B An] @sddlmZddlZddlmZddlmZddlZddlm Z m Z m Z ddlm Z e e e e e e dZ d Zd Zejd d defd d defdeddefdddefdee ddefdddefdddefdd!dd ZdS)")unicode_literalsN)Path)Printer) conllu2jsoniob2jsonconll_ner2json)ner_jsonl2json)Z conllubioZconlluZconllneriobjsonl)jsonr msg)r r z Input file positionalz!Output directory. '-' for stdout.zType of data to produce: {}optiontzNumber of sentences per docnz Converter: {}cz Language (if tokenizer required)lz#Enable appending morphology to tagsflagm) input_file output_dir file_typen_sents converterlang morphology-r FautocCst}t|}|tkr8|jd|ddtdd|tkr^|dkr^|jd|ddd|sv|jd |dd|dkrt|s|jd |dd|d kr|jdd }|t kr|jd |ddt |} |j ddd } | | |||d} |dkrd|} t|t|j d | } |dkr>t| | n.|dkrVt| | n|dkrlt| | |dt| | n.|dkrtd| n|dkrtd| d S)a Convert files into JSON format for use with train command and other experiment management functions. If no output_dir is specified, the data is written to stdout, so you can pipe them forward to a JSONL file: $ spacy convert some_file.conllu > some_file.jsonl zUnknown file type: '{}'zSupported file types: '{}'z, r)exitsrzCan't write .{} data to stdout.z#Please specify an output directory.zInput file not foundzOutput directory not foundrNzCan't find converter for {}rzutf-8)encoding)rZuse_morphologyrz.{}r r rz$Generated output file ({} documents))rr FILE_TYPESfailformatjoinFILE_TYPES_STDOUTexistssuffix CONVERTERSopenreadparts with_suffixsrsly write_json write_jsonl write_msgpackgoodlen)rrrrrrrr input_pathfuncZ input_datadatar*Z output_filer9t/home/app_decipher_dev_19-4/dev/decipher-analysis/serverless-application/helper/df_spacy/python/spacy/cli/convert.pyconvertsJ        r;)rr rFrN) __future__rplacpathlibrwasabirr0 convertersrrrr r+r$r( annotationsstrr&inttuplekeysboolr;r9r9r9r:s8