U C^@sddlmZddlmZmZddlmZmZddlm Z ddl m Z ddl m Z ddl mZddlmZmZmZmZmZdd lmZmZmZd d lmZmZmZmZmZmZd d ZedddZ ddZ!dS))unicode_literals)ModelMaxout) HashEmbed StaticVectors) ExtractWindow)Residual) LayerNorm)FeatureExtracter)layerizechainclone concatenate with_flatten)uniquedwrapnoop)IDORTHNORMPREFIXSUFFIXSHAPEc Ksddlm}ddlm}|dd}|dd}|dd}|d d }|rPd }|d d } |d d} tttttt g} t t t tdt||| tdd} |rt||d| tdd} t||d| tdd}t||d| tdd}n d\} }}|dk rt||| td}|r`t|| B| B|B|Btt||ddd?| t d}n,t|| Btt||ddd?| t d}n||rt| | B|B|Btt||d dd?| t d}n@|rt|dddt| t| ?}tt|d||d}n| }ttddtt||d|d?}|rN|t||| ?| d?}nt| t||| ?| d?}| dkr||||| ?}||_||_W5QRX|S) Nr)CharacterEmbed) PyTorchBiLSTMpretrained_vectorscnn_maxout_piecessubword_featuresT char_embedF conv_depth bilstm_depthr)z>>|z**Z embed_norm)columnnameZ embed_prefixZ embed_suffixZ embed_shape)NNN)r%)pieces@)ZnMZnCi)ZnWpad)Z_mlrrgetrrrrrrrZdefine_operatorsr rr rindexrrLNrconcatenate_listsr rrrZnOembed)widthZ embed_sizekwargsrrrrrr r!r#colsZnormprefixsuffixshapeZglover2Zreduce_dimensionsZ convolutionZtok2vecr9;/tmp/pip-install-6_kvzl1k/spacy/spacy/ml/_legacy_tok2vec.pyTok2Vecs               r;csDtjjdd|Dddd fdd }j|dd}||fS) NcSsg|] }t|qSr9len).0seqr9r9r: bszflatten..iZdtypecsj|ddS)Nrr,) unflatten)Zd_Xsgdlengthsopsr9r: finish_updatedszflatten..finish_updaterr,)N)rrHasarrayflatten)ZseqsdroprIXr9rFr:rK_s rKcsV|s tS|dd|djdd|D}t|d fdd }t|}|S) zCompose two or more models `f`, `g`, etc, such that their outputs are concatenated, i.e. `concatenate(f, g)(x)` computes `hstack(f(x), g(x))` drop_factorg?rcSsg|]}t|tqSr9)r rK)r?Zlayerr9r9r:rAssz%concatenate_lists..r<cs^|dk r|9}jdd|Ddd}j||d\}||}dfdd }||fS) NcSsg|] }t|qSr9r=)r?rMr9r9r:rAyszDconcatenate_lists..concatenate_lists_fwd..rBrC)rLcs||dS)N)rE)rK)Zd_ysrE) bp_flat_yrHr9r:concatenate_lists_bwd}szOconcatenate_lists..concatenate_lists_fwd..concatenate_lists_bwd)N)rJZ begin_updaterD)ZXsrLrGZflat_yZysrPconcatrNrH)rOr:concatenate_lists_fwdvs z0concatenate_lists..concatenate_lists_fwd)r<)rr.rHrr)Zlayersr4rSmodelr9rQr:r1ks   r1N)r<)" __future__rZ thinc.v2vrrZ thinc.i2vrrZ thinc.t2trZ thinc.miscrr r0r Z thinc.apir r r rrrrrattrsrrrrrrr;rKr1r9r9r9r:s      P