ó ùµÈ[c@sýdZddlmZddlZddlZddlmZmZmZddl m Z m Z d„Z d„Z d „Zd „Zd „Zdd „Zd efd„ƒYZdefd„ƒYZdefd„ƒYZdefd„ƒYZdefd„ƒYZdefd„ƒYZdefd„ƒYZdefd„ƒYZdefd„ƒYZdefd „ƒYZd!efd"„ƒYZd#efd$„ƒYZd%efd&„ƒYZ d'e fd(„ƒYZ!d)e fd*„ƒYZ"d+e fd,„ƒYZ#dS(-s5Definition of various recurrent neural network cells.iÿÿÿÿ(tprint_functionNi(tsymboltinittndarray(t string_typest numeric_typescCs#tg|D]}|j^q gƒS(N(tsumt state_shape(tcellstc((sR/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/rnn/rnn_cell.pyt_cells_state_shape scCs#tg|D]}|j^q gƒS(N(Rt state_info(RR ((sR/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/rnn/rnn_cell.pyt_cells_state_info#scKs)tg|D]}|j|^q gƒS(N(Rt begin_state(RtkwargsR ((sR/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/rnn/rnn_cell.pyt_cells_begin_state&scCs$x|D]}|j|ƒ}qW|S(N(tunpack_weights(Rtargstcell((sR/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/rnn/rnn_cell.pyt_cells_unpack_weights)s cCs$x|D]}|j|ƒ}qW|S(N(t pack_weights(RRR((sR/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/rnn/rnn_cell.pyt_cells_pack_weights.s c Css|dk stdƒ‚|jdƒ}|dk rB|jdƒn|}t|tjƒr·|tkr-t|jƒƒdksŠtdƒ‚t tj |d|d|ddƒƒ}q-nv|dksÛt|ƒ|ksÛt‚|t kr-g|D]}tj |d|ƒ^qî}tj d||Œ}|}nt|tjƒri||kritj|d |d |ƒ}n||fS( NsVunroll(inputs=None) has been deprecated. Please create input variables outside unroll.tTis|unroll doesn't allow grouped symbol as input. Please convert to list with list(inputs) first or let unroll handle splitting.taxist num_outputst squeeze_axistdimtdim0tdim1(tNonetAssertionErrortfindt isinstanceRtSymboltFalsetlent list_outputstlisttsplittTruet expand_dimstConcattswapaxes(tlengthtinputstlayouttmerget in_layoutRtin_axisti((sR/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/rnn/rnn_cell.pyt_normalize_sequence3s$ !  $ ( t RNNParamscBs#eZdZdd„Zd„ZRS(sðContainer for holding variables. Used by RNN cells for parameter sharing between cells. Parameters ---------- prefix : str Names of all variables created by this container will be prepended with prefix. tcCs||_i|_dS(N(t_prefixt_params(tselftprefix((sR/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/rnn/rnn_cell.pyt__init__Xs cKsC|j|}||jkr8tj|||j|=1) time steps. N(tNotImplementedError(R7R,tstates((sR/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/rnn/rnn_cell.pyt__call__scCst|_|jS(sParameters of this cell(R"RAR6(R7((sR/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/rnn/rnn_cell.pyRD©s cCs tƒ‚dS(s&shape and layout information of statesN(RI(R7((sR/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/rnn/rnn_cell.pyR ¯scCsg|jD]}|d^q S(sshape(s) of statestshape(R (R7tele((sR/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/rnn/rnn_cell.pyR´scCsdS(sname(s) of gates(((R7((sR/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/rnn/rnn_cell.pyt _gate_names¹scKs°|j stdƒ‚g}x|jD]‚}|jd7_|dkrl|dd|j|jf|}n/|j|ƒ|dd|j|jf|}|j|ƒq&W|S(s<Initial state for this cell. Parameters ---------- func : callable, default symbol.zeros Function for creating initial state. Can be symbol.zeros, symbol.uniform, symbol.Variable etc. Use symbol.Variable if you want to directly feed input as states. **kwargs : more keyword arguments passed to func. For example mean, std, dtype, etc. Returns ------- states : nested list of Symbol Starting states for the first RNN step. syAfter applying modifier cells (e.g. DropoutCell) the base cell cannot be called directly. Call the modifier cell instead.iR;s%sbegin_state_%dN(RBRR RFRR5tupdatetappend(R7tfuncRRJtinfotstate((sR/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/rnn/rnn_cell.pyR ¾s      c Cs|jƒ}|js|S|j}xÞddgD]Ð}|jd|j|fƒ}|jd|j|fƒ}xt|jƒD]~\}}d|j||f}||||d|!jƒ|| NDArray Dictionary containing packed weights. usually from `Module.get_params()[0]`. Returns ------- args : dict of str -> NDArray Dictionary with unpacked weights associated with this cell. See Also -------- pack_weights: Performs the reverse operation of this function. ti2hth2hs %s%s_weights %s%s_biass %s%s%s_weightis %s%s%s_bias(tcopyRNt _num_hiddentpopR5t enumerate( R7Rtht group_nametweighttbiastjtgatetwnametbname((sR/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/rnn/rnn_cell.pyRás   #+cCsì|jƒ}|js|SxÌddgD]¾}g}g}xi|jD]^}d|j||f}|j|j|ƒƒd|j||f}|j|j|ƒƒqBWtj|ƒ|d|j|f NDArray Dictionary containing unpacked weights. Returns ------- args : dict of str -> NDArray Dictionary with packed weights associated with this cell. RTRUs %s%s%s_weights %s%s%s_biass %s%s_weights %s%s_bias(RVRNR5RPRXRt concatenate(R7RR[R\R]R_R`Ra((sR/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/rnn/rnn_cell.pyR s   $tNTCc Cs®|jƒt|||tƒ\}}|dkr@|jƒ}n|}g}x:t|ƒD],} ||| |ƒ\} }|j| ƒqYWt||||ƒ\}}||fS(s¶Unroll an RNN cell across time steps. Parameters ---------- length : int Number of steps to unroll. inputs : Symbol, list of Symbol, or None If `inputs` is a single Symbol (usually the output of Embedding symbol), it should have shape (batch_size, length, ...) if layout == 'NTC', or (length, batch_size, ...) if layout == 'TNC'. If `inputs` is a list of symbols (usually output of previous unroll), they should all have shape (batch_size, ...). begin_state : nested list of Symbol, default None Input states created by `begin_state()` or output state of another cell. Created from `begin_state()` if None. layout : str, optional `layout` of input symbol. Only used if inputs is a single Symbol. merge_outputs : bool, optional If False, return outputs as a list of Symbols. If True, concatenate output across time steps and return a single symbol with shape (batch_size, length, ...) if layout == 'NTC', or (length, batch_size, ...) if layout == 'TNC'. If None, output whatever is faster. Returns ------- outputs : list of Symbol or Symbol Symbol (if `merge_outputs` is True) or list of Symbols (if `merge_outputs` is False) corresponding to the output from the RNN from this unrolling. states : nested list of Symbol The new state of this RNN after this unrolling. The type of this symbol is same as the output of begin_state(). N(RCR2R"RR trangeRP( R7R+R,R R-t merge_outputst_RJtoutputsR1toutput((sR/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/rnn/rnn_cell.pytunroll's*  cKs6t|tƒr%tj|d||S|||SdS(s-Get activation function. Convert if is stringtact_typeN(R RRt Activation(R7R,t activationR((sR/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/rnn/rnn_cell.pyt_get_activationbsN(R=R>R?RR9RCRKtpropertyRDR RRNRtzerosR RRRiRm(((sR/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/rnn/rnn_cell.pyR@ls   # ( ;tRNNCellcBsGeZdZdddd„Zed„ƒZed„ƒZd„ZRS(sÐSimple recurrent neural network cell. Parameters ---------- num_hidden : int Number of units in output symbol. activation : str or Symbol, default 'tanh' Type of activation function. Options are 'relu' and 'tanh'. prefix : str, default 'rnn_' Prefix for name of layers (and name of weight if params is None). params : RNNParams, default None Container for weight sharing between cells. Created if None. ttanhtrnn_cCs‰tt|ƒjd|d|ƒ||_||_|jjdƒ|_|jjdƒ|_|jjdƒ|_ |jjdƒ|_ dS(NR8RDt i2h_weightti2h_biast h2h_weightth2h_bias( tsuperRpR9RWt _activationRDR<t_iWt_iBt_hWt_hB(R7t num_hiddenRlR8RD((sR/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/rnn/rnn_cell.pyR9xs  cCsid|jfd6dd6gS(NiRLtNCt __layout__(RW(R7((sR/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/rnn/rnn_cell.pyR scCsdS(NR4(R4((R7((sR/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/rnn/rnn_cell.pyRN…sc CsÇ|jd7_d|j|jf}tjd|d|jd|jd|jdd|ƒ}tjd|d d|jd|jd|jdd |ƒ}|j |||j dd |ƒ}||gfS( Nis%st%d_tdataR\R]R}R;s%si2his%sh2hs%sout( RGR5RtFullyConnectedRyRzRWR{R|RmRx(R7R,RJR;RTRURh((sR/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/rnn/rnn_cell.pyRK‰s!  %   N( R=R>R?RR9RnR RNRK(((sR/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/rnn/rnn_cell.pyRpjs   tLSTMCellcBsGeZdZdddd„Zed„ƒZed„ƒZd„ZRS(sÜLong-Short Term Memory (LSTM) network cell. Parameters ---------- num_hidden : int Number of units in output symbol. prefix : str, default 'lstm_' Prefix for name of layers (and name of weight if params is None). params : RNNParams, default None Container for weight sharing between cells. Created if None. forget_bias : bias added to forget gate, default 1.0. Jozefowicz et al. 2015 recommends setting this to 1.0 tlstm_gð?cCs’tt|ƒjd|d|ƒ||_|jjdƒ|_|jjdƒ|_|jjddtj d|ƒƒ|_ |jjdƒ|_ dS( NR8RDRsRuRtRt forget_biasRv( RwR‚R9RWRDR<RyR{RtLSTMBiasRzR|(R7R}R8RDR„((sR/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/rnn/rnn_cell.pyR9¦s  'cCs8id|jfd6dd6id|jfd6dd6gS(NiRLR~R(RW(R7((sR/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/rnn/rnn_cell.pyR °scCsddddgS(Nt_it_ft_ct_o((R7((sR/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/rnn/rnn_cell.pyRNµsc Cs½|jd7_d|j|jf}tjd|d|jd|jd|jddd |ƒ}tjd|d d|jd|jd|jddd |ƒ}||}tj |d ddd |ƒ}tj |d dddd|ƒ}tj |ddddd|ƒ} tj |ddddd|ƒ} tj |ddddd|ƒ} tj j | |d|| dd|ƒ} tj j | tj | ddƒdd|ƒ} | | | gfS(Nis%st%d_R€R\R]R}iR;s%si2his%sh2hRs%ssliceRjtsigmoids%sis%sfiRqs%scis%sos%sstates%sout(RGR5RRRyRzRWR{R|t SliceChannelRkt _internalt_plust_mul(R7R,RJR;RTRUtgatest slice_gatestin_gatet forget_gatet in_transformtout_gatetnext_ctnext_h((sR/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/rnn/rnn_cell.pyRK¹s0!  %         ! N( R=R>R?RR9RnR RNRK(((sR/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/rnn/rnn_cell.pyR‚˜s   tGRUCellcBsDeZdZddd„Zed„ƒZed„ƒZd„ZRS(sØGated Rectified Unit (GRU) network cell. Note: this is an implementation of the cuDNN version of GRUs (slight modification compared to Cho et al. 2014). Parameters ---------- num_hidden : int Number of units in output symbol. prefix : str, default 'gru_' Prefix for name of layers (and name of weight if params is None). params : RNNParams, default None Container for weight sharing between cells. Created if None. tgru_cCs€tt|ƒjd|d|ƒ||_|jjdƒ|_|jjdƒ|_|jjdƒ|_|jjdƒ|_ dS(NR8RDRsRtRuRv( RwR—R9RWRDR<RyRzR{R|(R7R}R8RD((sR/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/rnn/rnn_cell.pyR9ãs  cCsid|jfd6dd6gS(NiRLR~R(RW(R7((sR/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/rnn/rnn_cell.pyR ëscCs dddgS(Nt_rt_zR‰((R7((sR/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/rnn/rnn_cell.pyRNðsc Cs |jd7_|j}d|j|f}|d}tjd|d|jd|jd|jdd d |ƒ}tjd|d|jd|jd|jdd d |ƒ}tj |d dd d |ƒ\}} }tj |d dd d|ƒ\} } }tj || ddd d|ƒ} tj | | ddd d|ƒ} tj || |ddd d|ƒ}tj j d| || |d d|ƒ}||gfS(Nis%st%d_iR€R\R]R}iR;s%s_i2hs%s_h2hRs %s_i2h_slices %s_h2h_sliceRjRŠs%s_r_acts%s_z_actRqs%s_h_actgð?s%sout( RGR5RRRyRzRWR{R|R‹RkRŒR(R7R,RJtseq_idxR;t prev_state_hRTRUti2h_rti2h_zth2h_rth2h_zt reset_gatet update_gatet next_h_tmpR–((sR/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/rnn/rnn_cell.pyRKôs2          ((    N( R=R>R?RR9RnR RNRK(((sR/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/rnn/rnn_cell.pyR—Õs  t FusedRNNCellc Bs›eZdZddededddd„Zed„ƒZed„ƒZed„ƒZ d „Z d „Z d „Z d „Z dd dd„Zd„ZRS(sÐFusing RNN layers across time step into one kernel. Improves speed but is less flexible. Currently only supported if using cuDNN on GPU. Parameters ---------- num_hidden : int Number of units in output symbol. num_layers : int, default 1 Number of layers in the cell. mode : str, default 'lstm' Type of RNN. options are 'rnn_relu', 'rnn_tanh', 'lstm', 'gru'. bidirectional : bool, default False Whether to use bidirectional unroll. The output dimension size is doubled if bidrectional. dropout : float, default 0. Fraction of the input that gets dropped out during training time. get_next_state : bool, default False Whether to return the states that can be used as starting states next time. forget_bias : bias added to forget gate, default 1.0. Jozefowicz et al. 2015 recommends setting this to 1.0 prefix : str, default '$mode_' such as 'lstm_' Prefix for names of layers (this prefix is also used for names of weights if `params` is None i.e. if `params` are being created and not reused) params : RNNParams, default None Container for weight sharing between cells. Created if None. itlstmggð?c CsÉ|dkrd|}ntt|ƒjd|d| ƒ||_||_||_||_||_||_ |r€ddgndg|_ t j d|||||ƒ} |j jdd| ƒ|_dS(Ns%s_R8RDtltrt parametersR(RRwR¤R9RWt _num_layerst_modet_bidirectionalt_dropoutt_get_next_statet _directionsRtFusedRNNRDR<t _parameter( R7R}t num_layerstmodet bidirectionaltdropouttget_next_stateR„R8RDt initializer((sR/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/rnn/rnn_cell.pyR94s         cCs^|jd}|jdkd}gt|ƒD]-}i||jd|jfd6dd6^q-S(NiR¥iRLtLNCR(R«RªRdR©RW(R7tbtnRf((sR/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/rnn/rnn_cell.pyR Fs cCsBidgd6dgd6ddddgd6d d dgd 6|jS( NR4trnn_relutrnn_tanhR†R‡RˆR‰R¥R™Rštgru(Rª(R7((sR/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/rnn/rnn_cell.pyRNMs  cCs t|jƒS(N(R#RN(R7((sR/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/rnn/rnn_cell.pyt _num_gatesTscCs:i}|j}|j}t|ƒ}d}x1t|jƒD] } x|D]} x¤|D]œ} d|j| | | f} | dkr¸|||} |||| !j|||fƒ|| } d|j| | | f} ||||!|| <||7}qˆWxF|D]>} d|j| | | f} ||||!|| <||7}qÑWq{WqnW||jks6tdƒ‚|S(sslice fused rnn weightsis%s%s%d_i2h%s_weights%s%s%d_h2h%s_weightis%s%s%d_i2h%s_biass%s%s%d_h2h%s_biass(Invalid parameters size for FusedRNNCell( RNR®R#RdR©R5treshapetsizeR(R7tarrtlitlhRt gate_namest directionsR¸tptlayert directionR_R;R¿((sR/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/rnn/rnn_cell.pyt_slice_weightsXs@      + $  $   cCs³|jƒ}|j|jjƒ}t|jƒ}|j}|j}|j||||j d|||d|d}|j |||jƒ}|j d„|j ƒDƒƒ|S(NiicSs%i|]\}}|jƒ|“qS((RV(t.0R;tnd((sR/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/rnn/rnn_cell.pys ˆs ( RVRXR°R;R#R®R½RWR¿R©RÈROtitems(R7RRÀR¸tmRZt num_inputtnargs((sR/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/rnn/rnn_cell.pyRs   8c Cs|jƒ}|jd}|j}|j}|j}|d|j|df}|jd}||d||||jd|||||d|}tj |fd|j d|j ƒ} x9|j | ||ƒj ƒD]\} } |j| ƒ| (qÝW| ||jj<|S(Nis%sl0_i2h%s_weightiitctxtdtype(RVR«R½RNRWR5RLR©RRotcontextRÐRÈRËRXR°R;( R7RR¸RÌR RZtw0RÍttotalRÀR;RÊ((sR/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/rnn/rnn_cell.pyR‹s      A$(cCstdƒ‚dS(Ns1FusedRNNCell cannot be stepped. Please use unroll(RI(R7R,RJ((sR/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/rnn/rnn_cell.pyRK›sRcc Cs*|jƒt|||tƒ\}}|dkr\tjdƒtj|ddddƒ}n|dksxtd|ƒ‚|dkr“|j ƒ}n|}|j dkrÇi|dd6|dd 6}ni|dd6}tj d |d |j d |j d |jd|jd|jd|jd|j d|jd| }idd6} |js_|g} }n||j dkr²|dj| |dj| |d|d|dg} }n)|dj| |d|dg} }|dkrtj| ddddƒ} nt|| ||ƒ\} } | |fS(NisINTC layout detected. Consider using TNC for FusedRNNCell for faster speedRitdim2sUnsupported layout %sR¥RSt state_cellR€R¨t state_sizeR±R³RÅt state_outputsR²R;trnnR·Ri(RCR2R'twarningstwarnRR*RRR RªtRNNR°RWR©R«R¬R­R5t _set_attr( R7R+R,R R-ReRRJRØtattrRgRf((sR/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/rnn/rnn_cell.pyRižs>        " c s=tƒ}i‡fd†d6‡fd†d6‡fd†d6‡fd†d6ˆj}xàtˆjƒD]Ï}ˆjrÇ|jt|d ˆj|fƒ|d ˆj|fƒd d ˆj|fƒƒn |j|d ˆj|fƒƒˆjd krf|ˆjdkrf|jt ˆjddˆj|fƒƒqfqfW|S(sÇUnfuse the fused RNN in to a stack of rnn cells. Returns ------- cell : SequentialRNNCell unfused cell that can be used for stepping, and can run on CPU. cstˆjddd|ƒS(NRltreluR8(RpRW(t cell_prefix(R7(sR/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/rnn/rnn_cell.pytÓs Rºcstˆjddd|ƒS(NRlRqR8(RpRW(Rß(R7(sR/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/rnn/rnn_cell.pyRàÖs R»cstˆjd|ƒS(NR8(R‚RW(Rß(R7(sR/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/rnn/rnn_cell.pyRàÙs R¥cstˆjd|ƒS(NR8(R—RW(Rß(R7(sR/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/rnn/rnn_cell.pyRàÛs R¼s%sl%d_s%sr%d_t output_prefixs %sbi_l%d_iiR8s %s_dropout%d_( tSequentialRNNCellRªRdR©R«taddtBidirectionalCellR5R¬t DropoutCell(R7tstacktget_cellR1((R7sR/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/rnn/rnn_cell.pytunfuseÊs      "0N(R=R>R?R"RR9RnR RNR½RÈRRRKRiRè(((sR/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/rnn/rnn_cell.pyR¤s   '  ,RâcBsheZdZd d„Zd„Zed„ƒZd„Zd„Z d„Z d„Z d dd d „Z RS( s·Sequantially stacking multiple RNN cells. Parameters ---------- params : RNNParams, default None Container for weight sharing between cells. Created if None. cCs;tt|ƒjddd|ƒ|dk |_g|_dS(NR8R4RD(RwRâR9Rt_override_cell_paramsRE(R7RD((sR/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/rnn/rnn_cell.pyR9ôscCsg|jj|ƒ|jrJ|js.tdƒ‚|jjj|jjƒn|jjj|jjƒdS(sAppend a cell into the stack. Parameters ---------- cell : BaseRNNCell The cell to be appended. During unroll, previous cell's output (or raw inputs if no previous cell) is used as the input to this cell. sEEither specify params for SequentialRNNCell or child cells, not both.N(RERPRéRARRDR6RO(R7R((sR/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/rnn/rnn_cell.pyRãùs    cCs t|jƒS(N(R RE(R7((sR/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/rnn/rnn_cell.pyR scKs&|j stdƒ‚t|j|S(NsyAfter applying modifier cells (e.g. ZoneoutCell) the base cell cannot be called directly. Call the modifier cell instead.(RBRRRE(R7R((sR/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/rnn/rnn_cell.pyR s  cCst|j|ƒS(N(RRE(R7R((sR/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/rnn/rnn_cell.pyRscCst|j|ƒS(N(RRE(R7R((sR/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/rnn/rnn_cell.pyRscCs¤|jd7_g}d}xs|jD]h}t|tƒ sAt‚t|jƒ}||||!}||7}|||ƒ\}}|j|ƒq%W|t|gƒfS(Nii( RGRER RäRR#R RPR(R7R,RJt next_statesRÅRR¹RS((sR/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/rnn/rnn_cell.pyRKs Rcc Csä|jƒt|jƒ}|dkr4|jƒ}nd}g}x—t|jƒD]†\} } t| jƒ} |||| !} || 7}| j|d|d| d|d| |dkrºdn|ƒ\}} |j| ƒqPW||fS(NiR,R R-Rei( RCR#RERR RYR Ritextend( R7R+R,R R-Ret num_cellsRÅRêR1RR¹RJ((sR/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/rnn/rnn_cell.pyRi's   %N( R=R>R?RR9RãRnR R RRRKRi(((sR/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/rnn/rnn_cell.pyRâìs      RåcBsGeZdZddd„Zed„ƒZd„Zdddd„ZRS(sÞApply dropout on input. Parameters ---------- dropout : float Percentage of elements to drop out, which is 1 - percentage to retain. prefix : str, default 'dropout_' Prefix for names of layers (this prefix is also used for names of weights if `params` is None i.e. if `params` are being created and not reused) params : RNNParams, default None Container for weight sharing between cells. Created if None. tdropout_cCsAtt|ƒj||ƒt|tƒs4tdƒ‚||_dS(Ns$dropout probability must be a number(RwRåR9R RRR´(R7R´R8RD((sR/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/rnn/rnn_cell.pyR9JscCsgS(N((R7((sR/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/rnn/rnn_cell.pyR OscCs7|jdkr-tjd|d|jƒ}n||fS(NiR€RÅ(R´RtDropout(R7R,RJ((sR/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/rnn/rnn_cell.pyRKSsRcc Css|jƒt||||ƒ\}}t|tjƒrD||gƒStt|ƒj||d|d|d|ƒSdS(NR R-Re(RCR2R RR!RwRåRi(R7R+R,R R-ReRf((sR/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/rnn/rnn_cell.pyRiXs  N( R=R>R?RR9RnR RKRi(((sR/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/rnn/rnn_cell.pyRå;s  t ModifierCellcBs_eZdZd„Zed„ƒZed„ƒZejd„Z d„Z d„Z d„Z RS(sBase class for modifier cells. A modifier cell takes a base cell, apply modifications on it (e.g. Zoneout), and returns a new cell. After applying modifiers the base cell should no longer be called directly. The modifer cell should be used instead. cCs)tt|ƒjƒt|_||_dS(N(RwRïR9R'RBt base_cell(R7Rð((sR/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/rnn/rnn_cell.pyR9ls cCst|_|jjS(N(R"RARðRD(R7((sR/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/rnn/rnn_cell.pyRDqs cCs |jjS(N(RðR (R7((sR/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/rnn/rnn_cell.pyR vscKsG|j stdƒ‚t|j_|jj||}t|j_|S(NsyAfter applying modifier cells (e.g. DropoutCell) the base cell cannot be called directly. Call the modifier cell instead.(RBRR"RðR R'(R7tinit_symRtbegin((sR/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/rnn/rnn_cell.pyR zs     cCs|jj|ƒS(N(RðR(R7R((sR/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/rnn/rnn_cell.pyRƒscCs|jj|ƒS(N(RðR(R7R((sR/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/rnn/rnn_cell.pyR†scCs t‚dS(N(RI(R7R,RJ((sR/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/rnn/rnn_cell.pyRK‰s( R=R>R?R9RnRDR RRoR RRRK(((sR/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/rnn/rnn_cell.pyRïcs   t ZoneoutCellcBs/eZdZddd„Zd„Zd„ZRS(sqApply Zoneout on base cell. Parameters ---------- base_cell : BaseRNNCell Cell on whose states to perform zoneout. zoneout_outputs : float, default 0. Fraction of the output that gets dropped out during training time. zoneout_states : float, default 0. Fraction of the states that gets dropped out during training time. gcCs“t|tƒ stdƒ‚t|tƒ s8tdƒ‚t|tƒ s^|j s^tdƒ‚tt|ƒj|ƒ||_ ||_ d|_ dS(Ns:FusedRNNCell doesn't support zoneout. Please unfuse first.s€BidirectionalCell doesn't support zoneout since it doesn't support step. Please add ZoneoutCell to the cells underneath instead.spBidirectional SequentialRNNCell doesn't support zoneout. Please add ZoneoutCell to the cells underneath instead.( R R¤RRäRâR«RwRóR9tzoneout_outputstzoneout_statesRt prev_output(R7RðRôRõ((sR/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/rnn/rnn_cell.pyR9™s     cCs tt|ƒjƒd|_dS(N(RwRóRCRRö(R7((sR/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/rnn/rnn_cell.pyRC¨sc Cs÷|j|j|j}}}|||ƒ\}}d„}|jdk rS|jn tjdƒ} |dkrŒtj|||ƒ|| ƒn|} |dkrÞgt||ƒD]0\} } tj||| ƒ| | ƒ^q®n|}| |_| |fS(NcSstjtj|ƒd|ƒS(NRÅ(RRît ones_like(RÅtlike((sR/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/rnn/rnn_cell.pyRà¯sig(ii( RðRôRõRöRRRotwheretzip( R7R,RJRt p_outputstp_statest next_outputRêtmaskRöRhtnew_stold_s((sR/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/rnn/rnn_cell.pyRK¬s '0R (R=R>R?R9RCRK(((sR/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/rnn/rnn_cell.pyRós  t ResidualCellcBs2eZdZd„Zd„Zdddd„ZRS(sAdds residual connection as described in Wu et al, 2016 (https://arxiv.org/abs/1609.08144). Output of the cell is output of the base cell plus input. Parameters ---------- base_cell : BaseRNNCell Cell on whose outputs to add residual connection. cCstt|ƒj|ƒdS(N(RwRR9(R7Rð((sR/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/rnn/rnn_cell.pyR9ÉscCsA|j||ƒ\}}tj||dd|jƒ}||fS(NR;s%s_plus_residual(RðRt elemwise_addR;(R7R,RJRh((sR/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/rnn/rnn_cell.pyRKÌsRcc Cs|jƒt|j_|jj|d|d|d|d|ƒ\}}t|j_|dkrpt|tj ƒn|}t ||||ƒ\}}|r¹tj ||dd|j ƒ}nAgt ||ƒD]+\} } tj | | dd| j ƒ^qÉ}||fS(NR,R R-ReR;s%s_plus_residual(RCR"RðRBRiR'RR RR!R2RR;Rú( R7R+R,R R-ReRgRJRft output_symt input_sym((sR/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/rnn/rnn_cell.pyRiÑs   ">N(R=R>R?R9RKRRi(((sR/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/rnn/rnn_cell.pyR½s   RäcBsbeZdZd dd„Zd„Zd„Zd„Zed„ƒZ d„Z d dd d „Z RS( s™Bidirectional RNN cell. Parameters ---------- l_cell : BaseRNNCell cell for forward unrolling r_cell : BaseRNNCell cell for backward unrolling params : RNNParams, default None. Container for weight sharing between cells. A new RNNParams container is created if `params` is None. output_prefix : str, default 'bi_' prefix for name of output tbi_cCsÕtt|ƒjdd|ƒ||_|dk |_|jr|jrO|js[tdƒ‚|jj j |jj ƒ|jj j |jj ƒn|jj j |jj ƒ|jj j |jj ƒ||g|_ dS(NR4RDsEEither specify params for BidirectionalCell or child cells, not both.( RwRäR9t_output_prefixRRéRARRDR6RORE(R7tl_celltr_cellRDRá((sR/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/rnn/rnn_cell.pyR9õs   cCst|j|ƒS(N(RRE(R7R((sR/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/rnn/rnn_cell.pyRscCst|j|ƒS(N(RRE(R7R((sR/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/rnn/rnn_cell.pyRscCstdƒ‚dS(Ns2Bidirectional cannot be stepped. Please use unroll(RI(R7R,RJ((sR/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/rnn/rnn_cell.pyRK scCs t|jƒS(N(R RE(R7((sR/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/rnn/rnn_cell.pyR scKs&|j stdƒ‚t|j|S(NsyAfter applying modifier cells (e.g. DropoutCell) the base cell cannot be called directly. Call the modifier cell instead.(RBRRRE(R7R((sR/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/rnn/rnn_cell.pyR s  Rcc Csd|jƒt|||tƒ\}}|dkr@|jƒ}n|}|j\}} |j|d|d|t|jƒ d|d|ƒ\} } | j|dt t |ƒƒd|t|jƒd|d|ƒ\} } |dkr‰t | t j ƒot | t j ƒ}|s‰t | t j ƒrGt t j| d|d|ddƒƒ} nt | t j ƒr†t t j| d|d|ddƒƒ} q†q‰n|r³| g} t j| d|ƒg} nt t | ƒƒ} gttt| ƒƒ| | ƒD]Q\}}}t j||d d|d |rd |jnd |j|fƒ^qä}|rN|d }n| | g}||fS(NR,R R-ReRRRiRR;s%souts%st%di(RCR2R"RR RERiR#R R%treversedR RR!R‹treverseRúRdR)R(R7R+R,R R-ReRRJRRt l_outputstl_statest r_outputstr_statesR1tl_otr_oRg((sR/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/rnn/rnn_cell.pyRisB     s  N( R=R>R?RR9RRRKRnR R Ri(((sR/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/rnn/rnn_cell.pyRäæs    tBaseConvRNNCellcBsPeZdZdddd„Zed„ƒZed„ƒZd„Zd„Z RS( s/Abstract base class for Convolutional RNN cellsR4tNCHWcCsïtt|ƒjd|d|ƒ||_|jdddkrV|jdddksltdt|ƒƒ‚|d|ddd|d|dddf|_||_||_||_ ||_ ||_ ||_ ||_ ||_| |_tjdƒ}tjd|d|j d |jd |j d |j d |j d |ƒ|_|jjd|ƒdd|_d|jd|_|jjdd| ƒ|_|jjdd| ƒ|_|jjdd| ƒ|_|jjdd| ƒ|_dS(NR8RDiiis+Only support odd number, get h2h_kernel= %sR€t num_filtertkerneltstridetpadtdilateR-RsRRuRtRv(i(RwRR9t _h2h_kernelRtstrt_h2h_padt _h2h_dilatet _i2h_kernelt _i2h_stridet_i2h_padt _i2h_dilateRWt _input_shapet _conv_layoutRxRR:t Convolutiont _state_shapet infer_shapeRDR<RyR{RzR|(R7t input_shapeR}t h2h_kernelt h2h_dilatet i2h_kernelt i2h_strideti2h_padt i2h_dilateti2h_weight_initializerth2h_weight_initializerti2h_bias_initializerth2h_bias_initializerRlR8RDt conv_layoutR€((sR/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/rnn/rnn_cell.pyR9Hs: 1                 cCs t|jƒS(N(R#RN(R7((sR/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/rnn/rnn_cell.pyR½sscCs2i|jd6|jd6i|jd6|jd6gS(NRLR(R#R!(R7((sR/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/rnn/rnn_cell.pyR wscCsátjdd|d|d|j|jd|jd|jd|jd|jd |jd |j d |j ƒ }tjdd |d|d d|j|jd|j d|j d|j ddd |jd |jd |j ƒ }||fS(NR;s%si2hR€RRRRRR\R]R-s%sh2hii(ii(RR"RWR½RRRRRyRzR!RRRR{R|(R7R,RJR;RTRU((sR/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/rnn/rnn_cell.pyt _conv_forward|s*              cCstdƒ‚dS(Ns7BaseConvRNNCell is abstract class for convolutional RNN(RI(R7R,RJ((sR/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/rnn/rnn_cell.pyRK•sN( R=R>R?RR9RnR½R R1RK(((sR/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/rnn/rnn_cell.pyRFs % t ConvRNNCellcBs€eZdZddddddddddejejddddƒddd d „Ze d „ƒZ d „Z e d „ƒZ RS(sLConvolutional RNN cells Parameters ---------- input_shape : tuple of int Shape of input in single timestep. num_hidden : int Number of units in output symbol. h2h_kernel : tuple of int, default (3, 3) Kernel of Convolution operator in state-to-state transitions. h2h_dilate : tuple of int, default (1, 1) Dilation of Convolution operator in state-to-state transitions. i2h_kernel : tuple of int, default (3, 3) Kernel of Convolution operator in input-to-state transitions. i2h_stride : tuple of int, default (1, 1) Stride of Convolution operator in input-to-state transitions. i2h_pad : tuple of int, default (1, 1) Pad of Convolution operator in input-to-state transitions. i2h_dilate : tuple of int, default (1, 1) Dilation of Convolution operator in input-to-state transitions. i2h_weight_initializer : str or Initializer Initializer for the input weights matrix, used for the convolution transformation of the inputs. h2h_weight_initializer : str or Initializer Initializer for the recurrent weights matrix, used for the convolution transformation of the recurrent state. i2h_bias_initializer : str or Initializer, default zeros Initializer for the bias vector. h2h_bias_initializer : str or Initializer, default zeros Initializer for the bias vector. activation : str or Symbol, default functools.partial(symbol.LeakyReLU, act_type='leaky', slope=0.2) Type of activation function. prefix : str, default 'ConvRNN_' Prefix for name of layers (and name of weight if params is None). params : RNNParams, default None Container for weight sharing between cells. Created if None. conv_layout : str, , default 'NCHW' Layout of ConvolutionOp iiRoRjtleakytslopegš™™™™™É?tConvRNN_Rc!Cswtt|ƒjd|d|d|d|d|d|d|d|d | d | d | d | d | d|d|d|ƒdS(NR%R}R&R'R(R)R*R+R,R-R.R/RlR8RDR0(RwR2R9(R7R%R}R&R'R(R)R*R+R,R-R.R/RlR8RDR0((sR/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/rnn/rnn_cell.pyR9Ás    cCsdS(NR4(R4((R7((sR/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/rnn/rnn_cell.pyRNÔscCsp|jd7_d|j|jf}|j|||ƒ\}}|j|||jdd|ƒ}||gfS(Nis%st%d_R;s%sout(RGR5R1RmRx(R7R,RJR;RTRURh((sR/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/rnn/rnn_cell.pyRKØs  cCsi|jd6|jd6gS(NRLR(R#R!(R7((sR/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/rnn/rnn_cell.pyR às(ii(ii(ii(ii(ii(iiN( R=R>R?Rt functoolstpartialRt LeakyReLUR9RnRNRKR (((sR/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/rnn/rnn_cell.pyR2˜s(  t ConvLSTMCellcBs€eZdZddddddddddejejddddƒddd d „Ze d „ƒZ d „Z e d „ƒZ RS(s†Convolutional LSTM network cell. Reference: Xingjian et al. NIPS2015 Parameters ---------- input_shape : tuple of int Shape of input in single timestep. num_hidden : int Number of units in output symbol. h2h_kernel : tuple of int, default (3, 3) Kernel of Convolution operator in state-to-state transitions. h2h_dilate : tuple of int, default (1, 1) Dilation of Convolution operator in state-to-state transitions. i2h_kernel : tuple of int, default (3, 3) Kernel of Convolution operator in input-to-state transitions. i2h_stride : tuple of int, default (1, 1) Stride of Convolution operator in input-to-state transitions. i2h_pad : tuple of int, default (1, 1) Pad of Convolution operator in input-to-state transitions. i2h_dilate : tuple of int, default (1, 1) Dilation of Convolution operator in input-to-state transitions. i2h_weight_initializer : str or Initializer Initializer for the input weights matrix, used for the convolution transformation of the inputs. h2h_weight_initializer : str or Initializer Initializer for the recurrent weights matrix, used for the convolution transformation of the recurrent state. i2h_bias_initializer : str or Initializer, default zeros Initializer for the bias vector. h2h_bias_initializer : str or Initializer, default zeros Initializer for the bias vector. activation : str or Symbol default functools.partial(symbol.LeakyReLU, act_type='leaky', slope=0.2) Type of activation function. prefix : str, default 'ConvLSTM_' Prefix for name of layers (and name of weight if params is None). params : RNNParams, default None Container for weight sharing between cells. Created if None. conv_layout : str, , default 'NCHW' Layout of ConvolutionOp iiRoRjR3R4gš™™™™™É?t ConvLSTM_Rc!Cswtt|ƒjd|d|d|d|d|d|d|d|d | d | d | d | d | d|d|d|ƒdS(NR%R}R&R'R(R)R*R+R,R-R.R/RlR8RDR0(RwR9R9(R7R%R}R&R'R(R)R*R+R,R-R.R/RlR8RDR0((sR/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/rnn/rnn_cell.pyR9s     cCsddddgS(NR†R‡RˆR‰((R7((sR/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/rnn/rnn_cell.pyRN%sc Csp|jd7_d|j|jf}|j|||ƒ\}}||}tj|ddd|jjdƒdd|ƒ}tj|d d d dd |ƒ}tj|dd d dd |ƒ} |j|d|j dd|ƒ} tj|dd d dd|ƒ} tj j | |d|| dd|ƒ} tj j | |j| |j ƒdd|ƒ} | | | gfS(Nis%st%d_RiRtCR;s%ssliceiRjRŠs%sis%sfis%scis%sos%sstates%sout( RGR5R1RR‹R!RRkRmRxRŒRRŽ(R7R,RJR;RTRURRR‘R’R“R”R•R–((sR/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/rnn/rnn_cell.pyRK)s& $      ! cCs2i|jd6|jd6i|jd6|jd6gS(NRLR(R#R!(R7((sR/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/rnn/rnn_cell.pyR ?s(ii(ii(ii(ii(ii(iiN( R=R>R?RR6R7RR8R9RnRNRKR (((sR/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/rnn/rnn_cell.pyR9ås+  t ConvGRUCellcBs€eZdZddddddddddejejddddƒddd d „Ze d „ƒZ e d „ƒZ d „Z RS(skConvolutional Gated Rectified Unit (GRU) network cell. Parameters ---------- input_shape : tuple of int Shape of input in single timestep. num_hidden : int Number of units in output symbol. h2h_kernel : tuple of int, default (3, 3) Kernel of Convolution operator in state-to-state transitions. h2h_dilate : tuple of int, default (1, 1) Dilation of Convolution operator in state-to-state transitions. i2h_kernel : tuple of int, default (3, 3) Kernel of Convolution operator in input-to-state transitions. i2h_stride : tuple of int, default (1, 1) Stride of Convolution operator in input-to-state transitions. i2h_pad : tuple of int, default (1, 1) Pad of Convolution operator in input-to-state transitions. i2h_dilate : tuple of int, default (1, 1) Dilation of Convolution operator in input-to-state transitions. i2h_weight_initializer : str or Initializer Initializer for the input weights matrix, used for the convolution transformation of the inputs. h2h_weight_initializer : str or Initializer Initializer for the recurrent weights matrix, used for the convolution transformation of the recurrent state. i2h_bias_initializer : str or Initializer, default zeros Initializer for the bias vector. h2h_bias_initializer : str or Initializer, default zeros Initializer for the bias vector. activation : str or Symbol, default functools.partial(symbol.LeakyReLU, act_type='leaky', slope=0.2) Type of activation function. prefix : str, default 'ConvGRU_' Prefix for name of layers (and name of weight if params is None). params : RNNParams, default None Container for weight sharing between cells. Created if None. conv_layout : str, , default 'NCHW' Layout of ConvolutionOp iiRoRjR3R4gš™™™™™É?tConvGRU_Rc!Cswtt|ƒjd|d|d|d|d|d|d|d|d | d | d | d | d | d|d|d|ƒdS(NR%R}R&R'R(R)R*R+R,R-R.R/RlR8RDR0(RwR<R9(R7R%R}R&R'R(R)R*R+R,R-R.R/RlR8RDR0((sR/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/rnn/rnn_cell.pyR9ms    cCs dddgS(NR™RšR‰((R7((sR/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/rnn/rnn_cell.pyRN€scCsi|jd6|jd6gS(NRLR(R#R!(R7((sR/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/rnn/rnn_cell.pyR „scCs?|jd7_|j}d|j|f}|j|||ƒ\}}tj|dddd|ƒ\}}}tj|dddd|ƒ\} } }tj|| dd dd |ƒ} tj|| dd dd |ƒ} |j|| ||jdd |ƒ} tjj d | | | |ddd|ƒ}||gfS(Nis%st%d_RiR;s %s_i2h_slices %s_h2h_sliceRjRŠs%s_r_acts%s_z_acts%s_h_actgð?is%sout( RGR5R1RR‹RkRmRxRŒR(R7R,RJR›R;RTRURRžRŸR R¡R¢R£R–((sR/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/rnn/rnn_cell.pyRKˆs ((   " (ii(ii(ii(ii(ii(iiN( R=R>R?RR6R7RR8R9RnRNR RK(((sR/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/rnn/rnn_cell.pyR<Ds( ($R?t __future__RRÙR6R4RRRtbaseRRR R RRRRR2tobjectR3R@RpR‚R—R¤RâRåRïRóRRäRR2R9R<(((sR/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/rnn/rnn_cell.pyts6        þ.=CÔO(*0)`RM_