ó šÄïYc@sždZddlmZddlmZmZddlmZmZm Z ddl m Z m Z ddl mZdd lmZd „Zd „Zd „Zd$d „Zde fd„ƒYZdee fd„ƒYZdefd„ƒYZdefd„ƒYZdefd„ƒYZdefd„ƒYZdefd„ƒYZdefd„ƒYZdefd„ƒYZd efd!„ƒYZd"efd#„ƒYZd$S(%s5Definition of various recurrent neural network cells.iÿÿÿÿ(tprint_functioni(tsymboltndarray(t string_typest numeric_typest_as_listi(tBlockt HybridBlock(t_indent(t tensor_typescCs)tg|D]}|j|ƒ^q gƒS(N(tsumt state_info(tcellst batch_sizetc((s8build/bdist.linux-armv7l/egg/mxnet/gluon/rnn/rnn_cell.pyt_cells_state_info scKs)tg|D]}|j|^q gƒS(N(R t begin_state(R tkwargsR((s8build/bdist.linux-armv7l/egg/mxnet/gluon/rnn/rnn_cell.pyt_cells_begin_state#sc CsŠ|dkr†|tkrht|tƒr0|jn |dj}| |jd|jd|ƒ}WdQXq†|jd|jd|ƒ}n|S(NitfuncR (tNoneRt isinstanceR tcontextRtzeros(tcelltFRtinputsR tctx((s8build/bdist.linux-armv7l/egg/mxnet/gluon/rnn/rnn_cell.pyt_get_begin_state&s  %$c CsQ|dk stdƒ‚|jdƒ}|jdƒ}d}|dk rW|jdƒn|}t|tjƒrÒt} |tkrt|jƒƒdks¥tdƒ‚t tj |d|d|d dƒƒ}qn6t|t j ƒr\t } |j |}|tkr|dks(||j |ks(t‚tt j |d|d|j |d dƒƒ}qn¬|dks€t|ƒ|ks€t‚t|dtjƒrŸt} nt } |dj |}|tkrg|D]} | j| d|ƒ^qÉ}| jd ||Œ}|}nt|tƒrA||krA| j|d |d |ƒ}n||| |fS( NsVunroll(inputs=None) has been deprecated. Please create input variables outside unroll.tTtNiis|unroll doesn't allow grouped symbol as input. Please convert to list with list(inputs) first or let unroll handle splitting.taxist num_outputst squeeze_axistdimtdim1tdim2(RtAssertionErrortfindRRtSymboltFalsetlent list_outputstlisttsplitRtNDArraytshapeRtTruet expand_dimstconcatR tswapaxes( tlengthRtlayouttmerget in_layoutRt batch_axisR tin_axisRti((s8build/bdist.linux-armv7l/egg/mxnet/gluon/rnn/rnn_cell.pyt_format_sequence0sB !    % $  ( t RecurrentCellcBsqeZdZd d d„Zd„Zd„Zdd„Zdej d„Z d dd d„Z d „Z d „Z RS( s®Abstract base class for RNN cells Parameters ---------- prefix : str, optional Prefix for names of `Block`s (this prefix is also used for names of weights if `params` is `None` i.e. if `params` are being created and not reused) params : Parameter or None, optional Container for weight sharing between cells. A new Parameter container is created if `params` is `None`. cCs6tt|ƒjd|d|ƒt|_|jƒdS(Ntprefixtparams(tsuperR;t__init__R(t _modifiedtreset(tselfR<R=((s8build/bdist.linux-armv7l/egg/mxnet/gluon/rnn/rnn_cell.pyR?hs cCsrd}t|dƒr"|d7}n|d7}|jrGdj|jn|j}|jd|jjd||jS(Ns{name}({mapping}t _activations, {_activation}t)s{_input_size} -> {_hidden_size}tnametmapping(thasattrt _input_sizetformatt__dict__t _hidden_sizet __class__t__name__(RBtsRF((s8build/bdist.linux-armv7l/egg/mxnet/gluon/rnn/rnn_cell.pyt__repr__ms   cCsd|_d|_dS(s1Reset before re-using the cell for another graph.iÿÿÿÿN(t _init_countert_counter(RB((s8build/bdist.linux-armv7l/egg/mxnet/gluon/rnn/rnn_cell.pyRAxs icCs tƒ‚dS(s&shape and layout information of statesN(tNotImplementedError(RBR ((s8build/bdist.linux-armv7l/egg/mxnet/gluon/rnn/rnn_cell.pyR }scKsš|j stdƒ‚g}xw|j|ƒD]f}|jd7_|dk r]|j|ƒn|}|dd|j|jf|}|j|ƒq,W|S(s=Initial state for this cell. Parameters ---------- func : callable, default symbol.zeros Function for creating initial state. For Symbol API, func can be `symbol.zeros`, `symbol.uniform`, `symbol.var etc`. Use `symbol.var` if you want to directly feed input as states. For NDArray API, func can be `ndarray.zeros`, `ndarray.ones`, etc. batch_size: int, default 0 Only required for NDArray API. Size of the batch ('N' in layout) dimension of input. **kwargs : Additional keyword arguments passed to func. For example `mean`, `std`, `dtype`, etc. Returns ------- states : nested list of Symbol Starting states for the first RNN step. syAfter applying modifier cells (e.g. ZoneoutCell) the base cell cannot be called directly. Call the modifier cell instead.iREs%sbegin_state_%dN(R@R%R RPRtupdatet_prefixtappend(RBR RRtstatestinfotstate((s8build/bdist.linux-armv7l/egg/mxnet/gluon/rnn/rnn_cell.pyRs    tNTCc Cs·|jƒt|||tƒ\}}}}t|||||ƒ}|} g} x:t|ƒD],} ||| | ƒ\} } | j| ƒq\Wt|| ||ƒ\} }}}| | fS(s½Unrolls an RNN cell across time steps. Parameters ---------- length : int Number of steps to unroll. inputs : Symbol, list of Symbol, or None If `inputs` is a single Symbol (usually the output of Embedding symbol), it should have shape (batch_size, length, ...) if `layout` is 'NTC', or (length, batch_size, ...) if `layout` is 'TNC'. If `inputs` is a list of symbols (usually output of previous unroll), they should all have shape (batch_size, ...). begin_state : nested list of Symbol, optional Input states created by `begin_state()` or output state of another cell. Created from `begin_state()` if `None`. layout : str, optional `layout` of input symbol. Only used if inputs is a single Symbol. merge_outputs : bool, optional If `False`, returns outputs as a list of Symbols. If `True`, concatenates output across time steps and returns a single symbol with shape (batch_size, length, ...) if layout is 'NTC', or (length, batch_size, ...) if layout is 'TNC'. If `None`, output whatever is faster. Returns ------- outputs : list of Symbol or Symbol Symbol (if `merge_outputs` is True) or list of Symbols (if `merge_outputs` is False) corresponding to the output from the RNN from this unrolling. states : list of Symbol The new state of this RNN after this unrolling. The type of this symbol is same as the output of `begin_state()`. (RAR:R(RtrangeRU( RBR3RRR4t merge_outputst_RR RVtoutputsR9toutput((s8build/bdist.linux-armv7l/egg/mxnet/gluon/rnn/rnn_cell.pytunrollªs* !!cKs6t|tƒr%|j|d||S|||SdS(s-Get activation function. Convert if is stringtact_typeN(RRt Activation(RBRRt activationR((s8build/bdist.linux-armv7l/egg/mxnet/gluon/rnn/rnn_cell.pyt_get_activationäscCs(|jd7_tt|ƒj||ƒS(s¦Unrolls the recurrent cell for one time step. Parameters ---------- inputs : sym.Variable Input symbol, 2D, of shape (batch_size * num_units). states : list of sym.Variable RNN state from previous step or the output of begin_state(). Returns ------- output : Symbol Symbol corresponding to the output from the RNN when unrolling for a single time step. states : list of Symbol The new state of this RNN after this unrolling. The type of this symbol is same as the output of `begin_state()`. This can be used as an input state to the next time step of this RNN. See Also -------- begin_state: This function can provide the states for the first time step. unroll: This function unrolls an RNN for a given number of (>=1) time steps. i(RQR>R;tforward(RBRRV((s8build/bdist.linux-armv7l/egg/mxnet/gluon/rnn/rnn_cell.pyRdësN(RMt __module__t__doc__RR?RORAR RRRR_RcRd(((s8build/bdist.linux-armv7l/egg/mxnet/gluon/rnn/rnn_cell.pyR;[s   ): tHybridRecurrentCellcBs&eZdZddd„Zd„ZRS(s'HybridRecurrentCell supports hybridize.cCs#tt|ƒjd|d|ƒdS(NR<R=(R>RgR?(RBR<R=((s8build/bdist.linux-armv7l/egg/mxnet/gluon/rnn/rnn_cell.pyR? scOs t‚dS(N(RR(RBRtxtargsR((s8build/bdist.linux-armv7l/egg/mxnet/gluon/rnn/rnn_cell.pythybrid_forwardsN(RMReRfRR?Rj(((s8build/bdist.linux-armv7l/egg/mxnet/gluon/rnn/rnn_cell.pyRg stRNNCellc BsMeZdZddddddddd„Zdd„Zd„Zd„ZRS( s´Simple recurrent neural network cell. Parameters ---------- hidden_size : int Number of units in output symbol activation : str or Symbol, default 'tanh' Type of activation function. i2h_weight_initializer : str or Initializer Initializer for the input weights matrix, used for the linear transformation of the inputs. h2h_weight_initializer : str or Initializer Initializer for the recurrent weights matrix, used for the linear transformation of the recurrent state. i2h_bias_initializer : str or Initializer Initializer for the bias vector. h2h_bias_initializer : str or Initializer Initializer for the bias vector. prefix : str, default 'rnn_' Prefix for name of `Block`s (and name of weight if params is `None`). params : Parameter or None Container for weight sharing between cells. Created if `None`. ttanhRic Csìtt|ƒjd|d| ƒ||_||_||_|jjdd||fd|dtƒ|_ |jjdd||fd|dtƒ|_ |jjdd|fd|dtƒ|_ |jjd d|fd|dtƒ|_ dS( NR<R=t i2h_weightR.tinittallow_deferred_initt h2h_weightti2h_biasth2h_bias( R>RkR?RKRCRHR=tgetR/RmRpRqRr( RBt hidden_sizeRbti2h_weight_initializerth2h_weight_initializerti2h_bias_initializerth2h_bias_initializert input_sizeR<R=((s8build/bdist.linux-armv7l/egg/mxnet/gluon/rnn/rnn_cell.pyR?-s       cCsi||jfd6dd6gS(NR.tNCt __layout__(RK(RBR ((s8build/bdist.linux-armv7l/egg/mxnet/gluon/rnn/rnn_cell.pyR BscCsdS(Ntrnn((RB((s8build/bdist.linux-armv7l/egg/mxnet/gluon/rnn/rnn_cell.pyt_aliasEsc Cs¦d|j}|jd|d|d|d|jd|dƒ} |jd|dd|d|d|jd|d ƒ} |j|| | |jd|d ƒ} | | gfS( Nst%d_tdatatweighttbiast num_hiddenREti2hith2htout(RQtFullyConnectedRKRcRC( RBRRRVRmRpRqRrR<R‚RƒR^((s8build/bdist.linux-armv7l/egg/mxnet/gluon/rnn/rnn_cell.pyRjHs      N(RMReRfRR?R R}Rj(((s8build/bdist.linux-armv7l/egg/mxnet/gluon/rnn/rnn_cell.pyRks  tLSTMCellcBsJeZdZdddddddd„Zdd„Zd„Zd„ZRS(sõLong-Short Term Memory (LSTM) network cell. Parameters ---------- hidden_size : int Number of units in output symbol. i2h_weight_initializer : str or Initializer Initializer for the input weights matrix, used for the linear transformation of the inputs. h2h_weight_initializer : str or Initializer Initializer for the recurrent weights matrix, used for the linear transformation of the recurrent state. i2h_bias_initializer : str or Initializer, default 'lstmbias' Initializer for the bias vector. By default, bias for the forget gate is initialized to 1 while all other biases are initialized to zero. h2h_bias_initializer : str or Initializer Initializer for the bias vector. prefix : str, default 'lstm_' Prefix for name of `Block`s (and name of weight if params is `None`). params : Parameter or None Container for weight sharing between cells. Created if `None`. Ric Csótt|ƒjd|d|ƒ||_||_|jjddd||fd|dtƒ|_|jjddd||fd|dtƒ|_ |jjd dd|fd|dtƒ|_ |jjd dd|fd|dtƒ|_ dS( NR<R=RmR.iRnRoRpRqRr( R>R†R?RKRHR=RsR/RmRpRqRr( RBRtRuRvRwRxRyR<R=((s8build/bdist.linux-armv7l/egg/mxnet/gluon/rnn/rnn_cell.pyR?qs     cCs8i||jfd6dd6i||jfd6dd6gS(NR.RzR{(RK(RBR ((s8build/bdist.linux-armv7l/egg/mxnet/gluon/rnn/rnn_cell.pyR †scCsdS(Ntlstm((RB((s8build/bdist.linux-armv7l/egg/mxnet/gluon/rnn/rnn_cell.pyR}Šsc Cs™d|j}|jd|d|d|d|jdd|dƒ} |jd|d d|d|d|jdd|d ƒ} | | } |j| d dd|d ƒ} |j| d d dd|dƒ} |j| dd dd|dƒ}|j| dd dd|dƒ}|j| dd dd|dƒ}|jj||d| |d|dƒ}|jj||j|d dƒd|dƒ}|||gfS(Nst%d_R~RR€RiRER‚iRƒR tsliceR`tsigmoidR9itfiRlRitoRXR„(RQR…RKt SliceChannelRat _internalt_plust_mul(RBRRRVRmRpRqRrR<R‚Rƒtgatest slice_gatestin_gatet forget_gatet in_transformtout_gatetnext_ctnext_h((s8build/bdist.linux-armv7l/egg/mxnet/gluon/rnn/rnn_cell.pyRjs   #### ! N(RMReRfRR?R R}Rj(((s8build/bdist.linux-armv7l/egg/mxnet/gluon/rnn/rnn_cell.pyR†Ws  tGRUCellcBsJeZdZdddddddd„Zdd„Zd„Zd„ZRS(sÜGated Rectified Unit (GRU) network cell. Note: this is an implementation of the cuDNN version of GRUs (slight modification compared to Cho et al. 2014). Parameters ---------- hidden_size : int Number of units in output symbol. i2h_weight_initializer : str or Initializer Initializer for the input weights matrix, used for the linear transformation of the inputs. h2h_weight_initializer : str or Initializer Initializer for the recurrent weights matrix, used for the linear transformation of the recurrent state. i2h_bias_initializer : str or Initializer Initializer for the bias vector. h2h_bias_initializer : str or Initializer Initializer for the bias vector. prefix : str, default 'gru_' prefix for name of `Block`s (and name of weight if params is `None`). params : Parameter or None Container for weight sharing between cells. Created if `None`. Ric Csótt|ƒjd|d|ƒ||_||_|jjddd||fd|dtƒ|_|jjddd||fd|dtƒ|_ |jjd dd|fd|dtƒ|_ |jjd dd|fd|dtƒ|_ dS( NR<R=RmR.iRnRoRpRqRr( R>R˜R?RKRHR=RsR/RmRpRqRr( RBRtRuRvRwRxRyR<R=((s8build/bdist.linux-armv7l/egg/mxnet/gluon/rnn/rnn_cell.pyR?¼s     cCsi||jfd6dd6gS(NR.RzR{(RK(RBR ((s8build/bdist.linux-armv7l/egg/mxnet/gluon/rnn/rnn_cell.pyR ÐscCsdS(Ntgru((RB((s8build/bdist.linux-armv7l/egg/mxnet/gluon/rnn/rnn_cell.pyR}Ósc Csvd|j}|d} |jd|d|d|d|jdd|d ƒ} |jd| d|d|d|jdd|d ƒ} |j| d dd|d ƒ\} } } |j| d dd|d ƒ\}}} |j| |ddd|dƒ}|j| |ddd|dƒ}|j| || ddd|dƒ}|jjd|||| d|dƒ}||gfS(Nst%d_iR~RR€RiRER‚RƒR t i2h_slicet h2h_sliceR`R‰tr_acttz_actRlth_actgð?R„(RQR…RKRŒRaRRŽ(RBRRRVRmRpRqRrR<t prev_state_hR‚Rƒti2h_rti2h_zth2h_rth2h_zt reset_gatet update_gatet next_h_tmpR—((s8build/bdist.linux-armv7l/egg/mxnet/gluon/rnn/rnn_cell.pyRjÖs2          N(RMReRfRR?R R}Rj(((s8build/bdist.linux-armv7l/egg/mxnet/gluon/rnn/rnn_cell.pyR˜¢s  tSequentialRNNCellcBszeZdZd d d„Zd„Zd„Zdd„Zd„Zd„Z d dd d „Z d „Z d „Z d „Z RS(s)Sequentially stacking multiple RNN cells.cCs#tt|ƒjd|d|ƒdS(NR<R=(R>R§R?(RBR<R=((s8build/bdist.linux-armv7l/egg/mxnet/gluon/rnn/rnn_cell.pyR?ûscCsnd}|jd|jjddjgt|jƒD]3\}}djd|dt|jƒdƒƒ^q1ƒƒS( Ns{name}( {modstr} )REtmodstrs s ({i}): {m}R9tmi(RIRLRMtjoint enumeratet _childrenRRO(RBRNR9R©((s8build/bdist.linux-armv7l/egg/mxnet/gluon/rnn/rnn_cell.pyROþs cCs|j|ƒdS(sfAppends a cell into the stack. Parameters ---------- cell : rnn cell N(tregister_child(RBR((s8build/bdist.linux-armv7l/egg/mxnet/gluon/rnn/rnn_cell.pytaddsicCst|j|ƒS(N(RR¬(RBR ((s8build/bdist.linux-armv7l/egg/mxnet/gluon/rnn/rnn_cell.pyR scKs&|j stdƒ‚t|j|S(NsyAfter applying modifier cells (e.g. ZoneoutCell) the base cell cannot be called directly. Call the modifier cell instead.(R@R%RR¬(RBR((s8build/bdist.linux-armv7l/egg/mxnet/gluon/rnn/rnn_cell.pyRs  cCs§|jd7_g}d}xv|jD]k}t|tƒ sAt‚t|jƒƒ}||||!}||7}|||ƒ\}}|j|ƒq%W|t|gƒfS(Nii( RQR¬RtBidirectionalCellR%R)R RUR (RBRRVt next_statestpRtnRX((s8build/bdist.linux-armv7l/egg/mxnet/gluon/rnn/rnn_cell.pyt__call__s RYc Cs|jƒt|||dƒ\}}}}t|jƒ} t|||||ƒ}d} g} xšt|jƒD]‰\} } t| jƒƒ}|| | |!}| |7} | j|d|d|d|d| | dkrÛdn|ƒ\}}| j |ƒqnW|| fS(NiRRR4R[i( RAR:RR)R¬RR«R R_textend(RBR3RRR4R[R\RR t num_cellsR±R°R9RR²RV((s8build/bdist.linux-armv7l/egg/mxnet/gluon/rnn/rnn_cell.pyR_#s ! %cCs |j|S(N(R¬(RBR9((s8build/bdist.linux-armv7l/egg/mxnet/gluon/rnn/rnn_cell.pyt __getitem__6scCs t|jƒS(N(R)R¬(RB((s8build/bdist.linux-armv7l/egg/mxnet/gluon/rnn/rnn_cell.pyt__len__9scOs t‚dS(N(RR(RBRiR((s8build/bdist.linux-armv7l/egg/mxnet/gluon/rnn/rnn_cell.pyRj<sN(RMReRfRR?ROR®R RR³R_R¶R·Rj(((s8build/bdist.linux-armv7l/egg/mxnet/gluon/rnn/rnn_cell.pyR§ùs      t DropoutCellcBsVeZdZd d d„Zd„Zdd„Zd„Zd„Zd dd d„Z RS( s¥Applies dropout on input. Parameters ---------- rate : float Percentage of elements to drop out, which is 1 - percentage to retain. cCsAtt|ƒj||ƒt|tƒs4tdƒ‚||_dS(Nsrate must be a number(R>R¸R?RRR%trate(RBR¹R<R=((s8build/bdist.linux-armv7l/egg/mxnet/gluon/rnn/rnn_cell.pyR?IscCs"d}|jd|jj|jS(Ns{name}(rate = {rate})RE(RIRLRMRJ(RBRN((s8build/bdist.linux-armv7l/egg/mxnet/gluon/rnn/rnn_cell.pyRONsicCsgS(N((RBR ((s8build/bdist.linux-armv7l/egg/mxnet/gluon/rnn/rnn_cell.pyR SscCsdS(Ntdropout((RB((s8build/bdist.linux-armv7l/egg/mxnet/gluon/rnn/rnn_cell.pyR}VscCsD|jdkr:|jd|d|jdd|jƒ}n||fS(NiR~R±REst%d_fwd(R¹tDropoutRQ(RBRRRV((s8build/bdist.linux-armv7l/egg/mxnet/gluon/rnn/rnn_cell.pyRjYs+RYc Csˆ|jƒt||||ƒ\}}}}t|tƒrY|j|||rR|ngƒStt|ƒj||d|d|d|ƒSdS(NRR4R[(RAR:RR RjR>R¸R_(RBR3RRR4R[R\R((s8build/bdist.linux-armv7l/egg/mxnet/gluon/rnn/rnn_cell.pyR_^s !N( RMReRfRR?ROR R}RjR_(((s8build/bdist.linux-armv7l/egg/mxnet/gluon/rnn/rnn_cell.pyR¸@s    t ModifierCellcBsSeZdZd„Zed„ƒZdd„Zejd„Z d„Z d„Z RS(sBase class for modifier cells. A modifier cell takes a base cell, apply modifications on it (e.g. Zoneout), and returns a new cell. After applying modifiers the base cell should no longer be called directly. The modifier cell should be used instead. cCs_|j std|jƒ‚t|_tt|ƒjd|j|jƒddƒ||_ dS(Ns>Cell %s is already modified. One cell cannot be modified twiceR<R=( R@R%RER/R>R¼R?R<R}Rt base_cell(RBR½((s8build/bdist.linux-armv7l/egg/mxnet/gluon/rnn/rnn_cell.pyR?ss   %cCs |jjS(N(R½R=(RB((s8build/bdist.linux-armv7l/egg/mxnet/gluon/rnn/rnn_cell.pyR={sicCs|jj|ƒS(N(R½R (RBR ((s8build/bdist.linux-armv7l/egg/mxnet/gluon/rnn/rnn_cell.pyR scKsJ|j stdƒ‚t|j_|jjd||}t|j_|S(NsyAfter applying modifier cells (e.g. DropoutCell) the base cell cannot be called directly. Call the modifier cell instead.R(R@R%R(R½RR/(RBRRtbegin((s8build/bdist.linux-armv7l/egg/mxnet/gluon/rnn/rnn_cell.pyR‚s     cCs t‚dS(N(RR(RBRRRV((s8build/bdist.linux-armv7l/egg/mxnet/gluon/rnn/rnn_cell.pyRj‹scCs"d}|jd|jj|jS(Ns{name}({base_cell})RE(RIRLRMRJ(RBRN((s8build/bdist.linux-armv7l/egg/mxnet/gluon/rnn/rnn_cell.pyROŽs( RMReRfR?tpropertyR=R RRRRjRO(((s8build/bdist.linux-armv7l/egg/mxnet/gluon/rnn/rnn_cell.pyR¼js   t ZoneoutCellcBsAeZdZddd„Zd„Zd„Zd„Zd„ZRS(sApplies Zoneout on base cell.gcCswt|tƒ stdƒ‚t|tƒ sB|j sBtdƒ‚tt|ƒj|ƒ||_||_ d|_ dS(Ns€BidirectionalCell doesn't support zoneout since it doesn't support step. Please add ZoneoutCell to the cells underneath instead.spBidirectional SequentialRNNCell doesn't support zoneout. Please add ZoneoutCell to the cells underneath instead.( RR¯R%R§t_bidirectionalR>RÀR?tzoneout_outputstzoneout_statesRt prev_output(RBR½RÂRÃ((s8build/bdist.linux-armv7l/egg/mxnet/gluon/rnn/rnn_cell.pyR?–s    cCs"d}|jd|jj|jS(NsF{name}(p_out={zoneout_outputs}, p_state={zoneout_states}, {base_cell})RE(RIRLRMRJ(RBRN((s8build/bdist.linux-armv7l/egg/mxnet/gluon/rnn/rnn_cell.pyRO¢scCsdS(Ntzoneout((RB((s8build/bdist.linux-armv7l/egg/mxnet/gluon/rnn/rnn_cell.pyR}§scCs tt|ƒjƒd|_dS(N(R>RÀRARRÄ(RB((s8build/bdist.linux-armv7l/egg/mxnet/gluon/rnn/rnn_cell.pyRAªscsý|j|j|j}}}|||ƒ\}}‡fd†} |j} | dkrhˆj|ƒ} n|dkr’ˆj| ||ƒ|| ƒn|} |dkrägt||ƒD]0\} } ˆj| || ƒ| | ƒ^q´n|}| |_| |fS(Ncsˆjˆj|ƒd|ƒS(NR±(R»t ones_like(R±tlike(R(s8build/bdist.linux-armv7l/egg/mxnet/gluon/rnn/rnn_cell.pyt±sg(R½RÂRÃRÄRt zeros_liketwheretzip(RBRRRVRt p_outputstp_statest next_outputR°tmaskRÄR^tnew_stold_s((Rs8build/bdist.linux-armv7l/egg/mxnet/gluon/rnn/rnn_cell.pyRj®s  0R (RMReRfR?ROR}RARj(((s8build/bdist.linux-armv7l/egg/mxnet/gluon/rnn/rnn_cell.pyRÀ”s    t ResidualCellcBs2eZdZd„Zd„Zdddd„ZRS(s§ Adds residual connection as described in Wu et al, 2016 (https://arxiv.org/abs/1609.08144). Output of the cell is output of the base cell plus input. cCstt|ƒj|ƒdS(N(R>RÒR?(RBR½((s8build/bdist.linux-armv7l/egg/mxnet/gluon/rnn/rnn_cell.pyR?ÈscCsA|j||ƒ\}}|j||dd|jƒ}||fS(NREst%d_fwd(R½t elemwise_addRQ(RBRRRVR^((s8build/bdist.linux-armv7l/egg/mxnet/gluon/rnn/rnn_cell.pyRjËsRYc Csí|jƒt|j_|jj|d|d|d|d|ƒ\}}t|j_|dkrmt|tƒn|}t ||||ƒ\}}} }|r¯| j ||ƒ}n4gt ||ƒD]\} } | j | | ƒ^q¿}||fS(NRRR4R[( RAR(R½R@R_R/RRR R:RÓRË( RBR3RRR4R[R]RVR\RR9tj((s8build/bdist.linux-armv7l/egg/mxnet/gluon/rnn/rnn_cell.pyR_Ðs   !4N(RMReRfR?RjRR_(((s8build/bdist.linux-armv7l/egg/mxnet/gluon/rnn/rnn_cell.pyRÒÁs  R¯cBsSeZdZdd„Zd„Zd„Zdd„Zd„Zd dd d „Z RS( s¸Bidirectional RNN cell. Parameters ---------- l_cell : RecurrentCell Cell for forward unrolling r_cell : RecurrentCell Cell for backward unrolling tbi_cCsFtt|ƒjddddƒ|j|ƒ|j|ƒ||_dS(NR<tR=(R>R¯R?RR­t_output_prefix(RBtl_celltr_cellt output_prefix((s8build/bdist.linux-armv7l/egg/mxnet/gluon/rnn/rnn_cell.pyR?ís  cCstdƒ‚dS(Ns2Bidirectional cannot be stepped. Please use unroll(RR(RBRRV((s8build/bdist.linux-armv7l/egg/mxnet/gluon/rnn/rnn_cell.pyR³óscCs6d}|jd|jjd|jdd|jdƒS(Ns+{name}(forward={l_cell}, backward={r_cell})RERØiRÙi(RIRLRMR¬(RBRN((s8build/bdist.linux-armv7l/egg/mxnet/gluon/rnn/rnn_cell.pyROös icCst|j|ƒS(N(RR¬(RBR ((s8build/bdist.linux-armv7l/egg/mxnet/gluon/rnn/rnn_cell.pyR üscKs&|j stdƒ‚t|j|S(NsyAfter applying modifier cells (e.g. DropoutCell) the base cell cannot be called directly. Call the modifier cell instead.(R@R%RR¬(RBR((s8build/bdist.linux-armv7l/egg/mxnet/gluon/rnn/rnn_cell.pyRÿs  RYc Cs |jƒt|||tƒ\}}}}t|||||ƒ}|} |j\} } | j|d|d| t| j|ƒƒ d|d|ƒ\} } | j|dtt |ƒƒd| t| j|ƒƒd|d|ƒ\}}|dkrSt | t ƒo t |t ƒ}td| ||ƒ\} }}}td|||ƒ\}}}}n|r–|j |d|ƒ}|j| |dddd |jƒ}n_gtt| t |ƒƒƒD]=\}\}}|j||dd dd |j|fƒ^q²}| |} || fS( NRRR4R[RR"iREs%soutis%st%d(RAR:R(RR¬R_R)R R+treversedRRR treverseR1R×R«RË(RBR3RRR4R[RRR RVRØRÙt l_outputstl_statest r_outputstr_statesR\R]R9tl_otr_o((s8build/bdist.linux-armv7l/egg/mxnet/gluon/rnn/rnn_cell.pyR_s0 !  !$(\ N( RMReRfR?R³ROR RRR_(((s8build/bdist.linux-armv7l/egg/mxnet/gluon/rnn/rnn_cell.pyR¯ãs      N( Rft __future__RRÖRRtbaseRRRtblockRRtutilsRR RRRRR:R;RgRkR†R˜R§R¸R¼RÀRÒR¯(((s8build/bdist.linux-armv7l/egg/mxnet/gluon/rnn/rnn_cell.pyts*   +¯ DKWG**-"