ó łµČ[c@sdZddgZddlmZmZmZmZddlmZm Z m Z ddl m Z defd„ƒYZ defd„ƒYZd S( s5Definition of various recurrent neural network cells.tVariationalDropoutCellt LSTMPCelli(tBidirectionalCelltSequentialRNNCellt ModifierCelltHybridRecurrentCell(t_format_sequencet_get_begin_statet_mask_sequence_variable_length(t tensor_typescBskeZdZdddd„Zd„Zd„Zd„Zd„Zd„Zd„Z d d d d d „Z RS( s; Applies Variational Dropout on base cell. (https://arxiv.org/pdf/1512.05287.pdf, https://www.stat.berkeley.edu/~tsmoon/files/Conference/asru2015.pdf). Variational dropout uses the same dropout mask across time-steps. It can be applied to RNN inputs, outputs, and states. The masks for them are not shared. The dropout mask is initialized when stepping forward for the first time and will remain the same until .reset() is called. Thus, if using the cell and stepping manually without calling .unroll(), the .reset() should be called after each sequence. Parameters ---------- base_cell : RecurrentCell The cell on which to perform variational dropout. drop_inputs : float, default 0. The dropout rate for inputs. Won't apply dropout if it equals 0. drop_states : float, default 0. The dropout rate for state inputs on the first state channel. Won't apply dropout if it equals 0. drop_outputs : float, default 0. The dropout rate for outputs. Won't apply dropout if it equals 0. gcCs | s#t|tƒ s#tdƒ‚| sPt|tƒ sP|j sPtdƒ‚tt|ƒj|ƒ||_||_ ||_ d|_ d|_ d|_dS(NsBidirectionalCell doesn't support variational state dropout. Please add VariationalDropoutCell to the cells underneath instead.sBidirectional SequentialRNNCell doesn't support variational state dropout. Please add VariationalDropoutCell to the cells underneath instead.(t isinstanceRtAssertionErrorRt_bidirectionaltsuperRt__init__t drop_inputst drop_statest drop_outputstNonetdrop_inputs_masktdrop_states_masktdrop_outputs_mask(tselft base_cellRRR((s`/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/gluon/contrib/rnn/rnn_cell.pyR3s       cCsdS(Ntvardrop((R((s`/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/gluon/contrib/rnn/rnn_cell.pyt_aliasCscCs2tt|ƒjƒd|_d|_d|_dS(N(R RtresetRRRR(R((s`/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/gluon/contrib/rnn/rnn_cell.pyRFs  cCs†|jrC|jdkrC|j|j|dƒd|jƒ|_n|jr‚|jdkr‚|j|j|ƒd|jƒ|_ndS(Nitp(RRRtDropoutt ones_likeRR(RtFtinputststates((s`/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/gluon/contrib/rnn/rnn_cell.pyt_initialize_input_masksLs cCsC|jr?|jdkr?|j|j|ƒd|jƒ|_ndS(NR(RRRRR(RRtoutput((s`/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/gluon/contrib/rnn/rnn_cell.pyt_initialize_output_maskUscCsŖ|j}|j|||ƒ|jrIt|ƒ}|d|j|d