ó šÄïYc@södZddlmZmZddlmZdefd„ƒYZdefd„ƒYZdefd „ƒYZd efd „ƒYZ d efd „ƒYZ defd„ƒYZ defd„ƒYZ defd„ƒYZ defd„ƒYZdS(sBasic neural network layers.i(tBlockt HybridBlock(t_indentt SequentialcBsJeZdZddd„Zd„Zd„Zd„Zd„Zd„Z RS(s Stacks `Block`s sequentially. Example:: net = nn.Sequential() # use net's name_scope to give child Blocks appropriate names. with net.name_scope(): net.add(nn.Dense(10, activation='relu')) net.add(nn.Dense(20)) cCs#tt|ƒjd|d|ƒdS(Ntprefixtparams(tsuperRt__init__(tselfRR((s;build/bdist.linux-armv7l/egg/mxnet/gluon/nn/basic_layers.pyR%scCs|j|ƒdS(sAdds block on top of the stack.N(tregister_child(Rtblock((s;build/bdist.linux-armv7l/egg/mxnet/gluon/nn/basic_layers.pytadd(scCs$x|jD]}||ƒ}q W|S(N(t _children(RtxR ((s;build/bdist.linux-armv7l/egg/mxnet/gluon/nn/basic_layers.pytforward,sc Csƒd}djgt|jƒD]B\}}t|tƒrdjd|dt|jƒdƒƒ^qƒ}|jd|jj d|ƒS( Ns{name}( {modstr} )s s ({key}): {block}tkeyR itnametmodstr( tjoint enumerateR t isinstanceRtformatRt__repr__t __class__t__name__(RtsRR R((s;build/bdist.linux-armv7l/egg/mxnet/gluon/nn/basic_layers.pyR1s  ?cCs |j|S(N(R (Rti((s;build/bdist.linux-armv7l/egg/mxnet/gluon/nn/basic_layers.pyt __getitem__:scCs t|jƒS(N(tlenR (R((s;build/bdist.linux-armv7l/egg/mxnet/gluon/nn/basic_layers.pyt__len__=sN( Rt __module__t__doc__tNoneRR RRRR(((s;build/bdist.linux-armv7l/egg/mxnet/gluon/nn/basic_layers.pyRs    tHybridSequentialcBsJeZdZddd„Zd„Zd„Zd„Zd„Zd„Z RS(sStacks `HybridBlock`s sequentially. Example:: net = nn.Sequential() # use net's name_scope to give child Blocks appropriate names. with net.name_scope(): net.add(nn.Dense(10, activation='relu')) net.add(nn.Dense(20)) cCs#tt|ƒjd|d|ƒdS(NRR(RR!R(RRR((s;build/bdist.linux-armv7l/egg/mxnet/gluon/nn/basic_layers.pyRLscCs|j|ƒdS(sAdds block on top of the stack.N(R (RR ((s;build/bdist.linux-armv7l/egg/mxnet/gluon/nn/basic_layers.pyR OscCs$x|jD]}||ƒ}q W|S(N(R (RtFR R ((s;build/bdist.linux-armv7l/egg/mxnet/gluon/nn/basic_layers.pythybrid_forwardSsc Csƒd}djgt|jƒD]B\}}t|tƒrdjd|dt|jƒdƒƒ^qƒ}|jd|jj d|ƒS( Ns{name}( {modstr} )s s ({key}): {block}RR iRR( RRR RRRRRRR(RRRR R((s;build/bdist.linux-armv7l/egg/mxnet/gluon/nn/basic_layers.pyRXs  ?cCs |j|S(N(R (RR((s;build/bdist.linux-armv7l/egg/mxnet/gluon/nn/basic_layers.pyRascCs t|jƒS(N(RR (R((s;build/bdist.linux-armv7l/egg/mxnet/gluon/nn/basic_layers.pyRdsN( RRRR RR R#RRR(((s;build/bdist.linux-armv7l/egg/mxnet/gluon/nn/basic_layers.pyR!As    tDensecBs;eZdZdedddd„Zdd„Zd„ZRS(sJust your regular densely-connected NN layer. `Dense` implements the operation: `output = activation(dot(input, weight) + bias)` where `activation` is the element-wise activation function passed as the `activation` argument, `weight` is a weights matrix created by the layer, and `bias` is a bias vector created by the layer (only applicable if `use_bias` is `True`). Note: the input must be a tensor with rank 2. Use `flatten` to convert it to rank 2 manually if necessary. Parameters ---------- units : int Dimensionality of the output space. activation : str Activation function to use. See help on `Activation` layer. If you don't specify anything, no activation is applied (ie. "linear" activation: `a(x) = x`). use_bias : bool Whether the layer uses a bias vector. weight_initializer : str or `Initializer` Initializer for the `kernel` weights matrix. bias_initializer: str or `Initializer` Initializer for the bias vector. in_units : int, optional Size of the input data. If not specified, initialization will be deferred to the first time `forward` is called and `in_units` will be inferred from the shape of input data. prefix : str or None See document of `Block`. params : ParameterDict or None See document of `Block`. Input shape: A 2D input with shape `(batch_size, in_units)`. Output shape: The output would have shape `(batch_size, units)`. tzerosic KsÙtt|ƒj||jƒ±||_||_|jjdd||fd|dtƒ|_ |r•|jjdd|fd|dtƒ|_ n d|_ |dk rÆt |d|dƒ|_ n d|_ WdQXdS(Ntweighttshapetinittallow_deferred_inittbiasRt_(RR$Rt name_scopet_unitst _in_unitsRtgettTrueR&R*R t Activationtact(Rtunitst activationtuse_biastweight_initializertbias_initializertin_unitstkwargs((s;build/bdist.linux-armv7l/egg/mxnet/gluon/nn/basic_layers.pyR“s      c Cs|dkr6|j||dtd|jddƒ}n$|j|||d|jddƒ}|jdk r{|j|ƒ}n|S(Ntno_biast num_hiddenRtfwd(R tFullyConnectedR0R-R2(RR"R R&R*R2((s;build/bdist.linux-armv7l/egg/mxnet/gluon/nn/basic_layers.pyR#¨s   c Cs^d}|jd|jjd|jr-|jndd|jrTdj|j|jƒn|jƒS(Ns{name}({layout}, {act})RR2tlineartlayouts {0} -> {1}(RRRR2R.R-(RR((s;build/bdist.linux-armv7l/egg/mxnet/gluon/nn/basic_layers.pyR³s !N(RRRR R0RR#R(((s;build/bdist.linux-armv7l/egg/mxnet/gluon/nn/basic_layers.pyR$hs *  R1cBs2eZdZd„Zd„Zd„Zd„ZRS(s(Applies an activation function to input. Parameters ---------- activation : str Name of activation function to use. See :func:`~mxnet.ndarray.Activation` for available choices. Input shape: Arbitrary. Output shape: Same shape as input. cKs#||_tt|ƒj|dS(N(t _act_typeRR1R(RR4R9((s;build/bdist.linux-armv7l/egg/mxnet/gluon/nn/basic_layers.pyRËs cCs|jS(N(R@(R((s;build/bdist.linux-armv7l/egg/mxnet/gluon/nn/basic_layers.pyt_aliasÏscCs|j|d|jddƒS(Ntact_typeRR<(R1R@(RR"R ((s;build/bdist.linux-armv7l/egg/mxnet/gluon/nn/basic_layers.pyR#ÒscCs"d}|jd|jj|jS(Ns{name}({_act_type})R(RRRt__dict__(RR((s;build/bdist.linux-armv7l/egg/mxnet/gluon/nn/basic_layers.pyRÕs(RRRRRAR#R(((s;build/bdist.linux-armv7l/egg/mxnet/gluon/nn/basic_layers.pyR1»s    tDropoutcBs)eZdZd„Zd„Zd„ZRS(sDApplies Dropout to the input. Dropout consists in randomly setting a fraction `rate` of input units to 0 at each update during training time, which helps prevent overfitting. Parameters ---------- rate : float Fraction of the input units to drop. Must be a number between 0 and 1. Input shape: Arbitrary. Output shape: Same shape as input. References ---------- `Dropout: A Simple Way to Prevent Neural Networks from Overfitting `_ cKs#tt|ƒj|||_dS(N(RRDRt_rate(RtrateR9((s;build/bdist.linux-armv7l/egg/mxnet/gluon/nn/basic_layers.pyRòscCs|j|d|jddƒS(NtpRR<(RDRE(RR"R ((s;build/bdist.linux-armv7l/egg/mxnet/gluon/nn/basic_layers.pyR#öscCs"d}|jd|jj|jS(Ns{name}(p = {_rate})R(RRRRC(RR((s;build/bdist.linux-armv7l/egg/mxnet/gluon/nn/basic_layers.pyRùs(RRRRR#R(((s;build/bdist.linux-armv7l/egg/mxnet/gluon/nn/basic_layers.pyRDÛs  t BatchNormc BsGeZdZdddeedddddd„ Zd„Zd „ZRS( s*Batch normalization layer (Ioffe and Szegedy, 2014). Normalizes the input at each batch, i.e. applies a transformation that maintains the mean activation close to 0 and the activation standard deviation close to 1. Parameters ---------- axis : int, default 1 The axis that should be normalized. This is typically the channels (C) axis. For instance, after a `Conv2D` layer with `layout='NCHW'`, set `axis=1` in `BatchNorm`. If `layout='NHWC'`, then set `axis=3`. momentum: float, default 0.9 Momentum for the moving average. epsilon: float, default 1e-5 Small float added to variance to avoid dividing by zero. center: bool, default True If True, add offset of `beta` to normalized tensor. If False, `beta` is ignored. scale: bool, default True If True, multiply by `gamma`. If False, `gamma` is not used. When the next layer is linear (also e.g. `nn.relu`), this can be disabled since the scaling will be done by the next layer. beta_initializer: str or `Initializer`, default 'zeros' Initializer for the beta weight. gamma_initializer: str or `Initializer`, default 'ones' Initializer for the gamma weight. moving_mean_initializer: str or `Initializer`, default 'zeros' Initializer for the moving mean. moving_variance_initializer: str or `Initializer`, default 'ones' Initializer for the moving variance. in_channels : int, default 0 Number of channels (feature maps) in input data. If not specified, initialization will be deferred to the first time `forward` is called and `in_channels` will be inferred from the shape of input data. Input shape: Arbitrary. Output shape: Same shape as input. igÍÌÌÌÌÌì?gñh㈵øä>R%tonesic KsHtt|ƒj| i|d6|d6|d6| d6|_| dkrT| |_n|jjdd|rodnd d | fd |d td |ƒ|_|jjdd|r±dnd d | fd |d td |ƒ|_ |jjddd d | fd |d td t ƒ|_ |jjddd d | fd | d td t ƒ|_ dS(Ntaxistepstmomentumt fix_gammaitgammatgrad_reqtwritetnullR'R(R)tdifferentiabletbetat running_meant running_var( RRHRt_kwargst in_channelsRR/R0RNRStFalseRTRU( RRJRLtepsilontcentertscaletbeta_initializertgamma_initializertrunning_mean_initializertrunning_variance_initializerRWR9((s;build/bdist.linux-armv7l/egg/mxnet/gluon/nn/basic_layers.pyR+s.  ! !    c Cs%|j|||||dd|jS(NRR<(RHRV(RR"R RNRSRTRU((s;build/bdist.linux-armv7l/egg/mxnet/gluon/nn/basic_layers.pyR#Hsc Cs”d}t|dƒr.|dj|jƒ7}n|d7}|jd|jjddjg|jjƒD]'\}}dj||jƒgƒ^qcƒƒS( Ns{name}({content}RWs, in_channels={0}t)Rtcontents, t=( thasattrRRWRRRRVtitemsR(RRtktv((s;build/bdist.linux-armv7l/egg/mxnet/gluon/nn/basic_layers.pyRLs  (RRRR0RR#R(((s;build/bdist.linux-armv7l/egg/mxnet/gluon/nn/basic_layers.pyRHÿs+  t LeakyReLUcBs)eZdZd„Zd„Zd„ZRS(s}Leaky version of a Rectified Linear Unit. It allows a small gradient when the unit is not active:: `f(x) = alpha * x for x < 0`, `f(x) = x for x >= 0`. Parameters ---------- alpha : float slope coefficient for the negative half axis. Must be >= 0. Input shape: Arbitrary. Output shape: Same shape as input. cKs#tt|ƒj|||_dS(N(RRgRt_alpha(RtalphaR9((s;build/bdist.linux-armv7l/egg/mxnet/gluon/nn/basic_layers.pyRjscCs"|j|ddd|jddƒS(NRBtleakytslopeRR<(RgRh(RR"R ((s;build/bdist.linux-armv7l/egg/mxnet/gluon/nn/basic_layers.pyR#nscCs%d}|jd|jjd|jƒS(Ns{name}({alpha})RRi(RRRRh(RR((s;build/bdist.linux-armv7l/egg/mxnet/gluon/nn/basic_layers.pyRqs(RRRRR#R(((s;build/bdist.linux-armv7l/egg/mxnet/gluon/nn/basic_layers.pyRgVs  t EmbeddingcBs/eZdZddd„Zd„Zd„ZRS(sgTurns non-negative integers (indexes/tokens) into dense vectors of fixed size. eg. [[4], [20]] -> [[0.25, 0.1], [0.6, -0.2]] Parameters ---------- input_dim : int Size of the vocabulary, i.e. maximum integer index + 1. output_dim : int Dimension of the dense embedding. dtype : str or np.dtype, default 'float32' Data type of output embeddings. weight_initializer : Initializer Initializer for the `embeddings` matrix. Input shape: 2D tensor with shape: `(N, M)`. Output shape: 3D tensor with shape: `(N, M, output_dim)`. tfloat32cKsett|ƒj|i|d6|d6|d6|_|jjdd||fd|dtƒ|_dS(Nt input_dimt output_dimtdtypeR&R'R(R)(RRlRRVRR/R0R&(RRnRoRpR6R9((s;build/bdist.linux-armv7l/egg/mxnet/gluon/nn/basic_layers.pyRŽs  cCs|j||dd|jS(NRR<(RlRV(RR"R R&((s;build/bdist.linux-armv7l/egg/mxnet/gluon/nn/basic_layers.pyR#—scCs"d}|jd|jj|jS(Ns2{block_name}({input_dim} -> {output_dim}, {dtype})t block_name(RRRRV(RR((s;build/bdist.linux-armv7l/egg/mxnet/gluon/nn/basic_layers.pyRšsN(RRRR RR#R(((s;build/bdist.linux-armv7l/egg/mxnet/gluon/nn/basic_layers.pyRlws   tFlattencBs)eZdZd„Zd„Zd„ZRS(sªFlattens the input to two dimensional. Input shape: Arbitrary shape `(N, a, b, c, ...)` Output shape: 2D tensor with shape: `(N, a*b*c...)` cKstt|ƒj|dS(N(RRrR(RR9((s;build/bdist.linux-armv7l/egg/mxnet/gluon/nn/basic_layers.pyR©scCs |jdƒS(Niiÿÿÿÿ(iiÿÿÿÿ(treshape(RR"R ((s;build/bdist.linux-armv7l/egg/mxnet/gluon/nn/basic_layers.pyR#¬scCs |jjS(N(RR(R((s;build/bdist.linux-armv7l/egg/mxnet/gluon/nn/basic_layers.pyR¯s(RRRRR#R(((s;build/bdist.linux-armv7l/egg/mxnet/gluon/nn/basic_layers.pyRr s  N(RR RRtutilsRRR!R$R1RDRHRgRlRr(((s;build/bdist.linux-armv7l/egg/mxnet/gluon/nn/basic_layers.pyts''S $W!)