ó R;]c@s%dZdddddgZddlmZddlZddlZd d lmZm Z d d l m Z m Z m Z mZd d lmZmZd d l mZddlmZmZe je jfZde fd„ƒYZdefd„ƒYZdefd„ƒYZdefd„ƒYZdS(sNeural network parameter.tDeferredInitializationErrort ParametertConstantt ParameterDictt tensor_typesiÿÿÿÿ(t OrderedDictNi(t mx_real_tt MXNetError(tsymboltndarrayt initializertcontext(tContexttcpu(tautogradi(t_indentt_brief_print_listcBseZdZRS(s-Error for unfinished deferred initialization.(t__name__t __module__t__doc__(((sU/usr/local/lib/python2.7/site-packages/mxnet-1.2.1-py2.7.egg/mxnet/gluon/parameter.pyR(sc Bs+eZdZddedddeed„Zd„Ze d„ƒZ e j d„ƒZ e d„ƒZ e j d„ƒZ d „Z d „Zd „Zd „Zd „Zd„Zddejƒed„Zd„Zd„Zdd„Zd„Zdd„Zd„Zd„Zd„Zd„Zd„ZRS(sp A Container holding parameters (weights) of Blocks. :py:class:`Parameter` holds a copy of the parameter on each :py:class:`Context` after it is initialized with ``Parameter.initialize(...)``. If :py:attr:`grad_req` is not ``'null'``, it will also hold a gradient array on each :py:class:`Context`:: ctx = mx.gpu(0) x = mx.nd.zeros((16, 100), ctx=ctx) w = mx.gluon.Parameter('fc_weight', shape=(64, 100), init=mx.init.Xavier()) b = mx.gluon.Parameter('fc_bias', shape=(64,), init=mx.init.Zero()) w.initialize(ctx=ctx) b.initialize(ctx=ctx) out = mx.nd.FullyConnected(x, w.data(ctx), b.data(ctx), num_hidden=64) Parameters ---------- name : str Name of this parameter. grad_req : {'write', 'add', 'null'}, default 'write' Specifies how to update gradient to grad arrays. - ``'write'`` means everytime gradient is written to grad :py:class:`NDArray`. - ``'add'`` means everytime gradient is added to the grad :py:class:`NDArray`. You need to manually call ``zero_grad()`` to clear the gradient buffer before each iteration when using this option. - 'null' means gradient is not requested for this parameter. gradient arrays will not be allocated. shape : tuple of int, default None Shape of this parameter. By default shape is not specified. Parameter with unknown shape can be used for :py:class:`Symbol` API, but ``init`` will throw an error when using :py:class:`NDArray` API. dtype : numpy.dtype or str, default 'float32' Data type of this parameter. For example, ``numpy.float32`` or ``'float32'``. lr_mult : float, default 1.0 Learning rate multiplier. Learning rate will be multiplied by lr_mult when updating this parameter with optimizer. wd_mult : float, default 1.0 Weight decay multiplier (L2 regularizer coefficient). Works similar to lr_mult. init : Initializer, default None Initializer of this parameter. Will use the global initializer by default. Attributes ---------- grad_req : {'write', 'add', 'null'} This can be set before or after initialization. Setting ``grad_req`` to ``'null'`` with ``x.grad_req = 'null'`` saves memory and computation when you don't need gradient w.r.t x. lr_mult : float Local learning rate multiplier for this Parameter. The actual learning rate is calculated with ``learning_rate * lr_mult``. You can set it with ``param.lr_mult = 2.0`` wd_mult : float Local weight decay multiplier for this Parameter. twritegð?c Cs”d|_d|_d|_d|_d|_d|_| |_||_d|_ ||_ ||_ ||_ ||_ ||_||_||_dS(N((tNonet_vart_datat_gradt _ctx_listt_ctx_mapt_deferred_initt_differentiablet_allow_deferred_initt _grad_reqt_shapetnametdtypetlr_multtwd_multtgrad_reqtinit( tselfR R$tshapeR!R"R#R%tallow_deferred_inittdifferentiable((sU/usr/local/lib/python2.7/site-packages/mxnet-1.2.1-py2.7.egg/mxnet/gluon/parameter.pyt__init__cs                cCs+d}|jd|jd|jd|jƒS(Ns/Parameter {name} (shape={shape}, dtype={dtype})R R'R!(tformatR R'R!(R&ts((sU/usr/local/lib/python2.7/site-packages/mxnet-1.2.1-py2.7.egg/mxnet/gluon/parameter.pyt__repr__wscCs|jS(N(R(R&((sU/usr/local/lib/python2.7/site-packages/mxnet-1.2.1-py2.7.egg/mxnet/gluon/parameter.pyR${scCs¶|dkstd|ƒ‚|js.d}n|j|krAdS||_|dkr–|jdk r–d|_g|jD]}|jƒ^qx|_n|jdk r²|jƒndS(NRtaddtnulls?grad_req must be one of 'write', 'add', or 'null', but got '%s'(RR.R/(tAssertionErrorRRRRRtdetacht _init_grad(R&treqti((sU/usr/local/lib/python2.7/site-packages/mxnet-1.2.1-py2.7.egg/mxnet/gluon/parameter.pyR$s     (cCs|jS(N(R(R&((sU/usr/local/lib/python2.7/site-packages/mxnet-1.2.1-py2.7.egg/mxnet/gluon/parameter.pyR'ŽscCs‹|jdkr||_dSt|jƒt|ƒkrYtd„t||jƒDƒƒs~tdt|ƒt|jƒfƒ‚||_dS(Ncss-|]#\}}|dkp$||kVqdS(iN((t.0R4tj((sU/usr/local/lib/python2.7/site-packages/mxnet-1.2.1-py2.7.egg/mxnet/gluon/parameter.pys ™ss6Expected shape %s is incompatible with given shape %s.(RRtlentalltzipR0tstr(R&t new_shape((sU/usr/local/lib/python2.7/site-packages/mxnet-1.2.1-py2.7.egg/mxnet/gluon/parameter.pyR'’s %cCs|dk rÌ|tkr|S|dkrQt|ƒdkrB|dStjƒ}n|j|jd@}|jt|ƒkrž||j}|dk rž||Sntd|j t |ƒt |j ƒfƒ‚n|j rët d|j ƒ‚ntd|j ƒ‚dS(NiisPParameter '%s' was not initialized on context %s. It was only initialized on %s.sGParameter '%s' has not been initialized yet because initialization was deferred. Actual initialization happens during the first forward pass. Please pass one batch of data through the network before accessing Parameters. You can also avoid deferred initialization by specifying in_units, num_features, etc., for network layers.sßParameter '%s' has not been initialized. Note that you should initialize parameters and create Trainer with Block.collect_params() instead of Block.params because the later does not include Parameters of nested child Blocks(RtlistR7R tcurrent_contextRt device_typeidt device_idt RuntimeErrorR R:RRR(R&tarr_listtctxtctx_listtidx((sU/usr/local/lib/python2.7/site-packages/mxnet-1.2.1-py2.7.egg/mxnet/gluon/parameter.pyt_check_and_getŸs,      ( cCs4|jr xit|j|jƒD]R\}}|dks||kstd|jt|jƒt|jƒfƒ‚qWtd„t|j|jƒDƒƒ|_n|jrøtj|jƒj|jksøtd|jt|jƒt|jƒfƒ‚nt |t ƒr|g}n|j dkrÂ|j r”|dks„t|ƒt|j dƒks„td|jt|ƒt|jƒƒfƒ‚|j d}n|dkr¯tƒg}n|j||ƒne|dkst|ƒt|jƒƒkstd|jt|ƒt|jƒƒfƒ‚|j|ƒd|_ dS( s%(Re)initializes by loading from data.is[Failed loading Parameter '%s' from saved params: shape incompatible expected %s vs saved %scss-|]#\}}|dkr!|n|VqdS(iN((R5R4R6((sU/usr/local/lib/python2.7/site-packages/mxnet-1.2.1-py2.7.egg/mxnet/gluon/parameter.pys Çss[Failed loading Parameter '%s' from saved params: dtype incompatible expected %s vs saved %sisNFailed to load Parameter '%s' on %s because it was previous initialized on %s.N((R'R9R0R R:ttupleR!tnpttypet isinstanceR RRRtsettlist_ctxR t _init_impltset_data(R&tdataRBtself_dimtdata_dim((sU/usr/local/lib/python2.7/site-packages/mxnet-1.2.1-py2.7.egg/mxnet/gluon/parameter.pyt _load_init¿s4 ",+ !+  .( -( c Cs|js dS|j\}}}}d|_|jdk rRtj|jƒdksttd|jt|jƒfƒ‚tj ƒz|dkræt j d|jd|j dt jƒƒ}tj|ƒtj|ji|d6ƒ|ƒn|j||ƒWdQXdS( s!Finishes deferred initialization.NiszCannot initialize Parameter '%s' because it has invalid shape: %s. Please specify in_units, in_channels, etc for `Block`s.R'R!RBR*((RR'RRGtprodR0R R:RtpauseR tzerosR!R R R tcreatetInitDescRL(R&R%RBt default_initRN((sU/usr/local/lib/python2.7/site-packages/mxnet-1.2.1-py2.7.egg/mxnet/gluon/parameter.pyt_finish_deferred_initás  *   #cCs¾t|ƒ|_ggg|_xgt|jƒD]V\}}|j|jd@}x&t|ƒ|jkrv|jdƒqQW|||jR7R?tappendRtcopytoRR2(R&RNRCR4RBtdev_list((sU/usr/local/lib/python2.7/site-packages/mxnet-1.2.1-py2.7.egg/mxnet/gluon/parameter.pyRLös(cCsj|jdkrd|_dSg|jD]}tj|ƒ^q&|_tj|jƒ|j ƒ|jƒdS(sInitialize grad buffers.R/N( R$RRRR t zeros_likeRtmark_variablest list_datat list_grad(R&R4((sU/usr/local/lib/python2.7/site-packages/mxnet-1.2.1-py2.7.egg/mxnet/gluon/parameter.pyR2s  (cCs3|jƒ}tjd„|DƒŒt|ƒ}|S(s"Reduce data from multiple context.css$|]}|jtjƒƒVqdS(N(R[R R (R5tw((sU/usr/local/lib/python2.7/site-packages/mxnet-1.2.1-py2.7.egg/mxnet/gluon/parameter.pys s(R_R tadd_nR7(R&tblockRN((sU/usr/local/lib/python2.7/site-packages/mxnet-1.2.1-py2.7.egg/mxnet/gluon/parameter.pyt_reduce s #cCs6|jdk r4| r4tjd|jddƒdSd|_|_|dkrbtjƒg}nt|t ƒr}|g}n|dkrª|j dkrž|n|j }n|j sÌt j |j ƒdkr|jrî|||df|_dStd|jt|j ƒfƒ‚n|||df|_|jƒdS(sŒInitializes parameter and gradient arrays. Only used for :py:class:`NDArray` API. Parameters ---------- init : Initializer The initializer to use. Overrides :py:meth:`Parameter.init` and default_init. ctx : Context or list of Context, defaults to :py:meth:`context.current_context()`. Initialize Parameter on given context. If ctx is a list of Context, a copy will be made for each context. .. note:: Copies are independent arrays. User is responsible for keeping their values consistent when updating. Normally :py:class:`gluon.Trainer` does this for you. default_init : Initializer Default initializer is used when both :py:func:`init` and :py:meth:`Parameter.init` are ``None``. force_reinit : bool, default False Whether to force re-initialization if parameter is already initialized. Examples -------- >>> weight = mx.gluon.Parameter('weight', shape=(2, 2)) >>> weight.initialize(ctx=mx.cpu(0)) >>> weight.data() [[-0.01068833 0.01729892] [ 0.02042518 -0.01618656]] >>> weight.grad() [[ 0. 0.] [ 0. 0.]] >>> weight.initialize(ctx=[mx.gpu(0), mx.gpu(1)]) >>> weight.data(mx.gpu(0)) [[-0.00873779 -0.02834515] [ 0.05484822 -0.06206018]] >>> weight.data(mx.gpu(1)) [[-0.00873779 -0.02834515] [ 0.05484822 -0.06206018]] sXParameter '%s' is already initialized, ignoring. Set force_reinit=True to re-initialize.t stackleveliNisBCannot initialize Parameter '%s' because it has invalid shape: %s.(RRtwarningstwarnR RR R=RIR R%R'RGRRRRt ValueErrorR:RX(R&R%RBRWt force_reinit((sU/usr/local/lib/python2.7/site-packages/mxnet-1.2.1-py2.7.egg/mxnet/gluon/parameter.pyt initializes(-     !" cCsÁ|dkrtjƒg}nt|tƒr9|g}n|jrt|jƒ}tjƒ|j ||ƒWdQXnI|j rª|j \}}}}||||f|_ nt d|j ƒ‚dS(s%Re-assign Parameter to other contexts. Parameters ---------- ctx : Context or list of Context, default ``context.current_context()``. Assign Parameter to given context. If ctx is a list of Context, a copy will be made for each context. NsLCannot reset context for Parameter '%s' because it has not been initialized.( RR R=RIR RRdRRSRLRRhR (R&RBRNR%t_RW((sU/usr/local/lib/python2.7/site-packages/mxnet-1.2.1-py2.7.egg/mxnet/gluon/parameter.pyt reset_ctxVs      cCsz|j|_|jdkrX|jdk s=td|jƒ‚|jd |f|_dSx|jƒD] }||(qeWdS(s,Sets this parameter's value on all contexts.s'Parameter '%s' has not been initializediN(R'RRRR0R R_(R&RNtarr((sU/usr/local/lib/python2.7/site-packages/mxnet-1.2.1-py2.7.egg/mxnet/gluon/parameter.pyRMos cCs|j|j|ƒS(sReturns a copy of this parameter on one context. Must have been initialized on this context before. Parameters ---------- ctx : Context Desired context. Returns ------- NDArray on ctx (RER(R&RB((sU/usr/local/lib/python2.7/site-packages/mxnet-1.2.1-py2.7.egg/mxnet/gluon/parameter.pyRN|s cCs|j|jtƒS(sXReturns copies of this parameter on all contexts, in the same order as creation.(RERR<(R&((sU/usr/local/lib/python2.7/site-packages/mxnet-1.2.1-py2.7.egg/mxnet/gluon/parameter.pyR_‹scCsG|jdk r4|jdkr4td|jƒ‚n|j|j|ƒS(sŸReturns a gradient buffer for this parameter on one context. Parameters ---------- ctx : Context Desired context. sDCannot get gradient array for Parameter '%s' because grad_req='null'N(RRRR@R RE(R&RB((sU/usr/local/lib/python2.7/site-packages/mxnet-1.2.1-py2.7.egg/mxnet/gluon/parameter.pytgrads cCsG|jdk r4|jdkr4td|jƒ‚n|j|jtƒS(sYReturns gradient buffers on all contexts, in the same order as :py:meth:`values`.sDCannot get gradient array for Parameter '%s' because grad_req='null'N(RRRR@R RER<(R&((sU/usr/local/lib/python2.7/site-packages/mxnet-1.2.1-py2.7.egg/mxnet/gluon/parameter.pyR`žs cCs@|jdkr9|jr#|jdStd|jƒ‚n|jS(s<Returns a list of contexts this parameter is initialized on.is'Parameter '%s' has not been initializedN(RRRR@R R(R&((sU/usr/local/lib/python2.7/site-packages/mxnet-1.2.1-py2.7.egg/mxnet/gluon/parameter.pyRK§s   cCs2|jdkrdSx|jD] }d|(qWdS(s€Sets gradient buffer on all contexts to 0. No action is taken if parameter is uninitialized or doesn't require gradient.Ni(RR(R&R4((sU/usr/local/lib/python2.7/site-packages/mxnet-1.2.1-py2.7.egg/mxnet/gluon/parameter.pyt zero_grad¯sc Cs[|jdkrTtj|jd|jd|jd|jd|jd|j ƒ|_n|jS(s-Returns a symbol representing this parameter.R'R!R"R#R%N( RRRtvarR R'R!R"R#R%(R&((sU/usr/local/lib/python2.7/site-packages/mxnet-1.2.1-py2.7.egg/mxnet/gluon/parameter.pyRp·s !cCs²||_|jdkrdStjƒ„g|jD]}|j|ƒ^q3|_|jdkrddSg|jD]}|j|ƒ^qn|_tj|j|j|jƒWdQXdS(s®Cast data and gradient of this Parameter to a new data type. Parameters ---------- dtype : str or numpy.dtype The new data type. N( R!RRRRStastypeRR^R$(R&R!R4((sU/usr/local/lib/python2.7/site-packages/mxnet-1.2.1-py2.7.egg/mxnet/gluon/parameter.pytcast¿s  ((N( RRRRRtFalsetTrueR*R-tpropertyR$tsetterR'RERQRXRLR2RdR tUniformRjRlRMRNR_RnR`RKRoRpRr(((sU/usr/local/lib/python2.7/site-packages/mxnet-1.2.1-py2.7.egg/mxnet/gluon/parameter.pyR,s66     "   C       cBs eZdZd„Zd„ZRS(sÇA constant parameter for holding immutable tensors. `Constant`s are ignored by `autograd` and `Trainer`, thus their values will not change during training. But you can still update their values manually with the `set_data` method. `Constant`s can be created with either:: const = mx.gluon.Constant('const', [[1,2],[3,4]]) or:: class Block(gluon.Block): def __init__(self, **kwargs): super(Block, self).__init__(**kwargs) self.const = self.params.get_constant('const', [[1,2],[3,4]]) Parameters ---------- name : str Name of the parameter. value : array-like Initial value for the constant. c s¯tˆtjƒs$tjˆƒ‰nˆ|_dtjf‡fd†ƒY}dj|t|ƒƒ}tj |ƒ|ƒt t |ƒj |dddˆj dˆjd|ƒdS( NtInitcseZ‡fd†ZRS(csˆj|ƒdS(N(R[(R&RkRm(tvalue(sU/usr/local/lib/python2.7/site-packages/mxnet-1.2.1-py2.7.egg/mxnet/gluon/parameter.pyt _init_weightðs(RRRz((Ry(sU/usr/local/lib/python2.7/site-packages/mxnet-1.2.1-py2.7.egg/mxnet/gluon/parameter.pyRxïssConstant_{}_{}R$R/R'R!R%(RIR tNDArraytarrayRyR t InitializerR+tidtaliastsuperRR*R'R!(R&R RyRxt init_name((RysU/usr/local/lib/python2.7/site-packages/mxnet-1.2.1-py2.7.egg/mxnet/gluon/parameter.pyR*ês cCs+d}|jd|jd|jd|jƒS(Ns.Constant {name} (shape={shape}, dtype={dtype})R R'R!(R+R R'R!(R&R,((sU/usr/local/lib/python2.7/site-packages/mxnet-1.2.1-py2.7.egg/mxnet/gluon/parameter.pyR-ùs(RRRR*R-(((sU/usr/local/lib/python2.7/site-packages/mxnet-1.2.1-py2.7.egg/mxnet/gluon/parameter.pyRÒs cBsàeZdZddd„Zd„Zd„Zd„Zd„Zd„Z d„Z e d „ƒZ d „Z d „Zdd „Zd „Zejƒdeed„Zd„Zd„Zd„Zdd„Zdeedd„ZRS(s·A dictionary managing a set of parameters. Parameters ---------- prefix : str, default ``''`` The prefix to be prepended to all Parameters' names created by this dict. shared : ParameterDict or None If not ``None``, when this dict's :py:meth:`get` method creates a new parameter, will first try to retrieve it from "shared" dict. Usually used for sharing parameters with another Block. tcCs"||_tƒ|_||_dS(N(t_prefixRt_paramst_shared(R&tprefixtshared((sU/usr/local/lib/python2.7/site-packages/mxnet-1.2.1-py2.7.egg/mxnet/gluon/parameter.pyR* s  c Csld}|jr|jdnd}|jd|ddjg|jƒD]}tdj|ƒdƒ^qDƒƒS( Ns{name}( {content} )t R‚R tcontents s {0}i(RƒR+tjointvaluesR(R&R,R tv((sU/usr/local/lib/python2.7/site-packages/mxnet-1.2.1-py2.7.egg/mxnet/gluon/parameter.pyR-s  cCs |j|S(N(R„(R&tkey((sU/usr/local/lib/python2.7/site-packages/mxnet-1.2.1-py2.7.egg/mxnet/gluon/parameter.pyt __getitem__scCs t|jƒS(N(titerR„(R&((sU/usr/local/lib/python2.7/site-packages/mxnet-1.2.1-py2.7.egg/mxnet/gluon/parameter.pyt__iter__scCs |jjƒS(N(R„titems(R&((sU/usr/local/lib/python2.7/site-packages/mxnet-1.2.1-py2.7.egg/mxnet/gluon/parameter.pyR‘scCs |jjƒS(N(R„tkeys(R&((sU/usr/local/lib/python2.7/site-packages/mxnet-1.2.1-py2.7.egg/mxnet/gluon/parameter.pyR’scCs |jjƒS(N(R„R‹(R&((sU/usr/local/lib/python2.7/site-packages/mxnet-1.2.1-py2.7.egg/mxnet/gluon/parameter.pyR‹"scCs|jS(snPrefix of this dict. It will be prepended to :py:class:`Parameter`s' name created with :py:func:`get`.(Rƒ(R&((sU/usr/local/lib/python2.7/site-packages/mxnet-1.2.1-py2.7.egg/mxnet/gluon/parameter.pyR†%scCsd||jkr|j|S|jdk r`||jjkr`|jj||j|<|jj|SdS(N(R„R…R(R&R ((sU/usr/local/lib/python2.7/site-packages/mxnet-1.2.1-py2.7.egg/mxnet/gluon/parameter.pyt _get_impl+s  !c KsÐ|j|}|j|ƒ}|dkrGt||}||j| s sˆParameter '%s' is missing in file '%s', which contains parameters: %s. Please make sure source and target networks have the same prefix.s¼Parameter '%s' loaded from file '%s' is not present in ParameterDict, choices are: %s. Set ignore_extra to True to ignore. Please make sure source and target networks have the same prefix.N( R’R¨R0R7R tloadR‘RR„RQ( R&RªRBt allow_missingt ignore_extraR®R tlprefixR™RŒtloadedR¬((R®sU/usr/local/lib/python2.7/site-packages/mxnet-1.2.1-py2.7.egg/mxnet/gluon/parameter.pyR¯ós* Y*  &N(RRRRR*R-RŽRR‘R’R‹RuR†R“RŸR£R¥R RwRsRjRoRlR–R©R¯(((sU/usr/local/lib/python2.7/site-packages/mxnet-1.2.1-py2.7.egg/mxnet/gluon/parameter.pyRþs*         7 (     (Rt__all__t collectionsRRftnumpyRGtbaseRRR‚RR R R R R RtutilsRRtSymbolR{RRtobjectRRR(((sU/usr/local/lib/python2.7/site-packages/mxnet-1.2.1-py2.7.egg/mxnet/gluon/parameter.pyts     "ÿ§,