ó šÄïYc@sîdZddlmZddlZddlZddlmZmZddl m Z m Z m Z m Z ddl mZddl mZd d lmZe je jfZd efd „ƒYZd efd„ƒYZdefd„ƒYZdS(sNeural network parameter.iÿÿÿÿ(t OrderedDictNi(t mx_real_tt MXNetError(tsymboltndarrayt initializertcontext(tContext(tautogradi(t_indenttDeferredInitializationErrorcBseZdZRS(s-Error for unfinished deferred initialization.(t__name__t __module__t__doc__(((s5build/bdist.linux-armv7l/egg/mxnet/gluon/parameter.pyR %st Parameterc BseZdZddedddeed„Zd„Ze d„ƒZ e j d„ƒZ dd„Z d„Z d „Zd „Zd „Zd „Zddejƒed „Zd„Zd„Zdd„Zd„Zdd„Zd„Zd„Zd„Zd„ZRS(sÛA Container holding parameters (weights) of `Block`s. `Parameter` holds a copy of the parameter on each `Context` after it is initialized with `Parameter.initialize(...)`. If `grad_req` is not `null`, it will also hold a gradient array on each `Context`:: ctx = mx.gpu(0) x = mx.nd.zeros((16, 100), ctx=ctx) w = mx.gluon.Parameter('fc_weight', shape=(64, 100), init=mx.init.Xavier()) b = mx.gluon.Parameter('fc_bias', shape=(64,), init=mx.init.Zero()) w.initialize(ctx=ctx) b.initialize(ctx=ctx) out = mx.nd.FullyConnected(x, w.data(ctx), b.data(ctx), num_hidden=64) Parameters ---------- name : str Name of this parameter. grad_req : {'write', 'add', 'null'}, default 'write' Specifies how to update gradient to grad arrays. - 'write' means everytime gradient is written to grad `NDArray`. - 'add' means everytime gradient is added to the grad `NDArray`. You need to manually call `zero_grad()` to clear the gradient buffer before each iteration when using this option. - 'null' means gradient is not requested for this parameter. gradient arrays will not be allocated. shape : tuple of int, default None Shape of this parameter. By default shape is not specified. Parameter with unknown shape can be used for `Symbol` API, but `init` will throw an error when using `NDArray` API. dtype : numpy.dtype or str, default 'float32' Data type of this parameter. For example, numpy.float32 or 'float32'. lr_mult : float, default 1.0 Learning rate multiplier. Learning rate will be multiplied by lr_mult when updating this parameter with optimizer. wd_mult : float, default 1.0 Weight decay multiplier (L2 regularizer coefficient). Works similar to lr_mult. init : Initializer, default None Initializer of this parameter. Will use the global initializer by default. Attributes ---------- grad_req : {'write', 'add', 'null'} This can be set before or after initialization. Setting grad_req to null with `x.grad_req = 'null'` saves memory and computation when you don't need gradient w.r.t x. twritegð?c Cs‚d|_d|_d|_d|_| |_d|_||_||_||_ ||_ ||_ ||_ ||_ ||_dS(N((tNonet_vart_datat_gradt_deferred_initt_differentiablet _grad_reqtnametshapetdtypetlr_multtwd_multtgrad_reqtinittallow_deferred_init( tselfRRRRRRRRtdifferentiable((s5build/bdist.linux-armv7l/egg/mxnet/gluon/parameter.pyt__init__Zs             cCsd}|j|jS(Ns/Parameter {name} (shape={shape}, dtype={dtype})(tformatt__dict__(Rts((s5build/bdist.linux-armv7l/egg/mxnet/gluon/parameter.pyt__repr__lscCs|jS(N(R(R((s5build/bdist.linux-armv7l/egg/mxnet/gluon/parameter.pyRpscCs¿|dkstd|ƒ‚|js.d}n|j|krAdS||_|dkrŸ|jdk rŸd|_xJ|jD] }|j|jƒ|j|Ýs(RGRtadd_ntlen(RtblockR8((s5build/bdist.linux-armv7l/egg/mxnet/gluon/parameter.pyt_reduceÚs #cCs*|jdk r.| r.tjd|jƒdSd|_|_|dkr\tjƒg}nt|t ƒrw|g}n|dkr¤|j dkr˜|n|j }n|j sÆt j |j ƒdkr |jrå|||f|_dStd|jt|j ƒfƒ‚n|||f|_|jƒdS(s!Initializes parameter and gradient arrays. Only used for `NDArray` API. Parameters ---------- init : Initializer The initializer to use. Overrides `Parameter.init` and default_init. ctx : Context or list of Context, defaults to `context.current_context()`. Initialize Parameter on given context. If ctx is a list of Context, a copy will be made for each context. .. note:: Copies are independent arrays. User is responsible for keeping their values consistent when updating. Normally `gluon.Trainer` does this for you. default_init : Initializer Default initializer is used when both `init` and `Parameter.init` are `None`. force_reinit : bool, default False Whether to force re-initialization if parameter is already initialized. Examples -------- >>> weight = mx.gluon.Parameter('weight', shape=(2, 2)) >>> weight.initialize(ctx=mx.cpu(0)) >>> weight.data() [[-0.01068833 0.01729892] [ 0.02042518 -0.01618656]] >>> weight.grad() [[ 0. 0.] [ 0. 0.]] >>> weight.initialize(ctx=[mx.gpu(0), mx.gpu(1)]) >>> weight.data(mx.gpu(0)) [[-0.00873779 -0.02834515] [ 0.05484822 -0.06206018]] >>> weight.data(mx.gpu(1)) [[-0.00873779 -0.02834515] [ 0.05484822 -0.06206018]] sVParameter %s is already initialized, ignoring. Set force_reinit=True to re-initialize.Nis@Cannot initialize Parameter %s because it has invalid shape: %s.(RRtwarningstwarnRRRtcurrent_contextR4RRRR2R<RRt ValueErrorR.RC(RRR,RBt force_reinit((s5build/bdist.linux-armv7l/egg/mxnet/gluon/parameter.pyt initializeàs&)     !" cCs»|dkrtjƒg}nt|tƒr9|g}n|jrt|jƒ}tjƒ|j ||ƒWdQXnC|j r¤|j \}}}|||f|_ nt d|j ƒ‚dS(sýRe-assign Parameter to other contexts. ctx : Context or list of Context, default `context.current_context()`. Assign Parameter to given context. If ctx is a list of Context, a copy will be made for each context. NsJCannot reset context for Parameter %s because it has not been initialized.( RRRQR4RRRNRR=R6RRRR(RR,R8Rt_RB((s5build/bdist.linux-armv7l/egg/mxnet/gluon/parameter.pyt reset_ctxs      cCsD|jdk s"td|jƒ‚x|jƒD] }||(q/WdS(s4Sets this parameter's value on all contexts to data.s%Parameter %s has not been initializedN(RRR(RRG(RR8tarr((s5build/bdist.linux-armv7l/egg/mxnet/gluon/parameter.pyR76scCs^|dkrF|jƒ}t|ƒdkr7|d}qFtjƒ}n|j|ƒ|j|S(sReturns a copy of this parameter on one context. Must have been initialized on this context before. Parameters ---------- ctx : Context Desired context. Returns ------- NDArray on ctx iiN(RR/RLRRQR0R(RR,R/((s5build/bdist.linux-armv7l/egg/mxnet/gluon/parameter.pyR8=s    cCs|jƒt|jjƒƒS(sXReturns copies of this parameter on all contexts, in the same order as creation.(R0tlistRtvalues(R((s5build/bdist.linux-armv7l/egg/mxnet/gluon/parameter.pyRGSs cCsƒ|dkrF|jƒ}t|ƒdkr7|d}qFtjƒ}n|j|ƒ|jdkrxtd|jƒ‚n|j|S(sŸReturns a gradient buffer for this parameter on one context. Parameters ---------- ctx : Context Desired context. iisBCannot get gradient array for Parameter %s because grad_req='null'N( RR/RLRRQR0RR-R(RR,R/((s5build/bdist.linux-armv7l/egg/mxnet/gluon/parameter.pytgradYs    cCs?|jƒ|jdk s,td|jƒ‚t|jjƒƒS(sPReturns gradient buffers on all contexts, in the same order as `values`.s<Parameter %s does not have gradients because grad_req='null'N(R0RRR(RRXRY(R((s5build/bdist.linux-armv7l/egg/mxnet/gluon/parameter.pyRHns cCsL|jdkr9|jr#|jdStd|jƒ‚nt|jjƒƒS(s<Returns a list of contexts this parameter is initialized on.is%Parameter %s has not been initializedN(RRRR-RRXtkeys(R((s5build/bdist.linux-armv7l/egg/mxnet/gluon/parameter.pyR/vs   cCs8|jdkrdSx|jjƒD] }d|(q#WdS(s€Sets gradient buffer on all contexts to 0. No action is taken if parameter is uninitialized or doesn't require gradient.Ni(RRRY(RR9((s5build/bdist.linux-armv7l/egg/mxnet/gluon/parameter.pyt zero_grad~sc Cs[|jdkrTtj|jd|jd|jd|jd|jd|j ƒ|_n|jS(s-Returns a symbol representing this parameter.RRRRRN( RRRtvarRRRRRR(R((s5build/bdist.linux-armv7l/egg/mxnet/gluon/parameter.pyR]†s !N(R R R RRtFalsetTrueR!R%tpropertyRtsetterR0R;RCR6R*RNRtUniformRTRVR7R8RGRZRHR/R\R](((s5build/bdist.linux-armv7l/egg/mxnet/gluon/parameter.pyR)s00          >        t ParameterDictcBsÑeZdZddd„Zd„Zd„Zd„Zd„Zd„Z d„Z e d „ƒZ d „Z d „Zd „Zejƒdeed „Zd„Zd„Zd„Zdd„Zeedd„ZRS(sªA dictionary managing a set of parameters. Parameters ---------- prefix : str, default '' The prefix to be prepended to all Parameters' names created by this dict. shared : ParameterDict or None If not `None`, when this dict's `get` method creates a new parameter, will first try to retrieve it from `shared` dict. Usually used for sharing parameters with another `Block`. tcCs"||_tƒ|_||_dS(N(t_prefixRt_paramst_shared(Rtprefixtshared((s5build/bdist.linux-armv7l/egg/mxnet/gluon/parameter.pyR!›s  c Csld}|jr|jdnd}|jd|ddjg|jƒD]}tdj|ƒdƒ^qDƒƒS( Ns{name}( {content} )t RdRtcontents s {0}i(ReR"tjoinRYR (RR$Rtv((s5build/bdist.linux-armv7l/egg/mxnet/gluon/parameter.pyR% s  cCs |j|S(N(Rf(Rtkey((s5build/bdist.linux-armv7l/egg/mxnet/gluon/parameter.pyt __getitem__§scCs t|jƒS(N(titerRf(R((s5build/bdist.linux-armv7l/egg/mxnet/gluon/parameter.pyt__iter__ªscCs |jjƒS(N(Rftitems(R((s5build/bdist.linux-armv7l/egg/mxnet/gluon/parameter.pyRr­scCs |jjƒS(N(RfR[(R((s5build/bdist.linux-armv7l/egg/mxnet/gluon/parameter.pyR[°scCs |jjƒS(N(RfRY(R((s5build/bdist.linux-armv7l/egg/mxnet/gluon/parameter.pyRY³scCs|jS(sYPrefix of this dict. It will be prepended to Parameters' name created with `get`.(Re(R((s5build/bdist.linux-armv7l/egg/mxnet/gluon/parameter.pyRh¶scCsd||jkr|j|S|jdk r`||jjkr`|jj||j|<|jj|SdS(N(RfRgR(RR((s5build/bdist.linux-armv7l/egg/mxnet/gluon/parameter.pyt _get_impl¼s  !c Ksñ|j|}|j|ƒ}|dkrGt||}||j|Xs s"Parameter %s is missing in file %ss@Parameter %s loaded from file %s is not present in ParameterDictN( R[R€R(RLRtloadRrRfR;( RR‚R,t allow_missingt ignore_extraR†RtlprefixR„((R†s5build/bdist.linux-armv7l/egg/mxnet/gluon/parameter.pyR‡Bs$ %  N(R R R RR!R%RoRqRrR[RYR`RhRsRzR|RRbR^RTR\RVRvRR‡(((s5build/bdist.linux-armv7l/egg/mxnet/gluon/parameter.pyRcs(         $     (R t collectionsRROtnumpyR2tbaseRRRdRRRRRRtutilsR tSymboltNDArrayt tensor_typesR tobjectRRc(((s5build/bdist.linux-armv7l/egg/mxnet/gluon/parameter.pyts  "ÿg