ó ùµÈ[c@sódZddlZddlZddlZddlZddlZddlmZddlm Z ddl m Z ddl m Z ddlmZdd lmZmZmZdd lmZd „Zd „Zd „Zdefd„ƒYZdS(s(`BaseModule` defines an API for modules.iÿÿÿÿNi(tmetric(tndarray(tcpu(t BatchEndParam(tUniform(tDataDesctDataItert DataBatch(t_as_listc CsÓ|jƒ}xÀ|D]¸}||kr+qng|D]L}|jdƒ r2|jdƒ r2|jdƒ r2|jdƒ r2|^q2}d|t|ƒ|dj|ƒf}|r¾t|ƒ‚qtj|ƒqWdS(s5Check that all input names are in symbol's arguments.t_weightt_biast_gammat_betas”You created Module with Module(..., %s_names=%s) but input with name '%s' is not found in symbol.list_arguments(). Did you mean one of: %ss N(tlist_argumentstendswithtstrtjoint ValueErrortwarningstwarn( tsymboltnamesttypenametthrowtargstnametargt candidatestmsg((sX/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/module/base_module.pyt_check_input_names&s    "cCs€g|D]}|d^q}t|ƒt|ƒkr|d||t|ƒt|ƒf}|rlt|ƒ‚q|tj|ƒndS(s6Check that input names matches input data descriptors.isNData provided by %s_shapes don't match names specified by %s_names (%s vs. %s)N(tsortedRRRR(t data_namest data_shapesRRtxtactualR((sX/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/module/base_module.pyt_check_names_match;scCsºg|D]'}t|tƒr"|n t|Œ^q}t||dtƒ|dk rg|D]'}t|tƒru|n t|Œ^qZ}t||dtƒnt|gdtƒ||fS(s@parse data_attrs into DataDesc format and check that names matchtdatatlabelN(t isinstanceRR#tTruetNonetFalse(Rt label_namesR t label_shapesR!((sX/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/module/base_module.pyt_parse_data_descGs4 4t BaseModulecBseZdZed„Zd„Zd%d%d%edd%d„Zd%ed%d„Z d%eee d%d„Z d%dd%d%dd d'd%d%e d ƒd%d%e e e dd%d%d%d%d „Z ed „ƒZed„ƒZed„ƒZed„ƒZed„ƒZd„Ze d ƒd%d%e e e d„Ze ee d„Zd„Zd„Zed„Zd%d%d„Zd„Zd%d„Zd%d„Zd%d„Zed„Zed„Z d„Z!e d „Z"d%ee e d%d!d"„Z#dd d)e d#„Z$ed$„ƒZ%RS(*söThe base class of a module. A module represents a computation component. One can think of module as a computation machine. A module can execute forward and backward passes and update parameters in a model. We aim to make the APIs easy to use, especially in the case when we need to use the imperative API to work with multiple modules (e.g. stochastic depth network). A module has several states: - Initial state: Memory is not allocated yet, so the module is not ready for computation yet. - Binded: Shapes for inputs, outputs, and parameters are all known, memory has been allocated, and the module is ready for computation. - Parameters are initialized: For modules with parameters, doing computation before initializing the parameters might result in undefined outputs. - Optimizer is installed: An optimizer can be installed to a module. After this, the parameters of the module can be updated according to the optimizer after gradients are computed (forward-backward). In order for a module to interact with others, it must be able to report the following information in its initial state (before binding): - `data_names`: list of type string indicating the names of the required input data. - `output_names`: list of type string indicating the names of the required outputs. After binding, a module should be able to report the following richer information: - state information - `binded`: `bool`, indicates whether the memory buffers needed for computation have been allocated. - `for_training`: whether the module is bound for training. - `params_initialized`: `bool`, indicates whether the parameters of this module have been initialized. - `optimizer_initialized`: `bool`, indicates whether an optimizer is defined and initialized. - `inputs_need_grad`: `bool`, indicates whether gradients with respect to the input data are needed. Might be useful when implementing composition of modules. - input/output information - `data_shapes`: a list of `(name, shape)`. In theory, since the memory is allocated, we could directly provide the data arrays. But in the case of data parallelism, the data arrays might not be of the same shape as viewed from the external world. - `label_shapes`: a list of `(name, shape)`. This might be `[]` if the module does not need labels (e.g. it does not contains a loss function at the top), or a module is not bound for training. - `output_shapes`: a list of `(name, shape)` for outputs of the module. - parameters (for modules with parameters) - `get_params()`: return a tuple `(arg_params, aux_params)`. Each of those is a dictionary of name to ``NDArray`` mapping. Those `NDArray` always lives on CPU. The actual parameters used for computing might live on other devices (GPUs), this function will retrieve (a copy of) the latest parameters. - ``set_params(arg_params, aux_params)``: assign parameters to the devices doing the computation. - ``init_params(...)``: a more flexible interface to assign or initialize the parameters. - setup - `bind()`: prepare environment for computation. - `init_optimizer()`: install optimizer for parameter updating. - `prepare()`: prepare the module based on the current data batch. - computation - `forward(data_batch)`: forward operation. - `backward(out_grads=None)`: backward operation. - `update()`: update parameters according to installed optimizer. - `get_outputs()`: get outputs of the previous forward operation. - `get_input_grads()`: get the gradients with respect to the inputs computed in the previous backward operation. - `update_metric(metric, labels, pre_sliced=False)`: update performance metric for the previous forward computed results. - other properties (mostly for backward compatibility) - `symbol`: the underlying symbolic graph for this module (if any) This property is not necessarily constant. For example, for `BucketingModule`, this property is simply the *current* symbol being used. For other modules, this value might not be well defined. When those intermediate-level API are implemented properly, the following high-level API will be automatically available for a module: - `fit`: train the module parameters on a data set. - `predict`: run prediction on a data set and collect outputs. - `score`: run prediction on a data set and evaluate performance. Examples -------- >>> # An example of creating a mxnet module. >>> import mxnet as mx >>> data = mx.symbol.Variable('data') >>> fc1 = mx.symbol.FullyConnected(data, name='fc1', num_hidden=128) >>> act1 = mx.symbol.Activation(fc1, name='relu1', act_type="relu") >>> fc2 = mx.symbol.FullyConnected(act1, name = 'fc2', num_hidden = 64) >>> act2 = mx.symbol.Activation(fc2, name='relu2', act_type="relu") >>> fc3 = mx.symbol.FullyConnected(act2, name='fc3', num_hidden=10) >>> out = mx.symbol.SoftmaxOutput(fc3, name = 'softmax') >>> mod = mx.mod.Module(out) cCsL||_t|_t|_t|_t|_t|_d|_d|_ dS(Ni( tloggerR)tbindedt for_trainingtinputs_need_gradtparams_initializedtoptimizer_initializedR(t_symbolt_total_exec_bytes(tselfR.((sX/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/module/base_module.pyt__init__µs       cCs!|j|dtƒ|jƒdS(sCA convenient function that calls both ``forward`` and ``backward``.tis_trainN(tforwardR'tbackward(R6t data_batch((sX/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/module/base_module.pytforward_backwardÂsic  CsÅ|jr|jst‚|r+|jƒnt|tjƒsOtj|ƒ}n|jƒd} x t|ƒD]ý\} } |d k r”| |kr”Pn|j | d|ƒ|j | dt ƒt| t ƒrø|j|g| D]} | j^qÙdtƒn|j|| jƒ|d k r_td|d| d|dtƒƒ} x!t|ƒD]}|| ƒqHWn| d 7} qlW|r»td|d| d|dtƒƒ}x!t|ƒD]}||ƒq¤Wn|jƒS( seRuns prediction on ``eval_data`` and evaluates the performance according to the given ``eval_metric``. Checkout `Module Tutorial `_ to see a end-to-end use-case. Parameters ---------- eval_data : DataIter Evaluation data to run prediction on. eval_metric : EvalMetric or list of EvalMetrics Evaluation metric to use. num_batch : int Number of batches to run. Defaults to ``None``, indicating run until the `DataIter` finishes. batch_end_callback : function Could also be a list of functions. reset : bool Defaults to ``True``. Indicates whether we should reset `eval_data` before starting evaluating. epoch : int Defaults to 0. For compatibility, this will be passed to callbacks (if any). During training, this will correspond to the training epoch number. sparse_row_id_fn : A callback function The function takes `data_batch` as an input and returns a dict of str -> NDArray. The resulting dict is used for pulling row_sparse parameters from the kvstore, where the str key is the name of the param, and the value is the row id of the param to pull. Examples -------- >>> # An example of using score for prediction. >>> # Evaluate accuracy on val_dataiter >>> metric = mx.metric.Accuracy() >>> mod.score(val_dataiter, metric) >>> mod.score(val_dataiter, ['mse', 'acc']) itsparse_row_id_fnR8t pre_slicedtepochtnbatcht eval_metrictlocalsiN(R/R2tAssertionErrortresetR&Rt EvalMetrictcreatet enumerateR(tprepareR9R)tlistt update_metricR%R'RRBRtget_name_value(R6t eval_dataRAt num_batchtbatch_end_callbacktscore_end_callbackRDR?R=tactual_num_batchR@t eval_batchtebtbatch_end_paramstcallbacktparams((sX/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/module/base_module.pytscoreÇs>(  /     c csÖ|jr|jst‚|r+|jƒnx¤t|ƒD]–\}}|dk r`||kr`Pn|j|d|ƒ|j|dtƒ|j }g|j ƒD]}|d|j d|!^qœ} | ||fVq8WdS(sIterates over predictions. Example Usage: ---------- >>> for pred, i_batch, batch in module.iter_predict(eval_data): ... # pred is a list of outputs from the module ... # i_batch is a integer ... # batch is the data batch from the data iterator Parameters ---------- eval_data : DataIter Evaluation data to run prediction on. num_batch : int Default is ``None``, indicating running all the batches in the data iterator. reset : bool Default is ``True``, indicating whether we should reset the data iter before start doing prediction. sparse_row_id_fn : A callback function The function takes `data_batch` as an input and returns a dict of str -> NDArray. The resulting dict is used for pulling row_sparse parameters from the kvstore, where the str key is the name of the param, and the value is the row id of the param to pull. R=R8iN( R/R2RCRDRGR(RHR9R)tpadt get_outputstshape( R6RLRMRDR=R@RQRWtouttoutputs((sX/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/module/base_module.pyt iter_predicts  1cCs|jr|jst‚t|tjtjfƒr{t|tjƒrWtj|ƒ}n|jt |gƒƒ|j ƒdSt|t ƒs™t dƒ‚n|r¬|j ƒng}x©t|ƒD]›\}} |dk rç||krçPn|j| d|ƒ|j| dtƒ| j} g|j ƒD]$} | d| jd| !jƒ^q#} |j| ƒq¿Wt|ƒdkrt|S|rt|dƒ} x0|D](} t| ƒ| ks‘tddƒ‚q‘Wgt| ƒD],}tjg|D]} | |^q݃^qÊ}| dkr| r|dS|S|S( sJRuns prediction and collects the outputs. When `merge_batches` is ``True`` (by default), the return value will be a list ``[out1, out2, out3]``, where each element is formed by concatenating the outputs for all the mini-batches. When `always_output_list` is ``False`` (as by default), then in the case of a single output, `out1` is returned instead of ``[out1]``. When `merge_batches` is ``False``, the return value will be a nested list like ``[[out1_batch1, out2_batch1], [out1_batch2], ...]``. This mode is useful because in some cases (e.g. bucketing), the module does not necessarily produce the same number of outputs. The objects in the results have type `NDArray`. If you need to work with a numpy array, just call ``.asnumpy()`` on each `NDArray`. Parameters ---------- eval_data : DataIter or NDArray or numpy array Evaluation data to run prediction on. num_batch : int Defaults to ``None``, indicates running all the batches in the data iterator. merge_batches : bool Defaults to ``True``, see above for return values. reset : bool Defaults to ``True``, indicates whether we should reset the data iter before doing prediction. always_output_list : bool Defaults to ``False``, see above for return values. sparse_row_id_fn : A callback function The function takes `data_batch` as an input and returns a dict of str -> NDArray. The resulting dict is used for pulling row_sparse parameters from the kvstore, where the str key is the name of the param, and the value is the row id of the param to pull. Returns ------- list of NDArray or list of list of NDArray Prediction results. Examples -------- >>> # An example of using `predict` for prediction. >>> # Predict on the first 10 batches of val_dataiter >>> mod.predict(eval_data=val_dataiter, num_batch=10) is-eval_data must be of type NDArray or DataIterR=R8s8Cannot merge batches, as num of outputs is not the same s)in mini-batches. Maybe bucketing is used?iN(R/R2RCR&RtNDArraytnptarrayR9RRXRRRDRGR(RHR)RWRYtcopytappendtlentranget concatenate(R6RLRMt merge_batchesRDtalways_output_listR=t output_listR@RQRWRZR[t num_outputstit output_list2((sX/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/module/base_module.pytpredict?sB/  7 <tacctlocaltsgdt learning_rateg{®Gáz„?c& Cs|dk stdƒ‚|jd|jd|jdtd|ƒ|dk r\|j|ƒn|jd| d| d| d |d |ƒ|jd |d |d |ƒ|dkrµ|}nt |t j ƒsÙt j |ƒ}nt j|ƒ}xt||ƒD]}tjƒ}|jƒ|jƒd}t|ƒ}t}t|ƒ}x™|sÝ|}|dk rj|jƒn|j|ƒ|jƒt |tƒrë|j|g|D]}|j^q dtƒ|j|g|D]}|j^qÌdtƒn&|j||jƒ|j||jƒy#t|ƒ}|j|d|ƒWntk rMt}nX|dk rg|jƒn|r||jƒ}n|dk rÐtd|d|d|dt ƒƒ} x!t!|ƒD]}!|!| ƒq¹Wn|d7}qEWx-|D]%\}"}#|j"j#d||"|#ƒqåWtjƒ}$|j"j#d||$|ƒ|j$ƒ\} } |j%| | ƒ|dk r’x-t!|ƒD]}!|!||j&| | ƒqoWn|rï|j'||d| d| d|ƒ}%x0|%D]%\}"}#|j"j#d||"|#ƒqÃWn|jƒqøWdS(s‚Trains the module parameters. Checkout `Module Tutorial `_ to see a end-to-end use-case. Parameters ---------- train_data : DataIter Train DataIter. eval_data : DataIter If not ``None``, will be used as validation set and the performance after each epoch will be evaluated. eval_metric : str or EvalMetric Defaults to 'accuracy'. The performance measure used to display during training. Other possible predefined metrics are: 'ce' (CrossEntropy), 'f1', 'mae', 'mse', 'rmse', 'top_k_accuracy'. epoch_end_callback : function or list of functions Each callback will be called with the current `epoch`, `symbol`, `arg_params` and `aux_params`. batch_end_callback : function or list of function Each callback will be called with a `BatchEndParam`. kvstore : str or KVStore Defaults to 'local'. optimizer : str or Optimizer Defaults to 'sgd'. optimizer_params : dict Defaults to ``(('learning_rate', 0.01),)``. The parameters for the optimizer constructor. The default value is not a dict, just to avoid pylint warning on dangerous default values. eval_end_callback : function or list of function These will be called at the end of each full evaluation, with the metrics over the entire evaluation set. eval_batch_end_callback : function or list of function These will be called at the end of each mini-batch during evaluation. initializer : Initializer The initializer is called to initialize the module parameters when they are not already initialized. arg_params : dict Defaults to ``None``, if not ``None``, should be existing parameters from a trained model or loaded from a checkpoint (previously saved model). In this case, the value here will be used to initialize the module parameters, unless they are already initialized by the user via a call to `init_params` or `fit`. `arg_params` has a higher priority than `initializer`. aux_params : dict Defaults to ``None``. Similar to `arg_params`, except for auxiliary states. allow_missing : bool Defaults to ``False``. Indicates whether to allow missing parameters when `arg_params` and `aux_params` are not ``None``. If this is ``True``, then the missing parameters will be initialized via the `initializer`. force_rebind : bool Defaults to ``False``. Whether to force rebinding the executors if already bound. force_init : bool Defaults to ``False``. Indicates whether to force initialization even if the parameters are already initialized. begin_epoch : int Defaults to 0. Indicates the starting epoch. Usually, if resumed from a checkpoint saved at a previous training phase at epoch N, then this value should be N+1. num_epoch : int Number of epochs for training. sparse_row_id_fn : A callback function The function takes `data_batch` as an input and returns a dict of str -> NDArray. The resulting dict is used for pulling row_sparse parameters from the kvstore, where the str key is the name of the param, and the value is the row id of the param to pull. Examples -------- >>> # An example of using fit for training. >>> # Assume training dataIter and validation dataIter are ready >>> # Assume loading a previously checkpointed model >>> sym, arg_params, aux_params = mx.model.load_checkpoint(model_prefix, 3) >>> mod.fit(train_data=train_dataiter, eval_data=val_dataiter, optimizer='sgd', ... optimizer_params={'learning_rate':0.01, 'momentum': 0.9}, ... arg_params=arg_params, aux_params=aux_params, ... eval_metric='acc', num_epoch=10, begin_epoch=3) splease specify number of epochsR R+R0t force_rebindt initializert arg_paramst aux_paramst allow_missingt force_inittkvstoret optimizertoptimizer_paramsiR>R=R?R@RARBisEpoch[%d] Train-%s=%fsEpoch[%d] Time cost=%.3fRORNsEpoch[%d] Validation-%s=%fN((R(RCtbindt provide_datat provide_labelR'tinstall_monitort init_paramstinit_optimizerR&RRERFR`tdeepcopyRcttimeRDtiterR)tnexttticR<tupdateRIRJR%RHt StopIterationt toc_printRKRRBRR.tinfot get_paramst set_paramsRRV(&R6t train_dataRLRAtepoch_end_callbackRNRvRwRxteval_end_callbackteval_batch_end_callbackRqRrRsRtRpRut begin_epocht num_epochtvalidation_metrictmonitorR=tepoch_eval_metricR?RƒR@t data_itert end_of_batchtnext_data_batchR;tdbteval_name_valsRSRTRtvalttoctres((sX/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/module/base_module.pytfitšsŠV                            cCs tƒ‚dS(s1A list of names for data required by this module.N(tNotImplementedError(R6((sX/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/module/base_module.pyRQscCs tƒ‚dS(s/A list of names for the outputs of this module.N(Rœ(R6((sX/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/module/base_module.pyt output_namesVscCs tƒ‚dS(sHA list of (name, shape) pairs specifying the data inputs to this module.N(Rœ(R6((sX/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/module/base_module.pyR ^scCs tƒ‚dS(sA list of (name, shape) pairs specifying the label inputs to this module. If this module does not accept labels -- either it is a module without loss function, or it is not bound for training, then this should return an empty list ``[]``. N(Rœ(R6((sX/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/module/base_module.pyR+cscCs tƒ‚dS(sDA list of (name, shape) pairs specifying the outputs of this module.N(Rœ(R6((sX/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/module/base_module.pyt output_shapeslscCs tƒ‚dS(s•Gets parameters, those are potentially copies of the the actual parameters used to do computation on the device. Returns ------- ``(arg_params, aux_params)`` A pair of dictionaries each mapping parameter names to NDArray values. Examples -------- >>> # An example of getting module parameters. >>> print mod.get_params() ({'fc2_weight': , 'fc1_weight': , 'fc3_bias': , 'fc3_weight': , 'fc2_bias': , 'fc1_bias': }, {}) N(Rœ(R6((sX/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/module/base_module.pyRˆtscCs tƒ‚dS(sœInitializes the parameters and auxiliary states. Parameters ---------- initializer : Initializer Called to initialize parameters if needed. arg_params : dict If not ``None``, should be a dictionary of existing `arg_params`. Initialization will be copied from that. aux_params : dict If not ``None``, should be a dictionary of existing `aux_params`. Initialization will be copied from that. allow_missing : bool If ``True``, params could contain missing values, and the initializer will be called to fill those missing params. force_init : bool If ``True``, `force_init` will force re-initialize even if already initialized. allow_extra : boolean, optional Whether allow extra parameters that are not needed by symbol. If this is True, no error will be thrown when arg_params or aux_params contain extra parameters that is not needed by the executor. Examples -------- >>> # An example of initializing module parameters. >>> mod.init_params() N(Rœ(R6RqRrRsRtRut allow_extra((sX/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/module/base_module.pyR}‡sc Cs2|jddd|d|d|d|d|ƒdS(s&Assigns parameter and aux state values. Parameters ---------- arg_params : dict Dictionary of name to value (`NDArray`) mapping. aux_params : dict Dictionary of name to value (`NDArray`) mapping. allow_missing : bool If ``True``, params could contain missing values, and the initializer will be called to fill those missing params. force_init : bool If ``True``, will force re-initialize even if already initialized. allow_extra : boolean, optional Whether allow extra parameters that are not needed by symbol. If this is True, no error will be thrown when arg_params or aux_params contain extra parameters that is not needed by the executor. Examples -------- >>> # An example of setting module parameters. >>> sym, arg_params, aux_params = mx.model.load_checkpoint(model_prefix, n_epoch_load) >>> mod.set_params(arg_params=arg_params, aux_params=aux_params) RqRrRsRtRuRŸN(R}R((R6RrRsRtRuRŸ((sX/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/module/base_module.pyR‰¦s cCsY|jƒ\}}d„|jƒDƒ}|jd„|jƒDƒƒtj||ƒdS(s Saves model parameters to file. Parameters ---------- fname : str Path to output param file. Examples -------- >>> # An example of saving module parameters. >>> mod.save_params('myfile') cSs/i|]%\}}|jtƒƒd|“qS(sarg:%s(t as_in_contextR(t.0tktv((sX/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/module/base_module.pys Òs cSs/i|]%\}}|jtƒƒd|“qS(saux:%s(R R(R¡R¢R£((sX/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/module/base_module.pys Ós N(RˆtitemsR„Rtsave(R6tfnameRrRst save_dict((sX/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/module/base_module.pyt save_paramsÄs c Cs¦tj|ƒ}i}i}xt|jƒD]f\}}|jddƒ\}}|dkre|||>> # An example of loading module parameters. >>> mod.load_params('myfile') t:iRtauxsInvalid param file N(RtloadR¤tsplitRR‰( R6R¦R§RrRsR¢tvaluetarg_typeR((sX/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/module/base_module.pyt load_paramsÖs     cCs)|jr|jst‚| s%t‚gS(sÜGets states from all devices If `merge_multi_context` is ``True``, returns output of form ``[out1, out2]``. Otherwise, it returns output of the form ``[[out1_dev1, out1_dev2], [out2_dev1, out2_dev2]]``. All output elements are `NDArray`. Parameters ---------- merge_multi_context : bool Defaults to ``True``. In the case when data-parallelism is used, the states will be collected from multiple devices. A ``True`` value indicates that we should merge the collected results so that they look like from a single executor. Returns ------- A list of ``NDArray`` or a list of list of ``NDArray``. (R/R2RC(R6tmerge_multi_context((sX/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/module/base_module.pyt get_statesðs cCs0|jr|jst‚| r&| s,t‚dS(seSets value for states. Only one of states & value can be specified. Parameters ---------- states : list of list of NDArray Source states arrays formatted like ``[[state1_dev1, state1_dev2], [state2_dev1, state2_dev2]]``. value : number A single scalar value for all state arrays. N(R/R2RC(R6tstatesR­((sX/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/module/base_module.pyt set_statess cCs tƒ‚dS(s"Installs monitor on all executors.N(Rœ(R6tmon((sX/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/module/base_module.pyR|scCs&|dk r"tjtdƒƒndS(s¥Prepares the module for processing a data batch. Usually involves switching bucket and reshaping. For modules that contain `row_sparse` parameters in KVStore, it prepares the `row_sparse` parameters based on the sparse_row_id_fn. When KVStore is used to update parameters for multi-device or multi-machine training, a copy of the parameters are stored in KVStore. Note that for `row_sparse` parameters, the `update()` updates the copy of parameters in KVStore, but doesn't broadcast the updated parameters to all devices / machines. The `prepare` function is used to broadcast `row_sparse` parameters with the next batch of data. Parameters ---------- data_batch : DataBatch The current batch of data for forward computation. sparse_row_id_fn : A callback function The function takes `data_batch` as an input and returns a dict of str -> NDArray. The resulting dict is used for pulling row_sparse parameters from the kvstore, where the str key is the name of the param, and the value is the row id of the param to pull. s/sparse_row_id_fn is not invoked for BaseModule.N(R(RRt UserWarning(R6R;R=((sX/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/module/base_module.pyRHs cCs tƒ‚dS(sŸForward computation. It supports data batches with different shapes, such as different batch sizes or different image sizes. If reshaping of data batch relates to modification of symbol or module, such as changing image layout ordering or switching from training to predicting, module rebinding is required. Parameters ---------- data_batch : DataBatch Could be anything with similar API implemented. is_train : bool Default is ``None``, which means `is_train` takes the value of ``self.for_training``. Examples -------- >>> import mxnet as mx >>> from collections import namedtuple >>> Batch = namedtuple('Batch', ['data']) >>> data = mx.sym.Variable('data') >>> out = data * 2 >>> mod = mx.mod.Module(symbol=out, label_names=None) >>> mod.bind(data_shapes=[('data', (1, 10))]) >>> mod.init_params() >>> data1 = [mx.nd.ones((1, 10))] >>> mod.forward(Batch(data1)) >>> print mod.get_outputs()[0].asnumpy() [[ 2. 2. 2. 2. 2. 2. 2. 2. 2. 2.]] >>> # Forward with data batch of different shape >>> data2 = [mx.nd.ones((3, 5))] >>> mod.forward(Batch(data2)) >>> print mod.get_outputs()[0].asnumpy() [[ 2. 2. 2. 2. 2.] [ 2. 2. 2. 2. 2.] [ 2. 2. 2. 2. 2.]] N(Rœ(R6R;R8((sX/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/module/base_module.pyR9:s$cCs tƒ‚dS(s°Backward computation. Parameters ---------- out_grads : NDArray or list of NDArray, optional Gradient on the outputs to be propagated back. This parameter is only needed when bind is called on outputs that are not a loss function. Examples -------- >>> # An example of backward computation. >>> mod.backward() >>> print mod.get_input_grads()[0].asnumpy() [[[ 1.10182791e-05 5.12257748e-06 4.01927764e-06 8.32566820e-06 -1.59775993e-06 7.24269375e-06 7.28067835e-06 -1.65902311e-05 5.46342608e-06 8.44196393e-07] ...]] N(Rœ(R6t out_grads((sX/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/module/base_module.pyR:`scCs tƒ‚dS(srGets outputs of the previous forward computation. If `merge_multi_context` is ``True``, it is like ``[out1, out2]``. Otherwise, it returns out put of form ``[[out1_dev1, out1_dev2], [out2_dev1, out2_dev2]]``. All the output elements have type `NDArray`. When `merge_multi_context` is ``False``, those `NDArray` instances might live on different devices. Parameters ---------- merge_multi_context : bool Defaults to ``True``. In the case when data-parallelism is used, the outputs will be collected from multiple devices. A ``True`` value indicates that we should merge the collected results so that they look like from a single executor. Returns ------- list of `NDArray` or list of list of `NDArray`. Output Examples -------- >>> # An example of getting forward output. >>> print mod.get_outputs()[0].asnumpy() [[ 0.09999977 0.10000153 0.10000716 0.10000195 0.09999853 0.09999743 0.10000272 0.10000113 0.09999088 0.09999888]] N(Rœ(R6R°((sX/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/module/base_module.pyRXvscCs tƒ‚dS(säGets the gradients to the inputs, computed in the previous backward computation. If `merge_multi_context` is ``True``, it is like ``[grad1, grad2]``. Otherwise, it is like ``[[grad1_dev1, grad1_dev2], [grad2_dev1, grad2_dev2]]``. All the output elements have type `NDArray`. When `merge_multi_context` is ``False``, those `NDArray` instances might live on different devices. Parameters ---------- merge_multi_context : bool Defaults to ``True``. In the case when data-parallelism is used, the gradients will be collected from multiple devices. A ``True`` value indicates that we should merge the collected results so that they look like from a single executor. Returns ------- list of NDArray or list of list of NDArray Input gradients. Examples -------- >>> # An example of getting input gradients. >>> print mod.get_input_grads()[0].asnumpy() [[[ 1.10182791e-05 5.12257748e-06 4.01927764e-06 8.32566820e-06 -1.59775993e-06 7.24269375e-06 7.28067835e-06 -1.65902311e-05 5.46342608e-06 8.44196393e-07] ...]] N(Rœ(R6R°((sX/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/module/base_module.pytget_input_grads”scCs tƒ‚dS(s†Updates parameters according to the installed optimizer and the gradients computed in the previous forward-backward batch. When KVStore is used to update parameters for multi-device or multi-machine training, a copy of the parameters are stored in KVStore. Note that for `row_sparse` parameters, this function does update the copy of parameters in KVStore, but doesn't broadcast the updated parameters to all devices / machines. Please call `prepare` to broadcast `row_sparse` parameters with the next batch of data. Examples -------- >>> # An example of updating module parameters. >>> mod.init_optimizer(kvstore='local', optimizer='sgd', ... optimizer_params=(('learning_rate', 0.01), )) >>> mod.backward() >>> mod.update() >>> print mod.get_params()[0]['fc3_weight'].asnumpy() [[ 5.86930104e-03 5.28078526e-03 -8.88729654e-03 -1.08308345e-03 6.13054074e-03 4.27560415e-03 1.53817423e-03 4.62131854e-03 4.69872449e-03 -2.42400169e-03 9.94111411e-04 1.12386420e-03 ...]] N(Rœ(R6((sX/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/module/base_module.pyR„´scCs tƒ‚dS(s‘Evaluates and accumulates evaluation metric on outputs of the last forward computation. Parameters ---------- eval_metric : EvalMetric Evaluation metric to use. labels : list of NDArray if `pre_sliced` parameter is set to `False`, list of lists of NDArray otherwise. Typically `data_batch.label`. pre_sliced: bool Whether the labels are already sliced per device (default: False). Examples -------- >>> # An example of updating evaluation metric. >>> mod.forward(data_batch) >>> mod.update_metric(metric, data_batch.label) N(Rœ(R6RAtlabelsR>((sX/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/module/base_module.pyRJÍstwritecCs tƒ‚dS(sBinds the symbols to construct executors. This is necessary before one can perform computation with the module. Parameters ---------- data_shapes : list of (str, tuple) or DataDesc objects Typically is ``data_iter.provide_data``. Can also be a list of (data name, data shape). label_shapes : list of (str, tuple) or DataDesc objects Typically is ``data_iter.provide_label``. Can also be a list of (label name, label shape). for_training : bool Default is ``True``. Whether the executors should be bind for training. inputs_need_grad : bool Default is ``False``. Whether the gradients to the input data need to be computed. Typically this is not needed. But this might be needed when implementing composition of modules. force_rebind : bool Default is ``False``. This function does nothing if the executors are already bound. But with this ``True``, the executors will be forced to rebind. shared_module : Module Default is ``None``. This is used in bucketing. When not ``None``, the shared module essentially corresponds to a different bucket -- a module with different symbol but with the same sets of parameters (e.g. unrolled RNNs with different lengths). grad_req : str, list of str, dict of str to str Requirement for gradient accumulation. Can be 'write', 'add', or 'null' (default to 'write'). Can be specified globally (str) or for each argument (list, dict). Examples -------- >>> # An example of binding symbols. >>> mod.bind(data_shapes=[('data', (1, 10, 10))]) >>> # Assume train_iter is already created. >>> mod.bind(data_shapes=train_iter.provide_data, label_shapes=train_iter.provide_label) N(Rœ(R6R R+R0R1Rpt shared_moduletgrad_req((sX/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/module/base_module.pyRyås'cCs tƒ‚dS(s7Installs and initializes optimizers, as well as initialize kvstore for distributed training Parameters ---------- kvstore : str or KVStore Defaults to `'local'`. optimizer : str or Optimizer Defaults to `'sgd'`. optimizer_params : dict Defaults to ``(('learning_rate', 0.01),)``. The default value is not a dictionary, just to avoid pylint warning of dangerous default values. force_init : bool Defaults to ``False``, indicates whether to force re-initializing an optimizer if it is already installed. Examples -------- >>> # An example of initializing optimizer. >>> mod.init_optimizer(optimizer='sgd', optimizer_params=(('learning_rate', 0.005),)) N(Rœ(R6RvRwRxRu((sX/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/module/base_module.pyR~scCs|jS(sGets the symbol associated with this module. Except for `Module`, for other types of modules (e.g. `BucketingModule`), this property might not be a constant throughout its life time. Some modules might not even be associated with any symbols. (R4(R6((sX/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/module/base_module.pyR*sN(s learning_rateg{®Gáz„?((s learning_rateg{®Gáz„?(s learning_rateg{®Gáz„?((s learning_rateg{®Gáz„?(&t__name__t __module__t__doc__tloggingR7R<R(R'RVR\R)RkRR›tpropertyRRR R+RžRˆR}R‰R¨R¯R±R³R|RHR9R:RXR·R„RJRyR~R(((sX/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/module/base_module.pyR-SsZa N( Z    °         &      '(R¾R€R¿RR`tnumpyR^tRRtcontextRtmodelRRqRtioRRRtbaseRRR#R,tobjectR-(((sX/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/module/base_module.pyts