Yc @@sdZddlmZmZddlZddlZddlZddlmZddl Z ddl m Z ddl m Z ddl mZdd l mZdd l mZdd l mZdd lmZmZdd lmZddlmZddlmZmZmZddl mZddl m!Z!e"Z#yddl$m%Z%e%Z#Wne&k rge'Z(nXedddddgZ)dZ*dZ+dZ,dddZ.dZ/ddddddddddd Z0dZ1dZ2dd l3m4Z4d!e#fd"YZ5dS(#sMXNet model modulei(tabsolute_importtprint_functionN(t namedtuplei(tio(tnd(tsymbol(t optimizer(tmetric(tkvstore(tContexttcpu(tUniform(t get_updater(tDataParallelExecutorManagert_check_argumentst _load_data(tDataDesc(t mx_real_t(t BaseEstimatortBatchEndParamstepochtnbatcht eval_metrictlocalscC@st}|dkrd}nt|tjr6|}nt|tr|dkrfd|krfd}qtj|}|dkrtd|jD}|d krt }qqn t d|dkrt }n||fS( sWCreate kvstore This function select and create a proper kvstore if given the kvstore type. Parameters ---------- kvstore : KVStore or str The kvstore. num_device : int The number of devices arg_params : dict of str to `NDArray`. Model parameter, dict of name to `NDArray` of net's weights. itdisttlocalcs@s!|]}tj|jVqdS(N(tnptprodtshape(t.0tparam((s+build/bdist.linux-armv7l/egg/mxnet/model.pys Tsiis$kvstore must be KVStore, str or NoneNii( tTruetNonet isinstancetkvstKVStoretstrtcreatetmaxtvaluestFalset TypeError(Rt num_devicet arg_paramstupdate_on_kvstoretkvtmax_size((s+build/bdist.linux-armv7l/egg/mxnet/model.pyt_create_kvstore9s$           cC@s_xXt|D]J\}}||}|j||||r |j||d| q q WdS(sInitialize kvstoretpriorityN(t enumeratetinittpull(Rt param_arraysR+t param_namesR,tidxt param_on_devstname((s+build/bdist.linux-armv7l/egg/mxnet/model.pyt_initialize_kvstore`s  c C@sx}tt||D]f\}}|\}}|ddkrDqn||}|j||d| |j||d| qWdS(s;Perform update of param_arrays from grad_arrays on kvstore.iR0N(R1tzipR tpushR3( R4t grad_arraysRR5tindextpairtarg_listt grad_listR8((s+build/bdist.linux-armv7l/egg/mxnet/model.pyt_update_params_on_kvstorejs"  cC@sxtt||D]\}}|\}} | ddkrDqn|r||} |j| | d| |j| | d| nxGtt|| D]0\} } | \} }|||| || qWqWdS(s?Perform update of param_arrays from grad_arrays not on kvstore.iR0N(R1R:R R;R3(R4R<tupdaterR*RR5R=R>R?R@R8tktptwtg((s+build/bdist.linux-armv7l/egg/mxnet/model.pyt_update_paramsvs"  " cO@sKt|tr1x|D]}|||qWdS|rG|||ndS(sSends args and kwargs to any configured callbacks. This handles the cases where the 'callbacks' variable is ``None``, a single function, or a list. N(R!tlist(t callbackstargstkwargstcb((s+build/bdist.linux-armv7l/egg/mxnet/model.pyt_multiple_callbackss  c%C@s|dkrt}ntd|d|d|d| d|d|d|d|d | }|rj|j|n|j||| st| }n| rtd | d |jd |d|jd | n| r| j | n| j x-t ||D]}t j }|j d}xt rt }x?| D]7}|j||dk rW|jn|jdt |j| rt|j|j| |jn4t|j|jd|dt|d | d|j|dk r|jn|j||j|d7}|dk r@td|d|d|dt}t||n| dk r+|| kr+t}Pq+q+W|r|jd|| j n| dks|| krPqqWt j } |jd|| ||s|d|kr|j||nt||||||r|j |j d}!xt|D]\}"}#|j|#|jdt|j||#j|dk rtd|d|"d|dt}t||n|!d7}!q8W|dk rtd|d|!d|dt}$t||$n|j qqWdS(sInternal training function on multiple devices. This function will also work for single device as well. Parameters ---------- symbol : Symbol The network configuration. ctx : list of Context The training devices. arg_names: list of str Name of all arguments of the network. param_names: list of str Name of all trainable parameters of the network. aux_names: list of str Name of all auxiliary states of the network. arg_params : dict of str to NDArray Model parameter, dict of name to NDArray of net's weights. aux_params : dict of str to NDArray Model parameter, dict of name to NDArray of net's auxiliary states. begin_epoch : int The begining training epoch. end_epoch : int The end training epoch. epoch_size : int, optional Number of batches in a epoch. In default, it is set to ``ceil(num_train_examples / batch_size)``. optimizer : Optimizer The optimization algorithm train_data : DataIter Training data iterator. eval_data : DataIter Validation data iterator. eval_metric : EvalMetric An evaluation function or a list of evaluation functions. epoch_end_callback : callable(epoch, symbol, arg_params, aux_states) A callback that is invoked at end of each epoch. This can be used to checkpoint model each epoch. batch_end_callback : callable(BatchEndParams) A callback that is invoked at end of each batch. This can be used to measure speed, get result from evaluation metric. etc. kvstore : KVStore The KVStore. update_on_kvstore : bool Whether or not perform weight updating on kvstore. logger : logging logger When not specified, default logger will be used. work_load_list : list of float or int, optional The list of work load for different devices, in the same order as ``ctx``. monitor : Monitor, optional Monitor installed to executor, for monitoring outputs, weights, and gradients for debugging. Notes ----- - This function will inplace update the NDArrays in `arg_params` and `aux_states`. Rtsym_gentctxt train_dataR5t arg_namest aux_namestwork_load_listtloggerRR4R+R,itis_trainRBR*iRRRRs!Epoch[%d] Resetting Data IteratorsEpoch[%d] Time cost=%.3fN( R tloggingR tinstall_monitort set_paramsR R9R4R5t set_optimizertresettrangettimeRtload_data_batchttictforwardtbackwardRAR<RGtlent toc_printt update_metrictlabelt BatchEndParamRRMR(tinfotcopy_toR1(%RRORQR5RRR+t aux_paramst begin_epocht end_epocht epoch_sizeRRR,RPt eval_dataRtepoch_end_callbacktbatch_end_callbackRTRStmonitorteval_end_callbackteval_batch_end_callbackRNtexecutor_managerRBRR^Rtdo_resett data_batchtbatch_end_paramsttocttotal_num_batchtit eval_batchteval_end_params((s+build/bdist.linux-armv7l/egg/mxnet/model.pyt_train_multi_devicesA                                       cC@s|dk r |jd|nd|jD}|jd|jDd||f}tj||tjd|dS(sBCheckpoint the model data into file. Parameters ---------- prefix : str Prefix of model name. epoch : int The epoch number of the model. symbol : Symbol The input Symbol. arg_params : dict of str to NDArray Model parameter, dict of name to NDArray of net's weights. aux_params : dict of str to NDArray Model parameter, dict of name to NDArray of net's auxiliary states. Notes ----- - ``prefix-symbol.json`` will be saved for symbol. - ``prefix-epoch.params`` will be saved for parameters. s%s-symbol.jsoncS@s/i|]%\}}|jtd|qS(sarg:%s(t as_in_contextR (RRCtv((s+build/bdist.linux-armv7l/egg/mxnet/model.pys ks cS@s/i|]%\}}|jtd|qS(saux:%s(R|R (RRCR}((s+build/bdist.linux-armv7l/egg/mxnet/model.pys ls s%s-%04d.paramssSaved checkpoint to "%s"N(R tsavetitemstupdateRRVRf(tprefixRRR+Rht save_dictt param_name((s+build/bdist.linux-armv7l/egg/mxnet/model.pytsave_checkpointTs c C@stjd|}tjd||f}i}i}xd|jD]V\}}|jdd\}} |dkr||| s c@s+i|]!\}}|kr||qS(((RRCR}(RR(s+build/bdist.linux-armv7l/egg/mxnet/model.pys s ( RRR RRRRR+tsettlist_argumentsRRhtlist_auxiliary_states(R((RQRRs+build/bdist.linux-armv7l/egg/mxnet/model.pyRs     % cC@s|jdp|jdS(s!Check if name is a data argument.tdataRd(tendswith(R8((s+build/bdist.linux-armv7l/egg/mxnet/model.pyt _is_data_argscC@spg|D]'}t|tr"|n t|^q}d|D}|jj|\}}}|dk sqtd|D}|jj|\} }} | dk st|jj} |j} g| D]} | | kr| ^q}|jj }gt | || D]}|d|kr|^q}d|D}gt ||| D]}|d|krU|^qU}d|D}x_|j D]Q\}}|j r||j kr| r|j |||(q|j ||qWx_|j D]Q\}}|jr7||jkr7| r7|j|||(q|j ||qW||_ ||_| t||fS(s2Initialize weight parameters and auxiliary states.cS@si|]}|j|jqS((RR8(Rtitem((s+build/bdist.linux-armv7l/egg/mxnet/model.pys s cS@si|]}|j|jqS((tdtypeR8(RR((s+build/bdist.linux-armv7l/egg/mxnet/model.pys s icS@s4i|]*\}}}tjd|d||qS(RR(Rtzeros(RRCtstt((s+build/bdist.linux-armv7l/egg/mxnet/model.pys s cS@s4i|]*\}}}tjd|d||qS(RR(RR(RRCRR((s+build/bdist.linux-armv7l/egg/mxnet/model.pys s N(R!RRt infer_shapeR Rt infer_typeRtkeysRR:RR+RRhRH(Rtinputst overwritetxt input_shapest arg_shapest_t aux_shapest input_dtypest arg_dtypest aux_dtypesRQt input_namestkeyR5RRtparam_name_attrsR+taux_name_attrsRhRCR}((s+build/bdist.linux-armv7l/egg/mxnet/model.pyt _init_paramss<4 %    cC@s|jj}d|d<|S(NR(t__dict__RR (Rtthis((s+build/bdist.linux-armv7l/egg/mxnet/model.pyt __getstate__/s cC@s|jj|dS(N(RR(Rtstate((s+build/bdist.linux-armv7l/egg/mxnet/model.pyt __setstate__4scC@s|jdk r}|jjt|\}}}|dk sHtdg|jjD]}|j^qU}||kr}dSn|jj|j dddd|t|}|j |j |j t |j||_dS(s7Initialize the predictor module for running prediction.sIncomplete input shapesNitgrad_reqtnullt type_dict(RR RRtdictRt arg_arraysRt simple_bindROtcopy_params_fromR+RhR(RRRRRRt pred_shapest pred_exec((s+build/bdist.linux-armv7l/egg/mxnet/model.pyt_init_predictor7s!"  % cC@st|tjtjfr^|d krU|r<tdqUtj|jd}nt|tjtjfst dn|jd|jdkrtdn|j dkr|jddkr|j }n|j dkrtdn|r/t j ||t|jd|jd|d d St j ||t|jd|jdtSnt|t jst d n|S( s$Initialize the iterator given input.s+y must be specified when X is numpy.ndarrayis)y must be ndarray when X is numpy.ndarrays/The numbers of data points and labels not equaliis3Label must be 1D or 2D (with 2nd dimension being 1)tshuffletlast_batch_handlet roll_overs,X must be DataIter, NDArray or numpy.ndarrayN(R!RtndarrayRtNDArrayR t ValueErrorRRR)tndimtflattenRt NDArrayItertminRR(tDataIter(RtXtyRU((s+build/bdist.linux-armv7l/egg/mxnet/model.pyt _init_iterGs( "% /cC@s|dkr|St|ttfrt|dkr|ddk r|ddkrut|dtjru|dSt|dtrtj|dn|d}t|dtrtj|dn|d}|j ||dt St dnt|tjst dn|S(s(Initialize the iterator given eval_data.iiiRUsEval data is NONEs\Eval data must be DataIter, or NDArray/numpy.ndarray/list pair (i.e. tuple/list of length 2)N( R R!ttupleRHRaRRRtarrayRRRR)(RRlt input_datat input_label((s+build/bdist.linux-armv7l/egg/mxnet/model.pyt_init_eval_iter`s '&& & cC@so|j|ddt}|r+|jn|j}g|D]}|d^q;}td|jjD}xA|jD]6}t|t r|j ||j t|jD]*\}}||j|d|!jq6Wn|d7}|dk ri||kriPqiqiWg| D]}tj|^q}t|dkr|d}n|rgg|D]}tj|^q}g|D]}tj|^q}t|dkr;|d}nt|dkrZ|d}n|||fS|SdS(sRun the prediction, always only use one device. Parameters ---------- X : mxnet.DataIter num_batch : int or None The number of batch to run. Go though all batches if ``None``. Returns ------- y : numpy.ndarray or a list of numpy.ndarray if the network has multiple outputs. The predicted value of the output. RUics@s$|]\}}||jfVqdS(N(R(RRtvalue((s+build/bdist.linux-armv7l/egg/mxnet/model.pys siN( RR R(RZt provide_dataRR+RR!RRR8RRt batch_sizeRtarg_dictR[Ratoutputst provide_labelRR_tpadR:tappendtasnumpyR1RRdRt concatenate(RRt num_batcht return_dataRZt data_shapesRt data_namesRRR8t data_arraysRt output_listt data_listt label_listRxtbatchtpaddedt real_sizeto_listto_ndtjRRRd((s+build/bdist.linux-armv7l/egg/mxnet/model.pytpredicttsZ    #+    ""% " ""   taccc C@st|tjs$tj|}n|j|d dt}|rO|jn|j}g|D]}|d^q_}t d|j j D} xA|jD]6}t|t r|j | |jsRRRRiN(R!Rt EvalMetricR%RR R(RZRRR+RRRR8RRRRR1RR_RRdRReRRMtget(RRRRRnRZRRRRR8RRxRRu((s+build/bdist.linux-armv7l/egg/mxnet/model.pytscores6  #    Rc *C@s|j||dt} |j|}|jrR|j| j|_|jn|j|jd<|j| j | j \}}}t |t j st j|}nt|t|j|j\}}i}|r|jt|nXxUt|D]G\}}x8tt|jD]!}|||t|j|sV         '     $