[c @`s9dZddlmZmZmZddlZddlZddlZddlZddl Z ddl Z ddl Z ddl Z ddl Z ddlZddlZddlZddlmZddlZddljZddljZyddljZWnek rdZnXyddl Z Wnek r<nXddl!Z"ddl#m$Z$m%Z%ddl&m'Z'ddl(m)Z)dd l*m+Z+d Z,d Z-d Z.dd Z/ddZ0dZ1dZ2ddZ3dZ4ddde5dZ6dddZ7dZ8dZ9dddddde5dZ:ddde5ddZ;ddddde5dZ<dddde5dZ=dddZ>dddd Z?dd!Z@d"ZAddd#ZBd$ZCdde5d%ZDdddZe5d(ZEddd)ZFddd[d*ZGd+ZHd,ZIde5d-ZJe.d.ZKe.d/ZLdd0eMe.d1ZNdd2d3ddeMdde.d4 ZOd0ddde5e.d5ZPd6ddd7dde5e.d8ZQddd9dd:d;ZRd<d7dddeMde5e5ejSd= ZTd>ZUdde5d?d@ZVdAZWdBZXdCZYdDZZdddEZ[dFZ\dGZ]dHdIZ^dJZ_edKZ`dLe"jajbfdMYZcdNZddOdPZedQdRZfdOdSZgdOdTZhdOd?dUdVZidddWZjdXdXd0d6eMdYZkdS(\sTools for testing.i(tabsolute_importtprint_functiontdivisionN(tcontextmanageri(tContexttcurrent_context(t_STORAGE_TYPE_STR_TO_ID(tarray(tSymbolcC`stS(s(Get default context for regression test.(R(((sP/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/test_utils.pytdefault_context5scC`s|tj_dS(sSet default context.N(Rt _default_ctxtvalue(tctx((sP/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/test_utils.pytset_default_context<scC`stjS(s*Get default data type for regression test.(tnptfloat32(((sP/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/test_utils.pyt default_dtypeAscC`s|dkrdS|S(s4Get default numerical threshold for regression test.g#B ;N(tNone(tatol((sP/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/test_utils.pytget_atolGscC`s|dkrdS|S(s4Get default numerical threshold for regression test.gh㈵>N(R(trtol((sP/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/test_utils.pytget_rtolNscG`sOg|D]$}tjj|jt^q}t|dkrK|dS|S(s"Generate some random numpy arrays.ii(RtrandomtrandntastypeRtlen(tshapeststarrays((sP/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/test_utils.pyt random_arraysUs .cC`sJd|kot|kns(t|}tjj||d|!S(sKReturn a k length list of the elements chosen from the population sequence.i(RtAssertionErrorRRtshuffle(t populationtktpopulation_copy((sP/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/test_utils.pyt random_sample^s(tuniformcC`st|||}|dks,|dkr;tdn|dksS|dkrbtdn|dkr|d|krtd|||fqndS( s9Validates inputs for csr generation helper functions iis!density has to be between 0 and 1s-num_rows or num_cols should be greater than 0tpowerlawisvnot supported for this density: %s for this shape (%s, %s) Please keep : num_rows * num_cols * density >= 2 * num_rowsN(tintt ValueError(tnum_rowstnum_colstdensityt distributiont total_nnz((sP/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/test_utils.pyt_validate_csr_generation_inputsfs cC`st|jd}xkt|D]]}|j|}|j|d}tj|j||!}tjj|||j||+q WdS(sShuffle CSR column indices per row This allows validation of unordered column indices, which is not a requirement for a valid CSR matrix iN(RtindptrtrangeRRtindicesRR(tcsrt row_counttit start_indext end_indextsublist((sP/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/test_utils.pytshuffle_csr_column_indiceszs g?c C`sBt|||ddyddlm}|j|||d|dd}|d k ri|jj|n|tkrt|nt j jj |j|j |j fd||fd|}Wntk r=|d kstd | std t j jjd||fd|} | | |k} | jd}nX|S( sIReturns CSRNDArray with uniform distribution This generates a csr matrix with totalnnz unique randomly chosen numbers from num_rows*num_cols and arranges them in the 2d array in the following way: row_index = (random_number_generated / num_rows) col_index = random_number_generated - row_index * num_cols R+R$i(tsparsetdtypetformatR1tshapes6data_init option is not supported when scipy is absents@shuffle_csr_indices option is not supported when scipy is absentN(R-tscipyR8trandRtdatatfilltTrueR7tmxtndt csr_matrixR0R.t ImportErrorRRR$ttostype( R(R)R*R9t data_inittshuffle_csr_indicestspspR1tresulttdnst masked_dns((sP/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/test_utils.pyt_get_uniform_dataset_csrs( !   $    $c C`st|||ddt|||}|}tj||fd|}xat|D]S}dtjdd||d<|d}|dkrXtjj |j dSqXWd}xt|D]}t ||} | |krB|| krBdtjdd||<|| d}|dkrtjj |j dSqnxdtd| D]S} dtjdd||| <|d}|dkrRtjj |j dSqRW|d}qW|dkrt d |||fntjj |j dSd S( s<Returns CSRNDArray with powerlaw distribution with exponentially increasing number of non zeros in each row. Not supported for cases where total_nnz < 2*num_rows. This is because the algorithm first tries to ensure that there are rows with no zeros by putting non zeros at beginning of each row. R+R%R9igMbP?iiR1s9not supported for this density: %s for this shape (%s,%s)N( R-R&RtzerosR/trndR$RARBRREtminR'( R(R)R*R9R,t unused_nnzt output_arrtrowtcol_maxt col_limitt col_index((sP/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/test_utils.pyt_get_powerlaw_dataset_csrs:      cC`s|dkrtj|}ntj|ddg}tj|j}tj|ddgddg}x;|js|d}|||d<|j|jqlW|S(sIReturn ndarray composed of passing each array value through some functiontflagstf_indextop_flagst writeonlyiN(RRRtnditerRMR;tfinishedtiternext(t the_inputtfunctiontoutputtit_inputtit_outt val_input((sP/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/test_utils.pyt assign_eachs !   c C`s|dkrtj|}n|j|jks6ttj|ddg}tj|ddg}tj|j}tj|ddgddg}xR|js|d}|d}||||d<|j|j|jqW|S(sIReturn ndarray composed of passing two array values through some functionRWRXRYRZiN( RRRR;RR[RMR\R]( tinput1tinput2R_R`t it_input1t it_input2Rbt val_input1t val_input2((sP/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/test_utils.pyt assign_each2s !     c  C`s|dkrtjn|}|dkr3tn|}|dkrKdn|}|dkr|dksytd||dk r|} t| |dkstn.tj|d} tj| |kj} | j ddkr1t j j |ddd|} | tj gd|tj gffStj| j d|dj|} |dk rs| j|n|dk rt| |} nt j jj| | fd|d|} | | | ffS|d krt|d kst|dkr@t|d|d|d |d |d|}||j|j|jffS|d krt|d|dd|d|}||j|j|jffStstd|tSntstdtSdS(sGenerate a random sparse ndarray. Returns the ndarray, value(np) and indices(np) Parameters ---------- shape: list or tuple stype: str, valid values: "csr" or "row_sparse" density, optional: float, should be between 0 and 1 distribution, optional: str, valid values: "uniform" or "powerlaw" dtype, optional: numpy.dtype, default value is None Returns ------- Result of type CSRNDArray or RowSparseNDArray Examples -------- Below is an example of the powerlaw distribution with csr as the stype. It calculates the nnz using the shape and density. It fills up the ndarray with exponentially increasing number of elements. If there are enough unused_nnzs, n+1th row will have twice more nnzs compared to nth row. else, remaining unused_nnzs will be used in n+1th row If number of cols is too small and we have already reached column size it will fill up all following columns in all followings rows until we reach the required density. >>> csr_arr, _ = rand_sparse_ndarray(shape=(5, 16), stype="csr", density=0.50, distribution="powerlaw") >>> indptr = csr_arr.indptr.asnumpy() >>> indices = csr_arr.indices.asnumpy() >>> data = csr_arr.data.asnumpy() >>> row2nnz = len(data[indptr[1]:indptr[2]]) >>> row3nnz = len(data[indptr[2]:indptr[3]]) >>> assert(row3nnz == 2*row2nnz) >>> row4nnz = len(data[indptr[3]:indptr[4]]) >>> assert(row4nnz == 2*row3nnz) R$t row_sparses,Distribution %s not supported for row_sparseitstypeR9iR;R1iRFRGR%R*sDistribution not supported: %ssunknown storage typeN(RRNR=RRRRtargwheretflattenR;RARBRMRRR?RdR8trow_sparse_arrayRLR.R0R>RVtFalse(R;RmR*R9R+RFt rsp_indicest modifier_funcRGR0t idx_sampleRItvaltarrR1((sP/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/test_utils.pytrand_sparse_ndarraysF'   (&  '   #c C`sd|dkr-tjjt|d|}n3t||d|d|d|d|d|\}}|S(NtdefaultR9R*RsRGR+(RARBRRRw( R;RmR*R9RsRGR+Rvt_((sP/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/test_utils.pyt rand_ndarraySs ! g?c C`s|dkrv|d k r4tj|}|jnd }t||d|d|d|d|d|\} \} } na|dkrt||d|d|d|d|d|\} \} } } nd |} t| | S( sHCreate a sparse array, For Rsp, assure indices are in a canonical formatRlR*RFRrR9RsR1RGsUnknown storage type: N(RRtasarraytsortRwR( R;RmRFRrR9RsR*RGt arr_indicestarr_dataRytmsg((sP/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/test_utils.pytcreate_sparse_array_s,       cC`sq|dkr@d}|d k r@t||dks=tq@nt||d|d|d|d|d|d |S( s@Create sparse array, using only rsp_indices to determine densityRlgiRFRrR9RsR*RGN(RRRR(R;RmR*RFRrR9RsRG((sP/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/test_utils.pytcreate_sparse_array_zd~s  " i cC`s*tjd|dtjd|dfS(Ni(RNtrandint(tdim0tdim1((sP/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/test_utils.pyt rand_shape_2dscC`s=tjd|dtjd|dtjd|dfS(Ni(RNR(RRtdim2((sP/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/test_utils.pyt rand_shape_3dscC`s ttjd|dd|S(Nitsize(ttupleRNR(tnum_dimtdim((sP/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/test_utils.pyt rand_shape_ndscC`st|tr|g}n-|dk r3t|ntt|j}|}x,tt|D]}||d|}qaW|rt|j}x|D]}d||Ms Rtis_trainii(R titemstbindtforwardtoutputstasnumpyR(tsymR RtinputstexetxR((sP/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/test_utils.pytsimple_forward:s" c`st|tttfsttjtjtjfksBtt|trt |j t |j krt dt t |j t t |j fqndt|j |D}fd|jD}|S(sParses the given location to a dictionary. Arguments of the provided op `sym` are used as dictionary keys and elements of `location` are used as values. Parameters ---------- sym : Symbol Symbol containing op location : list or tuple or dict Argument values location - if type is list or tuple of `np.ndarray` inner elements are arrays correspoding to ``sym.list_arguments()``. - if type is dict of str -> `np.ndarray` maps the name of arguments to the corresponding `np.ndarray`. *In either case, value of all the arguments must be provided.* ctx : Context Device context. dtype: np.float16 or np.float32 or np.float64 Datatype for mx.nd.array. Returns ------- dict Dictionary with `sym` arguments as keys and `location` elements as values. Examples ------- >>> a = mx.symbol.Variable('a') >>> b = mx.symbol.Variable('b') >>> l1 = np.ndarray([2,3]) >>> l2 = np.ndarray([3,4]) >>> _parse_location(a * b, [l1, l2], None) {'a': , 'b': } >>> _parse_location(a * b, {'a': l1, 'b': l2}, None) {'a': , 'b': } >>> _parse_location(a * b, {'a': l1}, None) ValueError: Symbol arguments and keys of the given location do not match. s_Symbol arguments and keys of the given location do not match.symbol args:%s, location.keys():%scS`si|]\}}||qS(((RR!R((sP/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/test_utils.pys s c`sOi|]E\}}t|tjrBtjj|ddn||qS(R R9(RRtndarrayRARBR(RR!R(R R9(sP/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/test_utils.pys s (RtdictRRRRtfloat16Rtfloat64tsettkeystlist_argumentsR'RtzipR(RtlocationR R9((R R9sP/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/test_utils.pyt_parse_locationVs+$$:c`stjtjtjfks$t|dk rt|trt|j t|j krt dt t|j t t|j fqn=t|t tfr|j }dt||D}nfd|jD}n|S(sParses the given auxiliary states to a dictionary. Auxiliary states of the provided op `sym` are used as dictionary keys and elements of `aux_states` are used as values. Parameters ---------- sym : Symbol Symbol containing op aux_states : None or list or dict Aux states - if type is list or tuple of `np.ndarray` inner elements are arrays correspoding to ``sym.list_auxiliary_states()``. - if type is dict of str -> `np.ndarray` maps the name of arguments to the corresponding `np.ndarray`. *In either case, all aux states of `sym` must be provided.* ctx : Context Device context. dtype: np.float16 or np.float32 or np.float64 Datatype for mx.nd.array. Returns ------- dict Dictionary with `sym` aux states as keys and `aux_states` elements as values. Examples ------- >>> data = mx.symbol.Variable('data') >>> weight = mx.sym.Variable(name='fc1_weight') >>> fc1 = mx.symbol.FullyConnected(data = data, weight=weight, name='fc1', num_hidden=128) >>> fc2 = mx.symbol.BatchNorm(fc1, name='batchnorm0') >>> mean_states = np.ones(3) >>> var_states = np.ones(3) >>> _parse_aux_states(fc2, [mean_states, var_states], None) {'batchnorm0_moving_var': , 'batchnorm0_moving_mean': } >>> _parse_aux_states(fc2, {'batchnorm0_moving_var': mean_states, ... 'batchnorm0_moving_mean': var_states}, None) {'batchnorm0_moving_var': , 'batchnorm0_moving_mean': } >>> _parse_aux_states(fc2, {'batchnorm0_moving_var': mean_states}, None) ValueError: Symbol aux_states names and given aux_states do not match. saSymbol aux_states names and given aux_states do not match.symbol aux_names:%s, aux_states.keys:%scS`si|]\}}||qS(((RR!R((sP/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/test_utils.pys s c`s7i|]-\}}tjj|dd|qS(R R9(RARBR(RR!R(R R9(sP/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/test_utils.pys s N(RRRRRRRRRRtlist_auxiliary_statesR'RRRRR(Rt aux_statesR R9t aux_names((R R9sP/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/test_utils.pyt_parse_aux_statess.$ $% "g-C6?c`sd}tjtjtjfks-tfd|jD}xV|jD]H\}} |j|j} | dkrV|| | d|j|(qVqVWx+|D]#}tj||dd||s RxR9tordertCRg@RiN(RRRRRRtarg_dictRmR{R9tkindRR/tprodR;travelRtaux_dictRRRtsum(texecutorRRtepstuse_forward_trainR9Rt approx_gradsR!RRmt old_valueR3tkeyRutf_pepstadstypetf_nepst approx_grad((R9sP/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/test_utils.pyt numeric_gradsH $  $ !   $!gMbP?g{Gz?c  `s!tjtjtjfks$ttjtjfkrQ|dksQtnd#kritnd} td|d|dd}d|jD} t d|d|dd}|d#k rd |jD} nd#} |d#kr |j }d |D}ndt |t t frTt |}d |D}n0t |tr~|j}|j}ntd |jD}|j|\}}}tjjd }||}tjj|}tt |jd tjj| |dddfg}tg|D].}|tjjddd||jf^q1d tjjddd|dfg}fd|jD}| d#k rOt | tstdxx| jD]g\}}||kr|tkr|dkrtjj||j||j||j||| numpy.ndarray maps the name of arguments to the corresponding numpy.ndarray. *In either case, value of all the arguments must be provided.* aux_states : list or tuple or dict, optional The auxiliary states required when generating the executor for the symbol. numeric_eps : float, optional Delta for the finite difference method that approximates the gradient. check_eps : float, optional relative error eps used when comparing numeric grad to symbolic grad. grad_nodes : None or list or tuple or dict, optional Names of the nodes to check gradient on use_forward_train : bool Whether to use is_train=True when computing the finite-difference. ctx : Context, optional Check the gradient computation on the specified device. grad_stype_dict : dict of str->str, optional Storage type dictionary for gradient ndarrays. dtype: np.float16 or np.float32 or np.float64 Datatype for mx.nd.array. References --------- ..[1] https://github.com/Theano/Theano/blob/master/theano/gradient.py gh㈵>cS`stjj|d}|S(sGet a random weight matrix with not too small elements Parameters ---------- shape : list or tuple g?(RRR=(R;tplain((sP/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/test_utils.pytrandom_projectionEs RRR R9cS`s%i|]\}}|j|qS((R(RR!R((sP/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/test_utils.pys Rs RcS`s%i|]\}}|j|qS((R(RR!R((sP/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/test_utils.pys Vs cS`si|]}d|qS(twrite((RR!((sP/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/test_utils.pys [s cS`si|]}d|qS(R((RR!((sP/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/test_utils.pys ^s cS`s"i|]\}}|j|qS((R;(RR!R((sP/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/test_utils.pys es t __random_projig{Gz?Rc`s7i|]-\}}tjj|dd|qS(R R9(RARBR(RR!R(R R9(sP/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/test_utils.pys rs sgrad_stype_dict must be a dictRxtgrad_reqRt args_gradsTExecutor arg_arrays and and location len do not match.Got %d inputs and %d locationsiRc`s&i|]}j|j|qS((t grad_dictR(RR!(R(sP/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/test_utils.pys s RRRs NUMERICAL_%ss BACKWARD_%staddtnulls#Invalid grad_req %s for argument %sN()RRRRRRR RRRRRRRRRRR't infer_shapeRARtVariablet make_lossRBRRtnormalR;RRMtcontextR9Rt arg_arraysRRRR@tbackwardRR(RRRt numeric_epsRRt grad_nodesRR tgrad_stype_dictR9Rt location_npytaux_states_npyRt input_shapeRyt out_shapetprojtoutR!t args_grad_npyRRtinpstsymbolic_gradstnumeric_gradientstnametfd_gradt orig_gradtsym_grad((R R9RsP/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/test_utils.pytcheck_numeric_gradients($   !       5, $!       c  `stjtjtjfks$tdkr<tntd|d|dd}td|d|dd}t |t rg|j D]} || ^q}nfd|j D} |j dd|d| d|} x| jD] } d | (qW| jd tg| jD]} | j^q+}xSt|j ||D]9\}}}t||||d |d |fd |q\W| jS(sSCompares a symbol's forward results with the expected ones. Prints error messages if the forward results are not the same as the expected ones. Parameters --------- sym : Symbol output symbol location : list of np.ndarray or dict of str to np.ndarray The evaluation point - if type is list of np.ndarray Contains all the numpy arrays corresponding to `sym.list_arguments()`. - if type is dict of str to np.ndarray Contains the mapping between argument names and their values. expected : list of np.ndarray or dict of str to np.ndarray The expected output value - if type is list of np.ndarray Contains arrays corresponding to exe.outputs. - if type is dict of str to np.ndarray Contains mapping between sym.list_output() and exe.outputs. check_eps : float, optional Relative error to check to. aux_states : list of np.ndarray of dict, optional - if type is list of np.ndarray Contains all the NumPy arrays corresponding to sym.list_auxiliary_states - if type is dict of str to np.ndarray Contains the mapping between names of auxiliary states and their values. ctx : Context, optional running context dtype: np.float16 or np.float32 or np.float64 Datatype for mx.nd.array. equal_nan: Boolean if True, `nan` is a valid value for checking equivalency (ie `nan` == `nan`) Example ------- >>> shape = (2, 2) >>> lhs = mx.symbol.Variable('lhs') >>> rhs = mx.symbol.Variable('rhs') >>> sym_dot = mx.symbol.dot(lhs, rhs) >>> mat1 = np.array([[1, 2], [3, 4]]) >>> mat2 = np.array([[5, 6], [7, 8]]) >>> ret_expected = np.array([[19, 22], [43, 50]]) >>> check_symbolic_forward(sym_dot, [mat1, mat2], [ret_expected]) RRR R9Rc`s:i|]0\}}tjj|jdd|qS(R R9(RARBtemptyR;(RR!R(R R9(sP/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/test_utils.pys s RRiRs EXPECTED_%ss FORWARD_%sRN(RRRRRRR RRRRt list_outputsRRt grad_arraysRRqRRRR(RRtexpectedRRRR RR9R!targs_grad_dataRtgRRt output_nametexpectR`((R R9sP/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/test_utils.pytcheck_symbolic_forwards&2$  ! &$ "( gh㈵>Rc  `sV| tjtjtjfks$t|dkr<t}ntd|d|d|d| }td|d|d|d| }t |t t frdt |j |D}nd|jD} i} x| jD]\}}tjj|d|d| }| dk rp|| krp| |}|dk r]|dkr]t|j|d d }n|}|| |str dictionary of mapping argument name to stype for the gradient equal_nan: Boolean if True, `nan` is a valid value for checking equivalency (ie `nan` == `nan`) dtype: np.float16 or np.float32 or np.float64 Datatype for mx.nd.array. Example ------- >>> lhs = mx.symbol.Variable('lhs') >>> rhs = mx.symbol.Variable('rhs') >>> sym_add = mx.symbol.elemwise_add(lhs, rhs) >>> mat1 = np.array([[1, 2], [3, 4]]) >>> mat2 = np.array([[5, 6], [7, 8]]) >>> grad1 = mx.nd.zeros(shape) >>> grad2 = mx.nd.zeros(shape) >>> exec_add = sym_add.bind(default_context(), args={'lhs': mat1, 'rhs': mat2}, ... args_grad={'lhs': grad1, 'rhs': grad2}, grad_req={'lhs': 'write', 'rhs': 'write'}) >>> exec_add.forward(is_train=True) >>> ograd = mx.nd.ones(shape) >>> grad_expected = ograd.copy().asnumpy() >>> check_symbolic_backward(sym_add, [mat1, mat2], [ograd], [grad_expected, grad_expected]) RRR R9RcS`si|]\}}||qS(((RR!R((sP/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/test_utils.pys -s cS`s1i|]'\}}tjjd|j|qS(R(RRRR;(RR!R((sP/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/test_utils.pys /s RxR*gc`si|]}|qS(((RR!(R(sP/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/test_utils.pys >s cS`si|]\}}||qS(((RR!R((sP/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/test_utils.pys @s RRRRcS`s%i|]\}}|j|qS((R(RR!R((sP/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/test_utils.pys [s Rs EXPECTED_%ss BACKWARD_%sRRRs#Invalid grad_req %s for argument %sN(RRRRRRR RRRRRRRRRARBRRR;RRRR@RtappendRRRR'(RRt out_gradsRRRRRR t grad_stypesRR9RRR!RRBRmRRtoutgRvtgradsR((RsP/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/test_utils.pytcheck_symbolic_backwardst=$  ! "  "  (  %      itwholecK`sl|d krt}n|d kr-d}n|d krp|jd|d||}d|jjD}nPt|tstdt||jd|d|d|jD}x;|jD]-\}} | j |j|j |j|(qW|dkr|j dt |j d |jx|jD]} | jq7Wtj} x7t|D])} |j dt |j d |jqdWtjjtj} | | d |}|S|d kr\|j dtx|jD]} | jqWtj} x$t|D]} |j dtqWtjjtj} | | d |}|Std d S(sSCheck the running speed of a symbol. Parameters ---------- sym : Symbol Symbol to run the speed test. location : none or dict of str to np.ndarray Location to evaluate the inner executor. ctx : Context Running context. N : int, optional Repeat times. grad_req : None or str or list of str or dict of str to str, optional Gradient requirements. typ : str, optional "whole" or "forward" - "whole" Test the forward_backward speed. - "forward" Only test the forward speed. RRR cS`s7i|]-\}}tjjd|jdd|qS(Rtscaleg?(RRRR;(RR!Rv((sP/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/test_utils.pys s sExpect dict, get "location"=%scS`s"i|]\}}|j|qS((R;(RR!R((sP/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/test_utils.pys s R(RR#g?Rs%typ can only be "whole" or "forward".N(RR t simple_bindRRRRRRRR9RR@RRt wait_to_readttimeR/RARBtwaitallRqR'(RRR tNRttypRRRtiarrR`tticRyttoctforward_backward_timet forward_time((sP/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/test_utils.pyt check_speednsL     %%        g?c  C`s|dkridtjtj6dtjtj6dtjtj6dtjtj6dtjtj6dtjtj6}nt |t j ri|tjtj6|tjtj6|tjtj6|tjtj6|tjtj6|tjtj6}nt |dks,t t |trQ|gt |}nt |t |ksot |dj} |dj} g}xit||D]X\}}|j| kst |j| kst |j|jd||qW|dkrin|}|dkr+in|}x|djjD]\}}||krE| rtjjddd d d |jj| ||>> # create the symbol >>> sym = mx.sym.Convolution(num_filter=3, kernel=(3,3), name='conv') >>> # initialize the running context >>> ctx_list =[{'ctx': mx.gpu(0), 'conv_data': (2, 2, 10, 10), 'type_dict': {'conv_data': np.float64}}, {'ctx': mx.gpu(0), 'conv_data': (2, 2, 10, 10), 'type_dict': {'conv_data': np.float32}}, {'ctx': mx.gpu(0), 'conv_data': (2, 2, 10, 10), 'type_dict': {'conv_data': np.float16}}, {'ctx': mx.cpu(0), 'conv_data': (2, 2, 10, 10), 'type_dict': {'conv_data': np.float64}}, {'ctx': mx.cpu(0), 'conv_data': (2, 2, 10, 10), 'type_dict': {'conv_data': np.float32}}] >>> check_consistency(sym, ctx_list) >>> sym = mx.sym.Concat(name='concat', num_args=2) >>> ctx_list = [{'ctx': mx.gpu(0), 'concat_arg1': (2, 10), 'concat_arg0': (2, 10), 'type_dict': {'concat_arg0': np.float64, 'concat_arg1': np.float64}}, {'ctx': mx.gpu(0), 'concat_arg1': (2, 10), 'concat_arg0': (2, 10), 'type_dict': {'concat_arg0': np.float32, 'concat_arg1': np.float32}}, {'ctx': mx.gpu(0), 'concat_arg1': (2, 10), 'concat_arg0': (2, 10), 'type_dict': {'concat_arg0': np.float16, 'concat_arg1': np.float16}}, {'ctx': mx.cpu(0), 'concat_arg1': (2, 10), 'concat_arg0': (2, 10), 'type_dict': {'concat_arg0': np.float64, 'concat_arg1': np.float64}}, {'ctx': mx.cpu(0), 'concat_arg1': (2, 10), 'concat_arg0': (2, 10), 'type_dict': {'concat_arg0': np.float32, 'concat_arg1': np.float32}}] >>> check_consistency(sym, ctx_list) g?gMbP?gh㈵>iiRtlowgq= ףpthighgq= ףp?RR)RR9RRRRRs#Predict Err: ctx %d vs ctx %d at %ss!Train Err: ctx %d vs ctx %d at %sN(/RRR9RRRtuint8tint32tint64RtnumberstNumberRRRRRRR"R*RRRR$R;RRRRRMRRt output_dictRtupdateRRRqt enumerateRRtprintt tracebackt print_excRR@R(Rtctx_listR)Rt arg_paramst aux_paramsRt raise_on_errt ground_truthRt use_uniformt rand_typet output_namest arg_namestexe_listRR RRvRRtdtypestmax_idxtgtR3tgtarrRtcurr((sP/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/test_utils.pytcheck_consistencys3        ',     (       (   cC`sd}dddg}xA|D]9}ytj|dgdt}Wqtk rTqXqWttg|jdD]}d|kro|^qoS( sReturn a list of GPUs Returns ------- list of int: If there are n GPUs, then return a list [0,1,...,n-1]. Otherwise returns []. ts nvidia-smis/usr/bin/nvidia-smis /usr/local/nvidia/bin/nvidia-smis-Ltuniversal_newliness tGPU(t subprocesst check_outputR@tOSErrorR/Rtsplit(tret nvidia_smitcmdR3((sP/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/test_utils.pyt list_gpusXs   ic C`s,|dkstd|dkr:|jdd}n|dkr[tjj|}ntjj||}|dkrtjj|sy!tj d|tj |Wqt k r}|j t j krt d|qqXqn| rtjj|rtj d||Sx|d dkrytj|d t}|jd ksgtd |t|d ;}x0|jddD]}|r|j|qqWPWdQXWq tk r} |d 8}|dkr| qtdj||d krdndq Xq Wtj d|||S(sDownload an given URL Parameters ---------- url : str URL to download fname : str, optional filename of the downloaded file. If None, then will guess a filename from url. dirname : str, optional output directory name. If None, then guess from fname or use the current directory overwrite : bool, optional Default is false, which means skipping download if the local file exists. If true, then download the url to overwrite the local file if exists. retries : integer, default 5 The number of times to attempt the download in case of failure or non 200 return codes Returns ------- str The filename of the downloaded file is&Number of retries should be at least 0t/iRSscreate directory %ssfailed to create s%s exists, skipping downloaditstreamisfailed to open %stwbt chunk_sizeiNs,download failed, retrying, {} attempt{} leftRs"downloaded %s into %s successfully(RRRYtostpathtdirnametjointexiststloggingtinfotmakedirsRXterrnotEEXISTtrequeststgetR@t status_codetopent iter_contentRt ExceptionR@R:( turltfnameRdt overwritetretriestexctrRtchunkR((sP/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/test_utils.pytdownloadjsD        'cC`sid}d}||d|d\}}||d|d\}}i|d6|d6|d 6|d 6S( smDownload and load the MNIST dataset Returns ------- dict A dict containing the data c S`stjtjj|>}tjd|jdtj |jdtj }WdQXtjtjj|d}tjd|jd\}}}}tj |jdtj j t |||}|j |jddd d jtjd }WdQX||fS( Ns>IIiR9trbs>IIIIiiiii(tgzipRoRAt test_utilsRytstructtunpacktreadRt frombuffertint8R8RRR;RR( t label_urlt image_urltflbltlabeltfimgRytrowstcolstimage((sP/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/test_utils.pyt read_datas$!'35s http://data.mxnet.io/data/mnist/strain-labels-idx1-ubyte.gzstrain-images-idx3-ubyte.gzst10k-labels-idx1-ubyte.gzst10k-images-idx3-ubyte.gzt train_datat train_labelt test_datat test_label((RRct train_lblt train_imgttest_lblttest_img((sP/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/test_utils.pyt get_mnists cC`sKtjjds"tjdntjjdsGtdddndS(skDownloads MNIST dataset as a pkl.gz into a directory in the current directory with the name `data` R>sdata/mnist.pkl.gzs/http://deeplearning.net/data/mnist/mnist.pkl.gzRdN(RbRctisdirRiRfRy(((sP/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/test_utils.pyt get_mnist_pkls  cC`stjjds"tjdntjjd sntjjd sntjjd sntjjd rtddd}tj|}|jdWdQXndS( sDownloads ubyte version of the MNIST dataset into a directory in the current directory with the name `data` and extracts all files in the zip archive to this directory. R>sdata/train-images-idx3-ubytesdata/train-labels-idx1-ubytesdata/t10k-images-idx3-ubytesdata/t10k-labels-idx1-ubytes)http://data.mxnet.io/mxnet/data/mnist.zipRdN( RbRcRRiRfRytzipfiletZipFilet extractall(t zip_file_pathtzf((sP/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/test_utils.pytget_mnist_ubytes  cC`stjjds"tjdntjjd sntjjd sntjjd sntjjd rtddd}tj|}|jdWdQXndS( sDownloads CIFAR10 dataset into a directory in the current directory with the name `data`, and then extracts all files into the directory `data/cifar`. R>sdata/cifar/train.recsdata/cifar/test.recsdata/cifar/train.lstsdata/cifar/test.lsts+http://data.mxnet.io/mxnet/data/cifar10.zipRdN( RbRcRRiRfRyRRR(RR((sP/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/test_utils.pyt get_cifar10s  cC`stt|dkrtnt}tjjddddd|d|dtd |d |d |}tjjdd dd d|d|d |d |d |}||fS(s@Returns training and validation iterators for MNIST dataset iRsdata/train-images-idx3-ubyteRsdata/train-labels-idx1-ubyteR t batch_sizeRtflatt num_partst part_indexsdata/t10k-images-idx3-ubytesdata/t10k-labels-idx1-ubyte(RRRqR@RAtiot MNISTIter(RR RRRttrain_dataitert val_dataiter((sP/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/test_utils.pytget_mnist_iterators(    cC`sctjj||}tjj|s_t|d|dttj|}|jd|ndS(sDownload and extract zip data. Parameters ---------- data_dir : str Absolute or relative path of the directory name to store zip files url : str URL to download data from data_origin_name : str Name of the downloaded zip file Examples -------- >>> get_zip_data("data_dir", "http://files.grouplens.org/datasets/movielens/ml-10m.zip", "ml-10m.zip") RdRtRcN( RbRcReRfRyRqRRR(tdata_dirRrtdata_origin_nametzip_file((sP/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/test_utils.pyt get_zip_datas cC`stjj||}tjj||}tjj|st|d|d|dttj|d}t|d/}x|D]}|j |qW|j WdQXtj |ndS(s:Download and extract bz2 data. Parameters ---------- data_dir : str Absolute or relative path of the directory name to store bz2 files data_name : str Name of the output file in which bz2 contents will be extracted url : str URL to download data from data_origin_name : str Name of the downloaded b2 file Examples -------- >>> get_bz2_data("data_dir", "kdda.t", "https://www.csie.ntu.edu.tw/~cjlin/libsvmtools/datasets/binary/kdda.t.bz2", "kdda.t.bz2") RsRdRtRzR`N( RbRcReRfRyRqtbz2tBZ2FileRoRtclosetremove(Rt data_nameRrRtbz_filetfouttline((sP/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/test_utils.pyt get_bz2_data)s RScC`s&tjj||}|tj|<|S(sVSet environment variable Parameters ---------- key : str Env var to set val : str New value assigned to the env var default_val : str, optional Default value returned if the env var doesn't exist Returns ------- str The value of env var before it is set to the new value (RbtenvironRm(RRut default_valtprev_val((sP/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/test_utils.pyt set_env_varJs cC`sY|d7(t|j|js5|d8(tS|d8(t|j|jS(s%Check whether two NDArrays sharing the same memory block Parameters ---------- array1 : NDArray First NDArray to be checked array2 : NDArray Second NDArray to be checked Returns ------- bool Whether two NDArrays share the same memory i(RRRq(tarray1tarray2((sP/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/test_utils.pyt same_array`s    c c`sttjdu}yUtjj}tj|}ztj|j|dVWdtj||XWntk rdVnXWdQXdS(sc Discards error output of a routine if invoked as: with discard_stderr(): ... twN( RoRbtdevnulltsyststderrtfilenotduptdup2tAttributeError(t bit_buckett stderr_filenot old_stderr((sP/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/test_utils.pytdiscard_stderrws  t DummyItercB`s)eZdZdZdZdZRS(sA dummy iterator that always returns the same batch of data (the first data batch of the real data iter). This is usually used for speed testing. Parameters ---------- real_iter: mx.io.DataIter The real data iterator where the first batch of data comes from cC`sStt|j||_|j|_|j|_|j|_t||_dS(N( tsuperRt__init__t real_itert provide_datat provide_labelRtnextt the_batch(tselfR((sP/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/test_utils.pyRs     cC`s|S(N((R((sP/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/test_utils.pyt__iter__scC`s|jS(sGet a data batch from iterator. The first data batch of real iter is always returned. StopIteration will never be raised. Returns ------- DataBatch The data of next batch. (R(R((sP/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/test_utils.pyRs (t__name__t __module__t__doc__RRR(((sP/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/test_utils.pyRs  cC`s|dkstgt|D]}d|^q}gt|D]6}||t|||dt|f^qB}||fS(sGenerate the buckets and probabilities for chi_square test when the ppf (Quantile function) is specified. Parameters ---------- ppf : function The Quantile function that takes a probability and maps it back to a value. It's the inverse of the cdf function nbuckets : int size of the buckets Returns ------- buckets : list of tuple The generated buckets probs : list The generate probabilities ig?i(RR/tfloat(tppftnbucketsRytprobsR3tbuckets((sP/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/test_utils.pytgen_buckets_probs_with_ppfs#Ii@BcC`sgtj||}|j}||d|tj|ko`||d|tj|k}|S(shTest the generator by matching the mean. We test the sample mean by checking if it falls inside the range (mu - 3 * sigma / sqrt(n), mu + 3 * sigma / sqrt(n)) References:: @incollection{goucher2009beautiful, title={Beautiful Testing: Leading Professionals Reveal How They Improve Software}, author={Goucher, Adam and Riley, Tim}, year={2009}, chapter=10 } Examples:: generator = lambda x: np.random.normal(0, 1.0, size=x) mean_check_ret = mean_check(generator, 0, 1.0) Parameters ---------- generator : function The generator function. It's expected to generate N i.i.d samples by calling generator(N). mu : float sigma : float nsamples : int Returns ------- ret : bool Whether the mean test succeeds i(RRtmeantsqrt(t generatortmutsigmatnsamplestsamplest sample_meanR((sP/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/test_utils.pyt mean_checks ! !!t MXNET_HOMEcC`s|tjkrtj|}ntjjtj}tjj|dd}tjj|rb|Stjj|dddd}tjj|r|StddS(sGet path to the im2rec.py tool Parameters ---------- home_env : str Env variable that holds the path to the MXNET folder Returns ------- str The path to im2rec.py ttoolss im2rec.pys..s&Could not find path to tools/im2rec.pyN( RbRRcRdRAt__file__RetisfiletIOError(thome_envt mxnet_patht im2rec_path((sP/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/test_utils.pytget_im2rec_pathscC`stj||}|jdd}||ddtjd|d|dko||ddtjd|d|dk}|S(sTest the generator by matching the variance. It will need a large number of samples and is not recommended to use We test the sample variance by checking if it falls inside the range (sigma^2 - 3 * sqrt(2 * sigma^4 / (n-1)), sigma^2 + 3 * sqrt(2 * sigma^4 / (n-1))) References:: @incollection{goucher2009beautiful, title={Beautiful Testing: Leading Professionals Reveal How They Improve Software}, author={Goucher, Adam and Riley, Tim}, year={2009}, chapter=10 } Examples:: generator = lambda x: np.random.normal(0, 1.0, size=x) var_check_ret = var_check(generator, 0, 1.0) Parameters ---------- generator : function The generator function. It's expected to generate N i.i.d samples by calling generator(N). sigma : float nsamples : int Returns ------- ret : bool Whether the variance test succeeds tddofiiii(RRRR(RRRRt sample_varR((sP/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/test_utils.pyt var_check s !11c C`s?tstdnt|ts*t||}t|t|ksTtt|dttfrIt}tj t|dfdtj }xt |D]\}}||d||dkst|t|dkr||d||ddkstn||d||d<||d||dd 0.05 threshold to get the final result. Examples:: buckets, probs = gen_buckets_probs_with_ppf(lambda x: ss.norm.ppf(x, 0, 1), 5) generator = lambda x: np.random.normal(0, 1.0, size=x) p = chi_square_check(generator=generator, buckets=buckets, probs=probs) assert(p > 0.05) Parameters ---------- generator: function A function that is assumed to generate i.i.d samples from a specific distribution. generator(N) should generate N random samples. buckets: list of tuple or list of number The buckets to run the chi-square the test. Make sure that the buckets cover the whole range of the distribution. Also, the buckets must be in ascending order and have no intersection probs: list or tuple The ground-truth probability of the random value fall in a specific bucket. nsamples:int The number of samples to generate for the testing Returns ------- p : float p value that the generator has the expected distribution. A higher value indicates a larger confidence obs_freq : list Observed frequency of buckets expected_freq : list The expected (ground-truth) frequency of the buckets sPscipy is not available. Please check if the scipy python bindings are installed.iiR9itsidetrightR;tf_obstf_exp(tssRDRRRRRR@RRMRR?RqRRR9t searchsortedR&R/Rt chisquare( RRRRRtcontinuous_distt buckets_npyR3Ryt expected_freqtsample_bucket_idstobs_freqtp((sP/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/test_utils.pytchi_square_check1s6* %")!( !g333333?c C`sg}g}g}xet|D]W} td|d|d|d|\} } } |j| |j| |j| qWtj|dkj} | ||krtdt|t|t|t|t|fn|S(sVerify whether the generator is correct using chi-square testing. The test is repeated for "nrepeat" times and we check if the success rate is above the threshold (25% by default). Parameters ---------- generator: function A function that is assumed to generate i.i.d samples from a specific distribution. generator(N) should generate N random samples. buckets: list of tuple or list of number The buckets to run the chi-square the test. Make sure that the buckets cover the whole range of the distribution. Also, the buckets must be in ascending order and have no intersection probs: list or tuple The ground-truth probability of the random value fall in a specific bucket. nsamples: int The number of samples to generate for the testing nrepeat: int The times to repeat the test success_rate: float The desired success rate Returns ------- cs_ret_l: list The p values of the chi-square test. RRRRg?sZGenerator test fails, Chi-square p=%s, obs_freq=%s, expected_freq=%s. buckets=%s, probs=%s(R/RR"RRRRR(RRRRtnrepeatt success_ratetcs_ret_lt obs_freq_ltexpected_freq_lRytcs_retRRt success_num((sP/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/test_utils.pytverify_generator{s  cC`s|dk r|dk rt|tr]xXt||D]\}}t||||q7Wqt|j|jd|d|ndS(sCompare ndarray tuple.RRN(RRRRtcompare_ndarray_tupleRR(tt1tt2RRts1ts2((sP/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/test_utils.pyRs Rxc C`s|dkrEtjjd|dtd|} | jt} n]|dks]|dkrt||ddd|} | jtjd} n td |dkrtjjd|dtd|} | jt} nW|dks|dkr2t||d|} | jtjd} n td |jd | } |jd | }|rxt | |n|j d | | | |j d | | ||rt | |d |d |nt | j | j d |d |d S(sCompare opt1 and opt2.RxR;R R9RlR1R*istype not supported yetiRRN( RARR$R tcopytoRzRERqtcreate_state_multi_precisionRtupdate_multi_precisionRR(topt1topt2R;R9tw_stypetg_stypeRRtcompare_statestw2tw1tg2tg1tstate1tstate2((sP/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/test_utils.pytcompare_optimizers. $  $ (RR(RR(lRt __future__RRRR,R{R}RAR;RVRRbRjRgRRt contextlibRtnumpyRt numpy.testingttestingRt numpy.randomRRNt scipy.statststatsRRDRRltmxnetRARRRtndarray.ndarrayRRRtsymbolRR R RRRRR#R-R7RqRLRVRdRkRwRzRRRRRRRRRRRRRRRRRR@RRR!R'R5RRRR]RyRRRRRRRRRRRtDataIterRRRRRRRRR(((sP/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/test_utils.pyts                        1     Q          9=I   H I   F     !    '  ' J.