R;]c @`sdZddlmZmZmZddlZddlZddlZddlZddl Z ddl Z ddl Z ddl Z ddl Z ddlZddlZddlZddlmZddlZddljZddljZyddljZWnek rdZnXyddl Z Wnek r<nXddl!Z"ddl#m$Z$ddl%m&Z&ddl'm(Z(dd l)m*Z*d Z+d Z,d Z-dd Z.ddZ/dZ0dZ1ddZ2dZ3ddde4dZ5dddZ6dZ7dZ8dddddde4dZ9ddde4ddZ:ddddde4dZ;dddde4dZ<dddZ=dddd Z>dd!Z?d"Z@ddd#ZAd$ZBdde4d%ZCdddWe4d(ZDddd)ZEdddXd*ZFd+ZGd,ZHde4d-ZIe-d.ZJe-d/ZKdd0eLe-d1ZMdd2d3ddeLdde-d4 ZNd0ddde4e-d5ZOd6ddd7dde4e-d8ZPddd9dd:d;ZQd<d7dddeLde4d=ZRd>ZSdde4d?ZTd@ZUdAZVdBZWdCZXdddDZYdEZZdFZ[dGdHZ\dIZ]edJZ^dKe"j_j`fdLYZadMZbdNdOZcdPdQZddNdRZedNdSZfdNdTdUdVZgdS(YsTools for testing.i(tabsolute_importtprint_functiontdivisionN(tcontextmanageri(tContext(t_STORAGE_TYPE_STR_TO_ID(tarray(tSymbolcC`stjS(s(Get default context for regression test.(Rt default_ctx(((sP/usr/local/lib/python2.7/site-packages/mxnet-1.2.1-py2.7.egg/mxnet/test_utils.pytdefault_context5scC`s |t_dS(sSet default context.N(RR(tctx((sP/usr/local/lib/python2.7/site-packages/mxnet-1.2.1-py2.7.egg/mxnet/test_utils.pytset_default_context<scC`stjS(s*Get default data type for regression test.(tnptfloat32(((sP/usr/local/lib/python2.7/site-packages/mxnet-1.2.1-py2.7.egg/mxnet/test_utils.pyt default_dtypeAscC`s|dkrdS|S(s4Get default numerical threshold for regression test.g#B ;N(tNone(tatol((sP/usr/local/lib/python2.7/site-packages/mxnet-1.2.1-py2.7.egg/mxnet/test_utils.pytget_atolGscC`s|dkrdS|S(s4Get default numerical threshold for regression test.gh㈵>N(R(trtol((sP/usr/local/lib/python2.7/site-packages/mxnet-1.2.1-py2.7.egg/mxnet/test_utils.pytget_rtolNscG`sOg|D]$}tjj|jt^q}t|dkrK|dS|S(s"Generate some random numpy arrays.ii(R trandomtrandntastypeRtlen(tshapeststarrays((sP/usr/local/lib/python2.7/site-packages/mxnet-1.2.1-py2.7.egg/mxnet/test_utils.pyt random_arraysUs .cC`sJd|kot|kns(t|}tjj||d|!S(sKReturn a k length list of the elements chosen from the population sequence.i(RtAssertionErrorR Rtshuffle(t populationtktpopulation_copy((sP/usr/local/lib/python2.7/site-packages/mxnet-1.2.1-py2.7.egg/mxnet/test_utils.pyt random_sample^s(tuniformcC`st|||}|dks,|dkr;tdn|dksS|dkrbtdn|dkr|d|krtd|||fqndS( s9Validates inputs for csr generation helper functions iis!density has to be between 0 and 1s-num_rows or num_cols should be greater than 0tpowerlawisvnot supported for this density: %s for this shape (%s, %s) Please keep : num_rows * num_cols * density >= 2 * num_rowsN(tintt ValueError(tnum_rowstnum_colstdensityt distributiont total_nnz((sP/usr/local/lib/python2.7/site-packages/mxnet-1.2.1-py2.7.egg/mxnet/test_utils.pyt_validate_csr_generation_inputsfs cC`st|jd}xkt|D]]}|j|}|j|d}tj|j||!}tjj|||j||+q WdS(sShuffle CSR column indices per row This allows validation of unordered column indices, which is not a requirement for a valid CSR matrix iN(RtindptrtrangeR RtindicesRR(tcsrt row_counttit start_indext end_indextsublist((sP/usr/local/lib/python2.7/site-packages/mxnet-1.2.1-py2.7.egg/mxnet/test_utils.pytshuffle_csr_column_indiceszs g?c C`sBt|||ddyddlm}|j|||d|dd}|d k ri|jj|n|tkrt|nt j jj |j|j |j fd||fd|}Wntk r=|d kstd | std t j jjd||fd|} | | |k} | jd}nX|S( sIReturns CSRNDArray with uniform distribution This generates a csr matrix with totalnnz unique randomly chosen numbers from num_rows*num_cols and arranges them in the 2d array in the following way: row_index = (random_number_generated / num_rows) col_index = random_number_generated - row_index * num_cols R)R"i(tsparsetdtypetformatR/tshapes6data_init option is not supported when scipy is absents@shuffle_csr_indices option is not supported when scipy is absentN(R+tscipyR6trandRtdatatfilltTrueR5tmxtndt csr_matrixR.R,t ImportErrorRRR"ttostype( R&R'R(R7t data_inittshuffle_csr_indicestspspR/tresulttdnst masked_dns((sP/usr/local/lib/python2.7/site-packages/mxnet-1.2.1-py2.7.egg/mxnet/test_utils.pyt_get_uniform_dataset_csrs( !   $    $c C`st|||ddt|||}|}tj||fd|}xat|D]S}dtjdd||d<|d}|dkrXtjj |j dSqXWd}xt|D]}t ||} | |krB|| krBdtjdd||<|| d}|dkrtjj |j dSqnxdtd| D]S} dtjdd||| <|d}|dkrRtjj |j dSqRW|d}qW|dkrt d |||fntjj |j dSd S( s<Returns CSRNDArray with powerlaw distribution with exponentially increasing number of non zeros in each row. Not supported for cases where total_nnz < 2*num_rows. This is because the algorithm first tries to ensure that there are rows with no zeros by putting non zeros at beginning of each row. R)R#R7igMbP?iiR/s9not supported for this density: %s for this shape (%s,%s)N( R+R$R tzerosR-trndR"R?R@RRCtminR%( R&R'R(R7R*t unused_nnzt output_arrtrowtcol_maxt col_limitt col_index((sP/usr/local/lib/python2.7/site-packages/mxnet-1.2.1-py2.7.egg/mxnet/test_utils.pyt_get_powerlaw_dataset_csrs:      cC`s|dkrtj|}ntj|ddg}tj|j}tj|ddgddg}x;|js|d}|||d<|j|jqlW|S(sIReturn ndarray composed of passing each array value through some functiontflagstf_indextop_flagst writeonlyiN(RR RtnditerRKR9tfinishedtiternext(t the_inputtfunctiontoutputtit_inputtit_outt val_input((sP/usr/local/lib/python2.7/site-packages/mxnet-1.2.1-py2.7.egg/mxnet/test_utils.pyt assign_eachs !   c C`s|dkrtj|}n|j|jks6ttj|ddg}tj|ddg}tj|j}tj|ddgddg}xR|js|d}|d}||||d<|j|j|jqW|S(sIReturn ndarray composed of passing two array values through some functionRURVRWRXiN( RR RR9RRYRKRZR[( tinput1tinput2R]R^t it_input1t it_input2R`t val_input1t val_input2((sP/usr/local/lib/python2.7/site-packages/mxnet-1.2.1-py2.7.egg/mxnet/test_utils.pyt assign_each2s !     c  C`s|dkrtjn|}|dkr3tn|}|dkrKdn|}|dkr|dksytd||dk r|} t| |dkstn.tj|d} tj| |kj} | j ddkr1t j j |ddd|} | tj gd|tj gffStj| j d|dj|} |dk rs| j|n|dk rt| |} nt j jj| | fd|d|} | | | ffS|d krt|d kst|dkr@t|d|d|d |d |d|}||j|j|jffS|d krt|d|dd|d|}||j|j|jffStstd|tSntstdtSdS(sGenerate a random sparse ndarray. Returns the ndarray, value(np) and indices(np) Parameters ---------- shape: list or tuple stype: str, valid values: "csr" or "row_sparse" density, optional: float, should be between 0 and 1 distribution, optional: str, valid values: "uniform" or "powerlaw" dtype, optional: numpy.dtype, default value is None Returns ------- Result of type CSRNDArray or RowSparseNDArray Examples -------- Below is an example of the powerlaw distribution with csr as the stype. It calculates the nnz using the shape and density. It fills up the ndarray with exponentially increasing number of elements. If there are enough unused_nnzs, n+1th row will have twice more nnzs compared to nth row. else, remaining unused_nnzs will be used in n+1th row If number of cols is too small and we have already reached column size it will fill up all following columns in all followings rows until we reach the required density. >>> csr_arr, _ = rand_sparse_ndarray(shape=(5, 16), stype="csr", density=0.50, distribution="powerlaw") >>> indptr = csr_arr.indptr.asnumpy() >>> indices = csr_arr.indices.asnumpy() >>> data = csr_arr.data.asnumpy() >>> row2nnz = len(data[indptr[1]:indptr[2]]) >>> row3nnz = len(data[indptr[2]:indptr[3]]) >>> assert(row3nnz == 2*row2nnz) >>> row4nnz = len(data[indptr[3]:indptr[4]]) >>> assert(row4nnz == 2*row3nnz) R"t row_sparses,Distribution %s not supported for row_sparseitstypeR7iR9R/iRDRER#R(sDistribution not supported: %ssunknown storage typeN(RRLR;RRRR targwheretflattenR9R?R@RKRRR=RbR6trow_sparse_arrayRJR,R.R<RTtFalse(R9RkR(R7R)RDt rsp_indicest modifier_funcRER.t idx_sampleRGtvaltarrR/((sP/usr/local/lib/python2.7/site-packages/mxnet-1.2.1-py2.7.egg/mxnet/test_utils.pytrand_sparse_ndarraysF'   (&  '   #c C`sd|dkr-tjjt|d|}n3t||d|d|d|d|d|\}}|S(NtdefaultR7R(RqRER)(R?R@RRRu( R9RkR(R7RqRER)Rtt_((sP/usr/local/lib/python2.7/site-packages/mxnet-1.2.1-py2.7.egg/mxnet/test_utils.pyt rand_ndarraySs ! g?c C`s|dkrv|d k r4tj|}|jnd }t||d|d|d|d|d|\} \} } na|dkrt||d|d|d|d|d|\} \} } } nd |} t| | S( sHCreate a sparse array, For Rsp, assure indices are in a canonical formatRjR(RDRpR7RqR/REsUnknown storage type: N(RR tasarraytsortRuR( R9RkRDRpR7RqR(REt arr_indicestarr_dataRwtmsg((sP/usr/local/lib/python2.7/site-packages/mxnet-1.2.1-py2.7.egg/mxnet/test_utils.pytcreate_sparse_array_s,       cC`sq|dkr@d}|d k r@t||dks=tq@nt||d|d|d|d|d|d |S( s@Create sparse array, using only rsp_indices to determine densityRjgiRDRpR7RqR(REN(RRRR~(R9RkR(RDRpR7RqRE((sP/usr/local/lib/python2.7/site-packages/mxnet-1.2.1-py2.7.egg/mxnet/test_utils.pytcreate_sparse_array_zd~s  " i cC`s*tjd|dtjd|dfS(Ni(RLtrandint(tdim0tdim1((sP/usr/local/lib/python2.7/site-packages/mxnet-1.2.1-py2.7.egg/mxnet/test_utils.pyt rand_shape_2dscC`s=tjd|dtjd|dtjd|dfS(Ni(RLR(RRtdim2((sP/usr/local/lib/python2.7/site-packages/mxnet-1.2.1-py2.7.egg/mxnet/test_utils.pyt rand_shape_3dscC`s ttjd|dd|S(Nitsize(ttupleRLR(tnum_dimtdim((sP/usr/local/lib/python2.7/site-packages/mxnet-1.2.1-py2.7.egg/mxnet/test_utils.pyt rand_shape_ndscC`st|tr|g}n-|dk r3t|ntt|j}|}x,tt|D]}||d|}qaW|rt|j}x|D]}d||tnptt build_err_msgtstrR( RRRRRRtindextrelR}((sP/usr/local/lib/python2.7/site-packages/mxnet-1.2.1-py2.7.egg/mxnet/test_utils.pytassert_almost_equals  ' cC`sitj|}tj|}tjtj|tj|}d||Os Rtis_trainii(R titemstbindtforwardtoutputstasnumpyR(tsymR RtinputstexetxR((sP/usr/local/lib/python2.7/site-packages/mxnet-1.2.1-py2.7.egg/mxnet/test_utils.pytsimple_forward<s" c`st|tttfsttjksQtjksQtjksQtt|trt |j t |j krt dt t |j t t |j fqndt|j |D}fd|jD}|S(sParses the given location to a dictionary. Arguments of the provided op `sym` are used as dictionary keys and elements of `location` are used as values. Parameters ---------- sym : Symbol Symbol containing op location : list or tuple or dict Argument values location - if type is list or tuple of `np.ndarray` inner elements are arrays correspoding to ``sym.list_arguments()``. - if type is dict of str -> `np.ndarray` maps the name of arguments to the corresponding `np.ndarray`. *In either case, value of all the arguments must be provided.* ctx : Context Device context. dtype: np.float16 or np.float32 or np.float64 Datatype for mx.nd.array. Returns ------- dict Dictionary with `sym` arguments as keys and `location` elements as values. Examples ------- >>> a = mx.symbol.Variable('a') >>> b = mx.symbol.Variable('b') >>> l1 = np.ndarray([2,3]) >>> l2 = np.ndarray([3,4]) >>> _parse_location(a * b, [l1, l2], None) {'a': , 'b': } >>> _parse_location(a * b, {'a': l1, 'b': l2}, None) {'a': , 'b': } >>> _parse_location(a * b, {'a': l1}, None) ValueError: Symbol arguments and keys of the given location do not match. s_Symbol arguments and keys of the given location do not match.symbol args:%s, location.keys():%scS`si|]\}}||qS(((RRR((sP/usr/local/lib/python2.7/site-packages/mxnet-1.2.1-py2.7.egg/mxnet/test_utils.pys s c`sOi|]E\}}t|tjrBtjj|ddn||qS(R R7(RR tndarrayR?R@R(RRR(R R7(sP/usr/local/lib/python2.7/site-packages/mxnet-1.2.1-py2.7.egg/mxnet/test_utils.pys s (RtdictRRRR tfloat16R tfloat64tsettkeystlist_argumentsR%RtzipR(RtlocationR R7((R R7sP/usr/local/lib/python2.7/site-packages/mxnet-1.2.1-py2.7.egg/mxnet/test_utils.pyt_parse_locationXs+3$:c`stjks3tjks3tjks3t|dk rt|trt|j t|j krt dt t|j t t|j fqn=t|t tfr|j }dt||D}nfd|jD}n|S(sParses the given auxiliary states to a dictionary. Auxiliary states of the provided op `sym` are used as dictionary keys and elements of `aux_states` are used as values. Parameters ---------- sym : Symbol Symbol containing op aux_states : None or list or dict Aux states - if type is list or tuple of `np.ndarray` inner elements are arrays correspoding to ``sym.list_auxiliary_states()``. - if type is dict of str -> `np.ndarray` maps the name of arguments to the corresponding `np.ndarray`. *In either case, all aux states of `sym` must be provided.* ctx : Context Device context. dtype: np.float16 or np.float32 or np.float64 Datatype for mx.nd.array. Returns ------- dict Dictionary with `sym` aux states as keys and `aux_states` elements as values. Examples ------- >>> data = mx.symbol.Variable('data') >>> weight = mx.sym.Variable(name='fc1_weight') >>> fc1 = mx.symbol.FullyConnected(data = data, weight=weight, name='fc1', num_hidden=128) >>> fc2 = mx.symbol.BatchNorm(fc1, name='batchnorm0') >>> mean_states = np.ones(3) >>> var_states = np.ones(3) >>> _parse_aux_states(fc2, [mean_states, var_states], None) {'batchnorm0_moving_var': , 'batchnorm0_moving_mean': } >>> _parse_aux_states(fc2, {'batchnorm0_moving_var': mean_states, ... 'batchnorm0_moving_mean': var_states}, None) {'batchnorm0_moving_var': , 'batchnorm0_moving_mean': } >>> _parse_aux_states(fc2, {'batchnorm0_moving_var': mean_states}, None) ValueError: Symbol aux_states names and given aux_states do not match. saSymbol aux_states names and given aux_states do not match.symbol aux_names:%s, aux_states.keys:%scS`si|]\}}||qS(((RRR((sP/usr/local/lib/python2.7/site-packages/mxnet-1.2.1-py2.7.egg/mxnet/test_utils.pys s c`s7i|]-\}}tjj|dd|qS(R R7(R?R@R(RRR(R R7(sP/usr/local/lib/python2.7/site-packages/mxnet-1.2.1-py2.7.egg/mxnet/test_utils.pys s N(R RR RRRRRRRtlist_auxiliary_statesR%RRRRR(Rt aux_statesR R7t aux_names((R R7sP/usr/local/lib/python2.7/site-packages/mxnet-1.2.1-py2.7.egg/mxnet/test_utils.pyt_parse_aux_statess.3 $% "g-C6?c`sd}tjks<tjks<tjks<tfd|jD}xV|jD]H\}} |j|j} | dkre|| | d|j|(qeqeWx%|D]}tj||||s RvR7Rg@RiN(R RR RRRtarg_dictRktascontiguousarrayR7tkindRR-tprodR9travelRtaux_dictRRRtsum(texecutorRRtepstuse_forward_trainR7Rt approx_gradsRRRkt old_valueR1tkeyRstf_pepstadstypetf_nepst approx_grad((R7sP/usr/local/lib/python2.7/site-packages/mxnet-1.2.1-py2.7.egg/mxnet/test_utils.pyt numeric_gradsH 3  $    $!gMbP?g{Gz?c  `stjks3tjks3tjks3td"krKtnd} td|d|dd}d|jD} t d|d|dd}|d"k rd|jD} nd"} |d"kr|j }d |D}ndt |t t fr6t |}d |D}n0t |tr`|j}|j}ntd |jD}|j|\}}}tjjd }||}tjj|}tt |jd tjj| |d ddfg}tg|D].}|tjjd dd||jf^qd tjjd dd|d fg}fd|jD}| d"k r1t | tstdxx| jD]g\}}||kr|tkr|dkrtjj||j||j||j||| numpy.ndarray maps the name of arguments to the corresponding numpy.ndarray. *In either case, value of all the arguments must be provided.* aux_states : list or tuple or dict, optional The auxiliary states required when generating the executor for the symbol. numeric_eps : float, optional Delta for the finite difference method that approximates the gradient. check_eps : float, optional relative error eps used when comparing numeric grad to symbolic grad. grad_nodes : None or list or tuple or dict, optional Names of the nodes to check gradient on use_forward_train : bool Whether to use is_train=True when computing the finite-difference. ctx : Context, optional Check the gradient computation on the specified device. grad_stype_dict : dict of str->str, optional Storage type dictionary for gradient ndarrays. dtype: np.float16 or np.float32 or np.float64 Datatype for mx.nd.array. References --------- ..[1] https://github.com/Theano/Theano/blob/master/theano/gradient.py cS`stjj|d}|S(sGet a random weight matrix with not too small elements Parameters ---------- shape : list or tuple g?(R RR;(R9tplain((sP/usr/local/lib/python2.7/site-packages/mxnet-1.2.1-py2.7.egg/mxnet/test_utils.pytrandom_projectionDs RRR R7cS`s%i|]\}}|j|qS((R(RRR((sP/usr/local/lib/python2.7/site-packages/mxnet-1.2.1-py2.7.egg/mxnet/test_utils.pys Qs RcS`s%i|]\}}|j|qS((R(RRR((sP/usr/local/lib/python2.7/site-packages/mxnet-1.2.1-py2.7.egg/mxnet/test_utils.pys Us cS`si|]}d|qS(twrite((RR((sP/usr/local/lib/python2.7/site-packages/mxnet-1.2.1-py2.7.egg/mxnet/test_utils.pys Zs cS`si|]}d|qS(R((RR((sP/usr/local/lib/python2.7/site-packages/mxnet-1.2.1-py2.7.egg/mxnet/test_utils.pys ]s cS`s"i|]\}}|j|qS((R9(RRR((sP/usr/local/lib/python2.7/site-packages/mxnet-1.2.1-py2.7.egg/mxnet/test_utils.pys ds t __random_projig{Gz?Rc`s7i|]-\}}tjj|dd|qS(R R7(R?R@R(RRR(R R7(sP/usr/local/lib/python2.7/site-packages/mxnet-1.2.1-py2.7.egg/mxnet/test_utils.pys qs sgrad_stype_dict must be a dictRvtgrad_reqRt args_gradsTExecutor arg_arrays and and location len do not match.Got %d inputs and %d locationsiRc`s&i|]}j|j|qS((t grad_dictR(RR(R(sP/usr/local/lib/python2.7/site-packages/mxnet-1.2.1-py2.7.egg/mxnet/test_utils.pys s RRRs NUMERICAL_%ss BACKWARD_%staddtnulls#Invalid grad_req %s for argument %sN()R RR RRRR RRRRRRRRRRR%t infer_shapeR?RtVariablet make_lossR@RRtnormalR9RRKtcontextR7Rt arg_arraysRRRR>tbackwardRR(RRRt numeric_epsRRt grad_nodesRR tgrad_stype_dictR7Rt location_npytaux_states_npyRt input_shapeRwt out_shapetprojtoutRt args_grad_npyRRtinpstsymbolic_gradstnumeric_gradientstnametfd_gradt orig_gradtsym_grad((R R7RsP/usr/local/lib/python2.7/site-packages/mxnet-1.2.1-py2.7.egg/mxnet/test_utils.pytcheck_numeric_gradients(3   !       5, $!       c  `stjks3tjks3tjks3tdkrKtntd|d|dd}td|d|dd}t |t rg|j D]} || ^q}nfd|j D} |j dd|d| d|} x| jD] } d | (qW| jd tg| jD]} | j^q:}xSt|j ||D]9\}}}t||||d |d |fd |qkW| jS(sSCompares a symbol's forward results with the expected ones. Prints error messages if the forward results are not the same as the expected ones. Parameters --------- sym : Symbol output symbol location : list of np.ndarray or dict of str to np.ndarray The evaluation point - if type is list of np.ndarray Contains all the numpy arrays corresponding to `sym.list_arguments()`. - if type is dict of str to np.ndarray Contains the mapping between argument names and their values. expected : list of np.ndarray or dict of str to np.ndarray The expected output value - if type is list of np.ndarray Contains arrays corresponding to exe.outputs. - if type is dict of str to np.ndarray Contains mapping between sym.list_output() and exe.outputs. check_eps : float, optional Relative error to check to. aux_states : list of np.ndarray of dict, optional - if type is list of np.ndarray Contains all the NumPy arrays corresponding to sym.list_auxiliary_states - if type is dict of str to np.ndarray Contains the mapping between names of auxiliary states and their values. ctx : Context, optional running context dtype: np.float16 or np.float32 or np.float64 Datatype for mx.nd.array. equal_nan: Boolean if True, `nan` is a valid value for checking equivalency (ie `nan` == `nan`) Example ------- >>> shape = (2, 2) >>> lhs = mx.symbol.Variable('lhs') >>> rhs = mx.symbol.Variable('rhs') >>> sym_dot = mx.symbol.dot(lhs, rhs) >>> mat1 = np.array([[1, 2], [3, 4]]) >>> mat2 = np.array([[5, 6], [7, 8]]) >>> ret_expected = np.array([[19, 22], [43, 50]]) >>> check_symbolic_forward(sym_dot, [mat1, mat2], [ret_expected]) RRR R7Rc`s:i|]0\}}tjj|jdd|qS(R R7(R?R@temptyR9(RRR(R R7(sP/usr/local/lib/python2.7/site-packages/mxnet-1.2.1-py2.7.egg/mxnet/test_utils.pys s RRiRs EXPECTED_%ss FORWARD_%sRN(R RR RRRR RRRRt list_outputsRRt grad_arraysRRoRRRR(RRtexpectedRRRR RR7Rtargs_grad_dataRtgRRt output_nametexpectR^((R R7sP/usr/local/lib/python2.7/site-packages/mxnet-1.2.1-py2.7.egg/mxnet/test_utils.pytcheck_symbolic_forwards&23  ! &$ "( gh㈵>Rc  `se| tjks3| tjks3| tjks3t|dkrKt}ntd|d|d|d| }td|d|d|d| }t |t t frdt |j |D}nd|jD} i} x| jD]\}}tjj|d|d| }| dk r|| kr| |}|dk rl|dkrlt|j|d d }n|}|| |str dictionary of mapping argument name to stype for the gradient equal_nan: Boolean if True, `nan` is a valid value for checking equivalency (ie `nan` == `nan`) dtype: np.float16 or np.float32 or np.float64 Datatype for mx.nd.array. Example ------- >>> lhs = mx.symbol.Variable('lhs') >>> rhs = mx.symbol.Variable('rhs') >>> sym_add = mx.symbol.elemwise_add(lhs, rhs) >>> mat1 = np.array([[1, 2], [3, 4]]) >>> mat2 = np.array([[5, 6], [7, 8]]) >>> grad1 = mx.nd.zeros(shape) >>> grad2 = mx.nd.zeros(shape) >>> exec_add = sym_add.bind(default_context(), args={'lhs': mat1, 'rhs': mat2}, ... args_grad={'lhs': grad1, 'rhs': grad2}, grad_req={'lhs': 'write', 'rhs': 'write'}) >>> exec_add.forward(is_train=True) >>> ograd = mx.nd.ones(shape) >>> grad_expected = ograd.copy().asnumpy() >>> check_symbolic_backward(sym_add, [mat1, mat2], [ograd], [grad_expected, grad_expected]) RRR R7RcS`si|]\}}||qS(((RRR((sP/usr/local/lib/python2.7/site-packages/mxnet-1.2.1-py2.7.egg/mxnet/test_utils.pys ,s cS`s1i|]'\}}tjjd|j|qS(R(R RRR9(RRR((sP/usr/local/lib/python2.7/site-packages/mxnet-1.2.1-py2.7.egg/mxnet/test_utils.pys .s RvR(gc`si|]}|qS(((RR(R(sP/usr/local/lib/python2.7/site-packages/mxnet-1.2.1-py2.7.egg/mxnet/test_utils.pys =s cS`si|]\}}||qS(((RRR((sP/usr/local/lib/python2.7/site-packages/mxnet-1.2.1-py2.7.egg/mxnet/test_utils.pys ?s RRRRcS`s%i|]\}}|j|qS((R(RRR((sP/usr/local/lib/python2.7/site-packages/mxnet-1.2.1-py2.7.egg/mxnet/test_utils.pys Zs Rs EXPECTED_%ss BACKWARD_%sRRRs#Invalid grad_req %s for argument %sN(R RR RRRR RRRRRRRRR?R@RR~R9RRRR>RtappendRRRR%(RRt out_gradsRRRRRR t grad_stypesRR7R RRRR@RkR RtoutgRttgradsR((RsP/usr/local/lib/python2.7/site-packages/mxnet-1.2.1-py2.7.egg/mxnet/test_utils.pytcheck_symbolic_backwardst=3  ! "  "  (  %      itwholecK`sl|d krt}n|d kr-d}n|d krp|jd|d||}d|jjD}nPt|tstdt||jd|d|d|jD}x;|jD]-\}} | j |j|j |j|(qW|dkr|j dt |j d |jx|jD]} | jq7Wtj} x7t|D])} |j dt |j d |jqdWtjjtj} | | d |}|S|d kr\|j dtx|jD]} | jqWtj} x$t|D]} |j dtqWtjjtj} | | d |}|Std d S(sSCheck the running speed of a symbol. Parameters ---------- sym : Symbol Symbol to run the speed test. location : none or dict of str to np.ndarray Location to evaluate the inner executor. ctx : Context Running context. N : int, optional Repeat times. grad_req : None or str or list of str or dict of str to str, optional Gradient requirements. typ : str, optional "whole" or "forward" - "whole" Test the forward_backward speed. - "forward" Only test the forward speed. RRR cS`s7i|]-\}}tjjd|jdd|qS(Rtscaleg?(R RRR9(RRRt((sP/usr/local/lib/python2.7/site-packages/mxnet-1.2.1-py2.7.egg/mxnet/test_utils.pys s sExpect dict, get "location"=%scS`s"i|]\}}|j|qS((R9(RRR((sP/usr/local/lib/python2.7/site-packages/mxnet-1.2.1-py2.7.egg/mxnet/test_utils.pys s R%RR g?Rs%typ can only be "whole" or "forward".N(RR t simple_bindRRRRRRRR7RR>RRt wait_to_readttimeR-R?R@twaitallRoR%(RRR tNRttypRRRtiarrR^tticRwttoctforward_backward_timet forward_time((sP/usr/local/lib/python2.7/site-packages/mxnet-1.2.1-py2.7.egg/mxnet/test_utils.pyt check_speedmsL     %%        g?c  C`so|dkrtidtjtj6dtjtj6dtjtj6dtjtj6dtjtj6}nzt|t j ri|tjtj6|tjtj6|tjtj6|tjtj6|tjtj6}nt |dkst t|t r+|gt |}nt |t |ksIt |dj} |dj} g} xit||D]X\} }| j| kst | j| kst | j| jd||qW|dkrin|}|dkrin|}xR| djjD]=\}}||krtjjd|jd|||>> # create the symbol >>> sym = mx.sym.Convolution(num_filter=3, kernel=(3,3), name='conv') >>> # initialize the running context >>> ctx_list =[{'ctx': mx.gpu(0), 'conv_data': (2, 2, 10, 10), 'type_dict': {'conv_data': np.float64}}, {'ctx': mx.gpu(0), 'conv_data': (2, 2, 10, 10), 'type_dict': {'conv_data': np.float32}}, {'ctx': mx.gpu(0), 'conv_data': (2, 2, 10, 10), 'type_dict': {'conv_data': np.float16}}, {'ctx': mx.cpu(0), 'conv_data': (2, 2, 10, 10), 'type_dict': {'conv_data': np.float64}}, {'ctx': mx.cpu(0), 'conv_data': (2, 2, 10, 10), 'type_dict': {'conv_data': np.float32}}] >>> check_consistency(sym, ctx_list) >>> sym = mx.sym.Concat(name='concat', num_args=2) >>> ctx_list = [{'ctx': mx.gpu(0), 'concat_arg1': (2, 10), 'concat_arg0': (2, 10), 'type_dict': {'concat_arg0': np.float64, 'concat_arg1': np.float64}}, {'ctx': mx.gpu(0), 'concat_arg1': (2, 10), 'concat_arg0': (2, 10), 'type_dict': {'concat_arg0': np.float32, 'concat_arg1': np.float32}}, {'ctx': mx.gpu(0), 'concat_arg1': (2, 10), 'concat_arg0': (2, 10), 'type_dict': {'concat_arg0': np.float16, 'concat_arg1': np.float16}}, {'ctx': mx.cpu(0), 'concat_arg1': (2, 10), 'concat_arg0': (2, 10), 'type_dict': {'concat_arg0': np.float64, 'concat_arg1': np.float64}}, {'ctx': mx.cpu(0), 'concat_arg1': (2, 10), 'concat_arg0': (2, 10), 'type_dict': {'concat_arg0': np.float32, 'concat_arg1': np.float32}}] >>> check_consistency(sym, ctx_list) g?gMbP?gh㈵>iiRRR&RR7RRRRRs#Predict Err: ctx %d vs ctx %d at %ss!Train Err: ctx %d vs ctx %d at %sN(-RR R7RR Rtuint8tint32RtnumberstNumberRRRRRRRR'RRRRR9RRRKRRt output_dictRtupdateRRRot enumerateRRRtprintt tracebackt print_excRR>R(Rtctx_listR&Rt arg_paramst aux_paramsRt raise_on_errt ground_truthRt output_namest arg_namestexe_listRR RRtRRtdtypestmax_idxtgtR1tgtarrRtcurr((sP/usr/local/lib/python2.7/site-packages/mxnet-1.2.1-py2.7.egg/mxnet/test_utils.pytcheck_consistencys*    )    ',     (       (   cC`sd}dddg}xA|D]9}ytj|dgdt}Wqtk rTqXqWttg|jdD]}d|kro|^qoS( sReturn a list of GPUs Returns ------- list of int: If there are n GPUs, then return a list [0,1,...,n-1]. Otherwise returns []. ts nvidia-smis/usr/bin/nvidia-smis /usr/local/nvidia/bin/nvidia-smis-Ltuniversal_newliness tGPU(t subprocesst check_outputR>tOSErrorR-Rtsplit(tret nvidia_smitcmdR1((sP/usr/local/lib/python2.7/site-packages/mxnet-1.2.1-py2.7.egg/mxnet/test_utils.pyt list_gpusHs   cC`s|d kr"|jdd}n|d krCtjj|}ntjj||}|dkrtjj|sy!tjd|tj |Wqt k r}|j t j krt d|qqXqn| rtjj|rtjd||St j|dt}|jdks9td |t|d :}x0|jd d D]}|r^|j|q^q^WWd QXtjd|||S(slDownload an given URL Parameters ---------- url : str URL to download fname : str, optional filename of the downloaded file. If None, then will guess a filename from url. dirname : str, optional output directory name. If None, then guess from fname or use the current directory overwrite : bool, optional Default is false, which means skipping download if the local file exists. If true, then download the url to overwrite the local file if exists. Returns ------- str The filename of the downloaded file t/iRKscreate directory %ssfailed to create s%s exists, skipping downloadtstreamisfailed to open %stwbt chunk_sizeiNs"downloaded %s into %s successfully(RRQtostpathtdirnametjointexiststloggingtinfotmakedirsRPterrnotEEXISTtrequeststgetR>t status_codeRtopent iter_contentR(turltfnameR\t overwritetexctrRtchunk((sP/usr/local/lib/python2.7/site-packages/mxnet-1.2.1-py2.7.egg/mxnet/test_utils.pytdownloadZs0   cC`sid}d}||d|d\}}||d|d\}}i|d6|d6|d 6|d 6S( smDownload and load the MNIST dataset Returns ------- dict A dict containing the data c S`stjtjj|>}tjd|jdtj |jdtj }WdQXtjtjj|d}tjd|jd\}}}}tj |jdtj j t |||}|j |jddd d jtjd }WdQX||fS( Ns>IIiR7trbs>IIIIiiiii(tgzipRgR?t test_utilsRotstructtunpacktreadR t frombuffertint8R3RRR9RR ( t label_urlt image_urltflbltlabeltfimgRwtrowstcolstimage((sP/usr/local/lib/python2.7/site-packages/mxnet-1.2.1-py2.7.egg/mxnet/test_utils.pyt read_datas$!'35s http://data.mxnet.io/data/mnist/strain-labels-idx1-ubyte.gzstrain-images-idx3-ubyte.gzst10k-labels-idx1-ubyte.gzst10k-images-idx3-ubyte.gzt train_datat train_labelt test_datat test_label((RR[t train_lblt train_imgttest_lblttest_img((sP/usr/local/lib/python2.7/site-packages/mxnet-1.2.1-py2.7.egg/mxnet/test_utils.pyt get_mnists cC`sKtjjds"tjdntjjdsGtdddndS(skDownloads MNIST dataset as a pkl.gz into a directory in the current directory with the name `data` R<sdata/mnist.pkl.gzs/http://deeplearning.net/data/mnist/mnist.pkl.gzR\N(RZR[tisdirRaR^Ro(((sP/usr/local/lib/python2.7/site-packages/mxnet-1.2.1-py2.7.egg/mxnet/test_utils.pyt get_mnist_pkls  cC`stjjds"tjdntjjd sntjjd sntjjd sntjjd rtddd}tj|}|jdWdQXndS( sDownloads ubyte version of the MNIST dataset into a directory in the current directory with the name `data` and extracts all files in the zip archive to this directory. R<sdata/train-images-idx3-ubytesdata/train-labels-idx1-ubytesdata/t10k-images-idx3-ubytesdata/t10k-labels-idx1-ubytes)http://data.mxnet.io/mxnet/data/mnist.zipR\N( RZR[RRaR^RotzipfiletZipFilet extractall(t zip_file_pathtzf((sP/usr/local/lib/python2.7/site-packages/mxnet-1.2.1-py2.7.egg/mxnet/test_utils.pytget_mnist_ubytes  cC`stjjds"tjdntjjd sntjjd sntjjd sntjjd rtddd}tj|}|jdWdQXndS( sDownloads CIFAR10 dataset into a directory in the current directory with the name `data`, and then extracts all files into the directory `data/cifar`. R<sdata/cifar/train.recsdata/cifar/test.recsdata/cifar/train.lstsdata/cifar/test.lsts+http://data.mxnet.io/mxnet/data/cifar10.zipR\N( RZR[RRaR^RoRRR(RR((sP/usr/local/lib/python2.7/site-packages/mxnet-1.2.1-py2.7.egg/mxnet/test_utils.pyt get_cifar10s  cC`stt|dkrtnt}tjjddddd|d|dtd |d |d |}tjjdd dd d|d|d |d |d |}||fS(s@Returns training and validation iterators for MNIST dataset iRsdata/train-images-idx3-ubyteR{sdata/train-labels-idx1-ubyteR t batch_sizeRtflatt num_partst part_indexsdata/t10k-images-idx3-ubytesdata/t10k-labels-idx1-ubyte(RRRoR>R?tiot MNISTIter(RR RRRttrain_dataitert val_dataiter((sP/usr/local/lib/python2.7/site-packages/mxnet-1.2.1-py2.7.egg/mxnet/test_utils.pytget_mnist_iterators(    cC`sctjj||}tjj|s_t|d|dttj|}|jd|ndS(sDownload and extract zip data. Parameters ---------- data_dir : str Absolute or relative path of the directory name to store zip files url : str URL to download data from data_origin_name : str Name of the downloaded zip file Examples -------- >>> get_zip_data("data_dir", "http://files.grouplens.org/datasets/movielens/ml-10m.zip", "ml-10m.zip") R\RkR[N( RZR[R]R^RoRoRRR(tdata_dirRitdata_origin_nametzip_file((sP/usr/local/lib/python2.7/site-packages/mxnet-1.2.1-py2.7.egg/mxnet/test_utils.pyt get_zip_datas cC`stjj||}tjj||}tjj|st|d|d|dttj|d}t|d/}x|D]}|j |qW|j WdQXtj |ndS(s:Download and extract bz2 data. Parameters ---------- data_dir : str Absolute or relative path of the directory name to store bz2 files data_name : str Name of the output file in which bz2 contents will be extracted url : str URL to download data from data_origin_name : str Name of the downloaded b2 file Examples -------- >>> get_bz2_data("data_dir", "kdda.t", "https://www.csie.ntu.edu.tw/~cjlin/libsvmtools/datasets/binary/kdda.t.bz2", "kdda.t.bz2") RjR\RkRpRXN( RZR[R]R^RoRotbz2tBZ2FileRgRtclosetremove(Rt data_nameRiRtbz_filetfouttline((sP/usr/local/lib/python2.7/site-packages/mxnet-1.2.1-py2.7.egg/mxnet/test_utils.pyt get_bz2_datas RKcC`s&tjj||}|tj|<|S(sVSet environment variable Parameters ---------- key : str Env var to set val : str New value assigned to the env var default_val : str, optional Default value returned if the env var doesn't exist Returns ------- str The value of env var before it is set to the new value (RZtenvironRe(RRst default_valtprev_val((sP/usr/local/lib/python2.7/site-packages/mxnet-1.2.1-py2.7.egg/mxnet/test_utils.pyt set_env_var)s cC`sY|d7(t|j|js5|d8(tS|d8(t|j|jS(s%Check whether two NDArrays sharing the same memory block Parameters ---------- array1 : NDArray First NDArray to be checked array2 : NDArray Second NDArray to be checked Returns ------- bool Whether two NDArrays share the same memory i(RRRo(tarray1tarray2((sP/usr/local/lib/python2.7/site-packages/mxnet-1.2.1-py2.7.egg/mxnet/test_utils.pyt same_array?s    c c`sttjdu}yUtjj}tj|}ztj|j|dVWdtj||XWntk rdVnXWdQXdS(sb Discards error output of a routine if invoked as: with discard_stderr(): ... twN( RgRZtdevnulltsyststderrtfilenotduptdup2tAttributeError(t bit_buckett stderr_filenot old_stderr((sP/usr/local/lib/python2.7/site-packages/mxnet-1.2.1-py2.7.egg/mxnet/test_utils.pytdiscard_stderrVs  t DummyItercB`s)eZdZdZdZdZRS(sA dummy iterator that always returns the same batch of data (the first data batch of the real data iter). This is usually used for speed testing. Parameters ---------- real_iter: mx.io.DataIter The real data iterator where the first batch of data comes from cC`sStt|j||_|j|_|j|_|j|_t||_dS(N( tsuperRt__init__t real_itert provide_datat provide_labelRtnextt the_batch(tselfR((sP/usr/local/lib/python2.7/site-packages/mxnet-1.2.1-py2.7.egg/mxnet/test_utils.pyRts     cC`s|S(N((R((sP/usr/local/lib/python2.7/site-packages/mxnet-1.2.1-py2.7.egg/mxnet/test_utils.pyt__iter__|scC`s|jS(sGet a data batch from iterator. The first data batch of real iter is always returned. StopIteration will never be raised. Returns ------- DataBatch The data of next batch. (R(R((sP/usr/local/lib/python2.7/site-packages/mxnet-1.2.1-py2.7.egg/mxnet/test_utils.pyRs (t__name__t __module__t__doc__RRR(((sP/usr/local/lib/python2.7/site-packages/mxnet-1.2.1-py2.7.egg/mxnet/test_utils.pyRks  cC`s|dkstgt|D]}d|^q}gt|D]6}||t|||dt|f^qB}||fS(sGenerate the buckets and probabilities for chi_square test when the ppf (Quantile function) is specified. Parameters ---------- ppf : function The Quantile function that takes a probability and maps it back to a value. It's the inverse of the cdf function nbuckets : int size of the buckets Returns ------- buckets : list of tuple The generated buckets probs : list The generate probabilities ig?i(RR-tfloat(tppftnbucketsRwtprobsR1tbuckets((sP/usr/local/lib/python2.7/site-packages/mxnet-1.2.1-py2.7.egg/mxnet/test_utils.pytgen_buckets_probs_with_ppfs#Ii@BcC`sgtj||}|j}||d|tj|ko`||d|tj|k}|S(shTest the generator by matching the mean. We test the sample mean by checking if it falls inside the range (mu - 3 * sigma / sqrt(n), mu + 3 * sigma / sqrt(n)) References:: @incollection{goucher2009beautiful, title={Beautiful Testing: Leading Professionals Reveal How They Improve Software}, author={Goucher, Adam and Riley, Tim}, year={2009}, chapter=10 } Examples:: generator = lambda x: np.random.normal(0, 1.0, size=x) mean_check_ret = mean_check(generator, 0, 1.0) Parameters ---------- generator : function The generator function. It's expected to generate N i.i.d samples by calling generator(N). mu : float sigma : float nsamples : int Returns ------- ret : bool Whether the mean test succeeds i(R Rtmeantsqrt(t generatortmutsigmatnsamplestsamplest sample_meanR((sP/usr/local/lib/python2.7/site-packages/mxnet-1.2.1-py2.7.egg/mxnet/test_utils.pyt mean_checks ! !!t MXNET_HOMEcC`s|tjkrtj|}ntjjtj}tjj|dd}tjj|rb|Stjj|dddd}tjj|r|StddS(sGet path to the im2rec.py tool Parameters ---------- home_env : str Env variable that holds the path to the MXNET folder Returns ------- str The path to im2rec.py ttoolss im2rec.pys..s&Could not find path to tools/im2rec.pyN( RZRR[R\R?t__file__R]tisfiletIOError(thome_envt mxnet_patht im2rec_path((sP/usr/local/lib/python2.7/site-packages/mxnet-1.2.1-py2.7.egg/mxnet/test_utils.pytget_im2rec_pathscC`stj||}|jdd}||ddtjd|d|dko||ddtjd|d|dk}|S(sTest the generator by matching the variance. It will need a large number of samples and is not recommended to use We test the sample variance by checking if it falls inside the range (sigma^2 - 3 * sqrt(2 * sigma^4 / (n-1)), sigma^2 + 3 * sqrt(2 * sigma^4 / (n-1))) References:: @incollection{goucher2009beautiful, title={Beautiful Testing: Leading Professionals Reveal How They Improve Software}, author={Goucher, Adam and Riley, Tim}, year={2009}, chapter=10 } Examples:: generator = lambda x: np.random.normal(0, 1.0, size=x) var_check_ret = var_check(generator, 0, 1.0) Parameters ---------- generator : function The generator function. It's expected to generate N i.i.d samples by calling generator(N). sigma : float nsamples : int Returns ------- ret : bool Whether the variance test succeeds tddofiiii(R RRR(RRRRt sample_varR((sP/usr/local/lib/python2.7/site-packages/mxnet-1.2.1-py2.7.egg/mxnet/test_utils.pyt var_checks !11c C`s?tstdnt|ts*t||}t|t|ksTtt|dttfrIt}tj t|dfdtj }xt |D]\}}||d||dkst|t|dkr||d||ddkstn||d||d<||d||dd 0.05 threshold to get the final result. Examples:: buckets, probs = gen_buckets_probs_with_ppf(lambda x: ss.norm.ppf(x, 0, 1), 5) generator = lambda x: np.random.normal(0, 1.0, size=x) p = chi_square_check(generator=generator, buckets=buckets, probs=probs) assert(p > 0.05) Parameters ---------- generator: function A function that is assumed to generate i.i.d samples from a specific distribution. generator(N) should generate N random samples. buckets: list of tuple or list of number The buckets to run the chi-square the test. Make sure that the buckets cover the whole range of the distribution. Also, the buckets must be in ascending order and have no intersection probs: list or tuple The ground-truth probability of the random value fall in a specific bucket. nsamples:int The number of samples to generate for the testing Returns ------- p : float p value that the generator has the expected distribution. A higher value indicates a larger confidence obs_freq : list Observed frequency of buckets expected_freq : list The expected (ground-truth) frequency of the buckets sPscipy is not available. Please check if the scipy python bindings are installed.iiR7itsidetrightR9tf_obstf_exp(tssRBRRRRRR>R RKR R9RoRRR4t searchsortedR$R-Rt chisquare( RRRRRtcontinuous_distt buckets_npyR1Rwt expected_freqtsample_bucket_idstobs_freqtp((sP/usr/local/lib/python2.7/site-packages/mxnet-1.2.1-py2.7.egg/mxnet/test_utils.pytchi_square_checks6* %")!( !ig333333?c C`sg}g}g}xet|D]W} td|d|d|d|\} } } |j| |j| |j| qWtj|dkj} | ||krtdt|t|t|t|t|fn|S(sVerify whether the generator is correct using chi-square testing. The test is repeated for "nrepeat" times and we check if the success rate is above the threshold (25% by default). Parameters ---------- generator: function A function that is assumed to generate i.i.d samples from a specific distribution. generator(N) should generate N random samples. buckets: list of tuple or list of number The buckets to run the chi-square the test. Make sure that the buckets cover the whole range of the distribution. Also, the buckets must be in ascending order and have no intersection probs: list or tuple The ground-truth probability of the random value fall in a specific bucket. nsamples: int The number of samples to generate for the testing nrepeat: int The times to repeat the test success_rate: float The desired success rate Returns ------- cs_ret_l: list The p values of the chi-square test. RRRRg?sZGenerator test fails, Chi-square p=%s, obs_freq=%s, expected_freq=%s. buckets=%s, probs=%s(R-RRR RRRR(RRRRtnrepeatt success_ratetcs_ret_lt obs_freq_ltexpected_freq_lRwtcs_retRRt success_num((sP/usr/local/lib/python2.7/site-packages/mxnet-1.2.1-py2.7.egg/mxnet/test_utils.pytverify_generatorXs  (RR(RR(hRt __future__RRRR)RqRsR;R5RNRRZRbR_RRt contextlibRtnumpyR t numpy.testingttestingRt numpy.randomRRLt scipy.statststatsRRBRRdtmxnetR?RRtndarray.ndarrayRRRtsymbolRR R RRRRR!R+R5RoRJRTRbRiRuRxR~RRRRRRRRRRRRRRRRR>RRRR$R2RJRURoRRRRRRRRRRRtDataIterRRRRRRR(((sP/usr/local/lib/python2.7/site-packages/mxnet-1.2.1-py2.7.egg/mxnet/test_utils.pyts                        1     Q          9=I   H J  5     !    '  ' J