ó šÄïYc@`sßdZddlmZmZmZddlZddlZddlZddlZddl Z ddl Z ddl Z ddl Z ddl Z ddlZddlmZddlZddljZddlZddlmZddlmZddlmZyddlZWnek r$nXej j!d ƒZ"d „Z#d „Z$d „Z%dd „Z'dd„Z(d„Z)d„Z*ddd„Z+d„Z,ddd„Z-ddd2d„Z.ddd„Z/ddd3d„Z0d„Z1de2d„Z3d„Z4d„Z5dde6d„Z7ddd dde6dd!„Z8ddddd"„Z9d#ddd$dd%„Z:ddd&dd'd(„Z;d)d$ddde6dd*„Z<d+„Z=dde2d,„Z>d-„Z?d.d/„Z@d0„ZAed1„ƒZBdS(4sTools for testing.i(tabsolute_importtprint_functiontdivisionN(tcontextmanageri(tContext(tarray(tSymboliÒcC`stjS(s(Get default context for regression test.(Rt default_ctx(((s0build/bdist.linux-armv7l/egg/mxnet/test_utils.pytdefault_context/scC`s |t_dS(sSet default context.N(RR(tctx((s0build/bdist.linux-armv7l/egg/mxnet/test_utils.pytset_default_context6scC`stjS(s*Get default data type for regression test.(tnptfloat32(((s0build/bdist.linux-armv7l/egg/mxnet/test_utils.pyt default_dtype;scC`s|dkrdS|S(s4Get default numerical threshold for regression test.g#B’ ¡œÇ;N(tNone(tatol((s0build/bdist.linux-armv7l/egg/mxnet/test_utils.pytget_atolAscC`s|dkrdS|S(s4Get default numerical threshold for regression test.gñh㈵øä>N(R(trtol((s0build/bdist.linux-armv7l/egg/mxnet/test_utils.pytget_rtolHscG`sOg|D]$}tjj|Œjtƒƒ^q}t|ƒdkrK|dS|S(s"Generate some random numpy arrays.ii(R trandomtrandntastypeR tlen(tshapeststarrays((s0build/bdist.linux-armv7l/egg/mxnet/test_utils.pyt random_arraysOs .cC`sÉt|tƒr|g}n-|dk r3t|ƒntt|jƒƒ}|}x,tt|ƒƒD]}||d|ƒ}qaW|rÅt|jƒ}x|D]}d||s RPtis_trainii(RtitemstbindtforwardtoutputstasnumpyR(tsymR R]tinputstexetxRa((s0build/bdist.linux-armv7l/egg/mxnet/test_utils.pytsimple_forwardòs" c`sÐt|tttfƒst‚t|tƒr‘t|jƒƒt|jƒƒkr°tdt t|jƒƒƒt t|jƒƒƒfƒ‚q°nd„t |jƒ|ƒDƒ}‡fd†|j ƒDƒ}|S(s2Parses the given location to a dictionary. Arguments of the provided op `sym` are used as dictionary keys and elements of `location` are used as values. Parameters ---------- sym : Symbol Symbol containing op location : list or tuple or dict Argument values location - if type is list or tuple of `np.ndarray` inner elements are arrays correspoding to ``sym.list_arguments()``. - if type is dict of str -> `np.ndarray` maps the name of arguments to the corresponding `np.ndarray`. *In either case, value of all the arguments must be provided.* ctx : Context Device context. Returns ------- dict Dictionary with `sym` arguments as keys and `location` elements as values. Examples ------- >>> a = mx.symbol.Variable('a') >>> b = mx.symbol.Variable('b') >>> l1 = np.ndarray([2,3]) >>> l2 = np.ndarray([3,4]) >>> _parse_location(a * b, [l1, l2], None) {'a': , 'b': } >>> _parse_location(a * b, {'a': l1, 'b': l2}, None) {'a': , 'b': } >>> _parse_location(a * b, {'a': l1}, None) ValueError: Symbol arguments and keys of the given location do not match. s_Symbol arguments and keys of the given location do not match.symbol args:%s, location.keys():%scS`si|]\}}||“qS(((RZR[R\((s0build/bdist.linux-armv7l/egg/mxnet/test_utils.pys >s c`s1i|]'\}}tjj|dˆƒ|“qS(R (tmxtndR(RZR[R\(R (s0build/bdist.linux-armv7l/egg/mxnet/test_utils.pys ?s ( RtdictRR$REtsettkeystlist_argumentst ValueErrorRDtzipR^(RctlocationR ((R s0build/bdist.linux-armv7l/egg/mxnet/test_utils.pyt_parse_locations)$:c`sß|dk rÛt|tƒrt|jƒƒt|jƒƒkr¼tdtt|jƒƒƒtt|jƒƒƒfƒ‚q¼n=t|tt fƒr¼|jƒ}d„t ||ƒDƒ}n‡fd†|j ƒDƒ}n|S(s”Parses the given auxiliary states to a dictionary. Auxiliary states of the provided op `sym` are used as dictionary keys and elements of `aux_states` are used as values. Parameters ---------- sym : Symbol Symbol containing op aux_states : None or list or dict Aux states - if type is list or tuple of `np.ndarray` inner elements are arrays correspoding to ``sym.list_auxiliary_states()``. - if type is dict of str -> `np.ndarray` maps the name of arguments to the corresponding `np.ndarray`. *In either case, all aux states of `sym` must be provided.* Returns ------- dict Dictionary with `sym` aux states as keys and `aux_states` elements as values. Examples ------- >>> data = mx.symbol.Variable('data') >>> weight = mx.sym.Variable(name='fc1_weight') >>> fc1 = mx.symbol.FullyConnected(data = data, weight=weight, name='fc1', num_hidden=128) >>> fc2 = mx.symbol.BatchNorm(fc1, name='batchnorm0') >>> mean_states = np.ones(3) >>> var_states = np.ones(3) >>> _parse_aux_states(fc2, [mean_states, var_states], None) {'batchnorm0_moving_var': , 'batchnorm0_moving_mean': } >>> _parse_aux_states(fc2, {'batchnorm0_moving_var': mean_states, ... 'batchnorm0_moving_mean': var_states}, None) {'batchnorm0_moving_var': , 'batchnorm0_moving_mean': } >>> _parse_aux_states(fc2, {'batchnorm0_moving_var': mean_states}, None) ValueError: Symbol aux_states names and given aux_states do not match. saSymbol aux_states names and given aux_states do not match.symbol aux_names:%s, aux_states.keys:%scS`si|]\}}||“qS(((RZR[R\((s0build/bdist.linux-armv7l/egg/mxnet/test_utils.pys vs c`s1i|]'\}}tjj|dˆƒ|“qS(R (RhRiR(RZR[R\(R (s0build/bdist.linux-armv7l/egg/mxnet/test_utils.pys ws N( RRRjRkRltlist_auxiliary_statesRnRDRR$RoR^(Rct aux_statesR t aux_names((R s0build/bdist.linux-armv7l/egg/mxnet/test_utils.pyt_parse_aux_statesCs* $% g-Cëâ6?cC`s!d„|jƒDƒ}x(|jƒD]\}}||j|(q#Wx%|D]}tj||ƒ||”s RUg@R]iN(R^targ_dictR tascontiguousarrayRvtkindRJRtprodR travelRtaux_dictR`RaRbtsum(texecutorRpRstepstuse_forward_traint approx_gradsR[R\t old_valueR)tkeytvaltf_pepstf_neps((s0build/bdist.linux-armv7l/egg/mxnet/test_utils.pyt numeric_grad{s:    "gü©ñÒMbP?g{®Gáz„?c  `s ˆdkrtƒ‰nd„} td|d|dˆƒ}d„|jƒDƒ} td|d|dˆƒ}|dk r’d„|jƒDƒ} nd} |dkrÃ|jƒ}d„|Dƒ} ndt|ttfƒr÷t|ƒ}d „|Dƒ} n0t|t ƒr!|j ƒ} |j ƒ}nt ‚d „|jƒDƒ} |j | \}}}tjjd ƒ}||}tjj|ƒ}t t|jƒƒd tjj| |d ƒdˆƒfgƒ}t g|D]+}|tjd d d||jƒf^qÎd tjd d d|d ƒfgƒ}‡fd†|jƒDƒ}|jˆd| d|d|d|ƒ‰ˆj}t|ƒt|ƒkr®t dt|ƒt|ƒfƒ‚ntˆjƒdksÉt‚ˆjdtƒˆjƒ‡fd†|Dƒ}tˆ| | d|d|ƒ}xï|D]ç}||}||}||}| |dkryt ||||d|d|fƒq| |dkr´t |||||d|d|fƒq| |dkrët ||||d|d|fƒqt d| ||fƒ‚qWdS( s{Verify an operation by checking backward pass via finite difference method. Based on Theano's `theano.gradient.verify_grad` [1] Parameters ---------- sym : Symbol Symbol containing op to test location : list or tuple or dict Argument values used as location to compute gradient - if type is list of numpy.ndarray inner elements should have the same order as mxnet.sym.list_arguments(). - if type is dict of str -> numpy.ndarray maps the name of arguments to the corresponding numpy.ndarray. *In either case, value of all the arguments must be provided.* aux_states : ist or tuple or dict, optional The auxiliary states required when generating the executor for the symbol. numeric_eps : float, optional Delta for the finite difference method that approximates the gradient. check_eps : float, optional relative error eps used when comparing numeric grad to symbolic grad. grad_nodes : None or list or tuple or dict, optional Names of the nodes to check gradient on use_forward_train : bool Whether to use is_train=True when computing the finite-difference. ctx : Context, optional Check the gradient computation on the specified device. References --------- ..[1] https://github.com/Theano/Theano/blob/master/theano/gradient.py cS`stj|Œd}|S(s„Get a random weight matrix with not too small elements Parameters ---------- shape : list or tuple gš™™™™™¹?(t_rngtrand(R tplain((s0build/bdist.linux-armv7l/egg/mxnet/test_utils.pytrandom_projectionÜs RcRpR cS`s%i|]\}}|jƒ|“qS((Rb(RZR[R\((s0build/bdist.linux-armv7l/egg/mxnet/test_utils.pys és RscS`s%i|]\}}|jƒ|“qS((Rb(RZR[R\((s0build/bdist.linux-armv7l/egg/mxnet/test_utils.pys ìs cS`si|]}d|“qS(twrite((RZR[((s0build/bdist.linux-armv7l/egg/mxnet/test_utils.pys ñs cS`si|]}d|“qS(R((RZR[((s0build/bdist.linux-armv7l/egg/mxnet/test_utils.pys ôs cS`s"i|]\}}|j|“qS((R (RZR[R\((s0build/bdist.linux-armv7l/egg/mxnet/test_utils.pys ûs t __random_projig{®Gáz„?tsizec`s1i|]'\}}tjj|dˆƒ|“qS(R (RhRiR(RZR[R\(R (s0build/bdist.linux-armv7l/egg/mxnet/test_utils.pys s tgrad_reqRPt args_gradsTExecutor arg_arrays and and location len do not match.Got %d inputs and %d locationsiR]c`s&i|]}ˆj|jƒ|“qS((t grad_dictRb(RZR[(R(s0build/bdist.linux-armv7l/egg/mxnet/test_utils.pys s R€RRs NUMERICAL_%ss BACKWARD_%staddtnulls#Invalid grad_req %s for argument %sN(!RRRqR^RuRmRRR$RjRJRlRnt infer_shapeRhRctVariabletMakeLossRiRR‰tnormalR R_t arg_arraysRRaRER`RAtbackwardRˆRI(RcRpRst numeric_epsRRt grad_nodesRR RŒt location_npytaux_states_npyRt input_shapeRRt out_shapetprojtoutR[t args_grad_npyR‘tinpstsymbolic_gradstnumeric_gradientstnametfd_gradt orig_gradtsym_grad((R Rs0build/bdist.linux-armv7l/egg/mxnet/test_utils.pytcheck_numeric_gradient·sn"         /8)      c `sdˆd krtƒ‰ntd|d|dˆƒ}td|d|dˆƒ}t|tƒrƒg|jƒD]}||^qj}n‡fd†|jƒDƒ}|jdˆd|d|d|ƒ} x| j D] } d| (qÍW| j d t ƒg| j D]} | j ƒ^qø} xMt|jƒ|| ƒD]3\} }}t||||d | d | fƒq)Wd S( s’Compares a symbol's forward results with the expected ones. Prints error messages if the forward results are not the same as the expected ones. Parameters --------- sym : Symbol output symbol location : list of np.ndarray or dict of str to np.ndarray The evaluation point - if type is list of np.ndarray Contains all the numpy arrays corresponding to `sym.list_arguments()`. - if type is dict of str to np.ndarray Contains the mapping between argument names and their values. expected : list of np.ndarray or dict of str to np.ndarray The expected output value - if type is list of np.ndarray Contains arrays corresponding to exe.outputs. - if type is dict of str to np.ndarray Contains mapping between sym.list_output() and exe.outputs. check_eps : float, optional Relative error to check to. aux_states : list of np.ndarray of dict, optional - if type is list of np.ndarray Contains all the NumPy arrays corresponding to sym.list_auxiliary_states - if type is dict of str to np.ndarray Contains the mapping between names of auxiliary states and their values. ctx : Context, optional running context Example ------- >>> shape = (2, 2) >>> lhs = mx.symbol.Variable('lhs') >>> rhs = mx.symbol.Variable('rhs') >>> sym_dot = mx.symbol.dot(lhs, rhs) >>> mat1 = np.array([[1, 2], [3, 4]]) >>> mat2 = np.array([[5, 6], [7, 8]]) >>> ret_expected = np.array([[19, 22], [43, 50]]) >>> check_symbolic_forward(sym_dot, [mat1, mat2], [ret_expected]) RcRpR Rsc`s4i|]*\}}tjj|jdˆƒ|“qS(R (RhRitemptyR (RZR[R\(R (s0build/bdist.linux-armv7l/egg/mxnet/test_utils.pys [s RPR‘iR]s EXPECTED_%ss FORWARD_%sN(RRRqRuRRjt list_outputsR^R_t grad_arraysR`tFalseRaRbRoRI(RcRptexpectedRRRsR R[targs_grad_dataRtgRfRat output_nametexpecttoutput((R s0build/bdist.linux-armv7l/egg/mxnet/test_utils.pytcheck_symbolic_forward(s,  &$ "(gñh㈵øä>Rc  `sêˆdkrtƒ‰ntd|d|dˆƒ}td|d|dˆƒ}t|ttfƒr…d„t|jƒ|ƒDƒ}nd„|j ƒDƒ} ‡fd†| j ƒDƒ} tˆt ƒrå‡fd†|jƒDƒ‰n7tˆttfƒrd „t|jƒˆƒDƒ‰n|j dˆd |d | d|ƒ} | j d t ƒt|ttfƒr“g|D]} tjj| dˆƒ^ql}n@t|tƒrÁ‡fd †|j ƒDƒ}n|dksÓt‚| j|ƒd„| j ƒDƒ} xí|D]å}ˆ|dkrBt||| |||d|d|fƒqýˆ|dkr‰t||| || |||d|d|fƒqýˆ|dkrÈt| || |||d|d|fƒqýtdˆ||fƒ‚qýWdS(sçCompares a symbol's backward results with the expected ones. Prints error messages if the backward results are not the same as the expected results. Parameters --------- sym : Symbol output symbol location : list of np.ndarray or dict of str to np.ndarray The evaluation point - if type is list of np.ndarray Contains all the NumPy arrays corresponding to ``mx.sym.list_arguments``. - if type is dict of str to np.ndarray Contains the mapping between argument names and their values. out_grads : None or list of np.ndarray or dict of str to np.ndarray NumPys arrays corresponding to sym.outputs for incomming gradient. - if type is list of np.ndarray Contains arrays corresponding to ``exe.outputs``. - if type is dict of str to np.ndarray contains mapping between mxnet.sym.list_output() and Executor.outputs expected : list of np.ndarray or dict of str to np.ndarray expected gradient values - if type is list of np.ndarray Contains arrays corresponding to exe.grad_arrays - if type is dict of str to np.ndarray Contains mapping between ``sym.list_arguments()`` and exe.outputs. check_eps: float, optional Relative error to check to. aux_states : list of np.ndarray or dict of str to np.ndarray grad_req : str or list of str or dict of str to str, optional Gradient requirements. 'write', 'add' or 'null'. ctx : Context, optional Running context. Example ------- >>> lhs = mx.symbol.Variable('lhs') >>> rhs = mx.symbol.Variable('rhs') >>> sym_add = mx.symbol.elemwise_add(lhs, rhs) >>> mat1 = np.array([[1, 2], [3, 4]]) >>> mat2 = np.array([[5, 6], [7, 8]]) >>> grad1 = mx.nd.zeros(shape) >>> grad2 = mx.nd.zeros(shape) >>> exec_add = sym_add.bind(default_context(), args={'lhs': mat1, 'rhs': mat2}, ... args_grad={'lhs': grad1, 'rhs': grad2}, grad_req={'lhs': 'write', 'rhs': 'write'}) >>> exec_add.forward(is_train=True) >>> ograd = mx.nd.ones(shape) >>> grad_expected = ograd.copy().asnumpy() >>> check_symbolic_backward(sym_add, [mat1, mat2], [ograd], [grad_expected, grad_expected]) RcRpR RscS`si|]\}}||“qS(((RZR[R\((s0build/bdist.linux-armv7l/egg/mxnet/test_utils.pys ¥s cS`s.i|]$\}}tjd|jƒ|“qS(R(R‰R˜R (RZR[R\((s0build/bdist.linux-armv7l/egg/mxnet/test_utils.pys ¦s c`s1i|]'\}}tjj|dˆƒ|“qS(R (RhRiR(RZR[R\(R (s0build/bdist.linux-armv7l/egg/mxnet/test_utils.pys §s c`si|]}ˆ|“qS(((RZR[(R(s0build/bdist.linux-armv7l/egg/mxnet/test_utils.pys ©s cS`si|]\}}||“qS(((RZR[R\((s0build/bdist.linux-armv7l/egg/mxnet/test_utils.pys «s RPR‘R]c`s1i|]'\}}tjj|dˆƒ|“qS(R (RhRiR(RZR[R\(R (s0build/bdist.linux-armv7l/egg/mxnet/test_utils.pys ²s cS`s%i|]\}}|jƒ|“qS((Rb(RZR[R\((s0build/bdist.linux-armv7l/egg/mxnet/test_utils.pys ·s Rs EXPECTED_%ss BACKWARD_%sR“R”s#Invalid grad_req %s for argument %sN(RRRqRuRRR$RoRmR^RDR_R`RARhRiRRjRERšRIRn(RcRpt out_gradsR°RRRsRR R£R±RR\tgradsR§((R Rs0build/bdist.linux-armv7l/egg/mxnet/test_utils.pytcheck_symbolic_backwardis@6  ""$.  itwholecK`sl|d krtƒ}n|d kr-d}n|d krp|jd|d||}d„|jjƒDƒ}nPt|tƒs•tdt|ƒƒ‚|jd|d|d„|jƒDƒ}x;|jƒD]-\}} | j |j|j ƒ|j|(qÍW|dkrÀ|j dt ƒ|j d |jƒx|jD]} | jƒq7Wtjƒ} x7t|ƒD])} |j dt ƒ|j d |jƒqdWtjjƒtjƒ} | | d |}|S|d kr\|j dtƒx|jD]} | jƒqæWtjƒ} x$t|ƒD]} |j dtƒqWtjjƒtjƒ} | | d |}|Std ƒ‚d S(sSCheck the running speed of a symbol. Parameters ---------- sym : Symbol Symbol to run the speed test. location : none or dict of str to np.ndarray Location to evaluate the inner executor. ctx : Context Running context. N : int, optional Repeat times. grad_req : None or str or list of str or dict of str to str, optional Gradient requirements. typ : str, optional "whole" or "forward" - "whole" Test the forward_backward speed. - "forward" Only test the forward speed. RRR cS`s4i|]*\}}tjd|jddƒ|“qS(Rtscalegð?(R‰R˜R (RZR[tarr((s0build/bdist.linux-armv7l/egg/mxnet/test_utils.pys ås sExpect dict, get "location"=%scS`s"i|]\}}|j|“qS((R (RZR[R\((s0build/bdist.linux-armv7l/egg/mxnet/test_utils.pys ês RºR]R·gð?R`s%typ can only be "whole" or "forward".N(RRt simple_bindRxR^RRjRERDRRvR`RARšRat wait_to_readttimeRRhRitwaitallR¯Rn(RcRpR tNRttypRQReR§tiarrRµtticRRttoctforward_backward_timet forward_time((s0build/bdist.linux-armv7l/egg/mxnet/test_utils.pyt check_speedÆsL     %%        gð?c  C`s$|dkrtidtjtjƒ6dtjtjƒ6dtjtjƒ6dtjtjƒ6dtjtjƒ6}nzt|t j ƒrîi|tjtjƒ6|tjtjƒ6|tjtjƒ6|tjtjƒ6|tjtjƒ6}nt |ƒdkst ‚t|t ƒr+|gt |ƒ}nt |ƒt |ƒksIt ‚|djƒ} |djƒ} g} xit||ƒD]X\} } | jƒ| ks£t ‚| jƒ| ks»t ‚| j| jd|| ƒqW|dkríin|}|dkrin|}xR| djjƒD]=\}}||krtjjd|jd|ƒ||>> # create the symbol >>> sym = mx.sym.Convolution(num_filter=3, kernel=(3,3), name='conv') >>> # initialize the running context >>> ctx_list =[{'ctx': mx.gpu(0), 'conv_data': (2, 2, 10, 10), 'type_dict': {'conv_data': np.float64}}, {'ctx': mx.gpu(0), 'conv_data': (2, 2, 10, 10), 'type_dict': {'conv_data': np.float32}}, {'ctx': mx.gpu(0), 'conv_data': (2, 2, 10, 10), 'type_dict': {'conv_data': np.float16}}, {'ctx': mx.cpu(0), 'conv_data': (2, 2, 10, 10), 'type_dict': {'conv_data': np.float64}}, {'ctx': mx.cpu(0), 'conv_data': (2, 2, 10, 10), 'type_dict': {'conv_data': np.float32}}] >>> check_consistency(sym, ctx_list) >>> sym = mx.sym.Concat(name='concat', num_args=2) >>> ctx_list = [{'ctx': mx.gpu(0), 'concat_arg1': (2, 10), 'concat_arg0': (2, 10), 'type_dict': {'concat_arg0': np.float64, 'concat_arg1': np.float64}}, {'ctx': mx.gpu(0), 'concat_arg1': (2, 10), 'concat_arg0': (2, 10), 'type_dict': {'concat_arg0': np.float32, 'concat_arg1': np.float32}}, {'ctx': mx.gpu(0), 'concat_arg1': (2, 10), 'concat_arg0': (2, 10), 'type_dict': {'concat_arg0': np.float16, 'concat_arg1': np.float16}}, {'ctx': mx.cpu(0), 'concat_arg1': (2, 10), 'concat_arg0': (2, 10), 'type_dict': {'concat_arg0': np.float64, 'concat_arg1': np.float64}}, {'ctx': mx.cpu(0), 'concat_arg1': (2, 10), 'concat_arg0': (2, 10), 'type_dict': {'concat_arg0': np.float32, 'concat_arg1': np.float32}}] >>> check_consistency(sym, ctx_list) gš™™™™™¹?gü©ñÒMbP?gñh㈵øä>iiRRR»R”R]RRs#Predict Err: ctx %d vs ctx %d at %ss!Train Err: ctx %d vs ctx %d at %sN(,RR Rvtfloat16R tfloat64tuint8tint32RtnumberstNumberRRERR­RmRotappendR½RxR^RR˜R R}RaR-t output_dictRJtupdateR’R`R¯t enumerateRRbRItprintt tracebackt print_excRDRARšR®(Rctctx_listR»Rt arg_paramst aux_paramsR3t raise_on_errt ground_trutht output_namest arg_namestexe_listRR RVR¼ReR§tdtypestmax_idxtgtR)tgtarrRStcurr((s0build/bdist.linux-armv7l/egg/mxnet/test_utils.pytcheck_consistencys¤*    )   ,     -      -  cC`s‘d}dddg}xA|D]9}ytj|dgdtƒ}Wqtk rTqXqWttg|jdƒD]}d|kro|^qoƒƒS( s¡Return a list of GPUs Returns ------- list of int: If there are n GPUs, then return a list [0,1,...,n-1]. Otherwise returns []. ts nvidia-smis/usr/bin/nvidia-smis /usr/local/nvidia/bin/nvidia-smis-Ltuniversal_newliness tGPU(t subprocesst check_outputRAtOSErrorRRtsplit(tret nvidia_smitcmdR)((s0build/bdist.linux-armv7l/egg/mxnet/test_utils.pyt list_gpus›s   cC`s›|d kr"|jdƒd}n|d krCtjj|ƒ}ntjj||ƒ}|dkrØtjj|ƒsØy!tjd|ƒtj |ƒWqÕt k rÑ}|j t j krÒt d|ƒ‚qÒqÕXqØn| rtjj|ƒrtjd|ƒ|St j|dtƒ}|jdks9td |ƒ‚t|d ƒ:}x0|jd d ƒD]}|r^|j|ƒq^q^WWd QXtjd||ƒ|S(slDownload an given URL Parameters ---------- url : str URL to download fname : str, optional filename of the downloaded file. If None, then will guess a filename from url. dirname : str, optional output directory name. If None, then guess from fname or use the current directory overwrite : bool, optional Default is false, which means skipping download if the local file exists. If true, then download the url to overwrite the local file if exists. Returns ------- str The filename of the downloaded file t/iÿÿÿÿRäscreate directory %ssfailed to create s%s exists, skipping downloadtstreamiÈsfailed to open %stwbt chunk_sizeiNs"downloaded %s into %s successfully(RRêtostpathtdirnametjointexiststloggingtinfotmakedirsRéterrnotEEXISTtrequeststgetRAt status_codeREtopent iter_contentR(turltfnameRõt overwritetexctrRUtchunk((s0build/bdist.linux-armv7l/egg/mxnet/test_utils.pytdownload­s0   cC`sid„}d}||d|dƒ\}}||d|dƒ\}}i|d6|d6|d 6|d 6S( smDownload and load the MNIST dataset Returns ------- dict A dict containing the data c S`stjtjj|ƒƒ>}tjd|jdƒƒtj |jƒdtj ƒ}WdQXtjtjj|ƒdƒ}tjd|jdƒƒ\}}}}tj |jƒdtj ƒj t |ƒ||ƒ}|j |jddd d ƒjtjƒd }WdQX||fS( Ns>IIiRvtrbs>IIIIiiiiiÿ(tgzipRRht test_utilsRtstructtunpacktreadR t fromstringtint8RËR#RR RR ( t label_urlt image_urltflbltlabeltfimgRRtrowstcolstimage((s0build/bdist.linux-armv7l/egg/mxnet/test_utils.pyt read_dataês$!'35s http://data.mxnet.io/data/mnist/strain-labels-idx1-ubyte.gzstrain-images-idx3-ubyte.gzst10k-labels-idx1-ubyte.gzst10k-images-idx3-ubyte.gzt train_datat train_labelt test_datat test_label((RRôt train_lblt train_imgttest_lblttest_img((s0build/bdist.linux-armv7l/egg/mxnet/test_utils.pyt get_mnistâs RäcC`s&tjj||ƒ}|tj|<|S(sVSet environment variable Parameters ---------- key : str Env var to set val : str New value assigned to the env var default_val : str, optional Default value returned if the env var doesn't exist Returns ------- str The value of env var before it is set to the new value (RótenvironRþ(R„R…t default_valtprev_val((s0build/bdist.linux-armv7l/egg/mxnet/test_utils.pyt set_env_varþs cC`sY|d7(t|jƒ|jƒƒs5|d8(tS|d8(t|jƒ|jƒƒS(s%Check whether two NDArrays sharing the same memory block Parameters ---------- array1 : NDArray First NDArray to be checked array2 : NDArray Second NDArray to be checked Returns ------- bool Whether two NDArrays share the same memory i(R9RbR¯(tarray1tarray2((s0build/bdist.linux-armv7l/egg/mxnet/test_utils.pyt same_arrays    cc`sqzOtjjƒ}tj|ƒ}ttjdƒ}tj|jƒ|ƒdVWdtj||ƒ|jƒXdS(sc Discards error output of a routine if invoked as: with discard_stderr(): ... twN( tsyststderrtfilenoRótdupRtdevnulltdup2tclose(t stderr_filenot old_stderrt bit_bucket((s0build/bdist.linux-armv7l/egg/mxnet/test_utils.pytdiscard_stderr+s  (R0R1(R0R1(Ct__doc__t __future__RRRR¿R R RÔRÍRçR+RóRûRøt contextlibRtnumpyR t numpy.testingttestingRBtmxnetRhtcontextRtndarrayRtsymbolRRýt ImportErrorRt RandomStateR‰RR R RRRRR+R7R9R;RIRNRORYR¯RgRqRuRARˆR«R¶R¹RÈRãRîRR"R&R)R5(((s0build/bdist.linux-armv7l/egg/mxnet/test_utils.pytsr                      5 8< p@\J ‰ 5