ó ùµÈ[c@@s.dZddlmZddlZddlZddlmZddlmZddlm Z m Z m Z m Z ddlm Z mZmZmZmZmZdd lmZmZmZmZmZdd lmZmZmZmZmZmZdd lm Z m!Z!dd l"m#Z#m$Z$m%Z%dd l&m'Z'm(Z(ddl&m)Z)m*Z*ddl&m+Z+m,Z,ddl"m-Z-e eƒZ.de/fd„ƒYZ0de0fd„ƒYZ1de0fd„ƒYZ2de/fd„ƒYZ3de/fd„ƒYZ4de/fd„ƒYZ5e5ƒZ6d„Z7e7dƒe4ƒdS(snumpy interface for operators.i(tabsolute_importN(tarray(tLock(t CFUNCTYPEtPOINTERt Structuretpointer(tc_void_ptc_inttc_chartc_char_ptcasttc_booli(t_LIBt check_calltMXCallbackListtc_arrayt c_array_buf(tc_strtmx_uinttmx_floattctypes2numpy_sharedt NDArrayHandletpy_str(tsymboltcontext(tNDArrayt_DTYPE_NP_TO_MXt_DTYPE_MX_TO_NP(t_STORAGE_TYPE_STR_TO_IDt_STORAGE_TYPE_ID_TO_STR(t_STORAGE_TYPE_UNDEFINEDt_STORAGE_TYPE_DEFAULT(t_STORAGE_TYPE_CSRt_STORAGE_TYPE_ROW_SPARSE(t _ndarray_clstPythonOpcB@sheZdZgZed„Zd„Zd„Zd„Zd„Z d„Z d„Z d„Z d „Z RS( s­Base class for operators implemented in Python. Parameters ---------- need_top_grad : bool the default need_top_grad() function returns this value. cC@s#d|_||_tjdƒdS(Ns1PythonOp has been deprecated. Please use CustomOp(tNonetinfo_tneed_top_grad_twarningstwarn(tselft need_top_grad((sN/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/operator.pyt__init__4s  cO@s|j||ŽS(N(t get_symbol(R*targstkwargs((sN/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/operator.pyt__call__9scO@stdƒ‚dS(sCCreate a symbol from numpy operator. This should only be called once per instance if the operator contains internal states. Parameters ---------- args : list a list of input arguments (symbols). Returns ------- sym : mxnet.symbol.Symbol sMust override thisN(tNotImplementedError(R*R.R/((sN/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/operator.pyR-<scC@s|d|d(dS(søForward interface. Override to create new operators. Parameters ---------- in_data, out_data: list input and output for forward. See document for corresponding arguments of Operator::Forward iN((R*tin_datatout_data((sN/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/operator.pytforwardLs cC@sd|d(dS(sBackward interface. Can override when creating new operators. Parameters ---------- out_grad, in_data, out_data, in_grad : list input and output for backward. See document for corresponding arguments of Operator::Backward gð?iN((R*tout_gradR2R3tin_grad((sN/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/operator.pytbackwardWs cC@s||dgfS(sInterface for ``infer_shape``. Can override when creating new operators. Parameters ---------- in_shape : list List of argument shapes in the same order as declared in list_arguments. Returns ------- in_shape : list List of argument shapes. Can be modified from in_shape. out_shape : list List of output shapes calculated from in_shape, in the same order as declared in list_arguments. i((R*tin_shape((sN/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/operator.pyt infer_shapecscC@sdgS(s±Interface for ``list_outputs``. Can override when creating new operators. Returns ------- outputs : list List of output blob names. toutput((R*((sN/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/operator.pyt list_outputsvscC@sdgS(sîInterface for ``list_arguments``. Can override when creating new operators. Returns ------- in_shape : list list of argument shapes in the same order as declared in list_arguments. tdata((R*((sN/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/operator.pytlist_arguments€s cC@s|jS(sìWhether this operator needs out_grad for backward. Returns ------- need_top_grad : bool Whether this operator needs out_grad for backward. Should be set to False for loss layers. (R'(R*((sN/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/operator.pyR+‹s (t__name__t __module__t__doc__t _ref_holdertTrueR,R0R-R4R7R9R;R=R+(((sN/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/operator.pyR$*s     tNumpyOpcB@s#eZdZed„Zd„ZRS(s¤Base class for numpy operators. numpy operators allow parts of computation in symbolic graph to be writen in numpy. This feature is intended for quickly hacking out a solution for non performance critical parts. Please consider write a c++ implementation if it becomes a bottleneck. Note that if your operator contains internal states (like arrays), it cannot be used for multi-gpu training. cC@s'tt|ƒj|ƒtjdƒdS(Ns0NumpyOp has been deprecated. Please use CustomOp(tsuperRCR,R(R)(R*R+((sN/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/operator.pyR,Ÿsc @s¤tdttttƒƒttƒtttƒƒttƒtƒ‰tdtttƒtttƒƒtƒ‰tdttttƒƒƒtƒ‰dtf‡‡‡fd†ƒY}‡fd†}‡fd†}‡fd†}‡fd†}‡fd†}|ˆ|ƒˆ|ƒˆ|ƒˆ|ƒˆ|ƒdddddƒ ˆ_ t t t ˆj ƒtƒj dƒ} tjjd | d ˆjƒ||Ž} tjjˆƒ| S( Nt NumpyOpInfoc @sneZdZdˆfdˆfdˆfdˆfdˆfdefdefdefd efd efg ZRS( s@Structure that holds Callback information. Passed to NumpyOpPropR4R7R9R;R=t p_forwardt p_backwardt p_infer_shapetp_list_outputstp_list_arguments(R>R?R@Rt_fields_((t fb_functypetinfer_functypet list_functype(sN/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/operator.pyRE©s         c @s«gtdƒD] }g^q }xgt|ƒD]Y}gt||ƒD]}|||^qC} t||| ƒ} |||j| ƒq,Wˆjd|dd|dƒdS(sC Callback for NumpyOp::ForwardiR2iR3iN(trangeRtappendR4( t num_tensort tensor_ptrst tensor_dimst tensor_shapest tensor_tagst_tittensorstjtshapetbuff(R*(sN/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/operator.pyt forward_entry·s +c @s¿gtdƒD] }g^q }xgt|ƒD]Y}gt||ƒD]}|||^qC} t||| ƒ} |||j| ƒq,Wˆjd|dd|dd|dd|d ƒd S( s C Callback for NumpyOp::BackwardiR2iR3iR6iR5iN(RORRPR7( RQRRRSRTRURVRWRXRYRZR[(R*(sN/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/operator.pytbackward_entryÁs+c @s;tˆjƒƒ}tˆjƒƒ}|||ks:t‚gt|ƒD]1}gt||ƒD]}|||^q^^qG}ˆj|ƒ\} } t| ƒ|ks«t‚t| ƒ|ksÃt‚t| ƒt| ƒ} x[t||ƒD]I}ttt t d| |ƒƒt t ƒƒ||R?R@RBR,R-(((sN/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/operator.pyRC–s t NDArrayOpcB@s,eZdZed„Zd„Zd„ZRS(s¤Base class for numpy operators. numpy operators allow parts of computation in symbolic graph to be writen in numpy. This feature is intended for quickly hacking out a solution for non performance critical parts. Please consider write a c++ implementation if it becomes a bottleneck. Note that if your operator contains internal states (like arrays), it cannot be used for multi-gpu training. cC@s'tt|ƒj|ƒtjdƒdS(Ns2NDArrayOp has been deprecated. Please use CustomOp(RDRuR,R(R)(R*R+((sN/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/operator.pyR,sc @s¼tttttƒttƒtƒ‰tttttƒtttƒƒtƒ‰ttttttƒƒƒtƒ‰ttttttttƒtƒ‰dtf‡‡‡‡fd†ƒY}‡fd†}‡fd†}‡fd†}‡fd†}‡fd†}‡fd†} |ˆ|ƒˆ|ƒˆ|ƒˆ|ƒˆ|ƒˆ| ƒddddddƒ ˆ_ t t t ˆj ƒtƒjd ƒ} tjjd | ||Ž} tjjˆƒ| S( Nt NDArrayOpInfoc @s€eZdZdˆfdˆfdˆfdˆfdˆfdˆfdefdefd efd efd efd efg ZRS( sBStructure that holds Callback information. Passed to NDArrayOpPropR4R7R9R;R=tdeclare_backward_dependencyRFRGRHRIRJtp_declare_backward_dependency(R>R?R@RRK((t deps_functypeRLRMRN(sN/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/operator.pyRvs           c@sïyÇgtdƒD] }g^q}xƒt|ƒD]u}||dkrv|||jtt||tƒdtƒƒq/|||jtt||tƒdtƒƒq/Wˆjd|dd|dƒWn!tk rêdt j ƒGHtSXtS(s!C Callback for NDArrayOp::ForwardiitwritableR2iR3sError in NDArrayOp.forward: %s( RORPRR RRBtFalseR4t Exceptiont tracebackt format_exc(t num_ndarrayt ndarraiesttagsRVRWRX(R*(sN/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/operator.pyR\!s$ $" c @syÛgtdƒD] }g^q}xƒt|ƒD]u}||dkrv|||jtt||tƒdtƒƒq/|||jtt||tƒdtƒƒq/Wˆjd|dd|dd|dd |d ƒWn!tk rþd t j ƒGHtSXtS( s"C Callback for NDArrayOp::BackwardiiRzR2iR3iR6R5isError in NDArrayOp.backward: %s( RORPRR RRBR{R7R|R}R~(RR€RRVRWRX(R*(sN/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/operator.pyR]2s$ $ c @scy;tˆjƒƒ}tˆjƒƒ}|||ks=t‚gt|ƒD]1}gt||ƒD]}|||^qa^qJ}ˆj|ƒ\} } t| ƒ|ks®t‚t| ƒ|ksÆt‚t| ƒt| ƒ} x[t||ƒD]I}ttt t d| |ƒƒt t ƒƒ||R?R@RBR,R-Rw(((sN/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/operator.pyRuýs  ˆtCustomOpcB@s2eZdZd„Zd„Zd„Zd„ZRS(s.Base class for operators implemented in pythoncC@sdS(N((R*((sN/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/operator.pyR,¬scC@sdS(sForward interface. Can override when creating new operators. Parameters ---------- is_train : bool whether this is for training req : list of str how to assign to out_data. can be 'null', 'write', or 'add'. You can optionally use self.assign(dst, req, src) to handle this. in_data, out_data, aux: list of NDArrays input, output, and auxiliary states for forward. See document for corresponding arguments of Operator::Forward N((R*tis_traintreqR2R3taux((sN/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/operator.pyR4¯scC@sdS(sØBackward interface. Can override when creating new operators. Parameters ---------- req : list of str how to assign to in_grad. can be 'null', 'write', or 'add'. You can optionally use self.assign(dst, req, src) to handle this. out_grad, in_data, out_data, in_grad, aux : list of NDArrays input and output for backward. See document for corresponding arguments of Operator::Backward N((R*RŠR5R2R3R6R‹((sN/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/operator.pyR7Às cC@sD|dkrdS|dkr&||(n|dkr@||7(ndS(sAHelper function for assigning into dst depending on requirements.tnullNtwritetinplacetadd(swritesinplace((R*tdstRŠtsrc((sN/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/operator.pytassignÏs     (R>R?R@R,R4R7R’(((sN/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/operator.pyRˆªs    t CustomOpPropcB@skeZdZed„Zd„Zd„Zd„Zd„Zd„Z d„Z d„Z d „Z d „Z RS( sBase class for operator property class implemented in python. Parameters ---------- need_top_grad : bool The default declare_backward_dependency function. Use this value to determine whether this operator needs gradient input. cC@s ||_dS(N(R'(R*R+((sN/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/operator.pyR,áscC@s$||dft|jƒƒdfS(s–infer_shape interface. Can override when creating new operators. Parameters ---------- in_shape : list List of argument shapes in the same order as declared in list_arguments. Returns ------- in_shape : list List of argument shapes. Can be modified from in_shape. out_shape : list List of output shapes calculated from in_shape, in the same order as declared in list_outputs. aux_shape : Optional, list List of aux shapes calculated from in_shape, in the same order as declared in list_auxiliary_states. i((R_R;(R*R8((sN/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/operator.pyR9äscC@s;||dgt|jƒƒ|dgt|jƒƒfS(sinfer_type interface. override to create new operators Parameters ---------- in_type : list of np.dtype list of argument types in the same order as declared in list_arguments. Returns ------- in_type : list list of argument types. Can be modified from in_type. out_type : list list of output types calculated from in_type, in the same order as declared in list_outputs. aux_type : Optional, list list of aux types calculated from in_type, in the same order as declared in list_auxiliary_states. i(R_R;tlist_auxiliary_states(R*tin_type((sN/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/operator.pyt infer_typeúscC@s~x@t|ƒD]2\}}|ttks td||fƒ‚q W|ttgt|jƒƒttgt|jƒƒfS(sÚinfer_storage_type interface. Used to infer storage type of inputs and outputs in the forward pass. When this interface is not implemented, all stypes will be inferred as default. Parameters ---------- in_stype : list of stypes, valid stypes are default, row_sparse and csr Returns ------- in_stype : list list of argument stypes. out_stype : list list of output types calculated from in_stype, in the same order as declared in list_outputs. aux_type : Optional, list list of aux types calculated from in_stype, in the same order as declared in list_auxiliary_states. s Default infer_storage_type implementation doesnt allow non default stypes: found non default stype '%s' for in_stype[%d]. Please implement infer_storage_type and infer_storage_type_backward interface in your custom operator if you have non-default input/output stypes(t enumerateRR R`R_R;R”(R*tin_stypeRWtstype((sN/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/operator.pytinfer_storage_typesc C@sx@t|ƒD]2\}}|ttks td||fƒ‚q Wx]t|ƒD]O\}}|ttkrytt}n|ttksPtd||fƒ‚qPW|||||g}x&|D]} t| ƒttg| (q¿W|d|d|d|d|dfS(sìinfer_storage_type_backward interface. Used to infer storage type of inputs and outputs in the backward pass. Will raise an error if undefined storage type is returned. Returned lists have to be the same size as the input lists to infer_storage_type_backward, otherwise an exception will be thrown. When this interface is not implemented, all stypes will be inferred as default. Parameters ---------- ograd_stype : list list of output gradient storage types in_stype : list list of input storage types out_stype : list list of output storage types igrad_stype : list list of input gradient storage types aux_stype : list list of auxiliary storage types Returns ------- ograd_stype : list list of inferred output gradient storage types in_stype : list list of inferred input storage types out_stype : list list of inferred output storage types igrad_stype : list list of inferred input gradient storage types aux_stype : list list of inferred storage types for auxiliary states sDefault infer_storage_type_backward implementation doesnt allow non default stypes: found non default stype '%s' for ograd_stype[%d]. Please implement infer_storage_type and infer_storage_type_backward interface in your custom operator if you have non-default output gradient stypessDefault infer_storage_type_backward implementation doesnt allow non default stypes: found non default stype '%s' for igrad_stype[%d]. Please implement infer_storage_type and infer_storage_type_backward interface in your custom operator if you have non-default input gradient stypesiiiii(R—RR R`RR_( R*t ograd_stypeR˜t out_stypet igrad_stypet aux_stypeRWR™t stype_listst stype_list((sN/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/operator.pytinfer_storage_type_backward0s#  cC@sdgS(s©list_outputs interface. Can override when creating new operators. Returns ------- outputs : list List of output blob names. R:((R*((sN/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/operator.pyR;fscC@sdgS(s¯list_arguments interface. Can override when creating new operators. Returns ------- arguments : list List of argument blob names. R<((R*((sN/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/operator.pyR=pscC@sgS(s¸list_auxiliary_states interface. Can override when creating new operators. Returns ------- auxs : list list of auxiliary state blob names. ((R*((sN/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/operator.pyR”zscC@s=g}|jr|j|ƒn|j|ƒ|j|ƒ|S(sŒDeclare dependencies of this operator for backward pass. Parameters ---------- out_grad : list of int ids of out_grad blobs. in_data : list of int ids of in_data blobs. out_data: list of int ids of out_data blobs. Returns ------- deps : list of int ids of the needed blobs. (R'R‡(R*R5R2R3R„((sN/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/operator.pyRw„s    cC@stƒS(swCreate an operator that carries out the real computation given the context, input shapes, and input data types.(Rˆ(R*tctxt in_shapest in_dtypes((sN/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/operator.pytcreate_operatorœs(R>R?R@RBR,R9R–RšR¡R;R=R”RwR¥(((sN/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/operator.pyR“Øs     6 t _RegistrycB@s eZdZd„Zd„ZRS(sCustomOp registry.cC@s.i|_d|_tƒ|_tƒ|_dS(Ni(t ref_holdertcountertsett result_depsRtlock(R*((sN/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/operator.pyR,¤s   cC@s6|jjƒ|j}|jd7_|jjƒ|S(sGet index for new entry.i(R«tacquireR¨trelease(R*tcur((sN/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/operator.pytincªs    (R>R?R@R,R¯(((sN/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/operator.pyR¦¢s c@s‡fd†}|S(sGRegister a subclass of CustomOpProp to the registry with name reg_name.c @sþtttttƒttƒttƒttƒ‰tttƒ‰tttttƒtttƒƒtƒ‰tttttƒtƒ‰tttttƒtƒ‰tttttƒttƒtƒ‰ttttttƒƒƒtƒ‰ttttttttƒtƒ‰tttttttƒƒttƒttƒttƒtƒ‰ddddf‰ ‡‡‡‡‡‡‡‡‡‡ ‡ ‡ f d†}ttttttƒttƒttƒƒ}||ƒ}t t j t ˆ ƒ|ƒƒt jƒ}|t j|<ˆ S(s4Register a subclass of CustomOpProp to the registry.RŒRRŽRc @sWt|ƒˆkst‚tgt|ƒD]&}t||ƒt||ƒf^q(ƒ}ˆ|‰ ‡‡ ‡fd†‰‡‡ ‡fd†‰‡‡ ‡fd†‰‡‡ ‡fd†‰‡‡ ‡fd†‰‡‡ ‡fd†‰‡‡ ‡fd†‰‡‡ ‡fd†‰‡ ‡ ‡ ‡‡fd †}tjƒ‰‡fd †}ˆ |ƒˆˆƒˆˆƒˆˆƒˆˆƒˆ ˆƒˆ |ƒˆˆƒˆˆƒˆˆƒg } g| D]}t|ttƒƒ^q¨} d gt | ƒ} t tt | ƒƒtt ttƒ| ƒt ttƒƒƒtt t| ƒt tƒƒƒ|d <|gˆ _ˆ tjˆs4  ".(. lg­.Ê  ÿš