ó R;]c @@s>dZddlmZyddlmZWnek rCdZnXddlZddlZddl Z ddl Z ddl m Z mZmZddl mZmZmZmZddl mZmZdd lmZdd lmZdd lmZdd lmZdd lmZdd lmZddl m!Z!m"Z"ddl#m$Z$d„Z%ddd„Z&de'fd„ƒYZ(de'fd„ƒYZ)d„Z*ddd„Z+dddd„Z,dddd„Z-dd„Z.ddd„Z/dddd„Z0ed „Z1ed!„Z2d&d'e!ƒdd$ddded%„ Z3dS((sLQuantization module for generating quantized (INT8) models from FP32 models.i(tabsolute_import(tstatsNi(t_LIBt check_calltpy_str(tc_arraytc_strtmx_uintt c_str_array(t NDArrayHandlet SymbolHandle(tSymbol(tload(tndarray(tNDArray(tDataIter(tcputContext(tModulec C@sÙ|jƒ}i}xÀ|D]¸}|jd ƒr´|tdƒ }||}tjjd|dtj|ƒdtj|ƒddƒ\}}} |||<|||d <| ||d NDArray tweight_quantizet bias_quantizet _quantizetdatat min_ranget max_rangetout_typetint8t_mint_max(RR(tlist_argumentstendswithtlenR tcontribtquantizetmintmax( tqsymtparamst inputs_nametquantized_paramstnamet original_nametparamtvaltvmintvmax((sZ/usr/local/lib/python2.7/site-packages/mxnet-1.2.1-py2.7.egg/mxnet/contrib/quantization.pyt_quantize_params+s      c C@s d}g}|dk r]t|tƒs-t‚t|ƒ}x!|D]}|j|jƒq@Wnd}g}|dk r¨t|ƒ}x$|D]}|jt|ƒƒqˆWntƒ} t t j |jt j | ƒt|ƒtt|ƒt|ƒtt j|ƒƒƒt| ƒS(s<Given a symbol object representing a neural network of data type FP32, quantize it into a INT8 network. Parameters ---------- sym : Symbol FP32 neural network symbol. excluded_symbols : list of symbols Nodes in the network that users do not want to replace with a symbol of INT8 data type. offline_params : list of strs Names of the parameters that users want to quantize offline. It's always recommended to quantize parameters offline so that quantizing parameters during the inference can be avoided. iN(tNonet isinstancetlisttAssertionErrorRtappendthandleRR RRtMXQuantizeSymboltctypestbyrefRRtc_char_pR ( tsymtexcluded_symbolstoffline_paramstnum_excluded_symbolstexcluded_handlestst num_offlinetofflinetktout((sZ/usr/local/lib/python2.7/site-packages/mxnet-1.2.1-py2.7.egg/mxnet/contrib/quantization.pyt_quantize_symbolKs*           t_LayerOutputCollectorcB@s&eZdZddd„Zd„ZRS(sÜSaves layer output NDArray in a dict with layer names as keys and lists of NDArrays as values. The collected NDArrays will be used for calculating the optimal thresholds for quantization using KL divergence. cC@si|_||_||_dS(N(tnd_dictt include_layertlogger(tselfRFRG((sZ/usr/local/lib/python2.7/site-packages/mxnet-1.2.1-py2.7.egg/mxnet/contrib/quantization.pyt__init__xs  cC@sÈt|ƒ}|jdk r/|j|ƒ r/dStj|tƒ}t|dtƒjt ƒƒ}|j dk rŽ|j j d||j fƒn||j kr´|j |j|ƒn|g|j |tatolgH¯¼šò×z>tdtypeiÿÿÿÿ( R0RtasnumpyR1RR2t enumerateRxR t TypeErrorRgRht concatenateR"R#tabst histogramtallclosetzerost zeros_liketint32R‡tcopyR{RzRwRyR|R…Rtentropytargmin(RPtnum_binstnum_quantized_binstitndtmin_valtmax_valtthR„t hist_edegest zero_bin_idxtnum_half_quantized_binst thresholdst divergencetquantized_binstp_bin_idx_starttp_bin_idx_stoptsliced_nd_histR}tleft_outlier_counttright_outlier_countR€tnum_merged_binstjtstarttstoptqtnormtmin_divergence_idxtmin_divergencetopt_th((sZ/usr/local/lib/python2.7/site-packages/mxnet-1.2.1-py2.7.egg/mxnet/contrib/quantization.pyt_get_optimal_thresholdùs~(             (     c C@stdkrtdƒ‚nt|tƒs0t‚|dk rV|jd||fƒni}t|jƒƒ}x|D]‡}||kst‚t ||d|d|ƒ\}}} } ||=| | f||<|dk ru|jd|||| | fƒququW|S(sVGiven a ndarray dict, find the optimal threshold for quantizing each value of the key.s·scipy.stats is required for running entropy mode of calculating the optimal thresholds for quantizing FP32 ndarrays into int8. Please check if the scipy python bindings are installed.snCalculating optimal thresholds for quantization using KL divergence with num_bins=%d and num_quantized_bins=%dR˜R™sIlayer=%s, min_val=%f, max_val=%f, min_divergence=%f, optimal_threshold=%fN( RR/t ImportErrorR0tdictR2RNR1tkeysR³( RER˜R™RGR\t layer_namesR(RœRR±R²((sZ/usr/local/lib/python2.7/site-packages/mxnet-1.2.1-py2.7.egg/mxnet/contrib/quantization.pyt_get_optimal_thresholdsPs$      cC@st|tƒr]tjjtjjtƒƒ}tjj||ƒ}|jd|ƒt |ƒSt|t ƒrp|St dtt |ƒƒƒ‚dS(sQGiven a str as a path the symbol .json file or a symbol, returns a Symbol object.sLoading symbol from file %ssP_load_sym only accepts Symbol or path to the symbol file, while received type %sN( R0Rgtostpathtdirnametrealpatht__file__tjoinRNtsym_loadR RfRh(R9RGtcur_pathtsymbol_file_path((sZ/usr/local/lib/python2.7/site-packages/mxnet-1.2.1-py2.7.egg/mxnet/contrib/quantization.pyt _load_symjs c C@s%t|tƒrÜtjjtjjtƒƒ}tjj||ƒ}|jd|ƒt |ƒ}i}i}xd|j ƒD]V\}}|j ddƒ\} } | dkrµ||| ëstsymbolt data_namest label_namestcontextit for_trainingt data_shapest label_shapesR–RFRpRGs9Collected layer outputs from FP32 model using %d exampless/Calculating optimal thresholds for quantizationtnaivesGCollected layer output min/max values from FP32 model using %d examplessLunknown calibration mode %s received, expected `none`, `naive`, or `entropy`sCalibrating quantized symbolN(R/R0R1RfRgRht get_internalst list_outputstindexR3RNRCR¶R.RRRRt provide_labeltbindRLt provide_datat set_paramsRvR¸RuRc(R9RËRÌRÔRÕtctxtexcluded_sym_namest calib_modet calib_datatnum_calib_examplest calib_layerRGt excluded_symstsym_nametnodestidxR$t qarg_paramsRnRERrR\((sZ/usr/local/lib/python2.7/site-packages/mxnet-1.2.1-py2.7.egg/mxnet/contrib/quantization.pytquantize_model‘sf>          !         (R(RÏ(4RTt __future__RtscipyRR´R/R6tloggingR¹tnumpyRxtbaseRRRRRRRR R RÓR R R¿tR RÆRtioRRÖRRtmoduleRR.RCtobjectRDRURcRtRuRvR…R³R¸RÂRÎRí(((sZ/usr/local/lib/python2.7/site-packages/mxnet-1.2.1-py2.7.egg/mxnet/contrib/quantization.pytsJ      " (   W