ó ùµÈ[c @@sJdZddlmZyddlmZWnek rCdZnXddlZddlZddl Z ddl Z ddl m Z mZmZddl mZmZmZmZddl mZmZdd lmZdd lmZdd lmZdd lmZdd lmZdd lmZddl m!Z!m"Z"ddl#m$Z$d„Z%ddde&d„Z'de(fd„ƒYZ)de(fd„ƒYZ*d„Z+ddd„Z,dddd„Z-dddd„Z.dd„Z/ddd„Z0dddd „Z1ed!„Z2ed"„Z3d'd(e!ƒdd%dddde&ed&„ Z4dS()sLQuantization module for generating quantized (INT8) models from FP32 models.i(tabsolute_import(tstatsNi(t_LIBt check_calltpy_str(tc_arraytc_strtmx_uintt c_str_array(t NDArrayHandlet SymbolHandle(tSymbol(tload(tndarray(tNDArray(tDataIter(tcputContext(tModulec C@s|jƒ}i}xh|D]`}|jdƒr´|tdƒ }||}tjjd|dtj|ƒdtj|ƒddƒ\}} } |||<| ||d <| ||d NDArray th_dict: dict of min/max pairs of layers' output tweight_quantizet bias_quantizet _quantizetdatat min_ranget max_rangetout_typetint8t_mint_maxt_outputii(sweight_quantizes bias_quantize( tlist_argumentstendswithtlenR tcontribtquantizetmintmaxtarray( tqsymtparamstth_dictt inputs_nametquantized_paramstnamet original_nametparamtvaltvmintvmaxtoutput((sZ/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/contrib/quantization.pyt_quantize_params+s0      $ (Rc C@sûd}|dk r6t|tƒs't‚t|ƒ}ng}d}g}|dk r‡t|ƒ}x$|D]}|jt|ƒƒqgWntƒ} tt j |j t j | ƒt|ƒt|ƒt|ƒtt j|ƒt|ƒt j|ƒƒƒt| ƒS(sGiven a symbol object representing a neural network of data type FP32, quantize it into a INT8 network. Parameters ---------- sym : Symbol FP32 neural network symbol. excluded_sym_names : list of strings A list of strings representing the names of the symbols that users want to excluding from being quantized. offline_params : list of strs Names of the parameters that users want to quantize offline. It's always recommended to quantize parameters offline so that quantizing parameters during the inference can be avoided. quantized_dtype: str The quantized destination type for input data. calib_quantize_op : bool Whether perform offline calibration for quantize op. iN(tNonet isinstancetlisttAssertionErrorR tappendRR RRtMXQuantizeSymbolthandletctypestbyrefRRRtc_char_ptc_boolR ( tsymtexcluded_symbolstoffline_paramstquantized_dtypetcalib_quantize_optnum_excluded_symbolst num_offlinetofflinetktout((sZ/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/contrib/quantization.pyt_quantize_symbolSs*          t_LayerOutputCollectorcB@s&eZdZddd„Zd„ZRS(sÜSaves layer output NDArray in a dict with layer names as keys and lists of NDArrays as values. The collected NDArrays will be used for calculating the optimal thresholds for quantization using KL divergence. cC@si|_||_||_dS(N(tnd_dictt include_layertlogger(tselfRKRL((sZ/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/contrib/quantization.pyt__init__‡s  cC@sÈt|ƒ}|jdk r/|j|ƒ r/dStj|tƒ}t|dtƒjt ƒƒ}|j dk rŽ|j j d||j fƒn||j kr´|j |j|ƒn|g|j |tatolgH¯¼šò×z>tdtypeiÿÿÿÿtinf(!R4RtasnumpyR5R R6t enumerateR|R t TypeErrorRkRlt concatenateR#R$tabst histogramtallclosetzerost zeros_liketint32R‹tcopyRR~R{R}R€R‰RjRtentropytargmin(RUtnum_binstnum_quantized_binstitndtmin_valtmax_valtthRˆt hist_edgest zero_bin_idxtnum_half_quantized_binst thresholdst divergencetquantized_binstp_bin_idx_starttp_bin_idx_stoptsliced_nd_histRtleft_outlier_counttright_outlier_countR„tnum_merged_binstjtstarttstoptqtnormtmin_divergence_idxtmin_divergencetopt_th((sZ/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/contrib/quantization.pyt_get_optimal_threshold s„(             (     c C@s#tdkrtdƒ‚nt|tƒs0t‚|dk rV|jd||fƒni}t|jƒƒ}x®|D]¦}||kst‚t ||d|d|ƒ\}}} } ||=|dkrÜ| | f||RLtcur_pathtsymbol_file_path((sZ/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/contrib/quantization.pyt _load_syms c C@s%t|tƒrÜtjjtjjtƒƒ}tjj||ƒ}|jd|ƒt |ƒ}i}i}xd|j ƒD]V\}}|j ddƒ\} } | dkrµ||| stsymbolt data_namest label_namestcontextit for_trainingt data_shapest label_shapesR›RKRtRLs9Collected layer outputs from FP32 model using %d exampless/Calculating optimal thresholds for quantizationtnaivesGCollected layer output min/max values from FP32 model using %d examplessLunknown calibration mode %s received, expected `none`, `naive`, or `entropy`sCalibrating quantized symbolsQuantizing parametersN(sint8suint8(R3R4R5RjRkRlRSRHR»RRRR t provide_labeltbindRQt provide_datat set_paramsRzR½RyRgR2(R>RÐRÑRÙRÚtctxtexcluded_sym_namest calib_modet calib_datatnum_calib_examplest calib_layerRARBRLR&R(RrRJRvt qarg_params((sZ/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/contrib/quantization.pytquantize_model§sfD         !          (sdata(s softmax_label(5RYt __future__RtscipyRR¹R3R:tloggingR¾tnumpyR|tbaseRRRRRRRR R RØR R RÄtR RËRtioRRÛRRtmoduleRR2RQRHtobjectRIRZRgRxRyRzR‰R¸R½RÇRÓRë(((sZ/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/contrib/quantization.pytsN      " (.   Z