3 ^p@sjdZddlmZddlZddlZddlZddlZddlZddlZ ddl Z Gddde Z e ZddZdS)zd ModelHandler defines an example model handler for load and inference requests for MXNet CPU models ) namedtupleNc@sPeZdZdZddZddZddZdd Zd d Zd d Z ddZ ddZ dS) ModelHandlerz0 A sample Model handler implementation. cCsd|_d|_d|_dS)NF) initializedmx_modelshapes)selfr`/home/ec2-user/SageMaker/[Blog] ezsmdeploy python package/ezsmdeploy/ezsmdeploy/model_handler.py__init__szModelHandler.__init__cCsHd}dj||}tj|d}tjj|j|d}tjdj||S)as Get the model prefix name for the model artifacts (symbol and parameter file). This assume model artifact directory contains a symbol file, parameter file, model shapes file and a synset file defining the labels :param model_dir: Path to the directory with model artifacts :return: prefix string for model artifact files z -symbol.jsonz{}/*{}rz"Prefix for the model artifacts: {})formatglobospathbasenamesplitlogginginfo)r model_dirZsym_file_suffixZcheckpoint_prefix_regexZcheckpoint_prefix_filenamecheckpoint_prefixrrr get_model_files_prefixs  z#ModelHandler.get_model_files_prefixc Cstjj|dj|d}tjj|s0tdj|t|}tj||_ WdQRXg}x0|j D]&}|d}|d}|j |t |fq\W|S)z Get the model input data shapes and return the list :param model_dir: Path to the directory with model artifacts :param checkpoint_prefix: Model files prefix name :return: prefix string for model artifact files z{}-{}z shapes.jsonzMissing {} file.Nnameshape) r rjoinr isfile RuntimeErroropenjsonloadrappendtuple) rrrZshapes_file_pathf data_shapes input_dataZ data_nameZ data_shaperrr get_input_data_shapes's   z"ModelHandler.get_input_data_shapesc Csd|_|j}|jd}|jd}|j|}|j||}ytj}tjj|d\}} } tj j ||dd|_ |j j d||j j d|j j| | dd td d } d d | D|_WdQRXWnVtjjtfk r} z0tjdt| tjrtjdj| tWYdd} ~ XnXdS)z Initialize model. This will be called during model loading time :param context: Initial context contains model server system properties. :return: Trgpu_idrN)symbolcontextZ label_namesF)Z for_trainingr!Z label_shapes)Z allow_missingz synset.txtrcSsg|] }|jqSr)rstrip).0lrrr Xsz+ModelHandler.initialize..zFailed to allocate (.*) MemoryzMemory allocation exception: {})rZsystem_propertiesgetrr#mxcpumodelZload_checkpointmodModulerbindZ _label_shapes set_paramsrlabelsbaseZ MXNetErrorrresearchstr IGNORECASErerrorr MemoryError) rr& propertiesrr$rr!ctxsymZ arg_paramsZ aux_paramsr Zmemerrrrr initialize>s(       zModelHandler.initializecCspg}xft|D]Z\}}|jd}tjj|}|dkr8dStjj|dd}|jd}|jdd}|j |qW|S) z Transform raw input into model input data. :param request: list of raw requests :return: list of preprocessed model input data bodyNr)axis)rBrrC) enumerater,r-imgZimdecodeimageZimresize transpose expand_dimsr)rrequestZimg_listidxdataZimg_arrrFrrr preprocess_s    zModelHandler.preprocesscCs2tddg}|jj|||jjdj}|S)z Internal inference methods :param model_input: transformed model input data list :return: list of inference output in NDArray BatchrLr)rrforwardZ get_outputsZasnumpy)r model_inputrNprobrrr inferenceys zModelHandler.inferencecs<tj|tjddd}fdd|ddDgS)z Return predict result in as list. :param inference_output: list of inference output :return: list of predict results NrCcs"g|]}d|j|fqS)zprobability=%f, class=%s)r4)r)i)rQrrr r+sz,ModelHandler.postprocess..r)npsqueezeargsort)rZinference_outputar)rQrr postprocesss zModelHandler.postprocesscCs|j|}|j|}|j|S)z Call preprocess, inference and post-process functions :param data: input data :param context: mms context )rMrRrZ)rrLr&rPZ model_outrrr handles  zModelHandler.handleN) __name__ __module__ __qualname____doc__r rr#r?rMrRrZr[rrrr rs!  rcCs(tjstj||dkrdStj||S)N)_servicerr?r[)rLr&rrr r[s  r[)r_ collectionsrr rrr r6mxnetr-numpyrVobjectrr`r[rrrr s