{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# BYOC Inference for paddleOCR"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We now want to use the model to perform inference, this includes: \n",
    "\n",
    "- Create model for the training outputCreate Endpoint Configuration \n",
    "- Create a configuration defining an endpoint.Create Endpoint \n",
    "- Use the configuration to create an inference endpoint.Perform Inference \n",
    "- Perform inference on some input data using the endpoint."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Deploy model\n",
    "we now create a SageMaker Model from the training output. Using the model we can create an Endpoint Configuration.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%time\n",
    "\n",
    "from sagemaker import get_execution_role\n",
    "role = get_execution_role()\n",
    "\n",
    "\n",
    "import sagemaker\n",
    "import boto3\n",
    "from time import gmtime, strftime\n",
    "\n",
    "sage = boto3.Session().client(service_name='sagemaker') \n",
    "sess = sagemaker.Session()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "account = sess.boto_session.client(\"sts\").get_caller_identity()[\"Account\"]\n",
    "region = sess.boto_session.region_name\n",
    "\n",
    "PROJECT_ID = \"sagemaker-p-5an0os9jqfdi\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "hosting_image = f'{account}.dkr.ecr.{region}.amazonaws.com/{PROJECT_ID}-inference-imagebuild:latest'\n",
    "print('Inference image location: ',hosting_image)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Replace with the correct previous training job name \n",
    "TrainingJobName = \"Replace your own training job name\"\n",
    "TrainingJobName = \"sagemaker-p-5an0os9jqfdi-training-image-2022-05-31-02-01-42-567\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "info = sage.describe_training_job(TrainingJobName=TrainingJobName)\n",
    "model_data = info['ModelArtifacts']['S3ModelArtifacts']\n",
    "print('Model artifacts location: ',model_data)\n",
    "\n",
    "primary_container = {\n",
    "    'Image': hosting_image,\n",
    "    'ModelDataUrl': model_data,\n",
    "}\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "model_name=\"paddle-v1\"\n",
    "\n",
    "create_model_response = sage.create_model(\n",
    "    ModelName = model_name,\n",
    "    ExecutionRoleArn = role,\n",
    "    PrimaryContainer = primary_container)\n",
    "\n",
    "print(create_model_response['ModelArn'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Create Endpoint Configuration\n",
    "\n",
    "At launch, we will support configuring REST endpoints in hosting with multiple models, e.g. for A/B testing purposes. In order to support this, customers create an endpoint configuration, that describes the distribution of traffic across the models, whether split, shadowed, or sampled in some way.In addition, the endpoint configuration describes the instance type required for model deployment, and at launch will describe the autoscaling configuration.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from time import gmtime, strftime\n",
    "import time \n",
    "\n",
    "job_name_prefix = \"paddle\"\n",
    "\n",
    "timestamp = time.strftime('-%Y-%m-%d-%H-%M-%S', time.gmtime())\n",
    "endpoint_config_name = job_name_prefix + '-epc-' + timestamp\n",
    "endpoint_config_response = sage.create_endpoint_config(\n",
    "    EndpointConfigName = endpoint_config_name,\n",
    "    ProductionVariants=[{\n",
    "        'InstanceType':'ml.p3.2xlarge',\n",
    "        'InitialInstanceCount':1,\n",
    "        'ModelName':model_name,\n",
    "        'VariantName':'AllTraffic'}])\n",
    "\n",
    "print('Endpoint configuration name: {}'.format(endpoint_config_name))\n",
    "print('Endpoint configuration arn:  {}'.format(endpoint_config_response['EndpointConfigArn']))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##  Create Endpoint\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%time\n",
    "import time\n",
    "\n",
    "\n",
    "timestamp = time.strftime('-%Y-%m-%d-%H-%M-%S', time.gmtime())\n",
    "endpoint_name = job_name_prefix + '-ep-' + timestamp\n",
    "print('Endpoint name: {}'.format(endpoint_name))\n",
    "\n",
    "endpoint_params = {\n",
    "    'EndpointName': endpoint_name,\n",
    "    'EndpointConfigName': endpoint_config_name,\n",
    "}\n",
    "endpoint_response = sage.create_endpoint(**endpoint_params)\n",
    "print('EndpointArn = {}'.format(endpoint_response['EndpointArn']))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can confirm the endpoint configuration and status by navigating to the \"Endpoints\" tab in the AWS SageMaker console.We will finally create a runtime object from which we can invoke the endpoint."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#  Perform Inference with image url "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import boto3\n",
    "from botocore.config import Config\n",
    "from sagemaker.session import Session\n",
    "\n",
    "config = Config(\n",
    "    read_timeout=120,\n",
    "    retries={\n",
    "        'max_attempts': 0\n",
    "    }\n",
    ")\n",
    "\n",
    "from boto3.session import Session\n",
    "import json"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import sagemaker as sage\n",
    "from time import gmtime, strftime\n",
    "from sagemaker import get_execution_role\n",
    "\n",
    "sess = sage.Session()\n",
    "WORK_DIRECTORY = \"./test/new_hkid_front.jpg\"\n",
    "\n",
    "# S3 prefix\n",
    "prefix = \"DEMO-paddle-byo\"\n",
    "bucket = sess.default_bucket()  \n",
    "\n",
    "image_uri = f'{prefix}/new_hkid_front.jpg'\n",
    "\n",
    "role = get_execution_role()\n",
    "\n",
    "data_location = sess.upload_data(WORK_DIRECTORY, key_prefix=prefix)\n",
    "print(data_location)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "\n",
    "test_data = {\n",
    "    'bucket' : bucket,\n",
    "    'image_uri' : image_uri,\n",
    "    'content_type': \"application/json\",\n",
    "}\n",
    "payload = json.dumps(test_data)\n",
    "print(payload)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'label': ['香港永久性居民身份', 'SAMP!', 'HONGKONGPERMANENTIDENTITYCARE', '而學永晴', 'Z683365', 'LOK，Wing', 'ching', 'MPLE', '2867', '3057', '2532', 'SAMPLE', '出生日期DateofBirth', 'Q3-06-1985', '女F', '大**AZ', 'SAMPLE', '發日期Dateoflssue', '（Q6-96)', '26-11-18', 'Z683365（5)'], 'confidences': [17.026626586914062, 17.13430404663086, 16.066162109375, 12.405200004577637, 16.365564346313477, 15.620736122131348, 16.953876495361328, 16.583984375, 18.67633056640625, 18.979454040527344, 20.221355438232422, 18.48903465270996, 15.502676963806152, 18.379587173461914, 13.910089492797852, 14.175936698913574, 17.855117797851562, 15.46212100982666, 15.202425003051758, 17.087268829345703, 16.568973541259766], 'bbox': [[[158.0, 10.0], [348.0, 11.0], [348.0, 31.0], [158.0, 30.0]], [[11.0, 21.0], [25.0, 21.0], [25.0, 66.0], [11.0, 66.0]], [[73.0, 36.0], [434.0, 36.0], [434.0, 52.0], [73.0, 52.0]], [[10.0, 63.0], [110.0, 67.0], [109.0, 91.0], [9.0, 87.0]], [[356.0, 65.0], [400.0, 65.0], [400.0, 78.0], [356.0, 78.0]], [[19.0, 96.0], [131.0, 96.0], [131.0, 112.0], [19.0, 112.0]], [[142.0, 92.0], [201.0, 95.0], [201.0, 112.0], [142.0, 109.0]], [[14.0, 113.0], [24.0, 113.0], [24.0, 154.0], [14.0, 154.0]], [[202.0, 121.0], [263.0, 121.0], [263.0, 134.0], [202.0, 134.0]], [[257.0, 121.0], [317.0, 121.0], [317.0, 134.0], [257.0, 134.0]], [[314.0, 121.0], [360.0, 121.0], [360.0, 135.0], [314.0, 135.0]], [[12.0, 161.0], [25.0, 161.0], [25.0, 225.0], [12.0, 225.0]], [[203.0, 164.0], [330.0, 166.0], [330.0, 179.0], [203.0, 177.0]], [[203.0, 185.0], [315.0, 184.0], [315.0, 198.0], [203.0, 199.0]], [[332.0, 185.0], [363.0, 185.0], [363.0, 199.0], [332.0, 199.0]], [[203.0, 206.0], [257.0, 208.0], [257.0, 222.0], [203.0, 219.0]], [[12.0, 231.0], [25.0, 231.0], [25.0, 299.0], [12.0, 299.0]], [[204.0, 228.0], [331.0, 229.0], [331.0, 244.0], [204.0, 242.0]], [[207.0, 251.0], [279.0, 251.0], [279.0, 265.0], [207.0, 265.0]], [[204.0, 279.0], [316.0, 279.0], [316.0, 299.0], [204.0, 299.0]], [[353.0, 278.0], [492.0, 276.0], [492.0, 296.0], [353.0, 298.0]]], 'shape': [322, 512, 3]}\n"
     ]
    }
   ],
   "source": [
    "sagemaker_runtime_client = boto3.client('sagemaker-runtime', config=config)\n",
    "session = Session(sagemaker_runtime_client)\n",
    "\n",
    "#     runtime = session.client(\"runtime.sagemaker\",config=config)\n",
    "response = sagemaker_runtime_client.invoke_endpoint(\n",
    "    EndpointName=endpoint_name,\n",
    "    ContentType=\"application/json\",\n",
    "    #ContentType=\"image/jpeg\",\n",
    "    Body=payload)\n",
    "\n",
    "result = json.loads(response[\"Body\"].read())\n",
    "print (result)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Clean up\n",
    "When we're done with the endpoint, we can just delete it and the backing instances will be released.  Run the following cell to delete the endpoint."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(endpoint_name)\n",
    "sage.delete_endpoint(EndpointName=endpoint_name)\n",
    "sage.delete_endpoint_config(EndpointConfigName=endpoint_name)\n",
    "sage.delete_model(ModelName=endpoint_name)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "instance_type": "ml.t3.medium",
  "kernelspec": {
   "display_name": "Python 3 (Data Science)",
   "language": "python",
   "name": "python3__SAGEMAKER_INTERNAL__arn:aws:sagemaker:eu-west-1:470317259841:image/datascience-1.0"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}