{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# AWS re:Invent 2019\n", "## Train and deploy custom deep learning models with AWS DeepLens and Amazon SageMaker\n", "## Lab 2. Train a Classification Model using Amazon SageMaker" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "When the notebook launchs for the first time, select `conda_mxnet_p36` for kernel." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# Run only once to install the gluoncv package with the following code:\n", "!pip install gluoncv==0.5.0 \n", "!pip install tqdm" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Download Data\n", "\n", "With this step, bear/non-bear image dataset will be downloaded into ./data folder." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "import requests\n", "import os\n", "import csv\n", "import zipfile\n", "from tqdm import tqdm\n", "import time\n", "\n", "ZIP_FILE = './open_images_bears.zip'\n", "ERRORS_FILE = 'download-errors.txt'\n", "CSV_DIR = './image_csv/'\n", "DATA_DIR = './data/'\n", "if not os.path.isdir(DATA_DIR):\n", " os.mkdir(DATA_DIR)\n", " \n", "with zipfile.ZipFile(ZIP_FILE, 'r') as f:\n", " f.extractall(os.path.expanduser(CSV_DIR))\n", " \n", "files = list(filter(lambda x: x.endswith('csv'), os.listdir(CSV_DIR)))\n", "\n", "f = files[0]\n", "with open(CSV_DIR + f, 'r') as f:\n", " reader = csv.reader(f)\n", " records = list(reader)\n", " \n", "def download(url, path):\n", " r = requests.get(url, allow_redirects=True)\n", " if len(r.content) < 1024:\n", " raise Exception((path.split('/')[-1]).split('.')[0])\n", " else:\n", " open(path, 'wb').write(r.content)\n", " \n", "with open(ERRORS_FILE,'w') as f:\n", " f.write('')\n", "for idx,fn in enumerate(files):\n", " print('{}/{} {} is being processed.'.format(idx, len(files), fn))\n", " time.sleep(1)\n", " with open(CSV_DIR + fn, 'r') as f:\n", " reader = csv.reader(f)\n", " records = list(reader)[1:] # no header row\n", " stage = fn.split('-')[0]\n", " lbl = fn.split('-')[1]\n", " dir_path = DATA_DIR + stage\n", " if not os.path.isdir(dir_path):\n", " os.mkdir(dir_path)\n", " dir_path = DATA_DIR + '{}/{}'.format(stage,lbl)\n", " if not os.path.isdir(dir_path):\n", " os.mkdir(dir_path)\n", " \n", " cnt = 0 \n", " for row in tqdm(records):\n", " path = dir_path + '/{}.jpg'.format(row[0])\n", " try:\n", " # If thumnail url is empty, download original url\n", " if not row[13]:\n", " download(row[5], path)\n", " else:\n", " download(row[13], path)\n", " except Exception as e:\n", " with open(ERRORS_FILE,'a') as f:\n", " f.write(e.args[0]+'\\n')\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Upload data to S3" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "import os\n", "import glob\n", "import boto3\n", "\n", "s3_bucket = '__PUT-S3-BUCKET-NAME-WHICH-WAS-CREATED-IN-LAB1__'\n", "\n", "SEARCH_CRITERION = '**/*.jpg'\n", "train_images = glob.glob(os.path.join(DATA_DIR + 'train', SEARCH_CRITERION), recursive=True)\n", "val_images = glob.glob(os.path.join(DATA_DIR + 'val', SEARCH_CRITERION), recursive=True)\n", "test_images = glob.glob(os.path.join(DATA_DIR + 'test', SEARCH_CRITERION), recursive=True)\n", "prefix = 'bear'\n", "\n", "for im_name in tqdm(train_images):\n", " boto3.Session().resource('s3').Bucket(s3_bucket).Object(os.path.join(prefix, 'train' + im_name.split('train')[1])).upload_file(im_name)\n", "for im_name in tqdm(val_images):\n", " boto3.Session().resource('s3').Bucket(s3_bucket).Object(os.path.join(prefix, 'val' + im_name.split('val')[1])).upload_file(im_name)\n", "for im_name in tqdm(test_images):\n", " boto3.Session().resource('s3').Bucket(s3_bucket).Object(os.path.join(prefix, 'test' + im_name.split('test')[1])).upload_file(im_name)\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Now we have all the data stored in S3 bucket. " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Fine-tuning the Image Classification Model\n", "Now that we are done with all the setup that is needed, we are ready to train our object detector. To begin, let us create a sagemaker.estimator.Estimator object. This estimator will launch the training job.\n", "\n", "### Bring your own script (BYOS)\n", "Amazon SageMaker provides pre-built containers to supports deep learning frameworks such as Apache MXNet, TensorFlow, PyTorch, and Chainer. We are going to bring in **bear-classification.py**, which is a image classification script using Gluon CV toolkit (Apache MXNet). The SageMaker MXNet estimator allows us to run single machine or distributed training in SageMaker, using CPU or GPU-based instances.\n", "\n", "\n", "### Training parameters\n", "There are parameters for the training job. These include:\n", "\n", "* **Training instance count**: This is the number of instances on which to run the training. When the number of instances is greater than one, then the image classification algorithm will run in distributed settings.\n", "* **Training instance type**: This indicates the type of machine on which to run the training. Typically, we use GPU instances for these training\n", "* **Output path**: This the s3 folder in which the training output is stored" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "import sagemaker\n", "from sagemaker.mxnet import MXNet\n", "\n", "data_channels = 's3://' + s3_bucket + '/' + prefix\n", "model_artifacts_location = 's3://' + s3_bucket + '/model_dir'\n", "\n", "gluon_bear_classification = MXNet(\"bear-classification.py\", \n", " role=sagemaker.get_execution_role(), \n", " train_instance_count=1,\n", " train_instance_type=\"ml.p2.xlarge\",\n", " output_path= model_artifacts_location,\n", " framework_version=\"1.4.1\",\n", " py_version = \"py3\",\n", " hyperparameters={'batch-size': 128, \n", " 'epochs': 10}) " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Running the Training Job\n", "Once MXNet object is constructed, we can fit it using data stored in S3.\n", "\n", "During training, SageMaker makes this data stored in S3 available in the local filesystem where the bear classification script is running. The **bear-classification.py** script simply loads the train and test data from disk." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "scrolled": true }, "outputs": [], "source": [ "gluon_bear_classification.fit(data_channels)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Once the training is done, you can proceed to Lab3 " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## (Optional) Optimize your edge deployment model using Amazon SageMaker Neo\n", "\n", "You can also optimize the trained model using Amazon SageMaker Neo. This is a sample script for that. \n", "\n", "```python\n", "neo_model = gluon_bear_classification.compile_model(target_instance_family='deeplens', \n", " input_shape={'data':[1, 3, 224, 224]}, # Batch size 1, 3 channels, 224x224 Images.\n", " output_path=model_artifacts_location + '/neo',\n", " framework='mxnet', framework_version='1.4.1')\n", "```" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## (Optional) Creating an inference Endpoint\n", "\n", "After training, we use the MXNet estimator object to build and deploy an MXNetPredictor. This creates a Sagemaker Endpoint -- a hosted prediction service that we can use to perform inference.\n", "\n", "The arguments to the deploy function allow us to set the number and type of instances that will be used for the Endpoint. These do not need to be the same as the values we used for the training job. For example, you can train a model on a set of GPU-based instances, and then deploy the Endpoint to a fleet of CPU-based instances. Here we will deploy the model to a single ml.c4.xlarge instance." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "from sagemaker.mxnet.model import MXNetModel\n", "\n", "training_job_name = gluon_bear_classification.latest_training_job.name\n", "sagemaker_model = MXNetModel(model_data= model_artifacts_location + '/{}/output/model.tar.gz'.format(training_job_name),\n", " role=sagemaker.get_execution_role(),\n", " framework_version=\"1.4.1\",\n", " py_version='py3',\n", " entry_point=\"bear-classification.py\")" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "predictor = sagemaker_model.deploy(initial_instance_count=1,\n", " instance_type='ml.c4.xlarge')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Predict with the finetuned model\n", "\n", "We can test the performance using finetuned weights." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "import json\n", "import numpy as np\n", "from PIL import Image as PILImage\n", "\n", "def test_image(filename):\n", " data = PILImage.open(file_name)\n", "\n", " predictor.content_type = 'application/json'\n", " predictor.accept = 'application/json'\n", " \n", " payload = np.expand_dims((np.asarray(data.resize((224,224))).astype('float16')/255.0).transpose((2,0,1)),0)\n", " result = predictor.predict(payload)[0]\n", " # the result will output the probabilities for all classes\n", " # find the class with maximum probability and print the class index\n", " index = np.argmax(result)\n", " object_categories = ['brown','no','polar']\n", " print(\"Result: label - \" + object_categories[index] + \", probability - \" + str(result[index]))" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "!wget -O /tmp/test.jpg https://19mvmv3yn2qc2bdb912o1t2n-wpengine.netdna-ssl.com/science/files/2013/12/tnc_17745326_preview-1260x708.jpg\n", "file_name = '/tmp/test.jpg'\n", "test_image(file_name)\n", "from IPython.display import Image\n", "Image(file_name)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "!wget -O /tmp/test_2.jpg https://www.nps.gov/lacl/learn/nature/images/Image-w-cred-cap_-1200w-_-Brown-Bear-page_-brown-bear-in-fog_2.jpg\n", "\n", "file_name = '/tmp/test_2.jpg'\n", "test_image(file_name)\n", "from IPython.display import Image\n", "Image(file_name)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "!wget -O /tmp/test_3.jpg https://www.dollargeneral.com/media/catalog/product/cache/image/beff4985b56e3afdbeabfc89641a4582/p/l/plush_teddy-bear_giant_092018.jpg\n", "file_name = '/tmp/test_3.jpg'\n", "test_image(file_name)\n", "from IPython.display import Image\n", "Image(file_name)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Cleanup\n", "After you have finished with this example, remember to delete the prediction endpoint to release the instance(s) associated with it." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "predictor.delete_endpoint()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## (Optional) Lab 2. Train a Classification Model Deep-Dive" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Install GluonCV and the required python packages.\n", "See the link below for GluonCV's `model_zoo` and `utils` packages.\n", "- `model_zoo`: [https://gluon-cv.mxnet.io/model_zoo/index.html](https://gluon-cv.mxnet.io/model_zoo/index.html)\n", "- `utils`: [https://gluon-cv.mxnet.io/api/utils.html](https://gluon-cv.mxnet.io/api/utils.html)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "scrolled": true }, "outputs": [], "source": [ "# Run only once to install the gluoncv package with the following code:\n", "!pip install gluoncv==0.5.0 \n", "!pip install tqdm" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Hyperparameters\n", "----------\n", "\n", "First, let's import all other necessary libraries.\n", "\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "jupyter": { "outputs_hidden": false } }, "outputs": [], "source": [ "import mxnet as mx\n", "import numpy as np\n", "import os, time, shutil\n", "\n", "from mxnet import gluon, image, init, nd\n", "from mxnet import autograd as ag\n", "from mxnet.gluon import nn\n", "from mxnet.gluon.data.vision import transforms\n", "from gluoncv.utils import makedirs\n", "from gluoncv.model_zoo import get_model" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We set the hyperparameters as following:\n", "\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "jupyter": { "outputs_hidden": false } }, "outputs": [], "source": [ "# class: brown, polar, no bear\n", "classes = 3\n", "epochs = 10\n", "lr = 0.001\n", "per_device_batch_size = 128\n", "num_gpus = len(list(mx.test_utils.list_gpus()))\n", "num_workers = 8\n", "ctx = [mx.gpu(i) for i in range(num_gpus)] if num_gpus > 0 else [mx.cpu()]\n", "batch_size = per_device_batch_size * max(num_gpus, 1)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Things to keep in mind:\n", "\n", "1. You can change the `epochs` value to a larger number in your experiments.\n", "\n", "2. `per_device_batch_size` can be a larger number. If you get `cudaMalloc failed: out of memory` error at the training loop, try to decrease this value (e.g. 128).\n", "\n", "3. remember to tune `num_gpus` and `num_workers` according to your machine.\n", "\n", "4. A pre-trained model is already in a pretty good status. So we can start with a small `lr`." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Data Augmentation\n", "-----------------\n", "\n", "In transfer learning, data augmentation can also help.\n", "We use the following augmentation in training:\n", "\n", "1. Randomly crop the image and resize it to 224x224\n", "2. Randomly flip the image horizontally\n", "3. Randomly jitter color and add noise\n", "4. Transpose the data from height \\* width \\* num_channels to num_channels \\* height \\* width, and map values from [0, 255] to [0, 1]\n", "\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "jupyter": { "outputs_hidden": false } }, "outputs": [], "source": [ "jitter_param = 0.4\n", "lighting_param = 0.1\n", "\n", "transform_train = transforms.Compose([\n", " transforms.RandomResizedCrop(224),\n", " transforms.RandomFlipLeftRight(),\n", " transforms.RandomColorJitter(brightness=jitter_param, \n", " contrast=jitter_param,\n", " saturation=jitter_param),\n", " transforms.RandomLighting(lighting_param),\n", " transforms.ToTensor(),\n", "])\n", "\n", "transform_test = transforms.Compose([\n", " transforms.Resize(256),\n", " transforms.CenterCrop(224),\n", " transforms.ToTensor(),\n", "])" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "With the data augmentation functions, we can define our data loaders:\n", "\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "jupyter": { "outputs_hidden": false } }, "outputs": [], "source": [ "path = 'data'\n", "\n", "train_path = os.path.join(path, 'train')\n", "val_path = os.path.join(path, 'val')\n", "test_path = os.path.join(path, 'test')\n", "\n", "train_data = gluon.data.DataLoader(\n", " gluon.data.vision.ImageFolderDataset(train_path).transform_first(transform_train),\n", " batch_size=batch_size, shuffle=True, num_workers=num_workers)\n", "\n", "val_data = gluon.data.DataLoader(\n", " gluon.data.vision.ImageFolderDataset(val_path).transform_first(transform_test),\n", " batch_size=batch_size, shuffle=False, num_workers = num_workers)\n", "\n", "test_data = gluon.data.DataLoader(\n", " gluon.data.vision.ImageFolderDataset(test_path).transform_first(transform_test),\n", " batch_size=batch_size, shuffle=False, num_workers = num_workers)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Note that only ``train_data`` uses ``transform_train``, while\n", "``val_data`` and ``test_data`` use ``transform_test`` to produce deterministic\n", "results for evaluation.\n", "\n", "Model and Trainer\n", "-----------------\n", "\n", "We use a pre-trained [``MobileNet1.0``](https://arxiv.org/pdf/1704.04861.pdf) model, which is useful for mobile and embedded vision applications due to its smaller model size and complexity.\n", "\n", "![alt text](https://3.bp.blogspot.com/-ujGePiv1gZ8/WUBjrgwrPmI/AAAAAAAAB14/zOw9URnrMnIbe7Vv8ftYT4PsnH7S-gJIQCLcBGAs/s1600/image1.png \"MobileNet1.0\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Figure 1. MobileNet use cases. Reprinted from “MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications,” by Andrew G. Howard, Menglong Zhu, Bo Chen, Dmitry Kalenichenko, Weijun Wang, Tobias Weyand, Marco Andreetto, Hartwig Adam, 2017, Retrieved from https://arxiv.org/abs/1704.04861. Copyright 2017 by Google." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "jupyter": { "outputs_hidden": false } }, "outputs": [], "source": [ "model_name = 'MobileNet1.0'" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Here we introduce a common technique in transfer learning: fine-tuning. As shown in the figure below, **fine-tuning** consists of the following steps:\n", "\n", "1. load the pre-trained model (e.g. `MobileNet1.0`)\n", "2. re-define the output layer whose output size is the number of target dataset categories to the target model, and randomly initialize the model parameters of this layer.\n", "3. train the target model on the target dataset.\n", "\n", "![alt text](https://www.d2l.ai/_images/finetune.svg \"Fine tuning\")" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "jupyter": { "outputs_hidden": false } }, "outputs": [], "source": [ "finetune_net = get_model(model_name, pretrained=True)\n", "\n", "with finetune_net.name_scope():\n", " finetune_net.output = nn.Dense(classes)\n", "finetune_net.output.initialize(init.Xavier(), ctx = ctx)\n", "finetune_net.collect_params().reset_ctx(ctx)\n", "finetune_net.hybridize()\n", "\n", "trainer = gluon.Trainer(finetune_net.collect_params(), 'adam', \n", " {'learning_rate': lr})\n", "\n", "metric = mx.metric.Accuracy()\n", "L = gluon.loss.SoftmaxCrossEntropyLoss()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We define a evaluation function for validation and testing." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "jupyter": { "outputs_hidden": false } }, "outputs": [], "source": [ "def test(net, val_data, ctx):\n", " metric = mx.metric.Accuracy()\n", " for i, batch in enumerate(val_data):\n", " data = gluon.utils.split_and_load(batch[0], ctx_list=ctx, batch_axis=0, even_split=False)\n", " label = gluon.utils.split_and_load(batch[1], ctx_list=ctx, batch_axis=0, even_split=False)\n", " outputs = [net(X) for X in data]\n", " metric.update(label, outputs)\n", "\n", " return metric.get()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Training Loop\n", "-------------\n", "\n", "Following is the main training loop." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "jupyter": { "outputs_hidden": false } }, "outputs": [], "source": [ "num_batch = len(train_data)\n", "\n", "for epoch in range(epochs):\n", " tic = time.time()\n", " train_loss = 0\n", " metric.reset()\n", "\n", " for i, batch in enumerate(train_data):\n", " data = gluon.utils.split_and_load(batch[0], ctx_list=ctx, batch_axis=0, even_split=False)\n", " label = gluon.utils.split_and_load(batch[1], ctx_list=ctx, batch_axis=0, even_split=False)\n", " with ag.record():\n", " outputs = [finetune_net(X) for X in data]\n", " loss = [L(yhat, y) for yhat, y in zip(outputs, label)]\n", " for l in loss:\n", " l.backward()\n", "\n", " trainer.step(batch_size)\n", " train_loss += sum([l.mean().asscalar() for l in loss]) / len(loss)\n", "\n", " metric.update(label, outputs)\n", "\n", " _, train_acc = metric.get()\n", " train_loss /= num_batch\n", "\n", " _, val_acc = test(finetune_net, val_data, ctx)\n", "\n", " print('[Epoch %d] Train-acc: %.3f, loss: %.3f | Val-acc: %.3f | time: %.1f' %\n", " (epoch, train_acc, train_loss, val_acc, time.time() - tic))\n", "\n", "_, test_acc = test(finetune_net, test_data, ctx)\n", "print('[Finished] Test-acc: %.3f' % (test_acc))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**NOTE**: If you get `cudaMalloc failed: out of memory` error at the training loop, you can:\n", "1. shutdown the previous Lab session. The Notebook Dashboard has a tab named `Running` that shows all the running notebooks and allows shutting them down (by clicking on a `Shutdown` button).\n", "    \n", "![](./images/jupyter_running.png)\n", "2. try to decrease the `per_device_batch_size` value (e.g. 128)." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Predict with finetuned model\n", "-------------\n", "\n", "We can test the performance using finetuned weights." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "%matplotlib inline\n", "import matplotlib.image as mpimg\n", "import matplotlib.pyplot as plt\n", "\n", "from gluoncv.utils import viz, download" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Let's test with the first picture." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "plt.rcParams['figure.figsize'] = (15, 9)\n", "\n", "url = 'https://cdn.pixabay.com/photo/2019/07/14/12/55/brown-bear-swimming-4337049_960_720.jpg'\n", "\n", "file = download(url, path='.')\n", "img = image.imread(file)\n", "\n", "viz.plot_image(img)\n", "plt.show()" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "transform_fn = transforms.Compose([\n", " transforms.Resize(size=(224, 224)),\n", " transforms.ToTensor(),\n", "])\n", "\n", "img = transform_fn(img)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "ctx = mx.gpu(0)\n", "pred = finetune_net(img.expand_dims(0).as_in_context(ctx))" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "class_names = ['brown', 'no', 'polar']\n", "\n", "topK = 3\n", "ind = nd.topk(pred, k=topK).astype('int')[0]\n", "for i in range(topK):\n", " print('[%s], with probability %.1f%%'%\n", " (class_names[ind[i].asscalar()], nd.softmax(pred)[0][ind[i]].asscalar()*100))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Let's test with another picture." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "url = 'https://cdn.pixabay.com/photo/2016/09/12/17/51/polar-bears-1665367_960_720.jpg'\n", "\n", "file = download(url, path='.')\n", "img = image.imread(file)\n", "\n", "viz.plot_image(img)\n", "plt.show()" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "img = transform_fn(img)\n", "pred = finetune_net(img.expand_dims(0).as_in_context(ctx))\n", "\n", "ind = nd.topk(pred, k=topK).astype('int')[0]\n", "for i in range(topK):\n", " print('[%s], with probability %.1f%%'%\n", " (class_names[ind[i].asscalar()], nd.softmax(pred)[0][ind[i]].asscalar()*100))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "This time let's try a more difficult example." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "url = 'https://cdn.pixabay.com/photo/2015/12/12/14/57/giant-rubber-bear-1089612_960_720.jpg'\n", "\n", "file = download(url, path='.')\n", "img = image.imread(file)\n", "\n", "viz.plot_image(img)\n", "plt.show()" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "img = transform_fn(img)\n", "pred = finetune_net(img.expand_dims(0).as_in_context(ctx))\n", "\n", "ind = nd.topk(pred, k=topK).astype('int')[0]\n", "for i in range(topK):\n", " print('[%s], with probability %.1f%%'%\n", " (class_names[ind[i].asscalar()], nd.softmax(pred)[0][ind[i]].asscalar()*100))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Congratulations! You have built your own object classification model based on a custom dataset." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Converting the trained model for DeepLens deployment" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Now we are saving the trained model into the format which can be deployed to your DeepLens device. Specifically, the model symbol and parameter files needs to be packaged together." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "s3_bucket = 'deeplens-'" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We will name the model as 'mobilenet1.0-bear', and this name will be used within the Lambda function of a DeepLens project package. Here we also add Softmax layer on top of the last layer which is required by Intel OpenVINO." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "model_name = 'mobilenet1.0-bear'\n", "\n", "finetune_net.export(model_name)\n", "net_with_softmax = finetune_net(mx.sym.var('data'))\n", "net_with_softmax = mx.sym.SoftmaxOutput(data=net_with_softmax, name='softmax')\n", "net_with_softmax.save('./{}-symbol.json'.format(model_name))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Let's tar the two files and upload the tar file to Amazon S3 bucket to be refered by DeepLens in the next Lab." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "!tar cvfz ./{model_name}.tar.gz ./{model_name}-*" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "!aws s3 cp {model_name}.tar.gz s3://{s3_bucket}/models/" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] } ], "metadata": { "kernelspec": { "display_name": "conda_mxnet_p36", "language": "python", "name": "conda_mxnet_p36" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.6.5" } }, "nbformat": 4, "nbformat_minor": 4 }