{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Celeribity Recognition using Amazon Rekognition" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "***\n", "This notebook provides a walkthrough of [celebrity recognition API](https://docs.aws.amazon.com/rekognition/latest/dg/celebrities.html) in Amazon Rekognition. You can quickly identify well known people in your video and image libraries to catalog footage and photos for marketing, advertising, and media industry use cases.\n", "***" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Initialize stuff" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# Initialise Notebook\n", "import boto3\n", "from IPython.display import HTML, display, Image as IImage\n", "from PIL import Image, ImageDraw, ImageFont\n", "import time\n", "import os" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# Curent AWS Region. Use this to choose corresponding S3 bucket with sample content\n", "\n", "mySession = boto3.session.Session()\n", "awsRegion = mySession.region_name" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# Init clients\n", "rekognition = boto3.client('rekognition')\n", "s3 = boto3.client('s3')" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# S3 bucket that contains sample images and videos\n", "\n", "# We are providing sample images and videos in this bucket so\n", "# you do not have to manually download/upload test images and videos.\n", "\n", "bucketName = \"aws-workshops-\" + awsRegion" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# Create temporary directory\n", "# This directory is not needed to call Rekognition APIs.\n", "# We will only use this directory to download images from S3 bucket and draw bounding boxes\n", "# around recognized celebrities to show them here in the notebook.\n", "\n", "!mkdir m1tmp\n", "tempFolder = 'm1tmp/'" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Recognize celebrities in image\n", "***" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "imageName = \"content-moderation/media/GrandTourjc.png\"" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "display(IImage(url=s3.generate_presigned_url('get_object', Params={'Bucket': bucketName, 'Key': imageName})))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Call Rekognition to recognize celebrities in the image" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# Call Amazon Rekognition to recognize celebrities in the image\n", "# https://docs.aws.amazon.com/rekognition/latest/dg/API_RecognizeCelebrities.html\n", "\n", "recognizeCelebritiesResponse = rekognition.recognize_celebrities(\n", " Image={\n", " 'S3Object': {\n", " 'Bucket': bucketName,\n", " 'Name': imageName,\n", " }\n", " }\n", ")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Review the raw JSON reponse from Rekognition" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# Show JSON response returned by Rekognition Celebrity Recognition API\n", "# In the JSON response below, you will see CelebrityFaces which contains information about recognized celebrities.\n", "# For each recognized celebrity, you will see information like Name, Id, Urls and additional information about \n", "# their facial attributes.\n", "\n", "display(recognizeCelebritiesResponse)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Show image with bounding boxes around recognized celebrities" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# Define a function that will display image with bounded boxes around recognized celebrites\n", "# We will call this function in next step\n", " \n", "def drawBoundingBoxes (sourceImage, boxes):\n", " # blue, green, red, grey\n", " colors = ((255,255,255),(255,255,255),(76,182,252),(52,194,123))\n", " \n", " # Download image locally\n", " imageLocation = tempFolder+os.path.basename(sourceImage)\n", " s3.download_file(bucketName, sourceImage, imageLocation)\n", "\n", " # Draws BB on Image\n", " bbImage = Image.open(imageLocation)\n", " draw = ImageDraw.Draw(bbImage)\n", " width, height = bbImage.size\n", " col = 0\n", " maxcol = len(colors)\n", " line= 3\n", " for box in boxes:\n", " x1 = int(box[1]['Left'] * width)\n", " y1 = int(box[1]['Top'] * height)\n", " x2 = int(box[1]['Left'] * width + box[1]['Width'] * width)\n", " y2 = int(box[1]['Top'] * height + box[1]['Height'] * height)\n", " \n", " draw.text((x1,y1),box[0],colors[col])\n", " for l in range(line):\n", " draw.rectangle((x1-l,y1-l,x2+l,y2+l),outline=colors[col])\n", " col = (col+1)%maxcol\n", " \n", " imageFormat = \"PNG\"\n", " ext = sourceImage.lower()\n", " if(ext.endswith('jpg') or ext.endswith('jpeg')):\n", " imageFormat = 'JPEG'\n", "\n", " bbImage.save(imageLocation,format=imageFormat)\n", "\n", " display(bbImage)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# Extract bounding box information from JSON response above and display image with bounding boxes around celebrites.\n", "\n", "boxes = []\n", "celebrities = recognizeCelebritiesResponse['CelebrityFaces']\n", "for celebrity in celebrities:\n", " boxes.append ((celebrity['Name'], celebrity['Face']['BoundingBox']))\n", " \n", "drawBoundingBoxes(imageName, boxes)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Recognize celebrities in video\n", " Celebrity recognition in video is an async operation. \n", " https://docs.aws.amazon.com/rekognition/latest/dg/API_StartCelebrityRecognition.html\n", " - We first start celebrity recognition job which returns a Job Id.\n", " - We can then call `get_celebrity_recognition` to get the job status and after job is complete, we can get celebrity metadata.\n", " - In production use cases, you would usually use StepFucntion or SNS topic to get notified when job is complete.\n", "***" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "videoName = \"content-moderation/media/GrandTour720.mp4\"" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Call Rekognition to start a job for celebrity rekognition" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# Start celebrity recognition job\n", "startCelebrityRekognition = rekognition.start_celebrity_recognition(\n", " Video={\n", " 'S3Object': {\n", " 'Bucket': bucketName,\n", " 'Name': videoName,\n", " }\n", " },\n", ")\n", "\n", "celebrityJobId = startCelebrityRekognition['JobId']\n", "display(\"Job Id: {0}\".format(celebrityJobId))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Wait for celebrity rekognition job to complete" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# Wait for celebrity recognition job to complete\n", "# In production use cases, you would usually use StepFucntion or SNS topic to get notified when job is complete.\n", "getCelebrityRecognition = rekognition.get_celebrity_recognition(\n", " JobId=celebrityJobId,\n", " SortBy='TIMESTAMP'\n", ")\n", "\n", "while(getCelebrityRecognition['JobStatus'] == 'IN_PROGRESS'):\n", " time.sleep(5)\n", " print('.', end='')\n", " \n", " getCelebrityRecognition = rekognition.get_celebrity_recognition(\n", " JobId=celebrityJobId,\n", " SortBy='TIMESTAMP')\n", " \n", "display(getCelebrityRecognition['JobStatus'])" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Review raw JSON reponse from Rekognition" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# Show JSON response returned by Rekognition Celebrity Recognition API\n", "# In the JSON response below, you will see list Celebrities which contains information about recognized celebrities.\n", "# For each recognized celebrity, you will see information like Timestamp, Name, Id, Urls\n", "# and additional information about their facial attributes.\n", "\n", "display(getCelebrityRecognition)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Dislpay names of recognized celebrities in the video" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "theCelebs = {}\n", "\n", "# Display timestamps and celebrites detected at that time\n", "strDetail = \"Celebrites detected in video
=======================================
\"\n", "strOverall = \"Celebrities in the overall video:
=======================================
\"\n", "\n", "# Celebrities detected in each frame\n", "for celebrity in getCelebrityRecognition['Celebrities']:\n", " if 'Celebrity' in celebrity :\n", " cconfidence = celebrity[\"Celebrity\"][\"Confidence\"]\n", " if(cconfidence > 95):\n", " ts = celebrity [\"Timestamp\"]\n", " cname = celebrity[\"Celebrity\"][\"Name\"]\n", " strDetail = strDetail + \"At {} ms: {} (Confidence: {})
\".format(ts, cname, round(cconfidence,2))\n", " if not cname in theCelebs:\n", " theCelebs[cname] = cname\n", " \n", "\n", "# Unique faces detected in video\n", "for theCeleb in theCelebs:\n", " strOverall = strOverall + \"Name: {}
\".format(theCeleb)\n", "\n", "# Display results\n", "display(HTML(strOverall))\n", "#display(HTML(strDetail))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Show video in the player" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "scrolled": true }, "outputs": [], "source": [ "# Show video in a player\n", "\n", "s3FilePrefix = \"https://s3.amazonaws.com\"\n", "if(not awsRegion == 'us-east-1'):\n", " s3FilePrefix = \"https://s3-{}.amazonaws.com\".format(awsRegion)\n", "\n", "s3VideoUrl = \"{0}/{1}/{2}\".format(s3FilePrefix, bucketName, videoName)\n", "\n", "videoTag = \"\".format(s3VideoUrl)\n", "\n", "videoui = \"
{}{}
\".format(videoTag, strDetail)\n", "\n", "display(HTML(videoui))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Try recognizing custom celebrities \n", "***" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# Now let us try an image with non-celebrities in the image.\n", "\n", "customCelebrityImageName = \"content-moderation/media/serverless-bytes.png\"" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "display(IImage(url=s3.generate_presigned_url('get_object', Params={'Bucket': bucketName, 'Key': customCelebrityImageName})))" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# Call Amazon Rekognition to recognize celebrities in the image\n", "\n", "customCelebrityResponse = rekognition.recognize_celebrities(\n", " Image={\n", " 'S3Object': {\n", " 'Bucket': bucketName,\n", " 'Name': customCelebrityImageName,\n", " }\n", " }\n", ")" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# Display Rekognition response\n", "# You will see Rekognition return an empty list for CelebrityFaces and \n", "# UnrecognizedFaces list with unrecognized faces that were detected in the image.\n", "# In the next module you will learn how to get custom-celebrity faces recognized.\n", "\n", "display(customCelebrityResponse)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "#Show image and bounded boxes around detected faces\n", "\n", "# Extract BB info from response\n", "cboxes = []\n", "faces = customCelebrityResponse['UnrecognizedFaces']\n", "for face in faces:\n", " cboxes.append (('Unrecognized Face', face['BoundingBox']))\n", " \n", "drawBoundingBoxes(customCelebrityImageName, cboxes)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "***\n", "### References\n", "- https://docs.aws.amazon.com/rekognition/latest/dg/celebrities.html\n", "- https://docs.aws.amazon.com/rekognition/latest/dg/API_RecognizeCelebrities.html\n", "- https://docs.aws.amazon.com/rekognition/latest/dg/API_StartCelebrityRecognition.html\n", "- https://docs.aws.amazon.com/rekognition/latest/dg/API_GetCelebrityRecognition.html\n", "\n", "***" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "You have successfully used Amazon Rekognition to identify celebrities in images an videos. In the next module, Recognize Custom Celebrities, you will learn how to recognize your custom celebrities in the images and videos." ] } ], "metadata": { "kernelspec": { "display_name": "conda_python3", "language": "python", "name": "conda_python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.6.5" } }, "nbformat": 4, "nbformat_minor": 2 }