{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Face search using Amazon Rekognition" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "***\n", "This notebook provides a walkthrough of [compare faces API]( https://docs.aws.amazon.com/rekognition/latest/APIReference/API_CompareFaces.html) and [face search API](https://docs.aws.amazon.com/rekognition/latest/APIReference/API_SearchFaces.html) in Amazon Rekognition to detect known faces.\n", "***" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "scrolled": true }, "outputs": [], "source": [ "# Initialize dependencies\n", "import boto3\n", "import botocore\n", "from IPython.display import HTML, display, Image as IImage\n", "import time\n", "\n", "# Initialize clients\n", "REGION = boto3.session.Session().region_name\n", "rekognition = boto3.client('rekognition', REGION)\n", "s3 = boto3.client('s3', REGION)\n", "\n", "%store -r bucket_name" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Detect known faces in image\n", "\n", "There are two main ways to detect known faces by using Amazon Rekognition.\n", "\n", "\n", "1. The first is by using [the compare faces API](https://docs.aws.amazon.com/rekognition/latest/dg/faces-comparefaces.html) to compare two images.\n", "2. The second is by creating a collection of known images using the [index faces API and then using the search faces API](https://docs.aws.amazon.com/rekognition/latest/dg/collections.html) to detect all the faces belonging to a particular collection on an image or video.\n", "***\n", "\n", "### 1. Call Rekognition to compare two faces\n", "\n", "" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# Show images\n", "source = \"media/identity.jpg\" # image profile\n", "target = \"media/looking_at_screen.jpg\" # screenshot\n", "\n", "display(IImage(url=s3.generate_presigned_url('get_object', Params={'Bucket': bucket_name, 'Key': source})))\n", "display(IImage(url=s3.generate_presigned_url('get_object', Params={'Bucket': bucket_name, 'Key': target})))" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "compare_faces_response = rekognition.compare_faces(\n", " SourceImage={\n", " 'S3Object': {\n", " 'Bucket': bucket_name,\n", " 'Name': source\n", " }\n", " },\n", " TargetImage={\n", " 'S3Object': {\n", " 'Bucket': bucket_name,\n", " 'Name': target\n", " } \n", " })" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Review the raw JSON reponse from Rekognition\n", "\n", "In the JSON response below, you will see detected faces, confidence score and additional information." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "print(\"Matched Faces: {}\".format(len(compare_faces_response['FaceMatches'])))\n", "display(compare_faces_response)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### 2. Use Index Faces and Search Faces\n", "\n", "Now let's detect faces using the Search functionality. \n", "\n", "#### Call Rekognition to create a new collection\n", "\n", "" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "collection_id = bucket_name\n", "\n", "# Create collection unless it already exsists\n", "try:\n", " rekognition.describe_collection(CollectionId=collection_id)\n", " print(\"Collection already exists\")\n", "except botocore.exceptions.ClientError as e:\n", " rekognition.create_collection(CollectionId=collection_id)\n", " print(\"Collection created\")" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# Show image\n", "image_name = \"media/looking_at_screen.jpg\"\n", "display(IImage(url=s3.generate_presigned_url('get_object', Params={'Bucket': bucket_name, 'Key': image_name})))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Call Rekognition to search faces in the collection\n", "\n", "By making this call, unsurpsingly we'll get no matches the first time, because no image has been indexed yet.\n", "\n", "" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "search_faces_response = rekognition.search_faces_by_image(\n", " CollectionId=collection_id,\n", " Image={\n", " 'S3Object': {\n", " 'Bucket': bucket_name,\n", " 'Name': image_name,\n", " }\n", " }\n", ")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Review the raw JSON reponse from Rekognition\n", "\n", "In the JSON response below, you will see detected faces, confidence score and additional information." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "print(\"Matched Faces: {}\".format(len(search_faces_response['FaceMatches'])))\n", "display(search_faces_response)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Index Face in the Collection\n", "\n", "Now let's call Amazon Rekognition to create a new identity for the given collection.\n", "" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "index_faces_response = rekognition.index_faces(\n", " CollectionId=collection_id,\n", " Image={\n", " 'S3Object': {\n", " 'Bucket': bucket_name,\n", " 'Name': 'media/identity.jpg',\n", " }\n", " }\n", ")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Review the raw JSON reponse from Rekognition\n", "\n", "In the JSON response below, you will see an ID identifying the newly created identity called FaceId.\n", "You can persist that ID in a Database and associate it with a Name for a future lookup." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "identities = {}\n", "identities[index_faces_response['FaceRecords'][0]['Face']['FaceId']] = 'John Doe'\n", "display(index_faces_response)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Call Rekognition again to search faces in the collection\n", "\n", "This time, the search should match the indexed identity.\n", "" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "search_faces_response = rekognition.search_faces_by_image(\n", " CollectionId=collection_id,\n", " Image={\n", " 'S3Object': {\n", " 'Bucket': bucket_name,\n", " 'Name': image_name,\n", " }\n", " }\n", ")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Review the raw JSON reponse from Rekognition\n", "\n", "In the JSON response below, you will see detected faces, confidence score and additional information." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "print(\"Matched Faces: {}\".format(len(search_faces_response['FaceMatches'])))\n", "for match in search_faces_response['FaceMatches']:\n", " print(\"Detected {} with Confidence: {}\".format(identities[match['Face']['FaceId']], match['Face']['Confidence']))\n", "\n", "display(search_faces_response)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Search faces in video\n", "Search recognition in video is an async operation. \n", "https://docs.aws.amazon.com/rekognition/latest/dg/procedure-person-search-videos.html. \n", "\n", "- First we start a search detection job which returns a Job Id.\n", "- We can then call `get_search_detection` to get the job status and after job is complete, we can get face metadata.\n", "- In production use cases, you would usually use StepFunction or SNS topic to get notified when job is complete.\n", "***" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "video_name = \"media/leaving.mp4\"" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Show video in the player" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "s3_video_url = s3.generate_presigned_url('get_object', Params={'Bucket': bucket_name, 'Key': video_name})\n", "\n", "video_tag = \"\".format(s3_video_url)\n", "video_ui = \"
{}
\".format(video_tag)\n", "display(HTML(video_ui))\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Call Rekognition to start a job for face search" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "start_search_detection = rekognition.start_face_search(\n", " CollectionId=collection_id,\n", " Video={\n", " 'S3Object': {\n", " 'Bucket': bucket_name,\n", " 'Name': video_name,\n", " }\n", " },\n", ")\n", "\n", "search_job_id = start_search_detection['JobId']\n", "display(\"Job Id: {0}\".format(search_job_id))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Additional (Optional) Request Attributes\n", "\n", "ClientRequestToken, FaceMatchThreshold, JobTag, NotificationChannel:\n", "https://docs.aws.amazon.com/rekognition/latest/APIReference/API_StartFaceSearch.html\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Wait for object detection job to complete\n", "\n", "In production use cases, you would usually use StepFunction or SNS topic to get notified when job is complete." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "get_search_detection = rekognition.get_face_search(\n", " JobId=search_job_id\n", ")\n", "\n", "while(get_search_detection['JobStatus'] == 'IN_PROGRESS'):\n", " time.sleep(5)\n", " print('.', end='')\n", " \n", " get_search_detection = rekognition.get_face_search(\n", " JobId=search_job_id)\n", " \n", "display(get_search_detection['JobStatus'])" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Review raw JSON reponse from Rekognition\n", "\n", "In the JSON response below, you will see list of detected faces and attributes.\n", "For each detected face, you will see information like Timestamp." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "scrolled": true }, "outputs": [], "source": [ "display(get_search_detection)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Display face detected by timestamp and alert when faces are not detected" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# Faces detected in each frame\n", "prev_ts = 0\n", "threshold = 1000 # ms\n", "for person in get_search_detection['Persons']:\n", " ts = person[\"Timestamp\"]\n", " if ts-prev_ts>threshold:\n", " print(\"ALERT - no face matched for {} seconds\".format((ts-prev_ts)/1000))\n", " for match in person[\"FaceMatches\"]:\n", " confidence = match[\"Face\"][\"Confidence\"]\n", " identity = identities[match[\"Face\"][\"FaceId\"]]\n", " print(\"Detected {} on Timestamp: {} with confidence: {}\".format(identity, ts, confidence))\n", " prev_ts = ts" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Cleanup resources\n", "\n", "Let's call Rekognition to delete the Collection.\n", "" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "rekognition.delete_collection(CollectionId=collection_id)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "***\n", "### References\n", "- https://docs.aws.amazon.com/rekognition/latest/APIReference/API_CreateCollection.html\n", "- https://docs.aws.amazon.com/rekognition/latest/APIReference/API_IndexFaces.html\n", "- https://docs.aws.amazon.com/rekognition/latest/APIReference/API_SearchFaces.html\n", "- https://docs.aws.amazon.com/rekognition/latest/APIReference/API_StartFaceSearch.html\n", "- https://docs.aws.amazon.com/rekognition/latest/APIReference/API_GetLabelDetection.html\n", "- https://docs.aws.amazon.com/rekognition/latest/APIReference/API_GetFaceSearch.html\n", "***" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "You have successfully used Amazon Rekognition to search for known faces in images and videos." ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.7.5" } }, "nbformat": 4, "nbformat_minor": 4 }