{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Authentication Use Case (1:n Identification)\n", "-----\n", "\n", "Here we are going to take a look at the Authentication use case. Here we simply ask a user to snap a selfie the system will then compare the selfie to registered users. The following diagram details the process. \n", "\n", "\n", "\n", "![Authentication](authentication_diagram.png \"Authentication\")\n", "\n", "1. user initiates an authentication to an application \n", "2. user snaps a selfie which will be used to compare to the drivers license\n", "3. system makes image quality checks\n", "4. system compares the identification image to the selfie image \n", " - if the similarity is above the specified threshold then we say that the faces match \n", " - if the similarity is below the specified threshold then we say that the faces DON'T match\n", " " ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "import io\n", "import boto3\n", "import json\n", "from IPython.display import Image as IImage\n", "import pandas as pd\n", "\n", "%store -r bucket_name\n", "mySession = boto3.session.Session()\n", "aws_region = mySession.region_name\n", "print(\"AWS Region: {}\".format(aws_region))\n", "print(\"AWS Bucket: {}\".format(bucket_name))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Setup Clients \n", "-----\n", "Here we are going to use both S3 and Rekognition apis " ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "s3_client = boto3.client('s3')\n", "rek_client = boto3.client('rekognition')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Setup Collection and Index Faces\n", "\n", "\n", "-----\n", "\n", "Here we'll create a collection and index faces from our image mapping file into a panda's dataframe\n", "\n", "
Next Steps: \n", "- CreateCollection: creates a searchable index of faces \n", "- IndexFaces: index a single image into our collection, index multiple faces into our collection \n", "
\n", "\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "## Name your Collection \n", "collection_name = 'identity-verification-collection'\n", "\n", "try:\n", " response = rek_client.create_collection(\n", " CollectionId=collection_name)\n", "except:\n", " rek_client.delete_collection(\n", " CollectionId=collection_name)\n", " response = rek_client.create_collection(\n", " CollectionId=collection_name)\n", " \n", "response" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Image map\n", "-----\n", "Here we read an excel file of images to index, the image_map data frame contains the external identifier (reference name) and a reference image as well as a sample selfie image. we'll use the selfie to search the collection. " ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# -- read the image map into a pandas dataframe --\n", "obj = s3_client.get_object(Bucket=bucket_name, Key='IDVImageMapping.xlsx')\n", "image_map = pd.read_excel(io.BytesIO(obj['Body'].read()))\n", "image_map.head()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Index lots of faces\n", "----\n", "\n", "Here we are going to index lots of faces from the image map data frame." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "## Index lots of faces \n", "dict_of_faces = image_map[[\"reference_name\",\"reference_image\"]].to_dict('records')\n", "\n", "for rec in dict_of_faces:\n", " \n", " response = rek_client.index_faces(\n", " CollectionId= collection_name,\n", " Image={\n", " 'S3Object': {\n", " 'Bucket': bucket_name,\n", " 'Name': rec[\"reference_image\"],\n", " }\n", " },\n", " ExternalImageId=rec['reference_name'],\n", " DetectionAttributes=[\n", " 'DEFAULT',\n", " ],\n", " MaxFaces=1, # maximum faces detected \n", " QualityFilter='AUTO' # apply the quality filter. \n", " )\n", " face_id = response['FaceRecords'][0]['Face']['FaceId']\n", " print(\"ImageName: {}, FaceID: {}\".format(rec[\"reference_image\"], face_id))\n", " \n", "\n", "print(\"--- indexing complete ---\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Step 2. User Snaps a Selfie \n", "-----\n", "The image below is a sample selfie which we'll use to search the collection with" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "## Image of a Face\n", "selfie_image = \"face_3_0.jpeg\"\n", "display(IImage(url=s3_client.generate_presigned_url('get_object', \n", " Params={'Bucket': bucket_name, \n", " 'Key' : selfie_image})))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Step 3. Image quality checks\n", "----\n", "\n", "Here we want to do some basic checks:\n", "1. that we can detect that there is only one face in the selfie \n", "2. the quality (sharpness and brightness) are sufficient to match with \n", "\n", "Note: we could do several other checks, but we'll see those in module 3.\n", "\n", "\n", "
DetectFaces \n", " \n", "The **DetectFaces** operation that looks for key facial features such as eyes, nose, and mouth to detect faces in an input image. Amazon Rekognition Image detects the 100 largest faces in an image.\n", "\n", "Here we actually will detect two faces in the driver's license. \n", "\n", "
\n", "\n", "
Note \n", " \n", "Take a look at the FaceDetails of each face detected. it will return several helpful details of image quality for each face in the image. \n", "\n", "- BoundingBox\n", "- AgeRange\n", "- Gender\n", "- Landmarks\n", "- Quality\n", "- Pose \n", "\n", "
" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# -- selfie check -- \n", "response = rek_client.detect_faces(Image={'S3Object':{\n", " 'Bucket':bucket_name,\n", " 'Name':selfie_image}},\n", " Attributes=['ALL'])\n", "response" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Step 4. Authentication 1:n Identification with SearchFaceByImage\n", "-----\n", "\n", "For a given input image, first detects the largest face in the image, and then searches the specified collection for matching faces. The operation compares the features of the input face with faces in the specified collection.\n", "\n", "
Note \n", " \n", "\n", "- SimilarityThreshold - The minimum level of confidence in the face matches that a match must meet to be included in the FaceMatches array. \n", "\n", "- QualityFilter - A filter that specifies a quality bar for how much filtering is done to identify faces. Filtered faces aren't compared. If you specify AUTO, Amazon Rekognition chooses the quality bar. If you specify LOW, MEDIUM, or HIGH, filtering removes all faces that do not meet the chosen quality bar. \n", "\n", "
\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "response = rek_client.search_faces_by_image(\n", " CollectionId=collection_name,\n", " Image={\n", " 'S3Object': {\n", " 'Bucket': bucket_name,\n", " 'Name': selfie_image,\n", " }\n", " },\n", " MaxFaces=1,\n", " FaceMatchThreshold=90,\n", " QualityFilter='AUTO'\n", ")\n", "response\n", "\n", "external_image_id = response[\"FaceMatches\"][0][\"Face\"][\"ExternalImageId\"]\n", "similarity_score = response[\"FaceMatches\"][0][\"Similarity\"]\n", "face_id = response[\"FaceMatches\"][0][\"Face\"][\"FaceId\"]\n", "\n", "print(\"-- FaceID : {} \".format(face_id))\n", "print(\"-- Similarity : {} \".format(similarity_score))\n", "print(\"-- ExternalImageID : {} \\n\".format(external_image_id))\n", "\n", "print(json.dumps(response, indent=3)) \n", "\n", " " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 1:n images in a collection?\n", "\n", "what happens when we have more than one image of a user in a collection? lets check it out! \n", "\n", "
Steps \n", " \n", "\n", "- Here we are going to index several images into our collection \n", "- then search the collection with a new selfie \n", "
" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "images_to_index = [\"face_52_0.jpeg\", \"face_52_1.jpeg\", \"face_52_2.jpeg\"]\n", "external_image_id = \"Entity_X_Cannon\"\n", "\n", "for image_name in images_to_index:\n", " \n", " response = rek_client.index_faces(\n", " CollectionId= collection_name,\n", " Image={\n", " 'S3Object': {\n", " 'Bucket': bucket_name,\n", " 'Name': image_name,\n", " }\n", " },\n", " ExternalImageId=external_image_id,\n", " DetectionAttributes=[\n", " 'DEFAULT',\n", " ],\n", " MaxFaces=1, # maximum faces detected \n", " QualityFilter='AUTO' # apply the quality filter. \n", " )\n", " face_id = response['FaceRecords'][0]['Face']['FaceId']\n", " print(\"ImageName: {}, FaceID: {}\".format(image_name, face_id))\n", " \n", "\n", "print(\"indexing complete\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Face's Indexed\n", "-----\n", "Now we should have a total of 4 image in the collection for \"Entity_X_Cannon\". Now the user is going to take a final selfie \"face_52_3.jpeg\" and we'll search search the index. \n", "\n", "
Results \n", " \n", "\n", "- Check out the similarity of each face found, it should range from 99.99 to 99.96 \n", "- Also note the ExternalImageId of each face, they should all match Entity_X_Cannon\n", " \n", "
" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "new_selfie = \"face_52_3.jpeg\"\n", "\n", "response = rek_client.search_faces_by_image(\n", " CollectionId=collection_name,\n", " Image={\n", " 'S3Object': {\n", " 'Bucket': bucket_name,\n", " 'Name': new_selfie,\n", " }\n", " },\n", " MaxFaces=10,\n", " FaceMatchThreshold=90,\n", " QualityFilter='AUTO'\n", ")\n", "\n", "print(\"SearchFacesByImage :\")\n", "for face in response['FaceMatches']:\n", " print(\"-- ExternalImageID : {} \\n\\tSimilarity : {:.4f} \\n\\tFaceID : {}\".format(face['Face']['ExternalImageId'],\n", " face['Similarity'],\n", " face['Face']['FaceId']))\n", "\n", "\n", "\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Search within Collection\n", "----- \n", "\n", "What faces matches another face within the collection? We now have several faces of the same person, lets see what that looks like." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "## -- Snag a FaceID from above --\n", "face_id = \" \" # enter FaceID from above\n", "try:\n", " response = rek_client.search_faces(\n", " CollectionId=collection_name,\n", " FaceId=face_id,\n", " MaxFaces=10\n", " )\n", " print(response)\n", "except:\n", " print(\"--- FaceId not Found ---\")\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Clean up\n", "------\n", "\n", "As part of our cleanup, we can delete our collection. This will delete the collection and all the face vectors contained within." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "try:\n", " rek_client.delete_collection(\n", " CollectionId=collection_name\n", " )\n", "except Exception as err:\n", " print(\"ERROR: {}\".format(err))" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.9.7" }, "vscode": { "interpreter": { "hash": "31f2aee4e71d21fbe5cf8b01ff0e069b9275f58929596ceb00d14d90e3e16cd6" } } }, "nbformat": 4, "nbformat_minor": 4 }