Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
SPDX-License-Identifier: CC-BY-SA-4.0

If you deployed a model to Amazon SageMaker hosting services in Step 6.1: Deploy the Model to Amazon SageMaker Hosting Services, you now have an endpoint that you can invoke to get inferences in real time. To validate the model, invoke the endpoint with example images from the test dataset and check whether the inferences you get match the actual labels of the images.

Topics + Validate a Model Deployed to Amazon SageMaker Hosting Services (Amazon SageMaker Python SDK) + Validate a Model Deployed to Amazon SageMaker Hosting Services (AWS SDK for Python (Boto 3))

To validate the model by using the Amazon SageMaker Python SDK, use the sagemaker.predictor.RealTimePredictor object that you created in Deploy the Model to Amazon SageMaker Hosting Services (Amazon SageMaker Python SDK). For information, see https://sagemaker.readthedocs.io/en/stable/predictors.html#sagemaker.predictor.RealTimePredictor.

To validate the model (Amazon SageMaker Python SDK)

1. Download the test data from Amazon S3.

   s3 = boto3.resource('s3')
   
   test_key = "{}/test/examples".format(prefix)
   
   s3.Bucket(bucket).download_file(test_key, 'test_data')

2. Plot the first 10 images from the test dataset with their labels.

   %matplotlib inline
   
   for i in range (0, 10):
       img = test_set[0][i]
       label = test_set[1][i]
       img_reshape = img.reshape((28,28))
       imgplot = plt.imshow(img_reshape, cmap='gray')
       print('This is a {}'.format(label))
       plt.show()

   

3. To get inferences for the first 10 examples in the test dataset, call the predict method of the sagemaker.predictor.RealTimePredictor object.

   with open('test_data', 'r') as f:
       for j in range(0,10):
           single_test = f.readline()
           result = xgb_predictor.predict(single_test)
           print(result)

   To see if the model is making accurate predictions, check the output from this step against the numbers that you plotted in the previous step.

You have now trained, deployed, and validated your first model in Amazon SageMaker.

Next Step
Step 8: Clean Up

To use the AWS SDK for Python (Boto 3) to validate the model, call the invoke_endpoint method. This method corresponds to the InvokeEndpoint API provided by the Amazon SageMaker runtime.

To validate the model (AWS SDK for Python (Boto 3))

1. Download the test data from Amazon S3.

   s3 = boto3.resource('s3')
   
   test_key = "{}/test/examples".format(prefix)
   
   s3.Bucket(bucket).download_file(test_key, 'test_data')

2. Plot the first 10 images from the test dataset with their labels.

   %matplotlib inline
   
   for i in range (0, 10):
       img = test_set[0][i]
       label = test_set[1][i]
       img_reshape = img.reshape((28,28))
       imgplot = plt.imshow(img_reshape, cmap='gray')
       print('This is a {}'.format(label))
       plt.show()

   

3. Get the Amazon SageMaker runtime client, which provides the invoke_endpoint method.

   runtime_client = boto3.client('runtime.sagemaker')

4. Get inferences from the first 10 examples in the test dataset by calling invoke_endpoint.

   with open('test_data', 'r') as f:
   
       for i in range(0,10):
           single_test = f.readline()
           response = runtime_client.invoke_endpoint(EndpointName = endpoint_name,
                                            ContentType = 'text/csv',
                                            Body = single_test)
           result = response['Body'].read().decode('ascii')
           print('Predicted label is {}.'.format(result))

5. To see if the model is making accurate predictions, check the output from this step against the numbers you plotted in the previous step.

You have now trained, deployed, and validated your first model in Amazon SageMaker.

Next Step
Step 8: Clean Up