{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Distributed Scikit Learn in SageMaker with Magic" ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "\u001B[0;31mDocstring:\u001B[0m\n", "::\n", "\n", " %sklearn [--estimator_name ESTIMATOR_NAME] [--entry_point ENTRY_POINT]\n", " [--source_dir SOURCE_DIR] [--role ROLE]\n", " [--framework_version FRAMEWORK_VERSION]\n", " [--py_version PY_VERSION] [--instance_type INSTANCE_TYPE]\n", " [--instance_count INSTANCE_COUNT] [--output_path OUTPUT_PATH]\n", " [--hyperparameters FOO:1,BAR:0.555,BAZ:ABC | 'FOO : 1, BAR : 0.555, BAZ : ABC']\n", " [--channel_training CHANNEL_TRAINING]\n", " [--channel_testing CHANNEL_TESTING]\n", " [--use_spot_instances [USE_SPOT_INSTANCES]]\n", " [--max_wait MAX_WAIT]\n", " [--enable_sagemaker_metrics [ENABLE_SAGEMAKER_METRICS]]\n", " [--metric_definitions ['Name: loss, Regex: Loss = .*?);' ['Name: loss, Regex: Loss = (.*?;' ...]]]\n", " [--name_contains NAME_CONTAINS] [--max_result MAX_RESULT]\n", " {submit,list,status,logs,delete,show_defaults}\n", "\n", "SKLearn magic command.\n", "\n", "methods:\n", " {submit,list,status,logs,delete,show_defaults}\n", "\n", "submit:\n", " --estimator_name ESTIMATOR_NAME\n", " estimator shell variable name\n", " --entry_point ENTRY_POINT\n", " notebook local code file\n", " --source_dir SOURCE_DIR\n", " Path (absolute, relative or an S3 URI) to a directory\n", " with any other training source code dependencies aside\n", " from the entry point file (default: None). If\n", " source_dir is an S3 URI, it must point to a tar.gz\n", " file. Structure within this directory are preserved\n", " when training on Amazon SageMaker.\n", " --role ROLE An AWS IAM role (either name or full ARN). The Amazon\n", " SageMaker training jobs and APIs that create Amazon\n", " SageMaker endpoints use this role to access training\n", " data and model artifacts. After the endpoint is\n", " created, the inference code might use the IAM role, if\n", " it needs to access an AWS resource.\n", " --framework_version FRAMEWORK_VERSION\n", " Scikit-learn version you want to use for executing\n", " your model training code. List of supported versions:\n", " https://github.com/aws/sagemaker-python-sdk#sklearn-\n", " sagemaker-estimators\n", " --py_version PY_VERSION\n", " Python version\n", " --instance_type INSTANCE_TYPE\n", " Type of EC2 instance to use for training, for example,\n", " ‘ml.c4.xlarge’.\n", " --instance_count INSTANCE_COUNT\n", " Number of Amazon EC2 instances to use for training.\n", " --output_path OUTPUT_PATH\n", " S3 location for saving the training result (model\n", " artifacts and output files). If not specified, results\n", " are stored to a default bucket. If the bucket with the\n", " specific name does not exist, the estimator creates\n", " the bucket during the fit() method execution.\n", " --hyperparameters \n", " Hyperparameters are passed to your script as arguments\n", " and can be retrieved with an argparse.\n", " --channel_training CHANNEL_TRAINING\n", " A string that represents the path to the directory\n", " that contains the input data for the training channel.\n", " --channel_testing CHANNEL_TESTING\n", " A string that represents the path to the directory\n", " that contains the input data for the testing channel.\n", "\n", "submit-spot:\n", " --use_spot_instances <[USE_SPOT_INSTANCES]>\n", " Specifies whether to use SageMaker Managed Spot\n", " instances for training. If enabled then the max_wait\n", " arg should also be set. More information:\n", " https://docs.aws.amazon.com/sagemaker/latest/dg/model-\n", " managed-spot-training.html\n", " --max_wait MAX_WAIT Timeout in seconds waiting for spot training instances\n", " (default: None). After this amount of time Amazon\n", " SageMaker will stop waiting for Spot instances to\n", " become available (default: None).\n", "\n", "submit-metrics:\n", " --enable_sagemaker_metrics <[ENABLE_SAGEMAKER_METRICS]>\n", " Enables SageMaker Metrics Time Series. For more\n", " information see: https://docs.aws.amazon.com/sagemaker\n", " /latest/dg/API_AlgorithmSpecification.html# SageMaker-\n", " Type-AlgorithmSpecification-\n", " EnableSageMakerMetricsTimeSeries\n", " --metric_definitions <['Name: loss, Regex: Loss = (.*?);' ['Name: loss, Regex: Loss = (.*?);' ...]]>\n", " A list of dictionaries that defines the metric(s) used\n", " to evaluate the training jobs. Each dictionary\n", " contains two keys: ‘Name’ for the name of the metric,\n", " and ‘Regex’ for the regular expression used to extract\n", " the metric from the logs. This should be defined only\n", " for jobs that don’t use an Amazon algorithm.\n", "\n", "list:\n", " --name_contains NAME_CONTAINS\n", " --max_result MAX_RESULT\n", "\u001B[0;31mFile:\u001B[0m /opt/conda/lib/python3.8/site-packages/sage_maker_kernel-0.1-py3.8.egg/sage_maker_kernel/kernelmagics.py\n" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "%%sklearn?" ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "defaults:\n", " {\n", " \"framework_version\": \"0.23-1\",\n", " \"instance_count\": 1,\n", " \"instance_type\": \"ml.c4.xlarge\",\n", " \"max_wait\": 86400,\n", " \"py_version\": \"py3\",\n", " \"use_spot_instances\": true\n", "}\n", "null\n" ] } ], "source": [ "%sklearn show_defaults" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Upload the data for training \n", "\n", "When training large models with huge amounts of data, you'll typically use big data tools, like Amazon Athena, AWS Glue, or Amazon EMR, to create your data in S3. For the purposes of this example, we're using a sample of the classic [Iris dataset](https://en.wikipedia.org/wiki/Iris_flower_data_set), which is included with Scikit-learn. We will load the dataset, write locally, then write the dataset to s3 to use.\n" ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [], "source": [ "import numpy as np\n", "import os\n", "from sklearn import datasets\n", "\n", "# Load Iris dataset, then join labels and features\n", "iris = datasets.load_iris()\n", "joined_iris = np.insert(iris.data, 0, iris.target, axis=1)\n", "\n", "# Create directory and write csv\n", "os.makedirs('./data', exist_ok=True)\n", "np.savetxt('./data/iris.csv', joined_iris, delimiter=',', fmt='%1.1f, %1.3f, %1.3f, %1.3f, %1.3f')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Lets create our Sagemaker session and create a S3 prefix to use for the notebook example.\n", "Once we have the data locally, we can use use the tools provided by the SageMaker Python SDK to upload the data to a default bucket. " ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "s3://sagemaker-eu-west-1-245582572290/Scikit-iris/data\n" ] } ], "source": [ "# S3 prefix\n", "prefix = 'Scikit-iris'\n", "\n", "import sagemaker\n", "from sagemaker import get_execution_role\n", "\n", "sagemaker_session = sagemaker.Session()\n", "\n", "WORK_DIRECTORY = 'data'\n", "\n", "train_input = sagemaker_session.upload_data(WORK_DIRECTORY, key_prefix=\"{}/{}\".format(prefix, WORK_DIRECTORY) )\n", "print(train_input)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Write the Scikit Learn script\n", "\n", "The source for a traning script is in the cell below. The cell uses the `%%sklearn submit` directive to submit python application from cell to Scikit Learn Estimator. " ] }, { "cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "Couldn't call 'get_role' to get Role ARN from role name workshop-sagemaker to get Role path.\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "submit:\n", " {\n", " \"channel_testing\": \"s3://sagemaker-eu-west-1-245582572290/Scikit-iris/data\",\n", " \"channel_training\": \"s3://sagemaker-eu-west-1-245582572290/Scikit-iris/data\",\n", " \"entry_point\": \"/tmp/tmp-ab309e54-b484-48ab-89eb-2850dcc5b4fc.py\",\n", " \"estimator_name\": \"___SKLearn_estimator\",\n", " \"framework_version\": \"0.23-1\",\n", " \"hyperparameters\": {\n", " \"max_leaf_nodes\": \"30\"\n", " },\n", " \"instance_count\": 1,\n", " \"instance_type\": \"ml.c4.xlarge\",\n", " \"max_result\": 10,\n", " \"max_wait\": 86400,\n", " \"name_contains\": \"sklearn\",\n", " \"py_version\": \"py3\",\n", " \"role\": \"arn:aws:iam::245582572290:role/workshop-sagemaker\",\n", " \"use_spot_instances\": true\n", "}\n", "{\n", " \"___SKLearn_latest_job_name\": \"sagemaker-scikit-learn-2021-03-02-15-04-35-077\",\n", " \"estimator_variable\": \"___SKLearn_estimator\"\n", "}\n" ] } ], "source": [ "%%sklearn submit --channel_training s3://sagemaker-eu-west-1-245582572290/Scikit-iris/data --channel_testing s3://sagemaker-eu-west-1-245582572290/Scikit-iris/data --hyperparameters 'max_leaf_nodes: 30' \n", "\n", "# Copyright 2018 Amazon.com, Inc. or its affiliates. All Rights Reserved.\n", "# \n", "# Licensed under the Apache License, Version 2.0 (the \"License\").\n", "# You may not use this file except in compliance with the License.\n", "# A copy of the License is located at\n", "# \n", "# http://www.apache.org/licenses/LICENSE-2.0\n", "# \n", "# or in the \"license\" file accompanying this file. This file is distributed \n", "# on an \"AS IS\" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either \n", "# express or implied. See the License for the specific language governing \n", "# permissions and limitations under the License.\n", "\n", "from __future__ import print_function\n", "\n", "import argparse\n", "import joblib\n", "import os\n", "import pandas as pd\n", "\n", "from sklearn import tree\n", "\n", "\n", "if __name__ == '__main__':\n", " parser = argparse.ArgumentParser()\n", "\n", " # Hyperparameters are described here. In this simple example we are just including one hyperparameter.\n", " parser.add_argument('--max_leaf_nodes', type=int, default=-1)\n", "\n", " # Sagemaker specific arguments. Defaults are set in the environment variables.\n", " parser.add_argument('--output-data-dir', type=str, default=os.environ['SM_OUTPUT_DATA_DIR'])\n", " parser.add_argument('--model-dir', type=str, default=os.environ['SM_MODEL_DIR'])\n", " parser.add_argument('--train', type=str, default=os.environ['SM_CHANNEL_TRAINING'])\n", " parser.add_argument('--test', type=str, default=os.environ['SM_CHANNEL_TESTING'])\n", "\n", " args = parser.parse_args()\n", "\n", " # Take the set of files and read them all into a single pandas dataframe\n", " input_files = [ os.path.join(args.train, file) for file in os.listdir(args.train) ]\n", " if len(input_files) == 0:\n", " raise ValueError(('There are no files in {}.\\n' +\n", " 'This usually indicates that the channel ({}) was incorrectly specified,\\n' +\n", " 'the data specification in S3 was incorrectly specified or the role specified\\n' +\n", " 'does not have permission to access the data.').format(args.train, \"train\"))\n", " raw_data = [ pd.read_csv(file, header=None, engine=\"python\") for file in input_files ]\n", " train_data = pd.concat(raw_data)\n", "\n", " # labels are in the first column\n", " train_y = train_data.iloc[:, 0]\n", " train_X = train_data.iloc[:, 1:]\n", "\n", " # Here we support a single hyperparameter, 'max_leaf_nodes'. Note that you can add as many\n", " # as your training my require in the ArgumentParser above.\n", " max_leaf_nodes = args.max_leaf_nodes\n", "\n", " # Now use scikit-learn's decision tree classifier to train the model.\n", " clf = tree.DecisionTreeClassifier(max_leaf_nodes=max_leaf_nodes)\n", " clf = clf.fit(train_X, train_y)\n", "\n", " # Print the coefficients of the trained classifier, and save the coefficients\n", " joblib.dump(clf, os.path.join(args.model_dir, \"model.joblib\"))\n", "\n", "\n", "def model_fn(model_dir):\n", " \"\"\"Deserialized and return fitted model\n", " \n", " Note that this should have the same name as the serialized model in the main method\n", " \"\"\"\n", " clf = joblib.load(os.path.join(model_dir, \"model.joblib\"))\n", " return clf\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Stop latest traning Job" ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "{\n", " \"AlgorithmSpecification\": {\n", " \"EnableSageMakerMetricsTimeSeries\": false,\n", " \"TrainingImage\": \"141502667606.dkr.ecr.eu-west-1.amazonaws.com/sagemaker-scikit-learn:0.23-1-cpu-py3\",\n", " \"TrainingInputMode\": \"File\"\n", " },\n", " \"CreationTime\": \"2021-03-02 14:58:22.381000+00:00\",\n", " \"DebugHookConfig\": {\n", " \"CollectionConfigurations\": [],\n", " \"S3OutputPath\": \"s3://sagemaker-eu-west-1-245582572290/\"\n", " },\n", " \"EnableInterContainerTrafficEncryption\": false,\n", " \"EnableManagedSpotTraining\": true,\n", " \"EnableNetworkIsolation\": false,\n", " \"HyperParameters\": {\n", " \"max_leaf_nodes\": \"\\\"30\\\"\",\n", " \"sagemaker_container_log_level\": \"20\",\n", " \"sagemaker_job_name\": \"\\\"sagemaker-scikit-learn-2021-03-02-14-58-21-806\\\"\",\n", " \"sagemaker_program\": \"\\\"tmp-d1536a9b-a65a-4c82-a0a5-f8b43bdf7775.py\\\"\",\n", " \"sagemaker_region\": \"\\\"eu-west-1\\\"\",\n", " \"sagemaker_submit_directory\": \"\\\"s3://sagemaker-eu-west-1-245582572290/sagemaker-scikit-learn-2021-03-02-14-58-21-806/source/sourcedir.tar.gz\\\"\"\n", " },\n", " \"InputDataConfig\": [\n", " {\n", " \"ChannelName\": \"training\",\n", " \"CompressionType\": \"None\",\n", " \"DataSource\": {\n", " \"S3DataSource\": {\n", " \"S3DataDistributionType\": \"FullyReplicated\",\n", " \"S3DataType\": \"S3Prefix\",\n", " \"S3Uri\": \"s3://sagemaker-eu-west-1-245582572290/Scikit-iris/data\"\n", " }\n", " },\n", " \"RecordWrapperType\": \"None\"\n", " },\n", " {\n", " \"ChannelName\": \"testing\",\n", " \"CompressionType\": \"None\",\n", " \"DataSource\": {\n", " \"S3DataSource\": {\n", " \"S3DataDistributionType\": \"FullyReplicated\",\n", " \"S3DataType\": \"S3Prefix\",\n", " \"S3Uri\": \"s3://sagemaker-eu-west-1-245582572290/Scikit-iris/data\"\n", " }\n", " },\n", " \"RecordWrapperType\": \"None\"\n", " }\n", " ],\n", " \"LastModifiedTime\": \"2021-03-02 14:58:34.650000+00:00\",\n", " \"OutputDataConfig\": {\n", " \"KmsKeyId\": \"\",\n", " \"S3OutputPath\": \"s3://sagemaker-eu-west-1-245582572290/\"\n", " },\n", " \"ProfilerConfig\": {\n", " \"ProfilingIntervalInMilliseconds\": 500,\n", " \"S3OutputPath\": \"s3://sagemaker-eu-west-1-245582572290/\"\n", " },\n", " \"ProfilerRuleConfigurations\": [\n", " {\n", " \"RuleConfigurationName\": \"ProfilerReport-1614697102\",\n", " \"RuleEvaluatorImage\": \"929884845733.dkr.ecr.eu-west-1.amazonaws.com/sagemaker-debugger-rules:latest\",\n", " \"RuleParameters\": {\n", " \"rule_to_invoke\": \"ProfilerReport\"\n", " },\n", " \"VolumeSizeInGB\": 0\n", " }\n", " ],\n", " \"ProfilerRuleEvaluationStatuses\": [\n", " {\n", " \"LastModifiedTime\": \"2021-03-02 14:58:23.489000+00:00\",\n", " \"RuleConfigurationName\": \"ProfilerReport-1614697102\",\n", " \"RuleEvaluationStatus\": \"InProgress\"\n", " }\n", " ],\n", " \"ProfilingStatus\": \"Enabled\",\n", " \"ResourceConfig\": {\n", " \"InstanceCount\": 1,\n", " \"InstanceType\": \"ml.c4.xlarge\",\n", " \"VolumeSizeInGB\": 30\n", " },\n", " \"ResponseMetadata\": {\n", " \"HTTPHeaders\": {\n", " \"content-length\": \"3111\",\n", " \"content-type\": \"application/x-amz-json-1.1\",\n", " \"date\": \"Tue, 02 Mar 2021 14:58:34 GMT\",\n", " \"x-amzn-requestid\": \"bda2faa9-56b4-41d4-b8dd-2523085495e4\"\n", " },\n", " \"HTTPStatusCode\": 200,\n", " \"RequestId\": \"bda2faa9-56b4-41d4-b8dd-2523085495e4\",\n", " \"RetryAttempts\": 0\n", " },\n", " \"RoleArn\": \"arn:aws:iam::245582572290:role/workshop-sagemaker\",\n", " \"SecondaryStatus\": \"Starting\",\n", " \"SecondaryStatusTransitions\": [\n", " {\n", " \"StartTime\": \"2021-03-02 14:58:22.381000+00:00\",\n", " \"Status\": \"Starting\",\n", " \"StatusMessage\": \"Launching requested ML instances\"\n", " }\n", " ],\n", " \"StoppingCondition\": {\n", " \"MaxRuntimeInSeconds\": 86400,\n", " \"MaxWaitTimeInSeconds\": 86400\n", " },\n", " \"TrainingJobArn\": \"arn:aws:sagemaker:eu-west-1:245582572290:training-job/sagemaker-scikit-learn-2021-03-02-14-58-21-806\",\n", " \"TrainingJobName\": \"sagemaker-scikit-learn-2021-03-02-14-58-21-806\",\n", " \"TrainingJobStatus\": \"Stopping\"\n", "}\n" ] } ], "source": [ "%sklearn delete" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Describe latest traning Job" ] }, { "cell_type": "code", "execution_count": 9, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "{\n", " \"AlgorithmSpecification\": {\n", " \"EnableSageMakerMetricsTimeSeries\": false,\n", " \"TrainingImage\": \"141502667606.dkr.ecr.eu-west-1.amazonaws.com/sagemaker-scikit-learn:0.23-1-cpu-py3\",\n", " \"TrainingInputMode\": \"File\"\n", " },\n", " \"CreationTime\": \"2021-03-02 14:43:54.283000+00:00\",\n", " \"DebugHookConfig\": {\n", " \"CollectionConfigurations\": [],\n", " \"S3OutputPath\": \"s3://sagemaker-eu-west-1-245582572290/\"\n", " },\n", " \"EnableInterContainerTrafficEncryption\": false,\n", " \"EnableManagedSpotTraining\": true,\n", " \"EnableNetworkIsolation\": false,\n", " \"HyperParameters\": {\n", " \"max_leaf_nodes\": \"\\\"30\\\"\",\n", " \"sagemaker_container_log_level\": \"20\",\n", " \"sagemaker_job_name\": \"\\\"sagemaker-scikit-learn-2021-03-02-14-43-53-685\\\"\",\n", " \"sagemaker_program\": \"\\\"tmp-9d940cb7-23ce-418d-b401-4783563fd641.py\\\"\",\n", " \"sagemaker_region\": \"\\\"eu-west-1\\\"\",\n", " \"sagemaker_submit_directory\": \"\\\"s3://sagemaker-eu-west-1-245582572290/sagemaker-scikit-learn-2021-03-02-14-43-53-685/source/sourcedir.tar.gz\\\"\"\n", " },\n", " \"InputDataConfig\": [\n", " {\n", " \"ChannelName\": \"training\",\n", " \"CompressionType\": \"None\",\n", " \"DataSource\": {\n", " \"S3DataSource\": {\n", " \"S3DataDistributionType\": \"FullyReplicated\",\n", " \"S3DataType\": \"S3Prefix\",\n", " \"S3Uri\": \"s3://sagemaker-eu-west-1-245582572290/Scikit-iris/data\"\n", " }\n", " },\n", " \"RecordWrapperType\": \"None\"\n", " },\n", " {\n", " \"ChannelName\": \"testing\",\n", " \"CompressionType\": \"None\",\n", " \"DataSource\": {\n", " \"S3DataSource\": {\n", " \"S3DataDistributionType\": \"FullyReplicated\",\n", " \"S3DataType\": \"S3Prefix\",\n", " \"S3Uri\": \"s3://sagemaker-eu-west-1-245582572290/Scikit-iris/data\"\n", " }\n", " },\n", " \"RecordWrapperType\": \"None\"\n", " }\n", " ],\n", " \"LastModifiedTime\": \"2021-03-02 14:44:17.681000+00:00\",\n", " \"OutputDataConfig\": {\n", " \"KmsKeyId\": \"\",\n", " \"S3OutputPath\": \"s3://sagemaker-eu-west-1-245582572290/\"\n", " },\n", " \"ProfilerConfig\": {\n", " \"ProfilingIntervalInMilliseconds\": 500,\n", " \"S3OutputPath\": \"s3://sagemaker-eu-west-1-245582572290/\"\n", " },\n", " \"ProfilerRuleConfigurations\": [\n", " {\n", " \"RuleConfigurationName\": \"ProfilerReport-1614696233\",\n", " \"RuleEvaluatorImage\": \"929884845733.dkr.ecr.eu-west-1.amazonaws.com/sagemaker-debugger-rules:latest\",\n", " \"RuleParameters\": {\n", " \"rule_to_invoke\": \"ProfilerReport\"\n", " },\n", " \"VolumeSizeInGB\": 0\n", " }\n", " ],\n", " \"ProfilerRuleEvaluationStatuses\": [\n", " {\n", " \"LastModifiedTime\": \"2021-03-02 14:44:17.677000+00:00\",\n", " \"RuleConfigurationName\": \"ProfilerReport-1614696233\",\n", " \"RuleEvaluationJobArn\": \"arn:aws:sagemaker:eu-west-1:245582572290:processing-job/sagemaker-scikit-learn-202-profilerreport-1614696233-5e67a902\",\n", " \"RuleEvaluationStatus\": \"InProgress\"\n", " }\n", " ],\n", " \"ProfilingStatus\": \"Enabled\",\n", " \"ResourceConfig\": {\n", " \"InstanceCount\": 1,\n", " \"InstanceType\": \"ml.c4.xlarge\",\n", " \"VolumeSizeInGB\": 30\n", " },\n", " \"ResponseMetadata\": {\n", " \"HTTPHeaders\": {\n", " \"content-length\": \"3070\",\n", " \"content-type\": \"application/x-amz-json-1.1\",\n", " \"date\": \"Tue, 02 Mar 2021 14:44:18 GMT\",\n", " \"x-amzn-requestid\": \"ed039292-693c-467b-900c-97d2ceaddcfb\"\n", " },\n", " \"HTTPStatusCode\": 200,\n", " \"RequestId\": \"ed039292-693c-467b-900c-97d2ceaddcfb\",\n", " \"RetryAttempts\": 0\n", " },\n", " \"RoleArn\": \"arn:aws:iam::245582572290:role/workshop-sagemaker\",\n", " \"SecondaryStatus\": \"Starting\",\n", " \"SecondaryStatusTransitions\": [\n", " {\n", " \"StartTime\": \"2021-03-02 14:43:54.283000+00:00\",\n", " \"Status\": \"Starting\",\n", " \"StatusMessage\": \"Launching requested ML instances\"\n", " }\n", " ],\n", " \"StoppingCondition\": {\n", " \"MaxRuntimeInSeconds\": 86400,\n", " \"MaxWaitTimeInSeconds\": 86400\n", " },\n", " \"TrainingJobArn\": \"arn:aws:sagemaker:eu-west-1:245582572290:training-job/sagemaker-scikit-learn-2021-03-02-14-43-53-685\",\n", " \"TrainingJobName\": \"sagemaker-scikit-learn-2021-03-02-14-43-53-685\",\n", " \"TrainingJobStatus\": \"InProgress\"\n", "}\n" ] } ], "source": [ "%sklearn status" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Show logs for latest traning Job" ] }, { "cell_type": "code", "execution_count": 16, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "2021-03-02 14:47:11 Starting - Preparing the instances for training\n", "2021-03-02 14:47:11 Downloading - Downloading input data\n", "2021-03-02 14:47:11 Training - Training image download completed. Training in progress.\n", "2021-03-02 14:47:11 Uploading - Uploading generated training model\n", "2021-03-02 14:47:11 Completed - Training job completed\u001B[34m2021-03-02 14:46:58,954 sagemaker-containers INFO Imported framework sagemaker_sklearn_container.training\u001B[0m\n", "\u001B[34m2021-03-02 14:46:58,957 sagemaker-training-toolkit INFO No GPUs detected (normal if no gpus installed)\u001B[0m\n", "\u001B[34m2021-03-02 14:46:58,966 sagemaker_sklearn_container.training INFO Invoking user training script.\u001B[0m\n", "\u001B[34m2021-03-02 14:46:59,380 sagemaker-training-toolkit INFO No GPUs detected (normal if no gpus installed)\u001B[0m\n", "\u001B[34m2021-03-02 14:46:59,392 sagemaker-training-toolkit INFO No GPUs detected (normal if no gpus installed)\u001B[0m\n", "\u001B[34m2021-03-02 14:46:59,403 sagemaker-training-toolkit INFO No GPUs detected (normal if no gpus installed)\u001B[0m\n", "\u001B[34m2021-03-02 14:46:59,412 sagemaker-training-toolkit INFO Invoking user script\n", "\u001B[0m\n", "\u001B[34mTraining Env:\n", "\u001B[0m\n", "\u001B[34m{\n", " \"additional_framework_parameters\": {},\n", " \"channel_input_dirs\": {\n", " \"testing\": \"/opt/ml/input/data/testing\",\n", " \"training\": \"/opt/ml/input/data/training\"\n", " },\n", " \"current_host\": \"algo-1\",\n", " \"framework_module\": \"sagemaker_sklearn_container.training:main\",\n", " \"hosts\": [\n", " \"algo-1\"\n", " ],\n", " \"hyperparameters\": {\n", " \"max_leaf_nodes\": \"30\"\n", " },\n", " \"input_config_dir\": \"/opt/ml/input/config\",\n", " \"input_data_config\": {\n", " \"testing\": {\n", " \"TrainingInputMode\": \"File\",\n", " \"S3DistributionType\": \"FullyReplicated\",\n", " \"RecordWrapperType\": \"None\"\n", " },\n", " \"training\": {\n", " \"TrainingInputMode\": \"File\",\n", " \"S3DistributionType\": \"FullyReplicated\",\n", " \"RecordWrapperType\": \"None\"\n", " }\n", " },\n", " \"input_dir\": \"/opt/ml/input\",\n", " \"is_master\": true,\n", " \"job_name\": \"sagemaker-scikit-learn-2021-03-02-14-43-53-685\",\n", " \"log_level\": 20,\n", " \"master_hostname\": \"algo-1\",\n", " \"model_dir\": \"/opt/ml/model\",\n", " \"module_dir\": \"s3://sagemaker-eu-west-1-245582572290/sagemaker-scikit-learn-2021-03-02-14-43-53-685/source/sourcedir.tar.gz\",\n", " \"module_name\": \"tmp-9d940cb7-23ce-418d-b401-4783563fd641\",\n", " \"network_interface_name\": \"eth0\",\n", " \"num_cpus\": 4,\n", " \"num_gpus\": 0,\n", " \"output_data_dir\": \"/opt/ml/output/data\",\n", " \"output_dir\": \"/opt/ml/output\",\n", " \"output_intermediate_dir\": \"/opt/ml/output/intermediate\",\n", " \"resource_config\": {\n", " \"current_host\": \"algo-1\",\n", " \"hosts\": [\n", " \"algo-1\"\n", " ],\n", " \"network_interface_name\": \"eth0\"\n", " },\n", " \"user_entry_point\": \"tmp-9d940cb7-23ce-418d-b401-4783563fd641.py\"\u001B[0m\n", "\u001B[34m}\n", "\u001B[0m\n", "\u001B[34mEnvironment variables:\n", "\u001B[0m\n", "\u001B[34mSM_HOSTS=[\"algo-1\"]\u001B[0m\n", "\u001B[34mSM_NETWORK_INTERFACE_NAME=eth0\u001B[0m\n", "\u001B[34mSM_HPS={\"max_leaf_nodes\":\"30\"}\u001B[0m\n", "\u001B[34mSM_USER_ENTRY_POINT=tmp-9d940cb7-23ce-418d-b401-4783563fd641.py\u001B[0m\n", "\u001B[34mSM_FRAMEWORK_PARAMS={}\u001B[0m\n", "\u001B[34mSM_RESOURCE_CONFIG={\"current_host\":\"algo-1\",\"hosts\":[\"algo-1\"],\"network_interface_name\":\"eth0\"}\u001B[0m\n", "\u001B[34mSM_INPUT_DATA_CONFIG={\"testing\":{\"RecordWrapperType\":\"None\",\"S3DistributionType\":\"FullyReplicated\",\"TrainingInputMode\":\"File\"},\"training\":{\"RecordWrapperType\":\"None\",\"S3DistributionType\":\"FullyReplicated\",\"TrainingInputMode\":\"File\"}}\u001B[0m\n", "\u001B[34mSM_OUTPUT_DATA_DIR=/opt/ml/output/data\u001B[0m\n", "\u001B[34mSM_CHANNELS=[\"testing\",\"training\"]\u001B[0m\n", "\u001B[34mSM_CURRENT_HOST=algo-1\u001B[0m\n", "\u001B[34mSM_MODULE_NAME=tmp-9d940cb7-23ce-418d-b401-4783563fd641\u001B[0m\n", "\u001B[34mSM_LOG_LEVEL=20\u001B[0m\n", "\u001B[34mSM_FRAMEWORK_MODULE=sagemaker_sklearn_container.training:main\u001B[0m\n", "\u001B[34mSM_INPUT_DIR=/opt/ml/input\u001B[0m\n", "\u001B[34mSM_INPUT_CONFIG_DIR=/opt/ml/input/config\u001B[0m\n", "\u001B[34mSM_OUTPUT_DIR=/opt/ml/output\u001B[0m\n", "\u001B[34mSM_NUM_CPUS=4\u001B[0m\n", "\u001B[34mSM_NUM_GPUS=0\u001B[0m\n", "\u001B[34mSM_MODEL_DIR=/opt/ml/model\u001B[0m\n", "\u001B[34mSM_MODULE_DIR=s3://sagemaker-eu-west-1-245582572290/sagemaker-scikit-learn-2021-03-02-14-43-53-685/source/sourcedir.tar.gz\u001B[0m\n", "\u001B[34mSM_TRAINING_ENV={\"additional_framework_parameters\":{},\"channel_input_dirs\":{\"testing\":\"/opt/ml/input/data/testing\",\"training\":\"/opt/ml/input/data/training\"},\"current_host\":\"algo-1\",\"framework_module\":\"sagemaker_sklearn_container.training:main\",\"hosts\":[\"algo-1\"],\"hyperparameters\":{\"max_leaf_nodes\":\"30\"},\"input_config_dir\":\"/opt/ml/input/config\",\"input_data_config\":{\"testing\":{\"RecordWrapperType\":\"None\",\"S3DistributionType\":\"FullyReplicated\",\"TrainingInputMode\":\"File\"},\"training\":{\"RecordWrapperType\":\"None\",\"S3DistributionType\":\"FullyReplicated\",\"TrainingInputMode\":\"File\"}},\"input_dir\":\"/opt/ml/input\",\"is_master\":true,\"job_name\":\"sagemaker-scikit-learn-2021-03-02-14-43-53-685\",\"log_level\":20,\"master_hostname\":\"algo-1\",\"model_dir\":\"/opt/ml/model\",\"module_dir\":\"s3://sagemaker-eu-west-1-245582572290/sagemaker-scikit-learn-2021-03-02-14-43-53-685/source/sourcedir.tar.gz\",\"module_name\":\"tmp-9d940cb7-23ce-418d-b401-4783563fd641\",\"network_interface_name\":\"eth0\",\"num_cpus\":4,\"num_gpus\":0,\"output_data_dir\":\"/opt/ml/output/data\",\"output_dir\":\"/opt/ml/output\",\"output_intermediate_dir\":\"/opt/ml/output/intermediate\",\"resource_config\":{\"current_host\":\"algo-1\",\"hosts\":[\"algo-1\"],\"network_interface_name\":\"eth0\"},\"user_entry_point\":\"tmp-9d940cb7-23ce-418d-b401-4783563fd641.py\"}\u001B[0m\n", "\u001B[34mSM_USER_ARGS=[\"--max_leaf_nodes\",\"30\"]\u001B[0m\n", "\u001B[34mSM_OUTPUT_INTERMEDIATE_DIR=/opt/ml/output/intermediate\u001B[0m\n", "\u001B[34mSM_CHANNEL_TESTING=/opt/ml/input/data/testing\u001B[0m\n", "\u001B[34mSM_CHANNEL_TRAINING=/opt/ml/input/data/training\u001B[0m\n", "\u001B[34mSM_HP_MAX_LEAF_NODES=30\u001B[0m\n", "\u001B[34mPYTHONPATH=/opt/ml/code:/miniconda3/bin:/miniconda3/lib/python37.zip:/miniconda3/lib/python3.7:/miniconda3/lib/python3.7/lib-dynload:/miniconda3/lib/python3.7/site-packages\n", "\u001B[0m\n", "\u001B[34mInvoking script with the following command:\n", "\u001B[0m\n", "\u001B[34m/miniconda3/bin/python tmp-9d940cb7-23ce-418d-b401-4783563fd641.py --max_leaf_nodes 30\n", "\n", "\u001B[0m\n", "\u001B[34m2021-03-02 14:47:01,080 sagemaker-containers INFO Reporting training SUCCESS\u001B[0m\n", "null\n" ] } ], "source": [ "%sklearn logs" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## List traning jobs" ] }, { "cell_type": "code", "execution_count": 13, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "{\n", " \"NextToken\": \"cIws2QhTXUIa8bi8X9aU7gCAR0Xdc3x9L/Ofg4vsVMTtcNqRqLcpBqE42+cDc29SYTx/csxWdtykePlH94xR4wYEkuOg2mnj9cwdVGP+NpZB72ngF4Jrl7LpzDZ0PJsqFrzBt+u31JpjCNRKIu0shJdmwQFN4q1dZVv7nlEQknn0Lq3OrzlpFDQiE18Oj1AJHnCfuBJZ7zOQ3sfYAfAdqYagLXLCDWuTdoa/hB7wYjqq3FrkadY0KvMW7eH0wtpZJayqKkHWdIyHy67DU4AmQJv8Gwc6g8JNdVWRblepG2ilsMjH/0avxXpG9AcUNA+GmZhYy57avBvkY5SgrCn6XoAyNX93/BeuXP7xKqEquvzZTiKq7vUo0p5kmdw1hhty1YLrJq3yMRVfoRqUsEh5wgMthHc9Nu5KmKdfgkL5XQFJUV3AwTw9ZTTZJbHJHSlmN2GESIAgBwwSmddHazfmhGUhL2ETbeAZ2FWN8vgWg94TN8cX++UfNBqWE7pZu+4LcXqHkd9SID/Jk95ZEzHBhapfc8O3+JZSZKOkE21Dm+Gs2YVOLrg6689YxbUCLnZtF4I8PwAdjp5pjUbZJQ==\",\n", " \"ResponseMetadata\": {\n", " \"HTTPHeaders\": {\n", " \"content-length\": \"1606\",\n", " \"content-type\": \"application/x-amz-json-1.1\",\n", " \"date\": \"Tue, 02 Mar 2021 15:18:50 GMT\",\n", " \"x-amzn-requestid\": \"2dd07e22-9075-4295-8e0a-a55f6ee249b0\"\n", " },\n", " \"HTTPStatusCode\": 200,\n", " \"RequestId\": \"2dd07e22-9075-4295-8e0a-a55f6ee249b0\",\n", " \"RetryAttempts\": 0\n", " },\n", " \"TrainingJobSummaries\": [\n", " {\n", " \"CreationTime\": \"2021-03-02 15:04:35.639000+00:00\",\n", " \"LastModifiedTime\": \"2021-03-02 15:08:41.523000+00:00\",\n", " \"TrainingEndTime\": \"2021-03-02 15:08:07.425000+00:00\",\n", " \"TrainingJobArn\": \"arn:aws:sagemaker:eu-west-1:245582572290:training-job/sagemaker-scikit-learn-2021-03-02-15-04-35-077\",\n", " \"TrainingJobName\": \"sagemaker-scikit-learn-2021-03-02-15-04-35-077\",\n", " \"TrainingJobStatus\": \"Completed\"\n", " },\n", " {\n", " \"CreationTime\": \"2021-03-02 14:58:22.381000+00:00\",\n", " \"LastModifiedTime\": \"2021-03-02 15:01:59.679000+00:00\",\n", " \"TrainingEndTime\": \"2021-03-02 15:00:56.060000+00:00\",\n", " \"TrainingJobArn\": \"arn:aws:sagemaker:eu-west-1:245582572290:training-job/sagemaker-scikit-learn-2021-03-02-14-58-21-806\",\n", " \"TrainingJobName\": \"sagemaker-scikit-learn-2021-03-02-14-58-21-806\",\n", " \"TrainingJobStatus\": \"Stopped\"\n", " },\n", " {\n", " \"CreationTime\": \"2021-03-02 14:43:54.283000+00:00\",\n", " \"LastModifiedTime\": \"2021-03-02 14:48:20.474000+00:00\",\n", " \"TrainingEndTime\": \"2021-03-02 14:47:11.062000+00:00\",\n", " \"TrainingJobArn\": \"arn:aws:sagemaker:eu-west-1:245582572290:training-job/sagemaker-scikit-learn-2021-03-02-14-43-53-685\",\n", " \"TrainingJobName\": \"sagemaker-scikit-learn-2021-03-02-14-43-53-685\",\n", " \"TrainingJobStatus\": \"Completed\"\n", " }\n", " ]\n", "}\n" ] } ], "source": [ "%sklearn list --name_contains scikit-learn" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Use estimator variable\n", "### Deploy the model " ] }, { "cell_type": "code", "execution_count": 11, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "---------------!" ] } ], "source": [ "# Deploy my estimator to a SageMaker Endpoint and get a Predictor\n", "predictor = ___SKLearn_estimator.deploy(instance_type='ml.m4.xlarge',\n", " initial_instance_count=1)\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Choose some data and use it for a prediction \n", "\n", "In order to do some predictions, we'll extract some of the data we used for training and do predictions against it. This is, of course, bad statistical practice, but a good way to see how the mechanism works." ] }, { "cell_type": "code", "execution_count": 14, "metadata": {}, "outputs": [], "source": [ "import itertools\n", "import pandas as pd\n", "\n", "shape = pd.read_csv(\"data/iris.csv\", header=None)\n", "\n", "a = [50*i for i in range(3)]\n", "b = [40+i for i in range(10)]\n", "indices = [i+j for i,j in itertools.product(a,b)]\n", "\n", "test_data = shape.iloc[indices[:-1]]\n", "test_X = test_data.iloc[:,1:]\n", "test_y = test_data.iloc[:,0]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Prediction is as easy as calling predict with the predictor we got back from deploy and the data we want to do predictions with. The output from the endpoint return an numerical representation of the classification prediction; in the original dataset, these are flower names, but in this example the labels are numerical. We can compare against the original label that we parsed." ] }, { "cell_type": "code", "execution_count": 15, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "[0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 2. 2. 2. 2.\n", " 2. 2. 2. 2. 2.]\n", "[0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 2. 2. 2. 2.\n", " 2. 2. 2. 2. 2.]\n" ] } ], "source": [ "print(predictor.predict(test_X.values))\n", "print(test_y.values)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Endpoint cleanup \n", "\n", "When you're done with the endpoint, you'll want to clean it up." ] }, { "cell_type": "code", "execution_count": 16, "metadata": {}, "outputs": [], "source": [ "predictor.delete_endpoint()" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] } ], "metadata": { "instance_type": "ml.t3.medium", "kernelspec": { "display_name": "SageMakerMagic (lblokhin/37)", "language": "python", "name": "sm__SAGEMAKER_INTERNAL__arn:aws:sagemaker:eu-west-1:245582572290:image-version/lblokhin/37" }, "language_info": { "codemirror_mode": { "name": "python", "version": 3 }, "mimetype": "text/x-python", "name": "sm_kernel", "pygments_lexer": "python" } }, "nbformat": 4, "nbformat_minor": 4 }