{ "cells": [ { "cell_type": "markdown", "id": "3606bcf8", "metadata": { "tags": [] }, "source": [ "# 1. SageMaker Training For AutoGluon" ] }, { "cell_type": "markdown", "id": "5f479074", "metadata": {}, "source": [ "### 작업 실행 시 필요 라이브러리 import" ] }, { "cell_type": "code", "execution_count": null, "id": "4ad2393d", "metadata": {}, "outputs": [], "source": [ "!pip install -U sagemaker-experiments" ] }, { "cell_type": "code", "execution_count": 2, "id": "e555f3f3", "metadata": {}, "outputs": [], "source": [ "!pip install sagemaker ipywidgets --upgrade --quiet" ] }, { "cell_type": "code", "execution_count": null, "id": "d2f08587", "metadata": {}, "outputs": [], "source": [ "!pip install autogluon" ] }, { "cell_type": "code", "execution_count": 6, "id": "62e04b57", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "'1.7.0'" ] }, "execution_count": 6, "metadata": {}, "output_type": "execute_result" } ], "source": [ "import mxnet\n", "mxnet.__version__" ] }, { "cell_type": "code", "execution_count": 7, "id": "06312623", "metadata": {}, "outputs": [], "source": [ "import pandas as pd" ] }, { "cell_type": "code", "execution_count": 83, "id": "139688e0", "metadata": {}, "outputs": [], "source": [ "import sys\n", "import os" ] }, { "cell_type": "code", "execution_count": 8, "id": "10fe2b70", "metadata": {}, "outputs": [], "source": [ "from sklearn.model_selection import train_test_split #splitting the dataset\n", "from autogluon.tabular import TabularDataset, TabularPredictor #to handle tabular data and train models" ] }, { "cell_type": "code", "execution_count": 4, "id": "d0300dc2", "metadata": {}, "outputs": [], "source": [ "import sagemaker, boto3, json\n", "from sagemaker import get_execution_role\n", "\n", "aws_role = get_execution_role()\n", "aws_region = boto3.Session().region_name\n", "sess = sagemaker.Session()" ] }, { "cell_type": "markdown", "id": "4b64bda5", "metadata": {}, "source": [ "### Dataset\n", "\n", "본 예제는 [ETDataset](https://github.com/zhouhaoyi/ETDataset)을 사용합니다. 데이터 셋은 하위와 같이 Github 리포지토리를 클론해 다운로드 받을 수 있도록 합니다." ] }, { "cell_type": "code", "execution_count": 5, "id": "1307c18d", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "fatal: destination path 'ETDataset' already exists and is not an empty directory.\n" ] } ], "source": [ "!git clone https://github.com/zhouhaoyi/ETDataset.git" ] }, { "cell_type": "code", "execution_count": 9, "id": "623dc48f", "metadata": {}, "outputs": [], "source": [ "df = pd.read_csv('./ETDataset/ETT-small/ETTh1.csv')" ] }, { "cell_type": "code", "execution_count": 10, "id": "4dfa1992", "metadata": {}, "outputs": [], "source": [ "df = df.drop(['date'], axis=1)" ] }, { "cell_type": "code", "execution_count": 11, "id": "362df7fc", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "((11671, 7), (5749, 7))" ] }, "execution_count": 11, "metadata": {}, "output_type": "execute_result" } ], "source": [ "df_train,df_test=train_test_split(df,test_size=0.33,random_state=1)\n", "df_train.shape,df_test.shape" ] }, { "cell_type": "code", "execution_count": 12, "id": "0a45e165", "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
HUFLHULLMUFLMULLLUFLLULLOT
151626.7651.5413.5530.3203.1980.97512.662000
1601315.0042.54510.5180.6044.6301.15711.607000
276410.9183.2157.2852.7723.594-0.48717.445999
17558.9754.7567.3563.4111.3400.70119.979000
69519.3572.81313.752-0.0715.9701.49240.450001
\n", "
" ], "text/plain": [ " HUFL HULL MUFL MULL LUFL LULL OT\n", "15162 6.765 1.541 3.553 0.320 3.198 0.975 12.662000\n", "16013 15.004 2.545 10.518 0.604 4.630 1.157 11.607000\n", "2764 10.918 3.215 7.285 2.772 3.594 -0.487 17.445999\n", "1755 8.975 4.756 7.356 3.411 1.340 0.701 19.979000\n", "695 19.357 2.813 13.752 -0.071 5.970 1.492 40.450001" ] }, "execution_count": 12, "metadata": {}, "output_type": "execute_result" } ], "source": [ "df_train.head()" ] }, { "cell_type": "code", "execution_count": 13, "id": "c2404479", "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
HUFLHULLMUFLMULLLUFLLULL
14198-16.2092.880-18.9050011.2442.4670.670
1351212.0562.8808.7420001.6352.8020.914
206912.5923.55010.6250002.3101.7060.701
83088.4391.4745.7570001.7411.919-0.609
1644811.9224.8907.1780002.3455.0561.614
\n", "
" ], "text/plain": [ " HUFL HULL MUFL MULL LUFL LULL\n", "14198 -16.209 2.880 -18.905001 1.244 2.467 0.670\n", "13512 12.056 2.880 8.742000 1.635 2.802 0.914\n", "2069 12.592 3.550 10.625000 2.310 1.706 0.701\n", "8308 8.439 1.474 5.757000 1.741 1.919 -0.609\n", "16448 11.922 4.890 7.178000 2.345 5.056 1.614" ] }, "execution_count": 13, "metadata": {}, "output_type": "execute_result" } ], "source": [ "test_data=df_test.drop(['OT'],axis=1)\n", "test_data.head()" ] }, { "cell_type": "markdown", "id": "969bd072", "metadata": {}, "source": [ "### AutoGluon " ] }, { "cell_type": "code", "execution_count": null, "id": "e889e8b7", "metadata": {}, "outputs": [], "source": [ "predictor= TabularPredictor(label ='OT').fit(train_data = df_train, verbosity = 2,presets='best_quality', hyperparameters = {\n", " # 'NN': {'num_epochs': 500},\n", " 'GBM': {},\n", " 'CAT': {},\n", " 'RF': {},\n", " 'XT': {},\n", " 'KNN': {},\n", "# 'custom': ['GBM'],\n", "})" ] }, { "cell_type": "code", "execution_count": null, "id": "6e66a841", "metadata": {}, "outputs": [], "source": [ "predictor.fit_summary()" ] }, { "cell_type": "code", "execution_count": 19, "id": "149acc33", "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
modelscore_testscore_valpred_time_testpred_time_valfit_timepred_time_test_marginalpred_time_val_marginalfit_time_marginalstack_levelcan_inferfit_order
0ExtraTrees_BAG_L1-2.373084-6.3902540.4252190.4749580.9125910.4252190.4749580.9125911True5
1RandomForest_BAG_L1-2.377621-6.4075530.2436810.4694961.4140260.2436810.4694961.4140261True3
2WeightedEnsemble_L2-3.472599-6.3287541.2735221.13500131.6409940.0040060.0005970.2406712True6
3CatBoost_BAG_L2-3.812389-6.3382751.3018231.15967736.4337940.0323070.0252745.0334722True9
4WeightedEnsemble_L3-3.863211-6.3318951.5519751.65098537.6567260.0019580.0006140.2064053True11
5LightGBM_BAG_L2-3.937479-6.4014701.3574381.16043634.2953420.0879220.0260322.8950192True7
6ExtraTrees_BAG_L2-4.205004-6.4055821.5177101.62509732.4168500.2481940.4906941.0165272True10
7RandomForest_BAG_L2-4.316848-6.4709871.5153111.63267434.6214630.2457960.4982713.2211402True8
8LightGBM_BAG_L1-5.231914-6.4707300.4405830.0568864.7713930.4405830.0568864.7713931True2
9CatBoost_BAG_L1-5.433174-6.4548640.0550130.02819224.2900920.0550130.02819224.2900921True4
10KNeighbors_BAG_L1-5.688029-7.0193380.1050190.1048710.0122220.1050190.1048710.0122221True1
\n", "
" ], "text/plain": [ " model score_test score_val pred_time_test pred_time_val \\\n", "0 ExtraTrees_BAG_L1 -2.373084 -6.390254 0.425219 0.474958 \n", "1 RandomForest_BAG_L1 -2.377621 -6.407553 0.243681 0.469496 \n", "2 WeightedEnsemble_L2 -3.472599 -6.328754 1.273522 1.135001 \n", "3 CatBoost_BAG_L2 -3.812389 -6.338275 1.301823 1.159677 \n", "4 WeightedEnsemble_L3 -3.863211 -6.331895 1.551975 1.650985 \n", "5 LightGBM_BAG_L2 -3.937479 -6.401470 1.357438 1.160436 \n", "6 ExtraTrees_BAG_L2 -4.205004 -6.405582 1.517710 1.625097 \n", "7 RandomForest_BAG_L2 -4.316848 -6.470987 1.515311 1.632674 \n", "8 LightGBM_BAG_L1 -5.231914 -6.470730 0.440583 0.056886 \n", "9 CatBoost_BAG_L1 -5.433174 -6.454864 0.055013 0.028192 \n", "10 KNeighbors_BAG_L1 -5.688029 -7.019338 0.105019 0.104871 \n", "\n", " fit_time pred_time_test_marginal pred_time_val_marginal \\\n", "0 0.912591 0.425219 0.474958 \n", "1 1.414026 0.243681 0.469496 \n", "2 31.640994 0.004006 0.000597 \n", "3 36.433794 0.032307 0.025274 \n", "4 37.656726 0.001958 0.000614 \n", "5 34.295342 0.087922 0.026032 \n", "6 32.416850 0.248194 0.490694 \n", "7 34.621463 0.245796 0.498271 \n", "8 4.771393 0.440583 0.056886 \n", "9 24.290092 0.055013 0.028192 \n", "10 0.012222 0.105019 0.104871 \n", "\n", " fit_time_marginal stack_level can_infer fit_order \n", "0 0.912591 1 True 5 \n", "1 1.414026 1 True 3 \n", "2 0.240671 2 True 6 \n", "3 5.033472 2 True 9 \n", "4 0.206405 3 True 11 \n", "5 2.895019 2 True 7 \n", "6 1.016527 2 True 10 \n", "7 3.221140 2 True 8 \n", "8 4.771393 1 True 2 \n", "9 24.290092 1 True 4 \n", "10 0.012222 1 True 1 " ] }, "execution_count": 19, "metadata": {}, "output_type": "execute_result" } ], "source": [ "predictor.leaderboard(df_train, silent=True)\n" ] }, { "cell_type": "code", "execution_count": 20, "id": "18e61403", "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "Computing feature importance via permutation shuffling for 6 features using 1000 rows with 3 shuffle sets...\n", "\t15.98s\t= Expected runtime (5.33s per shuffle set)\n", "\t5.08s\t= Actual runtime (Completed 3 of 3 shuffle sets)\n" ] }, { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
importancestddevp_valuenp99_highp99_low
LULL4.1863010.2398010.00054635.5603852.812217
LUFL3.1945130.1034890.00017533.7875152.601511
MUFL2.9997990.0573950.00006133.3286822.670917
MULL2.9177580.1326640.00034433.6779392.157576
HULL2.3345100.0607900.00011332.6828431.986177
HUFL1.8786860.1068050.00053832.4906931.266679
\n", "
" ], "text/plain": [ " importance stddev p_value n p99_high p99_low\n", "LULL 4.186301 0.239801 0.000546 3 5.560385 2.812217\n", "LUFL 3.194513 0.103489 0.000175 3 3.787515 2.601511\n", "MUFL 2.999799 0.057395 0.000061 3 3.328682 2.670917\n", "MULL 2.917758 0.132664 0.000344 3 3.677939 2.157576\n", "HULL 2.334510 0.060790 0.000113 3 2.682843 1.986177\n", "HUFL 1.878686 0.106805 0.000538 3 2.490693 1.266679" ] }, "execution_count": 20, "metadata": {}, "output_type": "execute_result" } ], "source": [ "predictor.feature_importance(data=df_train)" ] }, { "cell_type": "code", "execution_count": 22, "id": "5ae0b9e3", "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
OT
141987.291081
135128.206678
206916.144300
830815.873062
1644811.764364
......
47615.841290
50049.059195
1095418.083954
1621210.732670
1159910.766358
\n", "

5749 rows × 1 columns

\n", "
" ], "text/plain": [ " OT\n", "14198 7.291081\n", "13512 8.206678\n", "2069 16.144300\n", "8308 15.873062\n", "16448 11.764364\n", "... ...\n", "476 15.841290\n", "5004 9.059195\n", "10954 18.083954\n", "16212 10.732670\n", "11599 10.766358\n", "\n", "[5749 rows x 1 columns]" ] }, "execution_count": 22, "metadata": {}, "output_type": "execute_result" } ], "source": [ "y_pred = predictor.predict(test_data)\n", "y_pred=pd.DataFrame(y_pred,columns=['OT'])\n", "y_pred #print the DataFrame" ] }, { "cell_type": "code", "execution_count": 23, "id": "cf6b5d05", "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "Evaluation: root_mean_squared_error on test data: -6.391080107445289\n", "\tNote: Scores are always higher_is_better. This metric score can be multiplied by -1 to get the metric value.\n", "Evaluations on test data:\n", "{\n", " \"root_mean_squared_error\": -6.391080107445289,\n", " \"mean_squared_error\": -40.84590493978301,\n", " \"mean_absolute_error\": -4.507494016361431,\n", " \"r2\": 0.456628203348066,\n", " \"pearsonr\": 0.6770374919797111,\n", " \"median_absolute_error\": -3.2320613861083984\n", "}\n" ] }, { "data": { "text/plain": [ "{'root_mean_squared_error': -6.391080107445289,\n", " 'mean_squared_error': -40.84590493978301,\n", " 'mean_absolute_error': -4.507494016361431,\n", " 'r2': 0.456628203348066,\n", " 'pearsonr': 0.6770374919797111,\n", " 'median_absolute_error': -3.2320613861083984}" ] }, "execution_count": 23, "metadata": {}, "output_type": "execute_result" } ], "source": [ "predictor.evaluate(df_test)" ] }, { "cell_type": "markdown", "id": "8ba5f75b", "metadata": {}, "source": [ "### SageMaker" ] }, { "cell_type": "code", "execution_count": 41, "id": "b7e5fbab", "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "image_uri is not presented, retrieving image_uri based on instance_type, framework etc.\n" ] } ], "source": [ "from sagemaker import image_uris, model_uris, script_uris\n", "\n", "train_model_id, train_model_version, train_scope = \"autogluon-regression-ensemble\", \"*\", \"training\"\n", "\n", "training_instance_type = \"ml.m5.12xlarge\"\n", "\n", "# Retrieve the docker image\n", "train_image_uri = image_uris.retrieve(\n", " region=None,\n", " framework=None,\n", " model_id=train_model_id,\n", " model_version=train_model_version,\n", " image_scope=train_scope,\n", " instance_type=training_instance_type,\n", ")\n", "# Retrieve the training script\n", "train_source_uri = script_uris.retrieve(\n", " model_id=train_model_id, model_version=train_model_version, script_scope=train_scope\n", ")\n", "# Retrieve the pre-trained model tarball to further fine-tune. In tabular case, however, the pre-trained model tarball is dummy and fine-tune means training from scratch.\n", "train_model_uri = model_uris.retrieve(\n", " model_id=train_model_id, model_version=train_model_version, model_scope=train_scope\n", ")" ] }, { "cell_type": "code", "execution_count": 56, "id": "0c48e825", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "{\n", " \"AcceptRanges\": \"bytes\",\n", " \"LastModified\": \"Sat, 30 Jul 2022 18:40:11 GMT\",\n", " \"ContentLength\": 3198,\n", " \"ETag\": \"\\\"6d151a8ce01643d903391df6f5058390\\\"\",\n", " \"VersionId\": \"sy1FGcdI6WJN9p7oU_vrS3yeHB30WPI0\",\n", " \"ContentType\": \"binary/octet-stream\",\n", " \"Metadata\": {}\n", "}\n" ] } ], "source": [ "!aws s3api get-object --bucket jumpstart-cache-prod-ap-northeast-2 --key source-directory-tarballs/autogluon/transfer_learning/regression/v1.0.1/sourcedir.tar.gz sourcedir.tar.gz" ] }, { "cell_type": "code", "execution_count": 59, "id": "1b0db8b5", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "mkdir: cannot create directory ‘source’: File exists\n" ] } ], "source": [ "!mkdir source " ] }, { "cell_type": "code", "execution_count": 61, "id": "b69b0ce8", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "__init__.py\n", "constants/\n", "constants/__init__.py\n", "constants/__pycache__/\n", "constants/__pycache__/__init__.cpython-37.pyc\n", "constants/__pycache__/constants.cpython-37.pyc\n", "constants/constants.py\n", "transfer_learning.py\n", "version\n", "__pycache__/\n", "__pycache__/__init__.cpython-37.pyc\n" ] } ], "source": [ "!cd source && tar -zxvf sourcedir.tar.gz" ] }, { "cell_type": "code", "execution_count": 67, "id": "a0ef8df5", "metadata": {}, "outputs": [], "source": [ "# Sample training data is available in this bucket\n", "training_data_bucket = f\"yudong-data\"\n", "training_data_prefix = \"training-datasets/tabular_regression/train\"\n", "\n", "# training_dataset_s3_path = f\"s3://{training_data_bucket}/{training_data_prefix}\"\n", "\n", "output_prefix = \"training-datasets/tabular_regression\"\n", "\n", "s3_output_location = f\"s3://{training_data_bucket}/{output_prefix}/output\"" ] }, { "cell_type": "code", "execution_count": 68, "id": "9eb98655", "metadata": {}, "outputs": [], "source": [ "df_train.to_csv('train.csv', index=False)" ] }, { "cell_type": "code", "execution_count": 69, "id": "9c4ee260", "metadata": {}, "outputs": [], "source": [ "training_dataset_s3_path = session.upload_data('train.csv', key_prefix=training_data_prefix, bucket=training_data_bucket)" ] }, { "cell_type": "code", "execution_count": 71, "id": "9c875d98", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "{'eval_metric': 'auto', 'presets': 'medium_quality', 'auto_stack': 'True', 'num_bag_folds': '0', 'num_bag_sets': '1', 'num_stack_levels': '0', 'refit_full': 'False', 'set_best_to_refit_full': 'False', 'save_space': 'False', 'verbosity': '2'}\n" ] } ], "source": [ "from sagemaker import hyperparameters\n", "\n", "# Retrieve the default hyper-parameters for training the model\n", "hyperparameters = hyperparameters.retrieve_default(\n", " model_id=train_model_id, model_version=train_model_version\n", ")\n", "\n", "# [Optional] Override default hyperparameters with custom values\n", "hyperparameters[\"auto_stack\"] = \"True\"\n", "print(hyperparameters)" ] }, { "cell_type": "code", "execution_count": 80, "id": "604a151e", "metadata": {}, "outputs": [], "source": [ "training_instance_type = 'local'\n", "max_run = 1*60*60\n", "\n", "use_spot_instances = False\n", "if use_spot_instances:\n", " max_wait = 1*60*60\n", "else:\n", " max_wait = None" ] }, { "cell_type": "code", "execution_count": 84, "id": "29092309", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "file:///home/ec2-user/SageMaker/mlops-ramp/mlops-for-autogluon-step-fn/1_sagemaker_training/\n" ] } ], "source": [ "if training_instance_type == 'local':\n", " from sagemaker.local import LocalSession\n", " \n", " sagemaker_session = LocalSession()\n", " sagemaker_session.config = {'local': {'local_code': True}}\n", " local_data_path = \"file://\" + os.getcwd().replace('/lab_1_training', '') + \"/\"\n", " \n", " print(local_data_path)\n", " \n", " data_channels = {\n", " \"inputdata\": local_data_path, \n", " }\n", " \n", "else:\n", " sess = boto3.Session()\n", " sagemaker_session = sagemaker.Session()\n", " sm = sess.client('sagemaker')\n", " \n", " data_channels = {\n", " \"inputdata\": data_path, \n", " }" ] }, { "cell_type": "code", "execution_count": 101, "id": "0b78c985", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "{'eval_metric': 'auto',\n", " 'presets': 'medium_quality',\n", " 'auto_stack': 'True',\n", " 'num_bag_folds': '0',\n", " 'num_bag_sets': '1',\n", " 'num_stack_levels': '0',\n", " 'refit_full': 'False',\n", " 'set_best_to_refit_full': 'False',\n", " 'save_space': 'False',\n", " 'verbosity': '2',\n", " 'sagemaker_submit_directory': 'file:///home/ec2-user/SageMaker/mlops-ramp/mlops-for-autogluon-step-fn/1_sagemaker_training/source',\n", " 'sagemaker_program': 'transfer_learning.py',\n", " 'sagemaker_container_log_level': 20,\n", " 'sagemaker_job_name': 'autogluon-autogluon-regression-ensemble-2022-08-15-14-17-33-339',\n", " 'sagemaker_region': 'ap-northeast-2'}" ] }, "execution_count": 101, "metadata": {}, "output_type": "execute_result" } ], "source": [ "hyperparameters" ] }, { "cell_type": "code", "execution_count": 109, "id": "2da423dd", "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "Defaulting to the only supported framework/algorithm version: latest.\n", "Ignoring unnecessary instance type: None.\n", "Creating training-job with name: autogluon-autogluon-regression-ensemble-2022-08-15-20-29-33-019\n", "Starting training job\n", "No AWS credentials found in session but credentials from EC2 Metadata Service are available.\n", "docker compose file: \n", "networks:\n", " sagemaker-local:\n", " name: sagemaker-local\n", "services:\n", " algo-1-kyfef:\n", " command: train\n", " container_name: s25n1fxoqk-algo-1-kyfef\n", " environment:\n", " - '[Masked]'\n", " - '[Masked]'\n", " image: 763104351884.dkr.ecr.ap-northeast-2.amazonaws.com/autogluon-training:0.4.0-cpu-py38\n", " networks:\n", " sagemaker-local:\n", " aliases:\n", " - algo-1-kyfef\n", " stdin_open: true\n", " tty: true\n", " volumes:\n", " - /tmp/tmp0za9twmc/algo-1-kyfef/input:/opt/ml/input\n", " - /tmp/tmp0za9twmc/algo-1-kyfef/output/data:/opt/ml/output/data\n", " - /tmp/tmp0za9twmc/algo-1-kyfef/output:/opt/ml/output\n", " - /tmp/tmp0za9twmc/model:/opt/ml/model\n", " - /opt/ml/metadata:/opt/ml/metadata\n", " - /tmp/tmpqt_13ty6:/opt/ml/input/data/train\n", "version: '2.3'\n", "\n", "docker command: docker-compose -f /tmp/tmp0za9twmc/docker-compose.yaml up --build --abort-on-container-exit\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Creating s25n1fxoqk-algo-1-kyfef ... \n", "Creating s25n1fxoqk-algo-1-kyfef ... done\n", "Attaching to s25n1fxoqk-algo-1-kyfef\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m 2022-08-15 20:29:34,753 sagemaker-training-toolkit INFO Imported framework sagemaker_mxnet_container.training\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m 2022-08-15 20:29:34,755 sagemaker-training-toolkit INFO No GPUs detected (normal if no gpus installed)\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m 2022-08-15 20:29:34,755 sagemaker-training-toolkit INFO Failed to parse hyperparameter eval_metric value auto to Json.\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m Returning the value itself\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m 2022-08-15 20:29:34,755 sagemaker-training-toolkit INFO Failed to parse hyperparameter presets value medium_quality to Json.\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m Returning the value itself\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m 2022-08-15 20:29:34,755 sagemaker-training-toolkit INFO Failed to parse hyperparameter auto_stack value True to Json.\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m Returning the value itself\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m 2022-08-15 20:29:34,755 sagemaker-training-toolkit INFO Failed to parse hyperparameter refit_full value False to Json.\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m Returning the value itself\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m 2022-08-15 20:29:34,756 sagemaker-training-toolkit INFO Failed to parse hyperparameter set_best_to_refit_full value False to Json.\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m Returning the value itself\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m 2022-08-15 20:29:34,756 sagemaker-training-toolkit INFO Failed to parse hyperparameter save_space value False to Json.\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m Returning the value itself\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m 2022-08-15 20:29:34,765 sagemaker_mxnet_container.training INFO MXNet training environment: {'SM_HOSTS': '[\"algo-1-kyfef\"]', 'SM_NETWORK_INTERFACE_NAME': 'eth0', 'SM_HPS': '{\"auto_stack\":\"True\",\"eval_metric\":\"auto\",\"num_bag_folds\":0,\"num_bag_sets\":1,\"num_stack_levels\":0,\"presets\":\"medium_quality\",\"refit_full\":\"False\",\"save_space\":\"False\",\"set_best_to_refit_full\":\"False\",\"verbosity\":2}', 'SM_USER_ENTRY_POINT': 'transfer_learning.py', 'SM_FRAMEWORK_PARAMS': '{}', 'SM_RESOURCE_CONFIG': '{\"current_host\":\"algo-1-kyfef\",\"hosts\":[\"algo-1-kyfef\"]}', 'SM_INPUT_DATA_CONFIG': '{\"train\":{\"TrainingInputMode\":\"File\"}}', 'SM_OUTPUT_DATA_DIR': '/opt/ml/output/data', 'SM_CHANNELS': '[\"train\"]', 'SM_CURRENT_HOST': 'algo-1-kyfef', 'SM_MODULE_NAME': 'transfer_learning', 'SM_LOG_LEVEL': '20', 'SM_FRAMEWORK_MODULE': 'sagemaker_mxnet_container.training:main', 'SM_INPUT_DIR': '/opt/ml/input', 'SM_INPUT_CONFIG_DIR': '/opt/ml/input/config', 'SM_OUTPUT_DIR': '/opt/ml/output', 'SM_NUM_CPUS': '16', 'SM_NUM_GPUS': '0', 'SM_MODEL_DIR': '/opt/ml/model', 'SM_MODULE_DIR': 's3://yudong-data/autogluon-autogluon-regression-ensemble-2022-08-15-20-29-33-019/source/sourcedir.tar.gz', 'SM_TRAINING_ENV': '{\"additional_framework_parameters\":{},\"channel_input_dirs\":{\"train\":\"/opt/ml/input/data/train\"},\"current_host\":\"algo-1-kyfef\",\"framework_module\":\"sagemaker_mxnet_container.training:main\",\"hosts\":[\"algo-1-kyfef\"],\"hyperparameters\":{\"auto_stack\":\"True\",\"eval_metric\":\"auto\",\"num_bag_folds\":0,\"num_bag_sets\":1,\"num_stack_levels\":0,\"presets\":\"medium_quality\",\"refit_full\":\"False\",\"save_space\":\"False\",\"set_best_to_refit_full\":\"False\",\"verbosity\":2},\"input_config_dir\":\"/opt/ml/input/config\",\"input_data_config\":{\"train\":{\"TrainingInputMode\":\"File\"}},\"input_dir\":\"/opt/ml/input\",\"is_master\":true,\"job_name\":\"autogluon-autogluon-regression-ensemble-2022-08-15-20-29-33-019\",\"log_level\":20,\"master_hostname\":\"algo-1-kyfef\",\"model_dir\":\"/opt/ml/model\",\"module_dir\":\"s3://yudong-data/autogluon-autogluon-regression-ensemble-2022-08-15-20-29-33-019/source/sourcedir.tar.gz\",\"module_name\":\"transfer_learning\",\"network_interface_name\":\"eth0\",\"num_cpus\":16,\"num_gpus\":0,\"output_data_dir\":\"/opt/ml/output/data\",\"output_dir\":\"/opt/ml/output\",\"output_intermediate_dir\":\"/opt/ml/output/intermediate\",\"resource_config\":{\"current_host\":\"algo-1-kyfef\",\"hosts\":[\"algo-1-kyfef\"]},\"user_entry_point\":\"transfer_learning.py\"}', 'SM_USER_ARGS': '[\"--auto_stack\",\"True\",\"--eval_metric\",\"auto\",\"--num_bag_folds\",\"0\",\"--num_bag_sets\",\"1\",\"--num_stack_levels\",\"0\",\"--presets\",\"medium_quality\",\"--refit_full\",\"False\",\"--save_space\",\"False\",\"--set_best_to_refit_full\",\"False\",\"--verbosity\",\"2\"]', 'SM_OUTPUT_INTERMEDIATE_DIR': '/opt/ml/output/intermediate', 'SM_CHANNEL_TRAIN': '/opt/ml/input/data/train', 'SM_HP_EVAL_METRIC': 'auto', 'SM_HP_PRESETS': 'medium_quality', 'SM_HP_AUTO_STACK': 'True', 'SM_HP_NUM_BAG_FOLDS': '0', 'SM_HP_NUM_BAG_SETS': '1', 'SM_HP_NUM_STACK_LEVELS': '0', 'SM_HP_REFIT_FULL': 'False', 'SM_HP_SET_BEST_TO_REFIT_FULL': 'False', 'SM_HP_SAVE_SPACE': 'False', 'SM_HP_VERBOSITY': '2'}\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m 2022-08-15 20:29:34,961 sagemaker-training-toolkit INFO No GPUs detected (normal if no gpus installed)\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m 2022-08-15 20:29:34,961 sagemaker-training-toolkit INFO Failed to parse hyperparameter eval_metric value auto to Json.\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m Returning the value itself\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m 2022-08-15 20:29:34,961 sagemaker-training-toolkit INFO Failed to parse hyperparameter presets value medium_quality to Json.\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m Returning the value itself\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m 2022-08-15 20:29:34,961 sagemaker-training-toolkit INFO Failed to parse hyperparameter auto_stack value True to Json.\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m Returning the value itself\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m 2022-08-15 20:29:34,961 sagemaker-training-toolkit INFO Failed to parse hyperparameter refit_full value False to Json.\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m Returning the value itself\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m 2022-08-15 20:29:34,961 sagemaker-training-toolkit INFO Failed to parse hyperparameter set_best_to_refit_full value False to Json.\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m Returning the value itself\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m 2022-08-15 20:29:34,961 sagemaker-training-toolkit INFO Failed to parse hyperparameter save_space value False to Json.\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m Returning the value itself\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m 2022-08-15 20:29:34,973 sagemaker-training-toolkit INFO No GPUs detected (normal if no gpus installed)\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m 2022-08-15 20:29:34,973 sagemaker-training-toolkit INFO Failed to parse hyperparameter eval_metric value auto to Json.\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m Returning the value itself\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m 2022-08-15 20:29:34,973 sagemaker-training-toolkit INFO Failed to parse hyperparameter presets value medium_quality to Json.\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m Returning the value itself\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m 2022-08-15 20:29:34,974 sagemaker-training-toolkit INFO Failed to parse hyperparameter auto_stack value True to Json.\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m Returning the value itself\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m 2022-08-15 20:29:34,974 sagemaker-training-toolkit INFO Failed to parse hyperparameter refit_full value False to Json.\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m Returning the value itself\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m 2022-08-15 20:29:34,974 sagemaker-training-toolkit INFO Failed to parse hyperparameter set_best_to_refit_full value False to Json.\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m Returning the value itself\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m 2022-08-15 20:29:34,974 sagemaker-training-toolkit INFO Failed to parse hyperparameter save_space value False to Json.\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m Returning the value itself\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m 2022-08-15 20:29:34,986 sagemaker-training-toolkit INFO No GPUs detected (normal if no gpus installed)\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m 2022-08-15 20:29:34,986 sagemaker-training-toolkit INFO Failed to parse hyperparameter eval_metric value auto to Json.\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m Returning the value itself\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m 2022-08-15 20:29:34,986 sagemaker-training-toolkit INFO Failed to parse hyperparameter presets value medium_quality to Json.\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m Returning the value itself\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m 2022-08-15 20:29:34,986 sagemaker-training-toolkit INFO Failed to parse hyperparameter auto_stack value True to Json.\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m Returning the value itself\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m 2022-08-15 20:29:34,986 sagemaker-training-toolkit INFO Failed to parse hyperparameter refit_full value False to Json.\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m Returning the value itself\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m 2022-08-15 20:29:34,986 sagemaker-training-toolkit INFO Failed to parse hyperparameter set_best_to_refit_full value False to Json.\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m Returning the value itself\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m 2022-08-15 20:29:34,986 sagemaker-training-toolkit INFO Failed to parse hyperparameter save_space value False to Json.\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m Returning the value itself\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m 2022-08-15 20:29:34,996 sagemaker-training-toolkit INFO Invoking user script\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m \n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m Training Env:\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m \n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m {\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m \"additional_framework_parameters\": {},\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m \"channel_input_dirs\": {\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m \"train\": \"/opt/ml/input/data/train\"\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m },\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m \"current_host\": \"algo-1-kyfef\",\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m \"framework_module\": \"sagemaker_mxnet_container.training:main\",\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m \"hosts\": [\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m \"algo-1-kyfef\"\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m ],\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m \"hyperparameters\": {\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m \"eval_metric\": \"auto\",\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m \"presets\": \"medium_quality\",\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m \"auto_stack\": \"True\",\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m \"num_bag_folds\": 0,\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m \"num_bag_sets\": 1,\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m \"num_stack_levels\": 0,\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m \"refit_full\": \"False\",\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m \"set_best_to_refit_full\": \"False\",\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m \"save_space\": \"False\",\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m \"verbosity\": 2\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m },\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m \"input_config_dir\": \"/opt/ml/input/config\",\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m \"input_data_config\": {\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m \"train\": {\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m \"TrainingInputMode\": \"File\"\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m }\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m },\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m \"input_dir\": \"/opt/ml/input\",\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m \"is_master\": true,\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m \"job_name\": \"autogluon-autogluon-regression-ensemble-2022-08-15-20-29-33-019\",\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m \"log_level\": 20,\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m \"master_hostname\": \"algo-1-kyfef\",\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m \"model_dir\": \"/opt/ml/model\",\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m \"module_dir\": \"s3://yudong-data/autogluon-autogluon-regression-ensemble-2022-08-15-20-29-33-019/source/sourcedir.tar.gz\",\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m \"module_name\": \"transfer_learning\",\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m \"network_interface_name\": \"eth0\",\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m \"num_cpus\": 16,\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m \"num_gpus\": 0,\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m \"output_data_dir\": \"/opt/ml/output/data\",\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m \"output_dir\": \"/opt/ml/output\",\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m \"output_intermediate_dir\": \"/opt/ml/output/intermediate\",\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m \"resource_config\": {\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m \"current_host\": \"algo-1-kyfef\",\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m \"hosts\": [\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m \"algo-1-kyfef\"\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m ]\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m },\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m \"user_entry_point\": \"transfer_learning.py\"\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m }\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m \n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m Environment variables:\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m \n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m SM_HOSTS=[\"algo-1-kyfef\"]\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m SM_NETWORK_INTERFACE_NAME=eth0\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m SM_HPS={\"auto_stack\":\"True\",\"eval_metric\":\"auto\",\"num_bag_folds\":0,\"num_bag_sets\":1,\"num_stack_levels\":0,\"presets\":\"medium_quality\",\"refit_full\":\"False\",\"save_space\":\"False\",\"set_best_to_refit_full\":\"False\",\"verbosity\":2}\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m SM_USER_ENTRY_POINT=transfer_learning.py\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m SM_FRAMEWORK_PARAMS={}\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m SM_RESOURCE_CONFIG={\"current_host\":\"algo-1-kyfef\",\"hosts\":[\"algo-1-kyfef\"]}\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m SM_INPUT_DATA_CONFIG={\"train\":{\"TrainingInputMode\":\"File\"}}\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m SM_OUTPUT_DATA_DIR=/opt/ml/output/data\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m SM_CHANNELS=[\"train\"]\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m SM_CURRENT_HOST=algo-1-kyfef\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m SM_MODULE_NAME=transfer_learning\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m SM_LOG_LEVEL=20\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m SM_FRAMEWORK_MODULE=sagemaker_mxnet_container.training:main\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m SM_INPUT_DIR=/opt/ml/input\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m SM_INPUT_CONFIG_DIR=/opt/ml/input/config\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m SM_OUTPUT_DIR=/opt/ml/output\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m SM_NUM_CPUS=16\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m SM_NUM_GPUS=0\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m SM_MODEL_DIR=/opt/ml/model\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m SM_MODULE_DIR=s3://yudong-data/autogluon-autogluon-regression-ensemble-2022-08-15-20-29-33-019/source/sourcedir.tar.gz\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m SM_TRAINING_ENV={\"additional_framework_parameters\":{},\"channel_input_dirs\":{\"train\":\"/opt/ml/input/data/train\"},\"current_host\":\"algo-1-kyfef\",\"framework_module\":\"sagemaker_mxnet_container.training:main\",\"hosts\":[\"algo-1-kyfef\"],\"hyperparameters\":{\"auto_stack\":\"True\",\"eval_metric\":\"auto\",\"num_bag_folds\":0,\"num_bag_sets\":1,\"num_stack_levels\":0,\"presets\":\"medium_quality\",\"refit_full\":\"False\",\"save_space\":\"False\",\"set_best_to_refit_full\":\"False\",\"verbosity\":2},\"input_config_dir\":\"/opt/ml/input/config\",\"input_data_config\":{\"train\":{\"TrainingInputMode\":\"File\"}},\"input_dir\":\"/opt/ml/input\",\"is_master\":true,\"job_name\":\"autogluon-autogluon-regression-ensemble-2022-08-15-20-29-33-019\",\"log_level\":20,\"master_hostname\":\"algo-1-kyfef\",\"model_dir\":\"/opt/ml/model\",\"module_dir\":\"s3://yudong-data/autogluon-autogluon-regression-ensemble-2022-08-15-20-29-33-019/source/sourcedir.tar.gz\",\"module_name\":\"transfer_learning\",\"network_interface_name\":\"eth0\",\"num_cpus\":16,\"num_gpus\":0,\"output_data_dir\":\"/opt/ml/output/data\",\"output_dir\":\"/opt/ml/output\",\"output_intermediate_dir\":\"/opt/ml/output/intermediate\",\"resource_config\":{\"current_host\":\"algo-1-kyfef\",\"hosts\":[\"algo-1-kyfef\"]},\"user_entry_point\":\"transfer_learning.py\"}\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m SM_USER_ARGS=[\"--auto_stack\",\"True\",\"--eval_metric\",\"auto\",\"--num_bag_folds\",\"0\",\"--num_bag_sets\",\"1\",\"--num_stack_levels\",\"0\",\"--presets\",\"medium_quality\",\"--refit_full\",\"False\",\"--save_space\",\"False\",\"--set_best_to_refit_full\",\"False\",\"--verbosity\",\"2\"]\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m SM_OUTPUT_INTERMEDIATE_DIR=/opt/ml/output/intermediate\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m SM_CHANNEL_TRAIN=/opt/ml/input/data/train\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m SM_HP_EVAL_METRIC=auto\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m SM_HP_PRESETS=medium_quality\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m SM_HP_AUTO_STACK=True\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m SM_HP_NUM_BAG_FOLDS=0\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m SM_HP_NUM_BAG_SETS=1\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m SM_HP_NUM_STACK_LEVELS=0\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m SM_HP_REFIT_FULL=False\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m SM_HP_SET_BEST_TO_REFIT_FULL=False\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m SM_HP_SAVE_SPACE=False\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m SM_HP_VERBOSITY=2\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m PYTHONPATH=/opt/ml/code:/usr/local/bin:/usr/local/lib/python38.zip:/usr/local/lib/python3.8:/usr/local/lib/python3.8/lib-dynload:/usr/local/lib/python3.8/site-packages\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m \n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m Invoking script with the following command:\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m \n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m /usr/local/bin/python3.8 transfer_learning.py --auto_stack True --eval_metric auto --num_bag_folds 0 --num_bag_sets 1 --num_stack_levels 0 --presets medium_quality --refit_full False --save_space False --set_best_to_refit_full False --verbosity 2\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m \n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m \n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m ==== printing args ====\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m Namespace(auto_stack='True', current_host='algo-1-kyfef', eval_metric='auto', hosts=['algo-1-kyfef'], model_dir='/opt/ml/model', n_gpus=0, num_bag_folds='0', num_bag_sets=1, num_stack_levels=0, presets='medium_quality', pretrained_model=None, refit_full='False', save_space='False', set_best_to_refit_full='False', train='/opt/ml/input/data/train', verbosity=2)\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m /opt/ml/input/data/train\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m /opt/ml/input/data/train/train.csv\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m HUFL ... OT\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m 0 6.765 ... 12.662000\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m 1 15.004 ... 11.607000\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m 2 10.918 ... 17.445999\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m 3 8.975 ... 19.979000\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m 4 19.357 ... 40.450001\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m \n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m [5 rows x 7 columns]\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m Warning: path already exists! This predictor may overwrite an existing predictor! path=\"/opt/ml/model\"\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m Presets specified: ['medium_quality']\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m Beginning AutoGluon training ...\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m AutoGluon will save models to \"/opt/ml/model/\"\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m AutoGluon Version: 0.4.0\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m Python Version: 3.8.12\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m Operating System: Linux\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m Train Data Rows: 11671\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m Train Data Columns: 6\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m Label Column: OT\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m Preprocessing data ...\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m Using Feature Generators to preprocess the data ...\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m Fitting AutoMLPipelineFeatureGenerator...\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m \tAvailable Memory: 60212.54 MB\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m \tTrain Data (Original) Memory Usage: 0.56 MB (0.0% of available memory)\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m \tInferring data type of each feature based on column values. Set feature_metadata_in to manually specify special dtypes of the features.\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m \tStage 1 Generators:\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m \t\tFitting AsTypeFeatureGenerator...\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m \tStage 2 Generators:\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m \t\tFitting FillNaFeatureGenerator...\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m \tStage 3 Generators:\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m \t\tFitting IdentityFeatureGenerator...\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m \tStage 4 Generators:\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m \t\tFitting DropUniqueFeatureGenerator...\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m \tTypes of features in original data (raw dtype, special dtypes):\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m \t\t('float', []) : 6 | ['HUFL', 'HULL', 'MUFL', 'MULL', 'LUFL', ...]\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m \tTypes of features in processed data (raw dtype, special dtypes):\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m \t\t('float', []) : 6 | ['HUFL', 'HULL', 'MUFL', 'MULL', 'LUFL', ...]\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m \t0.0s = Fit runtime\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m \t6 features in original data used to generate 6 features in processed data.\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m \tTrain Data (Processed) Memory Usage: 0.56 MB (0.0% of available memory)\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m Data preprocessing and feature engineering runtime = 0.03s ...\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m AutoGluon will gauge predictive performance using evaluation metric: 'root_mean_squared_error'\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m \tTo change this, specify the eval_metric parameter of Predictor()\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m Automatically generating train/validation split with holdout_frac=0.1, Train Rows: 10503, Val Rows: 1168\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m Fitting 11 L1 models ...\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m Fitting model: KNeighborsUnif ...\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m \t-6.9259\t = Validation score (root_mean_squared_error)\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m \t0.01s\t = Training runtime\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m \t0.02s\t = Validation runtime\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m Fitting model: KNeighborsDist ...\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m \t-6.8851\t = Validation score (root_mean_squared_error)\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m \t0.01s\t = Training runtime\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m \t0.02s\t = Validation runtime\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m Fitting model: LightGBMXT ...\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m [1000]\tvalid_set's rmse: 6.36231\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m [2000]\tvalid_set's rmse: 6.28583\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m [3000]\tvalid_set's rmse: 6.25816\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m [4000]\tvalid_set's rmse: 6.25372\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m \t-6.2512\t = Validation score (root_mean_squared_error)\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m \t3.49s\t = Training runtime\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m \t0.03s\t = Validation runtime\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m Fitting model: LightGBM ...\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m \t-6.2815\t = Validation score (root_mean_squared_error)\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m \t0.65s\t = Training runtime\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m \t0.01s\t = Validation runtime\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m Fitting model: RandomForestMSE ...\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m \t-6.3053\t = Validation score (root_mean_squared_error)\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m \t1.41s\t = Training runtime\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m \t0.07s\t = Validation runtime\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m Fitting model: CatBoost ...\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m \t-6.2189\t = Validation score (root_mean_squared_error)\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m \t10.98s\t = Training runtime\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m \t0.0s\t = Validation runtime\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m Fitting model: ExtraTreesMSE ...\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m \t-6.2679\t = Validation score (root_mean_squared_error)\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m \t0.82s\t = Training runtime\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m \t0.07s\t = Validation runtime\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m Fitting model: NeuralNetFastAI ...\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m \t-6.2829\t = Validation score (root_mean_squared_error)\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m \t9.91s\t = Training runtime\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m \t0.02s\t = Validation runtime\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m Fitting model: XGBoost ...\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m \t-6.2431\t = Validation score (root_mean_squared_error)\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m \t0.81s\t = Training runtime\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m \t0.01s\t = Validation runtime\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m Fitting model: NeuralNetTorch ...\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m \t-6.2089\t = Validation score (root_mean_squared_error)\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m \t35.73s\t = Training runtime\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m \t0.02s\t = Validation runtime\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m Fitting model: LightGBMLarge ...\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m \t-6.2788\t = Validation score (root_mean_squared_error)\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m \t1.86s\t = Training runtime\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m \t0.04s\t = Validation runtime\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m Fitting model: WeightedEnsemble_L2 ...\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m \t-5.9727\t = Validation score (root_mean_squared_error)\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m \t0.39s\t = Training runtime\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m \t0.0s\t = Validation runtime\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m AutoGluon training complete, total runtime = 68.35s ... Best model: \"WeightedEnsemble_L2\"\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m TabularPredictor saved. To load, use: predictor = TabularPredictor.load(\"/opt/ml/model/\")\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef |\u001b[0m 2022-08-15 20:30:44,844 sagemaker-training-toolkit INFO Reporting training SUCCESS\n", "\u001b[36ms25n1fxoqk-algo-1-kyfef exited with code 0\n", "\u001b[0mAborting on container exit...\n", "===== Job Complete =====\n" ] } ], "source": [ "from sagemaker.estimator import Estimator\n", "from sagemaker.utils import name_from_base\n", "\n", "training_job_name = name_from_base(f\"autogluon-{train_model_id}-training\")\n", "\n", "# Create SageMaker Estimator instance\n", "tabular_estimator = Estimator(\n", " role=aws_role,\n", " image_uri=train_image_uri,\n", " source_dir='source',\n", "# model_uri=train_model_uri,\n", " entry_point=\"transfer_learning.py\",\n", " instance_count=1,\n", " instance_type=training_instance_type,\n", " max_run=360000,\n", " hyperparameters=hyperparameters,\n", " output_path=s3_output_location,\n", ")\n", "\n", "# Launch a SageMaker Training job by passing s3 path of the training data\n", "tabular_estimator.fit({\"train\": training_dataset_s3_path}, logs=False, job_name=training_job_name, wait=False)" ] }, { "cell_type": "markdown", "id": "d2ac099d", "metadata": { "tags": [] }, "source": [ "### 실험 결과 보기\n", "위의 실험한 결과를 확인 합니다.\n", "- 각각의 훈련잡의 시도에 대한 훈련 사용 데이터, 모델 입력 하이퍼 파라미터, 모델 평가 지표, 모델 아티펙트 결과 위치 등의 확인이 가능합니다.\n", "- **아래의 모든 내용은 SageMaker Studio 를 통해서 직관적으로 확인이 가능합니다.**" ] }, { "cell_type": "code", "execution_count": null, "id": "1e5e7bb6", "metadata": {}, "outputs": [], "source": [ "!rm -rf ./autogluon/\n", "!mkdir -p ./autogluon/result\n", "!aws s3 cp {ag_estimator.model_data} ./autogluon/" ] }, { "cell_type": "code", "execution_count": null, "id": "a88c3a0a", "metadata": {}, "outputs": [], "source": [ "!ls -alF ./autogluon/model.tar.gz" ] }, { "cell_type": "code", "execution_count": null, "id": "d7202da0", "metadata": {}, "outputs": [], "source": [ "!tar -xzf ./autogluon/model.tar.gz -C ./autogluon/result/" ] }, { "cell_type": "markdown", "id": "61fc568e", "metadata": {}, "source": [ "### Endpoint Deployment" ] }, { "cell_type": "code", "execution_count": null, "id": "e59b934e", "metadata": {}, "outputs": [], "source": [ "instance_type = \"ml.m5.2xlarge\"\n", "# instance_type = 'local'" ] }, { "cell_type": "code", "execution_count": null, "id": "7e9eef80", "metadata": {}, "outputs": [], "source": [ "if instance_type == 'local':\n", " from sagemaker.local import LocalSession\n", " sagemaker_session = LocalSession()\n", " sagemaker_session.config = {'local': {'local_code': True}}\n", "else:\n", " sess = boto3.Session()\n", " sagemaker_session = sagemaker.Session()" ] }, { "cell_type": "code", "execution_count": null, "id": "65914817", "metadata": {}, "outputs": [], "source": [ "model = AutoGluonInferenceModel(\n", " source_dir=os.getcwd() + \"/src\",\n", " entry_point=\"autogluon_serve.py\",\n", " model_data=ag_estimator.model_data,\n", " instance_type=instance_type,\n", " role=role,\n", " sagemaker_session=sagemaker_session,\n", " # region=region,\n", " framework_version=\"0.4\",\n", " py_version=\"py38\",\n", " predictor_cls=AutoGluonTabularPredictor\n", ")" ] }, { "cell_type": "code", "execution_count": null, "id": "09e2b8fb", "metadata": {}, "outputs": [], "source": [ "from sagemaker.serializers import CSVSerializer\n", "\n", "predictor = model.deploy(\n", " initial_instance_count=1, serializer=CSVSerializer(), instance_type=instance_type\n", ")" ] }, { "cell_type": "markdown", "id": "679e9ba2", "metadata": {}, "source": [ "### Predict on unlabeled test data\n", "\n", "Remove target variable (`fraud`) from the data and get predictions for a sample of 100 rows using the deployed endpoint." ] }, { "cell_type": "code", "execution_count": null, "id": "7f228dbb", "metadata": {}, "outputs": [], "source": [ "df = pd.read_csv(\"../data/dataset/test.csv\")\n", "data = df.drop(columns=\"fraud\")[:100].values" ] }, { "cell_type": "code", "execution_count": null, "id": "8f2d39f6", "metadata": {}, "outputs": [], "source": [ "preds = predictor.predict(data)\n", "pred_df = pd.DataFrame(json.loads(preds))" ] }, { "cell_type": "code", "execution_count": null, "id": "17c46e3f", "metadata": {}, "outputs": [], "source": [ "pred_df['fraud'].reset_index(drop=True, inplace=True)\n", "df[\"fraud\"][:len(pred_df)].reset_index(drop=True, inplace=True)" ] }, { "cell_type": "code", "execution_count": null, "id": "1af3f10f", "metadata": {}, "outputs": [], "source": [ "p = pd.DataFrame({\"preds\": pred_df['fraud'], \"actual\": df[\"fraud\"][: len(pred_df)]})\n", "p.head()" ] }, { "cell_type": "code", "execution_count": null, "id": "71eea765", "metadata": {}, "outputs": [], "source": [ "print(f\"{(p.preds==p.actual).astype(int).sum()}/{len(p)} are correct\")" ] }, { "cell_type": "markdown", "id": "16e992d4", "metadata": {}, "source": [ "### Cleanup Endpoint" ] }, { "cell_type": "code", "execution_count": null, "id": "9fe5e513", "metadata": {}, "outputs": [], "source": [ "# predictor.delete_endpoint()" ] }, { "cell_type": "markdown", "id": "b4886486", "metadata": { "tags": [] }, "source": [ "# Batch Transform\n", "\n", "학습된 모델을 호스트된 엔드포인트에 배포하는 것은 출시 이후 SageMaker에서 사용할 수 있으며 웹 사이트나 모바일 앱과 같은 서비스에 실시간 예측을 제공하는 좋은 방법입니다. 그러나 지연 시간을 최소화하는 것이 문제가 되지 않는 대규모 데이터 세트에서 학습된 모델에서 예측을 생성하는 것이 목표라면 배치 변환 기능이 더 쉽고, 더 확장 가능하며, 더 적절할 수 있다.\n", "\n", "[Read more about Batch Transform](https://docs.aws.amazon.com/sagemaker/latest/dg/batch-transform.html)." ] }, { "cell_type": "code", "execution_count": null, "id": "678d2f5c", "metadata": {}, "outputs": [], "source": [ "instance_type = \"ml.m5.2xlarge\"" ] }, { "cell_type": "code", "execution_count": null, "id": "4aec16da", "metadata": {}, "outputs": [], "source": [ "# model = AutoGluonInferenceModel(\n", "# source_dir=os.getcwd() + \"/src\",\n", "# entry_point=\"autogluon_serve.py\",\n", "# model_data=ag_estimator.model_data,\n", "# instance_type=instance_type,\n", "# role=role,\n", "# sagemaker_session=sagemaker_session,\n", "# region=region,\n", "# framework_version=\"0.4\",\n", "# py_version=\"py38\", \n", "# predictor_cls=AutoGluonTabularPredictor,\n", "# )" ] }, { "cell_type": "code", "execution_count": null, "id": "3bbec1f9", "metadata": {}, "outputs": [], "source": [ "transformer = model.transformer(\n", " instance_count=1,\n", " instance_type=instance_type,\n", " strategy=\"MultiRecord\",\n", " max_payload=6,\n", " max_concurrent_transforms=1,\n", " output_path=output_path,\n", " accept=\"application/json\",\n", " assemble_with=\"Line\",\n", ")\n" ] }, { "cell_type": "markdown", "id": "8e89ba77", "metadata": {}, "source": [ "Prepare data for batch transform" ] }, { "cell_type": "code", "execution_count": null, "id": "56558e0e", "metadata": {}, "outputs": [], "source": [ "pd.read_csv(f\"../data/dataset/test.csv\")[:100].to_csv(\"../data/dataset/test_no_header.csv\", header=False, index=False)" ] }, { "cell_type": "code", "execution_count": null, "id": "5cad618f", "metadata": {}, "outputs": [], "source": [ "test_input = transformer.sagemaker_session.upload_data(\n", " path=os.path.join(\"../data/dataset\", \"test_no_header.csv\"), key_prefix=f\"{bucket}/autogluon/dataset\"\n", ")\n", "test_input" ] }, { "cell_type": "code", "execution_count": null, "id": "50354cdc", "metadata": { "scrolled": true }, "outputs": [], "source": [ "transformer.transform(\n", " test_input,\n", " input_filter=\"$[1:]\", # filter-out target variable\n", " split_type=\"Line\",\n", " content_type=\"text/csv\",\n", " output_filter=\"$['fraud']\", # keep only prediction class in the output\n", ")\n", "\n", "transformer.wait()" ] }, { "cell_type": "markdown", "id": "7d16806f", "metadata": {}, "source": [ "batch transform 결과를 다운로드 받습니다." ] }, { "cell_type": "code", "execution_count": null, "id": "757e5717", "metadata": {}, "outputs": [], "source": [ "!rm -rf ./autogluon_batch_result\n", "!mkdir ./autogluon_batch_result" ] }, { "cell_type": "code", "execution_count": null, "id": "44cc858b", "metadata": {}, "outputs": [], "source": [ "transformer.output_path" ] }, { "cell_type": "code", "execution_count": null, "id": "83124688", "metadata": {}, "outputs": [], "source": [ "!aws s3 cp {transformer.output_path}/test_no_header.csv.out ./autogluon_batch_result/" ] }, { "cell_type": "code", "execution_count": null, "id": "ed5fae85", "metadata": {}, "outputs": [], "source": [ "p = pd.concat(\n", " [\n", " pd.read_json(\"./autogluon_batch_result/test_no_header.csv.out\", orient=\"index\")\n", " .sort_index()\n", " .rename(columns={0: \"preds\"}),\n", " pd.read_csv(\"../data/dataset/test.csv\")[[\"fraud\"]].iloc[:100].rename(columns={\"fraud\": \"actual\"}),\n", " ],\n", " axis=1,\n", ")\n", "p.head()" ] }, { "cell_type": "code", "execution_count": null, "id": "54ba65bc", "metadata": {}, "outputs": [], "source": [ "print(f\"{(p.preds==p.actual).astype(int).sum()}/{len(p)} are correct\")" ] }, { "cell_type": "markdown", "id": "863d6129", "metadata": { "tags": [] }, "source": [ "### Processing Evaluation 하기\n", "SageMaker Processing을 이용하여 Evalution을 수행하는 코드를 동작할 수 있습니다. MLOps에서 Processing을 적용하면 전처리, Evaluation 등을 serverless로 동작할 수 있습니다." ] }, { "cell_type": "code", "execution_count": null, "id": "cc0717c0", "metadata": {}, "outputs": [], "source": [ "from sagemaker.processing import FrameworkProcessor\n", "from sagemaker.processing import ProcessingInput, ProcessingOutput\n", "from sagemaker.estimator import Framework" ] }, { "cell_type": "code", "execution_count": null, "id": "f52ea248", "metadata": {}, "outputs": [], "source": [ "instance_count = 1\n", "instance_type = \"ml.m5.large\"\n", "# instance_type = 'local'" ] }, { "cell_type": "code", "execution_count": null, "id": "2f7400cd", "metadata": {}, "outputs": [], "source": [ "from sagemaker import image_uris\n", "\n", "image_uri = image_uris.retrieve(\n", " \"autogluon\",\n", " region=region,\n", " version=\"0.4\",\n", " py_version=\"py38\",\n", " image_scope=\"training\",\n", " instance_type=instance_type,\n", ")\n", "image_uri\n" ] }, { "cell_type": "code", "execution_count": null, "id": "7e7ad517", "metadata": {}, "outputs": [], "source": [ "script_eval = FrameworkProcessor(\n", " AutoGluonFramework,\n", " framework_version=\"0.4\",\n", " role=role,\n", " py_version=\"py38\",\n", " image_uri=image_uri,\n", " instance_type=instance_type,\n", " instance_count=instance_count\n", ")" ] }, { "cell_type": "code", "execution_count": null, "id": "90535db2", "metadata": {}, "outputs": [], "source": [ "detect_outputpath = f's3://{bucket}/autogluon/processing'" ] }, { "cell_type": "code", "execution_count": null, "id": "f4ed04de", "metadata": {}, "outputs": [], "source": [ "source_dir='src'\n", "\n", "if instance_type == 'local':\n", " from sagemaker.local import LocalSession\n", " from pathlib import Path\n", " \n", " sagemaker_session = LocalSession()\n", " sagemaker_session.config = {'local': {'local_code': True}}\n", " source_dir = f'{Path.cwd()}/src'\n", " s3_test_path=f'../data/dataset/test.csv'\n", "else:\n", " sagemaker_session = sagemaker.Session()\n", " s3_test_path = data_path + '/test.csv'" ] }, { "cell_type": "code", "execution_count": null, "id": "72143d60", "metadata": {}, "outputs": [], "source": [ "create_experiment(experiment_name)\n", "job_name = create_trial(experiment_name)\n", "\n", "script_eval.run(\n", " code=\"autogluon_evaluation.py\",\n", " source_dir=source_dir,\n", " inputs=[ProcessingInput(source=s3_test_path, input_name=\"test_data\", destination=\"/opt/ml/processing/test\"),\n", " ProcessingInput(source=ag_estimator.model_data, input_name=\"model_weight\", destination=\"/opt/ml/processing/model\")\n", " ],\n", " outputs=[\n", " ProcessingOutput(source=\"/opt/ml/processing/output\", output_name='evaluation', destination=detect_outputpath + \"/\" + job_name),\n", " ],\n", " job_name=job_name,\n", " experiment_config={\n", " 'TrialName': job_name,\n", " 'TrialComponentDisplayName': job_name,\n", " },\n", " wait=False\n", ")" ] }, { "cell_type": "code", "execution_count": null, "id": "3f7d2fff", "metadata": {}, "outputs": [], "source": [ "script_eval.latest_job.wait()" ] }, { "cell_type": "markdown", "id": "c2a16343", "metadata": {}, "source": [ "### Code repository 생성 및 push\n", "현재 사용하는 노트북의 iam role에 IAMFullAccess을 추가한 이후에 아래 작업을 수행합니다." ] }, { "cell_type": "code", "execution_count": null, "id": "e862bfa6", "metadata": {}, "outputs": [], "source": [ "from sagemaker import get_execution_role" ] }, { "cell_type": "code", "execution_count": null, "id": "df35f950", "metadata": {}, "outputs": [], "source": [ "iam_client = boto3.client('iam')\n", "\n", "role=get_execution_role()\n", "base_role_name=role.split('/')[-1]\n", "\n", "iam_client.attach_role_policy(\n", " RoleName=base_role_name,\n", " PolicyArn='arn:aws:iam::aws:policy/AWSCodeCommitFullAccess'\n", ")" ] }, { "cell_type": "code", "execution_count": null, "id": "0475cbda", "metadata": {}, "outputs": [], "source": [ "codecommit = boto3.client('codecommit')\n", "repository_name = 'autogluon_code'\n", "\n", "try:\n", " response = codecommit.create_repository(\n", " repositoryName=repository_name,\n", " repositoryDescription='Data Scientists share their training code using this Repository'\n", " )\n", "except:\n", " \n", " print(\"Repository already exists\")\n", " response = codecommit.get_repository(\n", " repositoryName=repository_name\n", " )" ] }, { "cell_type": "code", "execution_count": null, "id": "6171b328", "metadata": {}, "outputs": [], "source": [ "codecommit_repo = response['repositoryMetadata']['cloneUrlHttp']\n", "codecommit_repo" ] }, { "cell_type": "code", "execution_count": null, "id": "26a362be", "metadata": {}, "outputs": [], "source": [ "!git init\n", "!git remote add repo_codecommit $codecommit_repo\n", "!git checkout -b main\n", "!git add ./config ./src ./1.SageMaker-Training+Experiments+Processing-AutoGluon.ipynb\n", "!git commit -m \"autogluon-update\"\n", "!git push --set-upstream repo_codecommit main" ] }, { "cell_type": "code", "execution_count": null, "id": "726050b7", "metadata": {}, "outputs": [], "source": [] } ], "metadata": { "instance_type": "ml.t3.medium", "kernelspec": { "display_name": "conda_amazonei_mxnet_p36", "language": "python", "name": "conda_amazonei_mxnet_p36" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.6.13" } }, "nbformat": 4, "nbformat_minor": 5 }