Metadata-Version: 2.1 Name: sagemaker-experiments Version: 0.1.13 Summary: Open source library for Experiment Tracking in SageMaker Jobs and Notebooks Home-page: https://github.com/aws/sagemaker-experiment-tracking/ Author: Amazon Web Services License: Apache License 2.0 Keywords: ML Amazon AWS AI Tensorflow MXNet Platform: UNKNOWN Classifier: Development Status :: 5 - Production/Stable Classifier: Intended Audience :: Developers Classifier: Natural Language :: English Classifier: License :: OSI Approved :: Apache Software License Classifier: Programming Language :: Python Classifier: Programming Language :: Python :: 3.5 Classifier: Programming Language :: Python :: 3.6 Classifier: Programming Language :: Python :: 3.7 Description-Content-Type: text/x-rst Requires-Dist: boto3 (>=1.12.8) Provides-Extra: test Requires-Dist: tox (==3.13.1) ; extra == 'test' Requires-Dist: flake8 ; extra == 'test' Requires-Dist: pytest (==4.4.1) ; extra == 'test' Requires-Dist: pytest-cov ; extra == 'test' Requires-Dist: pytest-coverage ; extra == 'test' Requires-Dist: pytest-rerunfailures ; extra == 'test' Requires-Dist: pytest-xdist ; extra == 'test' Requires-Dist: docker ; extra == 'test' .. image:: https://github.com/aws/sagemaker-experiments/raw/master/branding/icon/sagemaker-banner.png :height: 100px :alt: SageMaker :target: https://aws.amazon.com/sagemaker/ ================================ SageMaker Experiments Python SDK ================================ .. image:: https://img.shields.io/pypi/v/sagemaker-experiments.svg :target: https://pypi.python.org/pypi/sagemaker-experiments :alt: Latest Version .. image:: https://img.shields.io/pypi/pyversions/sagemaker-experiments.svg :target: https://pypi.python.org/pypi/sagemaker-experiments :alt: Supported Python Versions .. image:: https://img.shields.io/pypi/l/sagemaker-experiments :target: https://github.com/aws/sagemaker-experiments/blob/master/LICENSE :alt: License .. image:: https://img.shields.io/pypi/dm/sagemaker-experiments :target: https://pypi.python.org/pypi/sagemaker-experiments :alt: PyPI - Downloads .. image:: https://codecov.io/gh/aws/sagemaker-experiments/branch/master/graph/badge.svg :target: https://codecov.io/gh/aws/sagemaker-experiments :alt: CodeCov .. image:: https://img.shields.io/pypi/status/sagemaker-experiments :target: https://github.com/aws/sagemaker-experiments :alt: PyPI - Status .. image:: https://img.shields.io/pypi/format/coverage.svg :target: https://pypi.org/project/coverage/ :alt: Kit format .. image:: https://img.shields.io/github/workflow/status/aws/sagemaker-experiments/Main :target: https://github.com/aws/sagemaker-experiments/actions :alt: GitHub Workflow Status .. image:: https://img.shields.io/github/stars/aws/sagemaker-experiments.svg?logo=github :target: https://github.com/aws/sagemaker-experiments/stargazers :alt: Github stars .. image:: https://img.shields.io/github/forks/aws/sagemaker-experiments.svg?logo=github :target: https://github.com/aws/sagemaker-experiments/network/members :alt: Github forks .. image:: https://img.shields.io/github/contributors/aws/sagemaker-experiments.svg?logo=github :target: https://github.com/aws/sagemaker-experiments/graphs/contributors :alt: Contributors .. image:: https://img.shields.io/github/search/aws/sagemaker-experiments/sagemaker :target: https://github.com/aws/sagemaker-experiments :alt: GitHub search hit counter .. image:: https://img.shields.io/badge/code_style-black-000000.svg :target: https://github.com/python/black :alt: Code style: black .. image:: https://readthedocs.org/projects/sagemaker-experiments/badge/?version=latest :target: https://readthedocs.org/projects/sagemaker-experiments/ :alt: Read the Docs - Sagemaker Experiments .. image:: https://mybinder.org/badge_logo.svg :target: https://mybinder.org/v2/gh/awslabs/amazon-sagemaker-examples/master?filepath=sagemaker-experiments%2Fmnist-handwritten-digits-classification-experiment.ipynb Experiment tracking in SageMaker Training Jobs, Processing Jobs, and Notebooks. Overview -------- SageMaker Experiments is an AWS service for tracking machine learning Experiments. The SageMaker Experiments Python SDK is a high-level interface to this service that helps you track Experiment information using Python. Experiment tracking powers the machine learning integrated development environment `Amazon SageMaker Studio `_. For detailed API reference please go to: `Read the Docs `_ Concepts -------- - **Experiment**: A collection of related Trials. Add Trials to an Experiment that you wish to compare together. - **Trial**: A description of a multi-step machine learning workflow. Each step in the workflow is described by a Trial Component. There is no relationship between Trial Components such as ordering. - **Trial Component**: A description of a single step in a machine learning workflow. For example data cleaning, feature extraction, model training, model evaluation, etc... - **Tracker**: A Python context-manager for logging information about a single TrialComponent. For more information see `Amazon SageMaker Experiments - Organize, Track, and Compare Your Machine Learning Trainings `_ Using the SDK ------------- You can use this SDK to: - Manage Experiments, Trials, and Trial Components within Python scripts, programs, and notebooks. - Add tracking information to a SageMaker notebook, allowing you to model your notebook in SageMaker Experiments as a multi-step ML workflow. - Record experiment information from inside your running SageMaker Training and Processing Jobs. Installation ------------ .. code-block:: bash pip install sagemaker-experiments Examples -------- .. code-block:: python import boto3 import pickle, gzip, numpy, urllib.request, json import io import numpy as np import sagemaker.amazon.common as smac import sagemaker from sagemaker import get_execution_role from sagemaker import analytics from smexperiments import experiment # Specify training container from sagemaker.amazon.amazon_estimator import get_image_uri container = get_image_uri(boto3.Session().region_name, 'linear-learner') # Load the dataset urllib.request.urlretrieve("http://deeplearning.net/data/mnist/mnist.pkl.gz", "mnist.pkl.gz") with gzip.open('mnist.pkl.gz', 'rb') as f: train_set, valid_set, test_set = pickle.load(f, encoding='latin1') vectors = np.array([t.tolist() for t in train_set[0]]).astype('float32') labels = np.where(np.array([t.tolist() for t in train_set[1]]) == 0, 1, 0).astype('float32') buf = io.BytesIO() smac.write_numpy_to_dense_tensor(buf, vectors, labels) buf.seek(0) key = 'recordio-pb-data' bucket = '{YOUR-BUCKET}' prefix = 'sagemaker/DEMO-linear-mnist' boto3.resource('s3').Bucket(bucket).Object(os.path.join(prefix, 'train', key)).upload_fileobj(buf) s3_train_data = 's3://{}/{}/train/{}'.format(bucket, prefix, key) output_location = 's3://{}/{}/output'.format(bucket, prefix) my_experiment = experiment.Experiment.create(experiment_name='MNIST') my_trial = my_experiment.create_trial(trial_name='linear-learner') role = get_execution_role() sess = sagemaker.Session() linear = sagemaker.estimator.Estimator(container, role, train_instance_count=1, train_instance_type='ml.c4.xlarge', output_path=output_location, sagemaker_session=sess) linear.set_hyperparameters(feature_dim=784, predictor_type='binary_classifier', mini_batch_size=200) linear.fit(inputs={'train': s3_train_data}, experiment_config={ "ExperimentName": my_experiment.experiment_name, "TrialName": my_trial.trial_name, "TrialComponentDisplayName": "MNIST-linear-learner", },) trial_component_analytics = analytics.ExperimentAnalytics(experiment_name=my_experiment.experiment_name) analytic_table = trial_component_analytics.dataframe() analytic_table For more examples, check out: `sagemaker-experiments `_ in `AWS Labs Amazon SageMaker Examples `_. License ------- This library is licensed under the Apache 2.0 License. Running Tests ------------- **Unit Tests** .. code-block:: bash tox tests/unit **Integration Tests** To run the integration tests, the following prerequisites must be met: - AWS account credentials are available in the environment for the boto3 client to use. - The AWS account has an IAM role with SageMaker permissions. .. code-block:: bash tox tests/integ - Test against different regions .. code-block:: bash tox -e py37 -- --region cn-north-1 Generate Docs ------------- .. code-block:: bash tox -e docs