/**
* Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
* SPDX-License-Identifier: Apache-2.0.
*/
#pragma once
#include Contains information about the output location for the compiled model and the
* target device that the model runs on. TargetDevice
and
* TargetPlatform
are mutually exclusive, so you need to choose one
* between the two to specify your target device or platform. If you cannot find
* your device you want to use from the TargetDevice
list, use
* TargetPlatform
to describe the platform of your edge device and
* CompilerOptions
if there are specific settings that are required or
* recommended to use for particular TargetPlatform.See Also:
AWS
* API Reference
Identifies the S3 bucket where you want Amazon SageMaker to store the model
* artifacts. For example, s3://bucket-name/key-name-prefix
.
Identifies the S3 bucket where you want Amazon SageMaker to store the model
* artifacts. For example, s3://bucket-name/key-name-prefix
.
Identifies the S3 bucket where you want Amazon SageMaker to store the model
* artifacts. For example, s3://bucket-name/key-name-prefix
.
Identifies the S3 bucket where you want Amazon SageMaker to store the model
* artifacts. For example, s3://bucket-name/key-name-prefix
.
Identifies the S3 bucket where you want Amazon SageMaker to store the model
* artifacts. For example, s3://bucket-name/key-name-prefix
.
Identifies the S3 bucket where you want Amazon SageMaker to store the model
* artifacts. For example, s3://bucket-name/key-name-prefix
.
Identifies the S3 bucket where you want Amazon SageMaker to store the model
* artifacts. For example, s3://bucket-name/key-name-prefix
.
Identifies the S3 bucket where you want Amazon SageMaker to store the model
* artifacts. For example, s3://bucket-name/key-name-prefix
.
Identifies the target device or the machine learning instance that you want
* to run your model on after the compilation has completed. Alternatively, you can
* specify OS, architecture, and accelerator using TargetPlatform
* fields. It can be used instead of TargetPlatform
.
Currently ml_trn1
is available only in US East (N. Virginia)
* Region, and ml_inf2
is available only in US East (Ohio) Region.
Identifies the target device or the machine learning instance that you want
* to run your model on after the compilation has completed. Alternatively, you can
* specify OS, architecture, and accelerator using TargetPlatform
* fields. It can be used instead of TargetPlatform
.
Currently ml_trn1
is available only in US East (N. Virginia)
* Region, and ml_inf2
is available only in US East (Ohio) Region.
Identifies the target device or the machine learning instance that you want
* to run your model on after the compilation has completed. Alternatively, you can
* specify OS, architecture, and accelerator using TargetPlatform
* fields. It can be used instead of TargetPlatform
.
Currently ml_trn1
is available only in US East (N. Virginia)
* Region, and ml_inf2
is available only in US East (Ohio) Region.
Identifies the target device or the machine learning instance that you want
* to run your model on after the compilation has completed. Alternatively, you can
* specify OS, architecture, and accelerator using TargetPlatform
* fields. It can be used instead of TargetPlatform
.
Currently ml_trn1
is available only in US East (N. Virginia)
* Region, and ml_inf2
is available only in US East (Ohio) Region.
Identifies the target device or the machine learning instance that you want
* to run your model on after the compilation has completed. Alternatively, you can
* specify OS, architecture, and accelerator using TargetPlatform
* fields. It can be used instead of TargetPlatform
.
Currently ml_trn1
is available only in US East (N. Virginia)
* Region, and ml_inf2
is available only in US East (Ohio) Region.
Identifies the target device or the machine learning instance that you want
* to run your model on after the compilation has completed. Alternatively, you can
* specify OS, architecture, and accelerator using TargetPlatform
* fields. It can be used instead of TargetPlatform
.
Currently ml_trn1
is available only in US East (N. Virginia)
* Region, and ml_inf2
is available only in US East (Ohio) Region.
Contains information about a target platform that you want your model to run
* on, such as OS, architecture, and accelerators. It is an alternative of
* TargetDevice
.
The following examples show how to configure
* the TargetPlatform
and CompilerOptions
JSON strings
* for popular target platforms:
Raspberry Pi 3 Model B+
* "TargetPlatform": {"Os": "LINUX", "Arch": "ARM_EABIHF"},
* "CompilerOptions": {'mattr': ['+neon']}
Jetson * TX2
"TargetPlatform": {"Os": "LINUX", "Arch": "ARM64",
* "Accelerator": "NVIDIA"},
"CompilerOptions": {'gpu-code':
* 'sm_62', 'trt-ver': '6.0.1', 'cuda-ver': '10.0'}
EC2 * m5.2xlarge instance OS
"TargetPlatform": {"Os": "LINUX", "Arch":
* "X86_64", "Accelerator": "NVIDIA"},
"CompilerOptions":
* {'mcpu': 'skylake-avx512'}
RK3399
* "TargetPlatform": {"Os": "LINUX", "Arch": "ARM64", "Accelerator":
* "MALI"}
ARMv7 phone (CPU)
* "TargetPlatform": {"Os": "ANDROID", "Arch": "ARM_EABI"},
* "CompilerOptions": {'ANDROID_PLATFORM': 25, 'mattr': ['+neon']}
*
ARMv8 phone (CPU)
"TargetPlatform": {"Os":
* "ANDROID", "Arch": "ARM64"},
"CompilerOptions":
* {'ANDROID_PLATFORM': 29}
Contains information about a target platform that you want your model to run
* on, such as OS, architecture, and accelerators. It is an alternative of
* TargetDevice
.
The following examples show how to configure
* the TargetPlatform
and CompilerOptions
JSON strings
* for popular target platforms:
Raspberry Pi 3 Model B+
* "TargetPlatform": {"Os": "LINUX", "Arch": "ARM_EABIHF"},
* "CompilerOptions": {'mattr': ['+neon']}
Jetson * TX2
"TargetPlatform": {"Os": "LINUX", "Arch": "ARM64",
* "Accelerator": "NVIDIA"},
"CompilerOptions": {'gpu-code':
* 'sm_62', 'trt-ver': '6.0.1', 'cuda-ver': '10.0'}
EC2 * m5.2xlarge instance OS
"TargetPlatform": {"Os": "LINUX", "Arch":
* "X86_64", "Accelerator": "NVIDIA"},
"CompilerOptions":
* {'mcpu': 'skylake-avx512'}
RK3399
* "TargetPlatform": {"Os": "LINUX", "Arch": "ARM64", "Accelerator":
* "MALI"}
ARMv7 phone (CPU)
* "TargetPlatform": {"Os": "ANDROID", "Arch": "ARM_EABI"},
* "CompilerOptions": {'ANDROID_PLATFORM': 25, 'mattr': ['+neon']}
*
ARMv8 phone (CPU)
"TargetPlatform": {"Os":
* "ANDROID", "Arch": "ARM64"},
"CompilerOptions":
* {'ANDROID_PLATFORM': 29}
Contains information about a target platform that you want your model to run
* on, such as OS, architecture, and accelerators. It is an alternative of
* TargetDevice
.
The following examples show how to configure
* the TargetPlatform
and CompilerOptions
JSON strings
* for popular target platforms:
Raspberry Pi 3 Model B+
* "TargetPlatform": {"Os": "LINUX", "Arch": "ARM_EABIHF"},
* "CompilerOptions": {'mattr': ['+neon']}
Jetson * TX2
"TargetPlatform": {"Os": "LINUX", "Arch": "ARM64",
* "Accelerator": "NVIDIA"},
"CompilerOptions": {'gpu-code':
* 'sm_62', 'trt-ver': '6.0.1', 'cuda-ver': '10.0'}
EC2 * m5.2xlarge instance OS
"TargetPlatform": {"Os": "LINUX", "Arch":
* "X86_64", "Accelerator": "NVIDIA"},
"CompilerOptions":
* {'mcpu': 'skylake-avx512'}
RK3399
* "TargetPlatform": {"Os": "LINUX", "Arch": "ARM64", "Accelerator":
* "MALI"}
ARMv7 phone (CPU)
* "TargetPlatform": {"Os": "ANDROID", "Arch": "ARM_EABI"},
* "CompilerOptions": {'ANDROID_PLATFORM': 25, 'mattr': ['+neon']}
*
ARMv8 phone (CPU)
"TargetPlatform": {"Os":
* "ANDROID", "Arch": "ARM64"},
"CompilerOptions":
* {'ANDROID_PLATFORM': 29}
Contains information about a target platform that you want your model to run
* on, such as OS, architecture, and accelerators. It is an alternative of
* TargetDevice
.
The following examples show how to configure
* the TargetPlatform
and CompilerOptions
JSON strings
* for popular target platforms:
Raspberry Pi 3 Model B+
* "TargetPlatform": {"Os": "LINUX", "Arch": "ARM_EABIHF"},
* "CompilerOptions": {'mattr': ['+neon']}
Jetson * TX2
"TargetPlatform": {"Os": "LINUX", "Arch": "ARM64",
* "Accelerator": "NVIDIA"},
"CompilerOptions": {'gpu-code':
* 'sm_62', 'trt-ver': '6.0.1', 'cuda-ver': '10.0'}
EC2 * m5.2xlarge instance OS
"TargetPlatform": {"Os": "LINUX", "Arch":
* "X86_64", "Accelerator": "NVIDIA"},
"CompilerOptions":
* {'mcpu': 'skylake-avx512'}
RK3399
* "TargetPlatform": {"Os": "LINUX", "Arch": "ARM64", "Accelerator":
* "MALI"}
ARMv7 phone (CPU)
* "TargetPlatform": {"Os": "ANDROID", "Arch": "ARM_EABI"},
* "CompilerOptions": {'ANDROID_PLATFORM': 25, 'mattr': ['+neon']}
*
ARMv8 phone (CPU)
"TargetPlatform": {"Os":
* "ANDROID", "Arch": "ARM64"},
"CompilerOptions":
* {'ANDROID_PLATFORM': 29}
Contains information about a target platform that you want your model to run
* on, such as OS, architecture, and accelerators. It is an alternative of
* TargetDevice
.
The following examples show how to configure
* the TargetPlatform
and CompilerOptions
JSON strings
* for popular target platforms:
Raspberry Pi 3 Model B+
* "TargetPlatform": {"Os": "LINUX", "Arch": "ARM_EABIHF"},
* "CompilerOptions": {'mattr': ['+neon']}
Jetson * TX2
"TargetPlatform": {"Os": "LINUX", "Arch": "ARM64",
* "Accelerator": "NVIDIA"},
"CompilerOptions": {'gpu-code':
* 'sm_62', 'trt-ver': '6.0.1', 'cuda-ver': '10.0'}
EC2 * m5.2xlarge instance OS
"TargetPlatform": {"Os": "LINUX", "Arch":
* "X86_64", "Accelerator": "NVIDIA"},
"CompilerOptions":
* {'mcpu': 'skylake-avx512'}
RK3399
* "TargetPlatform": {"Os": "LINUX", "Arch": "ARM64", "Accelerator":
* "MALI"}
ARMv7 phone (CPU)
* "TargetPlatform": {"Os": "ANDROID", "Arch": "ARM_EABI"},
* "CompilerOptions": {'ANDROID_PLATFORM': 25, 'mattr': ['+neon']}
*
ARMv8 phone (CPU)
"TargetPlatform": {"Os":
* "ANDROID", "Arch": "ARM64"},
"CompilerOptions":
* {'ANDROID_PLATFORM': 29}
Contains information about a target platform that you want your model to run
* on, such as OS, architecture, and accelerators. It is an alternative of
* TargetDevice
.
The following examples show how to configure
* the TargetPlatform
and CompilerOptions
JSON strings
* for popular target platforms:
Raspberry Pi 3 Model B+
* "TargetPlatform": {"Os": "LINUX", "Arch": "ARM_EABIHF"},
* "CompilerOptions": {'mattr': ['+neon']}
Jetson * TX2
"TargetPlatform": {"Os": "LINUX", "Arch": "ARM64",
* "Accelerator": "NVIDIA"},
"CompilerOptions": {'gpu-code':
* 'sm_62', 'trt-ver': '6.0.1', 'cuda-ver': '10.0'}
EC2 * m5.2xlarge instance OS
"TargetPlatform": {"Os": "LINUX", "Arch":
* "X86_64", "Accelerator": "NVIDIA"},
"CompilerOptions":
* {'mcpu': 'skylake-avx512'}
RK3399
* "TargetPlatform": {"Os": "LINUX", "Arch": "ARM64", "Accelerator":
* "MALI"}
ARMv7 phone (CPU)
* "TargetPlatform": {"Os": "ANDROID", "Arch": "ARM_EABI"},
* "CompilerOptions": {'ANDROID_PLATFORM': 25, 'mattr': ['+neon']}
*
ARMv8 phone (CPU)
"TargetPlatform": {"Os":
* "ANDROID", "Arch": "ARM64"},
"CompilerOptions":
* {'ANDROID_PLATFORM': 29}
Specifies additional parameters for compiler options in JSON format. The
* compiler options are TargetPlatform
specific. It is required for
* NVIDIA accelerators and highly recommended for CPU compilations. For any other
* cases, it is optional to specify CompilerOptions.
DTYPE
: Specifies the data type for the input. When compiling
* for ml_*
(except for ml_inf
) instances using PyTorch
* framework, provide the data type (dtype) of the model's input.
* "float32"
is used if "DTYPE"
is not specified. Options
* for data type are:
float32: Use either "float"
or
* "float32"
.
int64: Use either "int64"
* or "long"
.
For example, {"dtype" :
* "float32"}
.
CPU
: Compilation for CPU
* supports the following compiler options.
mcpu
:
* CPU micro-architecture. For example, {'mcpu': 'skylake-avx512'}
*
mattr
: CPU flags. For example, {'mattr':
* ['+neon', '+vfpv4']}
ARM
:
* Details of ARM CPU compilations.
NEON
: NEON is an
* implementation of the Advanced SIMD extension used in ARMv7 processors.
For example, add {'mattr': ['+neon']}
to the compiler options if
* compiling for ARM 32-bit platform with the NEON support.
NVIDIA
: Compilation for NVIDIA GPU supports the following
* compiler options.
gpu_code
: Specifies the
* targeted architecture.
trt-ver
: Specifies the
* TensorRT versions in x.y.z. format.
cuda-ver
:
* Specifies the CUDA version in x.y format.
For example,
* {'gpu-code': 'sm_72', 'trt-ver': '6.0.1', 'cuda-ver': '10.1'}
ANDROID
: Compilation for the Android OS supports the
* following compiler options:
ANDROID_PLATFORM
:
* Specifies the Android API levels. Available levels range from 21 to 29. For
* example, {'ANDROID_PLATFORM': 28}
.
* mattr
: Add {'mattr': ['+neon']}
to compiler options if
* compiling for ARM 32-bit platform with NEON support.
INFERENTIA
: Compilation for target ml_inf1 uses compiler
* options passed in as a JSON string. For example, "CompilerOptions":
* "\"--verbose 1 --num-neuroncores 2 -O2\""
.
For information about * supported compiler options, see * Neuron Compiler CLI.
CoreML
: Compilation
* for the CoreML OutputConfig
* TargetDevice
supports the following compiler options:
class_labels
: Specifies the classification labels file name
* inside input tar.gz file. For example, {"class_labels":
* "imagenet_labels_1000.txt"}
. Labels inside the txt file should be
* separated by newlines.
EIA
:
* Compilation for the Elastic Inference Accelerator supports the following
* compiler options:
precision_mode
: Specifies the
* precision of compiled artifacts. Supported values are "FP16"
and
* "FP32"
. Default is "FP32"
.
* signature_def_key
: Specifies the signature to use for models in
* SavedModel format. Defaults is TensorFlow's default signature def key.
output_names
: Specifies a list of output tensor names for
* models in FrozenGraph format. Set at most one API field, either:
* signature_def_key
or output_names
.
For example: {"precision_mode": "FP32", "output_names":
* ["output:0"]}
Specifies additional parameters for compiler options in JSON format. The
* compiler options are TargetPlatform
specific. It is required for
* NVIDIA accelerators and highly recommended for CPU compilations. For any other
* cases, it is optional to specify CompilerOptions.
DTYPE
: Specifies the data type for the input. When compiling
* for ml_*
(except for ml_inf
) instances using PyTorch
* framework, provide the data type (dtype) of the model's input.
* "float32"
is used if "DTYPE"
is not specified. Options
* for data type are:
float32: Use either "float"
or
* "float32"
.
int64: Use either "int64"
* or "long"
.
For example, {"dtype" :
* "float32"}
.
CPU
: Compilation for CPU
* supports the following compiler options.
mcpu
:
* CPU micro-architecture. For example, {'mcpu': 'skylake-avx512'}
*
mattr
: CPU flags. For example, {'mattr':
* ['+neon', '+vfpv4']}
ARM
:
* Details of ARM CPU compilations.
NEON
: NEON is an
* implementation of the Advanced SIMD extension used in ARMv7 processors.
For example, add {'mattr': ['+neon']}
to the compiler options if
* compiling for ARM 32-bit platform with the NEON support.
NVIDIA
: Compilation for NVIDIA GPU supports the following
* compiler options.
gpu_code
: Specifies the
* targeted architecture.
trt-ver
: Specifies the
* TensorRT versions in x.y.z. format.
cuda-ver
:
* Specifies the CUDA version in x.y format.
For example,
* {'gpu-code': 'sm_72', 'trt-ver': '6.0.1', 'cuda-ver': '10.1'}
ANDROID
: Compilation for the Android OS supports the
* following compiler options:
ANDROID_PLATFORM
:
* Specifies the Android API levels. Available levels range from 21 to 29. For
* example, {'ANDROID_PLATFORM': 28}
.
* mattr
: Add {'mattr': ['+neon']}
to compiler options if
* compiling for ARM 32-bit platform with NEON support.
INFERENTIA
: Compilation for target ml_inf1 uses compiler
* options passed in as a JSON string. For example, "CompilerOptions":
* "\"--verbose 1 --num-neuroncores 2 -O2\""
.
For information about * supported compiler options, see * Neuron Compiler CLI.
CoreML
: Compilation
* for the CoreML OutputConfig
* TargetDevice
supports the following compiler options:
class_labels
: Specifies the classification labels file name
* inside input tar.gz file. For example, {"class_labels":
* "imagenet_labels_1000.txt"}
. Labels inside the txt file should be
* separated by newlines.
EIA
:
* Compilation for the Elastic Inference Accelerator supports the following
* compiler options:
precision_mode
: Specifies the
* precision of compiled artifacts. Supported values are "FP16"
and
* "FP32"
. Default is "FP32"
.
* signature_def_key
: Specifies the signature to use for models in
* SavedModel format. Defaults is TensorFlow's default signature def key.
output_names
: Specifies a list of output tensor names for
* models in FrozenGraph format. Set at most one API field, either:
* signature_def_key
or output_names
.
For example: {"precision_mode": "FP32", "output_names":
* ["output:0"]}
Specifies additional parameters for compiler options in JSON format. The
* compiler options are TargetPlatform
specific. It is required for
* NVIDIA accelerators and highly recommended for CPU compilations. For any other
* cases, it is optional to specify CompilerOptions.
DTYPE
: Specifies the data type for the input. When compiling
* for ml_*
(except for ml_inf
) instances using PyTorch
* framework, provide the data type (dtype) of the model's input.
* "float32"
is used if "DTYPE"
is not specified. Options
* for data type are:
float32: Use either "float"
or
* "float32"
.
int64: Use either "int64"
* or "long"
.
For example, {"dtype" :
* "float32"}
.
CPU
: Compilation for CPU
* supports the following compiler options.
mcpu
:
* CPU micro-architecture. For example, {'mcpu': 'skylake-avx512'}
*
mattr
: CPU flags. For example, {'mattr':
* ['+neon', '+vfpv4']}
ARM
:
* Details of ARM CPU compilations.
NEON
: NEON is an
* implementation of the Advanced SIMD extension used in ARMv7 processors.
For example, add {'mattr': ['+neon']}
to the compiler options if
* compiling for ARM 32-bit platform with the NEON support.
NVIDIA
: Compilation for NVIDIA GPU supports the following
* compiler options.
gpu_code
: Specifies the
* targeted architecture.
trt-ver
: Specifies the
* TensorRT versions in x.y.z. format.
cuda-ver
:
* Specifies the CUDA version in x.y format.
For example,
* {'gpu-code': 'sm_72', 'trt-ver': '6.0.1', 'cuda-ver': '10.1'}
ANDROID
: Compilation for the Android OS supports the
* following compiler options:
ANDROID_PLATFORM
:
* Specifies the Android API levels. Available levels range from 21 to 29. For
* example, {'ANDROID_PLATFORM': 28}
.
* mattr
: Add {'mattr': ['+neon']}
to compiler options if
* compiling for ARM 32-bit platform with NEON support.
INFERENTIA
: Compilation for target ml_inf1 uses compiler
* options passed in as a JSON string. For example, "CompilerOptions":
* "\"--verbose 1 --num-neuroncores 2 -O2\""
.
For information about * supported compiler options, see * Neuron Compiler CLI.
CoreML
: Compilation
* for the CoreML OutputConfig
* TargetDevice
supports the following compiler options:
class_labels
: Specifies the classification labels file name
* inside input tar.gz file. For example, {"class_labels":
* "imagenet_labels_1000.txt"}
. Labels inside the txt file should be
* separated by newlines.
EIA
:
* Compilation for the Elastic Inference Accelerator supports the following
* compiler options:
precision_mode
: Specifies the
* precision of compiled artifacts. Supported values are "FP16"
and
* "FP32"
. Default is "FP32"
.
* signature_def_key
: Specifies the signature to use for models in
* SavedModel format. Defaults is TensorFlow's default signature def key.
output_names
: Specifies a list of output tensor names for
* models in FrozenGraph format. Set at most one API field, either:
* signature_def_key
or output_names
.
For example: {"precision_mode": "FP32", "output_names":
* ["output:0"]}
Specifies additional parameters for compiler options in JSON format. The
* compiler options are TargetPlatform
specific. It is required for
* NVIDIA accelerators and highly recommended for CPU compilations. For any other
* cases, it is optional to specify CompilerOptions.
DTYPE
: Specifies the data type for the input. When compiling
* for ml_*
(except for ml_inf
) instances using PyTorch
* framework, provide the data type (dtype) of the model's input.
* "float32"
is used if "DTYPE"
is not specified. Options
* for data type are:
float32: Use either "float"
or
* "float32"
.
int64: Use either "int64"
* or "long"
.
For example, {"dtype" :
* "float32"}
.
CPU
: Compilation for CPU
* supports the following compiler options.
mcpu
:
* CPU micro-architecture. For example, {'mcpu': 'skylake-avx512'}
*
mattr
: CPU flags. For example, {'mattr':
* ['+neon', '+vfpv4']}
ARM
:
* Details of ARM CPU compilations.
NEON
: NEON is an
* implementation of the Advanced SIMD extension used in ARMv7 processors.
For example, add {'mattr': ['+neon']}
to the compiler options if
* compiling for ARM 32-bit platform with the NEON support.
NVIDIA
: Compilation for NVIDIA GPU supports the following
* compiler options.
gpu_code
: Specifies the
* targeted architecture.
trt-ver
: Specifies the
* TensorRT versions in x.y.z. format.
cuda-ver
:
* Specifies the CUDA version in x.y format.
For example,
* {'gpu-code': 'sm_72', 'trt-ver': '6.0.1', 'cuda-ver': '10.1'}
ANDROID
: Compilation for the Android OS supports the
* following compiler options:
ANDROID_PLATFORM
:
* Specifies the Android API levels. Available levels range from 21 to 29. For
* example, {'ANDROID_PLATFORM': 28}
.
* mattr
: Add {'mattr': ['+neon']}
to compiler options if
* compiling for ARM 32-bit platform with NEON support.
INFERENTIA
: Compilation for target ml_inf1 uses compiler
* options passed in as a JSON string. For example, "CompilerOptions":
* "\"--verbose 1 --num-neuroncores 2 -O2\""
.
For information about * supported compiler options, see * Neuron Compiler CLI.
CoreML
: Compilation
* for the CoreML OutputConfig
* TargetDevice
supports the following compiler options:
class_labels
: Specifies the classification labels file name
* inside input tar.gz file. For example, {"class_labels":
* "imagenet_labels_1000.txt"}
. Labels inside the txt file should be
* separated by newlines.
EIA
:
* Compilation for the Elastic Inference Accelerator supports the following
* compiler options:
precision_mode
: Specifies the
* precision of compiled artifacts. Supported values are "FP16"
and
* "FP32"
. Default is "FP32"
.
* signature_def_key
: Specifies the signature to use for models in
* SavedModel format. Defaults is TensorFlow's default signature def key.
output_names
: Specifies a list of output tensor names for
* models in FrozenGraph format. Set at most one API field, either:
* signature_def_key
or output_names
.
For example: {"precision_mode": "FP32", "output_names":
* ["output:0"]}
Specifies additional parameters for compiler options in JSON format. The
* compiler options are TargetPlatform
specific. It is required for
* NVIDIA accelerators and highly recommended for CPU compilations. For any other
* cases, it is optional to specify CompilerOptions.
DTYPE
: Specifies the data type for the input. When compiling
* for ml_*
(except for ml_inf
) instances using PyTorch
* framework, provide the data type (dtype) of the model's input.
* "float32"
is used if "DTYPE"
is not specified. Options
* for data type are:
float32: Use either "float"
or
* "float32"
.
int64: Use either "int64"
* or "long"
.
For example, {"dtype" :
* "float32"}
.
CPU
: Compilation for CPU
* supports the following compiler options.
mcpu
:
* CPU micro-architecture. For example, {'mcpu': 'skylake-avx512'}
*
mattr
: CPU flags. For example, {'mattr':
* ['+neon', '+vfpv4']}
ARM
:
* Details of ARM CPU compilations.
NEON
: NEON is an
* implementation of the Advanced SIMD extension used in ARMv7 processors.
For example, add {'mattr': ['+neon']}
to the compiler options if
* compiling for ARM 32-bit platform with the NEON support.
NVIDIA
: Compilation for NVIDIA GPU supports the following
* compiler options.
gpu_code
: Specifies the
* targeted architecture.
trt-ver
: Specifies the
* TensorRT versions in x.y.z. format.
cuda-ver
:
* Specifies the CUDA version in x.y format.
For example,
* {'gpu-code': 'sm_72', 'trt-ver': '6.0.1', 'cuda-ver': '10.1'}
ANDROID
: Compilation for the Android OS supports the
* following compiler options:
ANDROID_PLATFORM
:
* Specifies the Android API levels. Available levels range from 21 to 29. For
* example, {'ANDROID_PLATFORM': 28}
.
* mattr
: Add {'mattr': ['+neon']}
to compiler options if
* compiling for ARM 32-bit platform with NEON support.
INFERENTIA
: Compilation for target ml_inf1 uses compiler
* options passed in as a JSON string. For example, "CompilerOptions":
* "\"--verbose 1 --num-neuroncores 2 -O2\""
.
For information about * supported compiler options, see * Neuron Compiler CLI.
CoreML
: Compilation
* for the CoreML OutputConfig
* TargetDevice
supports the following compiler options:
class_labels
: Specifies the classification labels file name
* inside input tar.gz file. For example, {"class_labels":
* "imagenet_labels_1000.txt"}
. Labels inside the txt file should be
* separated by newlines.
EIA
:
* Compilation for the Elastic Inference Accelerator supports the following
* compiler options:
precision_mode
: Specifies the
* precision of compiled artifacts. Supported values are "FP16"
and
* "FP32"
. Default is "FP32"
.
* signature_def_key
: Specifies the signature to use for models in
* SavedModel format. Defaults is TensorFlow's default signature def key.
output_names
: Specifies a list of output tensor names for
* models in FrozenGraph format. Set at most one API field, either:
* signature_def_key
or output_names
.
For example: {"precision_mode": "FP32", "output_names":
* ["output:0"]}
Specifies additional parameters for compiler options in JSON format. The
* compiler options are TargetPlatform
specific. It is required for
* NVIDIA accelerators and highly recommended for CPU compilations. For any other
* cases, it is optional to specify CompilerOptions.
DTYPE
: Specifies the data type for the input. When compiling
* for ml_*
(except for ml_inf
) instances using PyTorch
* framework, provide the data type (dtype) of the model's input.
* "float32"
is used if "DTYPE"
is not specified. Options
* for data type are:
float32: Use either "float"
or
* "float32"
.
int64: Use either "int64"
* or "long"
.
For example, {"dtype" :
* "float32"}
.
CPU
: Compilation for CPU
* supports the following compiler options.
mcpu
:
* CPU micro-architecture. For example, {'mcpu': 'skylake-avx512'}
*
mattr
: CPU flags. For example, {'mattr':
* ['+neon', '+vfpv4']}
ARM
:
* Details of ARM CPU compilations.
NEON
: NEON is an
* implementation of the Advanced SIMD extension used in ARMv7 processors.
For example, add {'mattr': ['+neon']}
to the compiler options if
* compiling for ARM 32-bit platform with the NEON support.
NVIDIA
: Compilation for NVIDIA GPU supports the following
* compiler options.
gpu_code
: Specifies the
* targeted architecture.
trt-ver
: Specifies the
* TensorRT versions in x.y.z. format.
cuda-ver
:
* Specifies the CUDA version in x.y format.
For example,
* {'gpu-code': 'sm_72', 'trt-ver': '6.0.1', 'cuda-ver': '10.1'}
ANDROID
: Compilation for the Android OS supports the
* following compiler options:
ANDROID_PLATFORM
:
* Specifies the Android API levels. Available levels range from 21 to 29. For
* example, {'ANDROID_PLATFORM': 28}
.
* mattr
: Add {'mattr': ['+neon']}
to compiler options if
* compiling for ARM 32-bit platform with NEON support.
INFERENTIA
: Compilation for target ml_inf1 uses compiler
* options passed in as a JSON string. For example, "CompilerOptions":
* "\"--verbose 1 --num-neuroncores 2 -O2\""
.
For information about * supported compiler options, see * Neuron Compiler CLI.
CoreML
: Compilation
* for the CoreML OutputConfig
* TargetDevice
supports the following compiler options:
class_labels
: Specifies the classification labels file name
* inside input tar.gz file. For example, {"class_labels":
* "imagenet_labels_1000.txt"}
. Labels inside the txt file should be
* separated by newlines.
EIA
:
* Compilation for the Elastic Inference Accelerator supports the following
* compiler options:
precision_mode
: Specifies the
* precision of compiled artifacts. Supported values are "FP16"
and
* "FP32"
. Default is "FP32"
.
* signature_def_key
: Specifies the signature to use for models in
* SavedModel format. Defaults is TensorFlow's default signature def key.
output_names
: Specifies a list of output tensor names for
* models in FrozenGraph format. Set at most one API field, either:
* signature_def_key
or output_names
.
For example: {"precision_mode": "FP32", "output_names":
* ["output:0"]}
Specifies additional parameters for compiler options in JSON format. The
* compiler options are TargetPlatform
specific. It is required for
* NVIDIA accelerators and highly recommended for CPU compilations. For any other
* cases, it is optional to specify CompilerOptions.
DTYPE
: Specifies the data type for the input. When compiling
* for ml_*
(except for ml_inf
) instances using PyTorch
* framework, provide the data type (dtype) of the model's input.
* "float32"
is used if "DTYPE"
is not specified. Options
* for data type are:
float32: Use either "float"
or
* "float32"
.
int64: Use either "int64"
* or "long"
.
For example, {"dtype" :
* "float32"}
.
CPU
: Compilation for CPU
* supports the following compiler options.
mcpu
:
* CPU micro-architecture. For example, {'mcpu': 'skylake-avx512'}
*
mattr
: CPU flags. For example, {'mattr':
* ['+neon', '+vfpv4']}
ARM
:
* Details of ARM CPU compilations.
NEON
: NEON is an
* implementation of the Advanced SIMD extension used in ARMv7 processors.
For example, add {'mattr': ['+neon']}
to the compiler options if
* compiling for ARM 32-bit platform with the NEON support.
NVIDIA
: Compilation for NVIDIA GPU supports the following
* compiler options.
gpu_code
: Specifies the
* targeted architecture.
trt-ver
: Specifies the
* TensorRT versions in x.y.z. format.
cuda-ver
:
* Specifies the CUDA version in x.y format.
For example,
* {'gpu-code': 'sm_72', 'trt-ver': '6.0.1', 'cuda-ver': '10.1'}
ANDROID
: Compilation for the Android OS supports the
* following compiler options:
ANDROID_PLATFORM
:
* Specifies the Android API levels. Available levels range from 21 to 29. For
* example, {'ANDROID_PLATFORM': 28}
.
* mattr
: Add {'mattr': ['+neon']}
to compiler options if
* compiling for ARM 32-bit platform with NEON support.
INFERENTIA
: Compilation for target ml_inf1 uses compiler
* options passed in as a JSON string. For example, "CompilerOptions":
* "\"--verbose 1 --num-neuroncores 2 -O2\""
.
For information about * supported compiler options, see * Neuron Compiler CLI.
CoreML
: Compilation
* for the CoreML OutputConfig
* TargetDevice
supports the following compiler options:
class_labels
: Specifies the classification labels file name
* inside input tar.gz file. For example, {"class_labels":
* "imagenet_labels_1000.txt"}
. Labels inside the txt file should be
* separated by newlines.
EIA
:
* Compilation for the Elastic Inference Accelerator supports the following
* compiler options:
precision_mode
: Specifies the
* precision of compiled artifacts. Supported values are "FP16"
and
* "FP32"
. Default is "FP32"
.
* signature_def_key
: Specifies the signature to use for models in
* SavedModel format. Defaults is TensorFlow's default signature def key.
output_names
: Specifies a list of output tensor names for
* models in FrozenGraph format. Set at most one API field, either:
* signature_def_key
or output_names
.
For example: {"precision_mode": "FP32", "output_names":
* ["output:0"]}
Specifies additional parameters for compiler options in JSON format. The
* compiler options are TargetPlatform
specific. It is required for
* NVIDIA accelerators and highly recommended for CPU compilations. For any other
* cases, it is optional to specify CompilerOptions.
DTYPE
: Specifies the data type for the input. When compiling
* for ml_*
(except for ml_inf
) instances using PyTorch
* framework, provide the data type (dtype) of the model's input.
* "float32"
is used if "DTYPE"
is not specified. Options
* for data type are:
float32: Use either "float"
or
* "float32"
.
int64: Use either "int64"
* or "long"
.
For example, {"dtype" :
* "float32"}
.
CPU
: Compilation for CPU
* supports the following compiler options.
mcpu
:
* CPU micro-architecture. For example, {'mcpu': 'skylake-avx512'}
*
mattr
: CPU flags. For example, {'mattr':
* ['+neon', '+vfpv4']}
ARM
:
* Details of ARM CPU compilations.
NEON
: NEON is an
* implementation of the Advanced SIMD extension used in ARMv7 processors.
For example, add {'mattr': ['+neon']}
to the compiler options if
* compiling for ARM 32-bit platform with the NEON support.
NVIDIA
: Compilation for NVIDIA GPU supports the following
* compiler options.
gpu_code
: Specifies the
* targeted architecture.
trt-ver
: Specifies the
* TensorRT versions in x.y.z. format.
cuda-ver
:
* Specifies the CUDA version in x.y format.
For example,
* {'gpu-code': 'sm_72', 'trt-ver': '6.0.1', 'cuda-ver': '10.1'}
ANDROID
: Compilation for the Android OS supports the
* following compiler options:
ANDROID_PLATFORM
:
* Specifies the Android API levels. Available levels range from 21 to 29. For
* example, {'ANDROID_PLATFORM': 28}
.
* mattr
: Add {'mattr': ['+neon']}
to compiler options if
* compiling for ARM 32-bit platform with NEON support.
INFERENTIA
: Compilation for target ml_inf1 uses compiler
* options passed in as a JSON string. For example, "CompilerOptions":
* "\"--verbose 1 --num-neuroncores 2 -O2\""
.
For information about * supported compiler options, see * Neuron Compiler CLI.
CoreML
: Compilation
* for the CoreML OutputConfig
* TargetDevice
supports the following compiler options:
class_labels
: Specifies the classification labels file name
* inside input tar.gz file. For example, {"class_labels":
* "imagenet_labels_1000.txt"}
. Labels inside the txt file should be
* separated by newlines.
EIA
:
* Compilation for the Elastic Inference Accelerator supports the following
* compiler options:
precision_mode
: Specifies the
* precision of compiled artifacts. Supported values are "FP16"
and
* "FP32"
. Default is "FP32"
.
* signature_def_key
: Specifies the signature to use for models in
* SavedModel format. Defaults is TensorFlow's default signature def key.
output_names
: Specifies a list of output tensor names for
* models in FrozenGraph format. Set at most one API field, either:
* signature_def_key
or output_names
.
For example: {"precision_mode": "FP32", "output_names":
* ["output:0"]}
The Amazon Web Services Key Management Service key (Amazon Web Services KMS) * that Amazon SageMaker uses to encrypt your output models with Amazon S3 * server-side encryption after compilation job. If you don't provide a KMS key ID, * Amazon SageMaker uses the default KMS key for Amazon S3 for your role's account. * For more information, see KMS-Managed * Encryption Keys in the Amazon Simple Storage Service Developer Guide. *
The KmsKeyId can be any of the following formats:
Key
* ID: 1234abcd-12ab-34cd-56ef-1234567890ab
Key
* ARN:
* arn:aws:kms:us-west-2:111122223333:key/1234abcd-12ab-34cd-56ef-1234567890ab
*
Alias name: alias/ExampleAlias
Alias name ARN:
* arn:aws:kms:us-west-2:111122223333:alias/ExampleAlias
The Amazon Web Services Key Management Service key (Amazon Web Services KMS) * that Amazon SageMaker uses to encrypt your output models with Amazon S3 * server-side encryption after compilation job. If you don't provide a KMS key ID, * Amazon SageMaker uses the default KMS key for Amazon S3 for your role's account. * For more information, see KMS-Managed * Encryption Keys in the Amazon Simple Storage Service Developer Guide. *
The KmsKeyId can be any of the following formats:
Key
* ID: 1234abcd-12ab-34cd-56ef-1234567890ab
Key
* ARN:
* arn:aws:kms:us-west-2:111122223333:key/1234abcd-12ab-34cd-56ef-1234567890ab
*
Alias name: alias/ExampleAlias
Alias name ARN:
* arn:aws:kms:us-west-2:111122223333:alias/ExampleAlias
The Amazon Web Services Key Management Service key (Amazon Web Services KMS) * that Amazon SageMaker uses to encrypt your output models with Amazon S3 * server-side encryption after compilation job. If you don't provide a KMS key ID, * Amazon SageMaker uses the default KMS key for Amazon S3 for your role's account. * For more information, see KMS-Managed * Encryption Keys in the Amazon Simple Storage Service Developer Guide. *
The KmsKeyId can be any of the following formats:
Key
* ID: 1234abcd-12ab-34cd-56ef-1234567890ab
Key
* ARN:
* arn:aws:kms:us-west-2:111122223333:key/1234abcd-12ab-34cd-56ef-1234567890ab
*
Alias name: alias/ExampleAlias
Alias name ARN:
* arn:aws:kms:us-west-2:111122223333:alias/ExampleAlias
The Amazon Web Services Key Management Service key (Amazon Web Services KMS) * that Amazon SageMaker uses to encrypt your output models with Amazon S3 * server-side encryption after compilation job. If you don't provide a KMS key ID, * Amazon SageMaker uses the default KMS key for Amazon S3 for your role's account. * For more information, see KMS-Managed * Encryption Keys in the Amazon Simple Storage Service Developer Guide. *
The KmsKeyId can be any of the following formats:
Key
* ID: 1234abcd-12ab-34cd-56ef-1234567890ab
Key
* ARN:
* arn:aws:kms:us-west-2:111122223333:key/1234abcd-12ab-34cd-56ef-1234567890ab
*
Alias name: alias/ExampleAlias
Alias name ARN:
* arn:aws:kms:us-west-2:111122223333:alias/ExampleAlias
The Amazon Web Services Key Management Service key (Amazon Web Services KMS) * that Amazon SageMaker uses to encrypt your output models with Amazon S3 * server-side encryption after compilation job. If you don't provide a KMS key ID, * Amazon SageMaker uses the default KMS key for Amazon S3 for your role's account. * For more information, see KMS-Managed * Encryption Keys in the Amazon Simple Storage Service Developer Guide. *
The KmsKeyId can be any of the following formats:
Key
* ID: 1234abcd-12ab-34cd-56ef-1234567890ab
Key
* ARN:
* arn:aws:kms:us-west-2:111122223333:key/1234abcd-12ab-34cd-56ef-1234567890ab
*
Alias name: alias/ExampleAlias
Alias name ARN:
* arn:aws:kms:us-west-2:111122223333:alias/ExampleAlias
The Amazon Web Services Key Management Service key (Amazon Web Services KMS) * that Amazon SageMaker uses to encrypt your output models with Amazon S3 * server-side encryption after compilation job. If you don't provide a KMS key ID, * Amazon SageMaker uses the default KMS key for Amazon S3 for your role's account. * For more information, see KMS-Managed * Encryption Keys in the Amazon Simple Storage Service Developer Guide. *
The KmsKeyId can be any of the following formats:
Key
* ID: 1234abcd-12ab-34cd-56ef-1234567890ab
Key
* ARN:
* arn:aws:kms:us-west-2:111122223333:key/1234abcd-12ab-34cd-56ef-1234567890ab
*
Alias name: alias/ExampleAlias
Alias name ARN:
* arn:aws:kms:us-west-2:111122223333:alias/ExampleAlias
The Amazon Web Services Key Management Service key (Amazon Web Services KMS) * that Amazon SageMaker uses to encrypt your output models with Amazon S3 * server-side encryption after compilation job. If you don't provide a KMS key ID, * Amazon SageMaker uses the default KMS key for Amazon S3 for your role's account. * For more information, see KMS-Managed * Encryption Keys in the Amazon Simple Storage Service Developer Guide. *
The KmsKeyId can be any of the following formats:
Key
* ID: 1234abcd-12ab-34cd-56ef-1234567890ab
Key
* ARN:
* arn:aws:kms:us-west-2:111122223333:key/1234abcd-12ab-34cd-56ef-1234567890ab
*
Alias name: alias/ExampleAlias
Alias name ARN:
* arn:aws:kms:us-west-2:111122223333:alias/ExampleAlias
The Amazon Web Services Key Management Service key (Amazon Web Services KMS) * that Amazon SageMaker uses to encrypt your output models with Amazon S3 * server-side encryption after compilation job. If you don't provide a KMS key ID, * Amazon SageMaker uses the default KMS key for Amazon S3 for your role's account. * For more information, see KMS-Managed * Encryption Keys in the Amazon Simple Storage Service Developer Guide. *
The KmsKeyId can be any of the following formats:
Key
* ID: 1234abcd-12ab-34cd-56ef-1234567890ab
Key
* ARN:
* arn:aws:kms:us-west-2:111122223333:key/1234abcd-12ab-34cd-56ef-1234567890ab
*
Alias name: alias/ExampleAlias
Alias name ARN:
* arn:aws:kms:us-west-2:111122223333:alias/ExampleAlias