/**
* Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
* SPDX-License-Identifier: Apache-2.0.
*/
#pragma once
#include Specifies additional configuration for hosting multi-model
* endpoints.See Also:
AWS
* API Reference
Whether to cache models for a multi-model endpoint. By default, multi-model
* endpoints cache models so that a model does not have to be loaded into memory
* each time it is invoked. Some use cases do not benefit from model caching. For
* example, if an endpoint hosts a large number of models that are each invoked
* infrequently, the endpoint might perform better if you disable model caching. To
* disable model caching, set the value of this parameter to
* Disabled
.
Whether to cache models for a multi-model endpoint. By default, multi-model
* endpoints cache models so that a model does not have to be loaded into memory
* each time it is invoked. Some use cases do not benefit from model caching. For
* example, if an endpoint hosts a large number of models that are each invoked
* infrequently, the endpoint might perform better if you disable model caching. To
* disable model caching, set the value of this parameter to
* Disabled
.
Whether to cache models for a multi-model endpoint. By default, multi-model
* endpoints cache models so that a model does not have to be loaded into memory
* each time it is invoked. Some use cases do not benefit from model caching. For
* example, if an endpoint hosts a large number of models that are each invoked
* infrequently, the endpoint might perform better if you disable model caching. To
* disable model caching, set the value of this parameter to
* Disabled
.
Whether to cache models for a multi-model endpoint. By default, multi-model
* endpoints cache models so that a model does not have to be loaded into memory
* each time it is invoked. Some use cases do not benefit from model caching. For
* example, if an endpoint hosts a large number of models that are each invoked
* infrequently, the endpoint might perform better if you disable model caching. To
* disable model caching, set the value of this parameter to
* Disabled
.
Whether to cache models for a multi-model endpoint. By default, multi-model
* endpoints cache models so that a model does not have to be loaded into memory
* each time it is invoked. Some use cases do not benefit from model caching. For
* example, if an endpoint hosts a large number of models that are each invoked
* infrequently, the endpoint might perform better if you disable model caching. To
* disable model caching, set the value of this parameter to
* Disabled
.
Whether to cache models for a multi-model endpoint. By default, multi-model
* endpoints cache models so that a model does not have to be loaded into memory
* each time it is invoked. Some use cases do not benefit from model caching. For
* example, if an endpoint hosts a large number of models that are each invoked
* infrequently, the endpoint might perform better if you disable model caching. To
* disable model caching, set the value of this parameter to
* Disabled
.