Model Warmup
model_navigator.api.triton.ModelWarmup
dataclass
Model warmup configuration.
Read more in Triton Inference server model configuration
Parameters:
-
batch_size
(
int
, default:1
) –The batch size of the inference request. This must be >= 1. For models that don't support batching, batch_size must be 1.
-
inputs
(
Dict[str, ModelWarmupInput]
) –The warmup meta data associated with every model input, including control tensors.
-
iterations
(
int
, default:0
) –The number of iterations that this warmup sample will be executed. For example, if this field is set to 2, 2 model executions using this sample will be scheduled for warmup. Default value is 0 which indicates that this sample will be used only once.
model_navigator.api.triton.ModelWarmupInput
dataclass
Model warmup input configuration.
Read more in Triton Inference server model configuration
Parameters:
-
shape
(
Tuple[int, ...]
) –Shape of the model input/output
-
dtype
(
Optional[Union[dtype, Type[numpy.dtype]]]
) –Data type
-
input_data_type
(
ModelWarmupInputDataType
) –Type of input data used for warmup
-
input_data_file
(
Optional[pathlib.Path]
, default:None
) –Path to file with input data. Provide the path where the file is located. Required only when input_data_type is
ModelWarmupInputDataType.DATA_FILE
model_navigator.api.triton.ModelWarmupInputDataType
Bases: Enum
Model warmup input data type.
Read more in Triton Inference server model configuration
Parameters:
-
ZERO
–
"ZERO"
-
RANDOM
–
"RANDOM"
-
FILE
–
"FILE"