Model Warmup

model_navigator.api.triton.ModelWarmup dataclass

Model warmup configuration.

Read more in Triton Inference server model configuration

Parameters:

• batch_size (int, default: 1 ) –

  The batch size of the inference request. This must be >= 1. For models that don't support batching, batch_size must be 1.

• inputs (Dict[str, ModelWarmupInput]) –

  The warmup meta data associated with every model input, including control tensors.

• iterations (int, default: 0 ) –

  The number of iterations that this warmup sample will be executed. For example, if this field is set to 2, 2 model executions using this sample will be scheduled for warmup. Default value is 0 which indicates that this sample will be used only once.

model_navigator.api.triton.ModelWarmupInput dataclass

Model warmup input configuration.

Read more in Triton Inference server model configuration

Parameters:

• shape (Tuple[int, ...]) –

  Shape of the model input/output

• dtype (Optional[Union[dtype, Type[numpy.dtype]]]) –

  Data type

• input_data_type (ModelWarmupInputDataType) –

  Type of input data used for warmup

• input_data_file (Optional[pathlib.Path], default: None ) –

  Path to file with input data. Provide the path where the file is located. Required only when input_data_type is ModelWarmupInputDataType.DATA_FILE

model_navigator.api.triton.ModelWarmupInputDataType

Bases: Enum

Model warmup input data type.

Read more in Triton Inference server model configuration

Parameters:

• ZERO –

  "ZERO"

• RANDOM –

  "RANDOM"

• FILE –

  "FILE"