Model Warmup

model_navigator.triton.ModelWarmup dataclass

ModelWarmup(inputs, batch_size=1, iterations=0)

Model warmup configuration.

Read more in Triton Inference server model configuration

Parameters:

• batch_size (int, default: 1 ) –

  The batch size of the inference request. This must be >= 1. For models that don't support batching, batch_size must be 1.

• inputs (Dict[str, ModelWarmupInput]) –

  The warmup meta data associated with every model input, including control tensors.

• iterations (int, default: 0 ) –

  The number of iterations that this warmup sample will be executed. For example, if this field is set to 2, 2 model executions using this sample will be scheduled for warmup. Default value is 0 which indicates that this sample will be used only once.

__post_init__

__post_init__()

Validate the configuration for early error handling.

Source code in model_navigator/triton/specialized_configs/common.py

def __post_init__(self):
    """Validate the configuration for early error handling."""
    if self.batch_size < 1:
        raise ModelNavigatorWrongParameterError("`batch_size` must be greater or equal 1.")

    if self.iterations < 0:
        raise ModelNavigatorWrongParameterError("`iterations` must be greater or equal 0.")

model_navigator.triton.ModelWarmupInput dataclass

ModelWarmupInput(shape, dtype, input_data_type, input_data_file=None)

Model warmup input configuration.

Read more in Triton Inference server model configuration

Parameters:

• shape (Tuple[int, ...]) –

  Shape of the model input/output

• dtype (Optional[Union[dtype, Type[dtype]]]) –

  Data type

• input_data_type (ModelWarmupInputDataType) –

  Type of input data used for warmup

• input_data_file (Optional[Path], default: None ) –

  Path to file with input data. Provide the path where the file is located. Required only when input_data_type is ModelWarmupInputDataType.DATA_FILE

__post_init__

__post_init__()

Validate the configuration for early error handling.

Source code in model_navigator/triton/specialized_configs/common.py

def __post_init__(self):
    """Validate the configuration for early error handling."""
    if self.input_data_type == ModelWarmupInputDataType.FILE and self.input_data_file is None:
        raise ModelNavigatorWrongParameterError("`input_data_file` is required. Set the file path.")

    if self.input_data_type != ModelWarmupInputDataType.FILE and self.input_data_file is not None:
        raise ModelNavigatorWrongParameterError("`input_data_file` is not required. Remove the parameter.")

model_navigator.triton.ModelWarmupInputDataType

Bases: Enum

Model warmup input data type.

Read more in Triton Inference server model configuration

Parameters:

• ZERO –

  "ZERO"

• RANDOM –

  "RANDOM"

• FILE –

  "FILE"