Model Inputs and Outputs

model_navigator.api.triton.InputTensorSpec dataclass

Bases: BaseTensorSpec

Stores specification of single input tensor.

This includes name, shape, dtype and more parameters available for input tensor in Triton Inference Server:

Read more in Triton Inference server model configuration

Parameters:

• optional (bool, default: False ) –

  Flag marking the input is optional for the model execution

• format (Optional[InputTensorFormat], default: None ) –

  The format of the input.

• allow_ragged_batch (bool, default: False ) –

  Flag marking the input is allowed to be "ragged" in a dynamically created batch.

model_navigator.api.triton.InputTensorFormat

Bases: Enum

Format for input tensor.

Read more in Triton Inference server model configuration

Parameters:

• FORMAT_NONE –

  0

• FORMAT_NHWC –

  1

• FORMAT_NCHW –

  2

model_navigator.api.triton.OutputTensorSpec dataclass

Bases: BaseTensorSpec

Stores specification of single output tensor.

This includes name, shape, dtype and more parameters available for output tensor in Triton Inference Server:

Read more in Triton Inference server model configuration

Parameters:

• label_filename (Optional[str], default: None ) –

  The label file associated with this output.