Model Inputs and Outputs
model_navigator.api.triton.InputTensorSpec
dataclass
Bases: BaseTensorSpec
Stores specification of single input tensor.
This includes name, shape, dtype and more parameters available for input tensor in Triton Inference Server:
Read more in Triton Inference server model configuration
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
optional |
bool
|
Flag marking the input is optional for the model execution |
False
|
format |
Optional[InputTensorFormat]
|
The format of the input. |
None
|
allow_ragged_batch |
bool
|
Flag marking the input is allowed to be "ragged" in a dynamically created batch. |
False
|
model_navigator.api.triton.InputTensorFormat
Format for input tensor.
Read more in Triton Inference server model configuration
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
FORMAT_NONE |
0 |
required | |
FORMAT_NHWC |
1 |
required | |
FORMAT_NCHW |
2 |
required |
model_navigator.api.triton.OutputTensorSpec
dataclass
Bases: BaseTensorSpec
Stores specification of single output tensor.
This includes name, shape, dtype and more parameters available for output tensor in Triton Inference Server:
Read more in Triton Inference server model configuration
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
label_filename |
Optional[str]
|
The label file associated with this output. |
None
|