Examples

We provide step-by-step examples that demonstrate how to use various features of Model Navigator. For the sake of readability and accessibility, we use a simple torch.nn.Linear model as an example. These examples illustrate how to optimize, test and deploy the model on the PyTriton and Triton Inference Server.

Step-by-step examples

1. Optimize model
2. Optimize model and verify model
3. Optimize model and save package
4. Load and optimize package
5. Optimize and server model on PyTriton
6. Optimize and serve model on Triton Inference Server
7. Optimize model and use for offline inference
8. Optimize PyTorch HiFi-GAN QAT model
9. Custom configuration for optimize

Example models

Inside example/models directory you can find ready to use example models in various frameworks.

Python: - Identity Model

PyTorch:

• Linear Model
• ResNet50
• BERT

TensorFlow:

• Linear Model
• EfficientNet
• BERT

JAX:

• Linear Model
• GPT-2

ONNX:

• Identity Model