Installation

Prerequisites

Before proceeding with the installation of the Triton Model Navigator, ensure your system meets the following criteria:

You can use NGC Containers for PyTorch and TensorFlow which contain all necessary dependencies:

The library can be installed in:

The NVIDIA optimized Docker images for Python frameworks could be obtained from NVIDIA NGC Catalog.

For using NVIDIA optimized Docker images, we recommend installing NVIDIA Container Toolkit to run model inference on NVIDIA GPU.

The Triton Model Navigator can be installed from pypi.org.

For installing with PyTorch dependencies, use:

pip install -U --extra-index-url https://pypi.ngc.nvidia.com triton-model-navigator[torch]

or with nvidia-pyindex:

pip install nvidia-pyindex
pip install -U triton-model-navigator[torch]

For installing with TensorFlow dependencies, use:

pip install -U --extra-index-url https://pypi.ngc.nvidia.com triton-model-navigator[tensorflow]

or with nvidia-pyindex:

pip install nvidia-pyindex
pip install -U triton-model-navigator[tensorflow]

For installing with JAX dependencies, use:

pip install -U --extra-index-url https://pypi.ngc.nvidia.com triton-model-navigator[jax]

or with nvidia-pyindex:

pip install nvidia-pyindex
pip install -U triton-model-navigator[jax]

The default CUDA version for ONNXRuntime since 1.19.0 is CUDA 12. To install with CUDA 11 support use following extra index url:

.. --extra-index-url https://aiinfra.pkgs.visualstudio.com/PublicPackages/_packaging/onnxruntime-cuda-11/pypi/simple/ ..

The Triton Model Navigator can be built as a wheel. We have prepared all necessary steps under Makefile command.

Firstly, install the Triton Model Navigator with development packages:

make install-dev

Next, simply run:

make dist

The wheel will be generated in the dist catalog.