Example with downloaded input data

In the following example, we will demonstrate how to effectively utilize PyTriton with downloaded input data. While the model itself does not possess any inputs, it utilize custom parameters or headers to extract a URL and download data from an external source, such as an S3 bucket.

The corresponding function can leverage the batch decorator since it does not rely on any parameters or headers.

Example

import numpy as np
from pytriton.model_config import ModelConfig, Tensor
from pytriton.triton import Triton, TritonConfig

@batch
def model_infer_function(**inputs):
    ...

def request_infer_function(requests):
    for request in requests:
        image_url = request.parameters["custom_url"]
        image_jpeg = download(image_url)
        image_data = decompress(image_jpeg)
        request['images_data'] = image_data
    outputs = model_infer_function(requests)
    return outputs

with Triton(config=TritonConfig(http_header_forward_pattern="custom.*")) as triton:
    triton.bind(
        model_name="ImgModel",
        infer_func=request_infer_function,
        inputs=[],
        outputs=[Tensor(name="out", dtype=np.float32, shape=(-1,))],
        config=ModelConfig(max_batch_size=128),
    )
    triton.serve()