Skip to content

Config

Classes, enums and types used to configure Model Navigator.

model_navigator.api.config

Definition of enums and classes representing configuration for Model Navigator.

CustomConfig

Bases: ABC

Base class used for custom configs. Input for Model Navigator optimize method.

defaults 

defaults()

Update parameters to defaults.

Source code in model_navigator/api/config.py 
487
488
489
def defaults(self) -> None:
    """Update parameters to defaults."""
    return None

from_dict classmethod 

from_dict(config_dict)

Instantiate CustomConfig from a dictionary.

Source code in model_navigator/api/config.py 
491
492
493
494
@classmethod
def from_dict(cls, config_dict: Dict[str, Any]) -> "CustomConfig":
    """Instantiate CustomConfig from a dictionary."""
    return cls(**config_dict)

name abstractmethod classmethod 

name()

Name of the CustomConfig.

Source code in model_navigator/api/config.py 
481
482
483
484
485
@classmethod
@abc.abstractmethod
def name(cls) -> str:
    """Name of the CustomConfig."""
    raise NotImplementedError()

CustomConfigForFormat dataclass

Bases: DataObject, CustomConfig

Abstract base class used for custom configs representing particular format.

format abstractmethod property 

format

Format represented by CustomConfig.

defaults 

defaults()

Update parameters to defaults.

Source code in model_navigator/api/config.py 
487
488
489
def defaults(self) -> None:
    """Update parameters to defaults."""
    return None

filter_data staticmethod 

filter_data(data, filter_fields)

Filter fields in dictionary.

Parameters:

  • data (Dict) –

    Dictionary with data to filter

  • filter_fields (List[str]) –

    Fields to filter

Returns:

  • –

    Filtered dictionary

Source code in model_navigator/utils/common.py 
103
104
105
106
107
108
109
110
111
112
113
114
115
@staticmethod
def filter_data(data: Dict, filter_fields: List[str]):
    """Filter fields in dictionary.

    Args:
        data: Dictionary with data to filter
        filter_fields: Fields to filter

    Returns:
        Filtered dictionary
    """
    filtered_data = {key: value for key, value in data.items() if key not in filter_fields}
    return filtered_data

from_dict classmethod 

from_dict(config_dict)

Instantiate CustomConfig from a dictionary.

Source code in model_navigator/api/config.py 
491
492
493
494
@classmethod
def from_dict(cls, config_dict: Dict[str, Any]) -> "CustomConfig":
    """Instantiate CustomConfig from a dictionary."""
    return cls(**config_dict)

name abstractmethod classmethod 

name()

Name of the CustomConfig.

Source code in model_navigator/api/config.py 
481
482
483
484
485
@classmethod
@abc.abstractmethod
def name(cls) -> str:
    """Name of the CustomConfig."""
    raise NotImplementedError()

parse_data staticmethod 

parse_data(data)

Parse values in provided data.

Parameters:

  • data (Dict) –

    Dictionary with data to parse

Returns:

  • –

    Parsed dictionary

Source code in model_navigator/utils/common.py 
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
@staticmethod
def parse_data(data: Dict):
    """Parse values in provided data.

    Args:
        data: Dictionary with data to parse

    Returns:
        Parsed dictionary
    """
    parsed_data = {}
    for key, value in data.items():
        parsed_data[key] = DataObject.parse_value(value)

    return parsed_data

parse_value staticmethod 

parse_value(value)

Parse value to jsonable format.

Parameters:

  • value (Any) –

    Value to be parsed.

Returns:

Source code in model_navigator/utils/common.py 
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
@staticmethod
def parse_value(value: Any) -> Union[str, Dict, List]:
    """Parse value to jsonable format.

    Args:
        value (Any): Value to be parsed.

    Returns:
        Union[str, Dict, List]: Jsonable value.
    """
    if isinstance(value, DataObject):
        value = value.to_dict(parse=True)
    elif hasattr(value, "to_json"):
        value = value.to_json()
    elif isinstance(value, (Mapping, Profile)):
        value = DataObject._from_dict(value)
    elif isinstance(value, list) or isinstance(value, tuple):
        value = DataObject._from_list(value)
    elif isinstance(value, Enum):
        value = value.value
    elif isinstance(value, pathlib.Path):
        value = str(value)
    elif isinstance(value, ShapeTuple):
        value = vars(value)

    return value

to_dict 

to_dict(filter_fields=None, parse=False)

Serialize to a dictionary.

Parameters:

  • filter_fields (Optional[List[str]], default: None ) –

    List of fields to filter out. Defaults to None.

  • parse (bool, default: False ) –

    If True recursively parse field values to jsonable representation. Defaults to False.

Returns:

  • Dict ( Dict ) –

    Data serialized to a dictionary.

Source code in model_navigator/utils/common.py 
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
def to_dict(self, filter_fields: Optional[List[str]] = None, parse: bool = False) -> Dict:
    """Serialize to a dictionary.

    Args:
        filter_fields (Optional[List[str]], optional): List of fields to filter out.
            Defaults to None.
        parse (bool, optional): If True recursively parse field values to jsonable representation.
            Defaults to False.

    Returns:
        Dict: Data serialized to a dictionary.
    """
    if filter_fields:
        filtered_data = DataObject.filter_data(
            data=self.__dict__,
            filter_fields=filter_fields,
        )
    else:
        filtered_data = self.__dict__

    if parse:
        data = DataObject.parse_data(filtered_data)
    else:
        data = filtered_data

    return data

CustomConfigForTensorRT dataclass

Bases: CustomConfigForFormat

Abstract base class used for custom configs representing particular TensorRT format.

format abstractmethod property 

format

Format represented by CustomConfig.

defaults 

defaults()

Update parameters to defaults.

Source code in model_navigator/api/config.py 
540
541
542
543
544
545
546
547
def defaults(self) -> None:
    """Update parameters to defaults."""
    self.precision = tuple(TensorRTPrecision(p) for p in DEFAULT_TENSORRT_PRECISION)
    self.precision_mode = DEFAULT_TENSORRT_PRECISION_MODE
    self.max_workspace_size = DEFAULT_MAX_WORKSPACE_SIZE
    self.trt_profiles = None
    self.trt_profile = None
    self.run_max_batch_size_search = None

filter_data staticmethod 

filter_data(data, filter_fields)

Filter fields in dictionary.

Parameters:

  • data (Dict) –

    Dictionary with data to filter

  • filter_fields (List[str]) –

    Fields to filter

Returns:

  • –

    Filtered dictionary

Source code in model_navigator/utils/common.py 
103
104
105
106
107
108
109
110
111
112
113
114
115
@staticmethod
def filter_data(data: Dict, filter_fields: List[str]):
    """Filter fields in dictionary.

    Args:
        data: Dictionary with data to filter
        filter_fields: Fields to filter

    Returns:
        Filtered dictionary
    """
    filtered_data = {key: value for key, value in data.items() if key not in filter_fields}
    return filtered_data

from_dict classmethod 

from_dict(config_dict)

Instantiate CustomConfig from a dictionary.

Source code in model_navigator/api/config.py 
491
492
493
494
@classmethod
def from_dict(cls, config_dict: Dict[str, Any]) -> "CustomConfig":
    """Instantiate CustomConfig from a dictionary."""
    return cls(**config_dict)

name abstractmethod classmethod 

name()

Name of the CustomConfig.

Source code in model_navigator/api/config.py 
481
482
483
484
485
@classmethod
@abc.abstractmethod
def name(cls) -> str:
    """Name of the CustomConfig."""
    raise NotImplementedError()

parse_data staticmethod 

parse_data(data)

Parse values in provided data.

Parameters:

  • data (Dict) –

    Dictionary with data to parse

Returns:

  • –

    Parsed dictionary

Source code in model_navigator/utils/common.py 
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
@staticmethod
def parse_data(data: Dict):
    """Parse values in provided data.

    Args:
        data: Dictionary with data to parse

    Returns:
        Parsed dictionary
    """
    parsed_data = {}
    for key, value in data.items():
        parsed_data[key] = DataObject.parse_value(value)

    return parsed_data

parse_value staticmethod 

parse_value(value)

Parse value to jsonable format.

Parameters:

  • value (Any) –

    Value to be parsed.

Returns:

Source code in model_navigator/utils/common.py 
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
@staticmethod
def parse_value(value: Any) -> Union[str, Dict, List]:
    """Parse value to jsonable format.

    Args:
        value (Any): Value to be parsed.

    Returns:
        Union[str, Dict, List]: Jsonable value.
    """
    if isinstance(value, DataObject):
        value = value.to_dict(parse=True)
    elif hasattr(value, "to_json"):
        value = value.to_json()
    elif isinstance(value, (Mapping, Profile)):
        value = DataObject._from_dict(value)
    elif isinstance(value, list) or isinstance(value, tuple):
        value = DataObject._from_list(value)
    elif isinstance(value, Enum):
        value = value.value
    elif isinstance(value, pathlib.Path):
        value = str(value)
    elif isinstance(value, ShapeTuple):
        value = vars(value)

    return value

to_dict 

to_dict(filter_fields=None, parse=False)

Serialize to a dictionary.

Parameters:

  • filter_fields (Optional[List[str]], default: None ) –

    List of fields to filter out. Defaults to None.

  • parse (bool, default: False ) –

    If True recursively parse field values to jsonable representation. Defaults to False.

Returns:

  • Dict ( Dict ) –

    Data serialized to a dictionary.

Source code in model_navigator/utils/common.py 
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
def to_dict(self, filter_fields: Optional[List[str]] = None, parse: bool = False) -> Dict:
    """Serialize to a dictionary.

    Args:
        filter_fields (Optional[List[str]], optional): List of fields to filter out.
            Defaults to None.
        parse (bool, optional): If True recursively parse field values to jsonable representation.
            Defaults to False.

    Returns:
        Dict: Data serialized to a dictionary.
    """
    if filter_fields:
        filtered_data = DataObject.filter_data(
            data=self.__dict__,
            filter_fields=filter_fields,
        )
    else:
        filtered_data = self.__dict__

    if parse:
        data = DataObject.parse_data(filtered_data)
    else:
        data = filtered_data

    return data

DeviceKind

Bases: Enum

Supported types of devices.

Parameters:

  • CPU (str) –

    Select CPU device.

  • GPU (str) –

    Select GPU with CUDA support.

Format

Bases: Enum

All model formats supported by Model Navigator 'optimize' function.

Parameters:

  • PYTHON (str) –

    Format indicating any model defined in Python.

  • TORCH (str) –

    Format indicating PyTorch model.

  • TENSORFLOW (str) –

    Format indicating TensorFlow model.

  • JAX (str) –

    Format indicating JAX model.

  • TORCHSCRIPT (str) –

    Format indicating TorchScript model.

  • TF_SAVEDMODEL (str) –

    Format indicating TensorFlow SavedModel.

  • TF_TRT (str) –

    Format indicating TensorFlow TensorRT model.

  • TORCH_TRT (str) –

    Format indicating PyTorch TensorRT model.

  • ONNX (str) –

    Format indicating ONNX model.

  • TENSORRT (str) –

    Format indicating TensorRT model.

JitType

Bases: Enum

TorchScript export parameter.

Used for selecting the type of TorchScript export.

Parameters:

  • TRACE (str) –

    Use tracing during export.

  • SCRIPT (str) –

    Use scripting during export.

OnnxConfig dataclass

Bases: CustomConfigForFormat

ONNX custom config used for ONNX export and conversion.

Parameters:

  • opset (Optional[int], default: DEFAULT_ONNX_OPSET ) –

    ONNX opset used for conversion.

  • dynamic_axes (Optional[Dict[str, Union[Dict[int, str], List[int]]]], default: None ) –

    Dynamic axes for ONNX conversion.

  • onnx_extended_conversion (bool, default: False ) –

    Enables additional conversions from TorchScript to ONNX.

format property 

format

Returns Format.ONNX.

Returns:

defaults 

defaults()

Update parameters to defaults.

Source code in model_navigator/api/config.py 
487
488
489
def defaults(self) -> None:
    """Update parameters to defaults."""
    return None

filter_data staticmethod 

filter_data(data, filter_fields)

Filter fields in dictionary.

Parameters:

  • data (Dict) –

    Dictionary with data to filter

  • filter_fields (List[str]) –

    Fields to filter

Returns:

  • –

    Filtered dictionary

Source code in model_navigator/utils/common.py 
103
104
105
106
107
108
109
110
111
112
113
114
115
@staticmethod
def filter_data(data: Dict, filter_fields: List[str]):
    """Filter fields in dictionary.

    Args:
        data: Dictionary with data to filter
        filter_fields: Fields to filter

    Returns:
        Filtered dictionary
    """
    filtered_data = {key: value for key, value in data.items() if key not in filter_fields}
    return filtered_data

from_dict classmethod 

from_dict(config_dict)

Instantiate CustomConfig from a dictionary.

Source code in model_navigator/api/config.py 
491
492
493
494
@classmethod
def from_dict(cls, config_dict: Dict[str, Any]) -> "CustomConfig":
    """Instantiate CustomConfig from a dictionary."""
    return cls(**config_dict)

name classmethod 

name()

Name of the config.

Source code in model_navigator/api/config.py 
720
721
722
723
@classmethod
def name(cls) -> str:
    """Name of the config."""
    return "Onnx"

parse_data staticmethod 

parse_data(data)

Parse values in provided data.

Parameters:

  • data (Dict) –

    Dictionary with data to parse

Returns:

  • –

    Parsed dictionary

Source code in model_navigator/utils/common.py 
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
@staticmethod
def parse_data(data: Dict):
    """Parse values in provided data.

    Args:
        data: Dictionary with data to parse

    Returns:
        Parsed dictionary
    """
    parsed_data = {}
    for key, value in data.items():
        parsed_data[key] = DataObject.parse_value(value)

    return parsed_data

parse_value staticmethod 

parse_value(value)

Parse value to jsonable format.

Parameters:

  • value (Any) –

    Value to be parsed.

Returns:

Source code in model_navigator/utils/common.py 
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
@staticmethod
def parse_value(value: Any) -> Union[str, Dict, List]:
    """Parse value to jsonable format.

    Args:
        value (Any): Value to be parsed.

    Returns:
        Union[str, Dict, List]: Jsonable value.
    """
    if isinstance(value, DataObject):
        value = value.to_dict(parse=True)
    elif hasattr(value, "to_json"):
        value = value.to_json()
    elif isinstance(value, (Mapping, Profile)):
        value = DataObject._from_dict(value)
    elif isinstance(value, list) or isinstance(value, tuple):
        value = DataObject._from_list(value)
    elif isinstance(value, Enum):
        value = value.value
    elif isinstance(value, pathlib.Path):
        value = str(value)
    elif isinstance(value, ShapeTuple):
        value = vars(value)

    return value

to_dict 

to_dict(filter_fields=None, parse=False)

Serialize to a dictionary.

Parameters:

  • filter_fields (Optional[List[str]], default: None ) –

    List of fields to filter out. Defaults to None.

  • parse (bool, default: False ) –

    If True recursively parse field values to jsonable representation. Defaults to False.

Returns:

  • Dict ( Dict ) –

    Data serialized to a dictionary.

Source code in model_navigator/utils/common.py 
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
def to_dict(self, filter_fields: Optional[List[str]] = None, parse: bool = False) -> Dict:
    """Serialize to a dictionary.

    Args:
        filter_fields (Optional[List[str]], optional): List of fields to filter out.
            Defaults to None.
        parse (bool, optional): If True recursively parse field values to jsonable representation.
            Defaults to False.

    Returns:
        Dict: Data serialized to a dictionary.
    """
    if filter_fields:
        filtered_data = DataObject.filter_data(
            data=self.__dict__,
            filter_fields=filter_fields,
        )
    else:
        filtered_data = self.__dict__

    if parse:
        data = DataObject.parse_data(filtered_data)
    else:
        data = filtered_data

    return data

OptimizationProfile dataclass

Bases: DataObject

Optimization profile configuration.

For each batch size profiler will run measurements in windows of fixed number of queries. Batch sizes are profiled in the ascending order.

Profiler will run multiple trials and will stop when the measurements are stable (within stability_percentage from the mean) within three consecutive windows. If the measurements are not stable after max_trials trials, the profiler will stop with an error. Profiler will also stop profiling when the throughput does not increase at least by throughput_cutoff_threshold.

Parameters:

  • max_batch_size (Optional[int], default: None ) –

    Maximal batch size used during conversion and profiling. None mean automatic search is enabled.

  • batch_sizes –

    List of batch sizes to profile. None mean automatic search is enabled.

  • window_size (int, default: 50 ) –

    Number of requests to measure in each window.

  • stability_percentage (float, default: 10.0 ) –

    Allowed percentage of variation from the mean in consecutive windows.

  • stabilization_windows (int, default: 3 ) –

    Number consecutive windows selected for stabilization.

  • min_trials (int, default: 3 ) –

    Minimal number of window trials.

  • max_trials (int, default: 10 ) –

    Maximum number of window trials.

  • throughput_cutoff_threshold (float, default: DEFAULT_PROFILING_THROUGHPUT_CUTOFF_THRESHOLD ) –

    Minimum throughput increase to continue profiling.

  • dataloader (Optional[SizedDataLoader], default: None ) –

    Optional dataloader for profiling. Use only 1 sample.

filter_data staticmethod 

filter_data(data, filter_fields)

Filter fields in dictionary.

Parameters:

  • data (Dict) –

    Dictionary with data to filter

  • filter_fields (List[str]) –

    Fields to filter

Returns:

  • –

    Filtered dictionary

Source code in model_navigator/utils/common.py 
103
104
105
106
107
108
109
110
111
112
113
114
115
@staticmethod
def filter_data(data: Dict, filter_fields: List[str]):
    """Filter fields in dictionary.

    Args:
        data: Dictionary with data to filter
        filter_fields: Fields to filter

    Returns:
        Filtered dictionary
    """
    filtered_data = {key: value for key, value in data.items() if key not in filter_fields}
    return filtered_data

from_dict classmethod 

from_dict(optimization_profile_dict)

Instantiate OptimizationProfile class from a dictionary.

Parameters:

  • optimization_profile_dict (Mapping) –

    Data dictionary.

Returns:

Source code in model_navigator/api/config.py 
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
@classmethod
def from_dict(cls, optimization_profile_dict: Mapping) -> "OptimizationProfile":
    """Instantiate OptimizationProfile class from a dictionary.

    Args:
        optimization_profile_dict (Mapping): Data dictionary.

    Returns:
        OptimizationProfile
    """
    return cls(
        max_batch_size=optimization_profile_dict.get("max_batch_size"),
        batch_sizes=optimization_profile_dict.get("batch_sizes"),
        window_size=optimization_profile_dict.get("window_size", 50),
        stability_percentage=optimization_profile_dict.get("stability_percentage", 10.0),
        stabilization_windows=optimization_profile_dict.get("stabilization_windows", 3),
        min_trials=optimization_profile_dict.get("min_trials", 3),
        max_trials=optimization_profile_dict.get("max_trials", 10),
        throughput_cutoff_threshold=optimization_profile_dict.get(
            "throughput_cutoff_threshold", DEFAULT_PROFILING_THROUGHPUT_CUTOFF_THRESHOLD
        ),
    )

parse_data staticmethod 

parse_data(data)

Parse values in provided data.

Parameters:

  • data (Dict) –

    Dictionary with data to parse

Returns:

  • –

    Parsed dictionary

Source code in model_navigator/utils/common.py 
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
@staticmethod
def parse_data(data: Dict):
    """Parse values in provided data.

    Args:
        data: Dictionary with data to parse

    Returns:
        Parsed dictionary
    """
    parsed_data = {}
    for key, value in data.items():
        parsed_data[key] = DataObject.parse_value(value)

    return parsed_data

parse_value staticmethod 

parse_value(value)

Parse value to jsonable format.

Parameters:

  • value (Any) –

    Value to be parsed.

Returns:

Source code in model_navigator/utils/common.py 
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
@staticmethod
def parse_value(value: Any) -> Union[str, Dict, List]:
    """Parse value to jsonable format.

    Args:
        value (Any): Value to be parsed.

    Returns:
        Union[str, Dict, List]: Jsonable value.
    """
    if isinstance(value, DataObject):
        value = value.to_dict(parse=True)
    elif hasattr(value, "to_json"):
        value = value.to_json()
    elif isinstance(value, (Mapping, Profile)):
        value = DataObject._from_dict(value)
    elif isinstance(value, list) or isinstance(value, tuple):
        value = DataObject._from_list(value)
    elif isinstance(value, Enum):
        value = value.value
    elif isinstance(value, pathlib.Path):
        value = str(value)
    elif isinstance(value, ShapeTuple):
        value = vars(value)

    return value

to_dict 

to_dict(filter_fields=None, parse=False)

Serialize to a dictionary.

Append dataloader field to filtered fields during dump.

Parameters:

  • filter_fields (Optional[List[str]], default: None ) –

    List of fields to filter out. Defaults to None.

  • parse (bool, default: False ) –

    If True recursively parse field values to jsonable representation. Defaults to False.

Returns:

  • Dict ( Dict ) –

    Data serialized to a dictionary.

Source code in model_navigator/api/config.py 
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
def to_dict(self, filter_fields: Optional[List[str]] = None, parse: bool = False) -> Dict:
    """Serialize to a dictionary.

    Append `dataloader` field to filtered fields during dump.

    Args:
        filter_fields (Optional[List[str]], optional): List of fields to filter out.
            Defaults to None.
        parse (bool, optional): If True recursively parse field values to jsonable representation.
            Defaults to False.

    Returns:
        Dict: Data serialized to a dictionary.
    """
    if not filter_fields:
        filter_fields = []

    filter_fields += ["dataloader"]
    return super().to_dict(filter_fields=filter_fields, parse=parse)

ShapeTuple dataclass

Bases: DataObject

Represents a set of shapes for a single binding in a profile.

Each element of the tuple represents a shape for a single dimension of the binding.

Parameters:

  • min (Tuple[int]) –

    The minimum shape that the profile will support.

  • opt (Tuple[int]) –

    The shape for which TensorRT will optimize the engine.

  • max (Tuple[int]) –

    The maximum shape that the profile will support.

__iter__ 

__iter__()

Iterate over shapes.

Source code in model_navigator/api/config.py 
205
206
207
def __iter__(self):
    """Iterate over shapes."""
    yield from [self.min, self.opt, self.max]

__repr__ 

__repr__()

Representation.

Source code in model_navigator/api/config.py 
201
202
203
def __repr__(self):
    """Representation."""
    return type(self).__name__ + self.__str__()

__str__ 

__str__()

String representation.

Source code in model_navigator/api/config.py 
197
198
199
def __str__(self):
    """String representation."""
    return f"(min={self.min}, opt={self.opt}, max={self.max})"

filter_data staticmethod 

filter_data(data, filter_fields)

Filter fields in dictionary.

Parameters:

  • data (Dict) –

    Dictionary with data to filter

  • filter_fields (List[str]) –

    Fields to filter

Returns:

  • –

    Filtered dictionary

Source code in model_navigator/utils/common.py 
103
104
105
106
107
108
109
110
111
112
113
114
115
@staticmethod
def filter_data(data: Dict, filter_fields: List[str]):
    """Filter fields in dictionary.

    Args:
        data: Dictionary with data to filter
        filter_fields: Fields to filter

    Returns:
        Filtered dictionary
    """
    filtered_data = {key: value for key, value in data.items() if key not in filter_fields}
    return filtered_data

parse_data staticmethod 

parse_data(data)

Parse values in provided data.

Parameters:

  • data (Dict) –

    Dictionary with data to parse

Returns:

  • –

    Parsed dictionary

Source code in model_navigator/utils/common.py 
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
@staticmethod
def parse_data(data: Dict):
    """Parse values in provided data.

    Args:
        data: Dictionary with data to parse

    Returns:
        Parsed dictionary
    """
    parsed_data = {}
    for key, value in data.items():
        parsed_data[key] = DataObject.parse_value(value)

    return parsed_data

parse_value staticmethod 

parse_value(value)

Parse value to jsonable format.

Parameters:

  • value (Any) –

    Value to be parsed.

Returns:

Source code in model_navigator/utils/common.py 
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
@staticmethod
def parse_value(value: Any) -> Union[str, Dict, List]:
    """Parse value to jsonable format.

    Args:
        value (Any): Value to be parsed.

    Returns:
        Union[str, Dict, List]: Jsonable value.
    """
    if isinstance(value, DataObject):
        value = value.to_dict(parse=True)
    elif hasattr(value, "to_json"):
        value = value.to_json()
    elif isinstance(value, (Mapping, Profile)):
        value = DataObject._from_dict(value)
    elif isinstance(value, list) or isinstance(value, tuple):
        value = DataObject._from_list(value)
    elif isinstance(value, Enum):
        value = value.value
    elif isinstance(value, pathlib.Path):
        value = str(value)
    elif isinstance(value, ShapeTuple):
        value = vars(value)

    return value

to_dict 

to_dict(filter_fields=None, parse=False)

Serialize to a dictionary.

Parameters:

  • filter_fields (Optional[List[str]], default: None ) –

    List of fields to filter out. Defaults to None.

  • parse (bool, default: False ) –

    If True recursively parse field values to jsonable representation. Defaults to False.

Returns:

  • Dict ( Dict ) –

    Data serialized to a dictionary.

Source code in model_navigator/utils/common.py 
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
def to_dict(self, filter_fields: Optional[List[str]] = None, parse: bool = False) -> Dict:
    """Serialize to a dictionary.

    Args:
        filter_fields (Optional[List[str]], optional): List of fields to filter out.
            Defaults to None.
        parse (bool, optional): If True recursively parse field values to jsonable representation.
            Defaults to False.

    Returns:
        Dict: Data serialized to a dictionary.
    """
    if filter_fields:
        filtered_data = DataObject.filter_data(
            data=self.__dict__,
            filter_fields=filter_fields,
        )
    else:
        filtered_data = self.__dict__

    if parse:
        data = DataObject.parse_data(filtered_data)
    else:
        data = filtered_data

    return data

SizedIterable

Bases: Protocol

Protocol representing sized iterable. Used by dataloader.

__iter__ 

__iter__()

Magic method iter.

Returns:

Source code in model_navigator/api/config.py 
72
73
74
75
76
77
78
def __iter__(self) -> Iterator:
    """Magic method __iter__.

    Returns:
        Iterator to next item.
    """
    ...

__len__ 

__len__()

Magic method len.

Returns:

  • int –

    Length of size iterable.

Source code in model_navigator/api/config.py 
80
81
82
83
84
85
86
def __len__(self) -> int:
    """Magic method __len__.

    Returns:
        Length of size iterable.
    """
    ...

TensorFlowConfig dataclass

Bases: CustomConfigForFormat

TensorFlow custom config used for SavedModel export.

Parameters:

  • jit_compile (Tuple[Optional[bool], ...], default: (None) ) –

    Enable or Disable jit_compile flag for tf.function wrapper for Jax infer function.

  • enable_xla (Tuple[Optional[bool], ...], default: (None) ) –

    Enable or Disable enable_xla flag for jax2tf converter.

format property 

format

Returns Format.TF_SAVEDMODEL.

Returns:

  • Format –

    Format.TF_SAVEDMODEL

defaults 

defaults()

Update parameters to defaults.

Source code in model_navigator/api/config.py 
577
578
579
580
def defaults(self) -> None:
    """Update parameters to defaults."""
    self.jit_compile = (None,)
    self.enable_xla = (None,)

filter_data staticmethod 

filter_data(data, filter_fields)

Filter fields in dictionary.

Parameters:

  • data (Dict) –

    Dictionary with data to filter

  • filter_fields (List[str]) –

    Fields to filter

Returns:

  • –

    Filtered dictionary

Source code in model_navigator/utils/common.py 
103
104
105
106
107
108
109
110
111
112
113
114
115
@staticmethod
def filter_data(data: Dict, filter_fields: List[str]):
    """Filter fields in dictionary.

    Args:
        data: Dictionary with data to filter
        filter_fields: Fields to filter

    Returns:
        Filtered dictionary
    """
    filtered_data = {key: value for key, value in data.items() if key not in filter_fields}
    return filtered_data

from_dict classmethod 

from_dict(config_dict)

Instantiate CustomConfig from a dictionary.

Source code in model_navigator/api/config.py 
491
492
493
494
@classmethod
def from_dict(cls, config_dict: Dict[str, Any]) -> "CustomConfig":
    """Instantiate CustomConfig from a dictionary."""
    return cls(**config_dict)

name classmethod 

name()

Name of the config.

Source code in model_navigator/api/config.py 
572
573
574
575
@classmethod
def name(cls) -> str:
    """Name of the config."""
    return "TensorFlow"

parse_data staticmethod 

parse_data(data)

Parse values in provided data.

Parameters:

  • data (Dict) –

    Dictionary with data to parse

Returns:

  • –

    Parsed dictionary

Source code in model_navigator/utils/common.py 
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
@staticmethod
def parse_data(data: Dict):
    """Parse values in provided data.

    Args:
        data: Dictionary with data to parse

    Returns:
        Parsed dictionary
    """
    parsed_data = {}
    for key, value in data.items():
        parsed_data[key] = DataObject.parse_value(value)

    return parsed_data

parse_value staticmethod 

parse_value(value)

Parse value to jsonable format.

Parameters:

  • value (Any) –

    Value to be parsed.

Returns:

Source code in model_navigator/utils/common.py 
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
@staticmethod
def parse_value(value: Any) -> Union[str, Dict, List]:
    """Parse value to jsonable format.

    Args:
        value (Any): Value to be parsed.

    Returns:
        Union[str, Dict, List]: Jsonable value.
    """
    if isinstance(value, DataObject):
        value = value.to_dict(parse=True)
    elif hasattr(value, "to_json"):
        value = value.to_json()
    elif isinstance(value, (Mapping, Profile)):
        value = DataObject._from_dict(value)
    elif isinstance(value, list) or isinstance(value, tuple):
        value = DataObject._from_list(value)
    elif isinstance(value, Enum):
        value = value.value
    elif isinstance(value, pathlib.Path):
        value = str(value)
    elif isinstance(value, ShapeTuple):
        value = vars(value)

    return value

to_dict 

to_dict(filter_fields=None, parse=False)

Serialize to a dictionary.

Parameters:

  • filter_fields (Optional[List[str]], default: None ) –

    List of fields to filter out. Defaults to None.

  • parse (bool, default: False ) –

    If True recursively parse field values to jsonable representation. Defaults to False.

Returns:

  • Dict ( Dict ) –

    Data serialized to a dictionary.

Source code in model_navigator/utils/common.py 
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
def to_dict(self, filter_fields: Optional[List[str]] = None, parse: bool = False) -> Dict:
    """Serialize to a dictionary.

    Args:
        filter_fields (Optional[List[str]], optional): List of fields to filter out.
            Defaults to None.
        parse (bool, optional): If True recursively parse field values to jsonable representation.
            Defaults to False.

    Returns:
        Dict: Data serialized to a dictionary.
    """
    if filter_fields:
        filtered_data = DataObject.filter_data(
            data=self.__dict__,
            filter_fields=filter_fields,
        )
    else:
        filtered_data = self.__dict__

    if parse:
        data = DataObject.parse_data(filtered_data)
    else:
        data = filtered_data

    return data

TensorFlowTensorRTConfig dataclass

Bases: CustomConfigForTensorRT

TensorFlow TensorRT custom config used for TensorRT SavedModel export.

Parameters:

  • minimum_segment_size (int, default: DEFAULT_MIN_SEGMENT_SIZE ) –

    Min size of subgraph.

format property 

format

Returns Format.TF_TRT.

Returns:

defaults 

defaults()

Update parameters to defaults.

Source code in model_navigator/api/config.py 
607
608
609
610
def defaults(self) -> None:
    """Update parameters to defaults."""
    super().defaults()
    self.minimum_segment_size = DEFAULT_MIN_SEGMENT_SIZE

filter_data staticmethod 

filter_data(data, filter_fields)

Filter fields in dictionary.

Parameters:

  • data (Dict) –

    Dictionary with data to filter

  • filter_fields (List[str]) –

    Fields to filter

Returns:

  • –

    Filtered dictionary

Source code in model_navigator/utils/common.py 
103
104
105
106
107
108
109
110
111
112
113
114
115
@staticmethod
def filter_data(data: Dict, filter_fields: List[str]):
    """Filter fields in dictionary.

    Args:
        data: Dictionary with data to filter
        filter_fields: Fields to filter

    Returns:
        Filtered dictionary
    """
    filtered_data = {key: value for key, value in data.items() if key not in filter_fields}
    return filtered_data

from_dict classmethod 

from_dict(config_dict)

Instantiate TensorFlowTensorRTConfig from adictionary.

Source code in model_navigator/api/config.py 
612
613
614
615
616
617
618
619
620
621
622
623
@classmethod
def from_dict(cls, config_dict: Dict[str, Any]) -> "TensorFlowTensorRTConfig":
    """Instantiate TensorFlowTensorRTConfig from  adictionary."""
    if config_dict.get("trt_profiles") is not None:
        # if config_dict.get("trt_profiles") is not None and not isinstance(config_dict["trt_profile"], TensorRTProfile):
        parsed_trt_profiles = []
        for trt_profile in config_dict.get("trt_profiles"):
            if not isinstance(trt_profile, TensorRTProfile):
                trt_profile = TensorRTProfile.from_dict(trt_profile)
            parsed_trt_profiles.append(trt_profile)
        config_dict["trt_profiles"] = parsed_trt_profiles
    return cls(**config_dict)

name classmethod 

name()

Name of the config.

Source code in model_navigator/api/config.py 
602
603
604
605
@classmethod
def name(cls) -> str:
    """Name of the config."""
    return "TensorFlowTensorRT"

parse_data staticmethod 

parse_data(data)

Parse values in provided data.

Parameters:

  • data (Dict) –

    Dictionary with data to parse

Returns:

  • –

    Parsed dictionary

Source code in model_navigator/utils/common.py 
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
@staticmethod
def parse_data(data: Dict):
    """Parse values in provided data.

    Args:
        data: Dictionary with data to parse

    Returns:
        Parsed dictionary
    """
    parsed_data = {}
    for key, value in data.items():
        parsed_data[key] = DataObject.parse_value(value)

    return parsed_data

parse_value staticmethod 

parse_value(value)

Parse value to jsonable format.

Parameters:

  • value (Any) –

    Value to be parsed.

Returns:

Source code in model_navigator/utils/common.py 
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
@staticmethod
def parse_value(value: Any) -> Union[str, Dict, List]:
    """Parse value to jsonable format.

    Args:
        value (Any): Value to be parsed.

    Returns:
        Union[str, Dict, List]: Jsonable value.
    """
    if isinstance(value, DataObject):
        value = value.to_dict(parse=True)
    elif hasattr(value, "to_json"):
        value = value.to_json()
    elif isinstance(value, (Mapping, Profile)):
        value = DataObject._from_dict(value)
    elif isinstance(value, list) or isinstance(value, tuple):
        value = DataObject._from_list(value)
    elif isinstance(value, Enum):
        value = value.value
    elif isinstance(value, pathlib.Path):
        value = str(value)
    elif isinstance(value, ShapeTuple):
        value = vars(value)

    return value

to_dict 

to_dict(filter_fields=None, parse=False)

Serialize to a dictionary.

Parameters:

  • filter_fields (Optional[List[str]], default: None ) –

    List of fields to filter out. Defaults to None.

  • parse (bool, default: False ) –

    If True recursively parse field values to jsonable representation. Defaults to False.

Returns:

  • Dict ( Dict ) –

    Data serialized to a dictionary.

Source code in model_navigator/utils/common.py 
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
def to_dict(self, filter_fields: Optional[List[str]] = None, parse: bool = False) -> Dict:
    """Serialize to a dictionary.

    Args:
        filter_fields (Optional[List[str]], optional): List of fields to filter out.
            Defaults to None.
        parse (bool, optional): If True recursively parse field values to jsonable representation.
            Defaults to False.

    Returns:
        Dict: Data serialized to a dictionary.
    """
    if filter_fields:
        filtered_data = DataObject.filter_data(
            data=self.__dict__,
            filter_fields=filter_fields,
        )
    else:
        filtered_data = self.__dict__

    if parse:
        data = DataObject.parse_data(filtered_data)
    else:
        data = filtered_data

    return data

TensorRTCompatibilityLevel

Bases: Enum

Compatibility level for TensorRT.

Parameters:

  • AMPERE_PLUS (str) –

    Support AMPERE plus architecture

TensorRTConfig dataclass

Bases: CustomConfigForTensorRT

TensorRT custom config used for TensorRT conversion.

Parameters:

  • optimization_level (Optional[int], default: None ) –

    Optimization level for TensorRT conversion. Allowed values are fom 0 to 5. Where default is 3 based on TensorRT API documentation.

format property 

format

Returns Format.TENSORRT.

Returns:

defaults 

defaults()

Update parameters to defaults.

Source code in model_navigator/api/config.py 
760
761
762
763
764
def defaults(self) -> None:
    """Update parameters to defaults."""
    super().defaults()
    self.optimization_level = None
    self.compatibility_level = None

filter_data staticmethod 

filter_data(data, filter_fields)

Filter fields in dictionary.

Parameters:

  • data (Dict) –

    Dictionary with data to filter

  • filter_fields (List[str]) –

    Fields to filter

Returns:

  • –

    Filtered dictionary

Source code in model_navigator/utils/common.py 
103
104
105
106
107
108
109
110
111
112
113
114
115
@staticmethod
def filter_data(data: Dict, filter_fields: List[str]):
    """Filter fields in dictionary.

    Args:
        data: Dictionary with data to filter
        filter_fields: Fields to filter

    Returns:
        Filtered dictionary
    """
    filtered_data = {key: value for key, value in data.items() if key not in filter_fields}
    return filtered_data

from_dict classmethod 

from_dict(config_dict)

Instantiate TensorRTConfig from adictionary.

Source code in model_navigator/api/config.py 
766
767
768
769
770
771
772
773
774
775
776
@classmethod
def from_dict(cls, config_dict: Dict[str, Any]) -> "TensorRTConfig":
    """Instantiate TensorRTConfig from  adictionary."""
    if config_dict.get("trt_profiles") is not None:
        parsed_trt_profiles = []
        for trt_profile in config_dict.get("trt_profiles"):
            if not isinstance(trt_profile, TensorRTProfile):
                trt_profile = TensorRTProfile.from_dict(trt_profile)
            parsed_trt_profiles.append(trt_profile)
        config_dict["trt_profiles"] = parsed_trt_profiles
    return cls(**config_dict)

name classmethod 

name()

Name of the config.

Source code in model_navigator/api/config.py 
755
756
757
758
@classmethod
def name(cls) -> str:
    """Name of the config."""
    return "TensorRT"

parse_data staticmethod 

parse_data(data)

Parse values in provided data.

Parameters:

  • data (Dict) –

    Dictionary with data to parse

Returns:

  • –

    Parsed dictionary

Source code in model_navigator/utils/common.py 
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
@staticmethod
def parse_data(data: Dict):
    """Parse values in provided data.

    Args:
        data: Dictionary with data to parse

    Returns:
        Parsed dictionary
    """
    parsed_data = {}
    for key, value in data.items():
        parsed_data[key] = DataObject.parse_value(value)

    return parsed_data

parse_value staticmethod 

parse_value(value)

Parse value to jsonable format.

Parameters:

  • value (Any) –

    Value to be parsed.

Returns:

Source code in model_navigator/utils/common.py 
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
@staticmethod
def parse_value(value: Any) -> Union[str, Dict, List]:
    """Parse value to jsonable format.

    Args:
        value (Any): Value to be parsed.

    Returns:
        Union[str, Dict, List]: Jsonable value.
    """
    if isinstance(value, DataObject):
        value = value.to_dict(parse=True)
    elif hasattr(value, "to_json"):
        value = value.to_json()
    elif isinstance(value, (Mapping, Profile)):
        value = DataObject._from_dict(value)
    elif isinstance(value, list) or isinstance(value, tuple):
        value = DataObject._from_list(value)
    elif isinstance(value, Enum):
        value = value.value
    elif isinstance(value, pathlib.Path):
        value = str(value)
    elif isinstance(value, ShapeTuple):
        value = vars(value)

    return value

to_dict 

to_dict(filter_fields=None, parse=False)

Serialize to a dictionary.

Parameters:

  • filter_fields (Optional[List[str]], default: None ) –

    List of fields to filter out. Defaults to None.

  • parse (bool, default: False ) –

    If True recursively parse field values to jsonable representation. Defaults to False.

Returns:

  • Dict ( Dict ) –

    Data serialized to a dictionary.

Source code in model_navigator/utils/common.py 
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
def to_dict(self, filter_fields: Optional[List[str]] = None, parse: bool = False) -> Dict:
    """Serialize to a dictionary.

    Args:
        filter_fields (Optional[List[str]], optional): List of fields to filter out.
            Defaults to None.
        parse (bool, optional): If True recursively parse field values to jsonable representation.
            Defaults to False.

    Returns:
        Dict: Data serialized to a dictionary.
    """
    if filter_fields:
        filtered_data = DataObject.filter_data(
            data=self.__dict__,
            filter_fields=filter_fields,
        )
    else:
        filtered_data = self.__dict__

    if parse:
        data = DataObject.parse_data(filtered_data)
    else:
        data = filtered_data

    return data

TensorRTPrecision

Bases: Enum

Precisions supported during TensorRT conversions.

Parameters:

  • INT8 (str) –

    8-bit integer precision.

  • FP16 (str) –

    16-bit floating point precision.

  • FP32 (str) –

    32-bit floating point precision.

TensorRTPrecisionMode

Bases: Enum

Precision modes for TensorRT conversions.

Parameters:

  • HIERARCHY (str) –

    Use TensorRT precision hierarchy starting from highest to lowest.

  • SINGLE (str) –

    Use single precision.

  • MIXED (str) –

    Use mixed precision.

TensorRTProfile

Bases: Dict[str, ShapeTuple]

Single optimization profile that can be used to build an engine.

More specifically, it is an Dict[str, ShapeTuple] which maps binding names to a set of min/opt/max shapes.

__getitem__ 

__getitem__(key)

Retrieves the shapes registered for a given input name.

Returns:

  • ShapeTuple – 
    A named tuple including ``min``, ``opt``, and ``max`` members for the shapes
corresponding to the input.
Source code in model_navigator/api/config.py 
353
354
355
356
357
358
359
360
361
362
363
def __getitem__(self, key):
    """Retrieves the shapes registered for a given input name.

    Returns:
        ShapeTuple:
                A named tuple including ``min``, ``opt``, and ``max`` members for the shapes
                corresponding to the input.
    """
    if key not in self:
        LOGGER.error(f"Binding: {key} does not have shapes set in this profile")
    return super().__getitem__(key)

__repr__ 

__repr__()

Representation.

Source code in model_navigator/api/config.py 
365
366
367
368
369
370
def __repr__(self):
    """Representation."""
    ret = "TensorRTProfile()"
    for name, (min, opt, max) in self.items():
        ret += f".add('{name}', min={min}, opt={opt}, max={max})"
    return ret

__str__ 

__str__()

String representation.

Source code in model_navigator/api/config.py 
372
373
374
375
376
377
378
379
def __str__(self):
    """String representation."""
    elems = []
    for name, (min, opt, max) in self.items():
        elems.append(f"{name} [min={min}, opt={opt}, max={max}]")

    sep = ",\n "
    return "{" + sep.join(elems) + "}"

add 

add(name, min, opt, max)

A convenience function to add shapes for a single binding.

Parameters:

  • name (str) –

    The name of the binding.

  • min (Tuple[int]) –

    The minimum shape that the profile will support.

  • opt (Tuple[int]) –

    The shape for which TensorRT will optimize the engine.

  • max (Tuple[int]) –

    The maximum shape that the profile will support.

Returns:

  • Profile –

    self, which allows this function to be easily chained to add multiple bindings, e.g., TensorRTProfile().add(...).add(...)

Source code in model_navigator/api/config.py 
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
def add(self, name, min, opt, max):
    """A convenience function to add shapes for a single binding.

    Args:
        name (str): The name of the binding.
        min (Tuple[int]): The minimum shape that the profile will support.
        opt (Tuple[int]): The shape for which TensorRT will optimize the engine.
        max (Tuple[int]): The maximum shape that the profile will support.

    Returns:
        Profile:
            self, which allows this function to be easily chained to add multiple bindings,
            e.g., TensorRTProfile().add(...).add(...)
    """
    self[name] = ShapeTuple(min, opt, max)
    return self

from_dict classmethod 

from_dict(profile_dict)

Create a TensorRTProfile from a dictionary.

Parameters:

  • profile_dict (Dict[str, Dict[str, Tuple[int, ...]]]) –

    A dictionary mapping binding names to a dictionary containing min, opt, and max keys.

Returns:

  • TensorRTProfile –

    A TensorRTProfile object.

Source code in model_navigator/api/config.py 
338
339
340
341
342
343
344
345
346
347
348
349
350
351
@classmethod
def from_dict(cls, profile_dict: Dict[str, Dict[str, Tuple[int, ...]]]):
    """Create a TensorRTProfile from a dictionary.

    Args:
        profile_dict (Dict[str, Dict[str, Tuple[int, ...]]]):
            A dictionary mapping binding names to a dictionary containing ``min``, ``opt``, and
            ``max`` keys.

    Returns:
        TensorRTProfile:
            A TensorRTProfile object.
    """
    return cls({name: ShapeTuple(**shapes) for name, shapes in profile_dict.items()})

TensorType

Bases: Enum

All model formats supported by Model Navigator 'optimize' function.

TorchConfig dataclass

Bases: CustomConfigForFormat

Torch custom config used for TorchScript export.

Parameters:

  • jit_type (Union[Union[str, JitType], Tuple[Union[str, JitType], ...]], default: (SCRIPT, TRACE) ) –

    Type of TorchScript export.

  • strict (bool, default: True ) –

    Enable or Disable strict flag for tracer used in TorchScript export, default: True.

format property 

format

Returns Format.TORCHSCRIPT.

Returns:

  • Format –

    Format.TORCHSCRIPT

defaults 

defaults()

Update parameters to defaults.

Source code in model_navigator/api/config.py 
658
659
660
661
def defaults(self) -> None:
    """Update parameters to defaults."""
    self.jit_type = (JitType.SCRIPT, JitType.TRACE)
    self.strict = True

filter_data staticmethod 

filter_data(data, filter_fields)

Filter fields in dictionary.

Parameters:

  • data (Dict) –

    Dictionary with data to filter

  • filter_fields (List[str]) –

    Fields to filter

Returns:

  • –

    Filtered dictionary

Source code in model_navigator/utils/common.py 
103
104
105
106
107
108
109
110
111
112
113
114
115
@staticmethod
def filter_data(data: Dict, filter_fields: List[str]):
    """Filter fields in dictionary.

    Args:
        data: Dictionary with data to filter
        filter_fields: Fields to filter

    Returns:
        Filtered dictionary
    """
    filtered_data = {key: value for key, value in data.items() if key not in filter_fields}
    return filtered_data

from_dict classmethod 

from_dict(config_dict)

Instantiate CustomConfig from a dictionary.

Source code in model_navigator/api/config.py 
491
492
493
494
@classmethod
def from_dict(cls, config_dict: Dict[str, Any]) -> "CustomConfig":
    """Instantiate CustomConfig from a dictionary."""
    return cls(**config_dict)

name classmethod 

name()

Name of the config.

Source code in model_navigator/api/config.py 
653
654
655
656
@classmethod
def name(cls) -> str:
    """Name of the config."""
    return "Torch"

parse_data staticmethod 

parse_data(data)

Parse values in provided data.

Parameters:

  • data (Dict) –

    Dictionary with data to parse

Returns:

  • –

    Parsed dictionary

Source code in model_navigator/utils/common.py 
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
@staticmethod
def parse_data(data: Dict):
    """Parse values in provided data.

    Args:
        data: Dictionary with data to parse

    Returns:
        Parsed dictionary
    """
    parsed_data = {}
    for key, value in data.items():
        parsed_data[key] = DataObject.parse_value(value)

    return parsed_data

parse_value staticmethod 

parse_value(value)

Parse value to jsonable format.

Parameters:

  • value (Any) –

    Value to be parsed.

Returns:

Source code in model_navigator/utils/common.py 
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
@staticmethod
def parse_value(value: Any) -> Union[str, Dict, List]:
    """Parse value to jsonable format.

    Args:
        value (Any): Value to be parsed.

    Returns:
        Union[str, Dict, List]: Jsonable value.
    """
    if isinstance(value, DataObject):
        value = value.to_dict(parse=True)
    elif hasattr(value, "to_json"):
        value = value.to_json()
    elif isinstance(value, (Mapping, Profile)):
        value = DataObject._from_dict(value)
    elif isinstance(value, list) or isinstance(value, tuple):
        value = DataObject._from_list(value)
    elif isinstance(value, Enum):
        value = value.value
    elif isinstance(value, pathlib.Path):
        value = str(value)
    elif isinstance(value, ShapeTuple):
        value = vars(value)

    return value

to_dict 

to_dict(filter_fields=None, parse=False)

Serialize to a dictionary.

Parameters:

  • filter_fields (Optional[List[str]], default: None ) –

    List of fields to filter out. Defaults to None.

  • parse (bool, default: False ) –

    If True recursively parse field values to jsonable representation. Defaults to False.

Returns:

  • Dict ( Dict ) –

    Data serialized to a dictionary.

Source code in model_navigator/utils/common.py 
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
def to_dict(self, filter_fields: Optional[List[str]] = None, parse: bool = False) -> Dict:
    """Serialize to a dictionary.

    Args:
        filter_fields (Optional[List[str]], optional): List of fields to filter out.
            Defaults to None.
        parse (bool, optional): If True recursively parse field values to jsonable representation.
            Defaults to False.

    Returns:
        Dict: Data serialized to a dictionary.
    """
    if filter_fields:
        filtered_data = DataObject.filter_data(
            data=self.__dict__,
            filter_fields=filter_fields,
        )
    else:
        filtered_data = self.__dict__

    if parse:
        data = DataObject.parse_data(filtered_data)
    else:
        data = filtered_data

    return data

TorchTensorRTConfig dataclass

Bases: CustomConfigForTensorRT

Torch custom config used for TensorRT TorchScript conversion.

format property 

format

Returns Format.TORCH_TRT.

Returns:

defaults 

defaults()

Update parameters to defaults.

Source code in model_navigator/api/config.py 
540
541
542
543
544
545
546
547
def defaults(self) -> None:
    """Update parameters to defaults."""
    self.precision = tuple(TensorRTPrecision(p) for p in DEFAULT_TENSORRT_PRECISION)
    self.precision_mode = DEFAULT_TENSORRT_PRECISION_MODE
    self.max_workspace_size = DEFAULT_MAX_WORKSPACE_SIZE
    self.trt_profiles = None
    self.trt_profile = None
    self.run_max_batch_size_search = None

filter_data staticmethod 

filter_data(data, filter_fields)

Filter fields in dictionary.

Parameters:

  • data (Dict) –

    Dictionary with data to filter

  • filter_fields (List[str]) –

    Fields to filter

Returns:

  • –

    Filtered dictionary

Source code in model_navigator/utils/common.py 
103
104
105
106
107
108
109
110
111
112
113
114
115
@staticmethod
def filter_data(data: Dict, filter_fields: List[str]):
    """Filter fields in dictionary.

    Args:
        data: Dictionary with data to filter
        filter_fields: Fields to filter

    Returns:
        Filtered dictionary
    """
    filtered_data = {key: value for key, value in data.items() if key not in filter_fields}
    return filtered_data

from_dict classmethod 

from_dict(config_dict)

Instantiate TorchTensorRTConfig from adictionary.

Source code in model_navigator/api/config.py 
682
683
684
685
686
687
688
689
690
691
692
693
@classmethod
def from_dict(cls, config_dict: Dict[str, Any]) -> "TorchTensorRTConfig":
    """Instantiate TorchTensorRTConfig from  adictionary."""
    if config_dict.get("trt_profiles") is not None:
        # if config_dict.get("trt_profiles") is not None and not isinstance(config_dict["trt_profile"], TensorRTProfile):
        parsed_trt_profiles = []
        for trt_profile in config_dict.get("trt_profiles"):
            if not isinstance(trt_profile, TensorRTProfile):
                trt_profile = TensorRTProfile.from_dict(trt_profile)
            parsed_trt_profiles.append(trt_profile)
        config_dict["trt_profiles"] = parsed_trt_profiles
    return cls(**config_dict)

name classmethod 

name()

Name of the config.

Source code in model_navigator/api/config.py 
677
678
679
680
@classmethod
def name(cls) -> str:
    """Name of the config."""
    return "TorchTensorRT"

parse_data staticmethod 

parse_data(data)

Parse values in provided data.

Parameters:

  • data (Dict) –

    Dictionary with data to parse

Returns:

  • –

    Parsed dictionary

Source code in model_navigator/utils/common.py 
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
@staticmethod
def parse_data(data: Dict):
    """Parse values in provided data.

    Args:
        data: Dictionary with data to parse

    Returns:
        Parsed dictionary
    """
    parsed_data = {}
    for key, value in data.items():
        parsed_data[key] = DataObject.parse_value(value)

    return parsed_data

parse_value staticmethod 

parse_value(value)

Parse value to jsonable format.

Parameters:

  • value (Any) –

    Value to be parsed.

Returns:

Source code in model_navigator/utils/common.py 
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
@staticmethod
def parse_value(value: Any) -> Union[str, Dict, List]:
    """Parse value to jsonable format.

    Args:
        value (Any): Value to be parsed.

    Returns:
        Union[str, Dict, List]: Jsonable value.
    """
    if isinstance(value, DataObject):
        value = value.to_dict(parse=True)
    elif hasattr(value, "to_json"):
        value = value.to_json()
    elif isinstance(value, (Mapping, Profile)):
        value = DataObject._from_dict(value)
    elif isinstance(value, list) or isinstance(value, tuple):
        value = DataObject._from_list(value)
    elif isinstance(value, Enum):
        value = value.value
    elif isinstance(value, pathlib.Path):
        value = str(value)
    elif isinstance(value, ShapeTuple):
        value = vars(value)

    return value

to_dict 

to_dict(filter_fields=None, parse=False)

Serialize to a dictionary.

Parameters:

  • filter_fields (Optional[List[str]], default: None ) –

    List of fields to filter out. Defaults to None.

  • parse (bool, default: False ) –

    If True recursively parse field values to jsonable representation. Defaults to False.

Returns:

  • Dict ( Dict ) –

    Data serialized to a dictionary.

Source code in model_navigator/utils/common.py 
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
def to_dict(self, filter_fields: Optional[List[str]] = None, parse: bool = False) -> Dict:
    """Serialize to a dictionary.

    Args:
        filter_fields (Optional[List[str]], optional): List of fields to filter out.
            Defaults to None.
        parse (bool, optional): If True recursively parse field values to jsonable representation.
            Defaults to False.

    Returns:
        Dict: Data serialized to a dictionary.
    """
    if filter_fields:
        filtered_data = DataObject.filter_data(
            data=self.__dict__,
            filter_fields=filter_fields,
        )
    else:
        filtered_data = self.__dict__

    if parse:
        data = DataObject.parse_data(filtered_data)
    else:
        data = filtered_data

    return data

map_custom_configs 

map_custom_configs(custom_configs)

Map custom configs from list to dictionary.

Parameters:

Returns:

  • Dict –

    Mapped configs to dictionary

Source code in model_navigator/api/config.py 
779
780
781
782
783
784
785
786
787
788
789
790
791
def map_custom_configs(custom_configs: Optional[Sequence[CustomConfig]]) -> Dict:
    """Map custom configs from list to dictionary.

    Args:
        custom_configs: List of custom configs passed to API method

    Returns:
        Mapped configs to dictionary
    """
    if not custom_configs:
        return {}

    return {config.name(): config for config in custom_configs}

model_navigator.api.MaxThroughputAndMinLatencyStrategy

Bases: RuntimeSearchStrategy

Get runtime with the highest throughput and the lowest latency.

model_navigator.api.MaxThroughputStrategy

Bases: RuntimeSearchStrategy

Get runtime with the highest throughput.

model_navigator.api.MinLatencyStrategy

Bases: RuntimeSearchStrategy

Get runtime with the lowest latency.