Common API

Framework indepedent definitions and enumerations.

model_navigator.api.config.SizedIterable

Bases: Protocol

Protocol representing sized iterable. Used by dataloader.

__iter__()

Magic method iter.

Returns:

Type	Description
Iterator	Iterator to next item.

Source code in model_navigator/api/config.py

def __iter__(self) -> Iterator:
    """Magic method __iter__.

    Returns:
        Iterator to next item.
    """
    ...

__len__()

Magic method len.

Returns:

Type	Description
int	Length of size iterable.

Source code in model_navigator/api/config.py

def __len__(self) -> int:
    """Magic method __len__.

    Returns:
        Length of size iterable.
    """
    ...

model_navigator.api.config.SizedDataLoader = Union[SizedIterable, Sequence] module-attribute

model_navigator.api.config.Format

Bases: Enum

All model formats supported by Model Navigator 'optimize' function.

model_navigator.api.config.CustomConfig

Bases: abc.ABC

Base class used for custom configs. Input for Model Navigator optimize method.

defaults()

Update parameters to defaults.

Source code in model_navigator/api/config.py

def defaults(self) -> None:
    """Update parameters to defaults."""
    return None

from_dict(config_dict) classmethod

Instantiate CustomConfig from a dictionary.

Source code in model_navigator/api/config.py

@classmethod
def from_dict(cls, config_dict: Dict[str, Any]) -> "CustomConfig":
    """Instantiate CustomConfig from a dictionary."""
    return cls(**config_dict)

name() classmethod abstractmethod

Name of the CustomConfig.

Source code in model_navigator/api/config.py

@classmethod
@abc.abstractmethod
def name(cls) -> str:
    """Name of the CustomConfig."""
    raise NotImplementedError()

model_navigator.api.config.CustomConfigForFormat

Bases: DataObject, CustomConfig

Abstract base class used for custom configs representing particular format.

format: Format property abstractmethod

Format represented by CustomConfig.

model_navigator.api.config.Sample = Dict[str, numpy.ndarray] module-attribute

model_navigator.api.config.VerifyFunction = Callable[[Iterable[Sample], Iterable[Sample]], bool] module-attribute

model_navigator.api.ProfilerConfig dataclass

Bases: DataObject

Profiler configuration.

For each batch size profiler will run measurments in windows. Depending on the measurement mode, each window will have fixed time length (MeasurementMode.TIME_WINDOWS) or fixed number of requests (MeasurementMode.COUNT_WINDOWS). Batch sizes are profiled in the ascending order.

Profiler will run multiple trials and will stop when the measurements are stable (within stability_percentage from the mean) within three consecutive windows. If the measurements are not stable after max_trials trials, the profiler will stop with an error. Profiler will also stop profiling when the throughput does not increase at least by throughput_cutoff_threshold.

Parameters:

Name	Type	Description	Default
run_profiling	bool	If True, run profiling, otherwise skip profiling.	True
batch_sizes	Optional[List[Union[int, None]]]	List of batch sizes to profile. None means that the model does not support batching.	None
measurement_mode	MeasurementMode	Measurement mode.	MeasurementMode.COUNT_WINDOWS
measurement_interval	Optional[float]	Measurement interval in milliseconds. Used only in MeasurementMode.TIME_WINDOWS mode.	5000
measurement_request_count	Optional[int]	Number of requests to measure in each window. Used only in MeasurementMode.COUNT_WINDOWS mode.	50
stability_percentage	float	Allowed percentage of variation from the mean in three consecutive windows.	10.0
max_trials	int	Maximum number of window trials.	10
throughput_cutoff_threshold	float	Minimum throughput increase to continue profiling.	DEFAULT_PROFILING_THROUGHPUT_CUTOFF_THRESHOLD

from_dict(profiler_config_dict) classmethod

Instantiate ProfilerConfig class from a dictionary.

Parameters:

Name	Type	Description	Default
profiler_config_dict	Mapping	Data dictionary.	required

Returns:

Type	Description
ProfilerConfig	ProfilerConfig

Source code in model_navigator/api/config.py

@classmethod
def from_dict(cls, profiler_config_dict: Mapping) -> "ProfilerConfig":
    """Instantiate ProfilerConfig class from a dictionary.

    Args:
        profiler_config_dict (Mapping): Data dictionary.

    Returns:
        ProfilerConfig
    """
    return cls(
        run_profiling=profiler_config_dict.get("run_profiling", True),
        batch_sizes=profiler_config_dict.get("batch_sizes"),
        measurement_interval=profiler_config_dict.get("measurement_interval"),
        measurement_mode=MeasurementMode(
            profiler_config_dict.get("measurement_mode", MeasurementMode.TIME_WINDOWS)
        ),
        measurement_request_count=profiler_config_dict.get("measurement_request_count"),
        stability_percentage=profiler_config_dict.get("stability_percentage", 10.0),
        max_trials=profiler_config_dict.get("max_trials", 10),
        throughput_cutoff_threshold=profiler_config_dict.get("throughput_cutoff_threshold", -2),
    )

model_navigator.api.MeasurementMode

Bases: Enum

Measurement mode.

TIME_WINDOWS mode run measurement windows with fixed time length. COUNT_WINDOWS mode run measurement windows with fixed number of requests.