Model Store API
model_navigator.triton.model_repository.add_model
model_navigator.triton.model_repository.add_model(
model_repository_path, model_name, model_path, config, model_version=1
)
Generate model deployment inside provided model store path.
The config requires specialized configuration to be passed for backend on which model is executed. Example:
- ONNX model requires ONNXModelConfig
- TensorRT model requires TensorRTModelConfig
- TorchScript or Torch-TensorRT models requires PyTorchModelConfig
- TensorFlow SavedModel or TensorFlow-TensorRT models requires TensorFlowModelConfig
- Python model requires PythonModelConfig
Parameters:
-
model_repository_path
(
Union[str, pathlib.Path]
) –Path where deployment should be created
-
model_name
(
str
) –Name under which model is deployed in Triton Inference Server
-
model_path
(
Union[str, pathlib.Path]
) –Path to model
-
config
(
Union[ONNXModelConfig, TensorRTModelConfig, PyTorchModelConfig, PythonModelConfig, TensorFlowModelConfig]
) –Specialized configuration of model for backend on which model is executed
-
model_version
(
int
) –Version of model that is deployed
Returns:
Source code in model_navigator/triton/model_repository.py
model_navigator.triton.model_repository.add_model_from_package
model_navigator.triton.model_repository.add_model_from_package(
model_repository_path, model_name, package, model_version=1, strategy=None, response_cache=False
)
Create the Triton Model Store with optimized model and save it to model_repository_path
.
Parameters:
-
model_repository_path
(
Union[str, pathlib.Path]
) –Path where the model store is located
-
model_name
(
str
) –Name under which model is deployed in Triton Inference Server
-
model_version
(
int
) –Version of model that is deployed
-
package
(
Package
) –Package for which model store is created
-
strategy
(
Optional[RuntimeSearchStrategy]
) –Strategy for finding the best runtime. When not set the
MaxThroughputAndMinLatencyStrategy
is used. -
response_cache
(
bool
) –Enable response cache for model
Returns:
-
–
Path to created model store
Source code in model_navigator/triton/model_repository.py
177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 |
|