Model Serving made Efficient in the Cloud
Project description
框架环境变量
| 环境变量 | 说明 | 默认值 |
|---|---|---|
| TI_MODEL_DIR | 模型路径 | /data/model |
| TI_PREPROCESS_NUMS | 预处理进程数 | 0 |
| TI_INFERENCE_NUMS | 推理进程数 | 1 |
| TI_POSTPROCESS_NUMS | 后处理进程数 | 0 |
| TI_INFERENCE_MAX_BATCH_SIZE | 推理Batch数 | 1 |
说明
-
当TI_PREPROCESS_NUMS==0且TI_POSTPROCESS_NUMS==0时;
model_service.py 文件中preprocess，postpress，predict 函数在一个进程中执行
-
否则，model_service.py 文件中preprocess, postpress, predict 函数分别处于不同进程中执行，
load 函数与predict 函数处于同一进程;
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distributions
No source distribution files available for this release.See tutorial on generating distribution archives.
Built Distribution
Close
Hashes for ti_cloud_infer_framework-0.1.6-py3-none-any.whl
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | 9bf087b192d60a441cb6c09e322521dab293d9fb597f8a9ee27440bd164cb0f2 |
|
| MD5 | e7af498c970ccc89198a055e9792709b |
|
| BLAKE2b-256 | d89eef84fb7025e2309c238728194faeb8c2215d13d00a6a12f1913406496ed2 |
