MLflow Knative Deployment Plugin

MLflow plugin adding a Knative deployment client to MLflow CLI and Python API.

Note: MLServer (V2 Inference API) is enabled for all Docker builds.

Requirements

• Python 3.10+
• MLflow 2+
• Docker

The target Kubernetes cluster must be running Knative 1.10+.

Installation

pip install mlflow-knative

Getting Started

The plugin adds support for a knative target scheme to the mlflow deployments CLI.

A Kubernetes context is used to define the Knative target as knative:/<context>.
You can list available contexts with kubectl config get-contexts.

Note: Passing only the base knative:/ (context omitted) as target lets the Kubernetes client pick a default configuration, which may be in-cluster. Kubernetes RBAC must be configured to allow the pod running this deployment client to create, list, update and delete Knative services.

Make sure Docker is running locally if you intend create or update a deployment, as this is required to build an image from the MLflow model.

Setting the image_repository config key is required to make the Docker image of the model available for deployment by Knative. Additionally you may also provide an image tag with the image_tag config key (defaults to latest).

mlflow deployments create \
  --target knative:/<context> \
  --name <deployment-name> \
  --model-uri models:/<model-name>/<model-version> \
  --config image_repository=<image-repository-URI> \
  --config image_tag=<image-tag>

The plugin provides detailed target help.

mlflow deployments help --target knative

All features are also available as a Python API deployment client.

from mlflow.deployments import get_deploy_client

client = get_deploy_client("knative:/my-cluster")
client.create_deployment(
	"hello-world",
	"models:/hello-world/1",
	config={
		"image_repository": "hello-world"
	}
)

Using a Private Image Repository

To use a private Docker image repository, simply run docker login defore running the deployment client, then use the full repository URI as value for the image_repository config key.

docker login --username <username> --password-stdin <private-repository-URL>

# If using AWS ECR:
aws ecr get-login-password | docker login --username AWS --password-stdin <private-repository-URL>

mlflow deployments create \
  --target knative:/<context> \
  --name <deployment-name> \
  --model-uri models:/<model-name>/<model-version> \
  --config image_repository=<image-repository-URI>  # e.g.: 000000000000.dkr.ecr.eu-west-3.amazonaws.com/model-name

Using a Remote MLflow Model Registry

Set an environment variable as export MLFLOW_TRACKING_URI=<tracking-server-uri> to use a remote MLflow model registry. This also works with a private model registry secured with OAuth 2, using the MLflow OIDC Client Plugin.

Knative Service Configuration

The deployment client can use any available namespace on the target cluster by setting the namespace config key. The default value is default.

mlflow deployments create \
  --target knative:/<context> \
  --name <deployment-name> \
  --model-uri models:/<model-name>/<model-version> \
  --config image_repository=<image-repository-URI> \
  --config namespace=<my-namespace>

To deploy a Knative service with a custom templated manifest, set the service_template config key. The value is a path to the YAML manifest you will be using.

mlflow deployments create \
  --target knative:/<context> \
  --name <deployment-name> \
  --model-uri models:/<model-name>/<model-version> \
  --config image_repository=<image-repository-URI> \
  --config service_template=<path/to/manifest>

$name, $namespace and $image templated values are respectively the deployment name, the provided namespace (or "default"), the image determined from the provided image repository and tag.

Caching Behavior

On deployment creation, the plugin always builds and pushes a new Docker image for the model.
On deployment update, if both:

• an image exists on the repository with the expected tag, and
• the MLflow run ID of the current deployment matches the run ID of the model to deploy (identified by its URI on the model registry) the plugin will skip the image build and push step.

Otherwise, the plugin will build and push a new image on the repository with the specified tag.

Using immutable tags is recommended, and we advise not relying on the default latest tag.

Updating a deployment will always attempt to patch the corresponding Knative service and rely on Knative handling the changes to determine if a new revision is required.
The generation service metadata property will be bumped when a new revision is created, which might be useful to compare service state before and after the update.

License

This project is licensed under the terms of the MIT license.

A yzr Free and Open Source project.