An example MLFlow project

You can Notebook see the Databricks code here: https://github.com/FourthBrain/databricks-zero-to-mlops/blob/main/src/week2-mlflow/AutoML/XGBoost-fake-news-automl.py

curl \
  -u token:$DATABRICKS_TOKEN \
  -X POST \
  -H "Content-Type: application/json; format=pandas-records" \
  [email protected] \
  https://adb-2951765055089996.16.azuredatabricks.net/model/Fake-News/1/invocations

• Azure Databricks Model Serving Architecture

• Databricks CLI

1. Create a databricks config: touch ~/.databrickscfg
2. Put in host and token
3. Query jobs

databricks jobs list --output JSON | jq 4. List clusters

databricks clusters list --output JSON | jq

5. List contents of DBFS

databricks fs ls dbfs:/

• Refer to Quikstart Python
• Refer to Create or Register Model
• Access tracking server external

You need to set the tracking URI.

export MLFLOW_TRACKING_URI=databricks

from pprint import pprint
from mlflow.tracking import MlflowClient
client = MlflowClient()
for rm in client.list_registered_models():
  pprint(dict(rm), indent=4)

• Download model artifacts

CLI version

mlflow artifacts download --artifact-uri models:/<name>/<version|stage>

To use Python do the following:

from mlflow.store.artifact.models_artifact_repo import ModelsArtifactRepository

model_uri = MlflowClient.get_model_version_download_uri(model_name, model_version)
ModelsArtifactRepository(model_uri).download_artifacts(artifact_path="")

• Register model for AWS Sagemaker

• Databricks Tensorflow run
• Feature Store
• Model Serving

https://docs.databricks.com/applications/mlflow/model-serving.html

Run it with mlserve

 mlflow models serve --model-uri  /workspaces/mlflow-project-best-practices/tf-model