2-2_Local_Deployment.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Lesson 2.2: Deployment and Inference with MLflow\n",
    "\n",
    "[![Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/zenml-io/zenbytes/blob/main/2-2_Local_Deployment.ipynb)\n",
    "\n",
    "***Key Concepts:*** *Model Deployer, MLflow*\n",
    "\n",
    "In the last lesson, we learned how to use MLflow and Weights & Biases to track our experiments and compare models. In the end, we found which hyperparameters produced the best-performing model on our validation dataset. How do we make this model available to our customers/users and enable them to query it?\n",
    "\n",
    "Setting up a dynamically scalable, highly-available, and reliable model service is a complex problem, and many companies hire large MLOps teams to build and maintain such services. With ZenML, we can build sophisticated ML services in a matter of minutes. In this lesson, we will start with a very basic model deployment, where we will use the [MLflow Models](https://mlflow.org/docs/latest/models.html) component to deploy our model as a local application that we can interact with via REST API. \n",
    "\n",
    "The beauty of ZenML is that our code can stay the same, no matter what tools or infrastructure we use. In a later chapter, we will see how this enables us to deploy the code we write here as a dynamically-scalable serverless microservice in the cloud. But more on that later.\n",
    "\n",
    "First, let's setup zenml and import some of the core steps we have created in previous lessons:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%pip install \"zenml[server]\" matplotlib\n",
    "!zenml integration install sklearn mlflow -y\n",
    "!rm -rf .zen\n",
    "!zenml init\n",
    "%pip install pyparsing==2.4.2  # required for Colab\n",
    "\n",
    "import IPython\n",
    "\n",
    "# automatically restart kernel\n",
    "IPython.Application.instance().kernel.do_shutdown(restart=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from zenml.environment import Environment\n",
    "\n",
    "if Environment.in_google_colab():  # Colab only setup\n",
    "\n",
    "    # clone zenbytes repo to get source code of previous lessons\n",
    "    !git clone https://github.com/zenml-io/zenbytes.git  # noqa\n",
    "    !mv zenbytes/steps .\n",
    "    !mv zenbytes/pipelines ."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from steps.evaluator import evaluator\n",
    "from steps.importer import importer\n",
    "from steps.mlflow_trainer import svc_trainer_mlflow"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "ZenML provides a standard step for deployment to MLflow, so we don't need to write any code ourselves. To deploy our model after training it, all we need to do is to add the `mlflow_model_deployer_step` into our pipeline. In addition to the trained model, this step expects a boolean argument of whether to deploy the model or not. This is very useful in practice, as it allows you to define some requirements for deploying your models, i.e., that it performs better than the currently deployed model or that no data drift is happening. For now, let us define a `deployment_trigger` that only deploys a model if the test accuracy is over 90%:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from zenml.integrations.mlflow.steps import (\n",
    "    MLFlowDeployerParameters,\n",
    "    mlflow_model_deployer_step\n",
    ")\n",
    "from zenml.pipelines import pipeline\n",
    "from zenml.steps import step\n",
    "\n",
    "\n",
    "@step\n",
    "def deployment_trigger(test_acc: float) -> bool:\n",
    "    \"\"\"Only deploy if the test accuracy > 90%.\"\"\"\n",
    "    return test_acc > 0.9\n",
    "\n",
    "\n",
    "@pipeline(enable_cache=False)\n",
    "def train_evaluate_deploy_pipeline(\n",
    "    importer,\n",
    "    trainer,\n",
    "    evaluator,\n",
    "    deployment_trigger,\n",
    "    model_deployer,\n",
    "):\n",
    "    \"\"\"Train and deploy a model with MLflow.\"\"\"\n",
    "    X_train, X_test, y_train, y_test = importer()\n",
    "    model = trainer(X_train=X_train, y_train=y_train)\n",
    "    test_acc = evaluator(X_test=X_test, y_test=y_test, model=model)\n",
    "    deployment_decision = deployment_trigger(test_acc)  # new\n",
    "    model_deployer(deployment_decision, model)  # new\n",
    "\n",
    "\n",
    "mlflow_pipeline = train_evaluate_deploy_pipeline(\n",
    "    importer=importer(),\n",
    "    trainer=svc_trainer_mlflow(),\n",
    "    evaluator=evaluator(),\n",
    "    deployment_trigger=deployment_trigger(),  # new\n",
    "    model_deployer=mlflow_model_deployer_step(\n",
    "        MLFlowDeployerParameters(timeout=20)\n",
    "    ),  # new\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Since we are using a new MLOps stack component, we need to register it with ZenML again before being able to run our pipeline. \n",
    "Similar to registering the experiment tracker in the last notebook, we first define a new model deployer, then add it to our ZenML stack."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Define MLflow experiment tracker from last lesson\n",
    "!zenml experiment-tracker register mlflow_tracker --flavor=mlflow\n",
    "\n",
    "# Register the MLflow model deployer\n",
    "!zenml model-deployer register mlflow --flavor=mlflow\n",
    "\n",
    "# Create a new stack with MLflow components\n",
    "!zenml stack register mlflow_stack -a default -o default -d mlflow -e mlflow_tracker\n",
    "\n",
    "# Set the new stack as active\n",
    "!zenml stack set mlflow_stack"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Executing `pipeline.run()` will now automatically deploy our model using MLflow. Let's try it out:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "mlflow_pipeline.run(unlisted=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's run the following command to get a list of all models currently deployed with our ZenML stack:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!zenml model-deployer models list"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If you see a checkmark under status, the model was correctly deployed. Congrats!\n",
    "\n",
    "To find the URL of a model deployed by a specific run, you can use the \n",
    "`deployed_model_url` metadata field of the model deployer step of your pipeline \n",
    "run, e.g.:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from zenml.post_execution import get_unlisted_runs\n",
    "\n",
    "last_run = get_unlisted_runs()[0]\n",
    "deployer_step = last_run.get_step(\"model_deployer\")\n",
    "deployed_model_url = deployer_step.metadata[\"deployed_model_url\"].value\n",
    "print(deployed_model_url)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To interact with our deployed model in Python, we can use the `find_model_server()` method of ZenMLs model-deployer stack component:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from zenml.client import Client\n",
    "\n",
    "client = Client()\n",
    "model_deployer = client.active_stack.model_deployer\n",
    "services = model_deployer.find_model_server(\n",
    "    pipeline_name=\"train_evaluate_deploy_pipeline\",\n",
    "    pipeline_step_name=\"model_deployer\",\n",
    "    running=True,\n",
    ")\n",
    "service = services[0]\n",
    "service.check_status()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's play with our model service a bit and send it a request. \n",
    "\n",
    "First, let's query the artifact store to get a sample from the test set of our last run."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "last_run = get_unlisted_runs()[0]\n",
    "X_test = last_run.steps[0].outputs[\"X_test\"].read()\n",
    "y_test = last_run.steps[0].outputs[\"y_test\"].read()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's use matplotlib to plot the sample and see what our model would predict:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "\n",
    "plt.axis(\"off\")\n",
    "plt.imshow(X_test[0].reshape(8, 8), cmap=plt.cm.gray_r, interpolation=\"nearest\")\n",
    "pred0 = service.predict(X_test[0:1])\n",
    "print(f\"Model predicted {pred0}, true label was {y_test[0]}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And that's it, we have deployed our first ML pipeline and learned how to interact with it. In practice, you would, of course, not query the model service manually but automatically send samples to it as new data comes in. That is what we will do in the [next lesson](2-3_Inference_Pipelines.ipynb), where we will build a basic inference pipeline. See you there!"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "zenml310",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.10"
  },
  "vscode": {
   "interpreter": {
    "hash": "569b3361e3ec4d7692543ddda480ca8173a6c158bb706498f2e35ca1687a80ea"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}