The Holy Bible of Azure Machine Learning Service. A walk-through for the believer (Part 3)
This is Part 3 of the series. If you missed the first one, or second one, worry-do-not, thank Gosh Internet is not like 90’s TV shows, so here you have the on-demand link to Part 1 and Part 2.
Let’s recap:
About this series
Azure Machine Learning Services (AML) provides a cloud-based environment you can use to develop, train, test, deploy, manage, and track machine learning models. I’m gonna cover three main topics in a sequence of posts:
- Part 1: Train the trainer in the cloud using AML. I will show you here how to create a run.py script to start training your models in the cloud with a couple of lines of code and without changing a single line of your training routine. [Link]
- Part 2: Making your trainer smarter for AML. I will show you here how you can modify your training routine script with some AML specifics lines of code to allow debugging, tracking, monitoring and manage outputs. If you are familiar with MLFlow, then you will find these capabilities a bit similar. [Link]
- Part 3: Promoting you model to production. I will show you how you can productionize your model using AML make it ready to be consumed by end users using web services. ← this post!
Publishing your model as a REST-enabled web service
AML has the capability to package your models inside a Docker container and publish it as a web service using either Azure Container Instances (ACS), Azure Kubernetes (AKS) or even on IoT devices on the edge using the capabilities of Azure IoT Edge (really cool feature). All this can be achieved using Python code, so open a Jupyter Notebook or your favorite IDE and start typing:
As usual, you first have to open your workspace:
from azureml.core import Workspacews = Workspace(workspace_name = workspace_name, subscription_id = subscription_id, resource_group=resource_group)
Step 1: Register your candidate model
You need to register your candidate model inside AML. This means uploading you model file into the AML repository of models:
from azureml.core.model import Modelmodel = Model.register(model_path = "/dogscats/export.pkl",
model_name = "cats_vs_dogs",
tags = {"key": "0.1"},
description = "cats_vs_dogs",
workspace = ws)
As you can see, AML keeps versioning of your models, which is a good thing when dealing with models in production.
Alternatively, you may want to register the best model available in your experiment. This is why having a code-first approach in AML is a good thing. It’s really easy to implement this in Python:
from azureml.core import Runexp = ws.experiments['azureml-cats-vs-dogs']
runs_accuracy = { run.id : run.get_metrics()['training_acc'] for run in exp.get_runs() }best_run_id = max(runs_accuracy, key=lambda k : runs_accuracy[k])
best_run = Run(exp, best_run_id)model = best_run.register_model(model_name='cats_vs_dogs',
model_path='outputs/cats-vs-dogs.pkl')
Step 2: Building your scoring logic
Once you model is registered, you have to build an score.py script with all the logic to use the model you just registered. The execution script receives data submitted to a deployed image, and passes it to the model. It then takes the response returned by the model and returns that to the client. The script is specific to your model; it must understand the data that the model expects and returns. The script usually contains two functions that load and run the model:
- init(): Typically this function loads the model into a global object. This function runs only once when the Docker container is started.
- run(input_data): This function uses the model to predict a value based on the input data. Inputs and outputs to the run typically use JSON for serialization and de-serialization. You can also work with raw binary data if it’s convenient to you.
Let’s see how the score.py would look like for the Cat’s vs Dogs experiment we were working before:
import torch
import os, base64, json
from fastai import *
from fastai.vision import *
from azureml.core.model import Modeldef init():
global learn
model_file = Model.get_model_path("cats_vs_dogs")
model_path = os.path.dirname(model_file)
learn = load_learner(model_path)def run(raw_data):
base64_string = json.loads(raw_data)['data']
base64_bytes = base64.b64decode(base64_string)
result = learn.predict(base64_bytes)
return json.dumps({'category':str(result[0]),
'confidence':result[2].data[1].item()})
Pay special attention to how the model is loaded inside the service in the init() function. the get_model_path function recieves the name of the model you are interested in. This is the model we have just registered in the AML service. The function returns the path of the file you uploaded.
Step 3: Create a docker image with the score.py file you created
Deployed models are packaged as a docker image. The image contains the dependencies needed to run the model all along with the logic to score new data.
For Azure Container Instance, Azure Kubernetes Service, and Azure IoT Edge deployments, the azureml.core.image.ContainerImage class is used to create an image configuration. The image configuration is then used to create a new Docker image.
from azureml.core.image import ContainerImage# Image configuration
image_config = ContainerImage.image_configuration(
execution_script = "score.py",
runtime = "python",
conda_file = "conda_dependencies.yml",
description = "Classification cats vs dogs",
tags = {"data": "cats-vs-dogs",
"type": "classification"})
You may be wondering about the conda_dependecies.yml file that is specified in one of the parameters right? This is a standard conda environment file specification. However, if you are using AML to train your model you don’t have to worry about it since it is automatically generated for you. Just look inside of your experiment folder and you will find a folder called .aml_config. Inside the folder you will find the file!
Step 4: Build the docker image
Once you have an image registered, you need to build the image:
# Register the image from the image configuration
image = ContainerImage.create(name = "azureml-gpubenchmark-fastai",
models = [model], # the model object
image_config = image_config,
workspace = ws)image.wait_for_creation(show_output=True)
Step 5: Deploy the image in your compute target
To deploy the image you created, you first need to specify the target you want to use. You can deploy it inside ACS, AKS or even in IoT devices running Azure IoT Edge. Really interesting feature if you are dealing with a solution in the IoT space and you have connectivity constrains.
In my case I will deploy the service inside ACS, so I have to specify the characteristics of the machine I want to use:
from azureml.core.webservice import AciWebserviceaciconfig = AciWebservice.deploy_configuration(
cpu_cores = 2,
memory_gb = 6,
tags = {"data": "cats-vs-dogs", "type": "classification"},
description = 'Image classficiation service cats vs dogs')
Finally, we deploy the image in our target compute:
from azureml.core.webservice import Webserviceservice = Webservice.deploy_from_image(
deployment_config = aciconfig,
image = image,
name = 'azureml-gpu-fastai-service',
workspace = ws)service.wait_for_deployment(show_output = True)
That’s all! By now, the service should be running in your target compute. You can see all your active deployments inside the Azure portal:
If you click on your deployment you will see all the details, including the scoring URL you have to use:
Wait! I got an error!
You may find yourself executing the method wait_for_deployment() and received a message saying that the container failed to start. This may happen if your score.py failed to execute the init() function. You will need to start debugging your file (buuuu). Use the following method to get the logs of the initialization procedure:
service.get_logs()Something else?
Hope I shed some light about how AML works and all the possibilities it opens. There are a couple of other features in the service, like Pipelines, that are also really interesting to see but I will leave that to you. Other features are also coming to the service in the near feature so stay tune!
