A typical ML process for training a model is as follows
Start with data prepossessing
Train the model
Evaluate the model
Iterate the above steps
These steps are often executed manually by running scripts, notebooks, and applying human judgment to validate the model as successfully trained. After model training for production, the process of training an ML model is more likely to be automated with an ML model training pipeline. This way, the model training can be reviewed, reproduced, tuned, and debugged at any point. When triggered, the components in the above pipeline will run in sequential order, and only if they are all successful, a micro-service that serves the trained model will be created or updated.
Most of the components in the above pipeline can be reused and have open-source implementations. There are all types of data pre-processing, model training, and validation components hosted on the AI Hub. You can go very far into production ML training and serving without the need for writing custom components.
Let’s go through the basics before using any components.
All of the components are applications in Docker containers. For example, to train an ML model, we would need the Docker registry URI that points to the container and pass arguments as training-data, learning-rate, and output-location. When we run that trainer image, the application will read the runtime arguments and use them to produce a trained model. There are many solutions for running docker containers on the Cloud. AI Platform training is a specialized service for running ML training containers.AI Platform training works better than other solutions because of the easy distributed training setup and the provision of TPUs.
To serve inference from your model, you need to create a serving REST endpoint. That is easy if you let AI Platform serving host your model. AI Platform serving creates a REST API that returns predictions and scales automatically according to demand. This eliminates the need for setup and maintenance of complex infrastructure. AI Platform also simplifies the process of updating prediction models by providing a model name and version. Note that the process of training and deployment will be the same for all TensorFlow components as well. The orchestration is the final stage of any pipeline development, and you should be able to execute every component separately before you put them in a pipeline.
So how do we chain our Docker ML containers?
Kubeflow Pipelines is a platform for building and deploying portable, scalable machine learning (ML) workflows based on Docker containers. To create a pipeline, we need to describe the DAG of components with the Kubeflow Pipelines Python SDK. KFP pipelines are composed of components, and components are Docker containers with the addition to some metadata as inputs, outputs, and Docker registry. That metadata is normally stored in a YAML file.
ML Container + component.yaml = Component
Most ML Containers are released with such a metadata file, and they are published under the “Kubeflow pipeline” category. Some ML containers don’t have a component.yaml file. They are just Docker containers that parse command line arguments.