An efficient, to-the-point, and easy-to-use checklist to following when deploying an ML model into production. Consider including a copy of this checklist alongside each model you plan to deploy to production.
MODEL NAME:___________________________________
MODEL ID:_____________________________________
MODEL VERSION:________________________________
- Model assets are stored in communal location (e.g. weights files, processing code, list of dependencies)
- Model particulars are stored in communal location (e.g. training data, features used, input/output formats)
- Model creator's name and contact information is logged with model assets
- Model monitoring is in place
- Model assets are assembled into final production format (e.g. container, zip file, script)
- New model version conforms to same input/output contract as previous model version -or- other production systems have been changed to accept a new contract
- Model is uploaded or otherwise pushed to
testenvironment - Model version in test environment matches model version from training
- Model successfully runs on same infrastructure that will be used in production (e.g. same CPU, GPU, RAM settings)
- Model validation metrics in test environment match validation metrics that were run at the end of training
- Model response times are within an acceptable range when tested with representative sample of production data
- Model is uploaded or otherwise pushed to production environment
- Model version in production matches model version in test
- Access to the model is secured and requires authentication
- Other production systems can communicate with the model
- Model returns results
- Production traffic is rerouted to new model version
- Old model version is shutdown
- Monitoring is in place to estimate and track model accuracy (aka model drift)
- Monitoring is in place to track model speed, latency, and throughput
- Monitoring is in place to track and log model errors including downtime
- Monitoring is in place to detect cybersecurity attacks