Deployment Basics

Serving, monitoring, and drift

A model in a notebook helps no one. Learn how models reach real users—and what can go wrong once they do.

Before We Begin

What we are learning today

Training a model is the beginning, not the end. Deployment is where ML meets the real world: wrapping a model behind an API so applications can call it, monitoring its predictions to catch silent failures, and detecting data drift—the slow shift in real-world data that makes yesterday's great model today's unreliable one. We also touch on versioning models, rolling back bad deployments, and the concept of a feedback loop where user behavior changes the data the model sees next.

How this lesson fits

A model that works is not enough—it also has to be understandable, fair, and ready for the real world. This module covers the human side of AI: explaining predictions, confronting bias, and deploying models responsibly.

The big question

How do we make sure an AI system is not just accurate but also transparent, fair, and safe to deploy?

Explain a model's prediction using interpretability toolsIdentify sources of bias and propose mitigation strategiesDescribe the basics of serving, monitoring, and handling drift in production

Why You Should Care

Most ML courses stop at training. But in industry, deployment, monitoring, and maintenance consume the majority of an ML engineer's time. Teaching these concepts early gives students a realistic picture of what ML work actually looks like—and prevents the common surprise that a "finished" model is really just the beginning.

Where this is used today

✓Streaming services: serving recommendation models to millions of users in real time
✓Banks: monitoring fraud models for drift as spending patterns change post-pandemic
✓Social media: feedback loops where engagement models shape the content users see, which changes future engagement data

Think of it like this

Think of opening a restaurant. Perfecting a recipe at home (training) is step one. But running a real kitchen means handling a dinner rush (serving at scale), checking that dishes taste the same every night (monitoring), and updating the menu when ingredients change seasonally (handling drift). Deployment is everything after the recipe is written.

Easy mistake to make

Deploying a model is not a one-time event. Real-world data changes constantly, so a deployed model requires ongoing monitoring, retraining schedules, and sometimes a full rebuild.

By the end, you should be able to say:

Describe how a trained model is served behind an API endpoint
Explain data drift and concept drift and why models degrade over time
Outline a basic monitoring dashboard: latency, error rate, prediction distribution
Define a feedback loop and explain why it can be dangerous if unchecked

Think about this first

You deploy a model that predicts ice-cream sales. It works great in summer. What happens in winter, and what should you do about it?

Words we will keep using

deploymentAPIservingmonitoringdata driftconcept driftfeedback looprollback

Deploying Is Not the End \u2014 It's the Beginning

A model that works in a notebook is not the same as a model that works in production. Real-world data changes constantly. Users behave differently over time. Systems fail in unexpected ways. Deployment is an ongoing commitment to monitoring, maintenance, and adaptation.

ServeWrap the model in an API that returns predictions on demand.

MonitorTrack accuracy, latency, and data distributions in real time.

DetectSpot data drift and concept drift before they degrade performance.

RollbackWhen things break, revert to the last known-good version instantly.

Interactive: Model Drift Over Time

A model is deployed at 92% accuracy. Over time, user behavior shifts and accuracy degrades. Adjust when drift begins and watch the model cross the alert threshold.

Drift starts at day: 45Monitoring window: 90 days

Without monitoring, drift goes unnoticed for weeks. With proper alerts, you catch it the moment accuracy crosses the threshold and trigger a retraining pipeline.

Two Types of Drift

Data Drift

The input distribution changes. E.g., a new user demographic appears, or sensor calibration shifts. The model sees data it was never trained on.

Detect: Monitor feature distributions (KS test, PSI).

Concept Drift

The relationship between inputs and outputs changes. E.g., a pandemic changes what "normal" spending looks like. The model's learned rules become outdated.

Detect: Monitor prediction accuracy and calibration over time.

Request Latency Distribution

In production, users care about tail latency, not average. The p99 (worst 1% of requests) reveals outliers that average hides. A model with 30ms average but 200ms p99 will frustrate users.

Serving Patterns

BatchRun predictions offline on a schedule. Cheapest, but not real-time. Use for reports and recommendations.

Real-time APIModel served behind a REST/gRPC endpoint. Low latency, scales horizontally. Use for search ranking and fraud detection.

EdgeModel runs on-device (phone, IoT). No network needed, privacy-preserving. Use for face ID and voice commands.

CI/CD for ML

Traditional CI/CD tests code. ML CI/CD must also test data, model quality, and inference performance.

# ML pipeline stages
data_validation \u2192 training \u2192 evaluation \u2192 shadow_deploy \u2192 canary \u2192 full_rollout
# If metrics drop, auto-rollback

Shadow DeployNew model runs alongside the old one, serving real traffic but its outputs are discarded. Compare predictions safely.

Canary ReleaseRoute 5% of traffic to the new model. If metrics hold, gradually increase to 100%. If not, rollback.