MLOps & serving interview questions
40 MLOps & serving interview questions with answers, from junior to mid level.
Foundations
Core
- Only one person can rebuild the production model, by running a long notebook by hand in the right order. What is the main fix?
- Which signal of data drift can you monitor in production even before any labels arrive?
- How can you release a new model version with low risk?
- What is a shadow deployment of a model?
- A new model version is live and the error rate and latency of the service jumped. What is the fastest safe action?
- What should experiment tracking record for each training run?
- Why is a model usually packaged in a container for serving?
- A model server groups several incoming requests into one batch before running the model on a GPU. What is the trade-off?
- What problem does a feature store mainly solve?
- A model service returns HTTP 200 for every request, but a bug makes it predict almost the same value for everyone. Which monitoring would catch this?
- Which is a sensible way to decide when to retrain a model?
- What are the stages of a production ML pipeline?
Intermediate
- To build a training set, each row is joined to the latest value of a feature
customer_total_spend, taken from a table that is updated daily. The model is great offline and poor online. What is the likely problem? - How can you check systematically for training-serving skew in a live system?
- With millions of rows per day, a statistical drift test flags almost every feature as "significantly different" every day, with tiny p-values. What is a better approach?
- You need real evidence that a new recommender raises purchases. Which rollout gives it?
- The training pipeline runs in CI with the same code and data as last month, but the metrics changed. What do you check first?
- The average latency of a model service is fine, but users complain about slow responses. What do you look at?
- A GPU-based model service has spiky traffic. Scaling on CPU usage reacts badly. What is a better signal for autoscaling?
- What does quantizing a model do?
- An upstream team renames a column. The pipeline silently fills the feature with nulls, and model quality collapses weeks later. What would have caught it early?
- A pipeline step failed halfway and was run again, and now the training table has duplicated rows. Which design principle prevents this?
- What should continuous integration for an ML project run, in addition to unit tests?
- An auditor asks which data and code produced the model that was in production on 3 March. What do you need to have kept?
- The same inputs reach a prediction service again and again, and each call is slow. What is a cheap way to speed it up?
- The online metric of a model slowly gets worse over several weeks. How do you investigate?
Advanced
- A recommender is retrained every week on clicks. Over months the catalogue it shows gets narrower and narrower. What is happening?
- What should an automated canary check before it increases the traffic share of a new model?
- The team that owns the "active user" feature changes its definition, and your model's quality drops without any change on your side. What helps prevent this?
- An automatic weekly retraining job trained on data corrupted by an upstream bug, and the worse model went live. Which safeguard was missing?
- A fraud model needs the feature "number of transactions by this card in the last 5 minutes", but features are computed in a daily batch job. Why is this not enough?
- Overall model quality is stable, but support tickets show it works badly for customers in one region. Which monitoring practice would have shown this?
- Design how you would deploy and monitor a fraud scoring model that must answer in under 100 ms.
- How do you decide between batch, online and streaming inference for a new use case?