Design how you would deploy and monitor a fraud scoring model that must answer in under 100 ms.
Answer
- Serving: an online API behind a container, with the model quantized or sized to meet the latency target, and the p95 and p99 latencies tracked by stage.
- Features: a feature store with an online part for fresh, windowed features and an offline part for training, from the same definitions and with point-in-time joins.
- Release: shadow first, then a small canary against the current model with automatic halt thresholds, and an instant rollback to the previous version in the registry.
- Monitoring: service health, input and prediction distributions by slice, and fraud labels as they arrive (often days later), with alerts on size of change.
- Retraining: on a schedule plus drift triggers, with data validation and a promotion gate on a fixed holdout that includes the recent period.
- Safety nets: a fallback rule-based score if the model or the feature store fails, caching where useful, and an audit trail of model version, features and decision for each score.
- Feedback: investigators' decisions go back as labels, taking care of selection bias, because only flagged cases get reviewed.
Practice this question in Repaso