A model service returns HTTP 200 for every request, but a bug makes it predict almost the same value for everyone. Which monitoring would catch this?
Options
- Uptime and HTTP status codes
- CPU usage graphs
- Monitoring of the distribution of predictions and of the inputs, not only service health (Correct option)
- The size of the log files
Answer
Monitoring of the distribution of predictions and of the inputs, not only service health
Why
Normal service monitoring (uptime, errors, latency, CPU) says the system is healthy. The prediction here is wrong, not the service.
ML monitoring adds the data: the distribution of inputs, the distribution of predictions (a sudden constant or a shifted mean is a warning), and quality metrics when labels arrive.
Both layers are needed.
Remember
Watch the data and predictions, not only the service.