A shop e-mails each customer a list of recommended products every night. Which kind of inference fits best?
Options
- Batch prediction: score all customers in one scheduled job (Correct option)
- Online (real-time) prediction through an API for every e-mail
- Prediction on the customer's phone
- Retraining the model for every customer
Answer
Batch prediction: score all customers in one scheduled job
Why
Nobody is waiting for the answer while it is computed, and the inputs are known in advance. A scheduled batch job is simpler and cheaper than keeping a real-time service running.
Online inference is for when a user or a system needs the prediction within milliseconds or seconds, for example during a payment.
Batch results are then stored and read when needed.
Remember
No one waiting: batch. Someone waiting: online.