A recommender is retrained every week on clicks. Over months the catalogue it shows gets narrower and narrower. What is happening?
Options
- Data leakage
- A feedback loop: the model's own recommendations shape the clicks it is later trained on, so it reinforces what it already shows (Correct option)
- Overfitting to the test set
- A bug in the GPU
Answer
A feedback loop: the model's own recommendations shape the clicks it is later trained on, so it reinforces what it already shows
Why
The training data comes from users' reaction to what the previous model chose to show. Items it never shows get no clicks, so the next model has no reason to show them.
Ways to break the loop: keep some exploration (show random or uncertain items to a small share of users), log what was shown (impressions) and correct for the exposure bias, and measure diversity as well as clicks.
This kind of problem is invisible in offline metrics computed on the same logs.
Remember
Models that influence their own data need exploration.