A chat assistant seems to remember what you said five messages ago. How does that usually work?
Options
- The model updates its own weights after every message
- The model keeps a private database about each user
- The app sends the earlier messages again with each new request (Correct option)
- The browser stores the model's memory
Answer
The app sends the earlier messages again with each new request
Why
A model call does not keep state: each request is independent.
The chat application keeps the conversation and sends it (or a summary of it) as part of every new request, so the model can read it again.
That is also why long chats get slower and more expensive, and why very old messages can fall out of the window.
Remember
The model is stateless; the app resends the history.