What should experiment tracking record for each training run?
Options
- Only the best metric
- Only the final model file
- The code version, data version, parameters, metrics, environment and output artifacts (Correct option)
- A screenshot of the notebook
Answer
The code version, data version, parameters, metrics, environment and output artifacts
Why
To compare runs and to repeat one later you need everything that influenced it: code commit, data snapshot, hyperparameters, library versions and random seed, plus the results and the files produced.
Tools such as MLflow store these in a searchable list, so "which run gave 0.91 and how?" has an answer.
A single metric without its context cannot be reproduced.
Remember
Track code, data, parameters, metrics and artifacts.