I find that a surprising number of people in the machine learning field do not track their metrics in a structured and automated way. Some only keep track of what their current single best model is, some put their faith in storing their whole experiment history in TensorBoard graphs, and some manually log their metrics in a Google Spreadsheet. While these methods might be sufficient in some cases, I find that they can be significantly improved in terms of the amount of insight they provide and resources they consume. In this post I will be talking about how to do this, and will go into depth about the why, what and how of tracking machine learning project metrics in a structured manner over time. I’ll be basing this on numerous projects I’ve been involved in, and also the many mistakes I’ve made in them. With metrics, I mean the final metrics you generate from an experiment, rather than the metrics you get per epoch during training.