Model Evaluation, Monitoring, and Error Analysis in Machine Learning. Is it all the same?
A gentle introduction to what model evaluation means in the ML development lifecycle, the requirements to implement it, and how to avoid common pitfalls.
One of the hottest topics in machine learning (besides the fact ChatGPT can explain jokes) is probably model monitoring and evaluation (read how much funding startups are getting to solve it). However, what exactly is Model Evaluation? How is it related to monitoring, error analysis, concept drift, and all those buzzwords?
In this post, we will go over the topic and focus on the role of Model Monitoring in the ML development lifecycle. We will also cover why some answers are not that straightforward (isn’t evaluation just calculated the ROC curve?) by combining them with some statistics (because statistics is what? … fun-da-mental!)
I will go over the concepts briefly in this post so it doesn’t become endless. However, I will create a series with implementation examples so you can see them in action. Stay tuned!
