Data Scientists and ML Engineers – How do you keep track of what you have tried?
2 comments
I am a manager. It is pretty bad in terms of tracking. Wandb looks great but really expensive (small team in a super large corp, pricing we were quoted plus the challenges of no-saas made this a no go for me). Been trying to get team members to mlflow and some adjacent tools but it is too hard to do it right.
Yeah, I totally relate — I often lose track of filtering logic, feature engineering, and other preprocessing steps. Those seem way harder to version and track than just model runs. I’m curious, how did your team build a workflow for managing that kind of complexity?
In your recent projects, how did you keep track of what you tried — preprocessing steps, model runs, or errors?
Did you have a process or system to look back at past experiments and learn from them?
Did you use any tools to help with this, like experiment tracking software? How did that work for you?
If you’ve ever struggled with this, what’s been the hardest part?