Ask HN: What are some good Machine Learning Experiment tracking platforms?
I'm talking about tools similar to Comet ML or MLFlow. I'm also interested in hearing about your experiences with them if possible.
3 comments
I don't have experience with this yet, but the CEO of comet.ml gave a demo of it at work this week and it's brilliant. You install 1 pip package, add 1 line of code, and you've got a log every time you train your model (with a diff of what changed). Super-easy to use and share with your team.
I'm thinking one cool use case is: easily let several team-mates try out different model hyperparameters, and you just pick the best one (easy to see in the comet.ml online chart)
I'm thinking one cool use case is: easily let several team-mates try out different model hyperparameters, and you just pick the best one (easy to see in the comet.ml online chart)
This is probably not what you're looking for, but I'll describe my setup. I do CT research, and would basically store information about how an image was generated in the filename and log files. This didn't scale well, so I developed https://github.com/stilley2/filesdb which is a wrapper on Python's sqlite library. It allows me keep track of the parameters used to generate each image and easily search/locate them.
Some tracking and training tools mentioned here:
https://medium.com/@iman.mirzadeh/4-helpful-websites-for-mac...
https://medium.com/@iman.mirzadeh/4-helpful-websites-for-mac...