I know the authors from the blog post quite well. Say what you will about the firm, but one of the authors have been investing in machine learning since 2016, and another has a PhD in CS (including a SIGCOMM test of time award!)
I come from a strong ML background (multiple publications, PhD dropout), I would say that the canon is actually quite good.
I wouldn't go so far. I know the authors quite well, and as someone who has multiple publications in machine learning confeerences (and started a PhD in ML), they know their stuff well.
Yeah, I strongly agree. While Nvidia is working on better hardware (and they're doing a great job at it!), we believe that better training methods should be a big source of efficiency. We've released a new PyTorch library for efficient training at http://github.com/mosaicml/composer.
Our combinations of methods can train CV models ~4x faster to the same accuracy on CV tasks, and ~2x faster to the same perplexity/GLUE score on NLP tasks!
I know the authors from the blog post quite well. Say what you will about the firm, but one of the authors have been investing in machine learning since 2016, and another has a PhD in CS (including a SIGCOMM test of time award!)
I come from a strong ML background (multiple publications, PhD dropout), I would say that the canon is actually quite good.