Abstract: Progress in machine learning (ML) stems from a combination of data availability, computational resources, and an appropriate encoding of inductive biases. Useful biases often exploit symmetries in the prediction problem, such as convolutional networks relying on translation equivariance. Automatically discovering these useful symmetries holds the potential to greatly improve the performance of ML systems, but still remains a challenge. In this work, we focus on sequential prediction problems and take inspiration from Noether's theorem to reduce the problem of finding inductive biases to meta-learning useful conserved quantities. We propose Noether Networks: a new type of architecture where a meta-learned conservation loss is optimized inside the prediction function. We show, theoretically and experimentally, that Noether Networks improve prediction quality, providing a general framework for discovering inductive biases in sequential problems.
I would love to know what "watching videos" means here. There's a big difference between educational YouTube (Kurzgesagt, Physics Girl, Vertitasium, etc) and TV.
For a long time, I was skeptical that model-free RL would ever be useful on robots. Dramatic advances in simulation have changed my mind. With domain randomization enabling sim2real transfer, it may soon be that classical control is rendered obsolete.
Why use a learned policy instead of a classical controller, you ask? The classical controller is only as good as the dynamics model. The learned policy, when trained with domain randomization, is robust to error in the dynamics model and external disturbances.
The demo here is underwhelming compared to old-school winamp visualizers.
But I think that with deep learning, there is now a real opportunity to make a kickass visualizer. Take a GAN and set the latent vector according to the DFT of the audio stream -- it would be super trippy. You could even use WaveNet and CLIP to make the visuals match the lyrics.
This description of Lex Fridman's research is generous. If you listen to his own podcast (previously called the Artificial Intelligence podcast, now the Lex Fridman Podcast), you'll find that he has a knack for missing the deeper insights of guests discussing those very fields of research.
"We found those who had both the poorest sleep quality and who exercised the least were most at risk of death from heart disease, stroke, and cancer"
If we make the (strong) assumption that sleep is primarily a cognitive repair period, then maybe this isn't too surprising? It would be interesting to do the same study, but measuring cognitive ability instead of risk of death.
Don't get me wrong, Bitcoin is an ecological nightmare and I think it should be shunned accordingly. But the "fun-fact" that it consumes more energy than solar produces is false.
"Power generation from solar PV is estimated to have increased by 22% in 2019, to 720 TWh" [1]
"Bitcoin miners are expected to consume roughly 130 Terawatt-hours of energy (TWh)" [2]
Agreed. That being the case, do you really want to read 20-200 pages of introduction and as much conclusion with a relatively low channel capacity? That simply cannot compare to watching a high quality educational youtube video at double speed.
"You want to consume content that took high amounts of effort to create because it gives you the most bang for your buck."
Youtube is free; books are not.
No, the real reason books are great is because they are slow enough that they give you the time to think about what you are reading. Not because they are highest value.
Shriram Krishnamurthi is the real deal. When I started at Brown I was majoring in Mech-E and took a programming course with Shriram on a whim. It was the hardest course I took in undergrad. Needless to say, I switched my major to Computer Science right after.
Totally agreed. It's so frustrating to me that out of the thousands of brilliant interviewers out there, we got this guy somehow bubbling up to the top. Maybe he brings the conversation down to the masses? His "philosophical" questions are often so bad that they don't even parse for his interviewees!!!
Abstract: Progress in machine learning (ML) stems from a combination of data availability, computational resources, and an appropriate encoding of inductive biases. Useful biases often exploit symmetries in the prediction problem, such as convolutional networks relying on translation equivariance. Automatically discovering these useful symmetries holds the potential to greatly improve the performance of ML systems, but still remains a challenge. In this work, we focus on sequential prediction problems and take inspiration from Noether's theorem to reduce the problem of finding inductive biases to meta-learning useful conserved quantities. We propose Noether Networks: a new type of architecture where a meta-learned conservation loss is optimized inside the prediction function. We show, theoretically and experimentally, that Noether Networks improve prediction quality, providing a general framework for discovering inductive biases in sequential problems.