Went to NeurIPS but couldn't hit all the talks I wanted, so I turned workshop sessions and invited talks into 11 podcast episodes (each under 20 min). Some findings that changed how I think about AI:
Reality checks:
- Best vision models: 28.8% accuracy on tasks requiring physics/time/causality
- FDA approvals for generative AI in clinical use: Zero
- 1000x gap in data efficiency between AI and human brains
- 10 years of daily financial data = 2,500 observations (way too small for transformers)
Many companies are commercializing open source, but what metrics really matter? TL;DR - it’s not GitHub Stars
We looked at all 134 oss companies founded since 2015 across their GitHub metrics to assess which are the most telling, from seed to growth (other channels next).
We hosted a virtual roundtable with execs at big companies and founders of startups all commercializing open source, sharing what we've learned about:
- Why you should build a freemium product on top of open source
- Product strategy for open source vs commercial product
- Who should be your first go-to-market hires
Reality checks:
- Best vision models: 28.8% accuracy on tasks requiring physics/time/causality
- FDA approvals for generative AI in clinical use: Zero
- 1000x gap in data efficiency between AI and human brains
- 10 years of daily financial data = 2,500 observations (way too small for transformers)
Actual breakthroughs:
- Brain-computer interfaces detecting subconscious error signals (90% to 99% accuracy)
- Apple Watch pulse geometry predicting heart disease better than cholesterol
- Multimodal fusion eliminating the need for 10,000 human labels per robot task
- Dynamic portfolios spontaneously switching hedging instruments during COVID
Episode list:
1. Society's Fragile Equilibrium - Why "good enough" AI at scale is more disruptive than superintelligence (AGEI vs AGI)
2. The Evaluation Crisis - Why we're testing AI wrong (the infant morality study, Clever Hans effect)
3. The Reasoning Revolution - How o1/o3 actually think, mode collapse, the artificial hivemind problem
4. Engineering Creative AI - DALL-E training 23x faster with one token, world models, IP questions
5. AI Transforms Scientific Discovery - Dolphin communication, ground squirrels to heart treatments, virtual cells
6. Foundation Models for Brain and Body - Biological age gap from wearables, BCI error detection, SSL for neural data
7. Computer Vision's Journey - AI vision is solved, AI reasoning is not (28.8% reality check)
8. Robots That Learn Without Humans - Zero-shot robotics training via multimodal fusion
9. The Autonomous Agent Revolution - Reward hacking everywhere, why rigid multi-agent systems fail
10. Generative AI in Finance - Four pitfalls that break standard AI, task-driven training solutions
11. AI as Time Machine for Science - Years to days, but zero FDA approvals (the central tension)
Sources: NeurIPS 2025 workshops and invited talks (Yejin Choi, Melanie Mitchell, Zeynep Tufekci, etc.)
https://www.basisset.com/#NeurIPS2025