> the researchers noticed a group of neurons that consistently signaled the wrong response. Instead of fading as learning improved, these neurons grew stronger, and occasionally even nudged the model toward an incorrect decision.
> “It’s counterintuitive,” Miller said. “You’d think neurons that signal the wrong pathway would go away with learning.”
Except they don't tweak weights when the model is incorrect, I'm puzzled why he's making that claim?? In equation 32 they show weights are adjusted in the form `dW = K * S * (Wmax - W) * F(A)`, where F(A) is the feedback given the chosen action A (i.e. reward). K is positive and S non-negative, so Weights can be nudged towards a maximal absolute value Wmax in the direction of F(A).
However, they set `F(A) = {1 if A is correct and 0 when incorrect}`. That means weights don't change when the model is wrong and can only be potentiated (`sign(Dw) == sign(F(A)) == +`)
hmm, can't tell if complete bullshit or a work of genius.
On the one hand, the approach overlaps a lot with my thinking, and has some original tweaks (like the emotionally valenced reward signals). Saying that as someone from a robotics/AI background nowadays involved in GenAI, with a few years of phd research on NeuroAI, curious about molecular neuroscience and the Free Energy Principle (as conceptualised by Karl Friston and Mark Solms).
On the other:
- this plausibility dilemma is the hallmark of LLMs
- has all the buzzwords imaginable
- no code, no raw outputs, no official confirmation (by ARC)
I started reading two recent neuroscience books (elusive cures and natural neuroscience) that while have different goals both highlight the utility of systems neuroscience. In elusive cures the author presented a brief history of the evolving ideological currents where neuroscientists first only cared about about the specific brain region where a stimulus or disorder is happening (first-order effects), then decades later realised the importance of downstream and upstream brain regions (second-order), and are finally coming to terms that the brain is a complex system with coupled regions (third-order).
Seems the article is a contemporary example of the first->second-order realisation...
“This was a very unexpected finding given the current assumptions about how psychedelic medicine works”
"Surprisingly, psychedelic treatment was still able to strongly boost connectivity onto these neurons”
Knowing (those types of) psychedelics bind to serotonin receptors scientists studied neurons with such receptors and didn't focus on the others. Their study looked at other neurons and found plasticity changes there too.
I know people are pushing back, taking "only" literally, but from a reasonable perspective what causes LLMs (technically their outputs) to give that impression is indeed the crux of what holds progress back: how/what LLMs learn from data. In my personal opinion, there's something fundamentally flawed the whole field has yet to properly pinpointing and fix.
That's an orthogonal concern IMO. Running a batch in Europe is about tapping into another source of opportunities. There are plenty of founders that won't or aren't able to attend YC in SF
disagree, there are a few organisations exploring novel paths. It's just that throwing new data at an "old" algorithm is much easier and has been a winning strategy. And, also, there's no incentive for a private org to advertise a new idea that seems to be working (mine's a notable exception :D).
yep, even with greedy sampling and fixed system state, numerical instability is sufficient to make output sequences diverge when processing the same exact input
It is a sensible position. YC and VCs are backing businesses, not charitable causes or research initiatives. It is the founders' responsibility to liaise the two sides in order to signal that the pursuit of their particular purpose is an undeniably attractive and fast-growing investment. Which obviously entails more work and has fewer market opportunities compared to the case "money is the purpose".
After getting backed and receiving adequate funding, all that matters is maintaining a good growth rate to remain a purpose-driven business.
aligns with (or is based on) Demis Hassabis' assessment from yesterday on missing cognitive capabilities for AGI: long-term memory, reasoning, hierarchical planning. He then goes on to suggest scientific creativity may be essential.
> “It’s counterintuitive,” Miller said. “You’d think neurons that signal the wrong pathway would go away with learning.”
Except they don't tweak weights when the model is incorrect, I'm puzzled why he's making that claim?? In equation 32 they show weights are adjusted in the form `dW = K * S * (Wmax - W) * F(A)`, where F(A) is the feedback given the chosen action A (i.e. reward). K is positive and S non-negative, so Weights can be nudged towards a maximal absolute value Wmax in the direction of F(A).
However, they set `F(A) = {1 if A is correct and 0 when incorrect}`. That means weights don't change when the model is wrong and can only be potentiated (`sign(Dw) == sign(F(A)) == +`)
Paper: https://www.nature.com/articles/s41467-025-67076-x