Neuromorphic computation has been hyped up for ~ 20 year by now. So far it has dramatically underperformed, at least vis-a-vis the hype.
The article does not distinguish between training and inference. Google Edge TPUs https://coral.ai/products/ each one is capable of performing 4 trillion operations per second (4 TOPS), using 2 watts of power—that's 2 TOPS per watt. So inference is already cheaper than the 20 watts the paper attributes to the brain. To be sure, LLM training is expensive, but so is raising a child for 20 years. Unlike the child, LLMs can share weights, and amortise the energy cost of training.
Another core problem with neuromorphic computation is that we currently have no meaningful idea how the brain produces intelligence, so it seems to be a bit premature to claim we can copy this mechanism. Here is what the Nvidia Chief Scientist B. Dally (and one of the main
developers of modern GPU architectures) says about the subject: "I keep getting those calls from those people who claim they are
doing neuromorphic computing and they claim there is something
magical about it because it's the way that the brain works ... but
it's truly more like building an airplane by putting feathers on it
and flapping with the wings!"
From "Hardware for Deep Learning" HotChips 2023 keynote.
https://www.youtube.com/watch?v=rsxCZAE8QNA
This is at 21:28.
The whole talk is brilliant and worth watching.
The reasoning for this choice in the base ISA is discussed in the RISCV ISA manual, Section 2.4 on "Integer Computational Instructions" [1]. Given that RISCV is a modular ISA, it should be possible in principle to have suitable ISA extensions that do integer overflow detection.
Maybe the absence of such an extension in 2025 indicates that this is not a pressing need for many RISC-V users?
Andy Pitts' writing is extremely clear, whatever he writes about. This clarity is not easy to achieve and shows mastery!
The full abstraction for PCF was solved in the mid 1990s by Abramsky/Jagadeesan/Malacaria [1] Hyland/Ong [2] and Nickau [3]. All three appeared simultaneously. This was a paradigm shift, because all three used used interactive rather than functional models of computation. (There was also later work on domain theoretic full abstraction, e.g. OHearn and Riecke [4], but I can't recall details. Maybe Streicher's work was in this direction?) The beauty of interative models like games is that they can naturally encode more complex behaviour, including parallelism.
[1] S. Abramsky, R. Jagadeesan, P. Malacaria, Full Abstraction for PCF.
[2] J.M. E. Hyland, C.-H. L. Ong, On Full Abstraction for PCF: I, II, and III.
[3] H. Nickau, Hereditarily sequential functionals.
[4] P. O'Hearn, J. G. Riecke, Kripke logical relations and PCF.
Domain theory has reduced to a trickle, with almost no new results since the late 1990s. Most domain theorist have retired, or moved on to other things. Aside, Andy Pitts has been made a fellow of the Royal Society a few days ago!
https://cointelegraph.com/news/neuromorphic-computing-breakt...