If you look at the timelines for the hiring of the hardware team, this was an extremely fast and high risk implementation from concept to tapeout. Amazing it works at all during bringup.
Regarding lack of p-hacking. The placebo arm of blinded trials breaks when your brain can detect a medication. The effect is tiny in these studies; approval was rushed to give hope to patients. The drug was discontinued later.
I work in drug discovery. For the past twenty years or so, my personal analogy for this hypothesis has been a fantasy story around the days after the bombing of Dresden, when a new civilization suddenly visits Dresden and has no priors about what may have happened there. The aliens see bricks all over the place and assume that the bricks were the cause of the catastrophe. They take great efforts to pick up the bricks and save a couple of lives from the people who were covered in the ruble. The aliens build better systems to pick up bricks in the future and get ready to act next time. When a nearby city gets bombed, they quickly visit and help recover bricks saving a couple more lives. A different civilization could have instead focused on reducing the bombs or detecting and defending against the attacking airplanes.
Our immune systems are complicated, much more so than airplanes and bombs. The amyloid deposits are very likely part of an immune response, and although in principle immune responses going wild are horrible and can be fixed, it is very important to work on identifying and addressing the causal factors of this disease. There have been more therapies tested on the amyloid hypothesis that mere statistical fluctuations could explain away. I don't always agree with Derek, but I'm with him on this one. New ideas are urgently needed here, or this horrible disease will be an increasingly common end state for our aging populations.
Non-US models are not banned in the US: they are used daily in every state of the US. Some misguided state governments temporarily banned employees from downloading the R1 models and variants released 16 months ago on state government computers. The article and your comment are misleading :-)
The GP did not try to time the market. He suggested a sensible strategy to exclude a tiny subset from an index (less expensive than maintaing the alternative index yourself).
Frankly, everyone in the industry knows. When people make these statements without additional clarity they always talk about API prices. You can look at the NVL72 specs and make estimates for electricity and ownership costs rather easily. Inference at data-center scale is dirt cheap, even with public codes using dynamo and sglang. The mystery is why the early misconceptions about inefficient inference persisted even after NVIDIA was very open about everything they did to help reduce costs dramatically in the last two years.
The key idea is to have many differently named shells. Typically, I group them by project (common prefix name), and the projects live in directories. I have some hacks to organize ibuffer, to split frames, to reflow buffers in the existing windows (eg to organize related project buffers (shells, magit, dired), or to show shells from multiple projects, or selected buffers, and so on). Emacs’ natural frame splitting and buffer selecting/switching commands are good enough if you dont display more than four buffers at a time, but soon you may need to show 5, 6, 8, 9, 10, 12, 15, 16, 18, 20, 24 or have funky arrangements, so you may collect functions to help along splitting n*m grids or keeping useful splits. The shells work very organically inside Emacs and you can still use it as a text editor. The so-called “dumb terminal” in M-x shell is a thing of beauty as it really is just a text buffer like any other; I think of it as a bash repl. If you are used to curses TUI commands it may not work, so for these rare occasions I also use eat (but tend to avoid). See also answer to a sibling comment.
Not sure, tbh. I use emacs -daemon to start a server; emacsclient -nw to connect. I use ssh and start a server on the remote. I spawn multiple shells with infinte buffer size and dumb terminals (M-x shell) so I can seamlessly edit. (These are based on comint, a neat command interpreter.) I use my own hacks for named shells (https://github.com/pjj/Emacs-nsh) and for rearranging/splitting windows, but any of the latest powerful LLMs can help with ergonomic modifications to M-x shell or the various improved terminal emulators (vterm, eat, ansi-term) or with renaming and moving/splitting windows. The Emacs manual is excellent but long; worth it IMHO, but focus on things you use. The tutorial is quick; worth it. I avoid curses programs (fancy TUI) or write wrappers around some of them. I love the -p option in codex/claude/copilot.
Nasdaq is about 8x higher now than then, so 4x higher M2 is tight. Ofc there is always a chance that this time is different and that the markets are genuinely much more efficient :-)
Congrats on the launch. If emacs was unavailable and I needed tmux, I would try it. I am old school, and use emacs daemons for all shell multiplexing. The agents dont need explanations and know how to use emacsclient to create, read, or send inputs to named buffers that run the shells. Elisp is powerful, so manipulating windows is a breeze. Lots of people on tmux would benefit from this design though.
Peter shows the near-term future. Raw API consumer price cost is arbitrary. (The frontier labs can put a 100x markup to cover other operational expenses.) The true cost of inference with same-capability models keeps dropping at dizzying rates, especially at the data-center batch size. (Due to both NVidia hardware and algorithmic changes.) So the developments that Peter can achieve today with internal support from OpenAI will be doable by anyone in a few years without breaking the bank.
I was answering to the question about how to know the probability from this comment:
> The sequence of tokens that would destroy your production environment can be produced by your agent, no matter how much prompting you use.
If you have a specific sequence of an agent that blows up production during debugging, you can certainly check its probability and compare it to one (of same length) that does not blow up your environment. If the two differ by a meteroic amount, it could be pointing to errors in your inference pipeline.
LLM inference is built upon a probability function over every possible token, given a stream of input tokens. If you serve the model yourself you can get the log prob for the next token, so you just add up a bunch of numbers to get the log probability of a sequence. Many API also provide these probabilities as additional outputs.
Not sure what you mean by efficiency as this was part of the article and I understand things differently—can you clarify? For the energy of 20 W in an hour on a laptop’s M4 pro, this model produces about 200k tokens (a book or two) at a typical electricity cost of less than a third of a US cent. Although clearly the intelligence of this particular model is unrelated to human intelligence, I always thought that there is no comparison between LLMs and humans in terms of efficiency: these models are way less energy expensive than humans. If you were to use data center scale optimizations, then serving LLMs is many additional orders of magnitude more efficient than serving LLMs at home. (The energy cost of inference on the M4 pro and iphone are listed in the article.)