I don't think there's good evidence that it directly jumps to truth. The aim of betting on a prediction market is to earn money. This incentivizes you to bet on whatever you believe is the most profitable, which is disproportionately going to be whatever you have insider information about, whatever you can influence the outcome of, and whatever you can most effectively fool other people into taking the wrong side of.
> unless you control the stack down to the hardware, it's basically a one way ticket to eternal slavery
Only if mind uploads are economically viable, which I doubt they ever will be. Organic brains are very efficient machines and it is far from clear that whole brain emulation could reproduce someone's intellect and consciousness within a smaller energy budget, or even accelerate it. The brain's behavior is so precisely adapted to its wetware that the emulation would have to reproduce many physical processes that are ultimately irrelevant to general cognition in order to not break the delicate equilibrium of the thing. The overhead would be gigantic.
Better just train from first principles, focusing on capabilities and skipping consciousness altogether. We can already see where this is going: LLMs in their current state would obsolete mmacevedo on many tasks. More likely than not, when it is possible to emulate brains, they will be terribly expensive, run like crap, and the only people interested in running them will be whoever had their own brains scanned.
> Our biology is ridiculously optimized. Like 6-7 orders of magnitude more energy efficient than current day AI/computation stack. It's plausible that AI can never outperform us at what we do best because we are already at the limit.
I do tend to agree with you on that, but with lesser confidence. It doesn't necessarily matter whether they outperform us at "what we do best" if they perform well at doing things that we didn't evolve to deal with. For example, we drove many animal species to extinction, not because we were better than them at anything they were good at, but because we were a novel threat outside their adaptive range. AI could very well do to us what we did to these animals, by acting aggressively enough in a direction that challenges our capacity for adaptation. Basically we have to both perform in our niche, and maintain the relevance of the niche itself in the face of whatever completely unforeseen BS these new technologies may bring about.
I suppose. I think it depends on your tooling somewhat. I mean, looking at the feature article, Bonsai lets you define TUI elements with an HTML-inspired syntax -- so if you're using that, it's a similar difficulty as writing a web interface, minus starting a server, Electron, or styling stuff yourself. You can just stay in the terminal. That's nice! But if you don't have such facilities and you want something more elaborate than just sequentially printing information, TUIs can be a bit rough.
> and then there's just TUIs, which look retrocool and run blazing fast, and are really easy to write
I mean, I don't even think that's true. Many TUIs are bloated, dogshit slow, and it's not trivial to write complex TUIs without glitches or flickering. The more people start making TUIs because it's the current fad, the worse they will get.
The point is more that you can well imagine a future where AI is both better and cheaper than humans at basically all types of work, leading to a situation where we would be entirely unable to fight back in the eventuality that machine owners, or the machines themselves, were to repurpose the resources used for our sustenance to other ends. And this would still fit in the universe-old pattern you've observed.
To put it bluntly, if the economic value of human labour drops to zero, or below the value of human sustenance, it is plausible that the consequence of that, from the cold perspective of cosmic logic, would be the extinction of humans. That's not to say there isn't any way to keep the genie in the bottle and create a utopia for ourselves (I very much doubt it), but that would be against the grain of nature. Call me a pessimist, but if we ever get outperformed in our own niche, our days are numbered.
Note that several of your historical examples didn’t involve humans, and presumably most future occurrences of better work enablers won’t involve humans either. The contention isn’t whether there will be an increase in diversity and amount of work done, it’s whether any of it will be done by us. Which would only be the case insofar that there exists categories of work we do better than AI at that juncture.
People who use AI set the bar themselves when they claim they generate "very high quality work using Claude". Humans more rarely make such claims about the code they write themselves, but when they do, I expect they face similar scrutiny.
AI code is competent, but it's not great or high quality unless you have a good enough eye for quality to steer it with an iron hand. But if you do, you know the quality comes from proper guidance, so you still wouldn't say AI code is great. If you do say exactly that, it comes across as having low standards (which is fine if you own it) and people are going to jump on that just to bring you down a peg.
> They launch fast, run fast, and you use them fast.
I don't know about that. The Gemini TUI takes like four full seconds to start on my machine. I have no idea what the hell it's doing. A lot of the fancy new TUIs that are coming on the crest of the current fad are hot garbage. I hate them.
Considering AC could persist indefinitely in hyperspace while interacting with normal matter, the answer would appear to be "hyperspace", whatever that is.
And my take! A fork of fish where any command that starts with > or a capital letter is fed to $fish_llm_command: https://github.com/breuleux/fish-shell. With Claude's help, that took all of 30 minutes to make.
I don’t tidy up very often, but when I do, it doesn’t take much time or energy. I just dump everything that isn’t version controlled into a junk folder, and it feels great.
It's generally easier to make such a process tamper-proof than an election. You can pick a cryptographically secure open source PRNG and determine the seed in a decentralized way by allowing anyone to contribute a salt into a list which is made public at the deciding moment. Then anyone can verify the integrity of the process by verifying the seed includes their contribution, and computing the candidates themselves.
> It's easier for a small number of people to coordinate, than a large number.
That's basically my main argument for replacing election-based democracy by lottery-based democracy. Electing the right representatives is a coordination problem in and of itself, a process which the wealthy are already quite adept at manipulating, so we might as well cut the middle man and pick a random representative sample of the population instead, who can then coordinate properly.
If the government doesn't have enough power, the wealthy won't need to bribe politicians to do their bidding. They will do their own bidding directly, and there will be nobody to stop them.
It's like, if you want to sell your cyanide penis pills under big government, you need to bribe someone. If you want to sell them under small government, you just... you just sell them, that's what.
There may be ways to design a government where power is better distributed, e.g. using sortition, but ultimately it needs to be richer and more powerful than its wealthiest citizens, otherwise these wealthy citizens will assess, correctly, that when push comes to shove, the laws won't apply to them, and they do not need the government's permission to do what they want.
In the case of LLMs, "prediction" is overselling it somewhat. They are token sequence generators. Calling these sequences "predictions" vaguely corresponds to our own intent with respect to training these machines, because we use the value of the next token as a signal to either reinforce or get away from the current behavior. But there's nothing intrinsic in the inference math that says they are predictors, and we typically run inference with a high enough temperature that we don't actually generate the max likelihood tokens anyway.
The whole terminology around these things is hopelessly confused.
The point is that "predicting the next token" is such a general mechanism as to be meaningless. We say that LLMs are "just" predicting the next token, as if this somehow explained all there was to them. It doesn't, not any more than "the brain is made out of atoms" explains the brain, or "it's a list of lists" explains a Lisp program. It's a platitude.
> These feel like they involve something beyond "predict the next token really well, with a reasoning trace."
I don't think there's anything you can't do by "predicting the next token really well". It's an extremely powerful and extremely general mechanism. Saying there must be "something beyond that" is a bit like saying physical atoms can't be enough to implement thought and there must be something beyond the physical. It underestimates the nearly unlimited power of the paradigm.
Besides, what is the human brain if not a machine that generates "tokens" that the body propagates through nerves to produce physical actions? What else than a sequence of these tokens would a machine have to produce in response to its environment and memory?
> So it looks as though this move — both mean-spirited and commercially incompetent — will result in the loss of about 50 book sales per year.