If our own brains were insufficient to demonstrate the fallacy of the Chinese Room; the particles, molecules, cells, synapses, and so forth of our bodies cannot be believed to have understanding, but we recognize it in the sum of the part anyway....
... we now have LLMs to even more pointedly show the deficiencies of the argument.
It’s funny you defend your independence by giving explanations of how you changed your peanut butter buying habits. The point was we don’t care what kind of peanut butter you buy; it’s not a meaningful kind of choice to have.
Because there is so much money for kindergarten teachers, paying an extra salary for a software dev for every kindergarten will certainly lead to better outcomes. The computer programs will make up for the lost teacher economically by teaching the children instead of people, raising the market share of my local kindergarten, or enticing people to have more babies. \s
This is very short sighted, and ignores the lethal trifecta insight.
The LLM doesn’t need to know what it is actually doing (it might think it is searching the web, installing a dev tool, or sending observability data (like metrics), when it is actually sending your API keys to an attacker (maybe in addition to what it thinks it is doing to keep it in the dark).
There have been some very clever things done I’ve seen… even a human reading the transcript may be surprised anything bad happened.
> I see no problem with betting on who will win a sports match...
I do wonder - what if the sports teams or politician loses intentionally; which could either be to profit off the loss or due to threats from an actor who seeks to profit?
I heard that Kalshi paid out for when Khamenei was killed in Iran (the bet was for when he would go out of power), so murdering people could be another way to win such a bet on who will lose. Even injuring a sports player could easily change a game result. With so much money on the line, it doesn't seem like a good mix.
This argument is also explored by the “Quantum Computing for the Very Curious” series that uses spaced repetition to teach an advanced topic. The series has been posted to HN more than once.
There is no real such thing as a read only GET request if we are talking about security issues here. Payloads with secrets can still be exfiltrated, and a server you don’t control can do what it wants when it gets the request.
I personally have jested many times I picked my career because the logical soundness of programming is comforting to me. A one is always a one; you don’t measure it and find it off by some error; you can’t measure it a second time and get a different value.
I’ve also said code is prose for me.
I am not some autistic programmer either, even if these statements out of context make me sound like one.
The non-determinism has nothing to do with temperature; it has everything to do with that fact that even at temp equal to zero, a single meaningless change can produce a different result. It has to do with there being no way to predict what will happen when you run the model on your prompt.
Coding with LLMs is not the same job. How could it be the same to write a mathematical proof compared to asking an LLM to generate that proof for you? These are different tasks that use different parts of the brain.
If you had infinite compute and time could you not just simulate all of “earth” itself? I think the compute needed for that would be absurd and unreasonable to build, but such a simulation would obviously have natural selection going on inside of it.
If that is possible; then perhaps there are many simplified simulations that can still be large and fast enough to support evolution. Or we can leverage actual reality as well, to give digital life embodied experiences in the real world.
Thinking the simulation needs to spread is the wrong way to think about it. The universe does not need to multiply into more universes. If I replace simulation with universe in your final paragraph, the argument makes little sense to me. There can be competition within a simulation, and consciousness seems useful where competition and survival are somehow part of that.
You don’t need consciousness to produce it though. The universe is not conscious, but the rules of how it works allows for them. I believe the universe is essentially computable, and therefore Turing machines can also support consciousness. There would be some class of algorithms we can run which simulate enough of reality for entities within the program to feel alive.
I think you are just misinterpreting my use of the word simulation… I mean the computer is performing math which produces a simulation, but the consciousness still feels alive to itself; and is real.
A simulation of a bomb does not produce a blast, but a true simulation containing a consciousness does produce consciousness.
This is a tired argument that is not worthy of all the long written articles about the impossibility of creating conscious machines. The universe itself appears to me to be some type of computer in a way. Physics cannot explain consciousness, and never will be able to. Particles, energy, whatever works in certain ways according to certain rules. The universe is not conscious, but yet it contains us, who are. I see no reason a sufficiently complex simulation cannot model the universe and contain consciousness within it. The argument that the brain is not a computer is immaterial; the universe is the computer; the brain is the data.
I'd be speaking out of my depth, but I think consciousness is experienced on a sort of information level, and that where ever it is found, some complex network will be found powering it. But that network could be virtual or physical.
I don't think the way we are currently producing LLMs creates consciousness; I just take a very dim view on the argument that computers are incapable of producing a simulation of consciousness; and further propose that such a simulation actually does produce consciousness in a very real sense.
If you know you will be pruning or otherwise reusing the context across multiple threads, the best place for context that will be retained is at the beginning due to prompt caching - it will reduce the cost and improve the speed.
If not, inserting new context any place other than at the end will cause cache misses and therefore slow down the response and increase cost.
Models also have some bias for tokens at start and end of the context window, so potentially there is a reason to put important instructions in one of those places.
Every model update would be a breaking change, an honest application of SemVer has no place in AI model versions.
Not saying using major.minor depending on architecture is a bad thing, but it wouldn’t be SemVer, and that doesn’t even cover all the different fine tuning / flavors that are done off those models, which generally have no way to order them.
The real problem is just managing identities for millions of people. Some of those people will voluntarily use their credentials for someone under 18. Some will sell their identities. There is no technical solution to that.