HackerLangs
TopNewTrendsCommentsPastAskShowJobs

Diogenesian

179 karmajoined 15 天前

comments

Diogenesian
·5 小時前·discuss
But that just takes us even further from physical relevance / universal fundamentals. Kolmogorov complexity is fundamental in computation, but its relevance in physical science is pretty selective.
Diogenesian
·6 小時前·discuss
Seems to be the first time this was confirmed via direct experimental observation of the orbitals:

  “This idea that relativity is important in heavy elements has been around since the 1970s,” said Lai-Sheng Wang, a professor of chemistry at Brown and the study’s corresponding author. “But we show direct spectroscopic evidence that what we learned in high school about chemical bonding isn’t true in heavy elements."
Diogenesian
·6 小時前·discuss
Information and computation are not the same thing. I was very specifically talking about Turing machines and other theoretical models of computation.

That said, you are still making the same mistake I pointed out, elevating human symbolic information to a higher plateau than it deserves. I think it's because you're being too vague about the connection, when it's fairly mundane. The "fundamental connection" is that physical quantities are information, and on the other hand information is always a large collection of semi-independent semi-stochastic physical objects and can be profitably modelled by some sort of statistical mechanics. Information theory is relevant all over physics because human agents collect all sorts of physical information. The universe "doesn't care" about information in and of itself. Shannon entropy and Boltzmann entropy have similar formulas because they measure precisely the same thing; put another way, a goofy but formally equivalent way to model a gas would be a noisy radio channel communicating each molecule's kinetic energy.

The problem with the black hole information paradox isn't that information is destroyed per se, but that it appears to be destroyed in a way that violates quantum mechanics (destroying quantum state without a measurement). The theoretically predicted destruction of information points to a more general problem.

The Laundauer limit is no more fundamental to the universe than "a mechanical crane cannot violate the laws of pulleys." It says that no matter how you design your binary (or whatever) computer, it must involve an ensemble of binary states, and statistical mechanics puts an absolute floor on how little heat is required to alter such states. Whether these states are gas molecules or the written symbols "0" and "1" is immaterial.
Diogenesian
·9 小時前·discuss
There are a lot of long comments basically saying what I am about to say so I will try to keep this brief:

Computation is a metaphysically universal and fundamental concept, since metaphysics is (tautologically) the domain of humans and we use symbolic communication. So of course very general theories of symbolic processes (e.g. Turing machines) are pertinent to the symbolic methodology we use to understand scientific processes.

But it is a fundamental mistake to jump from that to saying computation extends to a law of the universe. Computation reflects laws of the universe, but only in the exact same way that scientific and mathematical human speech do. The mystery (still totally unsolved) is how humans are able to intuitively understand space / time / causality / etc in order to define coherent symbolic rules that reflect real processes. That computers can seemingly always implement these rules having been given the symbols is of philosophical/scientific interest, but it's solipsistic to say it's a fundamental concept of the universe.
Diogenesian
·10 小時前·discuss
I didn't see the next paragraph after the proof. This typography is hard to read on a phone. Wish HN would let me delete the comment.
Diogenesian
·10 小時前·discuss
[deleted - the paragraph immediately following the proof of Lemma 2.1 is crucial and I found it hard to read correctly on my phone with the cramped typography. Having reread it I think the proof is correct.]
Diogenesian
·昨天·discuss
This shouldn't be ignored in the discussion here:

  The job performed by the humans was broader than what was requested of the model in this benchmark: humans also had to find the relevant invoices (searching through mailboxes, or requesting them from providers) and reason through any circumstances which cannot be inferred from the bank feed and invoices/receipts on their own. In the benchmark these circumstances are presented to the model as “user notes."
This is precisely the kind of fine print on white-collar AI capability that companies keep running into: pretty much any non-entry office job worth having involves a lot of undocumented (even undocumentable) problems requiring judgment and experience.

And I would be pretty nervous about asking any of the frontier LLMs to retrieve invoices: "cool, Claude logged that it found the May 6th bill from the paper supplier, I am sure it didn't just make something up arbitrary, then compound on the error by agentically iterating over the made-up invoice lurking in its reasoning traces. I checked the first 30 times and there were no problems!"
Diogenesian
·前天·discuss
It's true for lazy and cynical humans who don't care about the quality of their work, and there are more of those today than there were 10 years ago. It's also occassionally true for people with cognitive or psychological problems. Sometimes a bright youngster is simply green, and has to learn the hard way.

But it really is not "true for humans" the way it's true for LLMs. I've said this before: the most depressing thing about the 2020s AI boom is how certain tech folks explain away the lack of intelligence in LLMs by appealing to ignorant and misanthropic folk psychology.
Diogenesian
·3 天前·discuss
[dead]
Diogenesian
·3 天前·discuss
Not "the rest of history," but relative to the ~1800-2000 period of steadily increasing literacy and educational attainment.

The European Dark Ages after the fall of Western Rome was a real thing. Many people regressed to the Stone Age for hundreds of years, and we lost almost every written work from ancient Greece and Rome. That can absolutely happen with the US and EU by 2200, especially considering digital information is far more fragile over centuries than papyrus and parchment.
Diogenesian
·3 天前·discuss
> The reality is that before, you needed to read huge swaths of information to find/know the relevant information. Now you don’t.

> The density of useful information I gather from places like Wikipedia, even long form articles is substantially higher than I got reading non-fiction.

You're in good company. Sam Bankman-Freid:

  I would never read a book. I’m very skeptical of books. I don’t want to say no book is ever worth reading, but I actually do believe something pretty close to that. I think, if you wrote a book, you fucked up, and it should have been a six-paragraph blog post.
You do actually need to read huge swaths of information to understand the relevant information. A good nonfiction book isn't long because of low information density: it's because the ideas are so complicated that it takes an entire book to explain it. Your approach is emblematic of a modern trend where people know a bunch of smart factoids but have no broader wisdom or understanding.

Not reading books because of "information denisty" is a lazy rationalization for dumbing yourself down. Wikipedia is good as a quick reference if you already understand something, but a disaster for learning.
Diogenesian
·3 天前·discuss
Did you click the link? https://gracefulliberty.com/physics/

They are making little web applets demonstrating the physics concepts. You obviously can't do this in pure R6RS/etc., you need to pick some implementation that either has JS/WASM FFI or something built in. Likewise if they were doing this as a desktop application, etc. Pure Scheme can model the physics and return the correct numbers, but you need something more for graphics and interactivity.
Diogenesian
·5 天前·discuss
My source is "none of us have ever seen a robot that can navigate unfamiliar 3D spaces as well as a cockroach." If transformers were capable of the job we would have seen a smart robot by now. But all of our robots are truly mindless compared to the simplest insects.

I will change my mind if someone demonstrates such a robot. Absent this demonstration, cockroach-level AI is still an unsolved problem. Given how ignorant and arrogant and wealthy AI researchers are, it will remain unsolved. I don't think anyone alive today will live to see a robot smarter than an ant.
Diogenesian
·5 天前·discuss
"Virtual intelligence" is better. Transformer ANNs are dramatically dumber than cockroaches and it doesn't make sense to describe such a system as being artificially intelligent, for the same reason it doesn't make sense to describe Half-Life: Alyx as an "artificial reality." An artificial reality implies some sort of scientific fidelity to actual reality. A virtual reality just has to be temporarily convincing. Likewise transformer LLMs have essentially zero actual intelligence - e.g. SOTA "reasoning" models still seem much worse at small-integer quantitative reasoning than almost all vertebrates. But LLMs have an enormous amount of formal subject matter knowledge and inexhaustible stamina at solving tedious O(n) problems. So for many purposes they are an adequate virtual intelligence. At least temporarily.
Diogenesian
·5 天前·discuss
Humans are not stochastic parrots. You are 100% wrong about toddlers. This was clearly explained by St. Augustine 1500 years ago:

  Did I not, then, as I grew out of infancy, come next to boyhood, or rather did it not come to me and succeed my infancy? My infancy did not go away (for where would it go?). It was simply no longer present; and I was no longer an infant who could not speak, but now a chattering boy. I remember this, and I have since observed how I learned to speak. My elders did not teach me words by rote, as they taught me my letters afterward. But I myself, when I was unable to communicate all I wished to say to whomever I wished by means of whimperings and grunts and various gestures of my limbs (which I used to reinforce my demands), I myself repeated the sounds already stored in my memory by the mind which thou, O my God, hadst given me. When they called some thing by name and pointed it out while they spoke, I saw it and realized that the thing they wished to indicate was called by the name they then uttered. And what they meant was made plain by the gestures of their bodies, by a kind of natural language, common to all nations, which expresses itself through changes of countenance, glances of the eye, gestures and intonations which indicate a disposition and attitude--either to seek or to possess, to reject or to avoid. So it was that by frequently hearing words, in different phrases, I gradually identified the objects which the words stood for and, having formed my mouth to repeat these signs, I was thereby able to express my will. Thus I exchanged with those about me the verbal signs by which we express our wishes and advanced deeper into the stormy fellowship of human life, depending all the while upon the authority of my parents and the behest of my elders.
[https://faculty.georgetown.edu/jod/augustine/conf.pdf]

Humans learn language opportunistically. Toddlers start with a powerful "superchimpanzee" understanding of the real world, and use that to learn words in order to satisfy their needs and desires. Statistical frequency is incidental to what words a toddler learns: what matters is the real-world context. Also note how important it is that infants instinctively understand nonverbal communication.

The most depressing thing about the 2020s AI summer is watching ignorant tech workers use the success of LLMs to launder their own ignorant misanthropy. Your views are many many centuries out of date.
Diogenesian
·5 天前·discuss
This is a facile point. Lisp expert systems transparently don't understand the meaning of any symbols they process, yet with enough developer elbow grease they can do all the same things an LLM can do, with much higher reliability. The fact that LLMs are less transparent than Lisp expert systems (and easier to program) is extremely bad evidence that they understand language. Especially given that AFAICT Opus does not properly understand concepts like "four."
Diogenesian
·5 天前·discuss
"You can't spend your whole life worrying about your mistakes. You fucked up! You trusted us!"

https://youtu.be/MYQCb3qrBpo
Diogenesian
·8 天前·discuss
I am unaware of any healthy human who confabulates things as arbitrarily and disastrously as a SOTA reasoning model. It is childish to say stuff like "lawyers always made up court cases" - no they didn't!
Diogenesian
·11 天前·discuss
I would say high school geometry is still mostly "syllogistic" and not the formal philosophical / mathematical logic worked out between Kant and Gödel which forms the backbone of modern mathematics. It is good solid logical thinking, and mathematically correct! - but not really what the author means here.
Diogenesian
·11 天前·discuss
St Augustine had it right 1500 years ago: humans learn words opportunistically according to their desires and problems and existing deep understanding of the world, and (critically) humans really can't learn language without being natively fluent in great ape facial expressions, gestures, grunts, etc.

  Did I not, then, as I grew out of infancy, come next to boyhood, or rather did it not come to me and succeed my infancy? My infancy did not go away (for where would it go?). It was simply no longer present; and I was no longer an infant who could not speak, but now a chattering boy. I remember this, and I have since observed how I learned to speak. My elders did not teach me words by rote, as they taught me my letters afterward. But I myself, when I was unable to communicate all I wished to say to whomever I wished by means of whimperings and grunts and various gestures of my limbs (which I used to reinforce my demands), I myself repeated the sounds already stored in my memory by the mind which thou, O my God, hadst given me. When they called some thing by name and pointed it out while they spoke, I saw it and realized that the thing they wished to indicate was called by the name they then uttered. And what they meant was made plain by the gestures of their bodies, by a kind of natural language, common to all nations, which expresses itself through changes of countenance, glances of the eye, gestures and intonations which indicate a disposition and attitude--either to seek or to possess, to reject or to avoid. So it was that by frequently hearing words, in different phrases, I gradually identified the objects which the words stood for and, having formed my mouth to repeat these signs, I was thereby able to express my will. Thus I exchanged with those about me the verbal signs by which we express our wishes and advanced deeper into the stormy fellowship of human life, depending all the while upon the authority of my parents and the behest of my elders.
[https://faculty.georgetown.edu/jod/augustine/conf.pdf]