I think so, specifically lossy compression though.
A modern version of the book would include an extra section in the 'Lossy compression' chapter - 'Text' (alongside Images/Video/Audio) that would discuss LLM's.
LLM's seem to be the weird interesting outcome of applying lossy (de)compression concepts to text instead of the audio/image/video domains where they have traditionally been used.
Strange that they are feeding raw audio in. Even in humans, there is a hardware transform to the frequency domain (the cochlea) before data is fed to the brain, effectively doing this part in the LLM seems inefficient.
I'm more impressed by the eerie beauty of it than the technical achievement, even if the code was orders of magnitude larger it would still be wonderful.
I've always wondered if it was perhaps the inspiration for the novel Neuromancer (2 AI's in different continents plotting to combine with each other to form a global super-intelligence)
You are assuming that a battery is full at 4.2V and empty at 0V. In practice a Lithium Ion battery is empty at 3V. You are also assuming the relationship between capacity and voltage is linear - it is not.
This is arguably the reason why the Overton window has shifted towards the rejection of human slavery over the last century or so, with the growth of fossil fuel use.
Human slavery will thus likely swing back into fashion again in the future as oil, coal and natural gas run out.
People are losing their minds at the prospect of oil availability dropping just 20% for a month or two with the closing of the Strait of Hormuz - even just this could collapse the global economy.
So yea, no way is oil stopping or even dipping slightly any time soon.
I wonder if you could use the same technique (RAM models as ROM) for something like Whisper Speech-to-text, where the models are much smaller (around a Gigabyte) for a super-efficient single-chip speech recognition solution with tons of context knowledge.
I think this lack of 'G' (generality, or modality) is the problem. A human visualizes this kind of problem (a little video plays in my head of taking a car to a car wash). LLM's don't do this, they 'think' only in text, not visually.
A proper AGI would have have to have knowledge in video, image, audio and text domains to work properly.
A modern version of the book would include an extra section in the 'Lossy compression' chapter - 'Text' (alongside Images/Video/Audio) that would discuss LLM's.