Ireallyapart

> LLMs are not particularly good at arithmetic, counting syllables, or recognizing haikus, though, because (contrary to the thesis of the article) they don’t magically acquire whatever ability would “simplify” predicting the next token.

LLMs understand it to a certain extent. It's more then "predicting" the next token. When people ascribe "predicting the next token" it's a niave and unintelligent description to cover up what they don't understand.

I mean you can describe a human brain as simply wetware, a jumble of signals and chemical reactions that twitch muscles and react to pressure waves in the air and light. But obviously there is a higher level description of the human brain that is missing from that description.

The same thing could be said about LLMs. I can tell you this, researchers completely understand token prediction that much can be said. What we don't currently understand is the high level description. Perhaps it's not something we can understand as we've never been able to understand human consciousness at a high level either.

That's the thing with people. Nobody actually understands the high level description of a fully trained LLM. People are lambasting others because they "think" they understand when they only actually understand the low level primitives. We understand assembly, but you don't understand the Operating system written in assembly.

Take this for example:

     Me: 4320598340958340958340953095809348509348503480958340958304985038530495830 + 1
     chatGPT: 4320598340958340958340953095809348509348503480958340958304985038530495830 + 1 equals 4320598340958340958340953095809348509348503480958340958304985038530495831.

The chances of chatGPT memorizing or even predicting the next tokens here are in a probability too low to even consider. There are so many possible numbers here even numbers that aren't true but have a "higher probability" of being close to the truth from a token/edit-distance standpoint. It's safe to say, from a scientific standpoint, chatGPT in this scenario understands what it means to add 1.

Realize that this calculation results in an overflow. chatGPT needs symbolic understanding to perform the feat it did above.

But there are, of course, things it gets wrong. But again we don't truly understand what's going on here. Is it lying to us? Perhaps it can't differentiate between just a generated statistical token or a actual math equation. It's hard to say. But from the example above, by probability, we know that an aspect of true understanding and ability exists.

Ireallyapart

3 yıl öncediscuss

Doubtful. The developer is really old school. Additionally he's already spent years on integrating his huge lore generator into the fabric of the game. An LLM will throw a wrench into the whole process.

Ireallyapart

3 yıl öncediscuss

There's no books man. This stuff is too new. But all the components are in place and exist.

This guy just demonstrated what's required to generate a theme, and it's not far off from extrapolating further from that in using a LLM to generate Lore and just some random maze generator to create the base control image.

Ireallyapart

3 yıl öncediscuss

This is good for procedural generated 2D worlds. Think Hollow Knight, but expansive across infinite environments. Just randomly generate the control image and have the LLM generate the theme. Combine that with LLM generated lore and the possibilities are unlimited.

We have the technology to do this right now.

Ireallyapart

3 yıl öncediscuss

> The article claims that Chinese army researchers were at the lab researching viral weapons. That sounds like ill-intent to me.

The US does weapons research it does not mean ill intent. The Chinese does weapons research for the same reason the US does it.

Ireallyapart

3 yıl öncediscuss

Loma Linda is a bluezone because of diet and community. They are very far away from the ocean. The environment their is more desert rather then Mediterranean climate.

How do I know? I live in LA county about a couple miles due west of loma linda. That city is really inland and not considered to be near the ocean at all.

Ireallyapart

3 yıl öncediscuss

Also moisture is trapped in the enclosure between eye and screen.

Ireallyapart

3 yıl öncediscuss

of course they are terrified. Not being terrified is delusional. It's apple.

Ireallyapart

3 yıl öncediscuss

weird. I never had a problem with pyrex. Must be something to do with the environment the glass in your specific case is exposed to.

Ireallyapart

3 yıl öncediscuss

Get the expensive pyrex glass.

comments