> If an API call can cause a disaster then fix the API
By "API" I'm not referring just to publicly facing REST endpoints. I mean things like shell access for system maintenance, that normally only human professionals like you would be given. In the future it's not clear that humans will be able to dominate that role forever.
Hopefully the issues will be recognized while LLM-based agents are still only serving as retrieval systems.
Not sure if you meant this comment rhetorically or not, but try asking ChatGPT to do something useful like convert some text into JSON, or what conjugation of a verb you should use, or for writing feedback on an email. If you treat it like a toy it will be disappointing but it's useful as a tool.
> Is prompt injection even a problem worth worrying about?
It depends what API access the AI has. If it's just a chat bot, prompt injection can only reveal facts about its language model. But if the AI has POST access to something, depending on what it is, prompt injection can set off arbitrary human-caused disasters.
You might like some of the work being done under the label "Factored Cognition". It's an approach that treats LLMs as building blocks instead of being complete AIs. Instead of asking the LM to solve a problem directly in one pass, you ask it to divide the problem between several different virtual copies of itself, which then themselves subdivide further, and so on until each subtask is small enough that the LM can solve it directly. For this to work the original problem needs to be acyclic and fairly tree-like, i.e., not something that requires having a sudden "Eureka!" moment to solve.
But I've only seen this done with a single model. Sometimes it gets prompted to act like a different agent in different contexts, or given API access to external tools, but it's still just one set of weights.
> A few errors show quickly there is no such concept being weilded
I would have given similar examples to show that ChatGPT makes the same kinds of mistakes that humans do. The first one is good, because ChatGPT can solve it easily when you present it as a riddle rather than being a genuine question. Humans use context and framing in the same way; I'm sure you've heard of the Wason selection task:
https://en.wikipedia.org/wiki/Wason_selection_task
When posed as a logic problem, few people can solve it. But when framed in social terms, it becomes apparently simple. This shows how humans aren't using fundamental abstract concepts here, but rather heuristics and contextual information.
The second example you give is even better. It's designed to trick the reader into thinking of the number 30 by putting the phrase "half my age" before the number 60. It's using context as obfuscation. In this case, showing ChatGPT an analogous problem with different wording lets it see how to solve the first problem. You might even say it's able to notice the fundamental abstract concepts that both problems share.
The third problem is also a good example, but for the wrong reason: I can't solve it either. If you had spoken it to me slowly five times in a row, I doubt I could have given the right answer. If you gave me a pencil and paper, I could work through the steps one by one in a mechanical way... but solving it mentally? Impossible for me.
> It is run through a grammatical filter/generator at the end so it's usually grammatical, but no sort of truth filter (or ethical filter for that matter either).
I kind of thought it did get censored by a sort of "ethical filter" (very poorly, obviously), and also I wasn't aware of it needing grammatical assistance. Do you remember where you heard this?
But comparing 1 human to 1 GPT is mistaken to begin with. It's like comparing 1 human with 1 Wernicke's area or 1 angular gyrus. If you had 100 different ChatGPTs, each optimized for a different task and able to communicate with each other, then you'd have something more similar to the human brain.
Looks like IVF costs around $12,000 today. Considering that this is for your own child, I bet it would be one of the least expensive parts of raising them. I also expect it would become very cheap after it went mainstream.
> The Human Genome Project was the international research effort to determine the DNA sequence of the entire human genome. It took 13 years and was published in 2003, with an estimated cost of over $300 million. Today, a whole human genome can be sequenced in one day for under $1000.
"statistical re-mixer" doesn't describe these systems very well. I see this complaint a lot, that supposedly DL models can only manipulate existing content without creating anything of their own. That's just false, unless your standard for originality is so high that humans can't reach it either.
These models that have hundreds of billions of "synapses", it's not very shocking to me that they can learn the abstract form of concepts. In fact, it's kind of beautiful that human concepts have this mathematical nature. It vindicates Plato, and disappoints everyone who has claimed that language and meaning is arbitrary.
But the main issue here is that for every conceivable empirical test we can perform, you'll still make the same complaint. Even after it's demonstrated better ToM abilities than you, by predicting and explaining other people's mental states better than you can, you'll say the same thing.
Maybe it's because you think that "understanding" requires not just accuracy, but having a certain kind of inner experience that a human could relate to.
I'm always confused. The NWO conspiracy people say that gene editing will be prohibitively expensive (like owning a yacht is), but the real life examples I see don't seem that way. What are some good examples of services that could be done cheaply but are intentionally made expensive for the sake of exclusion?
> Reality, nor even driving, is not anywhere close to that.
This reminds me of the kinds of intuitions people had about Chess and Go. In retrospect they seem silly, but it made plenty of sense to them at the time. The fact was that there was a solution that machines could use that humans couldn't use. Naturally, a solution that humans couldn't use was hard for them to anticipate being effective.