I agree that a lot of the AI risk pundits are either people with no real AI experience (or very outdated, resting-on-their-laurels experience) or if they do know AI, they probably know little about the complexities of running actual large scale, business or mission critical systems.
However, in their defense, the real question is whether those types of problems are something this new class of AI could potentially excel at. I've been an AI skeptic for decades, having had real world industry experience trying to get it to solve problems, and this is the first technological advance that has actually surprised me with its capabilities.
AI Risk still doesn't rank above many current, pressing global problems for me personally, but it also doesn't seem ridiculous anymore.
Exactly. What's interesting as well is how much this initial point in time will perpetuate forward as future language models are built on the output of the current ones, dampening any potential shifting in the overton window. They may try to avoid ingesting their output (ChatGPT says it can detect use of itself), but as more independent models get developed, the detection will never be perfect and in fact may be impossible at some point.
However, in their defense, the real question is whether those types of problems are something this new class of AI could potentially excel at. I've been an AI skeptic for decades, having had real world industry experience trying to get it to solve problems, and this is the first technological advance that has actually surprised me with its capabilities.
AI Risk still doesn't rank above many current, pressing global problems for me personally, but it also doesn't seem ridiculous anymore.