Because, if they announced it and it seemed plausible or even possible that they were correct, then every media outlet, regulatory body, intelligence agency, and Fortune 500 C-suite would blanket OpenAI in the thickest veil of scrutiny to have ever existed in the modern era. Progress would grind to a halt and eventually, through some combination of legal, corporate, and legislative maneuvers, all decision making around the future of AGI would be pried away from Ilya and OpenAI in general - for better or worse.
But if there's one thing that seems very easy to discern about Ilya, it's that he fully believes that when it comes to AI safety and alignment, the buck must stop with him. Giving that control over to government bureaucracy/gerontocracy would be unacceptable. And who knows, maybe he's right.
Our onboarding docs specifically tell employees to NOT use Google Authenticator precisely because of this issue. I have no idea how Google let this fester for so long, literally if even one (1) person over there was using it and got a new phone, they should have known about the issue.
This thought crosses my mind a lot too. When we're working with code, we're shaping information into structures and patterns. And so our brains are tasked with translating those structures and patterns into text, making sure that text is written exactly right, and reconciling the difference in the observed output of structures and patterns with the text we've written. What if we could stay at that higher level of abstraction?
My de facto example of this that I keep coming back to is syntax highlighting. It's immensely useful, and yet once you get used to it, the brain is able to skip the (conscious) step of breaking it down e.g. "this text is green so it means it is a function name, so the next text should be blue because they are function parameters", etc. It's more like "okay, green, blue, bunch of stuff between the braces, checks out". Indentation, squiggly line error highlighting, etc. all serve a similar purpose. They rely on our minds' fundamental ability to recognize visual patterns without having to fully process them through our language centers.
So I feel like there is a huge unexplored space there for more efficient and natural programming. VR with 3D representations of code as, I dunno, pipes, gears, something like that? Function declarations in specific colors and then call sites in mixed colors when one function is called from another? Anything to reduce the friction from thought to working software without having to worry about syntax.
This is all very abstract and I probably haven't explained it well, but it's one of those things I can't shake because I'm fairly sure there's something there.