Eliezer Yudkowsky's written a response to this arguing that a six month ban is woefully insufficient. A thing I found particularly compelling is that there's a significant game-theoretic difference between "shut down some projects for six months" (which is easy to see as a political game between different AI players who want to get ahead in the race) and "humanity collectively decides to actually take the difficult actions necessary to solve problem, and sees it as a join endeavor of survival."
When I think about AI through an optimistic lens, I still think pausing to get a better handle on what's going on and how to evaluate it still seems like a useful step. GPT4 represents a substantial increase of capabilities we still don't fully understand.
I don't think it's a crazy luddite position to think "let's actually see how this initial experiment plays out" before racing ahead to the next even large blackbox advance.
https://time.com/6266923/ai-eliezer-yudkowsky-open-letter-no...
See comments on hackernews: https://news.ycombinator.com/item?id=35364833