Did they test whether GPT4 replicated already existing social science experiments? If so, this might have happened because the experiment was in the training data
I'm a programmer and data scientist. Using LLMs (specifically Claude 3.5 Sonnet) made me at least 10x as productive. I don't write boilerplate code anymore.
I agree that the feedback loop wouldn't make the models smarter than us, but training data would. I can see how with more computing power we will be able to feed it the whole of YouTube, Spotify and perhaps also millions of physics experiments.
With this data, perhaps a model will be able to understand the hidden patterns of the world (e.g. physics equations governing reality) better than us, don't you think?