I've been thinking about recursive self improvement. But especially the likelihood that it will be important soon. Soon means with current LLMs that might just give up or delete the test set or otherwise detach from reality. Call the probability of that happening P. You can estimate P by looking at research tasks and how much they need to get helped along by humans to stay on task.
Goedel machines (that prove that the next step is better) try to use maths as an oracle. Which relies on the mathematical foundations being true. Other oracles I've theorised could help are oracles from the future that can say whether the change detaches the system from reality or not, a sort of go/no-go signal for potentially detaching changes.
Are there any other classes of oracles?
There might be some complex quantum computation that I'm not thinking of, due to classical computer bias.
0 comments
—
Oracles and Recursive Self Improvement · HackerTrans
Goedel machines (that prove that the next step is better) try to use maths as an oracle. Which relies on the mathematical foundations being true. Other oracles I've theorised could help are oracles from the future that can say whether the change detaches the system from reality or not, a sort of go/no-go signal for potentially detaching changes.
Are there any other classes of oracles?
There might be some complex quantum computation that I'm not thinking of, due to classical computer bias.