In which GPT-3 answers the question "what is one hundred and five divided by three?" with "35.7". It also gave several other close-but-not-correct answers. It seems pretty unlikely these are all present in the training set, and surely can't all have been lifted verbatim.
I agree systematic testing is probably more useful, but find it really hard to believe this is all happening without any sort of model of arithmetic.
In which GPT-3 answers the question "what is one hundred and five divided by three?" with "35.7". It also gave several other close-but-not-correct answers. It seems pretty unlikely these are all present in the training set, and surely can't all have been lifted verbatim.
I agree systematic testing is probably more useful, but find it really hard to believe this is all happening without any sort of model of arithmetic.