I get what you're saying, but I think confidence is actually more important than enjoyment.
A lot of people confuse enjoying [x] with enjoying being good at [x]. This is why so many students switch subjects later on in life; when a field suddenly doesn't come naturally to them, they seek to play to their strengths elsewhere. Problems occur when they quit too early, and building confidence early on is important for stopping this.
In my experience, when you think you're bad at something, it's almost impossible to enjoy doing it, which makes preliminary mastery actually the first step to enjoyment and therefore downstream success.
I think that's more in line with transfer learning, a variant of fine-tuning. If I'm reading this article correctly, they're fine-tuning the LMs end-to-end.
My guess is that it's because they've already done RLHF on top of the standard next token prediction. In other words, they can't cheaply fine tune ChatGPT without undoing the RLHF objective by training on next token prediction with post-2021 data, and then retraining with RLHF to make sure it still gives good human-like output.
I mention the "undoing RLHF" since it's not uncommon for fine-tuned models to increase in error in the original training objective after being fine-tuned with a different one. I think people saw this happen in BERT.