The metric used is per-word surprisal: -logprob of each word you type. This is just the same thing as per-word cross entropy or KL-divergence where the user distribution is one-hot. Calibrating it so text generated by frontier models scored poorly was a challenge at first. Originally ChatGPT was scoring around 54%. I'm still having trouble assigning high scores to the personalized Gemini and ChatGPT responses when I'm logged in because all my personal context gives surprising responses.