Great article. One of the papers it cites is https://arxiv.org/abs/2403.07183, which is also great and looks at the issue of LLM usage to write peer reviews.
It’s an issue I’ve noticed personally, as I’m seeing an increasing number of reviews that lack substance and are almost entirely made of filler content. Here’s an excerpt from a particularly egregious recent example I ran into, which had this to say on the subject of meaningful comparison to recent work:
> Additionally, while the bibliography appears to be comprehensive, there could be some minor improvements, such as including more recent or relevant references if applicable.
The whole review was written like this, with no specific suggestions for improvement, just vague “if applicable” filler. Infuriating.