I was skeptical too, but Supreme Court cases give AI a significant advantage that your example is missing: dozens of pages of briefs describing the case and most relevant facts in great detail for the AI to reference.
In your dispute, the role of a mediator is primarily to find the relevant facts and/or judge the truth of the parties' statements. There's not really any complex legal question to be answered once you determine whose story to believe. This seems like it'd be the case for the vast majority of payment disputes.
The Supreme Court, on the other hand, is trying to decide complex or arguably ambiguous legal questions based on a large corpus of past law, all of which is almost certainly included in an AI's training data. I don't think of the Court as weighing evidence in the way your example requires; all the evidence is already there in the briefs.
So, I'm not sure payment dispute are really strictly simpler than Supreme Court cases, they require a whole different type of reasoning, going beyond the information in the prompt or training data in a way the Supreme Court doesn't have to and the AI cannot.
In your dispute, the role of a mediator is primarily to find the relevant facts and/or judge the truth of the parties' statements. There's not really any complex legal question to be answered once you determine whose story to believe. This seems like it'd be the case for the vast majority of payment disputes.
The Supreme Court, on the other hand, is trying to decide complex or arguably ambiguous legal questions based on a large corpus of past law, all of which is almost certainly included in an AI's training data. I don't think of the Court as weighing evidence in the way your example requires; all the evidence is already there in the briefs.
So, I'm not sure payment dispute are really strictly simpler than Supreme Court cases, they require a whole different type of reasoning, going beyond the information in the prompt or training data in a way the Supreme Court doesn't have to and the AI cannot.