AI ran into the cold hard reality of the legal profession(theregister.com)
theregister.com
AI ran into the cold hard reality of the legal profession
https://www.theregister.com/2026/04/13/ai_attorneys/
6 comments
This is an example of an insurance claim that was denied by AI. The insurance company was ordered to explain the AI decision making process, which they were unable to. Specifically they were unable to provide the llm settings such as temperature, the specific prompt used etc.
Legal AI tools are generally not used for decision making but for improving productivity of legal professionals, for example by helping to review large amounts of contracts. Ultimately lawyers have to review the work and take responsibility for it's correctness. We actually aim to reduce error rates overall. In the case of CompleteFlow, our agentic platform for contract and document processing, we also provide access to all those settings that were demanded by the judge in this case since we do single tenent private cloud hosting and customers can select the models and settings they want to use for a cost / performance trade-off.
Legal AI tools are generally not used for decision making but for improving productivity of legal professionals, for example by helping to review large amounts of contracts. Ultimately lawyers have to review the work and take responsibility for it's correctness. We actually aim to reduce error rates overall. In the case of CompleteFlow, our agentic platform for contract and document processing, we also provide access to all those settings that were demanded by the judge in this case since we do single tenent private cloud hosting and customers can select the models and settings they want to use for a cost / performance trade-off.
Legal AI tools are generally not used for decision making
Once AI is widely deployed and being used, it's not possible to judge or *know* the full extent of it's influence on any decision making process --- within a legal environment or elsewhere. It's pretty safe to say that without some influence, there wouldn't be much reason to use it.
One thing we do know for sure is that AI is inherently probabilistic --- i.e. not necessarily logical, reliable or trustworthy. The same can be said of people but at least people tend to document their decision making process in "important" cases for later review.
we also provide access to all those settings that were demanded by the judge
Knowing the settings still doesn't make any AI results reasonable, logical, free of errors or prudent.
The legal judgments thus far are pretty clear. If AI is used in "important" cases, you better be able to document and justify any conclusions without deferring to the AI "black box". The irony is that this need for documentation and the work required to produce it negates much of the incentive for using AI to begin with.
https://www.lawfuel.com/lawyers-sanctioned-again-for-relying...
Once AI is widely deployed and being used, it's not possible to judge or *know* the full extent of it's influence on any decision making process --- within a legal environment or elsewhere. It's pretty safe to say that without some influence, there wouldn't be much reason to use it.
One thing we do know for sure is that AI is inherently probabilistic --- i.e. not necessarily logical, reliable or trustworthy. The same can be said of people but at least people tend to document their decision making process in "important" cases for later review.
we also provide access to all those settings that were demanded by the judge
Knowing the settings still doesn't make any AI results reasonable, logical, free of errors or prudent.
The legal judgments thus far are pretty clear. If AI is used in "important" cases, you better be able to document and justify any conclusions without deferring to the AI "black box". The irony is that this need for documentation and the work required to produce it negates much of the incentive for using AI to begin with.
https://www.lawfuel.com/lawyers-sanctioned-again-for-relying...
Pessimistic prediction: Companies are already using mandatory arbitration provisions in contracts, some will want a not-really "Expert System" to do the arbitration.
That said, an unscrupulous and lazy judge can already betray their position by outsourcing to an LLM in a personal capacity.
That said, an unscrupulous and lazy judge can already betray their position by outsourcing to an LLM in a personal capacity.
Apparently The Register changed the title of this article. Someone posted it 10 hours before this post: https://news.ycombinator.com/item?id=47749485
Its the same thing , Happening for a while.
And not just in the legal field. The problem boils down to the very fundamental fact that AI is based on probability and statistics.
The results can't be legally defended in many cases because they are not derived from a reasonable, logical, reproduceable decision making process.
Here is an example from the medical industry. Really *important* results shouldn't be decided by a dice roll.
https://pub.towardsai.net/the-air-gapped-chronicles-the-cour...