Colombian judge used ChatGPT to make court decision(vice.com)
vice.com
Colombian judge used ChatGPT to make court decision
https://www.vice.com/en/article/k7bdmv/judge-used-chatgpt-to-make-court-decision
58 comments
TLDR: just to generate the legalese needed for the decision, not the decision it self.
This is how I use chatgpt. “Write me an email explaining to stakeholders that we have x” and then I mention as much detail and whatever it spits out I tweak and go from there. I have been told I write amazing emails but my problem is I spend a lot of time thinking my email over way too much, so I like to see how condensed chatgpt gives me of a template so I can reuse it.
I often take a five paragraph email and condense it back down to like three sentences before chagpt, now I can save myself the process.
I often take a five paragraph email and condense it back down to like three sentences before chagpt, now I can save myself the process.
I would honestely have doubts giving an unvetted third party, openai through ChatGPT, access to any prorietery or business relevant information.
> I would honestely have doubts giving an unvetted third party, openai through ChatGPT, access to any prorietery or business relevant information
I have done this, but I fill sensitive information with bullshit terms.
I have just uploaded the results for my analysis of <bullshit>. Some things to keep in mind. There was a request to highlight terms associated with <bullshit>. I have done so, please see the sections titled <bulshit, bullshit, and bullshit>....
Obviously, one can still worry that I may not have "redacted" enough information and that I'm still revealing sensitive information, but I'm comfortable with my ability to determine what's sensitive and what's not.
I have done this, but I fill sensitive information with bullshit terms.
I have just uploaded the results for my analysis of <bullshit>. Some things to keep in mind. There was a request to highlight terms associated with <bullshit>. I have done so, please see the sections titled <bulshit, bullshit, and bullshit>....
Obviously, one can still worry that I may not have "redacted" enough information and that I'm still revealing sensitive information, but I'm comfortable with my ability to determine what's sensitive and what's not.
If a person uses Windows, Microsoft already has access to any information which was ever typed on this computer. Nobody cares about it, despite the fact that it's known that Windows can literally send keystrokes as telemetry data to its home. Why ChatGPT would be different? You either trust that corporations don't abuse their power or you don't engage in any IT.
If a company uses MS or any other aoftware, they have all.konda of liscense agreements and conyracts with those software providers. If an individual employee decides to use ChatGPT to write a report, the comoany does not have any of those in place with openai. That is a major difference.
Depending on the industry and company this may be illegal and grounds for dismissal.
It would violate basically every confidentiality stipulations in any contract I ad in my career so far. I think there is one were it could even have got me jail, worst case. But then there are people leaking classified technical data about modern tanks on wargaming forums...
I give it really generic asks and it says things like [company name]
Yeah I've seen a lot of people who seem to believe that the prompts they give ChatGPT aren't being cataloged and saved when they certainly are.
It sends every prompt you give it to a server, by design it has to as the model is far too computationally expensive to run locally.
It sends every prompt you give it to a server, by design it has to as the model is far too computationally expensive to run locally.
> Yeah I've seen a lot of people who seem to believe that the prompts they give ChatGPT aren't being cataloged and saved when they certainly are.
Are you sure there are a lot of people who believe that? The UI literally saves the prompts on the left side.
Are you sure there are a lot of people who believe that? The UI literally saves the prompts on the left side.
Well, there are enough people using it for work. So at the very least, they don't care about potentially leaking confidential information.
Agreed. One could however use a nonconfidential placeholder for the sensitive information and then replace it with the real information while revising the ChatGPT output.
Even that isn’t enough since even if you use codenames there will be still enough complementary information to fill the gaps especially if OpenAI knows who you are and if you are coming out of a corporate network they probably have enough information based on your internet connection alone.
So say you work for PharmaCorp and you are developing a new drug even if you turn the name of the drug into a code name and you ask OpenAI to write an email about say a failed or successful FDA approval process that’s more than enough for someone to take advantage of it and for you to get fired over it too…
So say you work for PharmaCorp and you are developing a new drug even if you turn the name of the drug into a code name and you ask OpenAI to write an email about say a failed or successful FDA approval process that’s more than enough for someone to take advantage of it and for you to get fired over it too…
Isn’t this the case for all web services as well? Google doc, office 365, aws, Google cloud
No, both from a service and contractural and technical perspective using most SAAS is quite different than using ChatGPT currently.
And overall there is a reason why the USG has their own private AWS zones…
And overall there is a reason why the USG has their own private AWS zones…
I see, may be OpenAI should offer a plan to promise only run inferencing and not inspect the data. Or may be collaborate with amazon to serve the model on private AWS zones.
This is what Microsoft is doing by brining it to Azure.
It will allow you to use the inference model and possibly even train it further on your data without having all the inputs that are going into ChatGPT right now serve as future training content.
It will allow you to use the inference model and possibly even train it further on your data without having all the inputs that are going into ChatGPT right now serve as future training content.
Essentially, yes. One reason not to use them.
>According to the court document, the legal questions entered into the AI tool included “Is an autistic minor exonerated from paying fees for their therapies?” and “Has the jurisprudence of the constitutional court made favorable decisions in similar cases?”
It does say they were fact checked, but it also seems to leave room for questions. Like, for example, did ChatGPT choose the most appropriate answers/background, or just ones that fact-checked well enough? Might that matter later if the decision is rehashed or used as precedent?
It does say they were fact checked, but it also seems to leave room for questions. Like, for example, did ChatGPT choose the most appropriate answers/background, or just ones that fact-checked well enough? Might that matter later if the decision is rehashed or used as precedent?
> Like, for example, did ChatGPT choose the most appropriate answers/background, or just ones that fact-checked well enough?
Yeah. Train it on a rulings that pre-date the civil rights movement and see how great it is.
Yeah. Train it on a rulings that pre-date the civil rights movement and see how great it is.
On the other hand: Train it on rulings from the past 30 years, but remove all traces of gender, race, income, location and education. Then let the A.I. reveal a ton of bias in judges, prosecutors and juries.
Welcome to the era of bullshit AI bureaucracy, where humans aren't able to even understand the rules, but have to follow them.
[deleted]
> Although the Colombian court filing indicates that ... its responses were fact-checked
The problem is that the fact-checkers themselves can make mistakes and chatGPT is very good at making text that looks correct, even to experts. See the Meta StackOverflow post on banning ChatGPT answers for example
The problem is that the fact-checkers themselves can make mistakes and chatGPT is very good at making text that looks correct, even to experts. See the Meta StackOverflow post on banning ChatGPT answers for example
At that point we're back to the fact that humans make mistakes, so the AI is irrelevant. Had they not used AI, they could still make mistakes that look correct, even to experts
It is all a matter of degree and not a binary situation.
Oh interesting. I didn't realize chatgpt works in multiple languages
Not only that it is probably the best translation engine ever made. It translates incredibly well. This is likely one of the industries that is going to be hit severely.
I tried using it to translate subtitles for a movie and the results were pretty bad. I found DeepL giving marginally better results but those were pretty bad as well. To the point that I uploaded the subtitles in opensubtitles.org and they were taken down because of very low quality.
It’s able to translate dead languages reasonably well, too. I asked it to translate various text into Pali and it did a good job, and did a reasonable job providing the text of various Buddhist sutta and decent translations into English.
My wife speaks Bemba - a fairly common language in South-central Africa. I just asked it to translate a sentence to Bemba and it failed miserably. What it put out wasn't even Bemba, she didn't know what language it was.
I guess is it common on the internet? Pali is widely translated and referred to and available (in the Buddhist canon, the original Buddhist teaches were first written in Pali and so it’s widely studied ala Latin). I was impressed but not surprised it was able to do it.
depending on the domain !! it is absolutely confident, right, wrong or in the middle.. actually "dangerous" if you think about a few contexts
the article does a pretty bad job in answering this question, it says there was a translation, and it is implied that what is being talked about is the final decision was translated (by vice?) from spanish, but is silent of any translation of the AI narration, etc
"The arguments for this decision will be determined in line with the use of artificial intelligence (AI),” Garcia wrote in the decision, which was translated from Spanish. “Accordingly, we entered parts of the legal questions posed in these proceedings."
"The arguments for this decision will be determined in line with the use of artificial intelligence (AI),” Garcia wrote in the decision, which was translated from Spanish. “Accordingly, we entered parts of the legal questions posed in these proceedings."
So far my primary use case has been generating non-sense German metal songs and asking it how to more naturally phrase things as I learn Spanish.
I am not sure how it works in Colombia but in many countries, court judgements are lengthy documents that summarise all the aspects of a case before detailing the rationale for the decision. It feels awfully inefficient. Any tool that can help this process sounds like a good idea to me. It's no different to ChatGPT generating some dumb boiler plate code that you can adapt, correct and fine tune instead of having to write it from scratch.
I think it's pretty scary. The judge is asking it to make judgment calls.
> “Has the jurisprudence of the constitutional court made favorable decisions in similar cases?”
Whether or not similar rulings are "favorable" could contain a lot of nuance.
> It feels awfully inefficient.
Good? I think it's important for a judge to have a comprehensive understanding of the current case and all relevant precedents. If they're going to use ChatGPT to inform their opinion, they might as well let it make the ruling.
Imagine if the judge had another person that's not a lawyer read the relevant info and provide a summary / opinion. People would be appalled.
> “Has the jurisprudence of the constitutional court made favorable decisions in similar cases?”
Whether or not similar rulings are "favorable" could contain a lot of nuance.
> It feels awfully inefficient.
Good? I think it's important for a judge to have a comprehensive understanding of the current case and all relevant precedents. If they're going to use ChatGPT to inform their opinion, they might as well let it make the ruling.
Imagine if the judge had another person that's not a lawyer read the relevant info and provide a summary / opinion. People would be appalled.
If it’s making reviewed boilerplate how is it making a judgement call?
Having human beings write copious tedious boilerplate isn’t good. It takes the judges mind and applies to tedium rather than using judgement. Further under tedium humans make mistakes.
Having human beings write copious tedious boilerplate isn’t good. It takes the judges mind and applies to tedium rather than using judgement. Further under tedium humans make mistakes.
> “Has the jurisprudence of the constitutional court made favorable decisions in similar cases?”
That's a judgment call because "favorable" is subjective. Someone who gets sent to jail for 1 year might tell you the prosecutor got a favorable ruling while the prosecutor considers it unfavorable because they wanted the person sentenced to 5 years.
That's a judgment call because "favorable" is subjective. Someone who gets sent to jail for 1 year might tell you the prosecutor got a favorable ruling while the prosecutor considers it unfavorable because they wanted the person sentenced to 5 years.
Note if you read it carefully he only included the responses but wrote parallel arguments and then synthesized the two. So, like I said, it was reviewed.
It’s like complaining engineers use Monte Carlo simulations, which are random, to do analyses of structural integrity instead of doing all the calculations using a human computer[1]. I see these LLM chatbots as being akin to a computational tool like a calculator. It’s great for crunching and provides a force multiplier for a skilled person using it to augment their own abilities, but it’s (not yet) a replacement for the expert.
[1] https://en.m.wikipedia.org/wiki/Computer_(occupation)
It’s like complaining engineers use Monte Carlo simulations, which are random, to do analyses of structural integrity instead of doing all the calculations using a human computer[1]. I see these LLM chatbots as being akin to a computational tool like a calculator. It’s great for crunching and provides a force multiplier for a skilled person using it to augment their own abilities, but it’s (not yet) a replacement for the expert.
[1] https://en.m.wikipedia.org/wiki/Computer_(occupation)
It’s not really clear to me from reading the article what it did here, but it sounds like the judge asked the bot if there is precedent. If that’s the case, what if the issue is nuanced and the precedents are conflicting - the bot might provide favourable rulings which check out under review, but omit unfavourable ones.
The judge wrote a parallel analysis and included the chatgpt answer, an analysis comparing them, and a final analysis incorporating both his reasoning and the chatgpt reasoning.
And you now need to spend a lot of time on correcting. You need to super-carefully read every sentence, because they all sound correct, but may be factually wrong. Are you really saving time at that point?
You don't need to do that for SEO-copy, but for court judgements? I feel like that's one of those things where you don't want to take shortcuts to save and hour or three.
You don't need to do that for SEO-copy, but for court judgements? I feel like that's one of those things where you don't want to take shortcuts to save and hour or three.
That’s what clerks are for, judges rarely write their full judgements they’ll have a team of clerks which will do it based on the instructions from the judge.
There's a way of looking at this as pretty much normal. I think we have yet to really face the fact that systems of common-law are ad-hoc LLMs (in contrast to statutory-law, which is more similar to standard programming).
In particular, common-law uses an enormous "model" of case precedents that gets incrementally updated by human curators with minimal influence over it.
In particular, common-law uses an enormous "model" of case precedents that gets incrementally updated by human curators with minimal influence over it.
Could be interesting to use embeddings so you could get proper legal citations. This could be a great leveler, making it easier to self-represent…
Here's a video from Legal Eagle on the topic from yesterday:
https://youtu.be/Tpq3hRt0pmw
Tl;Dw it's a cold shower on this idea as it's implemented by the startup DoNotPay. There are a myriad of issues but the biggest one is that it isn't possible to foist the liability onto the customer and thus skirt the regulations around the unlicensed practice of law - if you run this service, you are either a law firm or you are a criminal enterprise.
Maybe it could be made to work though, this particular startup doesn't seem to have the deep understanding of the law that would be necessary for the project to succeed. If one started from the assumption they were an AI-assisted budget law firm, rather than an Uber or AirBNB style "we're just gunnuh break the law and get away with it, until we can eventually lobby to change the law," perhaps it could work.
https://youtu.be/Tpq3hRt0pmw
Tl;Dw it's a cold shower on this idea as it's implemented by the startup DoNotPay. There are a myriad of issues but the biggest one is that it isn't possible to foist the liability onto the customer and thus skirt the regulations around the unlicensed practice of law - if you run this service, you are either a law firm or you are a criminal enterprise.
Maybe it could be made to work though, this particular startup doesn't seem to have the deep understanding of the law that would be necessary for the project to succeed. If one started from the assumption they were an AI-assisted budget law firm, rather than an Uber or AirBNB style "we're just gunnuh break the law and get away with it, until we can eventually lobby to change the law," perhaps it could work.
Not all countries comprehensively forbid the unlicensed practice of law - one prominent exception is England, where only a specific set of reserved legal activities require authorization:
https://legalservicesboard.org.uk/enquiries/frequently-asked...
So a service like this would probably be legal there if they avoid the reserved activities.
https://legalservicesboard.org.uk/enquiries/frequently-asked...
So a service like this would probably be legal there if they avoid the reserved activities.
cf: the plain language movement, and the book Language of the Law (1963)