Study finds Wikipedia influences judicial behavior(news.mit.edu)
news.mit.edu
Study finds Wikipedia influences judicial behavior
https://news.mit.edu/2022/study-finds-wikipedia-influences-judicial-behavior-0727
117 comments
My guess is that law clerks are using wikipedia as an easy way to find info on cases, which isn't entirely surprising to me
I think law clerks and other legal professionals would be using a curated service (most likely LexisNexis) as their primary information source, not Wikipedia.
Wikipedia is quite unreliable on any major legal case, controversial issue, political personality, corporate entity etc., as the relevant wikipedia pages end up being heavily manipulated by agenda-driven editors who often erase or alter the most important information.
Wikipedia is quite unreliable on any major legal case, controversial issue, political personality, corporate entity etc., as the relevant wikipedia pages end up being heavily manipulated by agenda-driven editors who often erase or alter the most important information.
I would guess that google and other search engines have much better search functionality than curated services. You'd find a case via google, then review the details on a curated service.
Curated services like LexisNexis aren't necessarily free from bias either. One need not use a single service exclusively
Exactly, Wikipedia is the first step for almost every expert professional in every profession. We all do it, and the world is better for it.
Also the causality of the argument in the linked article seems clearly backwards anyway: it's not that "Getting a wikipedia page for your case makes it more likely to be cited", it's "More important/citable cases are more likely to have a wikipedia page". Which is sort of an obvious point.
Also the causality of the argument in the linked article seems clearly backwards anyway: it's not that "Getting a wikipedia page for your case makes it more likely to be cited", it's "More important/citable cases are more likely to have a wikipedia page". Which is sort of an obvious point.
It isn't that obvious that this is alarming. What are we alarmed about?
The point of judges is that they make judgements. The citing of other law is just to try and keep some consistency between the judgements to make them predictable. In theory they could even give out inconsistent judgements and that would also be workable, it just happens that it usually means something bad is happening because judges aren't naturally more pleasant people than the horrible Mass of Humans that causes so much grief. So we want judges to put in an effort at consistency and if they are consistently using Wikipedia as a reference then it is easier for the rest of us to guess what the system is about to do.
It is easy to influence judicial opinion on a case. There are even professionals hired to do it as a full time job! We call them lawyers.
The point of judges is that they make judgements. The citing of other law is just to try and keep some consistency between the judgements to make them predictable. In theory they could even give out inconsistent judgements and that would also be workable, it just happens that it usually means something bad is happening because judges aren't naturally more pleasant people than the horrible Mass of Humans that causes so much grief. So we want judges to put in an effort at consistency and if they are consistently using Wikipedia as a reference then it is easier for the rest of us to guess what the system is about to do.
It is easy to influence judicial opinion on a case. There are even professionals hired to do it as a full time job! We call them lawyers.
The part that concerns ('alarm' implies surprise) me is the confirmation that activist wikipedians have levers and knobs to play with that impact court decisions.
And if Wikipedia does, you can bet Google does too.
And if Wikipedia does, you can bet Google does too.
Seems to me that any concern here is more around case discovery. Whether judges learn about cases from Wikipedia, Lexis Nexus, university publications, etc there is always a bias in which cases get surfaced. Wikipedia being more open than most other venues can be good (more contributors/diversity of viewpoints, easier to critique) or bad (more potentially unsophisticated/ignorant contributions). I tend to bias toward more open platforms given the choice.
>>What are we alarmed about?
While I think the study has some major deficiencies addressed in other comments, if it was true, the alarm for me would be that Wikipedia has clearly become politically partisan in many area's and topics, including some that would intersect the law
Wikipedia is hardly a "neutral" site of just facts
While I think the study has some major deficiencies addressed in other comments, if it was true, the alarm for me would be that Wikipedia has clearly become politically partisan in many area's and topics, including some that would intersect the law
Wikipedia is hardly a "neutral" site of just facts
Do those have topics have anything to do with the class of Wikipedia article mentioned in the study?
I’m not saying there isn’t bias, but the existence of bias in certain controversial subjects isn’t necessarily evidence that articles about cases are subject to the same kinds of issues. I’m not saying they’re immune either, but it’s not clear that the two are related.
I’m not saying there isn’t bias, but the existence of bias in certain controversial subjects isn’t necessarily evidence that articles about cases are subject to the same kinds of issues. I’m not saying they’re immune either, but it’s not clear that the two are related.
But that is what influencing is. By definition, these results show that cases given Wikipedia articles exert an otherwise unproportional influence on the legal landscape.
Not a particularly surprising result, but it shows how much of an effect such a little thing can have, and how big of an influence for example controlling what articles published vs not published could have over time.
Not a particularly surprising result, but it shows how much of an effect such a little thing can have, and how big of an influence for example controlling what articles published vs not published could have over time.
Why would it be alarming. Wikipedia is a fine entry into any topic. Seeing PragerU influencing the court would be alarming.
> Wikipedia is a fine entry into any topic
It isn't, that's the point of the study - not all court decisions are on wikipedia and those that are, are getting cited more. Judges, Lawyers, Clerks have access to the original decisions and court transcripts in electronic form going back decades and longer. If they're relying on whats on wikipedia - they're either doing poor research or are too lazy to write their own summations.
It isn't, that's the point of the study - not all court decisions are on wikipedia and those that are, are getting cited more. Judges, Lawyers, Clerks have access to the original decisions and court transcripts in electronic form going back decades and longer. If they're relying on whats on wikipedia - they're either doing poor research or are too lazy to write their own summations.
... and then rely on LexisNexis et al.? Unfortunately it seems that (US) courts don't really have a good collation of cases (as opposed to collated laws which exists as the US Code).
[deleted]
Sir, I'm afraid your bias is showing...
Pretending PreagerU is anything but blatant propaganda is silly.
Pretending Wikipedia isn't also, is silly. This week's nonsense with the definition of 'recession' is proof enough of that. If you want to argue magnitude, you've already conceded what Wikipedia is (and I don't deny PragerU is what you claim it is).
Wikipedia is not perfectly unbiased, but if you honestly believe it is even comparable to PragerU you should seriously talk to someone about your biases and perception of the world.
There are left-wing equivalents of PragerU (ex: the Gravel Institute) that are just as far away from Wikipedia as PragerU is.
There are left-wing equivalents of PragerU (ex: the Gravel Institute) that are just as far away from Wikipedia as PragerU is.
The method of this study was to deliberately construct a bias on wikipedia, which the researchers were able to successfully pull off with measurable effect.
I think it depends on the topic and section of wikipedia.
If you get into the political area's of wikipeda, I think the bias of Wikipedia is pretty clear polar to that of PragerU.
If you stick to the pure hard sciences (physics as an example) and non-controversial events (like an earthquake) then sure they are unbiased
If you get into the political area's of wikipeda, I think the bias of Wikipedia is pretty clear polar to that of PragerU.
If you stick to the pure hard sciences (physics as an example) and non-controversial events (like an earthquake) then sure they are unbiased
Can you give an example of a wikipedia page that you think shows bias on the same level as PragerU?
Browse through https://en.wikipedia.org/wiki/Category:Wikipedia_neutral_poi... and you'll find plenty.
For example:
> Mohammed Zakir (also known as Meyra) was an Ethiopian Oromo nationalist. Regarded as "legendary Oromo hero", he is noted for his high contribution to keep the lights of Oromo nationalism shining after the martyrdom of his two hero colleagues called Elemo Qiltu and Ahmad Taqi ... the Oromos would never forget this early exemplary hero. And above all, history will always remember Meyra and his heroism.
For example:
> Mohammed Zakir (also known as Meyra) was an Ethiopian Oromo nationalist. Regarded as "legendary Oromo hero", he is noted for his high contribution to keep the lights of Oromo nationalism shining after the martyrdom of his two hero colleagues called Elemo Qiltu and Ahmad Taqi ... the Oromos would never forget this early exemplary hero. And above all, history will always remember Meyra and his heroism.
If you go to the page in question (available here: https://en.wikipedia.org/wiki/Mohammed_Zakir_Meyra) it has not one but two warnings at the top of the page about the content within. Is this not an example of Wikipedia's process working as intended?
There's been an NPOV warning on that page since 2011, and the content has remained substantially the same since 2010. So no, I don't think it is.
Wikipedia runs off of community contributions. This person seems to be extremely obscure, as searching the full name 'Mohammed Zakir Meyra' I found this Wikipedia article, websites that scrape Wikipedia and post its contents, and garbage SEO-optimized websites that had nothing to do with the person. I gave up trying to find anything that wasn't SEO garbage or a Wikipedia scrape on page 5 of the search results.
At least in the English speaking world, this person seems to be a nobody and nobody seems to care enough to fix this article. Honestly, someone that knows how the process works on Wikipedia more than me should probably submit it for deletion, as this (at least in my opinion) shouldn't even meet Wikipedia's notability standards.
At least in the English speaking world, this person seems to be a nobody and nobody seems to care enough to fix this article. Honestly, someone that knows how the process works on Wikipedia more than me should probably submit it for deletion, as this (at least in my opinion) shouldn't even meet Wikipedia's notability standards.
This is one of the most clear examples of bias in favor of leftism that I've seen on wikipedia: https://en.wikipedia.org/wiki/Cultural_marxism
> The term "Cultural Marxism" refers to a far-right antisemitic conspiracy theory which claims that Western Marxism is the basis of continuing academic and intellectual efforts to subvert Western culture.
I feel like a lot of actual marxists would disagree with this statement. People like Antonio Gramsci, who basically laid the foundations for cultural marxism in the early 20th century.
Oddly, if you read about Antonio Gramsci on wikipedia, they all but call him a cultural marxist: https://en.wikipedia.org/wiki/Antonio_Gramsci
> Gramsci is best known for his theory of cultural hegemony, which describes how the state and ruling capitalist class – the bourgeoisie – use cultural institutions to maintain power in capitalist societies.
Also, the last time I looked at the cultural marxism article, it simply said a "far-right conspiracy theory". Now they've somehow managed to roll up antisemitism in there too. Apparently it became antisemitic in just the past year or two!
> The term "Cultural Marxism" refers to a far-right antisemitic conspiracy theory which claims that Western Marxism is the basis of continuing academic and intellectual efforts to subvert Western culture.
I feel like a lot of actual marxists would disagree with this statement. People like Antonio Gramsci, who basically laid the foundations for cultural marxism in the early 20th century.
Oddly, if you read about Antonio Gramsci on wikipedia, they all but call him a cultural marxist: https://en.wikipedia.org/wiki/Antonio_Gramsci
> Gramsci is best known for his theory of cultural hegemony, which describes how the state and ruling capitalist class – the bourgeoisie – use cultural institutions to maintain power in capitalist societies.
Also, the last time I looked at the cultural marxism article, it simply said a "far-right conspiracy theory". Now they've somehow managed to roll up antisemitism in there too. Apparently it became antisemitic in just the past year or two!
> Also, the last time I looked at the cultural marxism article, it simply said a "far-right conspiracy theory". Now they've somehow managed to roll up antisemitism in there too. Apparently it became antisemitic in just the past year or two!
It's been there as long as the article has (almost two years)[1] and was on the Frankfurt School subsection on the Cultural Marxism conspiracy theory for the last 3.5 years[2], so you've off by a while there.
Even before that you can read about how "blacks, students, feminist women, and homosexuals" are the vanguard but its roots are in the anti-semitic criticism of the Frankfurt school, apparently peddled at, for instance, a Holocaust denial conference 20 years ago[3]. I'm not an expert on the subject but doesn't seem recent or much of a stretch to me.
[1] https://en.wikipedia.org/w/index.php?title=Cultural_Marxism_...
[2] https://en.wikipedia.org/w/index.php?title=Frankfurt_School&...
[3] https://www.splcenter.org/fighting-hate/intelligence-report/...
It's been there as long as the article has (almost two years)[1] and was on the Frankfurt School subsection on the Cultural Marxism conspiracy theory for the last 3.5 years[2], so you've off by a while there.
Even before that you can read about how "blacks, students, feminist women, and homosexuals" are the vanguard but its roots are in the anti-semitic criticism of the Frankfurt school, apparently peddled at, for instance, a Holocaust denial conference 20 years ago[3]. I'm not an expert on the subject but doesn't seem recent or much of a stretch to me.
[1] https://en.wikipedia.org/w/index.php?title=Cultural_Marxism_...
[2] https://en.wikipedia.org/w/index.php?title=Frankfurt_School&...
[3] https://www.splcenter.org/fighting-hate/intelligence-report/...
> It's been there as long as the article has (almost two years)[1]
Yes. This was already pointed out. I was mistaken about the antisemitic part not being there earlier.
> its roots are in the anti-semitic criticism of the Frankfurt school
I would have said its criticism is rooted in the fact that Mao rose to power through cultural subversion and is responsible for the deaths of tens of millions.
> apparently peddled at, for instance, a Holocaust denial conference 20 years ago
Interesting that is brought up. To deny the existence of cultural marxism, IMO, is akin to holocaust denial.
Yes. This was already pointed out. I was mistaken about the antisemitic part not being there earlier.
> its roots are in the anti-semitic criticism of the Frankfurt school
I would have said its criticism is rooted in the fact that Mao rose to power through cultural subversion and is responsible for the deaths of tens of millions.
> apparently peddled at, for instance, a Holocaust denial conference 20 years ago
Interesting that is brought up. To deny the existence of cultural marxism, IMO, is akin to holocaust denial.
>> its roots are in the anti-semitic criticism of the Frankfurt school
> I would have said its criticism is rooted in the fact that Mao rose to power through cultural subversion and is responsible for the deaths of tens of millions.
It seems like maybe you're just hung up on definitions and/or the inclusion of "cultural" here? It sounds a lot like you're just looking for https://en.wikipedia.org/wiki/Criticism_of_Marxism and its various offshoots.
The wikipedia article for a specific set of critiques of Marxism from a bunch of racists is not a good base article for a general/systematic critique of Marxism.
> I would have said its criticism is rooted in the fact that Mao rose to power through cultural subversion and is responsible for the deaths of tens of millions.
It seems like maybe you're just hung up on definitions and/or the inclusion of "cultural" here? It sounds a lot like you're just looking for https://en.wikipedia.org/wiki/Criticism_of_Marxism and its various offshoots.
The wikipedia article for a specific set of critiques of Marxism from a bunch of racists is not a good base article for a general/systematic critique of Marxism.
Just because they use the term "cultural" on Anotonio Gramsci's page doesn't make him part of "continuing academic and intellectual efforts to subvert Western culture". Can you substantiate that claim, or is that just made up?
> Also, the last time I looked at the cultural marxism article, it simply said a "far-right conspiracy theory". Now they've somehow managed to roll up antisemitism in there too. Apparently it became antisemitic in just the past year or two!
This is the first ever revision of the article that isn't simply a redirect, and it contains the term: https://en.wikipedia.org/w/index.php?title=Cultural_Marxism_...
> Also, the last time I looked at the cultural marxism article, it simply said a "far-right conspiracy theory". Now they've somehow managed to roll up antisemitism in there too. Apparently it became antisemitic in just the past year or two!
This is the first ever revision of the article that isn't simply a redirect, and it contains the term: https://en.wikipedia.org/w/index.php?title=Cultural_Marxism_...
Antonio Gramsci asserted that the reason Marx's predictions of late-stage capitalism never came to fruition was due to cultural institutions propping up capitalism. He states very clearly that the (new) objective of marxism should be to establish a counter-hegemony within the existing cultural hegemony, thereby subverting it.
This is all present in Gramsci's prison notebooks[1] and is common knowledge to anyone who has ever studied marxist intellectual literature at any length. I find it ridiculous that wikipedia would claim this is a "conspiracy theory". Marxists speak about this _very_ candidly in their writings.
[1] https://en.wikipedia.org/wiki/Prison_Notebooks
This is all present in Gramsci's prison notebooks[1] and is common knowledge to anyone who has ever studied marxist intellectual literature at any length. I find it ridiculous that wikipedia would claim this is a "conspiracy theory". Marxists speak about this _very_ candidly in their writings.
[1] https://en.wikipedia.org/wiki/Prison_Notebooks
This still does not provide any proof for the claim. Of course Marxists want to promote their ideology, even culturally. Nobody denies this. The claim made by people using the term "Cultural Marxism", and the reason it is a conspiracy theory, is that said Marxists do so for the purpose of destroying Western culture.
Again, to be clear: I am not asking for proof that Marxists promoted Marxism (obviously they did), but that they did so "to subvert Western culture".
Again, to be clear: I am not asking for proof that Marxists promoted Marxism (obviously they did), but that they did so "to subvert Western culture".
Here's a fun hour and a half analysis on a page related to Ukraine: https://youtu.be/3kaaYvauNho
> I think the bias of Wikipedia is pretty clear polar to that of PragerU
As if. This article: https://en.wikipedia.org/wiki/Holodomor_genocide_question is still up despite having almost zero pertinent sources.[1] Calling that "polar" is nothing short of a stretch.
At most, Wikipedia could be said to have a neo-liberal bias, an ideology upheld by both of the US' political parties, based on policies alone and not on each individual voter's personal and nuanced adherence. The parties are wrongly assumed to be polar opposites on the political spectrum by most US citizens, but they're not.
[1] https://www.youtube.com/watch?v=3kaaYvauNho
As if. This article: https://en.wikipedia.org/wiki/Holodomor_genocide_question is still up despite having almost zero pertinent sources.[1] Calling that "polar" is nothing short of a stretch.
At most, Wikipedia could be said to have a neo-liberal bias, an ideology upheld by both of the US' political parties, based on policies alone and not on each individual voter's personal and nuanced adherence. The parties are wrongly assumed to be polar opposites on the political spectrum by most US citizens, but they're not.
[1] https://www.youtube.com/watch?v=3kaaYvauNho
That holodomor example isn’t a good one. That article is up for, and mostly pertains to, the debate surrounding whether or not it was technically a genocide.
As the other poster pointed out, it's the pretense that Wikipedia is in any way shape or form neutral that shows the bias.
This study should have been controlled by asking whether courts had access to the curated legal databases (primarily LexisNexis, I imagine):
> "...the increase was bigger for citations by lower courts — the High Court — and mostly absent for citations by appellate courts — the Supreme Court and Court of Appeal."
Secondly, is it possible that the study designers themselves used LexisNexis to research their Wikipedia articles? The study design was based on:
> "...creating new legal Wikipedia articles to examine how they affect the legal decisions of judges. They set off by developing over 150 new Wikipedia articles on Irish Supreme Court decisions, written by law students. Half of these were randomly chosen to be uploaded online, where they could be used by judges, clerks, lawyers, and so on — the “treatment” group."
If these Wikipedia articles were based on LexisNexis research, and that same LexisNexis database was being used by judges and clerks, this would seem to introduce some major biases (making the study rather worthless in terms of measuring Wikipedia influence).
> "...the increase was bigger for citations by lower courts — the High Court — and mostly absent for citations by appellate courts — the Supreme Court and Court of Appeal."
Secondly, is it possible that the study designers themselves used LexisNexis to research their Wikipedia articles? The study design was based on:
> "...creating new legal Wikipedia articles to examine how they affect the legal decisions of judges. They set off by developing over 150 new Wikipedia articles on Irish Supreme Court decisions, written by law students. Half of these were randomly chosen to be uploaded online, where they could be used by judges, clerks, lawyers, and so on — the “treatment” group."
If these Wikipedia articles were based on LexisNexis research, and that same LexisNexis database was being used by judges and clerks, this would seem to introduce some major biases (making the study rather worthless in terms of measuring Wikipedia influence).
The 77 cases that were given a Wikipedia article saw a 17% increase in citations after publication.
One significant outcome of giving a case a Wikipedia article described in the paper but not in the article is that the case receives an infobox visible at the top of search results for most search engines.
One significant outcome of giving a case a Wikipedia article described in the paper but not in the article is that the case receives an infobox visible at the top of search results for most search engines.
Maybe there's a rising tide of interest in certain cases that lifts both Wikipedians and jurists.
Cases were randomly selected to have Wikipedia write-ups posted about them or not.
This is all in the article.
This is all in the article.
A while ago the Oxford English Dictionary wrote a letter to the US Supreme Court asking them not use the OED as an authority.
They said the OED definitions are based on imperfect collections of common use, and aren’t “right” or “wrong”.
They also said the OED often uses SCOTUS decisions as a source, which creates an incestuous link.
They said the OED definitions are based on imperfect collections of common use, and aren’t “right” or “wrong”.
They also said the OED often uses SCOTUS decisions as a source, which creates an incestuous link.
Really interesting.
I searched for this online, but couldn't find it. Might you have a link handy? Much appreciated!
I searched for this online, but couldn't find it. Might you have a link handy? Much appreciated!
Not exactly what you're looking for, but a super interesting article from 2013 from the Oxford University Press's Blog:
The highest dictionary in the land? https://blog.oup.com/2013/06/scotus-marriage-definition-dict...
The highest dictionary in the land? https://blog.oup.com/2013/06/scotus-marriage-definition-dict...
>Unsurprisingly, the increase was bigger for citations by lower courts... The researchers suspect this is showing that Wikipedia is used more by judges or clerks who have a heavier workload, for whom the convenience of Wikipedia offers a greater attraction.
This makes sense. I don't know anything about the legal profession but they must have their own search tools and databases? Is it not easily accessible or digitized for all levels? I don't get why it would be easier in the first place to search Google or Wikipedia.
This makes sense. I don't know anything about the legal profession but they must have their own search tools and databases? Is it not easily accessible or digitized for all levels? I don't get why it would be easier in the first place to search Google or Wikipedia.
If I had to bet, they have those systems but they are a pain to use. If I had to speculate, it would be a lot like those citation search sites you used in College, they simply didn’t have the usability of Google. You had to enter search terms very specifically, whereas with Google you just search for what you want and Google figures out how to structure the query best to give you the results you want.
I can totally see a scenario where a clerk needs to look something up on a Friday afternoon, the though of pulling up the proper search catalog pains them so they shortcut, googling the thing they are looking for a reading the citations on Wikipedia. If I was a law clerk I would 100% do this.
I can totally see a scenario where a clerk needs to look something up on a Friday afternoon, the though of pulling up the proper search catalog pains them so they shortcut, googling the thing they are looking for a reading the citations on Wikipedia. If I was a law clerk I would 100% do this.
Speaking from experience, you're right given its convenience. Though, I (and any responsible lawyer for that matter) will then also look up the case on WestLaw/Lexis given Wikipedia is very much an imperfect source.
Not to mention that Wikipedia also acts as a filter for importance. Important cases are more likely to have articles. This is bad for exhaustive research, but great for "I need to write a summary by tomorrow."
you would be surprised how the world operates outside of tech
it's scary excel sheets and a lot of ducttape
hell I wouldn't be surprised if the next round of innovation will be born out of tech people going into gov and realizing how much man hours there are to automate
it's scary excel sheets and a lot of ducttape
hell I wouldn't be surprised if the next round of innovation will be born out of tech people going into gov and realizing how much man hours there are to automate
That movement has been around a long time - see Code for America and its offshoots for example. It sometimes works, sometimes doesn't. It's easy to think that new tech will fix everything but you need to understand the bureaucracy the tech resides within.
Related discussion: https://news.ycombinator.com/item?id=32253053
It doesn’t seem unreasonable to extrapolate this finding to other areas of society where a small percentage of people write their opinions and then a much larger percent of people read them, eventually shaping aggregate opinion independently of any opinion held predominantly by “lurkers.”
How many elections have been shaped by 4chan, Reddit, and Facebook comments? There have been a few studies on this, eg in 2016 [0] and 2018 [1] which found results that probably will not shock you.
Personally, this might be confirmation/proximity bias, but I believe the effect of small internet communities is severely underestimated by the public and political consultants. This is an area that needs a lot more study, not so that we can stop it, but so we can raise public awareness of how memes spread and give people the tools to think critically without falling into the self-reinforcing cognitive traps that arise from filtering information with poor heuristics.
[0] https://arxiv.org/abs/1610.03452
[1] https://arxiv.org/abs/1805.12512v1
How many elections have been shaped by 4chan, Reddit, and Facebook comments? There have been a few studies on this, eg in 2016 [0] and 2018 [1] which found results that probably will not shock you.
Personally, this might be confirmation/proximity bias, but I believe the effect of small internet communities is severely underestimated by the public and political consultants. This is an area that needs a lot more study, not so that we can stop it, but so we can raise public awareness of how memes spread and give people the tools to think critically without falling into the self-reinforcing cognitive traps that arise from filtering information with poor heuristics.
[0] https://arxiv.org/abs/1610.03452
[1] https://arxiv.org/abs/1805.12512v1
> a small percentage of people write their opinions and then a much larger percent of people read them, eventually shaping aggregate opinion independently of any opinion held predominantly by “lurkers.”
ELI5 brainwashing
ELI5 brainwashing
How did clerks/lawyers find those articles on Wikipedia? Through Google I reckon. I wonder how bad/good the search is for court judgements of their existing system.
Sounds like they need to put those court judgements online, or if they already are, do some SEO work so they rank. Wikipedia was a great way to do that.
Sounds like they need to put those court judgements online, or if they already are, do some SEO work so they rank. Wikipedia was a great way to do that.
LexisNexis, which does other things now but started in the legal space, offers a huge collection of legal opinions with a fairly good search and linking capabilities. Most clerks and law professionals would have access to it.
I think the benefit of Wikipedia is not access to materials so much as it is the succinct summarization of the legal opinions. Perhaps now NLP could help with this, but it's a very complicated problem to provide a summary of the important bits from a 100+ page legal document.
I think the benefit of Wikipedia is not access to materials so much as it is the succinct summarization of the legal opinions. Perhaps now NLP could help with this, but it's a very complicated problem to provide a summary of the important bits from a 100+ page legal document.
> ... the succinct summarization of the legal opinions.
LexisNexis and Westlaw produce succinct summaries of legal opinions. That's the basis of their value, because the legal opinions themselves are not copyrighted. They also categorize everything about an opinion so that their database is searchable by area of law, etc.
LexisNexis and Westlaw produce succinct summaries of legal opinions. That's the basis of their value, because the legal opinions themselves are not copyrighted. They also categorize everything about an opinion so that their database is searchable by area of law, etc.
There are legal search engines which just about every lawyer should have access to. Resorting to google or wikipedia seems like a weird intersection of being comfortable with technology but unaware of any of the standard options.
Previous discussion on HN:
https://news.ycombinator.com/item?id=32253053
https://news.ycombinator.com/item?id=32253053
Lets check how google search results influence judicial behaviour. Ups we already know google creates echo chambers.
Curious what people think about the ethics of this study?
The researchers describe running an experiment on the public without opt-in consent. They also describe allowing an algorithm to do the editing. That raises questions about the kind of edits the algorithm made, how the edits influenced case outcomes, and if/how the research team will clean up Wikipedia now that they’ve concluded the study.
Are there professional guidelines that suggest how experiments like this should be conducted? Do any regulations exist to protect people who get exposed to information as a result of an experiment like this?
The researchers describe running an experiment on the public without opt-in consent. They also describe allowing an algorithm to do the editing. That raises questions about the kind of edits the algorithm made, how the edits influenced case outcomes, and if/how the research team will clean up Wikipedia now that they’ve concluded the study.
Are there professional guidelines that suggest how experiments like this should be conducted? Do any regulations exist to protect people who get exposed to information as a result of an experiment like this?
There is a paragraph on this in the publicly accessible pre-print [1] of the article.
"Approved by the respective ethics boards of MIT and Maynooth University, the experiment amounted to a friendly stress-test of the potential vulnerability of judicial legal reasoning to the limitations of reliance on Wikipedia, notably, its ad hoc topic coverage and unknown author/editorship. The experiment featured Wikipedia entries authored by faculty and by law students under faculty supervision, who each had access, through their university library, to all the relevant primary and secondary legal materials available to judges and their clerks. This assurance of accuracy and of informed analysis in the content of the entries—though short of that offered by a specialist textbook—indicates that judges or lawyers would be unlikely to be misled by what they might read. However, as the authorship of Wikipedia articles is opaque, this fact would not be known to any legal professional when using them. From the users’ perspective, there was no particular reason to imagine that the creators of the relevant entry had any legal expertise—or even that they lacked an ulterior agenda."
I'm curious where I missed the part about the algorithm did the editing? I don't see any matches for "algo" when I control + f the article. "Algo" doesn't appear in the article either. The excerpted paragraph says who wrote the entries and it wasn't an algorithm.
[1] https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4174200
"Approved by the respective ethics boards of MIT and Maynooth University, the experiment amounted to a friendly stress-test of the potential vulnerability of judicial legal reasoning to the limitations of reliance on Wikipedia, notably, its ad hoc topic coverage and unknown author/editorship. The experiment featured Wikipedia entries authored by faculty and by law students under faculty supervision, who each had access, through their university library, to all the relevant primary and secondary legal materials available to judges and their clerks. This assurance of accuracy and of informed analysis in the content of the entries—though short of that offered by a specialist textbook—indicates that judges or lawyers would be unlikely to be misled by what they might read. However, as the authorship of Wikipedia articles is opaque, this fact would not be known to any legal professional when using them. From the users’ perspective, there was no particular reason to imagine that the creators of the relevant entry had any legal expertise—or even that they lacked an ulterior agenda."
I'm curious where I missed the part about the algorithm did the editing? I don't see any matches for "algo" when I control + f the article. "Algo" doesn't appear in the article either. The excerpted paragraph says who wrote the entries and it wasn't an algorithm.
[1] https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4174200
Thanks. You summarized the paper better than the article this discussion thread is about. On closer reading, CSAIL had people write the articles and used natural language processing to detect “linguistic fingerprints of the Wikipedia articles that they’d created”.
On HN front page 4 days ago, with 190 comments: https://news.ycombinator.com/item?id=32253053
This is a repost from three days ago (190pts, 107 comments):
https://news.ycombinator.com/item?id=32253053
https://news.ycombinator.com/item?id=32253053
Knowledge influences behavior? No way, I don't believe it.
Say I want to learn how to do a backflip. I do a quick search, find an instructional video, and nope the fuck out because of how dangerous and difficult it seems to perform. A prime example of knowledge that influenced my behavior.
Pointless study since this is just common sense. So much time and money is wasted in science just so that people can make a name for themselves or get a promotion.
Say I want to learn how to do a backflip. I do a quick search, find an instructional video, and nope the fuck out because of how dangerous and difficult it seems to perform. A prime example of knowledge that influenced my behavior.
Pointless study since this is just common sense. So much time and money is wasted in science just so that people can make a name for themselves or get a promotion.
To me, the fact that Wikipedia influences law is surprising. I would not have expected judges or their clerks to rely on Wikipedia or a resource.
Wikipedia is an amazing wonder of the internet. I could spend ten hours a day reading deep into topics and related topics I only learned because of the initial curiosity I had. But that's beside the point. There is such thorough coverage of all topics of every field so I would indeed expect that judges are using it to look things up because of how convenient it is.
The problem is that in certain subjects (especially those with a potential ethical dimension like social and cultural pages) there is significant bias towards the left (I say that as a left-leaning person).
That's not healthy at all when being used as a resource without awareness and significant caution.
That's not healthy at all when being used as a resource without awareness and significant caution.
Everything is politicized and weaponized.
Can we stop calling stuff like this a "study"? This is not science, this is essentially journalism.
I mean technically yes they are "studying" a subject but this is not a scientific experiment, but it will almost certainly be cited as such just because it's called a study.
I mean technically yes they are "studying" a subject but this is not a scientific experiment, but it will almost certainly be cited as such just because it's called a study.
This is definitely a study and science. It's actually a rather high quality experiment too, I was in awe of the way they carefully constructed the pairs of decisions to make them as identical as possible, and then got the same law student to write an article for each member of a pair before randomizing to ensure they were written as similarly as possible. Everyone knows that you should be blocking to maximize statistical power, but very few people ever go that far. (And that's probably a good part of why despite the relatively small _n_, they still get very clear causal effects.)
I still don't understand why they bothered writing the articles for the ones that weren't being uploaded. I'm sure there's a reason, it just wasn't obvious to me.
Because that threatens the validity of randomization and the comparability of the counterfactual, if the article writer knows it'll be uploaded, they can do things like pick and choose which ones to work hard on. "Oh, I like this case, it accords with my personal politics, so I will work extra hard on it since I know it'll be uploaded." (One example from the RCT literature of how knowledge of the randomization before the intervention can be a bad thing: in one early study of steroids for babies, the hospital nurses 'knew' that steroids helped, and so they would pull out the randomizing ball and if it was 'wrong' for that baby, they'd put it back in and try again. This is why clinical trials try to use pregenerated randomness if they can't blind the nurses.) Even if mechanisms like that don't add a systematic bias, they do add noise and reduce the statistical power to detect an effect.
It also helps the ethics angle if you are simply holding back articles which you will upload eventually.
It also helps the ethics angle if you are simply holding back articles which you will upload eventually.
What about it makes it not science?
Concluding that knowledge influences behavior isn't science. It's common sense.
A conclusion supported by carefully collected data is science. A conclusion based on common sense is not.
You have no idea what science is. Suggesting a hypothesis and then proving it via rigorous testing is what science means. It isn’t whatever the hell you want it to be nor is “science” anything but a method to disprove hypotheses.
They quantified that though.
This is a statistical study, what are you talking about?
Is that necessarily bad?
A best-case-scenario example of influenced judicial behavior is that a judge is about to make a bad decision, but then is informed in some way by an excellent, well-referenced Wikipedia article.
A best-case-scenario example of influenced judicial behavior is that a judge is about to make a bad decision, but then is informed in some way by an excellent, well-referenced Wikipedia article.
This is why in itself Wikipedia cannot be a trusted source but should be treated as purely, an extension of the media. They choose who they can cite and who they cannot.
Wikipedia certainly isn't infallible, and on some topics reading the talk page or comparing how the article reads in other languages can broaden your understanding or completely change it.
But if that disqualifies Wikipedia as a "trusted source", which sources deserve that distinction?
But if that disqualifies Wikipedia as a "trusted source", which sources deserve that distinction?
Could you elaborate? I do see the issue about reliability, but I don’t see how this study is related? They added a random selection of cases and provided summaries without intentional bias. Except for a complete set, this is always the case for every database of cases.
Wikipedia has a very clear pro-science and pro-reality bias, and that's a good thing. You would not want news like OAN to be cited on their.
The problem is that the news sources that Wp trusts have gone completely bananas over the past decade.
Citation, please.
IIRC, as an example, Newsweek lost its trusted status as it adopted a news tabloid format. Further, they no longer have fact checkers and do not print corrections. Surely that's a disqualification right there.
Though I don't see that specific criteria listed:
https://en.wikipedia.org/wiki/Wikipedia:List_of_guidelines
FWIW, here's Wikipedia's roundup of sources:
https://en.wikipedia.org/wiki/Wikipedia:Reliable_sources/Per...
Though I don't see that specific criteria listed:
https://en.wikipedia.org/wiki/Wikipedia:List_of_guidelines
FWIW, here's Wikipedia's roundup of sources:
https://en.wikipedia.org/wiki/Wikipedia:Reliable_sources/Per...
If a news source loses reliability and wiki no longer uses it, how does that support the claim that wiki sources are bad?
Just ask anyone over 40...
I wish those people over 40 hadn't put a US government in place that abolished the fairness doctrine. Then our news sources would probably not be so bananas.
> IRC, as an example, Newsweek lost its trusted status
Precisely, it lost its trusted status, but the GP posts that WP trusts publications that have gone bananas. So your example speaks in favor of Wikipedia's reliability.
Precisely, it lost its trusted status, but the GP posts that WP trusts publications that have gone bananas. So your example speaks in favor of Wikipedia's reliability.
That is not a citation. It's an appeal to generational bias, and fails miserably.
Please provide an appropriate citation.
Please provide an appropriate citation.
You clearly don't edit much of Wikipedia if you think that. Their bias actually toward things that are in the news.
And since the news of today is only interested in the most extreme, that's the facts you'll find on Wikipedia.
Truthfully a much worse problem is that an average user cannot edit on Wikipedia. To edit on Wikipedia you need to have an obsessive personality, to constantly guard your edits and keep other editors from removing them.
And that bias is much worse than any other, you'll find a ton of information on matters that attract obsessive personalities, and much less on more normal subjects.
And since the news of today is only interested in the most extreme, that's the facts you'll find on Wikipedia.
Truthfully a much worse problem is that an average user cannot edit on Wikipedia. To edit on Wikipedia you need to have an obsessive personality, to constantly guard your edits and keep other editors from removing them.
And that bias is much worse than any other, you'll find a ton of information on matters that attract obsessive personalities, and much less on more normal subjects.
> Truthfully a much worse problem is that an average user cannot edit on Wikipedia. To edit on Wikipedia you need to have an obsessive personality, to constantly guard your edits and keep other editors from removing them.
This reads like something coming from the obsessive side.
This reads like something coming from the obsessive side.
I don't have patience to do any of that, and I'm mostly just gave up on editing Wikipedia.
Every once in awhile I go to try to edit something and I'm reminded again how time-consuming it is.
Every once in awhile I go to try to edit something and I'm reminded again how time-consuming it is.
Depends if it's true though, right? Let's not dismiss everything because culture wars.
If it's true there's probably a less controversial source that reported on it. If a controversial source is the only place to report on something, it's probably not true.
And the 'non-controversial' sources post only truth then?
This is really un-critical logic being applied to news consumption.
This is really un-critical logic being applied to news consumption.
[deleted]
Judges are biased, especially in political cases. Look at the joke of trial in Alex Jone's trial right now to see this. He's literally not allowed to plead his innocence to the jury. De facto ruling based on accusation of not turning over all the evidence.
Biased judges like this, love circular reasoning, to find justifications. Hence why Wikipedia is useful. Keep in mind wikipedia change the definition of a recession recently, than when challanged changed the definition of definition. Just last week.
Biased judges like this, love circular reasoning, to find justifications. Hence why Wikipedia is useful. Keep in mind wikipedia change the definition of a recession recently, than when challanged changed the definition of definition. Just last week.
[deleted]
in other news, reading books and attending judicial classes and conferences influences judicial behavior
After watching a female judge sentence a child with autism for killing his rapist and childhood abuser, after he was acquited by a male judge years before, I kind of lost faith in all types of judicial process. AI needs to take over for such things.
You mean when she sentenced a 25 year old with autism of stabbing a bedridden post-stroke 74 year old to death? https://www.bbc.com/news/uk-england-london-62332824
You're really misrepresenting the case, and I doubt a good AI would rule favorably on those circumstances. The judicial system doesn't tend to go easy on stabbing defenseless people to death, even if you can argue that they were bad people or otherwise deserved it.
You're really misrepresenting the case, and I doubt a good AI would rule favorably on those circumstances. The judicial system doesn't tend to go easy on stabbing defenseless people to death, even if you can argue that they were bad people or otherwise deserved it.
If I was bedridden for probably the rest of my life, I’d really appreciate if someone stabbed me to death.
An AI would be trained on human decisions, and unless we're careful it can easily have the same biases as human decision making.
[deleted]
at least some biases we can avoid. We can scrub the training data from mentioning gender or race. We can not include photos of people, to avoid the attractiveness halo effect.
This goes against double jeopardy. Do you have a reference?
I believe "he was acquitted" refers to the abuser, not the killer.
This makes more sense than my reading.
https://www.youtube.com/watch?v=X2NZtcfqjDs
Warning: it's very emotional
Warning: it's very emotional
It finds that "Getting a public Wikipedia article increased a case’s citations by more than 20 percent." That could mean judges are citing cases with wikipedia entries rather than similar cases without and their judicial behaviour remains the same.