Accused murderer wins right to check source code of DNA testing kit(theregister.com)

981 pointsby anfilt5 anni fa496 comments

Accused murderer wins right to check source code of DNA testing kit

https://www.theregister.com/2021/02/04/dna_testing_software/

519 comments

phire5 anni fa

> The co-founder of the company, Mark Perlin, is said to have argued against source code analysis by claiming that the program, consisting of 170,000 lines of MATLAB code, is so dense it would take eight and a half years to review at a rate of ten lines an hour.

First, the defence doesn't necessarily have to evaluate all 170,000 lines. They just need to find one buggy line which could potentially overturn the result.

Second, even if it did take a full 8 years, is that a good reason to deny the defendant due process?

breck5 anni fa

Lol. If it would take 8.5 yrs to review, it's probably god awful, and should never ever ever be used to convict someone of such a crime.

My prediction: this firm will probably try to get removed from the case, rather than open source their shitty code.

Source: I've worked on MATLAB codebases for various genomics research projects in the past.

knolan5 anni fa

170,000 lines of Matlab code for a project is not a good sign. Unless they’re also including the source of various Matlab toolboxes which are already tested by the Mathworks.

It’s such a high-level language it’s hard to imagine what the hell they’re doing with all that code. It’s probably mostly useless cruft from GUIDE.

sdenton45 anni fa

My guess is a bit of each: The company high-balling the LoC estimate to try to impress/scare the judge, but prooobably also has a truly terrible codebase.

breischl5 anni fa

How that number was probably arrived at...

PHB: Hey, how many lines of code do we have? CodeMonkey: You want a high estimate or a low estimate? PHB: High CodeMonkey: Well, including unit tests, comments, whitespace, build scripts, integration test harness... 170k

At least I hope they have enough testing code to be signficant...

h0l0cube5 anni fa

Having seen Matlab code that's been exported to C, it's mostly bloated by static arrays, and there's quite a lot of redundant functions. So maybe the LOC count is from a C export?

willis9365 anni fa

I’ve made 4 decent sized gui based tools before. I would never use GUIDE. At minimum it uses eval.

If you want a terrible gui-based gui creation interface: labview awaits.

knolan5 anni fa

I compiled eval once for fun.

willis9365 anni fa

A friend of mine did that. They were able to run arbitrary .m files from an executable. As far as I know, this doesn't violate any terms of use. I wonder what keeps someone from just downloading the freely available MATLAB runtime and running a compiled eval wrapper and suddenly have a freely available version of MATLAB.

Also just because eval can be compiled doesn't mean it should. It will forever be a security risk and I will not write code that uses it in good conscious. Fortunately, Mathworks has provided good alternatives. My personal favorite has been variable field names. It really opens up a lot of elegant coding.

https://www.mathworks.com/help/matlab/matlab_prog/string-eva...

nickodell5 anni fa

>My prediction: this firm will probably try to get removed from the case, rather than open source their shitty code.

That isn't necessarily their choice. The prosecutors will make the decision about whether to withdraw the DNA evidence. They probably won't, given that they would need to give the defendant a new trial, which could lead to an accused murderer getting off. A bad look for any prosecutor.

More to the point, if the firm withdraws from any case where their credibility is questioned, what does that say to law enforcement agencies who are thinking about using their software?

ahepp5 anni fa

My understanding is that (some) law enforcement agencies have been more than happy to drop cases rather than subject investigative tools to proper scrutiny[0]. They have no qualms resorting to "parallel construction"[1], and simply using the inadmissible (sometimes illegal) evidence to find admissible evidence.

[0] https://arstechnica.com/tech-policy/2015/04/fbi-would-rather...

[1] https://en.wikipedia.org/wiki/Parallel_construction

nickodell5 anni fa

Stingrays are more useful as an investigative tool than an evidentiary tool. DNA is the other way around.

canadianfella5 anni fa

gidan5 anni fa

That would be implying that the prosecutor would prefer taking the life of an innocent rather than having it hurt his career, making the prosecutor kind of a criminal.

melq5 anni fa

>making the prosecutor kind of a criminal.

Never met a lawyer before huh?

Jokes aside, prosecutors pushing through cases they know to be unsound isn't exactly uncommon. Many prosecutors are more concerned with their conviction rates than they are in justice, because that's what they are measured and rewarded by.

rtx5 anni fa

I often hear this, who is rewarding them for high conviction rate.

vkou5 anni fa

Voters, because when it comes to issues of criminal justice, crowds are rarely paragons of sober temperance and restraint.

jabits5 anni fa

I think you are wrong and that most prosecutors want to do the right thing, like most working people

JakeTheAndroid5 anni fa

"Right" and "wrong" are dependent upon the system and how it rewards you.I would agree that most prosecutors what to serve justice for malfeasance that has been committed. That's different than whether a case is the "right" or "wrong" one to take.

If a case seems unclear, and you could spend years working on a conviction that will ultimately fall through, that hurts your ability to do justice for more readily winnable cases. You have to spend the time building a case, do all the paperwork, go to trial, etc. That's opportunity cost. So spending that on a case you have 10% chance of winning just isn't a good use of time. Add that to the fact that conviction rate is a metric used to quantify skill, you're rewarded for serving justice successfully. And that then dictates how much money you can get which can help fund enforcing justice.

I believe you're looking at the moral right/wrong, and I don't believe that is the same right/wrong being discussed in terms of how lawyers often choose cases. At the end of the day, lawyers need work and they get that mostly through word of mouth and reputation. You don't really get either of those when you lose cases.

devlopr5 anni fa

You're version of the right thing and the prosecutors version might not align.

The right thing for them is to put as many criminals behind bars. They review cases and pick ones they can win. They will attack and find unrelated weak points in your character to win. They believe they are doing the right thing and will use whatever they can legally against you. You being innocent and going to court is means someone made a mistake. To confess to a mistake loses you credibility, to confess to an ongoing process mistake could open up other cases where dangerous people could be set free.

Is that your version of the right thing?

michaelmrose5 anni fa

It is trial if wrong to convince one's self that accused are probably guilty and that actions that convict them are moral even the proof is insufficient or weak or the procedure flawed.

Most people want to do the right thing wherein right thing is almost entirely defined by norms and customs of their environment. If the norms and expectations are high ethical and correct standards people will follow them to the degree they are able.

To what degree are such standards broken or defective in America though?

Lest we forget the head lawyer of Texas a state home to aprox 27 million people or around 8% of the nation is a man whose own prosecution has for years only been stymied by the difficulty of prosecuting the man at the head of the states justice department. Either 8 or 9 (I've lost track) directly beneath him have resigned and accused him of corruption.

This isn't even an isolated instance corruption is found in fact all over the united states.

Even when in theory we would like to do the right thing we have a hard time establishing what standards are even real. Look at the fact. For proof of that look no further than the science of hair analysis which the FBI spent decades using to convict the accused before we realized that they were incapable of differentiating dog hair from human hair.

Think of entire people going in to work producing work product about imaginary science they were pretending to do competently and sending people to death row in part because of their fake work product.

https://www.washingtonpost.com/local/crime/fbi-overstated-fo...

The justice system in America is a bad joke that is primarily differentiated from say Cuba in that bribes are paid to your lawyer instead of directly to government officials.

spoonjim5 anni fa

Prosecutors are shaped by an environment that equates “the right thing” to “punishing the guilty.” It’s like any profession... a surgeon will think you need surgery and a prosecutor will think the guy in handcuffs needs to go to jail.

hluska5 anni fa

Sadly, evidence contradicts that thought. It shouldn’t but it does.

hutzlibu5 anni fa

Everyone wants to do the right thing.

It is just that some think the right thing for themself is to maximize their career progress.

And I would not know in general about state prosecutors, but what I know anecdotally second hand, does not sound good.

melq5 anni fa

I believe that's true as well, and I never said otherwise.

XorNot5 anni fa

The prosecutor doesn't see it that way. They see it as just "knowing" the guy is "definitely guilty". It's just like, a feeling you know? And a win will look great when they go for re-election (why is that even a thing?).

Presuming rational actors in this case is missing the general problem with the system: people very easily convince themselves they know the truth despite how the validity of the evidence changes. Whatever it said initially, that must be right - it's misinformation 101. Once a belief is established it is much harder to change.

jfengel5 anni fa

And a win will look great when they go for re-election (why is that even a thing?).

You would prefer that they not be elected? That they would be appointed by some politician, with the public having no recourse?

The fact is that the public like prosecutors who convict people. That's deeply unfair. But it's also deeply democratic.

XorNot5 anni fa

You already elect politicians. If that system is producing people you don't trust to manage the affairs of state, why would electing prosecutors lead to different results?

jfengel5 anni fa

Hard to say, really. It's one of those compromises, quis custodiet and all that.

I very much agree with you: a government has a monopoly on violence and ultimately we all end up trusting it. Too many checks and balances lead to gridlock. Too few lead to oppression. Much of it ends up being decided on inertia. We do it both ways in different jurisdictions, with successes and failures in both.

mlindner5 anni fa

That's not how prosecuters work in the US. Their goal is to win the case, not make the "right" decision. They'll spin evidence as hard as they can against the accused.

nickodell5 anni fa

>taking the life of an innocent

The prosecutor isn't unilaterally deciding whether the DNA evidence is valid. There will be a public hearing where both the prosecution and defense show evidence about the validity of the DNA evidence, and a court will rule based on that evidence.

afthonos5 anni fa

You should read up on the rates of plea bargaining, as well as the methods prosecutors use to push defendants to do so, which include:

- Not revealing all information they are required to.

- Parallel construction (see above)

- Overcharging, with the goal of making the plea more palatable than the cost/risk of defending multiple absurd charges.

- Lying to you while getting to throw you in jail if you lie to them.

As a result, only 5% of federal cases go to trial.

None of behaviors these are rare. If your understanding of the legal system is based on popular culture, as most people’s is, it is basically law enforcement propaganda that has little relationship to reality.

nickodell5 anni fa

Believe it or not, I was already aware of all of those things, having followed a number of criminal defense blogs.

If you read the article and appellate decision which is linked, it says what I just said:

>On Wednesday, the appellate court sided with the defense [PDF] and sent the case back to a lower court directing the judge to compel Cybergenetics to make the TrueAllele code available to the defense team.

afthonos5 anni fa

Apologies, I thought you were stating general facts about DNA evidence in general, not about this specific case.

bscphil5 anni fa

Yeah, the system is in a pretty horrific state when you have to count on prosecutors' restraint for anything. Granted, we are in such a state, but it's beneficial not to just accept that as the status quo.

bdavisx5 anni fa

It would also give every person convicted using their software an incentive to open an appeal.

Spivak5 anni fa

I like how this is considered a bad thing. Like we can’t let this guy point out that he’s being convicted by an unauditable black box that suddenly isn’t worth using if it has to stand up to scrutiny because then everyone would want to. The horror.

Like I’m actually kinda shocked this is the reality. I would have assumed that DNA evidence would have some blessed methodologies and tools/algorithms, with a strict definition of what constitutes a match or partial match specifically so this wouldn’t happen.

belorn5 anni fa

Here in Sweden, there is a legal practice that you can't find someone guilty based on DNA evidence alone. Probabilistic evidence is nice to point law enforcement in a direction, but there is always a risk of false positives.

In this case we are also dealing with probabilistic genotyping involving DNA Mixtures with DNA from several individual contributors, and most likely degraded DNA. It is the tool the police can use when other more traditional methods is not possible because of the mixture. That should mean the qualitative value of the DNA evidence is lower, requiring even stronger additional evidence from other sources.

Blikkentrekker5 anni fa

In the U.S.A., a man can be convicted upon the word of a single witness, even if the defence poked significant holes into the reliability of said witness.

What can happen in the U.S.A. is that one lone man says “I saw the defendant do it.”; the defence attorney can point out that the witness was drunk at the time, that he has motive to lie, that he initially reported another story to the police and only later settled on this story, and what ever else to render him completely unreliable.

The jury can nevertheless return a verdict of guilty, and there are no grounds for appeal then, as it is the power of the jury to decide who is “reliable”, and it is not required to explain it's thought process at all.

What a shocking development that such would result into a criminal justice system where a defendant's race and gender plays such a factor.

mlindner5 anni fa

> What a shocking development that such would result into a criminal justice system where a defendant's race and gender plays such a factor.

It takes only one person in the jury to hang the jury. It's not a majority vote it's a unanimous vote.

Blikkentrekker5 anni fa

Bench trials are also required to be unanimous.

Methinks the U.S.A.-man often thinks that bench trials in other countries are done by a single juror; they are not and can range from three to twelve in how many professional jurors are required to reach a unanimous conclusion.

But this is not so much about lay fact finding vis-ǎ-vis trained fact-finding, but the rules of evidence.

Scotland also has jury trials, but does not permit that a man be convicted upon the word of a single witness; there must be further independent, corroborating evidence.

There are many other differences with, for instance, the Dutch system that guarantee a fairer trial. One very big one is that in the Netherlands both the defence and prosecution have one groundless appeal; either side if it not agree with the verdict can demand a fresh new trial with different jurors once. — this obviously reduces flukes of justice.

The other is far stronger rules of evidence and more consistent rulings. Juries are very fickle and legal experts rarely know what verdict they will return based on the evidence they saw before them; whereas with trained jurors, their verdict is often similar with the same evidence given to them.

Indeed, one might argue that the practice of plea bargains, which would be considered unconceivably unethical in most jurisdictions, are actually the saving grace, as they permit stability to this otherwise fickle system as the negotiations between both parties are more reproducible given the same evidence, than fickle juries.

MereInterest5 anni fa

Interesting. What does Swedish law consider non-probabilistic evidence? Even something like eye-witness testimony I would consider to be probabilistic, given how easy it is to manipulate memories, even unintentionally.

acct7765 anni fa

Clear videographic, use of a PIN that only the accused had access to, etc

gwd5 anni fa

How do you prove that only the accused had access to a PIN? Surely that's probabilistic as well?

ALittleLight5 anni fa

Isn't all evidence probabilistic? What's an example of something that isn't?

hluska5 anni fa

This is one of these scary areas where reality matches my teenaged experiences playing Shadowrun. I used to hope that the brutal dystopia we played through was just fun. Now I’m seeing that the present needs a word even more brutal than dystopia. :(

alchemism5 anni fa

Kafkatopia

hluska5 anni fa

You nailed it! Thanks friend, that’s some wicked writing and thinking.

Blikkentrekker5 anni fa

I do not find this reality worse at all than people being convicted upon the black box testimony of blood splatter analysts, which is simply an expert testifying that in his conclusion the blood indicated such-and-that.

Or of course, that the U.S.A. permits conviction based on the sworn testimony of a single eye witness, which is noteably unreliable.

All of these are black boxes that are routinely meant to convict. — it would not surprise me if such software were far more reliable than human eye witness accounts, but if there's one thing I noticed, it's that a man is seldom afraid of bad matters, he is only afraid of bad matters produced by new technology; far worse matters can stay, so long as they be ancient enough.

Aachen5 anni fa

> If it would take 8.5 yrs to review, it's probably god awful, and should never ever ever be used to convict someone of such a crime.

It's not like you review all scientific evidence and re-do the experiments that lead up to the discovery of <insert some evidence method> in the first place. Validating all that would also take years and much of it can be established as generally accepted by all parties. Similarly, there will be some trust involved with this source code as well. Getting the opportunity to look for bugs is essential in my opinion, but it needn't take multiple years. Focus on the parts you doubt, similar to what you'd do if you were reviewing the scientific method used in analog evidence.

Of course, the two aren't identical. Validating scientific methods and validating a program is different in that the program is proprietary and the science (usually) merely behind a paywall. The latter can then be replicated by others and becomes established. The former will only ever be seen by that company and doesn't become established. So scrutiny is necessary, but after a couple cases that used an identical version, requiring access without articulating particular doubts would unduly delay the case. It doesn't seem unreasonable to start trusting the program after a bunch of defendants had experts look at it and found no way to cast doubt on its result. If you don't think software of 180k lines can be used in court under such circumstances because it would take too long to review, we should throw out pretty much all software anywhere in the judicial system. (That's not what you said, but some of the replies including yours hint at that.)

inetknght5 anni fa

> It's not like you review all scientific evidence and re-do the experiments that lead up to the discovery of <insert some evidence method> in the first place.

Actually, it is. That's how science works and that's how convictions often get overturned.

> Validating all that would also take years

Are you suggesting that unvalidated data is being used to prosecute crimes?

> and much of it can be established as generally accepted by all parties.

The point here is that it isn't established as generally accepted by all parties.

> Similarly, there will be some trust involved with this source code as well.

"Trust but verify"

> If you don't think software of 180k lines can be used in court under such circumstances because it would take too long to review, we should throw out pretty much all software anywhere in the judicial system.

I firmly believe that if the source code isn't available to review by all parties, including the public, then it shouldn't be used in a criminal court.

gowld5 anni fa

> Are you suggesting that unvalidated data is being used to prosecute crimes?

Yes. Pseudoscience is the bread and butter of criminal forensics.

tylerhou5 anni fa

I think they meant "should be used to prosecute crimes."

vkou5 anni fa

> It's not like you review all scientific evidence and re-do the experiments that lead up to the discovery of <insert some evidence method> in the first place. Validating all that would also take years and much of it can be established as generally accepted by all parties. Similarly, there will be some trust involved with this source code as well

There are a few important differences between a generally accepted method, and some Matlab black-box that you feed an input into, and it prints out 'guilty' and 'not guilty'.

1. The former is based on centuries of peer review, where the best ideas eventually get selected for. The latter is an externally un-reviewed application, which encapsulates the best of whatever we could ship by Thursday.

2. You can call an expert witness to the stand, and ask them questions about the state of the art of <some evidence based method>. You can ask them why. You can ask them about how certain one should be about their statements. You can't cross-examine a black box.

The actual solution to your quandary is to require that forensic analysis services must pass an annual, independent, double-blind analysis of the accuracy of their methods, before they are used in a courtroom - and that the results of those audits are made available to the defense.

It's one thing for a man in a lab coat to take the microphone and say that their methods are accurate 'to within one in a million'. It's quite another to see an audit, where 100 samples were sent in for analysis over six weeks, and only 92 of them were analysed correctly.

A jury might still convict on the basis of that 92% accuracy, but only if other meaningful evidence points against the defendant.

Unfortunately, the reality of forensic science in 2021 is that most of it is sloppy bunk, with no assurances of accuracy.

SilasX5 anni fa

>The actual solution to your quandary is to require that forensic analysis services must pass an annual, independent, double-blind analysis of the accuracy of their methods, before they are used in a courtroom - and that the results of those audits are made available to the defense.

Agreed! But if that's the standard, it still doesn't involve letting the defendant see the source code.

gowld5 anni fa

In USA, accurate to 1 in a million means you can convict 300 innocent people for every guilty one.

Bad stats, especially around DNA, has convicted many innocent people.

BTW, law and Order did an episode on bad DNA science convicting someone.

vkou5 anni fa

As an innocent person, I'd much rather take a 1 in a million chance at a wrongful conviction, than risk a jury trial.

ClumsyPilot5 anni fa

the point he is making that 1 in a million was an outright lie used by prosecutors to secure convictions on innocent, while the real criminals are still out and about

dragonwriter5 anni fa

> Validating scientific methods and validating a program is different in that the program is proprietary and the science (usually) merely behind a paywall.

Or completely fictitious.

Have you heard the story about the FBI crime lab and the “science” of fiber analysis that they developed, and not only used in federal criminal trials but also provided as a service for state and local agencies for decades?

ArnoVW5 anni fa

Or the Shirley McKie debacle in Scotland in 2005, where it turned out that finger print detection was more of an 'art' than a science. The ball got rolling once they started convicting police officers (so at least the analysis was double-blind?)

Or the phantom of Heilbronn, where dozens of crimes were linked to a single woman. Who turned out to be the lab technician that assembled the kits. Doubts started once they discovered the caucasian female DNA in cells of the charred remains of a black male.

I often wonder how prosecuters defend against the use of these cases to create doubt.

https://en.wikipedia.org/wiki/Shirley_McKie

https://en.wikipedia.org/wiki/Phantom_of_Heilbronn

Blikkentrekker5 anni fa

It indeed doesn't seem fair to them that they are forced to share their trade secrets simply because the prosecutors elected to use their software.

hansel_der5 anni fa

it doesn't seem fair to sentence someone (to death?) based on a unaudited secret sauce tool.

the company can always retract their product if they want to keep it a trade secret.

Blikkentrekker5 anni fa

You assume it to be one or the other.

The real solution would obviously that the prosecutors cannot enter into evidence the conclusions of any closed source software.

Of course, this only displaces the issue, what of the black box c.p.u.'s whereupon this software ran?

wdobbels5 anni fa

If their argument is that their codebase is basically unreadable, then I see why they're scared someone might find some bugs here and there.

carlmr5 anni fa

I'd get an expert witness in there to testify basically that.

I also don't think you should code anything mission critical like this in Matlab. It's a decent language for prototyping, not for production.

ChrisLomont5 anni fa

The numerics in Matlab are far better than pretty much any developer can produce in production. This is why Matlab is used in production - it's vastly more reliable than people rebuilding the things it is good at by hand for bespoke solutions.

bigiain5 anni fa

"I could whip this up in JavaScript in about NaN or [object Object] hours over a weekend!"

WmyEE0UsWAwC2i5 anni fa

True. But I believe this is a case where correctness and clarity are the paramount concerns.

There should be a public reference implantation of these methods if they are going to be used in court.

generalizations5 anni fa

Seems like mathematical clarity is what Matlab is good at, at least compared to e.g. python and C.

carlmr5 anni fa

Most industry Matlab I've seen is similar to numpy code, heavily vectorized to make it work fast, somewhat inscrutable for everything that's not linear algebra, and a lot of assumptions about perfect floating point precision. Couple that with a unit-testing unfriendly culture and you have a code disaster. Especially on 170k lines.

ChrisLomont5 anni fa

Most industry C/C++/Java/Python/XXX code I have seen in production is a numerical disaster. I've been working in all these codebases for decades.

There's nothing you just wrote that is any better in any other language, except that Matlab provides a huge suite of state of the art numeric routines that almost no everyday developer could come close to making as solid.

Writing a nicely illustrated manual on brain surgery with nice fonts and proper grammar based on 11th century medicine is of little use for doing actual brain surgery.

Writing clean code based on bad numerics is also of little use for producing good results. Especially if you then have to defend that codebase in court.

Bad developers will make bad decisions in any language. At least using solid numerics underlying the code provides a huge benefit to building the entire codebase instead of on crap numerics. Every nice clean codebase I have been part of has still had crap numerics. Good numerics is nearly completely orthogonal to clean code, and it's a highly technical skill set that almost no developer has even an inkling of how to do well, no matter how pretty their formatting and documentation. I have never in 30+ years of working on highly technical teams worked with someone who really gets the nuances and details of how to do solid numerical code. I routinely get codebases and developers that do the absolute worst things numerically. I have only really good people in conferences on such topics, or online from similar filtering. These people are extremely rare in software development, to the point I don't think I've ever met on on an actual project (and the numerics when needed have always fallen to me, and I've often been selected for technical projects because such people are terribly hard to find when needed).

heavyset_go5 anni fa

> I'd get an expert witness in there to testify basically that.

Sounds really expensive.

sudosysgen5 anni fa

Not necessarily. I'd happily do it for a reasonable hourly rate + transportation. It would end up costing not more than a few thousand dollars, which is very reasonable given our legal system.

Hell, if it seemed outrageous enough I'd probably do it for transportation costs alone.

I'm sure I'm not the only one with this outlook.

cyberlurker5 anni fa

It might not be as much as you think. I know professionals who’ve been paid a few hundred dollars to be an expert witness more than once, but usually in medicine. It’s easy money for someone if a lawyer often does certain cases that require an expert witness.

Helloworldboy5 anni fa

themeiguoren5 anni fa

Laughs in autocoded control systems

kspacewalk25 anni fa

>Second, even if it did take a full 8 years, is that a good reason to deny the defendant due process?

"It's just gonna take so long, plus the code is a bit messy. We're gonna be doing all that work just because the rest of someone's life teeters on the results of the inquiry? Maan, that's a bummer."

Cthulhu_5 anni fa

> Second, even if it did take a full 8 years, is that a good reason to deny the defendant due process?

No, but the person that wants to have it analyzed will have to either spend the time themselves, or pay the expert witness for their time; it could be a costly affair.

But I think it's warranted. An independent software review, and a double blind assertion with the exact version of the software used in the conviction to test the accuracy and reliability of the application.

edgyquant5 anni fa

Any software used to convict people, especially on a such a serious crime, should be audited like the fed is. Twice yearly, once by a public firm and another by the government itself. It should have to pass both of these audits to be used.

contingencies5 anni fa

It should be audited by science.

edgyquant5 anni fa

Yes there should be teams comprised of people whose expertise is in biology (specifically whatever field is responsible for DNA matching) and other people who are programmers that review the code and make sure there’s no mistakes. That would do it and I’m surprised this isn’t already a thing.

bigiain5 anni fa

It should totally be available to be "red teamed", like the researchers who exploit fingerprint readers with gummy bear moulds made with laser prints of fingerprints, or who hold up photographs to face recognition systems.

The government should send a half a dozen to DefCon/CCC and let attendees loose trying to fool them.

contingencies5 anni fa

Hardware and business process would also need review. Its no good having perfect code if you can insert the wrong sample. It's also no good having perfect code if a well-timed EMI burst or power level shift games the result.

01343405 anni fa

Definitely. There was even a documentary about it on Netflix about how crimelab technicians were pushed for results rather than accuracy. A couple major scandals involved intentional faking of tests and resulting in countless people going to prison and the judicial consequences it caused as well as the friction involved to seek justice. Like, oh no, all these people convicted for life on flimsy grounds is such a burden of our time.

edgyquant5 anni fa

Eh that’s overkill. If can be proofed and guaranteed that it isn’t bugging out we don’t need to run through every building block this way.

contingencies5 anni fa

I would have said that too before starting a hardware company. Now, I'm not so sure.

bigiain5 anni fa

> but the person that wants to have it analyzed will have to either spend the time themselves, or pay the expert witness for their time; it could be a costly affair.

Sure.

And the prosecution using the company claiming to have "totally reliable DNA evidence" should be totally on the hook for those costs (plus damages) when that analysis or expert witnesses show up "reasonable doubt" flaws in the software or the processes in which that software is used, including then risking retrials or mistrials of all other cases in which it was used.

If the prosecutors want to play high stakes games with defendants lives and liberty using "evidence" from proprietary software or devices, they need to be held to the consequences of losing their stakes.

[Edit: I wonder what the legal system would think of a CyberGenetics competitor funding the expert witness analysis of their software on behalf of the defence???]

buzzerbetrayed5 anni fa

Wait until you hear how long it would take at a rate of 2 lines per hour, as long as we're just throwing out random numbers.

[deleted]5 anni fa

unishark5 anni fa

A lot of bashing over a vague third-hand quote without a source.

In the very next paragraph they say:

> The company offered the defense access under tightly controlled conditions outlined in a non-disclosure agreement, which included accepting a $1m liability fine in the event code details leaked. But the defense team objected to the conditions, which they argued would hinder their evaluation and would deter any expert witness from participating.

So it's a concern about IP protection for them.

breischl5 anni fa

Maybe, or maybe that's just the smokescreen they put up to prevent anyone looking at it.

I notice that they didn't say "here's the results of our last independency audit and verification of correctness", which I think would be a fantastic counter-argument... if they had one.

gowld5 anni fa

There is no legitimate need for IP protection against destroying innocent lives.

contingencies5 anni fa

Indeed. In fact, if there were a globally agreeable case for open science and open source, to which all governments could contribute, DNA analysis for criminal attribution would have to be it, no? A side benefit would be that sending evidence to numerous international labs would greatly frustrate attempts at domestic law enforcement / lab corruption.

colejohnson665 anni fa

“If the criminals know how our system works, they can work around it!” is probably what they’re thinking. i.e. security through obscurity.

himinlomax5 anni fa

> So it's a concern about IP protection for them.

For that kind of product, source code is not actually that valuable in itself; it's the standards compliance, reliability and trustworthiness. Most charitable explanation is that the vendor is clueless about what their value really is, least charitable is that they know exactly how fucked up their code is.

michaelmrose5 anni fa

Arguably the public analysis of the correctness of their methods ought to be done once in a fashion that is usable by all former and subsequent defendants. This is supported by their own claims that such an analysis will be extremely onerous.

If its a multi million dollar affair like they claim its virtually impossible that every defendant will be able to fund such an affair.

In fact in the case that a disastrous flaw is found it may be advantageous to simply drop the case and hope that past and future defendants wont be able to each afford to press the point.

maceurt5 anni fa

It shouldn't take 10 lines an hour should it? I don't have experience reviewing professional code of this size, so please correct me if my assumption is wrong, but that number doesn't seem right.

jjkaczor5 anni fa

As part of a quality control team, I personally went through over 1.2 million lines of working code (i.e. not including comments) over the span of about 8 months, M-F, 9am-5pm.

So, yeah - this number is bunk.

novembermike5 anni fa

It really isn't. Most of the code is probably going to be uninteresting and you can do 10 lines a minute or more. Some of the code will be more relevant and might take a day for 10 lines. This would just be checking for accuracy though so you could probably just ignore a huge chunk of it.

raldi5 anni fa

I'd say more like ten lines a second, if you're skimming it to look for something. And of course, the work can be parallelized.

lopatin5 anni fa

Is this satire?

raldi5 anni fa

I'm not sure what you mean. What numbers would you call realistic?

bigiain5 anni fa

"With enough eyeballs, all bugs are shallow^h^h^h^h^h^h buried in the critical open source dependancy underpinning the entire internet maintained by that one guy who's holding down a day job and doing it in his spare time." -- with apologies to Linus

(As always, XKCD beat me to this gag: https://xkcd.com/2347/ )

abfan11275 anni fa

10 lines per hour? doesn't that seem painfully slow?

Aachen5 anni fa

It depends on the level of scrutiny. It doesn't seem unreasonable. We review a lot more code per hour (usually C-like code though) but then we're not supposed to lock someone up for murder, we just find basic things like memory corruption. Don't even need to get into the business logic to find bugs that totally break the application, let alone all of it.

When writing Python (I don't have stats about reading), a 1.0 version of a small project took me 1.5 hours and consisted of 183 lines of code, so 2.2 lines per minute. That's much faster than this, but 183 lines is also a ton less complex than understanding the entirety of 180k lines and properly assessing whether it does exactly and only what it's supposed to.

10 lines per hour is probably taken as a lower bound to prove a point, especially because they argue about checking the whole thing (large parts can probably be skipped), but as a standalone statistic I would say it's probably within an order of magnitude from the true value. And for software time estimates that would be an amazing feat :p

techolic5 anni fa

If it means what I think it means - understanding the code - sometimes it takes days to understand just one line of code. Document digging, googling, asking around, fiddling with test cases, reading production log etc.

flavius296635 anni fa

have you ever seen MATLAB code?

ashtonkem5 anni fa

That’s actually a pretty good argument for banning such code from the criminal justice system. The idea that unreadable code is deciding who gets locked up is really worrying.

deathlight5 anni fa

This thread makes me sad because when I was taught and used matlab we had strong pressures to properly comment and document our code to make it legible (if only to our own future selves). It feels almost criminal to not do that in these circumstances.

cozzyd5 anni fa

and depending on which MATLAB version they run it in, it might have completely different results if they haven't tested carefully...

willis9365 anni fa

This is not my experience. Could you share an example?

cozzyd5 anni fa

well, this script, for example, gave me all sorts of fits trying to run on the then-new version of MATLAB I tried it on: http://cpc.cs.qub.ac.uk/summaries/AEJS_v1_0.html

willis9365 anni fa

Ah okay. I thought you meant numerical changes, which would be reason to not trust a language.

Mathworks has broken some legacy support in the past, but they have slowed down on that practice. They used to threaten that dll loading would go away “in a future version of MATLAB” but have since backpedaled on that. My biggest issue is writing code that leverages cool new features (especially timetables) but some people I work with never update their IDE.

ehwhyreally5 anni fa

Yes, You also dont read programs like a book, You generally follow the methods being used, Reading line to line would be like reading a a book with all its pages re arranged.

xirbeosbwo12345 anni fa

Both 0.1 lines an hour and 1000 lines per hour would be equally wrong. That isn't how people would review that sort of code. They would test it and then thoroughly examine any areas of concern that crop up.

I've run into 300-line programs that have taken me a month to figure out because the math was hard and I've run into 100,000 line programs that have taken me a few hours to tear apart.

[deleted]5 anni fa

sudosysgen5 anni fa

10 lines an hour seems quite low.

kenjackson5 anni fa

8 years is a long time. What he then wanted to code review Matlab, and then the compiler that Matlab used, then do some silicon verification...

Six to nine months seems like enough to do a very good code review with some testing. There's a good chance that 75% of that Matlab code doesn't execute for his test.

edgyquant5 anni fa

This is this persons life we’re talking about. No amount of time is too much and it isn’t hurting anyone to let him do a code review.

kenjackson5 anni fa

No. You give a reasonable amount of time. What if he asked for 100 years?

ClumsyPilot5 anni fa

The person's guilt must be proven beyond all reasonable doubt. If you are accusing him there is a range of possible proof. If all you have is based on an unreadable codebase, that takes 100 years to read, that's your (the prosecution's) fuckup. As a jury I would not convict.

I don't want prosecutors sleeping on the job, bringing in fraudsters laymen and psychics to accuse people, etc.

The prosecutor should use a company that can present independent proof that their system actually works.

kenjackson5 anni fa

So what if I say i need to review camera footage of me doing a crime and it will take me 100 years to validate it?

You need to establish reasonable timelines for this or any guilty person will claim any technology used will take 100 years to verify.

There is no reason you can’t analyze a DNA analysis codebase in 6 months. Unless you also need to verify the science.

ClumsyPilot5 anni fa

I am not clear what you are arguing - it there a real problem where people invent fantastical amount of time to review simple evidence?

The claim about the amount of time was not made by the defendant, it was from the company that produced the code.

kenjackson5 anni fa

Thanks for the clarification -- I actually misread it and thought the defendant was making the claim. With it being the company, then I agree that the company has an obligation to make review simple enough that it can be done in a reasonable amount of time (from someone knowledgeable in the field -- I still don't think we need to allow someone 10 years to understand statistics and progreamming before even beginning the code review) or recuse the tool from use in a trial.

PurpleFoxy5 anni fa

We should flip the responsibility. How does the company providing the software prove it is not flawed.

edgyquant5 anni fa

As long as he’s actually reading the code, and still in jail unable to hurt anyone, let him have it. He’ll likely pass before that time is reached and if he’s just wasting time he’ll get bored having to spend hours a day looking at code that he doesn’t care to look at. I get what you’re saying but my point is him wasting time reading code isn’t hurting anyone and is a better use of an inmates time than sitting around.

kenjackson5 anni fa

If he’s in jail doing it then that’s fine.

michaelmrose5 anni fa

For practical purposes its not in the interest of justice to punish people before we have found them guilty as a matter of law and justice. If it really requires 8 years of work to prove whether the tools they are using work or not then they haven't independently validated their work in the first place ergo we have no reason to suppose it works.

We should either pay for multiple people to work on it so we can have the answer in less than 8 years or we shouldn't use it at all.

willis9365 anni fa

The burden of proof does not suddenly fall on the defendant just because one for-profit conviction company can’t back up their product.

kenjackson5 anni fa

Who said they can’t back it up? The defendant claimed he needed to review it. The company presumably has done its own validation.

michaelmrose5 anni fa

There is no reason to presume they did their own validation correctly absent proof of same.

ALittleLight5 anni fa

Hopefully they have to have demonstrated some level of quality to be used as legal evidence in convicting someone. Although, I would hope "code analysis" is an infinitesimal part of the validation with the majority being real world end to end tests. (e.g. we can take 10,000 samples, divide them in two, mix them, then use our tool to pair up the samples with 100% accuracy).

kenjackson5 anni fa

I think this is the crux of the problem. If the defendant gets to decide when validation is sufficient then should they ever say it is sufficient?

willis9365 anni fa

When the jury agrees the evidence is there beyond a reasonable doubt. No one should trust any black box when it comes to criminal prosecution.

kenjackson5 anni fa

So the prosecution can present its case and say they gave the defendant the code for 9 months and here are five other independent reviews. The defense can argue they needed more time. The jury then decides if there is reasonable doubt.

michaelmrose5 anni fa

If the company can't show that the process has been independantly validated it probably should be tossed before the jury hears it.

kenjackson5 anni fa

There are papers that do validation of TrueAllelle. I don't know the product well enough to know if it is the same one used in the article, but there isn't info in the article to know if independent validation was done or not.

willis9365 anni fa

>presumably

>their own validation

That is not how things are proven.

“You don’t need to know how we came to this scientific conclusion.” An appeal to authority doesn’t fly in science and it certainly doesn’t in law.

michaelmrose5 anni fa

Such code should have tests, analysis, and documentation that ought to go a long way towards proving it correct in the time required to transmit data. If they can't produce THAT then they shouldn't use it in a court of law at all.

ChrisRR5 anni fa

This is similar to a case a few years back where someone asked to see the source code of a breathalyser that had found them guilty.

AFAIR, the breathalyser was incorrectly averaging the readings, giving disproportional weight to the first reading.

I don't know if it was enough to rule in their favour, but I'm sure it called the data into question

Edit: Looks like it was a Draeger breathalyser https://www.schneier.com/blog/archives/2009/05/software_prob...

text705 anni fa

A tool designed to find people guilty is biased to find people guilty.

As far as I know it is fairly easy to take a generic dna sequencer meant for healtcare diagnostics, and repurpose it for STR analysis. The only major difference between the healthcare versions and the forensic versions is the software i/o.

sdflhasjd5 anni fa

> A tool designed to find people guilty is biased to find people guilty.

I don't see those particular issues make it biased, just inaccurate - it could go either way.

radu_floricica5 anni fa

And yet somehow whenever you take a closer look at mislabeled product prices, the average is always in favor of the store. And that's far from the only industry.

Complex tools are the product of many thousands of individual decisions taken by humans, humans aware of who's the paying client.

jstanley5 anni fa

> And yet somehow whenever you take a closer look at mislabeled product prices, the average is always in favor of the store.

This could just as easily be selection bias: the errors in favour of the customer are less likely to get reported by customers.

boyesm5 anni fa

> And yet somehow whenever you take a closer look at mislabeled product prices, the average is always in favor of the store.

What is this based on?

sdenton45 anni fa

I had a 'fun' experience along these lines with health insurance and medical bills a couple years ago. I can confirm that in our case at least, /every/ error we found was not in our favor, and took usually about an hour on the phone to get fixed.

The somewhat-less-malicious interpretation is that the companies have a strong incentive to detect + fix errors that cost them money. Meanwhile, consumers are a) non-centralized, uncoordinated, and often unaware of errors, and b) have no way to fix systemic issues that impact them. And the companies therefore have no /real/ incentive to fix systemic problems. It is literally more profitable to fix the bills of the few people who complain, as they still make money on the remainder who don't notice the errors in the first place.

(on edit; exactly what the other comment one subthread over said. :P )

ClumsyPilot5 anni fa

decades of data?

sdflhasjd5 anni fa

I still believe Hanlons razor applies. I've seen products that have serious performance affecting bugs caused by similar mistakes.

ncallaway5 anni fa

I still think—even when applying Hanlon's razor—there's an imbalance in incentives that leads to a weight in favor of the interests of the party paying for the test.

Take the store pricing example. Suppose the store's pricing & labeling process produce an equal number of bugs at checkout in favor of the store and in opposition to the store.

The store is heavily incentivized to detect the errors that are opposed to them. They are much less likely to detect the errors in their favor. Consider the manager that looks at the cash at the end of the day and notices they are $500 short. They likely dig hard to find the root cause of the issue, detect the pricing disparity and correct it. Now consider the manager that is $500 over at the end of the day. They are much more likely to say: "that's weird", shrug their shoulders and move on.

The same applies to forensic tools. Even if they originally produced bugs in both directions, their own internal QA and the market of police officers are likely to work hard to detect bugs that make them less likely to allow them to make an arrest.

The net result is that the tools end up with a bias in one direction, even if the original developers made an equal number of mistakes in both directions.

jfengel5 anni fa

Most store managers get as grumpy about overages as undercounts. They mean that some customer got shortchanged. For $500, it probably means a lot of customers got shortchanged, or something even worse is going on. That makes customers grumpy, and it affects your future.

There are plenty of lazy managers who would sweep it under the rug once. But if it happens more than once, it can become their job on the line. They start looking for who's counting wrong. And if they can't figure that out, they get really worried.

I have no idea about police officers and prosecutors. But store managers care about accuracy of counts, not just profits.

bgirard5 anni fa

When running an experiment and following poor practices (i.e. p-hacking), results that fit the hypothesis will be accepted more readily and negative results will be debugged or re-ran more often.

i.e. The initial error may be randomly distributed. But the follow-up on the error will have a lot of bias.

ClumsyPilot5 anni fa

This is also similar to how Toyota hilled people with control software that would cause car to accelerate randomly. The software audit team concluded that they could not find the bug, but the code was totally unreadable and terrible. They settled.

https://www.nytimes.com/2013/10/26/business/toyota-agrees-to....

Also let's remember that a company in UK was selling fake bomb detectors to Israeli and other militaries, and it took them more than 10 years to notice!

https://www.bbc.co.uk/news/uk-29459896

There needs to be proper scrutiny into these things, I could start some random 'deep learning to find criminals' company tomorrow, and have less regulation than a car mechanic

abandonliberty5 anni fa

Every attempt to replicate this has failed, and other vehicle manufacturers have just as many 'sudden acceleration events'.

https://en.wikipedia.org/wiki/Sudden_unintended_acceleration

Occam's razer points to people just hitting the wrong pedal or people's floor mats getting stuck.

The reason Toyota ate it in the press for this was competitive.

appleflaxen5 anni fa

> The reason Toyota ate it in the press for this was competitive.

can you explain what you mean? I don't understand this sentence.

roryokane5 anni fa

I think they mean this: Toyota “ate it” (an idiom meaning to take losses) in the press by having bad stories written about them. These stories were written for a reason. This reason was not that Toyota deserving those stories, but that Toyota’s competition encouraged those stories.

azernik5 anni fa

From what I've seen of the Toyota case, the thing that Toyota actually got slammed on had nothing to do with control software, but was about the mechanical design of the pedals and the floor mats.

dathinab5 anni fa

I wouldn't be surprised if "incorrectly averaging" and similar are very common software errors.

The reasons are manifold, including:

- Normalized values need to be averaged differently the absolute values.

- Floating point has limited precision, even just correctly summing/multiplying numbers need special care if you care about correctness. Results can, in the worst case, be of by a massive amount.

Often you don't need to care about it so it's not uncommon for especially junior programmers to be not so aware about it.

I mean in the last 3 years of working as a professional software engineer/developer I didn't need any of this at all, but once I do I know what to look out for.

throw28385 anni fa

Draeger requires calibration prior each use for temperature and humidity. It is easy to get it thrown out as evidence.

lwigo5 anni fa

When the officer shows up and swears under oath that he "calibrated it that morning" it's not so easy. :shrug:

OnACoffeeBreak5 anni fa

Logs of calibration procedures along with calibration results should be stored on the device and auditable along with test results.

jdironman5 anni fa

Temps vary throughout the day.

lwigo5 anni fa

A rural small town judge probably doesn't have the best insight into that, just that Corporal Bubba said he calibrated it so it must've been right.

I know, this is exceedingly cynical.

klyrs5 anni fa

Your "Corporal Bubba" isn't just cynical, it leans heavily upon a harmful stereotype that folks in small towns are uneducated simpletons. Stop the polarization, please

owl575 anni fa

Why wouldn't they be? I went to university with people from all over Russia. They don't tend to return to their home towns after getting educated. Does this work differently in America?

zymhan5 anni fa

> Does this work differently in America?

No, not really.

[deleted]5 anni fa

edbob5 anni fa

Not all poor people are stupid.

owl575 anni fa

(1)

zymhan5 anni fa

I'd love it if more cops had college degrees in the US, but they don't. Ignoring the problem doesn't make it go away.

hluska5 anni fa

Sadly, when you consider the available evidence, Corporal Bubba isn’t too far from the truth. I say that as someone whose Dad is a retired police officer and who spent most of his youth in a very small town with sub-1200 people.

michaelmrose5 anni fa

Rural areas are subject to brain drain where the best and the brightest disproportionately leave for more urban settings where more money is available to be earned. Living in a rural setting is something one chooses not an immutable fact like skin color and by and large the harmful stereotype is spot on.

pvitz5 anni fa

I have worked with MATLAB code with 20,000 lines of code. Only over the past years, OOP and unit-testing has become properly available and usable. My guess is that this 170,000 lines are written in the old procedural way (also for performance reasons) and are full of bugs, also thanks to the lack of supporting tools.

Most likely, this grew out of a research prototype that just worked too well to be reimplemented in a proper production environment.

fatnoah5 anni fa

I wonder if bug tracker and other reports would be part of discovery in this case.

pvitz5 anni fa

Equally interesting is in my opinion who should do the review. Mathworks' own consulting service is probably the best to do so, but I wonder if they would objectively work against one of their own customers.

fatnoah5 anni fa

Yeah, any technical expert in a trial concerns me. I was an alternate juror (meaning I had to sit through the trial but was not allowed to take part in any deliberations) in a trial that involved the testimony of a computer "expert". The expert's testimony was 100% true and appeared to definitely prove X to someone who knew nothing about the subject matter. It was things analogous to saying the system was secure because it had a security chip.

There were 1,000,000 questions I wished had been asked.

mattkrause5 anni fa

Matlab has had classes (both types!) for ages. The unit-testing stuff dates back to at least 2013, and there were toolboxes to do similar things even before that.

The language certainly has some warts, but IMO, the bigger problem is that it's usually learned/used in contexts that focus on code quality: the goal is the resulting number or plot rather than the software that generates them.

pvitz5 anni fa

That's why I wrote "available and usable". What Mathworks called unit-testing back then was laughable. It only got interesting in the past two-three years. Same with OOP and the features added in the past years (e.g. type hinting etc.) You can see that Mathworks themselves preferred not to use OOP in their own toolboxes. Parts of the Financial toolbox use it (e.g. the SDE stuff) or the datafeed toolbox, but much stuff is still written the old way. Another example would be Appdesigner as the new preferred way of writing GUIs with OOP. It is still much slower than the old GUIDE functions.

ineedasername5 anni fa

If ever there was something that should be fully transparent it is the mechanisms by which a person might be found guilty of a crime. The defendant shouldn't even have had to fight for this. It should be a fundamental cornerstone of criminal prosecutions.

frongpik5 anni fa

Nothing is going to change until this software convicts a 10M net worth dude for something he didn't do.

gorgoiler5 anni fa

This is pretty depressing unless you can elaborate on your take.

Yes, the system is stacked against the poor, but there are people fighting that. If they are fighting and failing then we need to know why.

If they aren’t fighting at all then (in part, but it’s still a significant part I’m afraid) it’s because of attitudes like this.

JacobSuperslav5 anni fa

I think the point is that they are fighting but they are losing. Some 1st world countries are really bad at seemingly basic human rights for the poor. Doesn't mean they shouldn't fight, just an observation.

meowface5 anni fa

These cases also often come up with drug and alcohol detection tests, and as John Oliver points out in https://www.youtube.com/watch?v=1f2iawp0y5Y, software used to select jurors.

All of these companies claim that their source code is valuable intellectual property and that disclosing it can hurt their business. Even if this were true, when you're providing something that can be a significant factor in someone being imprisoned or executed, when creating the business you should accept that you're providing a public service that needs to be publicly accountable.

If it's not open source, at the very least there should be a requirement that software code and hardware designs must be provided on-demand to experts in court cases (with a non-disclosure clause to mitigate leaks and corporate espionage etc.).

bargl5 anni fa

Without jumping on the conspiracy bandwagon, I'd also like to see this applied to voting software. I know it's a hot topic, and I'm honestly not trying to get political.

Software that is critical to our fundamental human rights, and is being used by our government should be open source, or at least audited by a group of people who sign Non-competes/NDA and can't go work for competitors, or with some other mechanism to protect IP that I can't think of.

jedberg5 anni fa

The beauty of voting software is that you don't have to verify the code if you hold the vote correctly. If the software provides a voter verifiable paper trail, the voter can verify their vote before turning it in.

The county can then verify the software by manually counting a random selection of paper votes to see if they match the software. If they do, then the software is correct, otherwise it is not. You then have a full by-hand recount and tell the vendor to fix their software.

ynfnehf5 anni fa

I feel very strongly that all votes in all important elections should be counted by hand, and be open for anyone to observe the process (within reasonable limits on disruptive behavior).

Not because of the possibility of voting machines being hacked, but because it is important for the public to have trust in the system. It is difficult to trust a system you do not understand, and only a very small minority is ever going to be able to audit voting software.

(I'm not American, so this is in no way a comment on your current predicament.)

jjeaff5 anni fa

I agree, for my own piece of mind. But I am also certain that it would have made no difference in our current predicament with a third of the country thinking the election was stolen.

It has been shown to us time and time again that no actual evidence is required to get people to believe what they want to believe.

And the more technical the evidence (i.e. source code), the less helpful.

bargl5 anni fa

>But I am also certain that it would have made no difference in our current predicament with a third of the country thinking the election was stolen.

It would have changed some peoples minds I don't know if the change would have been a few thousand or 10s of millions. I can't say if it would have a dent in the 1/3 of people or not. I can't predict that. It would have helped me with my own peace of mind. And frank I think it's overall the right thing for us to do.

>And the more technical the evidence (i.e. source code), the less helpful.

Disinformation is powerful, I'm not suggesting this alone would fix that. I disagree that more technical evidence is harmful. Global warming is benefiting from transparency and evidence. It takes generations to change political will not years. The evidence there has shifted our whole economy, just maybe not fast enough.

There will always, always be deniers. Global warming, flat earth, vaccinations, etc. Evidence _helps_ battle deniers in these areas, but it takes generations for these ideas to become mainstream and the deniers to go from 99% of people to 2% of people.

Also, 2% of people think the earth is flat? Holy crap. https://www.sciencealert.com/one-third-millennials-believe-f...

michaelmrose5 anni fa

Why would people who weren't convinced by reputable evidence in the first place be convinced by slightly better evidence that is only better in a technical hard to express and prove fashion. This is especially true when the people doubting are the least educated and least intelligent.

It's like saying that better proof of evolution would convince some portion of creationists. That's just not how misinformation works.

Misinformation works by targeting vulnerable parties with misinformation that aligns with their existing vulnerabilities and beliefs in order to power relevant action with long stored and fruitful sources of hate, bias, and scorn in a fashion that bypasses the brain and goes right for the gut.

Like 30% in America believe in a young earth that is thousands not billions of years old.

If Bob is a scientist of some sort and presenting interesting scientific work to the community and incidentally advising the government on environmental policy that will harm some business and you want to crush support for this by playing on existing biases with this group you advertise to the young earth crowd about how bob is anti God and see if you can tie bob to as many negative things they already dislike as you can.

You aren't fighting an intellectual battle to set their ideas on bob let alone deeper ideas you are fighting an emotional battle to galvanize existing deeply held beliefs to obtain useful action like calling up and yelling at their congressman or voting.

In that context asking Bob to present a better case is laughable. The relevant parties never engaged their brain in the first place.

breischl5 anni fa

Seems like both you and your parent comment are talking about software audits & auditors. I don't know if that exists in this form, but it seems reasonable that if you can get a security audit you should be able to get a correctness audit. And of course those auditors would be under some heavy-duty NDAs, given the nature of the work.

ineedasername5 anni fa

I completely agree, but you can have an auditable voting record in an election without relying on software integrity. That's not really the case, from the defendant's point of view, in a trial that relies on probabilities implemented in the software. In the case of voting, it is software-assisted. In a case like this article, it is software driven.

acct7765 anni fa

It shouldn't be considered conspiracy theory that, technically speaking, many things in our nation are an insecure joke, including our system of voting.

alistairSH5 anni fa

...disclosing it can hurt their business...

And there's an entire body of law based around IP which they can use to protect their business, just like everybody else.

zymhan5 anni fa

Their business interests are also of minuscule importance compared to the impact to society these tools have.

zupreme5 anni fa

What is unfortunate is that it took going to appeal to force the judge to allow the code review at all.

Without, at minimum, an independent review (and preferably open source code) the software and lab processes being used constitute an inscrutable "black box" process within which any judgment can be made, for any conceivable reason, with life-changing effects for the defendant (and for the victims of a crime if, for example, a rapist or murderer is set free by a non-match decision).

One could even say that unreviewable code here falls under the umbrella of "secret evidence", which much of the world already knows can be easily misused and/or misapplied at the whim of the court.

vmception5 anni fa

People sometimes ask me what my “number” is, like how much net worth or “money” I want, what would I do with it

I say “I want to be able to afford appeals court where my rights matter”

Infinite appeals court!

Most people plea out, cant make bail, dont have counsel buddy buddy with the judge enough to get you bail, and lose the ability to keep good counsel for more and more motions and appeals

I want that, there is almost no pride in American rights if you cant afford them. People tie their whole identity to a system they arent even part of

ineedasername5 anni fa

Or just make the entire system based on a sort of "public defender" model. As it stands, a person accused of a crime and then found innocent has still been punished without even being found guilty due to enormous legal bills. It is a highly asymmetric power structure for anyone who isn't wealth: the prosecutors have massively more resources than the average person to call upon. Alternatively, when prosecuting the wealthy, that asymmetry is reversed, which might be equally problematic.

ashtonbaker5 anni fa

I've occasionally mused that funding for a legal case should go into a pool, which is divided equally between both sides. That way, any money thrown at a case is, at least in theory, aligned with the incentive of "getting at the truth" rather than overwhelming someone with a valid case, but lesser resources. I don't mind someone raising the profile of a case by adding funding, especially if the stakes are high for one party, but it shouldn't be at the detriment of justice.

It's kind of a half-baked idea, and I'm sure it's not totally watertight but the existing problems you've mentioned really bother me.

lotsofpulp5 anni fa

I have the same line of reasoning when people talk about having enough to feel secure. Even simple civil legal matters cost in the tens of thousands of dollars easy.

And the system works so that you’re either rich enough to be able to defend yourself and the money spent doesn’t affect you, you’re poor enough that you have nothing to lose, or you’re in the middle, busy trying to get from poor to rich, but you are vulnerable to losing it all because you don’t have enough to protect it, but you have enough that it’s worth for someone else to try and take it.

cyberlurker5 anni fa

Yes, that middle zone (that most of us probably live in) is terrifying. And it’s why we get conned into so many different types of insurance.

“One disaster and all that progress is gone.”

hutzlibu5 anni fa

"Yes, that middle zone (that most of us probably live in) is terrifying"

More terrifying than the bottom where you got nothing to loose? I doubt it. Otherwise, why be afraid of it?

vmception5 anni fa

I would say just because of the energy and sacrifices used

At the bottom you don't have to pretend that the circumstances will improve, and there is some freedom associated with some approaches to that. Careers don't need to have continuity, I know many people in hospitality and service industry whose vacation policy is saving and quitting one restuarant, travelling, and getting another job at a different restuarant when they get back. Sure other approaches have lots of energy used on finding food and shelter that day, and service and hospitality work is not necessarily at the bottom, my post isn't about those approaches and dilemmas.

People in the distinct category of "professional" careers, not my term, don't feel like they have that freedom to have any timegaps and are resigned to earning small periods of time off, and often times that is true.

hutzlibu5 anni fa

"At the bottom you don't have to pretend that the circumstances will improve"

Well, sorry, but I would also say, you don't know what you are talking about.

First of all, there is no bottom at the bottom - you can always fall deeper, until there is no more escape than suicide. I know people who did.

What you maybe mean, are people who don't care abobut materialism and live with little to no money by their choice. I lived with those people for quite some time and it was fun.

When you are young and healthy and on your own, you don't really have to worry about a lot of things. I worried about my backpack with my laptop and that was it. I slept in a tent or under the stars or wherever. When the money was gone, there were always places or ways to get food. Work a little, travel a little. Easygoing.

But now I have a family. Now I cannot not have money.

michaelmrose5 anni fa

You couldn't possibly be more wrong. Have you ever not been able to afford medication that you know you needed to breath and gone to sleep to have a nightmare about being attacked and suffocated and woke up to find it was real save for the fact that it was your own body?

Ever wondered if you could afford to keep a pet from dying due to being able to afford the care?

Ever wondered if losing your home was going to stress your marriage so much that it might splinter?

The only people who think the bottom is less stressful have never been there.

cyberlurker5 anni fa

To be clear, I don’t think the bottom is any less terrifying. It’s just the anxiety of being poor and powerless is now replaced by reality.

roywiggins5 anni fa

Appeals? Only ~3% of people charged even go to trial, the rest plead out.

Just giving everyone a substantive right to trial would amount to a revolution.

Gustomaximus5 anni fa

I've wondered what would happen if jail culture expected people to go to trial.

So similar to how snitches are targeted, if criminals in jail start violently targeting people that didn't go to trial they might be able to tear down the system...maybe?

And to be clear this is a loose idea as I don't really know the system but it seems courts would be so flooded if everyone took this route. Prosecuters would have to stop with these rediculous threats of trial jail time vs plea deal as jails would become too full. And authorities would be forced to stop charging people for smaller crimes as they simply couldn't handle the case load in courts.

Even getting juries might be tough and start the rest of society pushing back if people were regularly being called for jury duty and disrupting their own lives.

...or something else but this would be an interesting 'fight back' by criminals.

edbob5 anni fa

George Bush introduced a PREA/Safe Prisons Program that has radically changed prison culture. Prisons are far, far safer than ever before and getting even safer as time goes on. (Since I know about Texas in particular), Texas has spent millions per prison to install hundreds of HD cameras covering nearly every square inch of ground (outside of individual cells and showers). When someone commits serious violence, they are segregated for at least 5-10 years, depending on the severity.

There are still some gang-controlled areas, but they are an exception now rather than the rule. The nanny state is firmly in control of most of the prisons.

vmception5 anni fa

Yeah. But just giving myself that right first, forever.

criddell5 anni fa

In Canada you can't even get the breathalyzer maintenance records:

https://www.canadianlawyermag.com/news/general/maintenance-r...

undersuit5 anni fa

Guess you should just start doing these crucial DNA tests against some sort of panel of tests instead of just one lab. It would be a shame for the quality of the code in your one test to convict an innocent or free the guilty.

hastradamus5 anni fa

The co-founder of the company, Mark Perlin, is said to have argued against source code analysis by claiming that the program, consisting of 170,000 lines of MATLAB code, is so dense it would take eight and a half years to review at a rate of ten lines an hour.

This is hilarious. As if you need to read every damn line and you can’t skip blank lines? You can skip whole files that aren’t relevant. Weak excuse

heavyset_go5 anni fa

"It could take too long :(" is an interesting excuse not to examine the evidence that could put someone behind bars for the rest of their life.

jariel5 anni fa

The statement is odd, at the same time, it's not outrageous for him to make in the sense that - lines of code notwithstanding - the underlying science i.e. the application of the product is the thing in question.

It's a pretty interesting case.

At least the core nature of the algorithm should be made public if we're going to use it for public inquisition.

astura5 anni fa

This is a stupid "argument", I've personally participated in an audit of a program that has 5-6x as many lines of code.

sudosysgen5 anni fa

And of course, as soon as you find an issue (which you probably will given that it's 170k lines of MATLAB), you can stop then and there.

steerablesafe5 anni fa

Are forensic labs often get blind tested? If there is a bias for guilty cases then it should turn out in those blind tests. Source code is a red herring here, there should be independent evaluations of forensic laboratories/methodologies/etc... regardless of software source code availability. Maybe these checks are already in place, I genuinely don't know.

scott005 anni fa

I don't know the answer to your question, but blind testing is complementary to (not a substitute for) source code review.

It's very common for software to work correctly a high percentage of the time, but fail on rare input data. If, say, the software works correctly 999,999 times out of a million, you're going to be very unlikely to discover that error by throwing random samples at it, especially if you need a physical process (ie, drawing blood) in order to generate a test case.

On the other hand, once you have a known failing case (as you would if the defendant knows the result must be in error because he didn't commit the crime), it's often fairly straightforward to identify the error by reviewing the source and/or using a debugger to examine the progress of the algorithm.

fritzo5 anni fa

Agreed, blind testing is important for statistical correctness and code review is important to avoid adversarial backdoors like dieselgate.

cmpb5 anni fa

>Source code is a red herring here

If there were a way to ensure that the test suite applied to these forensic labs was all-encompassing w.r.t. the genetic variables at play, then maybe. But that sounds impossible. What if there's a coding error that causes the software to operate differently/incorrectly only for people with a certain (rare) genetic abnormality?

For what it's worth, I'm totally unversed in genetics, though I have a great deal of experience writing software tests (and seeing them come up short in adequately modelling real-world data).

WaitWaitWha5 anni fa

Not in my experience. Most labs of various types supposed to get certification, but these certifications primarily about chain of custody, operation protocols, record keeping, and such. It has little to do with the veracity of their conclusions.

hirundo5 anni fa

If an accused person has the right to see the source code that produced evidence against them, is it a violation of their rights for the source code to be obfuscated, or even just so spaghettified that not even an expert can understand it?

I kinda think that should be a violation. But deciding whether a particular piece of code is so bad is so subjective that I'm not sure on how you'd make a legal standard out of it. Maybe start with "the linter found a ratio of warnings to lines > X%" or some such.

Having a legal standard of code coherence/incoherence might help filter pull requests. "This PR cannot be merged to this project because it is configured to reject legally incoherent code."

As code becomes more complex it may become more meaningful to have access to the test suite, and to challenge the evidence if the tests are inadequate to demonstrate the correct code behavior.

londons_explore5 anni fa

You don't need a legal standard. You just need to put doubt into the minds of a jury. You can get an expert to stand up and say "I'm an expert in computers, and I couldn't understand how this DNA test works. I think it's likely there are mistakes in it that neither I nor the people that made it have discovered".

Spivak5 anni fa

“This system is an absolute mess that is impossible audit and so there is no way that the company using could find bugs in it either.”

[deleted]5 anni fa

ClumsyPilot5 anni fa

There was a court case against Toyota, where one of their cars would randomly accelerate and cause crashes. The software audit said the code was unreadable and nothing could be proven. They settled.

https://www.nytimes.com/2013/10/26/business/toyota-agrees-to....

Jakobeha5 anni fa

In the case where code is used to convict or acquit someone, I think it should be a well-tested and established program: generally something with the software quality of Linux, or in this case, whatever DNA testing kit is being used by scientists in top-ranking universities.

We could also use formal verification based on well-established axioms. For example, maybe we could "prove" that the DNA kit reports accurate results as long as the samples it's given are processed correctly.

WaitWaitWha5 anni fa

During discovery, the opposing lawyer can raise an alarm to the court of such obfuscation. There have been many cases where such behavior cost cases going against the party.

rland5 anni fa

These companies are disgusting. They peddle black box "models," that essentially ride the good reputation of DNA as infallible (which it is most certainly not) to get convictions on dubious or no evidence.

The way it works is that if there is a sample from a crime scene, they send it to these guys and they analyze it with their software to detect "statistical" DNA from the sample. These samples are the ones that are too crappy to actually make a definitive match -- they are a statistical match. So you say "I think Jim, Bob, and Alice were on scene," and it says "10% likelihood Jim DNA, 5% likelihood Bob DNA, 45% Alice DNA." Do you think it ever says "99% no DNA" in the sample?

It's basically Theranos, except instead of wasting $50 on a shitty blood test you get life in prison.

Ostensibly, it searches the entire DNA database for matches, and only returns a positive result if there's a positive match.

But it's a statistical model, using inputs that are crappy at best (because if it was an actual DNA match, they would send it off to in house forensics who would be able to do PCR...) and which includes inputs from circumstantial evidence as priors. Like we believe Alice was at the scene therefore if you find any statistical likelihood that this is Alice's DNA boost that.

They often run the model multiple times in a row, and use the result that the DA likes the most to enter into evidence. This is because the models return different results each time -- of course they'd say, iTs StAtiStIcaL, so they can do that...

And the source code is completely impenetrable. They argue that it's a "trade secret" that jeopardizes their ability to make future profits, so it cannot be open-sourced. These guys could have a model that just says "what percentage should the thing read, Señor D.A.?" The entire product is a sham. And because it's 170k LOC, no one has the time or the qualifications (Judges/Attorneys reading source code? Yeah right!) to review it, even if it were open source.

Pure quackery, and often times, decades-long sentences or life in prison for the defendant. These companies are pure filth worthy of the lowest revulsion. It's a wonder any convictions happen at all because of this stuff, but jurors have very inaccurate conceptions of forensic science, thanks to shit like CSI, Law and Order, etc. These companies happily play into that image and people really believe this stuff works.

jrm51005 anni fa

Issues with source code access aside, your description is mostly wrong. These programs take a DNA profile as input- it's just that the DNA profile is mixed (i.e. from multiple people). It reporting no DNA would be nonsensical. Figuring out exactly how many people are in a mixture isn't quite nailed down statistically (last I knew of), but it's usually pretty clear for up to 4 or so people.

Yes, you could run different models and get different probabilities. For example, the likelihood that the sample is a mixture of the suspect, the victim, and some unknown person vs victim and two unknown people compared to saying the victim isn't in the sample. However, the specification of those models is part of the trial process.

And the output probabilities (at least when being used to determine guilt) are usually quite high, orders of magnitude higher than 90% or even 99.99%.

My point is that the science behind these calculations is well developed- validation studies get published all the time. Whether or not the specific software has errors (or isn't coded exactly as modeled) is an entirely different matter, but it still isn't all that likely. All of these cases rely on expert witnesses anyway- it's not the prosecutor pressing some buttons and printing a report.

There is far more concerning quackery that gets used in forensics- bite marks, hair matching, etc.

[deleted]5 anni fa

qwertox5 anni fa

Here in Germany we have somewhat similar cases, but where the accusation is way less damaging than the case of this article, in which a false positive would have the drastic result of being labeled a murderer.

The cases are related to new speeding cameras which work with laser, where the defendants are complaining that these new devices are black boxes, and that they demand access to the raw data which these devices process. The problem is that these devices discard the raw data after having processed it and come to a conclusion that the driver was or was not speeding.

The devices in question are Traffistar S350 from Jenoptik and PoliScan SM1 from Vitronic.

There were discussions about a required software update which retains all this data, but apparently the devices lack the storage capability to do so. The National Metrology Institute of Germany (Physikalisch-Technische Bundesanstalt (PTB)) responded to this, that they would not re-certify these devices with updated software because from their point of view they work "as specified".

nolok5 anni fa

Now all you need is the proper amount of collusion/corruption between the certifying agency and the manufacturer to have a magic box that does whatever the one paying the bills want. Might seem far fetched in a developped country, until you read about the Boeing/FAA thing happened.

josefx5 anni fa

As far as I could find the courts basically decided that there has to be a way to examine how a result was reached if there was any doubt about it. That the PTB still allowed their use didn't change that and if a case got to court the results could get thrown out. The PTB probably doesn't care because only a small percentage of speeding cases end up in court.

Kinrany5 anni fa

Surely there aren't so many speeders that storing a few seconds per violation would add up to a significant amount.

spaetzleesser5 anni fa

There is so much junk science going on in forensics that it would be great to require everything to be open sourced. Same for voting machines and anything police in general is using (predictive policing is pretty scary). There is way too much stuff hidden and can be challenged only if you have very deep pockets.

egberts5 anni fa

Well, DNA matching is crapshoot and hazardous toward the innocents. We can still find unrelated folks with partial match by the virtue of segmentation.

https://dna-explained.com/2017/01/19/concepts-segment-size-l...

vlovich1235 anni fa

I'm curious. I've always been frustrated by this "closed" business model in the legal system. I feel like the entire process & details should be detailed in the open (code, methodology, controls, etc). Of course the counterpoint is that it makes it easier to copy this business & undercut all the time & energy spent on building it (copying is easier). Is that the only reason? I feel like open kimono is a critically important concept for anything related to the legal system because of how any perversion removes its legitimacy. If it really is that prohibitive to run a profitable business in this space, is there open standards that can be enforced (e.g. "this is the core algorithm that is approved" & businesses must get regular audits to continue to be used & any failed audit causes a reexamining of any court cases you were involved in the past year?). That's less ideal because then who audits the auditors but maybe at least it's an acceptable middle ground from where we are?

In general I've been extremely frustrated how regularly & consistently this entire industry keeps everything secretive & trust-based despite consistent examples of how insufficient trust is for this field & how devastating the results are when that trust is violated.

ABeeSea5 anni fa

> Mark Perlin, is said to have argued against source code analysis by claiming that the program, consisting of 170,000 lines of MATLAB code, is so dense it would take eight and a half years to review at a rate of ten lines an hour.

So it’s definitely riddled with bugs. And I can’t imagine that much matlab code following rigorous software engineering practices.

bitexploder5 anni fa

I have done a lot of source code review in my time. For security assessments. Our general rule of thumb was about 10k lines per week that we can really get deep on. 10 lines an hour would only be for the most dense code and critical path stuff. They will need a reviewer that knows the domain (DNA), but it’s perfectly reasonable to review that code on a weeks/months time scale, definitely not years.

Scandiravian5 anni fa

My exact thoughts. This sounds like a classic example of launching a prototype created by domain specialists (biostatisticians and bioinformaticians) as production software and skipping on the expensive stuff, like sound development practices

gww5 anni fa

There is so little emphasis in production software development in bioinformatics and biostatistics. Despite a lot of groups open sourcing their code it's is nearly unusable and not reproducible due to hard coding, ignoring edge cases, and dumping the majority of the code on a single giant R or python function.

natechols5 anni fa

It's a real problem, and I've been struggling with it for two decades, but even so I am legitimately impressed (and not in a good way) if they have 170,000 lines of Matlab code in their production software. That takes a really special combination of productivity and cluelessness, even for academic specialists. Regardless of the facts of this particular case, it should be absolutely horrifying that anyone's freedom is left up to a gigantic pile of unaudited Matlab code. (That said I am almost certain he added some zeros to the number, I have a hard time imagining what they're doing that could be that complex.)

Scandiravian5 anni fa

Choosing MATLAB as a language for software that could potentially lead to people dying (in areas where they still have the death penalty) is a gigantic red-flag

I don't know how many job postings ask for a software engineer who knows MATLAB, but I can't recall any

Scandiravian5 anni fa

That's my experience as well. My master's is in bioinformatics and I worked for several years in biotech.

I got frustrated because my concerns that my team's development practices were causing issues on a regular basis, were ignored. I was continuously able to predict what issues we would run into, but no-one seemed to care - I even had a manager tell me, that it was good that our software was buggy, since the client would continue paying us to fix it

I've since left the biotech industry. There's a limit to how many times I want to run my head against that particular wall

isolli5 anni fa

It also sounds exactly like Ferguson's Imperial College epidemiology model that apparently compelled politicians into imposing hard lockdowns (and was likely wrong by at least an order of magnitude):

- "a single 15k line C file that had been worked on for a decade" [0]

- code review of the model: [1]

- corresponding HN discussion: [2] (including sad appeals to authority: you're not an epidemiologist)

- other HN discussion [3] (including ridiculously blaming programmers for making C++ available to non-programmers)

[0] https://twitter.com/ID_AA_Carmack/status/1254872369556074496

[1] https://lockdownsceptics.org/code-review-of-fergusons-model/

[2] https://news.ycombinator.com/item?id=23093944

[3] https://news.ycombinator.com/item?id=23222338

antattack5 anni fa

For reference, whole Tesla Autopilot is 'few hundred thousand' loc.

https://youtu.be/YAtLTLiqNwg?t=947

partyboat15865 anni fa

>170,000 lines of MATLAB code

This is a deep problem. Many scientists don't understand software engineering and more and more need to write bigger and bigger programs. And most of the time they don't open source their code.

Open source science.

moron4hire5 anni fa

That's like an accountant, accused of embezalling, refusing to hand over the ledgers because "there are just too many records to go through". Yeah-no, that's kind of the whole point. We want to find out what you've been hiding in that wall of paper.

syshum5 anni fa

I also cant image anyone thinking that would be a winning legal argument... "This software is too complex to look at so just trust us" Really... that is what they went with...

delecti5 anni fa

Yeah, that's one of those "defenses" that feels like a confession.

avs7335 anni fa

unfortunately, the technical illiteracy of much of the US judiciary has made this effective.

bryanrasmussen5 anni fa

and surprisingly all of the code is of equal importance so you really need to review each line sequentially! Instead of finding stuff that you think is most likely to relate to what you're trying to figure out and debug from there. Wow I would like to see this marvel of engineering myself!

Scandiravian5 anni fa

Kind of makes me wonder: if the argument is that the code is too complex to review and understand, does that mean the company is not doing code reviews themselves?

bryanrasmussen5 anni fa

this has actually only happened to me a couple times but it has happened - someone tells me Bryan, go look at the code X did in Y, figure out if we refactor. X would then tell me - that code is really complicated is full of algorithms! I go and look at the code realize that for what it is trying to do can be cut down from 10 pages of printed code to less than 1 and it was incredibly simple what actually needed to be done.

In short when someone tells me the stuff is too complicated because too clever and advanced I tend to disbelieve them.

that said I have of course written my too complicated stuff lots of times, but if asked I don't say it was because I'm clever.

names anonymized so as to not accidentally hurt anyone's feelings.

on edit: actually one time the code was clever but not especially difficult, they just used the algorithms line because they didn't want anyone messing with their stuff.

Scandiravian5 anni fa

I think there's a bias towards judging things to be "clever" if they're hard to understand

It's a cliche to have a "what idiot wrote this" outburst, then realise it's your own code, because most of us have written our fair share of "clever" code

My boss explicitly stated that he doesn't want to see any "clever" or "smart" code in our product - write code based on simple fundamentals, benchmark before deciding to optimise, and be respectful in your reviews

I like my boss a lot

michaelmior5 anni fa

If the code was developed in less than 8.5 years, it seems like the answer must be yes based on their previous claims.

gambiting5 anni fa

Well I mean if something requires 8 man years of work you can do it in a year with 8 people. Not always, but usually.

markwkw5 anni fa

I would claim that if something requires 8 man years, that it will most definitely take more than a year to develop with 8 people.

Communication takes time, coordination takes time, there is an incremental cost to each news person added to a team. From experience, perhaps with 2-3 people who happen to gel well together you may get close to proportional scaling of output, but with 8 it’s really unlikely in the real world.

agravier5 anni fa

On the contrary, such linear scaling would be quite exceptional. I'm speaking from experience but you don't need to trust me; I invite you read any book on software engineering management, starting from The Mythical Man-Month by Brooks.

bryanrasmussen5 anni fa

the mythical man month was first published in 1975, I think the typical applications programmers work on today have changed significantly since then and encompass many different disciplines (to be thought professional) - so many disciplines that one developer is likely to be the master of all. It is true that there is a communication overhead to adding more people so it will not scale linearly, but if a single developer has taken 8 years to build something in our era it seems likely that having 8 people might get it done say 1 and a half to two years.

watwut5 anni fa

I would find it extremely unlikely to work that way.

watwut5 anni fa

You can do code review on every single commit while not being able to make overall analysis of the massive codebase.

Scandiravian5 anni fa

I completely agree, but if the reviewer isn't able to (with some amount of accuracy) predict the impact the committed code will have on overall behaviour, then there's very limited value on doing the review in the first place

watwut5 anni fa

In any larger project, the reviewer are not able to predict impact from reading commit.

More importantly, typical reviewer have only small partia area where he has good idea about which commit is bad idea. He however does not understand whole codebase.

Knowing what the whole does and knowing what my module does are two different things.

Scandiravian5 anni fa

I again completely agree with you

Looking back at my reply, I think I should have added a bit of background to clarify my comment

My master's degree is in bioinformatics and I worked in the biotech industry until about a year ago. I mainly worked as a consultant for top 20 pharma companies, but also did work on different in-house projects and in academia

From my experience in the industry, I find it very unlikely that the software mentioned in the article is structured in a modular way. I've yet to see good software practices outside one or two academic projects. Most pharma companies still use copying and renaming folders as version control. Naturally I'm sceptical of any code coming from the biotech industry

On top of that, it's written in MATLAB. I have only ever seen this used by statisticians and university researchers, never by software engineers

I'm therefore willing to bet, that when the reviewers open the source code, they'll find unstructured mess of spaghetti code, that has never been refactored, reviewed or tested

So yes - I agree in all your points, but I find it unlikely that they're being applied to this particular project

WaitWaitWha5 anni fa

The tone of this thread is leaning to "forensics evil, government lackeys out to get the little guy".

In the USA, all sides can call and rely on their own forensics. There is no government mandated & approved single-source-of-truth (with some minor exceptions).

> Forensic labs and companies are expert witnesses with black box processes and the incentive to protect the authority of their profession.

This statement is at minimum an extreme generalization. Forensic field is a very large field, with many government and private "forensicators".

There are no "black box processes". The very word "forensic" is based on presenting in full view, front of the fact finders (jury & judges usually).

How deep this gets dug into depends on the fact finders and attorneys.

In all court cases the forensic examiner can be called to demonstrate with extreme nuance how they performed the procedures. Any tools used are can be requested to be examined, including software. Vendors that I have worked with all have experts on staff specifically to appear in court and detail the inner workings of their tools.

Not only tools, processes, and environments, but the examiner can be drilled on their experience, education, degrees, previous cases, failures, etc. They do call it "voir dire" for a reason.

This does not negate the fact that there are some bad apples, bad prosecutors, and bad judges.

inetknght5 anni fa

> There are no "black box processes". The very word "forensic" is based on presenting in full view, front of the fact finders (jury & judges usually).

You're wrong.

When the defendant can't afford an expert witness then the defendant is literally shut out from being able to understand the technology involved.

Good luck proving to a jury made up of non-technical people how thread timing problems cause math problems in DNA analysis without an expert witness.

Good luck getting an expert witness to testify to that when the source code is hidden behind a black box "company trade secret".

ncallaway5 anni fa

> In the USA, all sides can call and rely on their own forensics

Ah, not quite.

More accurately: "all sides can call and rely on their own forensics _if they can afford it_"

In this country we _say_ that everyone has the right to an attorney, but that doesn't extend to expert witnesses.

This means forensic evidence becomes a _phenomenal_ tool for targeting the little guy, or the lower classes, while conveniently providing hooks for the more wealthy defendants to escape the system.

0xfffafaCrash5 anni fa

Sorry but forensic ”expertise” in the court room has always been laughable. In ”full view of the judge and jury” means absolutely nothing when these people are completely scientifically illiterate and defer to the so called expert who has a financial incentive to help the prosecutor. There’s an entire industry of people who make their livings by reliably testifying to the guilt of defendants and then being compensated for their “expertise.” They just need to be paraded as experts and how would a layperson be expected to know any better? The quality of defense in the American justice system has everything to do with who can afford to pay for it. I could cite a hundred sources but here’s a small sample...

[0] https://www.sciencemag.org/news/2016/03/reversing-legacy-jun...

[1] https://pubmed.ncbi.nlm.nih.gov/30447642/

[2] https://theconversation.com/how-corruption-in-forensic-scien...

[3] https://www.newyorker.com/magazine/2009/09/07/trial-by-fire

tal8d5 anni fa

> There are no "black box processes".

Do you know how much of the fingerprint match process is left up to the judgement of the examiner? You can't get much more "black box" than another person's brain. Last time I checked (several years ago), tool mark analysis was still without objective foundation. Also... bitemark analysis - that was a thing.

Falling35 anni fa

> In the USA, all sides can call and rely on their own forensics.

... to the extent their financial resources allow.

eeZah7Ux5 anni fa

...which can be extremely different between parties leading to EXTREMELY biased outcomes.

Falling35 anni fa

Exactly.

WaitWaitWha5 anni fa

I think your way of writing it is the the best expressed.

dhdhhdd5 anni fa

My experience is the following testimony: "i put the following parameters to the following program and this was the result.".

Potentially explainig why the given pareters were chosen.

anomaloustho5 anni fa

> Any tools used are can be requested to be examined, including software.

Out of curiosity, if this is a common occurrence that is willfully obeyed by all vendors and parties involved. Could you shed some light on why this vendor is resisting?

WaitWaitWha5 anni fa

No, it is not common but does happen. There are several cases on drug testing, DNA, bite mark, and software that comes to mind. Scientifically well established processes, tools, etc. are rarely called up.

Bite mark was a big deal in the industry because it turns out, it is not so unique and the methodologies developed were weak.

My personal opinion why the are resisting? They are resisting because they are bottom dwellers. Forensicators whom I associate with, tool vendors, and I understand that we have to share knowledge. I dare say, we are the most open source scientific knowledge industry.

This is not because of some altruistic reason, but because tomorrow they can be called on to explain.

WaitWaitWha5 anni fa

There are several comments regarding that if one cannot pay for an opposing forensic investigation, then forensics is a black box.

Either you are moving the goal post, or I am to donnish.

Let's agree that it is scientifically not a black box, but some may not be able to pay for such service.

There were suggestions of nationalizing, centralizing or governing forensics and just have one, unbiased working for the courts.

This diametrically opposed to the problem pointed out by many where prosecutors will use specific labs because they return more positives. If the prosecution and the forensicator work for the same employers, how is that prevent further erosion of this problem?

dingoegret5 anni fa

dannykwells5 anni fa

"170,000 lines of MATLAB code" (used as software to determine if humans are spending their lives in jail).

Well if that's not a line to make you feel awful on a Monday morning.

motohagiography5 anni fa

It's an important development. Forensic labs and companies are expert witnesses with black box processes and the incentive to protect the authority of their profession. They are as likely to lie as any other witness. Perhaps even moreso.

"Those arguing on behalf of the defense cited past problems with other genetic testing software such as STRmix and FST (Forensic Statistical Tool). Defense expert witnesses Mats Heimdahl and Jeanna Matthews, for example, said that STRmix had 13 coding errors that affected 60 criminal cases, errors not revealed until a source code review." "They also pointed out, as the appeals court ruling describes, how an FST source code review "uncovered that a 'secret function . . . was present in the software, tending to overestimate the likelihood of guilt.'"

An analogous situation with an alcohol and drug testing lab caused a scandal in Canada and called into question 16,000 child protective services cases: https://en.wikipedia.org/wiki/Motherisk , and then at the same hospital(!), a forensic pathologist was giving fake prosecution evidence, https://en.wikipedia.org/wiki/Charles_Smith_(pathologist)

Maybe there's just something about Toronto and compromised processes, but defense challenges to the integrity of automated systems looks like a growth field.

MaxBarraclough5 anni fa

> Forensic labs and companies are expert witnesses with black box processes and the incentive to protect the authority of their profession.

I imagine they're also incentivised to favour a result indicating guilt.

heavyset_go5 anni fa

They fill a similar role to prosecutors that field drug tests serve to cops[1]: they aren't meant to be accurate, they exist to give the cops legitimacy as they proceed to do whatever it is they wanted to do to you. In the cops' case, they need legitimacy to search and arrest you, and in the prosecutors' case, they need legitimacy to charge and convict you.

If field drug tests were actually accurate, they wouldn't be bought and used. Similarly, if an expert witness fails to give prosecutors the results and testimony that they want, they wouldn't be hired again.

[1] "How a $2 Roadside Drug Test Sends Innocent People to Jail": https://www.nytimes.com/2016/07/10/magazine/how-a-2-roadside...

jdsalaro5 anni fa

> They fill a similar role to prosecutors that field drug tests serve to cops[1]: they aren't meant to be accurate, they exist to give the the cops legitimacy as they proceed to do whatever it is they wanted to do to you. In the cops' case, they need legitimacy to search and arrest you, and in the prosecutors' case, they need legitimacy to charge and convict you.

In case any of you are interested, or completely flabbergasted as I was, by the idea that law enforcement's purpose and raison d'etre could ever become as distorted and contorted as this comment mentions, you should definitely read more on Walter "Johnny D." Macmillan [1] or watch the movie based on his story: Just Mercy[2].

Absolutely mind-blowing stuff.

[1] https://en.wikipedia.org/wiki/Just_Mercy

[2] https://en.wikipedia.org/wiki/Walter_McMillian

bluntfang5 anni fa

Wow, think about what it takes to be an elected official (Sheriff) and not only perjure, lie, and coerce witnesses, but be outed for all of those crimes, and then continue to run for election and continue your tenure for decades. How does someone live with themselves knowing they purposely put an innocent man on death row, and then CONTINUE YOUR CAREER, probably thinking yourself as successful?

The guilt I feel when I introduce a software bug that effects my corporate customers can sometimes burn my motivation for WEEKS. Who are these people?!

anonAndOn5 anni fa

> incentivised to favour a result indicating guilt

Exhibit A: Annie Dookhan of the Massachusetts State Crime Lab who is suspected of falsifying thousands of drug tests

[0]https://www.cbsnews.com/news/massachusetts-lab-tech-arrested...

HarryHirsch5 anni fa

Annie Dookhan did about 3 years of jail, which is risible, considering that her cooking of evidence affected > 30000 cases. No visible consequences for the supervisor, which is scandalous, considering that the had been carrying on for at least 4 years.

neodymiumphish5 anni fa

This absolutely depends on the agency/situation.

My agency used to rate our agents based on the amount of cases that led to convictions. Years back, we recognized that disproving an allegation was equally worthwhile, and settings promotions/evaluations based on "proving or disproving" the allegation was much more objectively reasonable than focusing strictly on convictions. There is still a reference to case completion speed, value of recovered money/property, and conviction results (providing a thorough enough investigation that the suspect is convicted for X years, for example), but disproving an allegation or proving that a different person committed the crime is far more appreciable to the agency.

We're fortunate in that we are a federal agency, instead of local/state level, where they can be significantly more busy with countless lesser offenses, while still running some extremely high-profile stuff, but I think it would be a huge boon to law enforcement if states mandated an approach focused on this objective metric that isn't strictly on "getting him".

tantalor5 anni fa

Why? The risk to their reputation & livelihood for a false positive should temper that.

MaxBarraclough5 anni fa

The hands-off reputation-based approach isn't enough to regulate food and drugs, or aviation. I don't see why for-profit forensics companies should be trusted implicitly.

Also related: we know that funding bias [0] is a real problem in science, despite that scientists' reputations should (ideally) motivate them to resist such biases.

[0] https://en.wikipedia.org/wiki/Funding_bias

kortilla5 anni fa

No, the question is what is the incentive for lying and claiming guilty? Do prosecutors choose different labs?

Ma8ee5 anni fa

Of course prosecutors will favour expert witnesses that will help them get convictions. Why isn’t that obvious?

drdrey5 anni fa

There are countries where the job of the prosecutor is to uncover the truth, not to convict someone at all costs:

https://en.wikipedia.org/wiki/Inquisitorial_system

jerf5 anni fa

I doubt there's any system in the world where it's officially the job of a prosecutor to "convict someone at all costs". It certainly isn't in the US. Nevertheless....

21125 anni fa

Most of the very real dynamics that affect all of our lives at a policy-making / systemic level are never official. If you think the official stuff is a reflection of reality ... I don't know how to finish this sentence. Safe from it being "official", it is very much the job of a prosecutor in the US to convict someone at all costs. This is common knowledge.

Aerroon5 anni fa

The Soviet Union is a perfect example of this. They had set up a proper republic. It had a constitution, elections, there were government posts etc. The problem was that the Party came first. The official that held the government post might've had the legal power, but in reality they were rubber stamps for the Party.

Ma8ee5 anni fa

I know. I live in such country. But even here it is better for the prosecutor’s career with many convictions.

heavyset_go5 anni fa

Conviction rates are surprisingly high in a lot of the first world.

Ma8ee5 anni fa

I’m not sure what conclusion we should draw from that isolated fact. It can mean that people are convicted on too weak evidence, and it can mean that people are only put on trial when the evidence already is very strong.

simple_phrases5 anni fa

I think we can look at the plea bargain rates and assume that most people do not have the resources to stand trial, and thus plea out whether or not the charges against them are valid.

bardworx5 anni fa

Ok, maybe I’m naive, but why is that obvious?

Not arguing just trying to understand.

My rationale is: prosecutors have to weigh an “easy” conviction vs the possibility that labs that always guarantee results may be biased. In the event that they get caught up with a biased lab, wouldn’t it look bad for their reputation? As in, they can’t even perform due diligence on their sources, how can you trust them in a promoted role?

Or am I just naive on how the legal system works?

benlivengood5 anni fa

> In the event that they get caught up with a biased lab, wouldn’t it look bad for their reputation? As in, they can’t even perform due diligence on their sources, how can you trust them in a promoted role?

I am not a lawyer but as I understand the U.S. criminal justice system (which is adversarial between defense and prosecution with the judge officiating and the jury picking the winner) prosecutors are not experts in particular technical fields which is why courts allow expert witnesses. The defense is responsible for calling their own experts who can dispute other biased witnesses. If no one successfully disputes an expert witness over time the prosecutor's trust in them is very likely to grow.

A prosecutor will always claim they did due diligence by selecting a reputable expert witness up until it's proven that the witness was not in fact reliable, but claim (likely in good faith) that they had no way of knowing the expert witness was biased.

A better solution might be to mandate that expert witnesses get independently tested but again that's something a defense council could facilitate and bring into evidence. If no defense team finds it valuable to do this kind of work then it's unlikely a prosecutor will see it as useful either. Lack of challenges implies that the evidence is more or less indisputable from a legal perspective.

mulmen5 anni fa

I wonder how much of this has to do with funding differences. Public defenders are famously underfunded but prosecutors have essentially unlimited resources. As a result they are on a fast-track to political fortune, especially if they "do their job well" and "win" (get a lot of convictions).

How can the truth-finding benefits of the adversarial system be preserved while leveling the playing field by equalizing resources?

My naive thought is that both parties pay into a common fund that is split equally between the prosecution and defense. But that seems like it has it's own set of pitfalls. Are there other models we can look to for ideas?

curryst5 anni fa

So there's an underlying problem here, which is that at our current rate of indictments, our court systems cannot support everyone going to trial. One of the reasons people in charge of funding want to starve the public defenders is that if they're actually good, people will be less willing to just plea out to a lower charge. So the growth will be non-linear as their success rate increases due to the funding, which they now need more of to service the extra cases they get from being good.

One idea is that whoever introduces an expert witness must also pay for the other side's expert witness, up to the cost of your expert. The defense doesn't need to prove anything due to the presumption of innocence, so the defense gets free counter-experts and only have to pay for the counter-expert if they want to bring up an expert about something the prosecution doesn't want to talk about.

mulmen5 anni fa

Huh, that’s an interesting idea but it seems to only address one form of inequality. Clearly the incentives are perverse. If there are more cases brought than the system can handle then this just sounds like a band aid at best.

The real issue is why are there so many cases?

nobody99995 anni fa

>I am not a lawyer but as I understand the U.S. criminal justice system (which is adversarial between defense and prosecution with the judge officiating and the jury picking the winner)

That process happens very rarely. Federal criminal cases are resolved via plea bargain in ~97% of cases and state criminal cases are resolved via plea bargain in ~94% of cases[0].

This is a travesty of justice, especially since most defendants are severely over-charged and often face long prison sentences if they actually insist on (and can afford) a trial.

Those practices, along with cash bail, force even innocent people to plead guilty to avoid having their lives destroyed by bankruptcy, loss of employment, homes and even custody of their children.

And once they have a criminal conviction, they are stigmatized for life and are shut out of many jobs.

In most of these cases, the forensic evidence (if any) is never presented. For a discussion of this, as well as American forensic practices, see Blood, Powder and Residue[1], by Beth Bechky (Author discussion can be found here[2]).

While there are no required standards for forensics labs in the US (with some exceptions[4]), there are accreditation programs (example[3]), and not all forensic laboratories are for-profit companies.

I'm not defending the quality or independence of any particular forensic lab, but it's definitely more diverse than just a bunch of corrupt, rapacious scum sucking at the teat of prosecutors' budgets.

That said, most criminal defendants are at a significant disadvantage when it comes to performing/challening forensic research, as many state and federal labs provide such services for prosecutors, while defendants need to pay, often through the nose for them.

It's just another way the US "justice" system is stacked against criminal defendants.

[0] https://www.nbcnews.com/think/opinion/prisons-are-packed-bec...

[1] https://press.princeton.edu/books/hardcover/9780691183589/bl...

[2] https://www.c-span.org/video/?508130-1/blood-powder-residue

[3] https://anab.ansi.org/en/forensic-accreditation

[4] https://www.justice.gov/opa/pr/justice-department-announces-...

Edit: Added notation for reference [1]

horsawlarway5 anni fa

I'll take a different approach than the other folks responding to you -

There are plenty of profit motives involved that have little to do with securing future contracts with police departments.

At the most basic - Source code reviews are expensive. They can often throw thousands of false flags that developers have to individually track down and verify are not problems.

Then there's the reputation hit of admitting a mistake - If you've had a bug for the last 5 years that makes you less reliable, that alone is likely to impact future contracts. So given the option of revealing this, admitting the mistake, and tackling the cases it impacted... OR... simply sweeping it under the rug and fixing it internally - some companies will take option number 2 (possibly most companies).

the_local_host5 anni fa

I think you're underestimating the usefulness of scapegoats. If biased results exist, a lot of people look more effective. If such a bias is discovered, only a subset of those people will be blamed.

eptcyka5 anni fa

Prosecutors don't seek justice, they seek a conviction.

JoeAltmaier5 anni fa

Sometimes the expert witness is in the department's own lab. It seems a tautology that they will favor conviction. Every officer bringing in evidence is begging "Give me something I can use to get this scumbag!" The pressure to fudge is enormous.

A fairly simple step is to certify equipment independently, and to do blind testing (one sample is provided from evidence; another by an independent unrelated source). Of course this costs more, but anything else is effectively an experiment without a control.

pelliphant5 anni fa

That seems plausible.

I would argue that if prosecutors and/or cops get to pick the tool, false positives would result in more sales.

Just as false negatives would result in more sales if the defendant gets to make the pick.

thgaway175 anni fa

false positives would result in more sales.

That seems highly unlikely. Prosecutors, like all attorneys, talk. It wouldn’t take too many getting burned by a false positive before word would get out (regardless of any nda).

For example, once an expert gets Dauberted, you’ve got to think long and hard about being the next guy to use them.

pelliphant5 anni fa

But how would one actually know if a result is a false positive or not?

Sure, in extreme cases it might be obvious, but if you just know that one test gives more matches than another, not knowing which one is the one giving false results?

merpnderp5 anni fa

But genetic evidence is often enough to convict by itself. A false positive is far more likely to land a conviction than an embarrassment to the prosecutor. I mean we live in a world where DNA evidence will easily override solid alibis.

[deleted]5 anni fa

65105 anni fa

A product that doesn't work wouldn't be considered.

ncallaway5 anni fa

You're a prosecutor. Your goal is to get convictions. You are objectively graded largely on your ability to secure convictions.

Every incentive you have leads you to preferring convictions.

You have a choice between two labs to hire for your field test. Lab A and Lab B.

Lab A gives you conclusive evidence leading to a conviction 95% of the time. Lab B gives you conclusive evidence leading to a conviction 65% of the time. Price and speed are roughly comparable between the labs. Which lab do you select?

---

You are now a lab director. You are under pressure to improve the sales of the lab's forensic services. You have pretty much maxed out making changes to improve price and speed, and have been matched by other labs. You understand very well the decision making process prosecutors will use to hire your lab.

---

The incentives clearly lead prosecutors to pursuing labs that deliver better results. Labs know this, and so are under market pressure to delivery convictions for prosecutors.

ashtonkem5 anni fa

Yes, yes they do. Prosecutors often campaign on being “tough on crime”, and even judges (where elected) are known to sentence more harshly during election season.

The ability for an expert witness to deliver more convictions, even if by lying, is a feature for prosecutors, not a bug.

geocar5 anni fa

If you knew someone was guilty, because of a preponderance of other evidence you have the utmost faith in, such as a professional law enforcement officers’ eyewitness account, and the DNA testing vendor you were evaluating said “not guilty”, what exactly would you expect to happen?

boomboomsubban5 anni fa

That the eyewitness account would be discarded, they make for terrible evidence anyway.

tekromancr5 anni fa

I think it is REALLY hard to overstate how dangerously naive a view of the american justice system that is

boomboomsubban5 anni fa

I don't understand what you mean. I would expect them to make the obviously correct decision, though I'm fully aware that they very often don't.

tsimionescu5 anni fa

It depends on the definition of 'expect'. You should expect it of them, as in consider it unethical if they don't. You shouldn't expect them to live up to this standard, as in, if you were to bet on it, you'd likely lose.

buttercraft5 anni fa

Paying investigators per conviction is a sure way to create incentive.

Eric_WVGG5 anni fa

Exactly. "Product X caught 25% more bad guys than Product Y" this stuff just writes itself.

Policing is driven by numbers. There was a whole season of The Wire about this.

irjustin5 anni fa

A lot of DA and other prosecutors are measured by the number of convictions they get vs the number of cases they take to court.

Their goals are always to get plea bargain because that is automatic guilt or if they take it to court, they'd better win. That's a lot of state resources just to lose.

Overtime, you'll get people willing to bend the rules, turn a blind eye to 'weak evidence' or whatever.

You're only as good as your measurement/incentivisation.

treeman795 anni fa

Deal: Parole for a year. If not, full resources of the government to convict you, jail time in the decades. Evan a failure to convict can ruin you for life.

Even innocent people with a good alibi are going to take the deal.

chopin5 anni fa

Why should that come to light if the process is opaque? Most defendants don't have the resources to challenge that.

And if it comes to light, acknowledge a one-time error and carry on.

wtetzner5 anni fa

I seems like there are no repercussions for sending an innocent person to prison. I don't know if that's true, but it definitely seems like it.

jimz5 anni fa

Depends on what you mean by repercussions. Since there is at least informal and sometimes fairly open scorekeeping happening at prosecutor's offices there's some hurt pride if your conviction is overturned later. Everything else either gets taken care of by the taxpayer or prosecutors' immunity for their official acts so on a personal level, no, there won't be any actual repercussions.

dathinab5 anni fa

Because they likely use metrics which highlight how many criminals where caught due to their system.

On the other side a metric which says how many times no conclusive evidence was found with DNA testing sounds more like a metric of failure (but is not, if you don't have qualitative good enough evidence including "complete enough" DNA you should not make up results which simply don't exists, but then courts ruling without conclusive evidence isn't that rare and some would say even necessary to some degree. It's just that many believe DNA tests are somehow unfailable perfects ways to prove the presence of an specific person at a specific place, but they are often not).

andi9995 anni fa

The prosecution might call in a different expert in the next trial if the current one is not 'good'.

refurb5 anni fa

Then they get hired by the defense.

MaxBarraclough5 anni fa

The defence can rarely afford their services.

jimz5 anni fa

Prosecutors aren't necessarily blessed with bigger budgets in a lot of places, they simply have a separately-funded (and very well funded at that) investigative body that is the police or sheriff's office doing the really expensive part that the defense, especially a public defender's office, will only have a skeleton crew to do. The problem is that defense may not have the resources or knowledge to use inadequacies of something like this as a defense in the first place, and may not be able to effectively cross or direct an expert because of the lack of specific expertise and the reactive nature of the job.

It's not that the venn diagram of tech-literate and criminal-defense lawyers are entirely separate circles, but having been "that guy" in a public defender's office for even basic stuff like cell tower triangulation accuracy to finding proprietary surveillance video codecs to decode exculpatory evidence, you really need to start at square one while the prosecution have the whole police department's resources, expertise, and initiative at their disposal. You can afford the expert, it just won't do any good when you don't know what questions to ask that will actually be effective. And unsurprisingly those who do have experience in technical, specialized fields tend to get poached into the private sector or out of trial (really plea) practice all together, so the knowledge/bullshit gap will still exist and there's no real consistent way to bridge it.

refurb5 anni fa

The one trial I sat in on, it was the public defender who hired the expert.

Go figure.

andi9995 anni fa

Also the defense is many customers while the prosecution is bulk.

_jal5 anni fa

Livelihood is frequently the reason why biases creep in. Independent labs tend to do better.

Here's a look at two different Texas crime labs. Compare and contrast, and see if you can spot the differences.

https://gritsforbreakfast.blogspot.com/2021/02/beyond-aid-to...

It starts with one pledging to:

   aid law enforcement in the detection, suppression, and prosecution of crime

whereas the other seeks to

    provide medical examiner and crime laboratory services of the highest quality in an unbiased manner with uncompromised integrity.

and goes from there.

vkou5 anni fa

When a DNA lab issues a false positive, its CEO doesn't go to prison, nor do they get hit with a charge-back for all the tests they've done in the past year.

Their reputation doesn't matter as much as you think. Police procurement isn't based on the reputation of the vendor. It's based on whether or not the vendor delivers the results they want.

In fact, government procurement in general, isn't based on the reputation of the vendor. This is by design.

macinjosh5 anni fa

Prosecutors want a 100% conviction rate. They are the ones who hire these companies. Generally speaking companies give customers what they are looking for.

linuxftw5 anni fa

Perhaps in a just society. Most places, the state and it's agents can do no wrong, there's almost no incentive to do the right thing.

dimitrios15 anni fa

This is a long tail regulatory force, but unfortunately, does nothing to stop bad actors from jumping in and making a quick profit for as long as they can until it goes noticed.

Society also no longer has tolerance for processes that are not fully auditable and transparent, especially when it comes to our already marred justice system.

acranox5 anni fa

Maybe for some, but look what happened in Massachusetts. https://en.wikipedia.org/wiki/Annie_Dookhan

avs7335 anni fa

The risk to their business of negatives whether or not they are false negatives, is likely higher. The resource asymmetry between a tax payer funded prosecution and a defendant funded defense is enormous.

AntiImperialist5 anni fa

JoeAltmaier5 anni fa

While there have been cases of technicians guilty of systemic fudging of data, that is a tiny exception to the vast majority of honest technicians with self-respect.

kingkawn5 anni fa

Your sentimentality is not enough for a good justice system.

JoeAltmaier5 anni fa

On the contrary, that's what the American justice system is entirely based upon. Fair judges, juries of peers and so on. Its the bedrock of justice.

Engineers have some notion that it can all be replaced with science. In so far as science can certify the reliability of tests, that is good. But in the end one has to trust the humans between those tests and the courtroom.

tsimionescu5 anni fa

The whole point of this discussion is that this trust is being largely misplaced.

A lot of forensic science in itself is essentially phrenology (tooth prints, hair analysis - the non DNA kind, even fingerprinting is of exaggerated value). A lot of labs systematically turn out biased results with generally no expectation of risk, either personally to researchers and definitely not to higher ups.

Ultimately, forensic evidence should be seen as a signal, but not nearly as trustworthy as witness testimony (which in itself is not very trustworthy), despite what many believe.

And this is important for the general public to understand, the people who will participate in juries. The mystique that forensic experts have can make juries give extremely wrong decisions (even bad aquitalls on lack of forensic evidence).

JoeAltmaier5 anni fa

Yes there is a need for controls and independent certification.

tsimionescu5 anni fa

There is also a need to recognize that a lot of this science just doesn't work (to the extent that it is often presented). The right way to phrase a lot of the conclusions of these experts, at the state of the art, would be 'in my expert opinion, it looks like it might be their DNA/fingerprint/hair/teeth', not 'in my expert opinion it is [...]'. Because even the state of the art is often about at that level - probably closer to 75% accuracy than 99.99% like it's often treated. Especially on real world, partial, corrupted samples.

And to emphasize again, I'm talking about the state of the art without the biases being discussed. The biases only compound this problem significantly.

m4635 anni fa

Imagine what an innocent man goes through.

If I was innocent, I'd be 100% certain that the system was screwed up, and if DNA evidence "proved" it, I'd want to look at the code too.

I remember reading about the husband who was falsely accused of murdering his wife, and it was very very sad.

https://www.texasmonthly.com/politics/the-innocent-man-part-...

GizmoSwan5 anni fa

DNA testing should be double blind. I have no idea if it is.

Sample contamination can be arranged before sent to the lab though.

IG_Semmelweiss5 anni fa

i think we can all agree that black boxes are not desirable.

But let's not jump the gun here, and assume ill-intent.... to explain something that can be attributed to negligence, or simple human error.

IgorPartola5 anni fa

When your negligence results in people wrongfully spending decades in prison I would argue that it ought to be criminal negligence. If your software does not produce results that are accurate, precise, and have confidence intervals included, how OK are you with people going to prison based on them?

caconym_5 anni fa

Yeah, there need to be consequences for this. The alternative is empowering people and corporations to destroy lives with impunity in pursuit of their own goals, as long as they can later claim it wasn’t intentional.

What really pisses me off is how we fail again and again to ask the question of whether these entities should have been doing what they were doing in the first place, if it even can be done safely. Facebook, Equifax, police surveillance and misconduct, totally unaccountable “forensics” techniques—in all these cases where incalculable damage is done in totally predictable ways, we only address it reactively (if we address it at all) and completely fail to fix or even really consider the root cause(s). It’s like we have this huge blind spot where we take as a given that established entities doing thing(s) just have a right to keep doing them, regardless of any other factors.

In this case, forensics “experts” (essentially witch doctors) and fly-by-night black-box “labs” with unproven methodologies an undergrad intern might have developed in a week or two are institutionalized in our courts. This has happened because there is zero oversight and zero accountability for their claims, and lives are destroyed as a result.

It is insane not to pursue criminal charges for these indiscriminate arsonists of justice—if they can’t do what they claim to do fairly and accurately, they should not do it. They don’t have a right to “try” just because it will make them money, regardless of the consequences.

tsimionescu5 anni fa

A lot of people respond to this as if the fact that it would be nice if forensics worked justifies treating them as if they do. 'What do you want them to do, stop looking for fingerprints and DNA evidence?' Yes! Just like I don't want them to use 'lie detectors' or hire psychics.

caconym_5 anni fa

Exactly! It’s pure wishful thinking, and in a lot of cases those wishes are (at best) frighteningly amoral in context.

It’s like a hack to get around the whole “reasonable doubt” thing: just sequester all that pesky doubt in a black box so that people can’t see it, and if anyone asks just play dumb.

crazygringo5 anni fa

The quality standards need to be set by the courts or by leglislation.

No software is perfect. It's unreasonable for any bug to be considered criminal negligence, or else the entire software development profession would be in prison.

Instead, software that can result in life or death or prison scenarios needs to have tested, documented, verified quality controls, just like we do for vehicles.

But that means courts or the legislature have to decide what those standards are.

drstewart5 anni fa

>I would argue that it ought to be criminal negligence

I would like a thorough analysis of everything you've ever learned that led you to this conclusion, and if we find one mistake in your lifetime of learning then this is also criminal negligence on your part.

[deleted]5 anni fa

kspacewalk25 anni fa

I agree that we shouldn't assume ill intent, but we should recognize it is a very real possibility. Ill intent comes in when you realize the simple human error. Alone, in your office, poring over code or reviewing past files. Then you weigh the impact on your career and reputation of honestly fessing up, and put it off for a while, then a while longer.

anamexis5 anni fa

Why shouldn't we assume ill-intent? The precedent and the incentives are there.

AS375 anni fa

> a forensic pathologist was giving fake prosecution evidence

The doctor handled thousands of cases in his career, and a well-funded inquiry found issues with less than 1% of them.

Sure we'd like that to be 0%, and society should spend time thinking of how much they rely on complicated processes of reasoning, but that's a really good accuracy.

pc865 anni fa

You say that as if these were mistakes or unfortunate culminations of circumstance. The man committed fraud and sent innocent, grieving parents to prison by proxy. I don't think it's unreasonable to expect that to be 0.00% and punish those harshly who exceed that metric.

badRNG5 anni fa

If I'm a juror, and the evidence presented happens to have a false positive rate of 1%, that certainly meets my understanding of "a reasonable doubt."

londons_explore5 anni fa

I would need a very very big pile of very clear cut evidence to find someone guilty of a crime where the likely outcome is a substantial prison time.

It's all tradeoffs, but "his blood was found on the murder weapon" wouldn't be sufficient for me - some childhood enemy could easily have planted a bunch of forensic evidence next to a crime scene.

michaelt5 anni fa

Unfortunately, the vast majority of crimes don't produce evidence that rises to the level you call for.

After all, if my next door neighbour barged into my house and beat me up, the only evidence would be my visible injuries and my statement that it was my neighbour.

I'm not sure if I'd prefer a society where he would be convicted, or where he wouldn't.

londons_explore5 anni fa

Maybe your neighbour has a nicer car than you which makes you feel insecure so you decide to beat yourself up and go to the police saying your neighbour did it.

Considering that possibility, I don't want to put your neighbour in prison.

michaelt5 anni fa

And presumably, if having been denied justice I pursue revenge instead, I can barge into his home and beat him up, as he did to me, and avoid punishment for the same reason?

While I admire your devotion to Blackstone's Ratio, this doesn't seem like a recipe for stability and rule of law to me.

Nightshaxx5 anni fa

A murderer kills less than 1% of all the people they meet. Does that mean we should just accept their behavior?

slickQ5 anni fa

A murderer who kills 1% of the people he meets has certainly committed a crime. A person who has a 99% chance of being a murderer has not certainly committed a crime.

That is depending on your threshold of certainty. 1% is not that high considering that, according to the OJJDP, 5 milion people were arrested for serious charges in 2019 so with a 1% false positive rate that would be 100,000 people falsely imprisoned every year.

It does not have to be and really can't be 0% but 1% is unreasonably high in my opinion. If it can't be helped then it can't be helped but that isn't necessarily the case with these devices.

Nightshaxx5 anni fa

The doctor did not have the _potential_ to fake evidence. He _did_ fake evidence.

throwaway22455 anni fa

Is it bad that I'd guess you would be excluded from a jury if you admitted this upfront?

yarcob5 anni fa

The wikipedia page is heartbreaking. Griefing parents sent to jail, their other children taken away and placed in foster care... So many lifes made miserable by one person...

mschuster915 anni fa

Additionally it could be made a requirement that, like in professional sports, two or three blood drafts are taken - one sample is analyzed and one is either kept until the case is closed or analyzed at a different lab if the evidence is contested.

londons_explore5 anni fa

This method doesn't protect against evidence tampering. Someone can just take a drop of that blood and smear it on the victims clothes. Now it doesn't matter how many labs analyze it, it's always going to come back a match...

yarcob5 anni fa

From the Wikipedia page

> In June 2005, the Chief Coroner of Ontario ordered a review of 44 autopsies carried out by Smith. Thirteen of these cases had resulted in criminal charges and convictions. The report was released in April 2007, indicating that there were substantial problems with 20 of the autopsies.

That sounds more like 45% to me.

Also this bit from the article about a baby that allegedly had a skull fracture:

> Later exhumation of the child and examination of the skull have shown that there was no skull fracture. It is thought Dr. Smith confused the normal gap between the baby's skull plates for an injury.

Holy shit that is bad. Even as a lay person I know about these gaps.

dathinab5 anni fa

> there are gaps between skull plates.

Between Baby skull plates, not adult ones ;-)

They grow together at some point.

yarcob5 anni fa

I edited my post to make it clear I don't think skulls have gaps in general.

AnIdiotOnTheNet5 anni fa

> Holy shit that is bad. Even as a lay person I know that there are gaps between skull plates.

Sure, but can you tell them apart from a fracture caused by injury? Presumably someone knowledgeable in medicine should be able to, but maybe it is one of those things that isn't as obvious as we, the uneducated outsider, might think.

R0b0t15 anni fa

That's an extremely flawed argument. If you can't explain it to a layman then you don't understand it. If you can't tell whether it's a gap or a fracture -- you can't get a conviction.

"It just looks like it" is not a valid reason.

AnIdiotOnTheNet5 anni fa

It is possible that they could have well supported their assessment with evidence and still been wrong.

AS375 anni fa

Interestingly there's an area of psychology called Naturalistic Decision Making which studies how experts make decisions that they can't explain. (Example: a firefighter may be able to pinpoint where a fire is before they enter a house and see it.)

R0b0t15 anni fa

I know, but that is reason for suspicion, not evidence for a conviction.

sethammons5 anni fa

If the error goes against innocent people, then it must be unacceptable. If the error sides with the “would-be” criminal, that is more acceptable as we, as a society, have decided that letting a few bad folks go is preferred to imprisoning innocents.

35fbe7d3d5b95 anni fa

> as we, as a society, have decided that letting a few bad folks go is preferred to imprisoning innocents.

We, as a society, like to say this because it makes us feel good.

But a critical analysis of our actions reveals that we don't believe it for a second. We are happy to write overbroad laws that allow most behaviors to be criminalized, over police marginalized communities, and place those who have been convicted by this flawed system in deeply dysfunctional prisons that may well violate their human rights.

dghlsakjg5 anni fa

The end result of that tiny little error rate was tens of millions in settlements and legal costs, families permanently separated by government mandate, and multiple people spending more than a decade in prison.

JoeAltmaier5 anni fa

I'm not so sure we should be allowing challenges to expert witness, with no evidence. That sounds like a rich source of abuse of the system, like a big company filing thousands of briefs to delay a case.

It's important for experts to use validated, tested equipment. That's clear. And by the sound of it that is happening (stories of how code was tested and found wanting confirm that; stories of double-blind testing).

Maybe what is needed is, an 'underwriters lab' certification for such devices. To forstal the inevitable gaming of the 'right to challenge'.

josephorjoe5 anni fa

defendants should always be able to challenge expert witnesses qualifications.

and how could anyone on hacker news claim there is 'no evidence' that unreviewed software may do something other than what people claim it does? especially if it 170,000 lines of matlab?

from the article:

> "Without scrutinizing its software's source code – a human-made set of instructions that may contain bugs, glitches, and defects – in the context of an adversarial system, no finding that it properly implements the underlying science could realistically be made," the ruling says.

x0x05 anni fa

Particularly when TrueAllele specializes in

>probabilistic genotyping [1]

and

>solves mixed DNA samples, low template DNA and kinship problems [1]

And the product sheet discusses

> models for PCR artifacts [2]

This sure feels like they go way beyond typical dna match technology into maximally extracting information via statistics, so their methods definitely should be open for review.

[1] https://www.cybgen.com/products/casework.shtml

[2] https://www.cybgen.com/solutions/brochures/unique_features.p...

jonhohle5 anni fa

In most criminal complaints the defendant has far fewer resources than then the prosecution. The prosecution has a conflict of interest to get a conviction. Absolutely everything about their process needs to be above reproach, including their experts. A challenge to that process, approved by a judge, seems like a small consolation in a biased system.

sixothree5 anni fa

When I was a juror on a murder trial the one thing that was most clear was the disparity in resources.

On one side were 6 impeccably dressed men in what I presumed were multi-thousand dollar suits. On the other side was single overwhelmed, overworked, and under-dressed defender who was steam-rolled at every sentence she spoke.

I ended up in traffic court last year for a parking ticket that I can only think must have been stolen from my car during a festival. After spending just a few hours waiting my turn, it was pretty clear who exactly the police target for traffic violations. But this is a tangent.

salamanderman5 anni fa

Grossly oversimplifying it, but it is what my lawyer uncles say, and is consistent with what I saw as a juror on a civil trial involving many topics I'm personally an expert in, expert witnesses are paid to lie on the stand. At the very least they are paid very very well (the amount they were paid was emphasized at the trial I was at) to state only the information consistent with what the person paying them wanted to them to say, regardless of whether they know information that would be more helpful to the other side.

AS375 anni fa

> At the very least they are paid very very well... to state only the information consistent with what the person paying them wanted to them to say

Anecdotally, if you replace 'paid' with 'incentivized' my understanding is the same.

And here's a totally unrelated Wikipedia link: https://en.wikipedia.org/wiki/Michael_Baden

kspacewalk25 anni fa

Of course this is the case, but rather than lying, I think it is only a consequence of selection bias. You don't see the experts rejected by the prosecution (or defence) for opinions that aren't helpful, because of course they won't be invited to testify.

JoeAltmaier5 anni fa

They answer any question put to them. The defense has to ask the right questions of course.

I know expert witnesses. They are of impeccable character, and will seek to educate as well as they are allowed.

salamanderman5 anni fa

The experience I had, which was pretty cool (although the case overall was very very depressing) in that we the jury were allowed to anonymously submit questions for the judge to ask the witnesses as well."They answer any question put to them." definitely did not apply to all of them. There were many, painful to watch, who at best gave half answers. I'd say the expert witness in the movie "my cousin Vinny" who testifies that the tire marks between the two cars are identical and that it is possible for both cars to have made the tire marks, while knowing that it was literally impossible for the defendants car to make the marks on the road, is very very close to what I saw experts do. In my case it was dumber, though. We had experts talking about how far a driver can see to recognize that a pedestrian is walking across a road and all of his citations were referencing daylight illumination. Thankfully, us jurors were able ask questions like "All of your citations are for daylight illumination. You know this case is about an accident at 3am and the streetlights were broken. What's the distance in that case?" And they would answer that question appropriately, but I didn't consider it impeccable character that they had acted all the while up to that point like the numbers they were giving were for the conditions of the case. And yes, they would always say they were familiar with the conditions of the case before answering any questions, so I generally felt is was lying by omission.

tsimionescu5 anni fa

You know some expert witnesses. I doubt you have a good acquaintance of a significant percentage of all expert witnesses.

tehwebguy5 anni fa

Absolutely not! Please, if you want to know why, look into expert witness testimony as a whole, possibly can start with “burn pattern experts”.

Not to mention we have a constitutional right (6th amendment) to face and cross examine witnesses against us in the US, a black box should not void that right.

gamblor9565 anni fa

When I was a public defender I made my career by cross-examining expert witnesses. There are a dozen innocent people walking around free today because I challenged the expert despite going into the cross "with no evidence."

What you call abuse, I call due process.

(And your proposed solution is already part of the expert certification and questioning process in criminal trials, so it isn't a solution. Experts must testify that the equipment they used was calibrated/certified/etc based on the standards applicable to the field.)

JoeAltmaier5 anni fa

Testify does not equal independently certified. Also, double-blind testing is not really subject to faking. Its the standard of science.

Responses to my comment are all around 'how it is now' which seems not to be very reliable. A responsible testing/certification process would address that.

gamblor9565 anni fa

You've just moved the goal posts from someone who can be crossed in court to someone who can't, which is worse, because then you have now way of verifying that the "independent" certifier is actually independent or that the certification was properly done.

You still need to allow the "independent certification" to be examined in court to validate both the independence and the certification. The means testimony on the applicable standards, and how the specific equipment used satisfied those standards. And right now, the expert witness already does the latter with respect to calibration, etc., for the equipment they used.

JoeAltmaier5 anni fa

The skepticism has to end somewhere. Else ever court case becomes an infinite regressing into doubting the foundations of everything.

Of course self-certification is not the best. To bring up that straw man (as most of these posts do) is not advancing the conversation.

gamblor9565 anni fa

Yes, the skepticism has to end somewhere. But when someone is facing years, or life, in prison, that skepticism absolutely should not end before it validates the primary evidence of guilt, and the tools used to purportedly analyze that evidence.

If you can't validate the equipment used to analyze the DNA, gunshot residue, or other scientific evidence, then that evidence should not be allowed in court, because that's prima facie reasonable doubt.

tsimionescu5 anni fa

That straw man is essentially the status quo at the moment...

JoeAltmaier5 anni fa

Reread the comment being responded to? It started with that, suggested an alternative. Anything helpful to add?

yholio5 anni fa

> I'm not so sure we should be allowing challenges to expert witness, with no evidence

Any challenge is "allowed" in the sense that the parties can try to convince the judge there is a flaw.

The question is one of burden of proof: should the expert prove they have a trustworthy result, or should the expert testimony stand on their credibility alone, and the defense scramble to poke holes into it armed with incomplete information.

For a system that can put people into jail for life and that acts as a direct extension of state power, it's obvious the latter can have disastrous consequences. Effectively, any one who acquirers the "judicial expert" title can put people in jail at their whim.

JumpCrisscross5 anni fa

It seems reasonable to have a lower bar for challenging expert witnesses in criminal cases versus civil ones.

anfilt5 anni fa

Link to the ruling: https://www.eff.org/files/2021/02/03/decision_appdiv_232021....

chadlavi5 anni fa

While this may lead to more transparent technology use in the justice system, a more realistic outcome is that everyone accused of any crime where a technology was used in the process of determining guilt (which could get quite picayune; did the prosecutor's office use Excel? Is that something the defendant could demand to check the source code on?) will use this as a way to slow-roll the process indefinitely.

tehwebguy5 anni fa

Well sure, if they rely entirely on Excel to generate and spit out an expert witness testimony. If that ever happens it will be extremely important to understand what’s happening under the hood in Excel.

Spivak5 anni fa

Yes? I mean the intimate details of how cell networks work have been the difference between guilty and not before when the only evidence is some black-box report generated by AT&T that seems more accurate than is actually guaranteed.

This wouldn’t be necessary if the expert on the stand was a geneticist who ran the test. But when the expert literally is the software you can’t really cross-examine the company’s sales rep.

bagacrap5 anni fa

Definitely if an excel macro determined my guilt based on a dataset, I would like to be able to read that macro.

abfan11275 anni fa

or at least double check the math.

Jolter5 anni fa

Since when do defendants have an interest in prolonged sentencing processes? For major crimes like this, they are in jail with restrictions. They would get better treatment once sentenced, in the prison system, wouldn’t they?

giantg25 anni fa

Now if this can become precedent for all evidence derived from automated systems. There have been some past battles about BAC machines in this regard.

marcodiego5 anni fa

Wouldn't this set a precedent to eventually force any forensic software to become open source?

jMyles5 anni fa

It's unfathomable that the state can introduce evidence of any variety that isn't open source.

Secret policing and secrecy in prosecution have no place in decent society.

vincnetas5 anni fa

Code inspections is not same as open source.

Zenst5 anni fa

Not per-say, though it certainly affords it the same level of scrutiny.

Which may well be a win as open source already has that.

However, even open source has bugs that pass scrutiny as many CVE's can attest, so whilst a code review may find nothing wrong, that in itself could be used by a lawyer to create reasonable doubt, if the lawyer is good.

jlgaddis5 anni fa

> per-say

https://en.wiktionary.org/wiki/per_se

[deleted]5 anni fa

schmorptron5 anni fa

That'd be ideal

jimnotgym5 anni fa

I guess open-source implementations of various algorithms that had been demonstrated effective, if produced by a not-for-profit maybe, could be very disruptive to this 'industry'.

gsich5 anni fa

Hopefully.

Snoozus5 anni fa

Why don't they just redo the analysis with a few other software packages?

elliekelly5 anni fa

The defendant has a right to “confront” their accuser. So when an algorithm is the one doing the accusing it seems only fair that the defendant should be able to understand how and why the algorithm arrived at that conclusion. Running the analysis through every software package available wouldn’t satisfy the defendant’s 6th amendment rights.

carlmr5 anni fa

It wouldn't satisfy them, but if one of the software packages gave a different answer they can't both be correct.

elliekelly5 anni fa

Or worse, they could give the same incorrect answer.

MaxBarraclough5 anni fa

That would help with variance, but not with bias.

Business incentives presumably reward all such companies for helping the police get convictions, so it doesn't seem a far-fetched concern.

jonnycomputer5 anni fa

Who is "they"

unnouinceput5 anni fa

Probably "they" refer to prosecution. And yeah, why is prosecution going with that lab/software only? One reason is probably the DNA available for more testing is not enough / one testing could been done reliable with DNA taken from crime scene. Other reason might be behind the scenes incentives, which seems is the defense strategy here

vaduz5 anni fa

Even if you had plenty of DNA material, going with multiple labs comes with a risk that one of them comes with a "no match" result - and that is something that is supposed to be disclosed to defense as potential exculpatory evidence - and is a nice source of reasonable doubt for a jury. It's something that is terribly inconvenient for the prosecution when someone has already been charged with the crime...

jlgaddis5 anni fa

> why is prosecution going with that lab/software only?

Presumably becquse that lab/software gave them the answer they were looking for.

Why pay for even more lab/software testing when you've already got what you want (especially when it's a possibility that the additional testing will contradict the first)?

zyngaro5 anni fa

This kind of software should be required by law to be open source.

fsflover5 anni fa

https://publiccode.eu

TheRealDunkirk5 anni fa

Yes, just like voting machines. But how do we confirm that the software which was vetted is the software that was actually _used_?

calciphus5 anni fa

Compile it, run it with the same inputs the prosecution claims was provided, and see if you get the same result. That's sorta the point of computers, right? Same inputs on the same program, same output?

curryst5 anni fa

> Compile it, run it with the same inputs the prosecution claims was provided, and see if you get the same result. That's sorta the point of computers, right? Same inputs on the same program, same output?

One difficulty here is that the input is a real world effect. That means the answers are rarely exactly the same. The last time I looked, breathalyzers have a ~50% margin of error. So if you blow a 0.06, your BAC is really somewhere between 0.03 and 0.09. You could mod your breathalyzer to always assume that the reading was 25% higher than the reality without being noticed, as long as your breathalyzer doesn't return a result that's already 25+% higher during testing. Even if it does, your other results will be in range.

They need to just stop using breathalyzers. They're inaccurate pieces of machinery, operated by people without any medical knowledge to speak of, who have a vested interest in the readings being high. The whole thing, from top to bottom, is just a long chain of poorly aligned incentives. Field sobriety tests are even worse. The fact that your license can be revoked for refusing the test under the above grounds is nothing short of an end-run around the 4th amendment by declaring driving a "privilege, not a right" despite the fact that it is extremely hard to live in most of the US and not be able to drive. How the fuck do I survive in rural Oklahoma/Georgia/Kentucky/Texas/etc, 40 miles from the nearest bus stop, without a car? And all without any due process. It's never impacted me, but it bothers me nonetheless.

They need to force people to go in for a blood draw to establish BAC. It is an accurate test, administered by competent medical professionals, who have no vested interest in the outcome either way. Yes it does take longer. Yes, some people who were at 0.08 will be down to 0.07 by the time the test is administered. I still find that more comforting than the fact that some people at 0.06 will read at 0.09, and some people at 0.14 are going to read at 0.07.

There's also the fact that a cop who doesn't calibrate their breathalyzer is merely going to get a slap on the wrist, and a medical professional who does the same is probably going to end up with a huge malpractice suit and possibly lose their license if it was egregious in some way.

themaninthedark5 anni fa

> The whole thing, from top to bottom, is just a long chain of poorly aligned incentives. Field sobriety tests are even worse. The fact that your license can be revoked for refusing the test under the above grounds is nothing short of an end-run around the 4th amendment by declaring driving a "privilege, not a right"

I agree and I wish I had a better solution.

>They need to force people to go in for a blood draw to establish BAC.

I know the intention is good, but that seems like a very scary proposition to me. I think the same poorly aligned incentives will line up here too.

"We are now partnered exclusively with X-clinic to do all the blood tests."

X-clinic just happens to tweak their procedure so that borderline samples end up showing under influence.

Jolter5 anni fa

Many jurisdictions would have enough cases come through to justify employing someone trained to draw a blood sample.

Btw here (Sweden) the breathalyzer is only used to help decide whether someone gets picked up for a blood draw or not. Only a blood test can be used to convict someone.

themaninthedark5 anni fa

To be honest, having someone affiliated with the justice department would not be seen as something that increases trust.

There are historical and current issues that cause this distrust, having an adversarial or multiple third parties is probably the best option for the US.

kart235 anni fa

>The last time I looked, breathalyzers have a ~50% margin of error

Source on this? I know the portable ones are like this, but I thought the actual court-admissible ones are more accurate, and the calibration procedures need to be followed to get a conviction. Also, FSTs are quite good for preliminary screening, a proper HGN test especially. And if you ask for a blood test, they are required to give you one and consider it in court.

Cops have no incentive to make people who aren't drinking take FSTs or the breathalyzer, its just a waste of their time. They don't want to be bothering normal people, they want to be taking real dangerous and irresponsible drivers off the road. You survive by not drinking before driving, or having a DD, its really not that hard.

Implied consent is the reason for the tests, and you agree to it when you sign for a license. And I think its a worthwhile tradeoff if it actually works to reduce DUI deaths.

tsimionescu5 anni fa

> [Cops] don't want to be bothering normal people, they want to be taking real dangerous and irresponsible drivers off the road.

That's the nice theory we all wish we lived in, but the reality all over the world is that police will extremely often do things for their own petty reasons, often systematically. Police departments often have quotas for tickets, they can often extract a bribe by threatening someone with a DUI, they can simply want to scare 'undesirables' out of a 'nice neighborhood'.

In general, there are good reasons to be extremely weary of the police and their motives.

kart235 anni fa

I agree. The solution is to make people aware of their rights and know what to do when a cop is doing something shady, not weaken the tools that the justice system has for going after things like DUI. Most departments have bodycam requirements now, and citizens have the power to impose steep penalties upon dirty cops.

I stand by that statement though, giving bullshit tickets and fines means that cop will likely just have to spend more time in court trying to defend hopeless cases and eventually get investigated for this kind of activity.

JoeAltmaier5 anni fa

Not true universally. In-car breathalyzers are calibrated regularly; have margins much closer to 10%. They are portable and simple to use.

Maybe in the 'bad old days' it was worse. But today its been scrutinized in court so many times, it has to be bulletproof to get deployed.

Now, blood testing may differ from breath testing. Which is aligned with impaired driving? Breath testing may be the better measure. If state laws are often couched in terms of breath alcohol, then they are the only correct measure.

[deleted]5 anni fa

glitchc5 anni fa

Same compiler? Same configuration? Same OS? There will be variations in the output. How much variation is reasonable?

fsflover5 anni fa

https://reproducible-builds.org/

AntiImperialist5 anni fa

antidocker5 anni fa

CivBase5 anni fa

At some point you just have to trust people to not intentionally lie in a trial. We have very stiff penalties for anyone caught lying in court, but there isn't much more we can realistically do.

jsjsbdkj5 anni fa

We have stiff penalties for civilians lying in court. Cops can lie in court, get caught, and still have the court accept their version of events as basis for proscution.

heavyset_go5 anni fa

Well, for voting, you just don't use machines. Counting by hand is an easy algorithm to audit, and literally any adult that can count is able to audit the process themselves.

razakel5 anni fa

It's staggering that a slot machine in Vegas has more official scrutiny than a voting machine.

pacamara6195 anni fa

It's not just like voting machines if it's just software. Just give it the same DNA as input and check if your results match the others.

[deleted]5 anni fa

mseidl5 anni fa

Nobody should use electronic voting.

Spivak5 anni fa

People keep saying that but electronic voting would be great! Set it up so the process is exactly like mail-in ballots except digital and you can’t have made it any worse. It’s not like emailing a PDF to be opened by an election official is any different than mailing a document to be opened by that same official.

ryukafalz5 anni fa

How many Nigerian princes have asked for your bank account number via physical mail? How about via email?

The internet lowers the barrier to certain forms of abuse substantially enough that I don't think you can so easily say you won't have made it any worse.

Spivak5 anni fa

I mean none because Gmail's spam filters are pretty good these days but I take your point. I'm not at all saying people should be trained to vote via unsolicited email. You would have to ask for digital voting when you register. And you would of course be able to check that your ballot was received through the same online form that already exists for mail-in ballots.

Right now you already give election officials your email for mail-in ballots and if there's a problem with your ballot they'll email you!

I think we could do a lot better than this when it comes to online voting but as a baseline optionally replacing the transport from physical mail to email I don't think would be the end of the world.

Ekaros5 anni fa

No one should use mail in voting either...

Spivak5 anni fa

Except for the sick, disabled, elderly, deceased, people overseas or out-of-town, anyone in the military that's deployed, people who don't have good access to transportation, people who work long continuous shifts like healthcare workers and firefighters, poll workers, people in jail awaiting their trials, people with stalkers...

Ekaros5 anni fa

Deceased? Why should dead people get to vote? Their votes should be first ones to be invalidated...

kazinator5 anni fa

> Mark Perlin, is said to have argued against source code analysis by claiming that the program, consisting of 170,000 lines of MATLAB code, is so dense it would take eight and a half years to review at a rate of ten lines an hour

What an deplorably dishonest argument to present to laypeople.

If there is some weakness in it so the accused has a way to plausibly deny the DNA test, it will take a consultant who has relevant competence something like week or two to find it.

Finding a problem doesn't require looking at every line of 170,000.

protomyth5 anni fa

I do wonder if the next step is check the compiler / interpreter the code is running on. Is Matlab assumed to be error free in this situation? Will certified compilers and CPUs be required in the future? Looks like someone could end up reviving a modern version of the Viper https://en.wikipedia.org/wiki/VIPER_microprocessor

fritzo5 anni fa

Code review is not the way to validate statistical inferences.

Even during internal review of statistical inference code, committers argue correctness through a combination of readable code, readable tests, and statistical validation on a combination of synthetic and real data. While I agree the TrueAllele source should be provided, the "zero-defects" bar is neither sufficient nor necessary for correctness of the inferences made.

akyu5 anni fa

> Code review is not the way to validate statistical inferences.

But it is a way to invalidate the inference, which is what the defense is interested in.

damagednoob5 anni fa

How much uncertainty in the system are you willing to live with if you were falsely accused of murder?

fritzo5 anni fa

That is a question for the judge or jury. The statistician's job is to provide a probability distribution, and to argue for the statistical correctness of that distribution based on similar inferences made on validation data.

damagednoob5 anni fa

In that case, at least with respect to the death penalty, "zero-defects" is precisely what the courts have moved towards.

onetimemanytime5 anni fa

Makes perfect sense. You have the right to cross-examine the witness, especially when you're being accused of murder.

lowbloodsugar5 anni fa

"The co-founder of the company, Mark Perlin, is said to have argued against source code analysis by claiming that the program, consisting of 170,000 lines of MATLAB code, is so dense it would take eight and a half years to review at a rate of ten lines an hour."

Well then the probability that it works correctly is zero.

femto1135 anni fa

Interesting that they're putting so much focus on the DNA piece, since the actual crime was witnessed by a police officer and there is circumstantial evidence tying the suspect to the weapon (it was found along the path they used to flee from the scene).

kovac5 anni fa

I don't think it's meaningful to let defendant review the code each trial. Such code reviews should have been done before it was ever allowed to be used in the legal system.

On the other hand, there's always the more practical option of getting the dna testing done by a second, third or even a tenth different approved company if the defendants want to pursue this line of reasoning. From what I understand, dna testing is statistical which means even if they find some bug in the code there will have to be extended debates on whether or not a particular bug affected the credibility of this test result. This is just bullocks.

Having said that, the excuse the software company is giving is a bit lame.

fxleach5 anni fa

> it would take eight and a half years to review at a rate of ten lines an hour

Wow, the co-founder's argument to not disclose the source code to the defendant was that it's too many lines of code. Also... ten lines reviewed in an hour!?

meowster5 anni fa

In the TV shows and books, opposing counsel always want to inundate the protagonist with truckloads of boxes full of printed papers when they could just handover a flashdrive instead. Are there any lawyers here who can speak to the accuracy of such portrayal?

It would seem the next logical step would be for every other lawyer to say they shouldn't hand over discovery because it would take to long for the otherside when they bury them in paperwork.

_Wintermute5 anni fa

Not sure if it's the best source, but according to a "Youtube laywer reacts" video [0], it's a common tactic but you can complain about it to the judge and request documents in a more appropriate form.

[0] https://youtu.be/spr5smxuO5E?t=1284

[deleted]5 anni fa

brutal_chaos_5 anni fa

Disclaimer, I am FOR seeing the source in these cases.

Say the defense finds the software in a very, very troubling state. Could that be telling enough to not trust the outcomes of the software? Assuming the DNA parts were correct, but the rest is junk, this is where the plaintiff would bring in their expert to counter the arguments?

Perhaps ^ is just a weak argument. I don't know how well versed courts are in these matters. (My guess is NOT that well versed because of the CFAA of the 80's up to Aaron Schwartz, and more).

My legal background is Law and Order, so I have no idea what I'm talking about, just curious.

michaelmrose5 anni fa

One would suppose that for a word of such import there would be a detailed analysis of the work product along with testing they had done to prove it actually worked.

They ought to ask for access to THAT in addition to the source code. One would imagine that it doesn't come with 170k lines of documentation or analysis and it would be orders of magnitude easier to review than the source code alone.

zyxzevn5 anni fa

All source code for important decisions should be open and be verified by different parties. Just as that for voting machines.

Abimelex5 anni fa

I'm not familiar with genome sequencing, but why are they even relying only on one program? Wouldn't it be possible to check it with different programs to reduce the possibility of failure here? And just assuming they find a possible bug, would that mean all the sentences based on the programs result will be obsolete?

LockAndLol5 anni fa

For the Europeans among us, I would like to remind you that there's an initiative to opensource all publicly funded code

https://publiccode.eu/

If you care, add your signature and vote for a party during the next elections that considers opensource at least somewhat important.

LatteLazy5 anni fa

We had a case in the uk with accounting software. Dozens of people were convicted over decades because the software couldn't be wrong. Until eventually someone actually checked.

https://www.bbc.co.uk/news/business-54384427

Thorentis5 anni fa

> If TrueAllele is found wanting, presumably that will not affect the dozen individuals said to have been exonerated by the software.

Buried right at the end, but an interesting thought. It would depend on the individual cases, but if there was a whole body of evidence vs. one DNA test, surely these cases would need to be retried?

clarle5 anni fa

IANAL but if the individuals were already found not guilty, I think double jeopardy laws would apply.

The prosecution wouldn't be able to try again unless they made an appeal that there's a substantially different crime.

duluca5 anni fa

Things are only going to get murkier when leveraging ML. All of this needs to become open source.

fritzo5 anni fa

Agreed, and with ML we'll need both open source and open training data.

pnathan5 anni fa

This is excellent! As a principle, we should always be able to properly critique the expert witnesses and analyses! Or, at least, hire an expert firm to do the critique. It at least restores a proper ability to challenge an assertion of guilt.

iamleppert5 anni fa

Any company who has software that is used in such a way should have independent review of their source code. It’s amazing to me they didn’t think of this originally and have the necessary endorsements already available for viewing.

dwg5 anni fa

> The maker of the software, Cybergenetics, has insisted in lower court proceedings that the program's source code is a trade secret.

Should this be, in general, a disqualifying condition when something is to be used as evidence?

rudylee5 anni fa

Why can't just they run the DNA test again in 2 or 3 other different software ? If all of them come back with the same result then we can assume that the first software is not buggy right ?

handoflixue5 anni fa

It could be that they're all copying the same Stack Exchange answer, or making the same basic mistake - If 50% of programmers do averages wrong, it's not ridiculous to claim that 3 in a row got it wrong. Tripling your costs and still having a 12.5% failure rate isn't great.

jxramos5 anni fa

Aren't there lower hanging fruits to go after? Like are there not competitor or predicate devices to run the same DNA against and see what it says as far as a match goes?

Mashimo5 anni fa

Yay, free code review?

johnnyfived5 anni fa

How would a team of independent reviewers even go about examining source code of this scale? How can you possibly find bugs that aren't super obvious?

pavel_lishin5 anni fa

> How would a team of independent reviewers even go about examining source code of this scale? How can you possibly find bugs that aren't super obvious?

Something tells me that 170k of Matlab code is going to contain plenty of obvious bugs.

twobitshifter5 anni fa

The defense team only needs to provide the jury with a reasonable doubt. They don’t have to prove that it’s entirely wrong, only the existence of mistakes that call into question the overall accuracy. The founder has already made their argument for them by claiming that the entire codebase is impossible to review. “since a complete review is not possible, we reviewed a sample of x files and found y errors in the code. It is obvious that the developers has not adhered to the strict software code quality review standards that are necessary when dealing with life and death situations.”

trynton5 anni fa

In such a serious case, once the accused was identified by TrueAllele, shouldn't a further test be carried out by a human to verify the results.

astura5 anni fa

There should be no such thing as a "trade secret" when talking about evidence that is used to convict someone. Correct judgement here.

amelius5 anni fa

How does this work? They only have to find 1 bug and the entire case is dropped / postponed?

carlmr5 anni fa

I guess they have to show this bug changes the result.

But if you look at e.g. the case against Toyota where they had expert witnesses from Carnegie Melon and NASA testifying that the code was a horrible mess, I think that was sufficient for a multi billion dollar fine. So I guess with the admission from the DNA company that their code is basically unreviewable, written in a language known for prototyping, if they get an expert witness to testify this it may be enough without showing how the software misbehaves exactly.

aaron6955 anni fa

This isn't a DNA testing kit as one would normally think -

"TrueAllele uses a hierarchical Bayesian probability model that adds genotype alleles, accounts for artifacts, and determines variance to explain STR data and derive parameter values and their uncertainty. The computer employs Markov chain Monte Carlo (MCMC) statistical sampling to solve the Bayesian equations. The resulting joint posterior probability provides marginal distributions for contributor genotypes, mixture weights, and other explanatory variables."

https://onlinelibrary.wiley.com/doi/full/10.1111/1556-4029.1...

Scandiravian5 anni fa

How is that different from a DNA testing kit "as one would normally think"?

known5 anni fa

Sounds rational since conviction should be on irrefutable evidence;

[deleted]5 anni fa

(4)