The linked report seems almost useless -- it doesn't say anything about an error rate or a sample size, so it's a mystery whether 9 out of 20 systems “fabricated information and made suggestions to patients' treatment plans” ten out of ten times, or one out of a thousand times.
If we just postulate that the systems have a high error rate, I wonder why they are being adopted. They seem extremely easy to test, so I don't see why doctors or hospitals or governments should be getting tricked into buying them if they suck.
> Of course the safest (first) option is the correct option from a liability standpoint, which is all a company should operate on since it's first responsibility is to protect the company for those that are still there.
Isn't this an unrealistically black-and-white mode of thinking? Humans are complicated and have many values and perceived responsibilities. It's not healthy for them to throw them all out and act as if they only have one responsibility that needs to be maximally upheld at all costs. They should balance their actions thoughtfully.
That rule is a rule that excludes many socially excluded nerds. So if there are more and more spaces like that, then the nerds that those spaces exclude, are probably going to go make new communities and attempt to keep out the people who just excluded them.
Your comment is framed like "giving a student a personal AI datacenter to carry with them" is unrealistic, but in fact it is easy for anyone with access to $1000-$2000 worth of compute to download and operate exactly that for free, with performance perhaps a year behind the state of the art.
This comment would make more sense if it were before the new wave of prediction markets, which are high-profile gambling products clearly largely made and popularized by true believers who think they are making the world a better place.
I think it's slightly less ridiculous than it sounds, because governments have much more power over their own citizens. As an American I would dramatically prefer the Chinese government to spy on me than the American government, because the Chinese government probably isn't going to do anything about whatever they find out.
(That logic breaks down somewhat in the case of explicitly negotiated surveillance sharing agreements.)
I don't really understand the criticism. The authors aren't claiming to have the strongest chess engine without search. They are just showing that they got a chess engine to a respectable level with their process, which is somewhat different from LC0. They do in fact explain that explicitly:
> Leela Chess Zero’s networks, which are trained with self-play and RL, achieve higher Elo ratings without using explicit search at test time than our transformers, which we trained via supervised learning. However, in contrast to our work, very strong chess performance (at low computational cost)
is the explicit goal of this open source project (which they have clearly achieved via domain-specific adaptations). We refer interested readers to [https://arxiv.org/abs/2409.12272] (which was published concurrently to our work) for details on the current state-of-the-art and a comparison against our network.
And I don't think the criticism of their writing is on point either. I don't think they are secretly implying that their engine is better than Stockfish. And it's 100% plausible for human masters to rigorously analyze many positions with engine assistance and correctly establish whether Stockfish's evaluation is right or not.
Actually, the main reason I used Nebula was that they advertised a credible-to-me promise that you could download and permanently delete your data upon request. That was some years ago, so I don't know if I would trust them today. But that was their claim, and I have no reason to believe they didn't delete my data.
I was interested to read this because some time ago I had my genome sequenced by Nebula. If you look at the lawsuit you can see that what Nebula did was use off-the-shelf third-party analytics products on their website, including recording analytics pings when users buy a kit, and pings when users use the Nebula website to browse Nebula's high-level analysis of their traits (leaking that the user has those traits to the analytics provider.)
This behavior represents a contemptible lack of respect for users' privacy, but it's important to distinguish it from Nebula selling access to users' genomes.
If kids could do math they would be able to divide the yearly American military aid to Israel ($18b last year) by the American population (340m), so they probably wouldn't conclude that fifty bucks per year per person was the main reason why their classmates are poor or their parents can't afford healthcare.
If we just postulate that the systems have a high error rate, I wonder why they are being adopted. They seem extremely easy to test, so I don't see why doctors or hospitals or governments should be getting tricked into buying them if they suck.