Did Claude increase bugs in rsync?(alexispurslane.github.io)
alexispurslane.github.io
Did Claude increase bugs in rsync?
https://alexispurslane.github.io/rsync-analysis/
584 comments
> I thought it would be a good idea to do the core structure for the new test suite in public on master first though given all the rage that has generated maybe that was a bad idea.
I don't entirely understand what this is saying. People wouldn't have been outraged if only the tests had been updated and/or he pushed solely on master - but he pushed breaking changes onto the release branch(es) too. Breaking workflows that have worked for years is a prime way to get people irate, and then seeing "Claude" in the commits just pours gasoline onto the fire.
I don't entirely understand what this is saying. People wouldn't have been outraged if only the tests had been updated and/or he pushed solely on master - but he pushed breaking changes onto the release branch(es) too. Breaking workflows that have worked for years is a prime way to get people irate, and then seeing "Claude" in the commits just pours gasoline onto the fire.
It seems that wasn't the Claude part, though I haven't seen a full analysis of exactly what broke. I also only saw one report: are there multiple, or do you just perceive that?
Rsync has many options: I can totally believe that fixing a bug in one place broke someone's usage, to be fair.
Rsync has many options: I can totally believe that fixing a bug in one place broke someone's usage, to be fair.
"yes, there were regressions in some use cases of rsync in the 3.4.3 release. I quite deliberately tried to err on the side of fixing security issues for that release, and there were some valid (but unusual) use cases that got caught up in the changes"
This should be the top comment.
I think it's pretty sad that he even had to write it. Quite a lot of judgement from people who aren't paying his bills.
I think it's pretty sad that he even had to write it. Quite a lot of judgement from people who aren't paying his bills.
Yeah a big reason you see so much pushback on clanker slop is that it's having (and there was certainly the expectation of it having) a negative impact on the ability of plenty of people to pay their bills.
Nice to know we've got slurs for LLMs and their users now. Very nice.
It's not a slur, clankers ain't people.
How do your achievements match up to the developer behind samba & rsync?
If corporations can be people, there is no reason why AI can't be considered a person.
Nobody human considers a corporation to be a person. It is merely a legal expedient.
I used to think that until I did a little digging into it. Specifically, corporations are considered legal persons because they are separate legal entities, thereby creating a wall between your personal affairs and your business. It protects your personal assets from any liabilities, debts, or lawsuits.
Also, since it's a separate entity, its lifetime is not tied to the owner. So if the owner dies, their shares are inherited by somebody else, and the company keeps operating.
It helps in raising money for business operations. A corporation raises capital by issuing and selling shares of stock. However, if a physical person did that, I think it would be called indentured servitude.
Also, since it's a separate entity, its lifetime is not tied to the owner. So if the owner dies, their shares are inherited by somebody else, and the company keeps operating.
It helps in raising money for business operations. A corporation raises capital by issuing and selling shares of stock. However, if a physical person did that, I think it would be called indentured servitude.
I'm confused about why you believe this contrasts with what the above poster said. You've described a bunch of practical reasons why this is legally expedient and also at least one that seems to contrast with your own concept of personhood
Yes, it does conflict with my own concept of personhood. I forgot to add that to my comment.
I was trying to show that it is not "merely a legal expedient", that corporate personhood had a specific purpose, and that it differed from a real person. I think that the confusion about legal personhood in corporations comes from how lawyers explain its existence. A couple of lawyers I've had explained it as, it's just like a person in the law, except where it's different.
The problem is that we haven't created a clear enough distinction between a natural person and a legal person. In many cases, corporations have rights but not the responsibility. For example, they have speech rights, but they don't go to jail when the corporation commits a crime. The judicial inequalities between ordinary people and rich people are even greater between natural persons and corporations.
I was trying to show that it is not "merely a legal expedient", that corporate personhood had a specific purpose, and that it differed from a real person. I think that the confusion about legal personhood in corporations comes from how lawyers explain its existence. A couple of lawyers I've had explained it as, it's just like a person in the law, except where it's different.
The problem is that we haven't created a clear enough distinction between a natural person and a legal person. In many cases, corporations have rights but not the responsibility. For example, they have speech rights, but they don't go to jail when the corporation commits a crime. The judicial inequalities between ordinary people and rich people are even greater between natural persons and corporations.
Yea, I mean, I think the reason people balk at corporate personhood are to do with both the iniquities committed by corporate actors and the fact that personhood is a really confusing model for all this
A model that treats what effectively amounts to a body of assets united by a charter as equivalent to a person - except when it isn't - is inherently confusing because these are not at all similar kinds of entities. While it's clear that this model has a purpose, I think people are right to point out that the equivalence is drawn by rather stilted logic and even more right to question whether the consequences of this legal framing are desirable from their perspective
A model that treats what effectively amounts to a body of assets united by a charter as equivalent to a person - except when it isn't - is inherently confusing because these are not at all similar kinds of entities. While it's clear that this model has a purpose, I think people are right to point out that the equivalence is drawn by rather stilted logic and even more right to question whether the consequences of this legal framing are desirable from their perspective
It takes something far more hollow and soulless even than a clanker to type a thing like this.
Beep boop. It comes from recognizing that, yet again, humans are way too full of themselves and are not as unique as they think they are.
Corporations are aggregations of people.
Are AIs?
Are AIs?
A corporation is a separate entity. They hire people to work for them, but any liabilities incurred by the Corporation are rarely passed through to the workers. If you pay attention, it's more likely for a worker to go to jail or be held responsible for a screw-up by the legal system than one of the suits at the top.
also people calling all AI is shite, might not have used top tier models. They put $20, tried Sonnet, got sloppy code and called it a day.
The title at least sounds less like judgement and more analysis and more about AI assistance (and claude in particular) than rsync. Maybe I am too used to postmortems!
I think they're talking about the whole twitter and github issue things.
> Now if any of the people posting the rage stuff want to actually review any of the code I’ve published and make constructive criticisms then that would be great!
When you quickly churn more lines of code in a few days than you changed in months, and then release them as a normal, not sure you're expecting "constructive criticism"
Also if I suspect the project is just slopping high amount of code without proper thought, I probably won't invest my time into reading those changes
When you quickly churn more lines of code in a few days than you changed in months, and then release them as a normal, not sure you're expecting "constructive criticism"
Also if I suspect the project is just slopping high amount of code without proper thought, I probably won't invest my time into reading those changes
I get that there's a lot of loud nonsense flying around about AI, both positive and negative, and I echo the sentiment that people should have some damn perspective when talking to FOSS maintainers, but I think writing a bunch of AI-assisted code that causes regressions and then responding to that by throwing out a strawman about how critics (with PhDs no less!) are telling him these things can't do anything at all and can't possibly understand how literally everything has fundamentally changed in the last few months sounds way more like a guy who has a motivated (and understandable - he's retired ffs) reason to... a little bit buy into the hype
I think he makes a lot of good points here, but also think that kind of statement is unlikely to assuage the real concerns of people using the software. I think people are more likely to fork rsync now rather than rely on a more diverged earlier alternative implementation though
I think he makes a lot of good points here, but also think that kind of statement is unlikely to assuage the real concerns of people using the software. I think people are more likely to fork rsync now rather than rely on a more diverged earlier alternative implementation though
Was just looking at commits and came across a commit and its revert
original commit: https://github.com/RsyncProject/rsync/commit/d046525de39315d...
```
- if (!ptr)
- ptr = malloc(num * size);
- else if (ptr == do_calloc)
+ if (!ptr || ptr == do_calloc)
Written with claude. This is a good example of what slips through LLM attention. It forces all allocations to be calloc as if it is a strict upgrade. For large and recursive allocations, this becomes a significant cost.
reverted in https://github.com/RsyncProject/rsync/commit/7db73ad9a1b8721...
if you read the description of revert half carefully, it's easy to tell that even that was written by an LLM .
I can understand the sentiment of whoever posted the original thread.
original commit: https://github.com/RsyncProject/rsync/commit/d046525de39315d...
```
- if (!ptr)
- ptr = malloc(num * size);
- else if (ptr == do_calloc)
+ if (!ptr || ptr == do_calloc)
ptr = calloc(num, size);
Written with claude. This is a good example of what slips through LLM attention. It forces all allocations to be calloc as if it is a strict upgrade. For large and recursive allocations, this becomes a significant cost.
reverted in https://github.com/RsyncProject/rsync/commit/7db73ad9a1b8721...
if you read the description of revert half carefully, it's easy to tell that even that was written by an LLM .
I can understand the sentiment of whoever posted the original thread.
Also the amount of commits is suspicious. In the last two months, rsync had about as much commits as in the last two years before that. Most of them written with claude. And then stuff like this is in there.
That's exactly what I'd expect when someone is excited about AI usage and becomes... well, sloppy.
That's exactly what I'd expect when someone is excited about AI usage and becomes... well, sloppy.
Tridge already explains this:
"Like many developers of open source packages I’ve been hit by a flood of security reports lately in my role as the rsync maintainer. Many of those reports are AI generated (not all though, there are some notable ones with very careful and high quality manual analysis).
As this flood started to get more intense I realised I needed to raise the defences on rsync a lot — we needed much more thorough test suites, code coverage analysis, CI testing on a lot more platforms, deliberate and thorough scanning for possible security issues (so I find at least some of them before other people!) and the addition of a whole lot of defence-in-depth hardening techniques. This is all a huge amount of work. "
https://medium.com/@tridge60/rsync-and-outrage-d9849599e5a0
"Like many developers of open source packages I’ve been hit by a flood of security reports lately in my role as the rsync maintainer. Many of those reports are AI generated (not all though, there are some notable ones with very careful and high quality manual analysis).
As this flood started to get more intense I realised I needed to raise the defences on rsync a lot — we needed much more thorough test suites, code coverage analysis, CI testing on a lot more platforms, deliberate and thorough scanning for possible security issues (so I find at least some of them before other people!) and the addition of a whole lot of defence-in-depth hardening techniques. This is all a huge amount of work. "
https://medium.com/@tridge60/rsync-and-outrage-d9849599e5a0
I think Tridge is simultaneously trying to be proactive and kinda giving too much credit to marketing. Anthropic has not been able to really give numbers or actual values on what Mythos can really do. It just waved Mythos in front of the public like a boogeyman screaming that AI is going to cause a security nightmare (and it has, but mostly through vibe coded trash from what I’ve noticed); I’m hard pressed to find their statement that they spent less than $20,000 to find a Kerberos bug in FreeBSD a compelling win without a lot more context and they seem disinclined to provide that data. I really do wonder what evidence they have provided to their approved partners, all of this smells…weird.
I honestly think the main problem is Tridge just failed at communicating any of this correctly and I don’t think the implication he gives that all of this was due to the urgency of the impending security apocalypse really holds water.
Why was all of this written straight to the master branch? Now that the release is out, why not better explain what the urgency of this release was? Why wasn’t he proactive in communicating this and instead let the mob make up their own story? I think a lot of people are inclined to give Tridge a lot of leeway due to the fact that he literally is the reason why rsync exists, but this was avoidable and I think the comment in his response post where he mentions that, “I’d rather be out sailing than working on rsync security issues, so I have reached for several AI tools to help with what needs to be done,” speaks volumes as to what is going on.
I honestly think the main problem is Tridge just failed at communicating any of this correctly and I don’t think the implication he gives that all of this was due to the urgency of the impending security apocalypse really holds water.
Why was all of this written straight to the master branch? Now that the release is out, why not better explain what the urgency of this release was? Why wasn’t he proactive in communicating this and instead let the mob make up their own story? I think a lot of people are inclined to give Tridge a lot of leeway due to the fact that he literally is the reason why rsync exists, but this was avoidable and I think the comment in his response post where he mentions that, “I’d rather be out sailing than working on rsync security issues, so I have reached for several AI tools to help with what needs to be done,” speaks volumes as to what is going on.
As a long-time open-source maintainer, I find all the second-guessing and armchair psychoanalysis here (not just in this comment, all over HN) about Tridge's motivations, state of mind, and so on incredibly off-putting.
Tridge doesn't owe anyone anything as far as rsync is concerned. Yet he is spending his time maintaining it, only to be attacked for his efforts.
To respond to the specific technical point, there really _is_ a flood of security reports arriving everywhere in the past few months. The jury is out on whether Mythos is that much better than alternatives, but even the publicly available models are _highly_ capable of finding real problems, and they are being employed to that end quite effectively. Here are the counts of security issues fixed in each monthly Go minor release going back to the start of 2024:
You can see that 2026 has been quite different from the previous years. There are plenty of other contemporaneous accounts from other security teams about the load increase they've seen (which again is almost entirely not Mythos).
Also, the number of reports we are receiving has gone up far faster than the number of actual vulnerabilities. Over the 75-month period from January 2020 to early April 2026, the final 30 days accounted for ~16% of the reports.
It is easy to believe that Tridge is seeing a similar flood of reports. More reports means more fixes means more code changes means more bugs.
Tridge doesn't owe anyone anything as far as rsync is concerned. Yet he is spending his time maintaining it, only to be attacked for his efforts.
To respond to the specific technical point, there really _is_ a flood of security reports arriving everywhere in the past few months. The jury is out on whether Mythos is that much better than alternatives, but even the publicly available models are _highly_ capable of finding real problems, and they are being employed to that end quite effectively. Here are the counts of security issues fixed in each monthly Go minor release going back to the start of 2024:
0 2024-01-09 Go 1.21.6, Go 1.20.13
0 2024-02-06 Go 1.21.7, Go 1.20.14
5 2024-03-05 Go 1.22.1, Go 1.21.8
1 2024-04-03 Go 1.22.2, Go 1.21.9
2 2024-05-07 Go 1.22.3, Go 1.21.10
2 2024-06-04 Go 1.22.4, Go 1.21.11
1 2024-07-02 Go 1.22.5, Go 1.21.12
0 2024-08-06 Go 1.22.6, Go 1.21.13
3 2024-09-05 Go 1.23.1, Go 1.22.7
0 2024-10-01 Go 1.23.2, Go 1.22.8
0 2024-11-06 Go 1.23.3, Go 1.22.9
0 2024-12-03 Go 1.23.4, Go 1.22.10
2 2025-01-16 Go 1.23.5, Go 1.22.11
1 2025-02-04 Go 1.23.6, Go 1.22.12
1 2025-03-04 Go 1.24.1, Go 1.23.7
1 2025-04-01 Go 1.24.2, Go 1.23.8
1 2025-05-06 Go 1.24.3, Go 1.23.9
3 2025-06-05 Go 1.24.4, Go 1.23.10
1 2025-07-08 Go 1.24.5, Go 1.23.11
2 2025-08-06 Go 1.24.6, Go 1.23.12
1 2025-09-03 Go 1.25.1, Go 1.24.7
10 2025-10-07 Go 1.25.2, Go 1.24.8
* 2025-10-13 Go 1.25.3, Go 1.24.9
0 2025-11-05 Go 1.25.4, Go 1.24.10
2 2025-12-02 Go 1.25.5, Go 1.24.11
6 2026-01-15 Go 1.25.6, Go 1.24.12
2 2026-02-04 Go 1.25.7, Go 1.24.13
5 2026-03-05 Go 1.26.1, Go 1.25.8
10 2026-04-07 Go 1.26.2, Go 1.25.9
11 2026-05-07 Go 1.26.3, Go 1.25.10
3 2026-06-02 Go 1.26.4, Go 1.25.11
You can see that 2026 has been quite different from the previous years. There are plenty of other contemporaneous accounts from other security teams about the load increase they've seen (which again is almost entirely not Mythos).
Also, the number of reports we are receiving has gone up far faster than the number of actual vulnerabilities. Over the 75-month period from January 2020 to early April 2026, the final 30 days accounted for ~16% of the reports.
It is easy to believe that Tridge is seeing a similar flood of reports. More reports means more fixes means more code changes means more bugs.
> Yet he is spending his time maintaining it, only to be attacked for his efforts.
Which, in general, is totally legit. Doing something voluntarily doesn't relieve you from criticism if what you are doing isn't good.
Which, in general, is totally legit. Doing something voluntarily doesn't relieve you from criticism if what you are doing isn't good.
You can criticize all you want, but he can also just stop maintaining it if he gets too annoyed by the criticism. Maybe that's a better outcome for you, idk.
Yes, I agree. Voluntarily forming a mob to flood issue trackers with garbage shouldn't relieve the mob members from receiving criticism.
Agreed. Just like one doesn't owe the society their voluntary work, the society doesn't owe one protection from criticism.
I follow Go security issues and many recent ones are consequences of features added to Go and also security researches following up on an area after one issue is found.
Recent examples are certification validation logic, one issue after an another... because it's a mess of thing to implement.
Recent examples are certification validation logic, one issue after an another... because it's a mess of thing to implement.
I agree, it's very off-putting, and I totally understand that the amount of reports are overwhelming for maintainers of popular libraries.
> More reports means more fixes means more code changes means more bugs.
Sounds like we'll be riding a downward spiral for the foreseeable future? It will be very interesting to see how stats like the ones you shared develop in the coming year(s).
From the article I find this a bit concerning:
> So: the Claude releases changed way more lines of code than historical ones, but didn't have more bugs. More code, same bugs. That's not what you'd expect if Claude were making things worse.
More code, same bugs, is a net negative, no? I mean unless it's strictly needed for the inherent complexity of the program. But I've seen a tokenizer written by Rob Pike and I've seen a tokenizer written by Claude.... they are not the same :D
> More reports means more fixes means more code changes means more bugs.
Sounds like we'll be riding a downward spiral for the foreseeable future? It will be very interesting to see how stats like the ones you shared develop in the coming year(s).
From the article I find this a bit concerning:
> So: the Claude releases changed way more lines of code than historical ones, but didn't have more bugs. More code, same bugs. That's not what you'd expect if Claude were making things worse.
More code, same bugs, is a net negative, no? I mean unless it's strictly needed for the inherent complexity of the program. But I've seen a tokenizer written by Rob Pike and I've seen a tokenizer written by Claude.... they are not the same :D
What Tridge says is that the "more code" is more fixes and more thorough test suites, not random changes made by LLMs.
> As a long-time open-source maintainer, I find all the second-guessing and armchair psychoanalysis here (not just in this comment, all over HN) about Tridge's motivations, state of mind, and so on incredibly off-putting.
Much of the language from both groups is incredibly off-putting, frankly. Tridge in his blog post describes people as "foaming at the mouth"?!
The rhetoric around this has gotten way too emotional from both groups.
I'm glad I'm just a hobbyist.
Much of the language from both groups is incredibly off-putting, frankly. Tridge in his blog post describes people as "foaming at the mouth"?!
The rhetoric around this has gotten way too emotional from both groups.
I'm glad I'm just a hobbyist.
Tridge in his blog post describes people as "foaming at the mouth"?!
Did you see the picture in the article where the user posted a picture of them strangling the maintainer? I think “foaming at the mouth” is probably gentler than how I would characterise that.
Did you see the picture in the article where the user posted a picture of them strangling the maintainer? I think “foaming at the mouth” is probably gentler than how I would characterise that.
IMHO, the whole episode is just embarrassing. I have no doubt he’s just trying to do the right thing. You can disagree with the tactics, but the vitriol is outrageous. rsync is a gift to the world and we should be grateful and mindful of how much it has been quietly woven into the fabric of computing. rsync is taken for granted. This is not okay.
> This is not okay.
Agreed. The way to address it though, is through calm analysis and reason. The emotional language from both groups is not helping.
If there's one problem with Claude et al, it's that it's all happened way too quickly for people to keep up. We're all at different stages of acceptance and I think that's what we're seeing manifest in the various discussions.
Agreed. The way to address it though, is through calm analysis and reason. The emotional language from both groups is not helping.
If there's one problem with Claude et al, it's that it's all happened way too quickly for people to keep up. We're all at different stages of acceptance and I think that's what we're seeing manifest in the various discussions.
>We're all at different stages of acceptance
I do hope you see the irony of accusing people of armchair psychology and then hitting us with the five stages of grief.
I trust rsync (which handles critical data on my system) because I know a veteran of 40 years wrote the code it runs. If I see code like the one above posted by the OP, that the author wouldn't have written, I start to pay attention. When I then read the blog post of him saying that he'd "rather go sailing than fix rsync issues", I start to question whether the software is still written in a way I can trust and where it's going quality wise.
The problem isn't this weird gaslighting attempt that we just haven't let Claude in our hearts and souls yet which you seem to have determined is inevitable (spoiler alert, it is not), it's that a bot wrote crappy code and I wasn't even aware I was running it and now don't know to what standard this project is held.
I do hope you see the irony of accusing people of armchair psychology and then hitting us with the five stages of grief.
I trust rsync (which handles critical data on my system) because I know a veteran of 40 years wrote the code it runs. If I see code like the one above posted by the OP, that the author wouldn't have written, I start to pay attention. When I then read the blog post of him saying that he'd "rather go sailing than fix rsync issues", I start to question whether the software is still written in a way I can trust and where it's going quality wise.
The problem isn't this weird gaslighting attempt that we just haven't let Claude in our hearts and souls yet which you seem to have determined is inevitable (spoiler alert, it is not), it's that a bot wrote crappy code and I wasn't even aware I was running it and now don't know to what standard this project is held.
> If I see code like the one above posted by the OP, that the author wouldn't have written, I start to pay attention.
Except the author did write it. https://github.com/RsyncProject/rsync/issues/959#issuecommen...
Which is part of the problem with all of this nonsense right now - everyone is running off of emotion and not looking to see if what is being said is actually true. Which is somewhat ironic, considering the message of the article.
Except the author did write it. https://github.com/RsyncProject/rsync/issues/959#issuecommen...
Which is part of the problem with all of this nonsense right now - everyone is running off of emotion and not looking to see if what is being said is actually true. Which is somewhat ironic, considering the message of the article.
> I do hope you see the irony of accusing people of armchair psychology and then hitting us with the five stages of grief.
I just want to point out that those were two different commenters.
I just want to point out that those were two different commenters.
> The problem isn't this weird gaslighting attempt that we just haven't let Claude in our hearts and souls yet which you seem to have determined is inevitable (spoiler alert, it is not),
I don't believe it's inevitable and in fact, I'm thoroughly against the use of tools like Claude.
My reference to "different stages of acceptance" was only to indicate that people have embraced these things to varying degrees, and it that it seems to be this difference which is causing conflict in discussions like this. (I doubt I will ever fully accept it. A lot needs to change for that to happen).
I didn't really have the "five stages of grief" in mind when I wrote it.
I don't believe it's inevitable and in fact, I'm thoroughly against the use of tools like Claude.
My reference to "different stages of acceptance" was only to indicate that people have embraced these things to varying degrees, and it that it seems to be this difference which is causing conflict in discussions like this. (I doubt I will ever fully accept it. A lot needs to change for that to happen).
I didn't really have the "five stages of grief" in mind when I wrote it.
> As a long-time open-source maintainer, I find all the second-guessing and armchair psychoanalysis here (not just in this comment, all over HN) about Tridge's motivations, state of mind, and so on incredibly off-putting.
I agree that the entire episode is obscene, but I am also unsure of what to do here either. On some level this is the same problem movie stars run into. I agree that guessing or waxing about the motivations of anyone is a nosy and overall unproductive exercise (yet paparazzi exist because of this very human behavior), but I also think that there is a modest duty owed to users to explain things.
> Tridge doesn't owe anyone anything as far as rsync is concerned. Yet he is spending his time maintaining it, only to be attacked for his efforts.
I am reminded of this piece: https://mikemcquaid.com/open-source-maintainers-owe-you-noth...
Which, I empathize with, but I fundamentally disagree that maintainers owe users nothing. I will die on that hill. If you are getting to that point where you actively loathe working on the project, I agree you should be able to walk away. However, I strongly believe that when you create something for people to use that there’s an implicit social contract about how to go about doing certain things.
I suppose in a very extreme and intentionally histrionic example, having a project carry the MIT license, getting frustrated and then changing the project to delete the entire system is a crime. The average person and the courts don’t care if the license is “as-is”. There is a duty that is understood that you don’t do that and I think we need to make it clear what that duty is for OSS.
Ultimately, though, I think this is all symptomatic of the fact that the OSS model has gaps that the increase in security reports whether AI generated or not has exerted more pressure on. I have certainly been on the receiving end of a lot of frivolous security reports that were discarded because it was obvious that it was just someone with a security scanner wandering around the Internet. You still have to review that nonsense and it eats into your time. Doing this on your own time, without pay and having to listen to the peanut gallery is just infuriating.
Is any business built on top of rsync going to donate their money in a sustainable manner?
I agree that the entire episode is obscene, but I am also unsure of what to do here either. On some level this is the same problem movie stars run into. I agree that guessing or waxing about the motivations of anyone is a nosy and overall unproductive exercise (yet paparazzi exist because of this very human behavior), but I also think that there is a modest duty owed to users to explain things.
> Tridge doesn't owe anyone anything as far as rsync is concerned. Yet he is spending his time maintaining it, only to be attacked for his efforts.
I am reminded of this piece: https://mikemcquaid.com/open-source-maintainers-owe-you-noth...
Which, I empathize with, but I fundamentally disagree that maintainers owe users nothing. I will die on that hill. If you are getting to that point where you actively loathe working on the project, I agree you should be able to walk away. However, I strongly believe that when you create something for people to use that there’s an implicit social contract about how to go about doing certain things.
I suppose in a very extreme and intentionally histrionic example, having a project carry the MIT license, getting frustrated and then changing the project to delete the entire system is a crime. The average person and the courts don’t care if the license is “as-is”. There is a duty that is understood that you don’t do that and I think we need to make it clear what that duty is for OSS.
Ultimately, though, I think this is all symptomatic of the fact that the OSS model has gaps that the increase in security reports whether AI generated or not has exerted more pressure on. I have certainly been on the receiving end of a lot of frivolous security reports that were discarded because it was obvious that it was just someone with a security scanner wandering around the Internet. You still have to review that nonsense and it eats into your time. Doing this on your own time, without pay and having to listen to the peanut gallery is just infuriating.
Is any business built on top of rsync going to donate their money in a sustainable manner?
> However, I strongly believe that when you create something for people to use that there’s an implicit social contract about how to go about doing certain things.
Wow.
The entitlement in this statement is outrageous.
Wow.
The entitlement in this statement is outrageous.
> I also think that there is a modest duty owed to users to explain things.
> I fundamentally disagree that maintainers owe users nothing.
> I strongly believe that when you create something for people to use that there’s an implicit social contract about how to go about doing certain things.
do you realize how unhinged this all reads like?
there is no duty. nothing is owed to no one. there is no implicit anything. this is all happening in your head. you are making up things that don't exist. the social contract is not a real thing either. the only contract you can have with the author of rsync is the GNU GENERAL PUBLIC LICENSE Version 3, and then, only when you get a copy of rsync.
> getting frustrated and then changing the project to delete the entire system is a crime
boop: strawman argument — you have been disqualified
> Is any business built on top of rsync going to donate their money in a sustainable manner?
does it matter? do you have an invoice for rsync?
the author wrote it themselves, he is retired, and sailing. unless google is buying him a new boat, i doubt he gives a crap what anyone has to offer.
truly obscene is the fabricated idea that you are owed anything after downloading code from github.
> I am also unsure of what to do here either.
touch grass?
> I fundamentally disagree that maintainers owe users nothing.
> I strongly believe that when you create something for people to use that there’s an implicit social contract about how to go about doing certain things.
do you realize how unhinged this all reads like?
there is no duty. nothing is owed to no one. there is no implicit anything. this is all happening in your head. you are making up things that don't exist. the social contract is not a real thing either. the only contract you can have with the author of rsync is the GNU GENERAL PUBLIC LICENSE Version 3, and then, only when you get a copy of rsync.
> getting frustrated and then changing the project to delete the entire system is a crime
boop: strawman argument — you have been disqualified
> Is any business built on top of rsync going to donate their money in a sustainable manner?
does it matter? do you have an invoice for rsync?
the author wrote it themselves, he is retired, and sailing. unless google is buying him a new boat, i doubt he gives a crap what anyone has to offer.
truly obscene is the fabricated idea that you are owed anything after downloading code from github.
> I am also unsure of what to do here either.
touch grass?
> the courts don’t care if the license is “as-is”.
There isn't any case law to show that. Certainly not in the age of AI. On the criminal side, the CFAA requires "intentionally causes damage" and that's entirely impossible to prove in the age of AI. On the civil side, liability waivers and warranty disclaimers generally cannot shield intentional or willful misconduct or gross negligence.
There isn't any case law to show that. Certainly not in the age of AI. On the criminal side, the CFAA requires "intentionally causes damage" and that's entirely impossible to prove in the age of AI. On the civil side, liability waivers and warranty disclaimers generally cannot shield intentional or willful misconduct or gross negligence.
Yeah the maintainers don’t owe users nothing is a disgusting sentiment that doesn’t stand real scrutiny. There is a social contract here. If you want to be respected and get recognized as “tridge” or whatever your name is, you owe the people that recognize you and that wider community in general.
First off: I don't agree that there's a social contract here at all. That's just some imaginary thing that you (and others) have decided exists. It's funny how lots of people who aren't open source maintainers seem to think it's ok to make up social contracts for other people without their consent.
But ok, let's just pretend for a second that maintainers have indeed entered into some sort of social contract that gives them an obligation to support their software, uncompensated. But if we have this contract, then it cuts both ways. The users then have entered into a social contract of their own: they agree to treat me with respect when they deal with me, to not act entitled, to not demand things of me, to not be rude, and to do their part in being a helpful, productive partner in helping to solve any issues they report.
If a user breaks their part of the contract, then I have no obligation to fulfill my side of it.
It's a bit bizarre to me that non-maintainers have decided to invent some sort of "social contract" that benefits them (while putting a sizeable burden on maintainers), but seem to think that they aren't entering into a social contract of their own when they decide to use the software. (And that there are consequences for not upholding the user side of the social contract.)
Put another way: in contract law (in the US, at least), there's the concept of "consideration". It's the idea that both parties are getting something out of a deal. Some of that can be monetary, but it can also be other things. If a contract is one sided, that is, if one party isn't getting any consideration, then the contract can often be unenforceable.
That seems to be what people like you are doing here: requiring that open source maintainers enter into a social contract, but not give them any consideration in return for it. (And no, some sort of ill-defined concepts like "reputation" or "large user base" don't pass my threshold for meaningful consideration.)
That's one more thing, even: contracts are voluntary. All involved parties must agree for there to be a contract. I don't agree to your bullshit contract of one-sided obligation, so there is no contract.
But ok, let's just pretend for a second that maintainers have indeed entered into some sort of social contract that gives them an obligation to support their software, uncompensated. But if we have this contract, then it cuts both ways. The users then have entered into a social contract of their own: they agree to treat me with respect when they deal with me, to not act entitled, to not demand things of me, to not be rude, and to do their part in being a helpful, productive partner in helping to solve any issues they report.
If a user breaks their part of the contract, then I have no obligation to fulfill my side of it.
It's a bit bizarre to me that non-maintainers have decided to invent some sort of "social contract" that benefits them (while putting a sizeable burden on maintainers), but seem to think that they aren't entering into a social contract of their own when they decide to use the software. (And that there are consequences for not upholding the user side of the social contract.)
Put another way: in contract law (in the US, at least), there's the concept of "consideration". It's the idea that both parties are getting something out of a deal. Some of that can be monetary, but it can also be other things. If a contract is one sided, that is, if one party isn't getting any consideration, then the contract can often be unenforceable.
That seems to be what people like you are doing here: requiring that open source maintainers enter into a social contract, but not give them any consideration in return for it. (And no, some sort of ill-defined concepts like "reputation" or "large user base" don't pass my threshold for meaningful consideration.)
That's one more thing, even: contracts are voluntary. All involved parties must agree for there to be a contract. I don't agree to your bullshit contract of one-sided obligation, so there is no contract.
This. Best writeup I've seen on the topic of entitled/abusive users. You should publish this as a blog post or launch some sort of campaign or something, something people can refer to. I haven't encountered entitled users myself, but my gawd, I'm so annoyed at users who feel entitled to other open source maintainers. I'm raging with a drive to protest against people who treat the rsync maintainer with such disrespect.
This is great, but I can shorten it for you for times in the future you need to deploy it: "DM for my rate card".
No there isn't.
I just cannot understand this logic, can you explain why there is no responsibility whatsoever on the part of a maintainer towards the users?
Selling a toaster has an implicit warranty of merchantability. Society expects that if you sell me something, it should have certain promises. Yes, there’s no monetary exchange here, the work is given gratis, but there’s still a relationship and an interaction here and I think it is clear some people, like myself, believe that there are implied expectations. Just because it is “free” doesn’t mean it allows one to have a seemingly psychopathic attitude on the matter. It doesn’t absolve people of societal obligations.
I read that article by Mike McQuaid and I don’t get the impression that, “Yes, project maintainers should be allowed to run projects as they see fit and they put up with a lot of drive-by insults and hostile users. You don’t understand how hard all of this is and I’m doing it for free.” I get, “I hate my users and you should be grateful that I give you anything.”
Selling a toaster has an implicit warranty of merchantability. Society expects that if you sell me something, it should have certain promises. Yes, there’s no monetary exchange here, the work is given gratis, but there’s still a relationship and an interaction here and I think it is clear some people, like myself, believe that there are implied expectations. Just because it is “free” doesn’t mean it allows one to have a seemingly psychopathic attitude on the matter. It doesn’t absolve people of societal obligations.
I read that article by Mike McQuaid and I don’t get the impression that, “Yes, project maintainers should be allowed to run projects as they see fit and they put up with a lot of drive-by insults and hostile users. You don’t understand how hard all of this is and I’m doing it for free.” I get, “I hate my users and you should be grateful that I give you anything.”
If I hated my users I wouldn’t work on Homebrew for 17 years. I do hate a small subset of hostile users.
The selling metaphor doesn’t work. Homebrew is not sold and its license, effectively a EULA, discloses all warranties because it is not sold and we are not paid a wage to build it.
I have also built a bunch of proprietary software for money where my obligations are different. I also enjoy that and my responsibilities differ there.
Users should be grateful that they are given anything. We do not get anything from their use. For the vast majority, it is a one way relationship (contributors excluded of course).
If they don’t like the choices made by me or the project: they can fork it. They won’t, though, because the closest friend of entitlement is laziness. They can use Nix or MacPorts instead which may be a better fit for them and, if they are not contributing, does not disadvantage Homebrew.
The selling metaphor doesn’t work. Homebrew is not sold and its license, effectively a EULA, discloses all warranties because it is not sold and we are not paid a wage to build it.
I have also built a bunch of proprietary software for money where my obligations are different. I also enjoy that and my responsibilities differ there.
Users should be grateful that they are given anything. We do not get anything from their use. For the vast majority, it is a one way relationship (contributors excluded of course).
If they don’t like the choices made by me or the project: they can fork it. They won’t, though, because the closest friend of entitlement is laziness. They can use Nix or MacPorts instead which may be a better fit for them and, if they are not contributing, does not disadvantage Homebrew.
Thanks for chiming in. I appreciate that this is the position of you and a large chunk of folks, but I don’t think I’m ever going to fully understand it.
If you don’t mind me probing a little further, what is the motivation to work on it?
> they can fork it
I get that, but I also think it is too pat a narrative at the same time. I think the success of the project is both a testament to the effort that you and the Homebrew team have put into it. It is also an example of just how much effort any project really takes; this stuff doesn’t set itself up nor do all the patching required to make sure things behave as well as they do.
If you don’t mind me probing a little further, what is the motivation to work on it?
> they can fork it
I get that, but I also think it is too pat a narrative at the same time. I think the success of the project is both a testament to the effort that you and the Homebrew team have put into it. It is also an example of just how much effort any project really takes; this stuff doesn’t set itself up nor do all the patching required to make sure things behave as well as they do.
> If you don’t mind me probing a little further, what is the motivation to work on it?
Not the person you're replying to, but I do it because it's fun. Programming is a passion of mine, and has been a part of my life since my dad gave me a book on BASIC when I was 8 years old. I love solving problems with code. Giving it away as open source is, in a way, philanthropy to me, with the hope that at least some of the things I create are useful to others. There's also a bit of a "political" aspect to it, in that I think it is bad for society for all useful programs to be locked up in proprietary software, making everyone dependent on profit-seeking corporations (whose interests and incentives are often hostile to their users) to provide the software they need to use in their daily lives. My work is a small contribution to combat that.
That joy I feel hits a wall when I run into an entitled, lazy user who thinks that I owe them something more than what I've already given. If most users were like that, I just wouldn't do it. Or at most I'd do it, releasing under a pseudonym, and have no public issue tracker, pull request mechanism, or public contact information. That would make the project worse, not better, of course, but the most important thing to me is my mental health and my happiness. If that's selfish, so be it.
> > they can fork it
> I get that, but I also think it is too pat a narrative at the same time.
I'm not sure what you expect someone to do with that statement. So what if it's "too pat"? That's the reality of the situation. It's the maintainer's way or the highway. If you don't like it, then open source has a truly wonderful escape hatch that proprietary software doesn't: you can fork and go your own way with it.
Many open source communities have problems, certainly, but I think many of the better ones are the some of the closest things we have to true meritocracies. If you do the work, and the work is good and valued, you get a say. If you don't, you don't. And yes, I would say "providing good, helpful, actionable feedback" can be part of "doing the work", so people who don't write code can have a say, depending on how well they are able to provide value to the process. But people who just want to take: no, they don't get a say, and that's exactly how it should -- and must -- be.
Not the person you're replying to, but I do it because it's fun. Programming is a passion of mine, and has been a part of my life since my dad gave me a book on BASIC when I was 8 years old. I love solving problems with code. Giving it away as open source is, in a way, philanthropy to me, with the hope that at least some of the things I create are useful to others. There's also a bit of a "political" aspect to it, in that I think it is bad for society for all useful programs to be locked up in proprietary software, making everyone dependent on profit-seeking corporations (whose interests and incentives are often hostile to their users) to provide the software they need to use in their daily lives. My work is a small contribution to combat that.
That joy I feel hits a wall when I run into an entitled, lazy user who thinks that I owe them something more than what I've already given. If most users were like that, I just wouldn't do it. Or at most I'd do it, releasing under a pseudonym, and have no public issue tracker, pull request mechanism, or public contact information. That would make the project worse, not better, of course, but the most important thing to me is my mental health and my happiness. If that's selfish, so be it.
> > they can fork it
> I get that, but I also think it is too pat a narrative at the same time.
I'm not sure what you expect someone to do with that statement. So what if it's "too pat"? That's the reality of the situation. It's the maintainer's way or the highway. If you don't like it, then open source has a truly wonderful escape hatch that proprietary software doesn't: you can fork and go your own way with it.
Many open source communities have problems, certainly, but I think many of the better ones are the some of the closest things we have to true meritocracies. If you do the work, and the work is good and valued, you get a say. If you don't, you don't. And yes, I would say "providing good, helpful, actionable feedback" can be part of "doing the work", so people who don't write code can have a say, depending on how well they are able to provide value to the process. But people who just want to take: no, they don't get a say, and that's exactly how it should -- and must -- be.
kelnos beat me to it but: because working on it is fun. I don't think most people in our industry will understand decade+ open source maintainers of large projects and that's fine. I don't understand how designers can do what they do either. I would say that those of us who stick around and don't burn out quickly tend to share my ideas of "owe nothing" and "have fun" much more than "I must make sure I am always responsible and responsive to my users desires" who tend to seem permanently stressed before burning out.
I use my words carefully: I don't "owe" my users anything but that doesn't mean I don't "give" them anything. It's charity as opposed to taxes; I do so freely and on my own terms rather than obligation.
On forking: yes, it's a lot of work and forking would also be a lot of work. That's exactly the point. Lots of people over the years could have forked Homebrew but no meaningful forks have taken off because those who are most dissatisfied with our decisions are least willing to do the work to solve these problems.
Hope that clarifies. Thanks for the polite discussion :)
I use my words carefully: I don't "owe" my users anything but that doesn't mean I don't "give" them anything. It's charity as opposed to taxes; I do so freely and on my own terms rather than obligation.
On forking: yes, it's a lot of work and forking would also be a lot of work. That's exactly the point. Lots of people over the years could have forked Homebrew but no meaningful forks have taken off because those who are most dissatisfied with our decisions are least willing to do the work to solve these problems.
Hope that clarifies. Thanks for the polite discussion :)
> Selling a toaster has an implicit warranty of merchantability. Society expects that if you sell me something, it should have certain promises. Yes, there’s no monetary exchange here, the work is given gratis, but there’s still a relationship and an interaction here and I think it is clear some people, like myself, believe that there are implied expectations.
No there isn't.
Pay money and there's a contract.
Anything else is in your head.
No there isn't.
Pay money and there's a contract.
Anything else is in your head.
If you induce someone to expend resources you can have liability even if those resources are not a payment to you. You can’t license your way out liability if you advertised, formally or informally, certain features and functionality that cause people to act on that advertisement. It’s called reliance interest. It’s an actual legal principle with case law supporting it.
full-disclosure: i skimmed the wiki on reliance damages, and concluded you're wrong. it goes something like this: reliance damages require you produce a contract, or some other evidence, that demonstrates you were promised some thing you did not receive, or some outcome you didn't experience. essentially, your claim is: a README file has more standing, in a court of law, than the LICENSE file sitting next to it at the root. cute, but preposterous.
anyway, to the gist of this reply: you disagree with the license conditions. an important, but rather obvious, observation to be had is that, the rights the LICENSE offers, are contingent on your acceptance of the LICENSE conditions. one cannot be had without the other.
the LICENSE is real, it's a contract, and is in effect the moment you obtain a copy.
> You can’t [un-]license your way out [of the] liability if you [copied], formally or informally, [wares] that [you have no rights to, because you disagreed with its license conditions].
anyway, to the gist of this reply: you disagree with the license conditions. an important, but rather obvious, observation to be had is that, the rights the LICENSE offers, are contingent on your acceptance of the LICENSE conditions. one cannot be had without the other.
the LICENSE is real, it's a contract, and is in effect the moment you obtain a copy.
> You can’t [un-]license your way out [of the] liability if you [copied], formally or informally, [wares] that [you have no rights to, because you disagreed with its license conditions].
Can you cite the case law about an open source project having a reliance interest?
Fortunately this is completely wrong.
> Selling a toaster has an implicit warranty of merchantability.
Why would you think this is worth mentioning here?
Instead of explaining, just try to do something, that people actually use, for free, in the open, for some time. It doesn't have to be software, can be work for a nonprofit or a charity etc. I'm sure you will be enlightened.
Why would you think this is worth mentioning here?
Instead of explaining, just try to do something, that people actually use, for free, in the open, for some time. It doesn't have to be software, can be work for a nonprofit or a charity etc. I'm sure you will be enlightened.
I volunteer and I don't tell people or believe they should be grateful that an event is happening because of the volunteers. I just don't find this logic compelling in the same way that you don't find my logic compelling either.
It's not about telling other people how they should feel, it's about managing your own emotions as an unpaid maintainer of a somewhat public good facing unreasonably entitled members of public (often not even users of your product).
I'm a little confused at this comment, because the person you're replying to didn't say anything to the effect of what you're arguing against here.
That's not the argument. No one is asking users to be grateful for OSS. They just want users to not be dicks to the maintainer.
Would you continue volunteering if the beneficiaries spat in your face and cursed you out for it?
Would you continue volunteering if the beneficiaries spat in your face and cursed you out for it?
That toaster example sounds so nonsensical that I'm expecting you to deliver on your indirect promise of backing that up with evidence because of this special relationship of ours you established via interaction so these expectations are obviously implied by you commenting here.
Please continue.
Please continue.
Some people do not realize that they're in a parasocial relationships with content creators like streamers and youtubers and feel that it is reasonable to have expectations. For me, applying your argument, that there is some responsibility for a creator towards their users, within that domain seems farfetched. Like, I can wish that they'd continue producing worthwhile content but apart from that, how would their responsibility toward me actually manifest itself?
You don't have the impression that project managers should "be allowed" to run projects "as they see fit"?
it would save everyone a lot of hurt feelings, and unexpected surprises, if access to open source software was treated as a privilege, instead of treating it like a right
You have given the maintainer nothing. There is no relationship, no interaction. If you want to change open source code fork it and do as you please. No one owes you free labor.
> can you explain why there is no responsibility whatsoever on the part of a maintainer towards the users?
Because I don't. It's that simple. There is nothing that says I have a responsibility, and the license I release under even makes it clear and explicit that I have no responsibility. So I don't.
If you are going to claim that I do have a responsibility, then the onus is on you to present some solid, convincing, extraordinary evidence or argumentation to support that. And you haven't succeeded in doing so.
> Selling
That's part of it, right there. If I sell my open source software, then yes, I may have created an implied warranty of merchantability, even if my license disclaims that.
But if I haven't sold it to you, then no such warranty or obligation exists.
> Yes, there’s no monetary exchange here, the work is given gratis, but there’s still a relationship and an interaction here
So you admit that, but seem to ignore the idea that there's a difference between selling something and giving it away for free. I fundamentally disagree with that. If I give away something for free, the person accepting it has zero claim on me or my time. If I sell something, then there's some claim there, depending on the terms of sale that we both agreed to before I took payment.
> It doesn’t absolve people of societal obligations.
This is something you've invented out of whole cloth. There's no societal obligation to maintain something (for free) that you've given away for free. And on top of that, there's no societal obligation to deal with demanding, entitled, sometimes angry people, who want more of your time for free.
Let's actually look at it from a paid perspective. Let's say I release some software (open- or closed-source; I suppose the distinction doesn't matter for this example), and also offer paid support for that software. Some people use it without paying for support, some people pay for support. Let's say some of the people who are paying for support are demanding and rude when reporting issues and asking for fixes. Even then, I still don't have to put up with it. I can "fire" those customers if I want, either by cancelling and refunding their remaining support contract, or by deciding not to renew them when their current contract runs out.
I don't think anyone would reasonably require a company to continue to have a business relationship with a customer that is causing too many problems for them. I think the reason we are fine with this concept is because there's a remedy that gives both parties something: if we refund the customer some portion or all of what they've paid, we consider that a reasonable way to terminate that relationship. With gratis open source software, there's no such monetary arrangement, so it feels a fuzzier what the author-user relationship even is. But to me, this makes an even stronger case for the idea that open source maintainers have no obligations to their users, aside from any that they voluntarily take on, and can also decide to terminate at any point they like.
Because I don't. It's that simple. There is nothing that says I have a responsibility, and the license I release under even makes it clear and explicit that I have no responsibility. So I don't.
If you are going to claim that I do have a responsibility, then the onus is on you to present some solid, convincing, extraordinary evidence or argumentation to support that. And you haven't succeeded in doing so.
> Selling
That's part of it, right there. If I sell my open source software, then yes, I may have created an implied warranty of merchantability, even if my license disclaims that.
But if I haven't sold it to you, then no such warranty or obligation exists.
> Yes, there’s no monetary exchange here, the work is given gratis, but there’s still a relationship and an interaction here
So you admit that, but seem to ignore the idea that there's a difference between selling something and giving it away for free. I fundamentally disagree with that. If I give away something for free, the person accepting it has zero claim on me or my time. If I sell something, then there's some claim there, depending on the terms of sale that we both agreed to before I took payment.
> It doesn’t absolve people of societal obligations.
This is something you've invented out of whole cloth. There's no societal obligation to maintain something (for free) that you've given away for free. And on top of that, there's no societal obligation to deal with demanding, entitled, sometimes angry people, who want more of your time for free.
Let's actually look at it from a paid perspective. Let's say I release some software (open- or closed-source; I suppose the distinction doesn't matter for this example), and also offer paid support for that software. Some people use it without paying for support, some people pay for support. Let's say some of the people who are paying for support are demanding and rude when reporting issues and asking for fixes. Even then, I still don't have to put up with it. I can "fire" those customers if I want, either by cancelling and refunding their remaining support contract, or by deciding not to renew them when their current contract runs out.
I don't think anyone would reasonably require a company to continue to have a business relationship with a customer that is causing too many problems for them. I think the reason we are fine with this concept is because there's a remedy that gives both parties something: if we refund the customer some portion or all of what they've paid, we consider that a reasonable way to terminate that relationship. With gratis open source software, there's no such monetary arrangement, so it feels a fuzzier what the author-user relationship even is. But to me, this makes an even stronger case for the idea that open source maintainers have no obligations to their users, aside from any that they voluntarily take on, and can also decide to terminate at any point they like.
> “solid, convincing, extraordinary evidence or argumentation to support that.”
Just ordinary evidence. If there was a charity event which asked for a volunteer to organise drinks, and you volunteered, and then there were no drinks, and you said “I don’t owe you anything stop being entitled, if you want an event with drinks you can fork the idea and organise your own”, people would be unhappy and reasonably so. It’s not that you had a legal obligation to do that work, it’s that you told everyone you would and that stopped other people from doing it.
If rsync had no maintainer and someone publicly offered to take it on and maintain it, that would also block other people taking that spot. It stops people investing time effort and money into a fork or replacement to an abandoned project. If the volunteer then either didn’t do anything or wrecked it and said “I don’t owe you anything etc.” that would be bad in a similar way.
If you want to be able to tell people you are the maintainer, that the thing is maintained, and you get to control what happens to a widely used project, you can’t really stand by the position “why did people expect me to maintain it? I only told them I would maintain it, why would they believe me, that’s not fair”.
Make it clear that it’s abandonware and has no maintainer, and you can totally uphold the “not my problem, says so in the license, deal with it” position. But if your thing becomes popular then you should expect a company like RedHat to fork it into ‘redsync’ and run it their way as their project, not look to you as ‘upstream’ and sideline you completely. Which is what a lot of open source people say they want but don’t behave as if they want that. Probably because there actually is some prestige and power and status and reputation involved, even though people try to claim there isn’t.
Just ordinary evidence. If there was a charity event which asked for a volunteer to organise drinks, and you volunteered, and then there were no drinks, and you said “I don’t owe you anything stop being entitled, if you want an event with drinks you can fork the idea and organise your own”, people would be unhappy and reasonably so. It’s not that you had a legal obligation to do that work, it’s that you told everyone you would and that stopped other people from doing it.
If rsync had no maintainer and someone publicly offered to take it on and maintain it, that would also block other people taking that spot. It stops people investing time effort and money into a fork or replacement to an abandoned project. If the volunteer then either didn’t do anything or wrecked it and said “I don’t owe you anything etc.” that would be bad in a similar way.
If you want to be able to tell people you are the maintainer, that the thing is maintained, and you get to control what happens to a widely used project, you can’t really stand by the position “why did people expect me to maintain it? I only told them I would maintain it, why would they believe me, that’s not fair”.
Make it clear that it’s abandonware and has no maintainer, and you can totally uphold the “not my problem, says so in the license, deal with it” position. But if your thing becomes popular then you should expect a company like RedHat to fork it into ‘redsync’ and run it their way as their project, not look to you as ‘upstream’ and sideline you completely. Which is what a lot of open source people say they want but don’t behave as if they want that. Probably because there actually is some prestige and power and status and reputation involved, even though people try to claim there isn’t.
That metaphor doesn't operate here. People are building stuff and making it available. Nobody's making a commitment. Nobody's "volunteering" for you.
This "social contract" seems to be vocabulary that people use to rationalize their assumptions about how the world should work. A contract requires consent of both parties.
> “I’d rather be out sailing than working on rsync security issues, so I have reached for several AI tools to help with what needs to be done,”
Well, then maybe it's already overdue to find a new maintainer for the project and let someone else continue it? The tool will not get better from someone working on it who doesn't want to.
Well, then maybe it's already overdue to find a new maintainer for the project and let someone else continue it? The tool will not get better from someone working on it who doesn't want to.
He explicitly addresses that in the article.
> Luckily I’ve been joined by some other very good developers with great systems development skills and security knowledge... Watch out for some credits for some great new rsync developers in the next release.
> Luckily I’ve been joined by some other very good developers with great systems development skills and security knowledge... Watch out for some credits for some great new rsync developers in the next release.
Unless you're willing to step up and be that person, it's not your place for you to suggest it.
I don't agree with that, I can very well still discuss that. He clearly sounds like someone who doesn't want to do this work anymore and should have searched for a successor.
That's my impression from that sentence, at least. Don't you agree?
So, why didn't he do it? Because just firing up Claude and let it rip is way easier than finding real people and building up trust?
Did Claude increase bugs in rsync? Or did Claude just gave some basically retired programmer, who doesn't even want to work on his project anymore, the impression that he can replace finding a successor with just handing it to AI?
That's my impression from that sentence, at least. Don't you agree?
So, why didn't he do it? Because just firing up Claude and let it rip is way easier than finding real people and building up trust?
Did Claude increase bugs in rsync? Or did Claude just gave some basically retired programmer, who doesn't even want to work on his project anymore, the impression that he can replace finding a successor with just handing it to AI?
Because just firing up Claude and let it rip
Based on Tridge’s post, this seems an unfair characterisation of how he used Claude.
Did Claude increase bugs in rsync?
TFA answered this, the answer is “no”.
Based on Tridge’s post, this seems an unfair characterisation of how he used Claude.
Did Claude increase bugs in rsync?
TFA answered this, the answer is “no”.
You're highly critical. What would you be doing differently? So far Tridge has elected to:
- generally decide to fix security issues over preserving compatibility - rewritten an aging test suite in what appears to be a highly responsible way - brought on additional qualified developers to help with the workload
Not bad for a guy who's retired.
You care enough to complain on HN. You could be a part of the solution.
What were you going to do differently, specifically?
- generally decide to fix security issues over preserving compatibility - rewritten an aging test suite in what appears to be a highly responsible way - brought on additional qualified developers to help with the workload
Not bad for a guy who's retired.
You care enough to complain on HN. You could be a part of the solution.
What were you going to do differently, specifically?
> That's my impression from that sentence, at least. Don't you agree?
No. Given a choice between doing laundry and driving Lamborghinis, I would probably choose the latter. But I still have to do my laundry. I might use a washing machine to do so. It's just a responsibility among many responsibilities. It isn't that deep, really.
The reality few people want to admit is that maintaining open-source software is often closer for many people to "doing laundry" than like, being the software equivalent of Atticus Finch.
> Or did Claude just gave some basically retired programmer, who doesn't even want to work on his project anymore,
The only thing Claude has "done" apparently is give a bunch of annoying people online a license to engage in armchair psychoanalysis of someone they don't know at all, from what I can tell.
No. Given a choice between doing laundry and driving Lamborghinis, I would probably choose the latter. But I still have to do my laundry. I might use a washing machine to do so. It's just a responsibility among many responsibilities. It isn't that deep, really.
The reality few people want to admit is that maintaining open-source software is often closer for many people to "doing laundry" than like, being the software equivalent of Atticus Finch.
> Or did Claude just gave some basically retired programmer, who doesn't even want to work on his project anymore,
The only thing Claude has "done" apparently is give a bunch of annoying people online a license to engage in armchair psychoanalysis of someone they don't know at all, from what I can tell.
> and should have searched for a successor.
He doesn't have to do that. If he ever do not care enough he can just stop maintaining it and that's it.
He doesn't have to do that. If he ever do not care enough he can just stop maintaining it and that's it.
I think many would prefer that to the situation that happened.
Congratulations, you have an opinion.
Yeah, we definitely need to make sure that we take the considerations of the mob into account.
The person owning the project is using the master branch in the way he sees fit.
Incidentally, there is no amount of communicating "correctly" that quells a mob. There's a Venn diagram of concerns, and those with concerns not being met will generate (now infinite) outrage.
The person owning the project is using the master branch in the way he sees fit.
Incidentally, there is no amount of communicating "correctly" that quells a mob. There's a Venn diagram of concerns, and those with concerns not being met will generate (now infinite) outrage.
Is using calloc for everything fixing a security issue or hardening it?
Calloc is generally hardening, because it zeros out any stale memory contents left over from previous uses of the memory.
You can avoid this overhead if you use a language that forbids reading from uninitialized memory, but C is not that language.
You can avoid this overhead if you use a language that forbids reading from uninitialized memory, but C is not that language.
Uninitialized memory is not a problem (the OS is never going to give a program memory that has data in it from another program). The problem is memory that you allocated in the past, have freed, but hasn't been returned to the OS[0]. It might have key material or other sensitive data in it[1]. Or it might just have random garbage in it that could be misinterpreted by the code that's about to use it, if it hasn't been initialized to a known state.
For some uses, you do genuinely need (specifically) zeroed-out memory before you start to use it, and that's where calloc() is truly useful. But that need not have anything to do with security.
[0] The allocator will often hold onto memory that has been freed in order to quickly service future requests for new allocations, without needing a context switch into kernel space.
[1] Granted, the correct way to handle that is to zero it out before freeing it, in a way that the compiler won't optimize out.
For some uses, you do genuinely need (specifically) zeroed-out memory before you start to use it, and that's where calloc() is truly useful. But that need not have anything to do with security.
[0] The allocator will often hold onto memory that has been freed in order to quickly service future requests for new allocations, without needing a context switch into kernel space.
[1] Granted, the correct way to handle that is to zero it out before freeing it, in a way that the compiler won't optimize out.
> The problem is memory that you allocated in the past, have freed, but hasn't been returned to the OS[0].
There are at least two different ways in which memory might be semantically "uninitialized":
1. The memory was provided by the OS. On modern desktop and mobile OSes, this memory will normally be zeroed automatically. 2. The memory was provided by the language's allocator. This may contain a mix of data used by previous allocations and memory that has never been touched (perhaps because previous allocations reserved it as end-of-array "capacity" that never got used). From the perspective of a language like Rust, this memory is considered uninitialized, and safe code should never be able to read it without first setting it.
In ancient C code, it makes a fair bit of sense to preemptively calloc everything. Or better, to wrap the allocator with one that zeroes on free. Though even there, you need to be careful not to expose recycled heap block headers in the middle of newly allocated objects.
My opinion for the last 30+ years has been that C is unfit for purpose, and that using it almost inevitably introduces large numbers of dire security holes. But until the last 10-15 years, there hasn't been any seriously viable alternatives.
There are at least two different ways in which memory might be semantically "uninitialized":
1. The memory was provided by the OS. On modern desktop and mobile OSes, this memory will normally be zeroed automatically. 2. The memory was provided by the language's allocator. This may contain a mix of data used by previous allocations and memory that has never been touched (perhaps because previous allocations reserved it as end-of-array "capacity" that never got used). From the perspective of a language like Rust, this memory is considered uninitialized, and safe code should never be able to read it without first setting it.
In ancient C code, it makes a fair bit of sense to preemptively calloc everything. Or better, to wrap the allocator with one that zeroes on free. Though even there, you need to be careful not to expose recycled heap block headers in the middle of newly allocated objects.
My opinion for the last 30+ years has been that C is unfit for purpose, and that using it almost inevitably introduces large numbers of dire security holes. But until the last 10-15 years, there hasn't been any seriously viable alternatives.
mythical man month only gets more prescient as time passes
I would expect a 10x change rate, even carried out by clones of the existing maintainers to result in more bugs.
> Also the amount of commits is suspicious. In the last two months, rsync had about as much commits as in the last two years before that.
I wonder if the data looks worse or better when not doing per-10commit and instead do per-commit.
I wonder if the data looks worse or better when not doing per-10commit and instead do per-commit.
Seems like someone could use Claude to port rsync to Rust and the whole enterprise would be safer from things like this.
Start with unsafe then gradually convert into idiomatic Rust.
Start with unsafe then gradually convert into idiomatic Rust.
Your let's redo this in Rust made me wonder if generative AI will also be susceptible to software fads. One LLM writes a few blog posts extoling a new framework/lanaguge. Other agentics read these and get 'influenced'. Then they start clamoring for 'lets redo this in X!'. Can't wait to see it. /g
You can get 80% there with rust which is what is impressive. Then you have a reference implementation that you can always check against. If a Rust library have 0 unsafe, i dont care if it is written by a dog, it still have 0 UB.
UB is especially bad but also not as big as all other concerns combined. Two of the most reliable software ever to exist, curl and SQLite, are C/C++. There are also cases in system programming, drivers etc where the unsafe is necessary and then your code is only as good as the boundary, and lots of bugs can seep in. Another issue with Rust is ecosystem - the dependency trees required to do fairly basic things are often deep and vast, meaning other risks.
That said if something like rsync was written today, I still think Rust may be a better choice. Mainly because a 95 percentile skilled Rust programmer is less dangerous than for C. The people that are skilled enough to be trusted with C are few and diminishing every year.
That said if something like rsync was written today, I still think Rust may be a better choice. Mainly because a 95 percentile skilled Rust programmer is less dangerous than for C. The people that are skilled enough to be trusted with C are few and diminishing every year.
Prompt: automate writing commits to increase safety in these software projects so that my profile increases and I can snag a high-paying Rust job.
LLM: this commit changes whole codebase to Rust!
LLM: this commit changes whole codebase to Rust!
We will need rigorous agnostic statistical experiments to know what stuff is better
its bad enough when humans do it
> Then they start clamoring for 'lets redo this in X!'
Elon announces that spacex wrote its AI software in C. And now suddenly, C has become the new (old) kid on the block. Now we have folks saying, lets redo this in C as it gives you full power over the machine since we are 10x engineers. Earlier it was rust this or rust that. So, fads work both ways.
Elon announces that spacex wrote its AI software in C. And now suddenly, C has become the new (old) kid on the block. Now we have folks saying, lets redo this in C as it gives you full power over the machine since we are 10x engineers. Earlier it was rust this or rust that. So, fads work both ways.
> Written with claude.
No.
The reversion commit references https://github.com/RsyncProject/rsync/issues/959. In that GitHub issue is this comment:
> The change to zero memory was my idea and my change. It was a reaction to a security report I got which caused use of an element past the end of an array. By zeroing the allocation I could ensure that misuse of that memory if a similar bug came up in the future could only cause a null ptr deref, which is better than the chance of a valid pointer.
> It got a claude co-authored tag on it as I got it to do some tidy ups of a series of commits, and that is just what it does when it makes any modification. It doesn't mean the change was written by claude. It was written by me.
No.
The reversion commit references https://github.com/RsyncProject/rsync/issues/959. In that GitHub issue is this comment:
> The change to zero memory was my idea and my change. It was a reaction to a security report I got which caused use of an element past the end of an array. By zeroing the allocation I could ensure that misuse of that memory if a similar bug came up in the future could only cause a null ptr deref, which is better than the chance of a valid pointer.
> It got a claude co-authored tag on it as I got it to do some tidy ups of a series of commits, and that is just what it does when it makes any modification. It doesn't mean the change was written by claude. It was written by me.
> This is a good example of what slips through LLM attention. It forces all allocations to be calloc as if it is a strict upgrade.
I wouldn't assume Claude made that decision; it's not as if that was some incidental thing that it snuck into a large commit. The commit message starts with "zero all new memory from allocations", and that's exactly what the commit does. What do you imagine the prompt was?
It seems totally plausible to me that a human initially thought this was an improvement, then rethought after discovering the RSS regression. And it's not a law of nature anyway that this change has to increase RSS; calloc could special-case the case in which memory was freshly returned from the OS, knowing fresh memory mappings are zeroed anyway.
I blame AI for these regressions mostly in the sense that it caused a flurry of vulnerability reports. Those led to a flurry of quick fixes. Sometimes quick fixes cause other problems.
I wouldn't assume Claude made that decision; it's not as if that was some incidental thing that it snuck into a large commit. The commit message starts with "zero all new memory from allocations", and that's exactly what the commit does. What do you imagine the prompt was?
It seems totally plausible to me that a human initially thought this was an improvement, then rethought after discovering the RSS regression. And it's not a law of nature anyway that this change has to increase RSS; calloc could special-case the case in which memory was freshly returned from the OS, knowing fresh memory mappings are zeroed anyway.
I blame AI for these regressions mostly in the sense that it caused a flurry of vulnerability reports. Those led to a flurry of quick fixes. Sometimes quick fixes cause other problems.
You don't really have to guess. The guy told us the AI didn't suggest this specific change:
> The change to zero memory was my idea and my change. It was a reaction to a security report I got which caused use of an element past the end of an array. By zeroing the allocation I could ensure that misuse of that memory if a similar bug came up in the future could only cause a null ptr deref, which is better than the chance of a valid pointer. It got a claude co-authored tag on it as I got it to do some tidy ups of a series of commits, and that is just what it does when it makes any modification. It doesn't mean the change was written by claude. It was written by me.
https://github.com/RsyncProject/rsync/issues/959#issuecommen...
> The change to zero memory was my idea and my change. It was a reaction to a security report I got which caused use of an element past the end of an array. By zeroing the allocation I could ensure that misuse of that memory if a similar bug came up in the future could only cause a null ptr deref, which is better than the chance of a valid pointer. It got a claude co-authored tag on it as I got it to do some tidy ups of a series of commits, and that is just what it does when it makes any modification. It doesn't mean the change was written by claude. It was written by me.
https://github.com/RsyncProject/rsync/issues/959#issuecommen...
> … By zeroing the allocation …
How does that prevent reading past the end of the buffer? Or change how bytes outside the buffer are used? Are these arrays of pointers so that the “null ptr deref” comment makes sense?
Or am I the bozo and don’t know what’s happening here?
How does that prevent reading past the end of the buffer? Or change how bytes outside the buffer are used? Are these arrays of pointers so that the “null ptr deref” comment makes sense?
Or am I the bozo and don’t know what’s happening here?
It doesn’t. It’s just that dereferencing a zeroed pointer reliably crashes the program (unless you specifically do funky things with mmap) but dereferencing garbage memory as a pointer could do a lot more insidious damage.
My point is that the developer's comment doesn't make sense. Zeroing the allocated memory doesn't change anything about overrunning the buffer.
edit: removed unnecessary examples
edit: removed unnecessary examples
Haven't looked at the code, but the allocated memory could be larger than necessary to make "off-by-one" or "off-by-a-few" errors less deadly. Then zeroing it out makes it even less so. Defense in depth.
Or it's an allocation for an arena? The zeroing might help trigger 0 derefs earlier if the overrun happens for the object that are then allocated in the arena (and not by allocating more objects than the arena can provide)
Or it's an allocation for an arena? The zeroing might help trigger 0 derefs earlier if the overrun happens for the object that are then allocated in the arena (and not by allocating more objects than the arena can provide)
This doesn't prevent overrunning the buffer -- it means that when you do overrun the buffer, it does less damage
The code is part of a function called expand item list. It looks like it over allocates memory and uses a bump pointer for internal allocation, only expanding the allocation when necessary. Thus OOB writes to the list would hit the allocated memory.
You’re not a bozo but it is helpful to read the code.
You’re not a bozo but it is helpful to read the code.
[deleted]
okay I had not read this or any discussions there (except the one linked in the post), but this looks weirder. the comment you linked is a dev responding to what is very clearly a bot comment. I am sure they have good intentions and I have no reason to believe otherwise as I have no connection to the project whatsoever, but the original commit being 4-5 lines long (what did claude do then?) and the revert description is almost certainly written by an LLM makes in my mind the slop argument stronger.
I hope if this doesn't come across as unkind towards the dev who gives their time and energy to the project. Grateful for that.
I hope if this doesn't come across as unkind towards the dev who gives their time and energy to the project. Grateful for that.
> the original commit being 4-5 lines long (what did claude do then?)
I've said "rebase onto <newbase>" and let it handle all the merge conflicts. I wouldn't expect this particular commit to conflict with anything, but it could have been part of a big series where it'd be worth doing that instead of running the rebase command yourself. It wouldn't surprise me if I picked up some Co-Authored-By:s along the way.
I've said "rebase onto <newbase>" and let it handle all the merge conflicts. I wouldn't expect this particular commit to conflict with anything, but it could have been part of a big series where it'd be worth doing that instead of running the rebase command yourself. It wouldn't surprise me if I picked up some Co-Authored-By:s along the way.
It is certainly unkind, when a developer asserts the opposite of what you have assumed about their code, to double down and imply they are lying.
AI multiplied by Linux overcommit. What times we live in!
(My own view: 10.8 GB is nothing these days. Your sprintf buffers are probably larger than that. (And if they aren't: they should be. That, or you should start using snprintf...))
(My own view: 10.8 GB is nothing these days. Your sprintf buffers are probably larger than that. (And if they aren't: they should be. That, or you should start using snprintf...))
sprintf() should be a longer way to write abort(), change my mind
I'll change your mind:
If you pass NULL as the destination pointer, it doesn't write any string. If you combine this with %n at the end of the format string, you can get the exact length that the output string would be. Then you allocate that, then you print again, into the actual destination buffer this time.
If you pass NULL as the destination pointer, it doesn't write any string. If you combine this with %n at the end of the format string, you can get the exact length that the output string would be. Then you allocate that, then you print again, into the actual destination buffer this time.
If anything I just got entrenched in my opinion. The second best option, maybe, would be to not accept any destination pointers and have the default and only possible behavior just like what you describe.
I agree. I tend to use the gnu "asprintf" which simply returns a properly allocated char buffer with the formatted string in it. And on platforms that don't feature asprintf (windows) you can build your own using sprintf!
AI is fine, and in fact fun to use... committing AI written code without understanding Every. Single. Line. Of. Changes is on the committer. You can't LGFM for vibe code ffs
I don't have a dog in this fight, but a few points that look a little suspicious:
- The release with the highest number of attributed bugs is the release _right before_ the first release with Claude-coauthored commits, released in January; is there a chance that unattributed LLM-authored commits made it into this release?
- The release attribution methodology is not great, since it will tend to attribute bugs introduced in a minor version update to the longest-lived patch release of that minor version. I doubt that 3.4.1 actually introduced a lot of bugs, but since it was released a day after 3.4.0, bugs that were introduced in that release get attributed to 3.4.1.
- Relatedly, more recent releases have had less time to have bugs filed against them, so there may be a bit of a bias toward evaluating recent releases as less buggy.
- The release with the highest number of attributed bugs is the release _right before_ the first release with Claude-coauthored commits, released in January; is there a chance that unattributed LLM-authored commits made it into this release?
- The release attribution methodology is not great, since it will tend to attribute bugs introduced in a minor version update to the longest-lived patch release of that minor version. I doubt that 3.4.1 actually introduced a lot of bugs, but since it was released a day after 3.4.0, bugs that were introduced in that release get attributed to 3.4.1.
- Relatedly, more recent releases have had less time to have bugs filed against them, so there may be a bit of a bias toward evaluating recent releases as less buggy.
Agree. From the article:
> Here's my favorite part, though. Digging into the data, one of the first things that jumped out at me with blinding clarity was that the worst release, by far, in rsync history was entirely prior to the introduction of Claude ... And yet nobody noticed.
Language really does suggest the article's author does have a dog in this fight and is cloaking opinion in fancy statistics jargon. "Blinding clarity"? All you have to do is draw a plot. And anyway, v3.4.1 was 2025-01-16, technically well within the AI assisted coding era and before attribution was becoming standard practice.
> Here's my favorite part, though. Digging into the data, one of the first things that jumped out at me with blinding clarity was that the worst release, by far, in rsync history was entirely prior to the introduction of Claude ... And yet nobody noticed.
Language really does suggest the article's author does have a dog in this fight and is cloaking opinion in fancy statistics jargon. "Blinding clarity"? All you have to do is draw a plot. And anyway, v3.4.1 was 2025-01-16, technically well within the AI assisted coding era and before attribution was becoming standard practice.
Also from the article:
> "Claude clearly made things worse" &emdash; the main claim
This article was clearly generated by AI, yet I found no mention/attribution of that by author.
How likely is it than someone who vibe codes articles would also vibe code the underlying analysis and be eager to accept an outcome that is highly validating of that person’s workflow? I’d say very.
> "Claude clearly made things worse" &emdash; the main claim
This article was clearly generated by AI, yet I found no mention/attribution of that by author.
How likely is it than someone who vibe codes articles would also vibe code the underlying analysis and be eager to accept an outcome that is highly validating of that person’s workflow? I’d say very.
The &emdash; is probably human error, other parts of the HTML correctly use — or Unicode em-dashes. Also: https://github.com/alexispurslane/rsync-analysis/commit/740b...
He did admit as much:
> "The scripts used to fetch the data, collate it into a DuckDB database file, construct the views on that DB, and then do the statistical analysis on that data, were indeed written by GLM 5.1, as was the HTML and much of the original prose for the final report webpage you're looking at right now."
> "The scripts used to fetch the data, collate it into a DuckDB database file, construct the views on that DB, and then do the statistical analysis on that data, were indeed written by GLM 5.1, as was the HTML and much of the original prose for the final report webpage you're looking at right now."
But: "After posting this on Hacker News and recieving [sic] almost no substantive input, discussion, or response on the actual content of the article, I decided to rewrite all of the prose in my own voice. If anyone complains about my verbosity or sentence structure — as they usually do, which is the reason I originally let the AI write the prose, among other reasons obsoleted by templating — they can go fuck themselves."
So rewritten in his own voice. Maybe the m-dashes are from GLM, maybe from the author.
So rewritten in his own voice. Maybe the m-dashes are from GLM, maybe from the author.
Are the numbers wrong? That's the only relevant thing here.
Also, humans do use em dashes, just FYI.
Also, humans do use em dashes, just FYI.
> Are the numbers wrong? That's the only relevant thing here.
Data without interpretation is irrelevant, and correct numbers can be interpreted wrongly, either on purpose or by mistake.
I’m not saying any of that happened here, only that “are the numbers wrong” is not the only thing that is relevant.
> humans do use em dashes, just FYI.
Your parent comment is not complaining about em dashes, they are pointing out the article has a literal “&emdash;” in it.
Data without interpretation is irrelevant, and correct numbers can be interpreted wrongly, either on purpose or by mistake.
I’m not saying any of that happened here, only that “are the numbers wrong” is not the only thing that is relevant.
> humans do use em dashes, just FYI.
Your parent comment is not complaining about em dashes, they are pointing out the article has a literal “&emdash;” in it.
Yes, I do for example.
And the author discussed the use of AI pretty exhaustively in point 0 of the post.
And the author discussed the use of AI pretty exhaustively in point 0 of the post.
[deleted]
You can use LLMs in multiple ways, from very hands on to make local changes to completely hands-off.
I've seen plenty of code that was LLM generated but the commit message itself did not have the co-author attached to it. This only seems to happen when someone's interface to the codebase is completely though Claude/Codex/..., and those are usually the most verbose commits, and yet they say the least, because they just summarize the code changes, not the why.
On the other hand I've seen developers using Claude as a tool. They have VSCode open and a terminal window with Claude and go back and forth, ensuring they write correct code, and leave the plumbing to Claude.
So maybe the author of the code started off small and it grew over time?
I've seen plenty of code that was LLM generated but the commit message itself did not have the co-author attached to it. This only seems to happen when someone's interface to the codebase is completely though Claude/Codex/..., and those are usually the most verbose commits, and yet they say the least, because they just summarize the code changes, not the why.
On the other hand I've seen developers using Claude as a tool. They have VSCode open and a terminal window with Claude and go back and forth, ensuring they write correct code, and leave the plumbing to Claude.
So maybe the author of the code started off small and it grew over time?
I would expect a mature code base like rsync to have a lot of unit tests and integration tests and frankly if there's not enough that such bugs haven't been caught; that should be your first use of LLMs in order to setup some deterministic guidelines when you do start making changes to your actual code.
I have been experimenting with both aforementioned styles with interesting results.
I have been experimenting with both aforementioned styles with interesting results.
> I would expect a mature code base like rsync to have a lot of unit tests and integration tests
You might be surprised. C applications which interact heavily with the system - like rsync - can be tricky to test comprehensively, as it's nontrivial to inject faults into system calls. If the application is architected to support this kind of testing, or uses a HAL, that may make matters easier - but an older codebase like rsync probably isn't.
You might be surprised. C applications which interact heavily with the system - like rsync - can be tricky to test comprehensively, as it's nontrivial to inject faults into system calls. If the application is architected to support this kind of testing, or uses a HAL, that may make matters easier - but an older codebase like rsync probably isn't.
I've had a local LLM spending weeks trying to write tests. then debug those tests. then write antipatterns and patterns for those tests.
It's amusing. It's not terrible, but tests arn't going to save you from a malicious tester.
It's amusing. It's not terrible, but tests arn't going to save you from a malicious tester.
Your first and second points seem to contradict each other because if all of the bugs for 3.4.1 should be attributed to 3.4.0, that pushes the timetable back even further that unattributed LLM commits would have to have been being committed to the project, which just makes your point even more absurd.
Which brings me to my overall response, which is that there is absolutely no evidence, and nothing even intimating this hypothesis, that LLM commits were secretly being added to earlier releases before they were attributed, and that's why the rate of bugs is higher. There's no reason to think that it's an unreasonable thing to think, and there's no evidence for that whatsoever unless you beg the question and assume that higher bug counts must automatically indicate AI involvement, which is just circular reasoning. You're essentially just making up a hypothesis out of thin air to preserve your point.
Regarding your third point, that one's fair, but I've done the analysis and I can put it up if you want, as to how long it usually takes to find bugs and how far through the release cycle we are for each version.
Which brings me to my overall response, which is that there is absolutely no evidence, and nothing even intimating this hypothesis, that LLM commits were secretly being added to earlier releases before they were attributed, and that's why the rate of bugs is higher. There's no reason to think that it's an unreasonable thing to think, and there's no evidence for that whatsoever unless you beg the question and assume that higher bug counts must automatically indicate AI involvement, which is just circular reasoning. You're essentially just making up a hypothesis out of thin air to preserve your point.
Regarding your third point, that one's fair, but I've done the analysis and I can put it up if you want, as to how long it usually takes to find bugs and how far through the release cycle we are for each version.
Sorry, I should have said this explicitly in the original comment: I think you're likely _correct_ that there isn't a clear increase in the rate of bugs attributable to LLM-authored code in rsync. Your analysis provides evidence in this direction; these are just the things that made me go "hmm". They're not accusations or claims that the conclusion is invalid. But they're definitely things to be curious about.
Regarding unlabeled LLM-authored commits, I don't think it's unreasonable in general to think that an open-source project might have had unlabeled LLM-authored commits at some point before 2026. Looking more closely at rsync's recent commit history, I think it's less likely in this case. There's just a low number of commits in general, _until_ large batches of Claude-authored commits start showing up early this year. But this then raises some questions about the bugs-per-commit metric; it does correct for something like "size of release", but also obscures a significant shift in commit velocity that may be downstream of adding LLM development tools to the workflow.
Like I said, I don't have a dog in this fight, and I try not to approach sorts of questions from a position of explicit advocacy. I do think it's an interesting question, though, and we should try to understand what the data is actually telling us.
Regarding unlabeled LLM-authored commits, I don't think it's unreasonable in general to think that an open-source project might have had unlabeled LLM-authored commits at some point before 2026. Looking more closely at rsync's recent commit history, I think it's less likely in this case. There's just a low number of commits in general, _until_ large batches of Claude-authored commits start showing up early this year. But this then raises some questions about the bugs-per-commit metric; it does correct for something like "size of release", but also obscures a significant shift in commit velocity that may be downstream of adding LLM development tools to the workflow.
Like I said, I don't have a dog in this fight, and I try not to approach sorts of questions from a position of explicit advocacy. I do think it's an interesting question, though, and we should try to understand what the data is actually telling us.
Isn't the metric that you've used "bugs per commit ~ per new line of code" going to miss the issue?
All code is technical debt.
If rsync releases used to have 500 lines changed and 5 bugs in and AI-powered rsync releases have 50000 lines and 500 bugs, it's the same bugs/line but much worse experience for the user?
I've not looked into the details of this case and I do use AI assistance coding at work but in my experience, the problem is that it's too easy to write lots of code and therefore hard to review the huge volumes of code and this analysis will ignore that?
edit: actually your table shows there weren't unusually large numbers of commits in this release, so perhaps my initial skepticism shows a bias I have?
All code is technical debt.
If rsync releases used to have 500 lines changed and 5 bugs in and AI-powered rsync releases have 50000 lines and 500 bugs, it's the same bugs/line but much worse experience for the user?
I've not looked into the details of this case and I do use AI assistance coding at work but in my experience, the problem is that it's too easy to write lots of code and therefore hard to review the huge volumes of code and this analysis will ignore that?
edit: actually your table shows there weren't unusually large numbers of commits in this release, so perhaps my initial skepticism shows a bias I have?
OpenBSD used to have sqlite in base, but the code churn rate was too high to review. This was well before the recent LLM craze, so a human (perhaps not a normal one, though) already sufficies to generate too many changes for others to check for errors.
I started to look into the same thing considering releases are quite infrequent. To avoid the issue of unattributed LLM-authored commits, in my opinion the analysis should include a comparison to bug severity before and after release v3.3.0 (date April 6th, 2024)
Let's start with most outright alarming error - the claude statistics are taken out of whole 2 data points
That's sort of the point. There isn't enough data to extrapolate, and yet that's exactly what those outraged about AI were doing, and when you do do the very minimal types of analyses (permutation tests, and looking at distributions, mostly) that are actually valid, safe, standard, and useful to do on such low amounts of date, again, no evidence for the outrage shows up, and the two releases look so normal that it sort of shows no one would've cared if they hadn't known or found out that Claude was involved.
I really think this a much better standard of evidence — limited though it is — to outrage-fueled cherry-picked anecdotes, which is what has been driving this whole thing. If you disagree, and think the outrage should go one when I've shown there's an absence of evidence entirely for it (although of course, that's not evidence of absence; maybe I'll have to eat my words 5 releases down the line, but appealing to that now feels like a Russell's Teapot), would you care to explain why?
I really think this a much better standard of evidence — limited though it is — to outrage-fueled cherry-picked anecdotes, which is what has been driving this whole thing. If you disagree, and think the outrage should go one when I've shown there's an absence of evidence entirely for it (although of course, that's not evidence of absence; maybe I'll have to eat my words 5 releases down the line, but appealing to that now feels like a Russell's Teapot), would you care to explain why?
you could've literally just waited few more releases, but no, have to catch the hype wave before news are cold
> that are actually valid, safe, standard, and useful to do on such low amounts of date,
if you presented paper with that amount of data points you'd be laughed out of the room
> that are actually valid, safe, standard, and useful to do on such low amounts of date,
if you presented paper with that amount of data points you'd be laughed out of the room
The interpretations of the p-value is also alarming. One of the first thing they teach you in statistics class is: “an absence of evidence is not evidence of absence”.
This analysis showed that there is indeed an absence of evidence, but it concludes there is evidence of absence.
Traditional p-hacking is done by oversampling and overtesting. If you do 20 analysis on average one will show p < 0.05 by random chance. This analysis is doing the inverse of that. Under-sampling, and concluding with p > 0.05
This analysis showed that there is indeed an absence of evidence, but it concludes there is evidence of absence.
Traditional p-hacking is done by oversampling and overtesting. If you do 20 analysis on average one will show p < 0.05 by random chance. This analysis is doing the inverse of that. Under-sampling, and concluding with p > 0.05
> This analysis showed that there is indeed an absence of evidence, but it concludes there is evidence of absence.
I tried pretty hard to avoid saying that, can you point me at how to rephrase? The point I'm trying to make is just that there is absolutely no evidence at all for what people are saying with such absolutism and claimed objectivity (that Claude made rsync worse), and thus it doesn't justify the outrage.
> Under-sampling, and concluding with p > 0.05
How would I avoid under-sampling here? And if you're going to say it's because I only have 2 data points, well, the side making the positive claim — that Claude made rsync worse — only had two as well, and unremarkable ones at that, as I've tried very hard to show.
I tried pretty hard to avoid saying that, can you point me at how to rephrase? The point I'm trying to make is just that there is absolutely no evidence at all for what people are saying with such absolutism and claimed objectivity (that Claude made rsync worse), and thus it doesn't justify the outrage.
> Under-sampling, and concluding with p > 0.05
How would I avoid under-sampling here? And if you're going to say it's because I only have 2 data points, well, the side making the positive claim — that Claude made rsync worse — only had two as well, and unremarkable ones at that, as I've tried very hard to show.
You are interpreting the p-values on their own merit rather then using them to test a null-hypothesis. Quotes like:
> With a p-value of 74%, the answer is a decisive no. The odds ratio is 1.06 — essentially 1:1. Claude releases are no more likely to be above the median than any other releases.
are problematic in this context as the correct conclusion here is you just don‘t have enough data conclude whether or not you are more likely to encounter a bug after a Claude commit.
> How would I avoid under-sampling here?
You don‘t. You admit that you don’t have enough data and move on. What you are trying to do here is prove a negative, which is extremely hard to do. In your discussion you claim that the users complaining had no right to, however nothing in your analysis showed they were wrong. We simply don‘t have enough data (yet) to say either way. When we have enough data they may be proven right or wrong, but until then, we cannot conclude either way.
If you insist still, I recommend looking into bayesian analysis. Theoretically at least the posterior distribution from a bayesian analysis can be interpreted directly and analyses on its own merits. However I suspect your posterior will have way too much uncertainty to reach any conclusions.
> With a p-value of 74%, the answer is a decisive no. The odds ratio is 1.06 — essentially 1:1. Claude releases are no more likely to be above the median than any other releases.
are problematic in this context as the correct conclusion here is you just don‘t have enough data conclude whether or not you are more likely to encounter a bug after a Claude commit.
> How would I avoid under-sampling here?
You don‘t. You admit that you don’t have enough data and move on. What you are trying to do here is prove a negative, which is extremely hard to do. In your discussion you claim that the users complaining had no right to, however nothing in your analysis showed they were wrong. We simply don‘t have enough data (yet) to say either way. When we have enough data they may be proven right or wrong, but until then, we cannot conclude either way.
If you insist still, I recommend looking into bayesian analysis. Theoretically at least the posterior distribution from a bayesian analysis can be interpreted directly and analyses on its own merits. However I suspect your posterior will have way too much uncertainty to reach any conclusions.
Edited that claim, and made several clarifications elsewhere. The whole point of this analysis is that outrage is unjustified on the basis of two totally statistically unremarkable releases that no one would have remarked on pre-AI (my further proof of this is that there was a pre-AI remarkably broken release, and no one did comment!) and zero positive evidence outside cherry-picked anecdotes for any negative impact. We should wait for outrage and version pinning and cancelation until there is evidence, no? I'm just trying to say that these specific releases are unremarkable, and there's no evidence at all of harm currently; I'm not trying to build any kind of predictive model for future Claude releases to say anything grander than "these specific releases are fine, what are we freaking out about?", not some claim about what Claude-exposed releases will look like or trend like in the future or in general.
There is a lot more context to the outrage which is missing from your analysis. People have multiple reasons to be mad at AI usage, you mention some of them in your introduction, and you put a (statistically insignificant) measure on only one of them. In your analysis you have shown that exactly one of these reasons is anecdotal. That does not mean they are wrong, and it especially does not mean they are unjustified.
That you found a single pre-AI release which did not cause outrage is proof of nothing. This single release is equally anecdotal, and statistically insignificant.
So, the biggest context that is missing here is that people hate AI for various reasons, and they don‘t want their favorite tools to fall victim to AI for equally many reasons. It is only natural that people who hate AI react this way when they find out their favorite tool uses AI, and doubly so when they sniff correlation between their favorite tools use of AI and bugs.
> I'm just trying to say that these specific releases are unremarkable, and there's no evidence at all of harm currently.
Well, there is no evidence against harm either. But what you did here is a bit of a slight of hand. In your analysis your null hypothesis is: “There is no difference in bug count between releases which includes code commits from Claude Code and releases which don‘t”. (You then go about doing what every psychology major is taught not to do; find evidence for the null hypothesis, not against it). However what hypothesis testing is for is to use a representative sample to generalize over a wider population. You do hypothesis testing because you want to demonstrate that your sample is representative of a wider population, that you just so happened to have picked the two sample, by random chance, which shows the effect regardless of the experiment.
By calculating the p-values you were telling me that you were in fact ready to make generalizing statements over a wider population of commits, but your results were statically insignificant, so really you should not draw any conclusions from them. You have not, in fact, shown that they aren’t different from the rest of the population.
That you found a single pre-AI release which did not cause outrage is proof of nothing. This single release is equally anecdotal, and statistically insignificant.
So, the biggest context that is missing here is that people hate AI for various reasons, and they don‘t want their favorite tools to fall victim to AI for equally many reasons. It is only natural that people who hate AI react this way when they find out their favorite tool uses AI, and doubly so when they sniff correlation between their favorite tools use of AI and bugs.
> I'm just trying to say that these specific releases are unremarkable, and there's no evidence at all of harm currently.
Well, there is no evidence against harm either. But what you did here is a bit of a slight of hand. In your analysis your null hypothesis is: “There is no difference in bug count between releases which includes code commits from Claude Code and releases which don‘t”. (You then go about doing what every psychology major is taught not to do; find evidence for the null hypothesis, not against it). However what hypothesis testing is for is to use a representative sample to generalize over a wider population. You do hypothesis testing because you want to demonstrate that your sample is representative of a wider population, that you just so happened to have picked the two sample, by random chance, which shows the effect regardless of the experiment.
By calculating the p-values you were telling me that you were in fact ready to make generalizing statements over a wider population of commits, but your results were statically insignificant, so really you should not draw any conclusions from them. You have not, in fact, shown that they aren’t different from the rest of the population.
> In your analysis you have shown that exactly one of these reasons is anecdotal.
This was actually the convincing one for me though. “Did AI increase the rsync bug rate? Dunno, can’t tell yet” seems like a fine conclusion to me. Plenty of people in this thread and previous ones on the topic seem convinced one way or another, so it’s nice to see actual numbers.
This was actually the convincing one for me though. “Did AI increase the rsync bug rate? Dunno, can’t tell yet” seems like a fine conclusion to me. Plenty of people in this thread and previous ones on the topic seem convinced one way or another, so it’s nice to see actual numbers.
The numbers are statistically insignificant though. So you cannot use them to generalize over a wider population.
I think in this era of scientific literacy people tend to overcorrect in the absence of evidence. Anecdotal evidence still evidence though, and people are right to react to them.
If we remove our frequentis hats and put on our baysian hat (which is a wise thing to do when n is very low) we can take into consideration evidence from multiple direction at the same time as we upgrade our belief. A baysian might start with the prior that claude assisted commits have the same distribution as non-assisted commits. I would start with a Poisson distribution as my prior, and then they would factor inn all the evidence of AI slop they have seen in their lives and update their posteriors accordingly. Claude caude has been wrong about so many things in the past, which should contribute to a smaller lambda then the control group.
I think in this era of scientific literacy people tend to overcorrect in the absence of evidence. Anecdotal evidence still evidence though, and people are right to react to them.
If we remove our frequentis hats and put on our baysian hat (which is a wise thing to do when n is very low) we can take into consideration evidence from multiple direction at the same time as we upgrade our belief. A baysian might start with the prior that claude assisted commits have the same distribution as non-assisted commits. I would start with a Poisson distribution as my prior, and then they would factor inn all the evidence of AI slop they have seen in their lives and update their posteriors accordingly. Claude caude has been wrong about so many things in the past, which should contribute to a smaller lambda then the control group.
The concept you need here is "Statistical Power".
The ELI5 version is that there are two mistakes you can make when looking at a P value:
Type I error, where your P value is falsely low. In the experiment being discussed here, it would lead one to conclude that AI code is worse. Otherwise known as a false positive.
Type II error, where your P value is falsely high, leading you to conclude that AI code is no different. Otherwise known as a false negative.
https://en.wikipedia.org/wiki/Power_(statistics)
One can calculate statistical power for a given experimental protocol.
My hunch is that if you did this, you would find this experiment is grossly under-powered.
This means you can't make the "absence of evidence" claim.
The ELI5 version is that there are two mistakes you can make when looking at a P value:
Type I error, where your P value is falsely low. In the experiment being discussed here, it would lead one to conclude that AI code is worse. Otherwise known as a false positive.
Type II error, where your P value is falsely high, leading you to conclude that AI code is no different. Otherwise known as a false negative.
https://en.wikipedia.org/wiki/Power_(statistics)
One can calculate statistical power for a given experimental protocol.
My hunch is that if you did this, you would find this experiment is grossly under-powered.
This means you can't make the "absence of evidence" claim.
He can't make the evidence of absence claim, but he can absolutely make the absence of evidence claim.
Perhaps in an “everyday language” way, but not in the technical, statistical sense.
In an underpowered statistical study, a claim that two experimental conditions did not differ are not persuasive.
In an underpowered statistical study, a claim that two experimental conditions did not differ are not persuasive.
No. It's a description of the result of the maybe underpowered study. the underpowered study did not find evidence. Evidence is absent. Because it is underpowered, it's not evidence that the effect is absent.
The claim is not "two experimental conditions did not differ". The claim is "The data do not show evidence that the experimental conditions did differ".
The claim is not "two experimental conditions did not differ". The claim is "The data do not show evidence that the experimental conditions did differ".
You say "the underpowered study did not find evidence". Not true, it found quite a bit of evidence - many statistics were presented. There is no absence of evidence. The author wrote about the evidence, presenting P values and other statistics.
Of course the critical part is not the numbers, but what they mean.
So, what does the evidence mean?
The author interprets it to mean that there is no difference. They state this several times:
"46% EXACT PERMUTATION TEST P-VALUE (ONE-SIDED, H₁: CLAUDE MEAN > HISTORICAL)[...] What this p-value tells us is There's nothing unusual about the Claude group."
"74% ONE-SIDED P-VALUE (H₁: CLAUDE MORE LIKELY ABOVE MEDIAN) Fisher's exact test asks: if we split all releases at the historical median (0.74 sev/10c), are these Claude releases significantly buggy than previous releases (more likely to land above the median)? With a p-value of 74%, the answer is a decisive no. "
In an under-powered study, when a P value is above your alpha level cutoff (.05, .01, whatever was chosen) you can't distinguish between "no effect" and "could be an effect, but I didn't see one".
Of course the critical part is not the numbers, but what they mean.
So, what does the evidence mean?
The author interprets it to mean that there is no difference. They state this several times:
"46% EXACT PERMUTATION TEST P-VALUE (ONE-SIDED, H₁: CLAUDE MEAN > HISTORICAL)[...] What this p-value tells us is There's nothing unusual about the Claude group."
"74% ONE-SIDED P-VALUE (H₁: CLAUDE MORE LIKELY ABOVE MEDIAN) Fisher's exact test asks: if we split all releases at the historical median (0.74 sev/10c), are these Claude releases significantly buggy than previous releases (more likely to land above the median)? With a p-value of 74%, the answer is a decisive no. "
In an under-powered study, when a P value is above your alpha level cutoff (.05, .01, whatever was chosen) you can't distinguish between "no effect" and "could be an effect, but I didn't see one".
Many statistics were presented. In the view of the author (and I think he is correct), none of them show evidence for an increased bug rate from Claude. That is absence of evidence (...for the increased bug rate).
The two examples you bring are not claims of absence of evidence, but claims of evidence of absence. The author takes the result as evidence that there is no effect. As I wrote, the author shouldn't do that, because indeed you cannot distinguish between "no effect exists" and "no effect observed". But again, these are (wrong) claims for evidence of absence.
The author can absolutely claim: I did these statistical tests, and none showed evidence that there is an effect. Absence of evidence. It's not a claim that there will never be evidence. Just that there is none from these tests.
Edit: To convert the absence of evidence into evidence for absence, indeed you need to understand the statistical power of your test, and how it is affected by alternate hypotheses. And for that, without having done the math, having only two data points seems very thin.
The two examples you bring are not claims of absence of evidence, but claims of evidence of absence. The author takes the result as evidence that there is no effect. As I wrote, the author shouldn't do that, because indeed you cannot distinguish between "no effect exists" and "no effect observed". But again, these are (wrong) claims for evidence of absence.
The author can absolutely claim: I did these statistical tests, and none showed evidence that there is an effect. Absence of evidence. It's not a claim that there will never be evidence. Just that there is none from these tests.
Edit: To convert the absence of evidence into evidence for absence, indeed you need to understand the statistical power of your test, and how it is affected by alternate hypotheses. And for that, without having done the math, having only two data points seems very thin.
If one asks "Is the house on 123 Road Street, NJ, taller than the statistical average", then that there is only 1 datapoint for the house on 123 Road Street, NJ. Which is also 100% of the houses on 123 Road Street, NJ.
You can apply that to the outrage too: the people pissed off about this are going off 2 measly data points.
There's a meta-level of irony here that's important to note.
TFA is defending the use of AI, and it very clearly (to me) used AI to analyze the data and present the results.
In doing so, the author used statistics in a way they do not appear to understand, and ended up making numerous false claims (you can see the thread discussing these here https://news.ycombinator.com/item?id=48417626 )
In short, the study doesn't have sufficient statistical power, and is making "no difference" claims that aren't justified.
The meta-irony is this: the author used an LLM to interpret data in this study, and seems to have made the same category of mistake (confidently asserting falsehoods) that the study was supposed to be investigating (confidently submitting bad commits to the rsync project).
TFA is defending the use of AI, and it very clearly (to me) used AI to analyze the data and present the results.
In doing so, the author used statistics in a way they do not appear to understand, and ended up making numerous false claims (you can see the thread discussing these here https://news.ycombinator.com/item?id=48417626 )
In short, the study doesn't have sufficient statistical power, and is making "no difference" claims that aren't justified.
The meta-irony is this: the author used an LLM to interpret data in this study, and seems to have made the same category of mistake (confidently asserting falsehoods) that the study was supposed to be investigating (confidently submitting bad commits to the rsync project).
The meta-meta level irony is that the reaction to this post is based on vibes and misunderstands the point of the article to wage ideological warfare -- much the same way the original github issue was written.
AI is so much like a religion. There is nothing you can say to a believer that will make them question their believes. Or more generally, you cannot reason anyone out of something that they want to believe.
AI is nothing like religion. People behave similarly to AI when debating their favorite sports team, or for Java coders, Checked vs Runtime exceptions.
Religion is about faith and what people feel and sense as much as believe.
Religion is about faith and what people feel and sense as much as believe.
It gets pretty dark if you pull that thread of reason. =3
https://en.wikipedia.org/wiki/The_True_Believer
https://en.wikipedia.org/wiki/The_True_Believer
The statistical methodology I used is mine. As is the interpretation. Completely. To the degree that I misunderstood statistics (and it is under debate even in the thread you link, and the people accusing me of misunderstanding statistics there are universally misrepresenting my point, which is to point out a total absence of evidence for any difference, not to prove the null hypothesis) that's on me
FWIW I understood your point just fine. It seemed to me that you made a clear enough distinction between "evidence that Claude didn't increase bugs" and "no good evidence either way".
I don’t think that point is clear at all, at least in this comment sections you can find a few posts like these:
> Did Claude increase bugs in rsync?
>
> TFA answered this, the answer is “no”.
https://news.ycombinator.com/item?id=48423811
> Did Claude increase bugs in rsync?
>
> TFA answered this, the answer is “no”.
https://news.ycombinator.com/item?id=48423811
It seems like based on e.g. [1] the article originally made some stronger claims about “no difference in bugs” that have been corrected. I agree that now it seems fine, but those edits might be why it feels like some commenters read a different article than you.
[1]: https://news.ycombinator.com/item?id=48418186
[1]: https://news.ycombinator.com/item?id=48418186
Unfortunately for the people mad about this, I predict the only thing they will accomplish by pressuring the rsync maintainers, is to discourage everyone else from responsibly disclosing their use of AI. You’re just going to make people disable Claude attribution on their commits to avoid drama.
I never care about AI usage disclosure, because I don't believe that human produced code is necessarily better than AI produced code, unless it's someone I personally know.
People need to be responsible for code they commit and push anyways. This has never changed. Whether the code is written by hand, by their cat walking over keyboard, or by AI, is not my concern.
A project's code quality can decline for all kinds of reasons. I don't think it's productive to laser-focus on whether it's produced by AI or not. That's a distraction. If a person just want to find excuse to criticize AI, and another person wants to fight back and defend AI, sure, go for it. But that's not how you would want to assess a project's code quality.
People need to be responsible for code they commit and push anyways. This has never changed. Whether the code is written by hand, by their cat walking over keyboard, or by AI, is not my concern.
A project's code quality can decline for all kinds of reasons. I don't think it's productive to laser-focus on whether it's produced by AI or not. That's a distraction. If a person just want to find excuse to criticize AI, and another person wants to fight back and defend AI, sure, go for it. But that's not how you would want to assess a project's code quality.
something as simple as requiring sign-offs like the DCO maybe relevant to people who care. I do think the driveby stuff may get smaller. People dont need to get stuff upstream. I have lots of patches I am keeping downmstrea and instead have a trigger system when new packages updates drop into debian and i rebuild the package with my patches on top using quill. Other systems like gentoo basically always supported this flow.
So - why bother forking or going upstream? maybe its selfish. I think publishing the patches are cool but I feel less of a need to force other people into doing what I want or even writing every possible configuration or solution. I just hack it for me
So - why bother forking or going upstream? maybe its selfish. I think publishing the patches are cool but I feel less of a need to force other people into doing what I want or even writing every possible configuration or solution. I just hack it for me
> People need to be responsible for code they commit and push anyways.
Well the GPL (which rsync is licensed under) says: "This program comes with ABSOLUTELY NO WARRANTY" so actually nobody is responsible for anything.
Well the GPL (which rsync is licensed under) says: "This program comes with ABSOLUTELY NO WARRANTY" so actually nobody is responsible for anything.
> You’re just going to make people disable Claude attribution on their commits to avoid drama.
People should be doing this regardless of drama. No reason to provide free advertising for trillion dollar corporations. Generated-by trailers are only relevant when contributing to third party projects, in that case disclosure is polite.
People should be doing this regardless of drama. No reason to provide free advertising for trillion dollar corporations. Generated-by trailers are only relevant when contributing to third party projects, in that case disclosure is polite.
The value of the Claude attribution is that you can tell at a glance who used AI.
I don't care about the advertising angle. We all know Claude by now. I want some indicator that AI was used.
I don't care about the advertising angle. We all know Claude by now. I want some indicator that AI was used.
At my employer, if AI is not used, it shows up on your performance report and you’ll be told if you don’t start using it, you will be dismissed. I work at a medium sized successful YC-backed SaaS. So here, the attribution is meaningless - they look at your Bedrock and LLM API calls as well as Claude Code history.
If the company policy is to have everyone using it then everyone is going to assume you're using it.
I don't see a need for an attribution line in this case.
I don't see a need for an attribution line in this case.
Do you fellow ICs have access to those reports and can correlate commits from you to the prompts used to create them easily?
Not currently. Each IC's report is kept private unless they voluntarily share it, and IC's don't have visibility into other IC's Claude Code or Cursor logs. I think we're moving toward a model where it will be easier to correlate commits with chats, but timeline is not clear.
Seems far more efficient to just have a line the commit message then.
> they look at your Bedrock and LLM API calls as well as Claude Code history.
This is fucking insane. How does this correlate with productivity in any way? The results are all that matters, who cares how you got there?
This is fucking insane. How does this correlate with productivity in any way? The results are all that matters, who cares how you got there?
> The results are all that matters, who cares how you got there?
If it's decent code, but attributed to AI, how does that change things? What real-world impacts does that have?
That's what I can't for the life of me figure out. Bad code is bad code regardless of who is writing it. Adding a disclaimer about how it was written is meaningless. Hell, it could say "Written by the Easter bunny" and that would have 0 impact on it's utility.
That's what I can't for the life of me figure out. Bad code is bad code regardless of who is writing it. Adding a disclaimer about how it was written is meaningless. Hell, it could say "Written by the Easter bunny" and that would have 0 impact on it's utility.
Not the commenter you replied to:
I think many people in this camp have political or ethical concerns and want to avoid contributing to or supporting the companies behind frontier-AI tools. Or they have moral or technical concerns and want to boycott usage to maintain their principles.
It should be fairly widely known at this point.
I think many people in this camp have political or ethical concerns and want to avoid contributing to or supporting the companies behind frontier-AI tools. Or they have moral or technical concerns and want to boycott usage to maintain their principles.
It should be fairly widely known at this point.
Obviously people have those concerns. The comment above specifically said:
> The value of the Claude attribution is that you can tell at a glance who used AI.
Specifies none of that, which is why I was asking the question.
> technical concerns
Which is exactly why I asked what I did. What technical concerns could possibly exist if the code is good? What does adding that attribution remove or add to technical concerns that you can't already see from the code itself?
> The value of the Claude attribution is that you can tell at a glance who used AI.
Specifies none of that, which is why I was asking the question.
> technical concerns
Which is exactly why I asked what I did. What technical concerns could possibly exist if the code is good? What does adding that attribution remove or add to technical concerns that you can't already see from the code itself?
Maybe you want to resist normalizing the use of GenAI for programming?
I know my personal choice doesn’t make much of a difference but I refuse to own a car. I advocate at my local city council to remove car storage from streets, remove parking minimums, add better transit, make the core of our city car-free. It sometimes feels easier to join in and just accept that this is the way of the world but I refuse to believe in inevitability: building cities for the benefit of cars is a choice.
Maybe some folks want to avoid AI code because they don’t want to make that choice?
I can’t say for them. But I do know there’s no sense pretending like they don’t have a point or feigning shock that someone might not have the same view as you do.
I know my personal choice doesn’t make much of a difference but I refuse to own a car. I advocate at my local city council to remove car storage from streets, remove parking minimums, add better transit, make the core of our city car-free. It sometimes feels easier to join in and just accept that this is the way of the world but I refuse to believe in inevitability: building cities for the benefit of cars is a choice.
Maybe some folks want to avoid AI code because they don’t want to make that choice?
I can’t say for them. But I do know there’s no sense pretending like they don’t have a point or feigning shock that someone might not have the same view as you do.
Are you intentionally misreading my comments?
I asked a simple question: What technical concerns could possibly exist if the code is good?
I made it very clear I wasn't talking about personal, political, or ethical arguments.
> But I do know there’s no sense pretending like they don’t have a point or feigning shock that someone might not have the same view as you do.
Where am I feigning shock? Are you reading the right comment thread before you're replying?
I asked a simple question: What technical concerns could possibly exist if the code is good?
I made it very clear I wasn't talking about personal, political, or ethical arguments.
> But I do know there’s no sense pretending like they don’t have a point or feigning shock that someone might not have the same view as you do.
Where am I feigning shock? Are you reading the right comment thread before you're replying?
> That's what I can't for the life of me figure out. Bad code is bad code regardless of who is writing it. Adding a disclaimer about how it was written is meaningless.
Maybe I was reading too much into this part of your comment.
Plenty of folks don’t separate the ethical, political, or moral from the technology. For them using it is condoning it. Like for me, owning a car is contributing to car culture. It might be inconvenient for me or seem backwards to others but it’s worth resisting. They want to know that something was written with AI so they can avoid supporting AI or condoning its use.
Maybe I was reading too much into this part of your comment.
Plenty of folks don’t separate the ethical, political, or moral from the technology. For them using it is condoning it. Like for me, owning a car is contributing to car culture. It might be inconvenient for me or seem backwards to others but it’s worth resisting. They want to know that something was written with AI so they can avoid supporting AI or condoning its use.
That's fair. I guess at that point I think adding a label to the repo as opposed to the commit would make more sense to me.
And why do you want to know that? So you can call our projects slop? Ostracize us?
Because LLMs are not humans, and the code they produce will have a different distribution of failure modes than human written code, so attribution is useful info while reviewing?
> while reviewing
As I said, disclosure is polite when contributing code to third party projects which will undergo human review.
No need for such things in one's own projects.
As I said, disclosure is polite when contributing code to third party projects which will undergo human review.
No need for such things in one's own projects.
>which will undergo human review
This can be largely assumed to be true for any open source code. It's kinda the point of open source.
This can be largely assumed to be true for any open source code. It's kinda the point of open source.
Nope. It cannot be assumed at all. Maintainer could just as easily tell Claude to review the hand written code you sent instead of spending any effort on it. Maintainer could sit on the patch for months on end only to swoop in later and rewrite it instead of engaging with you, thereby erasing your contribution and attribution. Maintainer could just ignore you entirely despite the pervasive "patches welcome" attitude.
If there's one thing I learned not to do in open source, it's to assume nonsense like that.
If there's one thing I learned not to do in open source, it's to assume nonsense like that.
I'm referring to the fact that "open source" quite literally means "readable by humans [and machines]", and anything beyond that is a subject of debate. There are more users than readers in nearly all cases, but being able to read the code as a user is a significant benefit at times, and it's one of the reasons it's such a large ecosystem in terms of both users and contributors. (it usually being free is another big reason, of course)
Even with coding agents gaining popularity, many humans still look at the code at some point.
Even with coding agents gaining popularity, many humans still look at the code at some point.
I see. That depends on how much I care about the project. My favorite ones get weeks of review and refinement, to the point I still consider them to be more or less hand written. Not all projects get to be that important.
for the same reason we want to know who wrote an article, a book, a movie, a song, a play, a journal paper, a painting, and on and on.
why do you so many people want to hide who the real author is?
we should be very weary of anyone claiming they’re the author of something when they’re absolutely not. if jon wrote a book and i take credit, that’s shady as hell.
why do you so many people want to hide who the real author is?
we should be very weary of anyone claiming they’re the author of something when they’re absolutely not. if jon wrote a book and i take credit, that’s shady as hell.
Ghostwriting is a thing.
Yes, and I respect ghostwritten work less than I do a work with a disclosed true author. Same would be true of unattributed AI-generated code.
yes because there's people who can't write but want to pretend that they can, just like the people who don't disclose they're using these tools. If you're the Gwyneth Paltrow of programming you're not making a great case for yourself, and I'd like to know before touching any of the software.
I don't know, am I? Why don't you check out my work and decide for yourself? Better than forming prejudiced opinions about others.
>Why don't you check out my work and decide for yourself?
because no person can read every line of code written in software they use, or track every commit made to a project. Integrity and authorship matters. If a person lies or obfuscates the origin of what they produce, an article, software, what have you they're doing it for a reason, otherwise they would be honest. That's not prejudice, that's recognizing deceit. And you don't eat fruit from a rotten tree.
because no person can read every line of code written in software they use, or track every commit made to a project. Integrity and authorship matters. If a person lies or obfuscates the origin of what they produce, an article, software, what have you they're doing it for a reason, otherwise they would be honest. That's not prejudice, that's recognizing deceit. And you don't eat fruit from a rotten tree.
> because no person can read every line of code written in software they use, or track every commit made to a project
Ask Claude to do it for you.
> they're doing it for a reason
And you concluded that the reason was they were pretenders who can't hack it.
That's your prejudice. Not interested in helping you categorize me, thanks.
Ask Claude to do it for you.
> they're doing it for a reason
And you concluded that the reason was they were pretenders who can't hack it.
That's your prejudice. Not interested in helping you categorize me, thanks.
That's fine; don't contribute to projects that ask for proper attribution, then, and I suppose everyone will be happy.
As I said, attribution is polite. If you're going to openly pre-judge and disrespect people on that basis though, the calculus changes.
So that the AI model that generated code can get proper credit and we'll know to use (or not use it) next time.
That's not at all what someone who wants to "tell at a glance who used AI" actually wants to know.
You don't need an AI attribution tag to recognize slop. In my experience reviewing PRs, the slop-pushers are most aggressive about stripping the AI attribution anyway. It's the normal devs who use a little bit of AI who leave it in.
The tag is helpful because AI authorship is different than the human authorship. When you work with a project or team for long enough you start to trust certain people and their intuition, but when they start submitting AI-produced code you have to reset and review it like AI code.
I use these tools a lot, too. But I want to know where the code came from so I can review it accordingly. The source matters.
> Ostracize us?
I don't know why you're so defensive. If AI wrote the code just be honest about it.
If you outsourced the code writing to some guy named Bob on Fiverr, I'd want to know that too.
The tag is helpful because AI authorship is different than the human authorship. When you work with a project or team for long enough you start to trust certain people and their intuition, but when they start submitting AI-produced code you have to reset and review it like AI code.
I use these tools a lot, too. But I want to know where the code came from so I can review it accordingly. The source matters.
> Ostracize us?
I don't know why you're so defensive. If AI wrote the code just be honest about it.
If you outsourced the code writing to some guy named Bob on Fiverr, I'd want to know that too.
> I don't know why you're so defensive.
Check it out:
https://lobste.rs/s/29pm2f/llm_generated_submissions_should_...
https://lobste.rs/s/ytim7h/collection_small_low_stakes_low_e...
Check it out:
https://lobste.rs/s/29pm2f/llm_generated_submissions_should_...
https://lobste.rs/s/ytim7h/collection_small_low_stakes_low_e...
I'm not interesting in joining into some argument you're having with someone on lobste.rs
You're not supposed to join. You said you didn't know why I was defensive. I showed you those posts as evidence of the stigma attached to LLMs and their usage. Now you know why.
Maybe you should step back and see if there's a reason why there's a stigma, instead of stubbornly insisting that there's nothing different between submitting work that you wrote yourself, vs. work done by an LLM.
Maybe you should stop making excuses for the summary dismissal of people's years old projects as slop on the basis that an LLM touched it.
It doesn’t help your case that your response is to say “well I’ll just hide my use.” That’s fraud.
No.
Some people prefer organic grown food for all kinds of reasons, does it matter to you they would want the same for code? (Also, I'm not picking a side here)
It matters when I'm contributing to their projects. In that case I'll go out of my way to be polite and learn their rules.
[deleted]
That's really all anyone is asking of you. It's odd that this is your position, and yet you seem to be arguing (in your other comments) in a way that seems like you think that you should be able to do whatever you want, with any project, their requirements be damned.
Because the reasons for doing it matter. The "different failure modes" argument is a fair point. Since it changes the way code is reviewed, it is polite to disclose use of LLMs.
But you and others in this thread seem hellbent on stigmatizing it to the point you take it as evidence of someone's incompetence. So I'm not at all sympathetic to your "requirements".
That's really all anyone's asking of you: enough respect for your fellow programmers that you avoid pre-judging them. If you can't do that, then what do we care about your "requirements"?
But you and others in this thread seem hellbent on stigmatizing it to the point you take it as evidence of someone's incompetence. So I'm not at all sympathetic to your "requirements".
That's really all anyone's asking of you: enough respect for your fellow programmers that you avoid pre-judging them. If you can't do that, then what do we care about your "requirements"?
So we can know which commits will be infringing others’ copyright.
If Claude is actually good enough to commit to rsync, of course I'm going to look at that and think "it's good enough for my side project too." And (benefit to companies aside) that is info it is useful to know, if it's true.
Yeah, this is why it's obnoxious and this is why scummy marketers do it. If you don't aggressively turn it off, they leech an implicit endorsement out of you.
- Sent from my iPhone
- Sent from my iPhone
Alto hug the iphone sigoff is hilaripus sonce fhe meyboard is so bad it always comes across asa an ask doe forgivebeds
— Sent from my iPhone
— Sent from my iPhone
Indeed. The best endorsement is done explicitly by obnoxious users.
I use Linux, btw.
I use Linux, btw.
Is that a bad thing? I mean from the perspective of Anthropic's marketing department sure, but if agents are just another type of tool in developer's tool belt - as I see people recently like to claim - attribution feels kinda weird. In the end it is the developer who is responsible for their commits.
Yeah I think it's a bad thing. It's context about how open source code was written that is lost.
And I guess maybe there's no such thing as bad press but at least in this cases it doesn't seem like effective marketing for Anthropic.
And I guess maybe there's no such thing as bad press but at least in this cases it doesn't seem like effective marketing for Anthropic.
“Don’t get mad at people for doing something unethical or immoral, or they’ll do something unethical or immoral!”
Disabling attribution of LLM-generated code is fraud, because you’re saying you wrote the code.
Of course that fits right in with the use of an LLM to generate code in the first place, since what it’s actually doing is regurgitating its inputs stripped of any license and copyright notice.
Disabling attribution of LLM-generated code is fraud, because you’re saying you wrote the code.
Of course that fits right in with the use of an LLM to generate code in the first place, since what it’s actually doing is regurgitating its inputs stripped of any license and copyright notice.
I'm very certain that this is not fraud, across multiple legal systems, both roman and common law. In both cases fraud requires a person is deprived of a material good. Neither the defrauded person or their material loss is present in this case. Maybe there is a oddball legal system somewhere in the world where fraud is something entirely different, but i doubt it. "Fraud", just like "Decorator Pattern" is a well established concept and pretty simple concept, even if there are edge cases. This does not fit at all.
In academia this is miss-attribution, outside of academia this does not exist.
This is clearly not not copyright infringement either as LLMs do not claim copyright, nor could they. Just like the photograph taken by the monkey, or pictures drawn by crows. LLM output is not a creative work either.
If this is unethical or immoral is a totaly different question. I really dont think so and I dont think you argue that position well.
In academia this is miss-attribution, outside of academia this does not exist.
This is clearly not not copyright infringement either as LLMs do not claim copyright, nor could they. Just like the photograph taken by the monkey, or pictures drawn by crows. LLM output is not a creative work either.
If this is unethical or immoral is a totaly different question. I really dont think so and I dont think you argue that position well.
It is misrepresentation for gain, that gain does not need to be monetary to be material. For example, it can be reputational.
It also is copyright infringement, because what the LLM “generates” are actually portions of its training set, which were covered by copyright. Just passing through an LLM does not remove that copyright from that work.
It also is copyright infringement, because what the LLM “generates” are actually portions of its training set, which were covered by copyright. Just passing through an LLM does not remove that copyright from that work.
No, you are wrong.
In German and French (roman) legal systems this is a "Vermögensdelikt", and explicitly about material damage and gain. Yes, common law can be more broad (in canada it isn't really, it just also includes service, btw.), and yet it clearly does not meet the definition, as there is a damaged/defraued party and fraudulent/gaining party. We are not talking about somebody usurping somebody else reputation, after all.
You misuse a technical term that is well established since antiquity.
You do not know what this word means. If you want to argue about semantics, look up the definition. This works especially well for legal terms as laws define them.
(That said, IANAL and there are very many different legal systems and I am not ruling out there exists one that is competently different - laws can be changed a will, after all.)
It is also obviously not copyright infringement, because this is simply not how copyright works, at all. I cannot and will explain of all copyright here. Instead I will point this out: Every code produced by a human who read copyrighted code would fall under your definition.
In German and French (roman) legal systems this is a "Vermögensdelikt", and explicitly about material damage and gain. Yes, common law can be more broad (in canada it isn't really, it just also includes service, btw.), and yet it clearly does not meet the definition, as there is a damaged/defraued party and fraudulent/gaining party. We are not talking about somebody usurping somebody else reputation, after all.
You misuse a technical term that is well established since antiquity.
You do not know what this word means. If you want to argue about semantics, look up the definition. This works especially well for legal terms as laws define them.
(That said, IANAL and there are very many different legal systems and I am not ruling out there exists one that is competently different - laws can be changed a will, after all.)
It is also obviously not copyright infringement, because this is simply not how copyright works, at all. I cannot and will explain of all copyright here. Instead I will point this out: Every code produced by a human who read copyrighted code would fall under your definition.
No, you are wrong. You are either willfully misunderstanding what I’m calling fraud, or you are misinformed as to what “material gain” means in many legal systems.
With respect to the former, “fraud” is a shorthand for “fraudulent misrepresentation,” which is what you’re doing when you take someone else’s IP and try to contribute it to a project without securing the right to do so. It can be read as implicit in the attempt to contribute to the project that you have secured this permission (or do not need to, because the work is original to you). Whether the code came out of an LLM or was copied from another project or Stack Overflow doesn’t matter, it’s that you’re misrepresenting the rights you have that’s the fraudulent part.
For the latter, I specifically pointed out that the gain from fraudulent misrepresentation need not be monetary. The gain can be reputational or any other sort of benefit. For example, someone pretending to a fictional person to gain access to a space they otherwise wouldn’t is still committing fraud.
Finally, you’re wrong about whether the output of an LLM infringes copyright of material in its training set. Just running a copyrighted work through an LLM does not remove the copyright on that work if reproduced by the LLM.
With respect to the former, “fraud” is a shorthand for “fraudulent misrepresentation,” which is what you’re doing when you take someone else’s IP and try to contribute it to a project without securing the right to do so. It can be read as implicit in the attempt to contribute to the project that you have secured this permission (or do not need to, because the work is original to you). Whether the code came out of an LLM or was copied from another project or Stack Overflow doesn’t matter, it’s that you’re misrepresenting the rights you have that’s the fraudulent part.
For the latter, I specifically pointed out that the gain from fraudulent misrepresentation need not be monetary. The gain can be reputational or any other sort of benefit. For example, someone pretending to a fictional person to gain access to a space they otherwise wouldn’t is still committing fraud.
Finally, you’re wrong about whether the output of an LLM infringes copyright of material in its training set. Just running a copyrighted work through an LLM does not remove the copyright on that work if reproduced by the LLM.
You are misinformed, I suspect you have no idea what you are talking about.
As I said, I do not know all legal systems in the world. If there one where "material gain" matches your idea, please cite the law or a case that includes LLM usage. As I explained in the canadian law even includes services and yet it is so much very much not matching the defintion for reasons explained.
I do understand very well what you mean by "fraud", I do not miss represent it - your opinion on what it should be is plain and simple wrong. I explained why in my previous posts.
You are under the impression that legal science is some kind of folk etymology. It absolutely is not. Fraud is §263 StGB, Art. 313-1 Code penal or §380 of the canadian criminal code. (They all are remarkably similar, because they share a millennia old tradition. Making them IMHO fascinating cultural artifacts.) Here [0] is a structured version of on of these texts. Think of it as a symbolic execution of the law. You can see there is structural mismatch with your "case". Nobody ubsurbs anything from somebody else, and all three laws incude that in their defintion. That was my original claim.
You think you somehow can make up your own private definitions, develop your own private theories about them, apply them and argue about the semantics your made up terms. That is the opposite of how jurisprudence works. It rigorous, with well established scientific and scholastic methods. It operates on term defined by the law. In the case of "fraud" the previous citations, especially in criminal law, and nothing else. German legal science has its own theory what counts as "nothing else" under the name "Wortlautgrenze". These terms and methods vary from jurisdiction to jurisdiction, but by surprisingly little.
Dont call your code a decorator pattern, because you think it is decorative. Different pattern libraries have definitions for that and you need to be able to argue it fits. Like wise, if you feel something involves some kind of misrepresentation its probably not fraud. If things have different names, that probably for a good reasons, especially in legal science.
[0] https://www.iurastudent.de/schemata/schema-zum-betrug-263-i-...
As I said, I do not know all legal systems in the world. If there one where "material gain" matches your idea, please cite the law or a case that includes LLM usage. As I explained in the canadian law even includes services and yet it is so much very much not matching the defintion for reasons explained.
I do understand very well what you mean by "fraud", I do not miss represent it - your opinion on what it should be is plain and simple wrong. I explained why in my previous posts.
You are under the impression that legal science is some kind of folk etymology. It absolutely is not. Fraud is §263 StGB, Art. 313-1 Code penal or §380 of the canadian criminal code. (They all are remarkably similar, because they share a millennia old tradition. Making them IMHO fascinating cultural artifacts.) Here [0] is a structured version of on of these texts. Think of it as a symbolic execution of the law. You can see there is structural mismatch with your "case". Nobody ubsurbs anything from somebody else, and all three laws incude that in their defintion. That was my original claim.
You think you somehow can make up your own private definitions, develop your own private theories about them, apply them and argue about the semantics your made up terms. That is the opposite of how jurisprudence works. It rigorous, with well established scientific and scholastic methods. It operates on term defined by the law. In the case of "fraud" the previous citations, especially in criminal law, and nothing else. German legal science has its own theory what counts as "nothing else" under the name "Wortlautgrenze". These terms and methods vary from jurisdiction to jurisdiction, but by surprisingly little.
Dont call your code a decorator pattern, because you think it is decorative. Different pattern libraries have definitions for that and you need to be able to argue it fits. Like wise, if you feel something involves some kind of misrepresentation its probably not fraud. If things have different names, that probably for a good reasons, especially in legal science.
[0] https://www.iurastudent.de/schemata/schema-zum-betrug-263-i-...
"Disabling attribution of LLM-generated code is fraud, because you’re saying you wrote the code."
Should there by attribution for Google or Stack Overflow copy/paste? Who should we bully about this?
Should there by attribution for Google or Stack Overflow copy/paste? Who should we bully about this?
> Should there by attribution for Google or Stack Overflow copy/paste?
Obviously, and I'm a bit taken aback that anyone thinks otherwise.
Obviously, and I'm a bit taken aback that anyone thinks otherwise.
Yes, in fact, this is why people who do that are looked down upon.
They are in fact committing fraud if they do not attribute the code in their commit properly, because by committing it they’re claiming to have rights by virtue of authorship that they do not have. (Namely, the right to contribute that code to the project,.) They may also be committing copyright infringement, depending on the copyright and license status of some code they found via Google or Stack Overflow.
It’s always fascinating to me to see how many people on Hacker News have such extremely poor understanding of how intellectual property actually works, and how misrepresenting themselves or their work can actually have consequences.
They are in fact committing fraud if they do not attribute the code in their commit properly, because by committing it they’re claiming to have rights by virtue of authorship that they do not have. (Namely, the right to contribute that code to the project,.) They may also be committing copyright infringement, depending on the copyright and license status of some code they found via Google or Stack Overflow.
It’s always fascinating to me to see how many people on Hacker News have such extremely poor understanding of how intellectual property actually works, and how misrepresenting themselves or their work can actually have consequences.
Are there any court cases you can point to that have clearly established that using LLM generated code can be a copyright violation? My understanding is that this is very far from being settled law.
What cases can you cite that have determined it’s not?
It’s clear on its face that LLMs can and do store and reproduce copyrighted works; using a form of (somewhat) lossy data compression. And using a lossy stochastic or perceptual form of compression to reproduce a copyrighted work doesn’t somehow make it not storage or reproduction, otherwise sharing MP3 files wouldn’t be copyright infringement.
Anyone engaging in responsible risk management should assume that anything LLM-generated is infringing until determined otherwise by the courts, not the other way around.
It’s clear on its face that LLMs can and do store and reproduce copyrighted works; using a form of (somewhat) lossy data compression. And using a lossy stochastic or perceptual form of compression to reproduce a copyrighted work doesn’t somehow make it not storage or reproduction, otherwise sharing MP3 files wouldn’t be copyright infringement.
Anyone engaging in responsible risk management should assume that anything LLM-generated is infringing until determined otherwise by the courts, not the other way around.
There are billion-dollar entities preparing to fight this very question out in court as we speak.
Your interpretation of the law is certainly plausible, but it is clearly not a settled question.
If you really are so confident, go bet on Kalshi and make some easy money: https://kalshi.com/markets/kxnytoai/new-york-times-wins-open...
Your interpretation of the law is certainly plausible, but it is clearly not a settled question.
If you really are so confident, go bet on Kalshi and make some easy money: https://kalshi.com/markets/kxnytoai/new-york-times-wins-open...
Outside of situations where it is required by contract, attributing AI usage is a courtesy, nothing more.
It's only fraud if a person signed their name stating such.
Their name being attached to the commit is itself, irrelevant, as their is no way to submit a patch otherwise. You could use a fake name, but you're just moving this fraud problem around.
You're going to have a hard time convincing anyone that using a tool constitutes fraud. Frankly, it's silly, if not genuinely stupid.
Film photographers in the early 2000s routinely called digital "not real photography" and Photoshop "cheating" because you could delete bad shots and fix everything later. Traditional musicians and critics dismissed drum machines, synthesizers, and autotune as soulless tools.
Their name being attached to the commit is itself, irrelevant, as their is no way to submit a patch otherwise. You could use a fake name, but you're just moving this fraud problem around.
You're going to have a hard time convincing anyone that using a tool constitutes fraud. Frankly, it's silly, if not genuinely stupid.
Film photographers in the early 2000s routinely called digital "not real photography" and Photoshop "cheating" because you could delete bad shots and fix everything later. Traditional musicians and critics dismissed drum machines, synthesizers, and autotune as soulless tools.
Intent and custom both matter quite a bit in law. It is customary to treat the name attached to a commit as the copyright holder of any changes represented by that commit, just as it was for the sender of an email containing a patch back when that was how such work was done.
Often this is also spelled out in a project’s contribution guidelines, and some projects have even had more explicit copyright assignment policies they required contributors to agree to, but the lack of such guidelines or assignment policies does not mean the custom as normally observed in the field is irrelevant.
Often this is also spelled out in a project’s contribution guidelines, and some projects have even had more explicit copyright assignment policies they required contributors to agree to, but the lack of such guidelines or assignment policies does not mean the custom as normally observed in the field is irrelevant.
> Intent and custom both matter quite a bit in law.
Indeed, and I'm not aware of any (Western, at least) legal system that would consider it fraud to not disclose that an LLM had generated some code.
I'd like to gently point out that your insistence of fraud here is hurting your overall argument, and is causing people to focus on the language you're using, instead of the substance of what you're trying to say. I do agree with you that people should disclose LLM generation when writing commits. But the way you're going about arguing this "fraud" thing is an unproductive dead end.
Indeed, and I'm not aware of any (Western, at least) legal system that would consider it fraud to not disclose that an LLM had generated some code.
I'd like to gently point out that your insistence of fraud here is hurting your overall argument, and is causing people to focus on the language you're using, instead of the substance of what you're trying to say. I do agree with you that people should disclose LLM generation when writing commits. But the way you're going about arguing this "fraud" thing is an unproductive dead end.
The fraud isn’t (directly) in hiding that the LLM generated some code. The fraud is in the (implicit) misrepresentation of ownership of and/or rights to the code.
When you send a patch or pull request to a project, you’re saying (implicitly) that you have the necessary rights to contribute the intellectual property it contains. If you used an LLM to “generate” some of it, that is not necessarily the case.
A similar situation would occur if you agreed to pay someone else to create a patch, and then submitted it under your own name without paying them. Because it’s a work for hire, it’s not yours until they’re paid for it, so you’re fraudulently misrepresenting your rights to that patch to the project. If you did pay the creator, you don’t have to attribute them unless it’s in the contract between you and the creator, or unless the project requires such attribution.
When you send a patch or pull request to a project, you’re saying (implicitly) that you have the necessary rights to contribute the intellectual property it contains. If you used an LLM to “generate” some of it, that is not necessarily the case.
A similar situation would occur if you agreed to pay someone else to create a patch, and then submitted it under your own name without paying them. Because it’s a work for hire, it’s not yours until they’re paid for it, so you’re fraudulently misrepresenting your rights to that patch to the project. If you did pay the creator, you don’t have to attribute them unless it’s in the contract between you and the creator, or unless the project requires such attribution.
[deleted]
This argument gets trotted out every time but it doesn't convince me of anything. Yes, calling things out creates an incentive for people to hide them, but so what?
Setting aside the whole AI = bad argument, let's do a metaphor. Tax evasion is bad and unethical and you should call it out where you see it. But wait, that creates an incentive for people to hide it! So I'd better not call it out, it's best to just keep my mouth shut.
Setting aside the whole AI = bad argument, let's do a metaphor. Tax evasion is bad and unethical and you should call it out where you see it. But wait, that creates an incentive for people to hide it! So I'd better not call it out, it's best to just keep my mouth shut.
I'd be willing to be that an undisclosed LLM disclosure will follow a developer around for the rest of their career
That kind of fraud absolutely should. (I suspect you mean “undisclosed LLM use.”)
Thank you, that's what I meant
I'm willing to be that in two years that's going to be completely irrelevant because the amount of code written by hand will drop to less than 10%.
I mean, I don't think commits are the place for tool attributions. I want to know what the change was, I'm not really interested in your tool selection (put that in the PR if it's relevant). It'd be just as irrelevant to see "written on my macbook in neovim"
potsandpans(2)
This article is a rant disguised as data analysis.
I don't know how to word this in a non-confrontational, respectful way, but this article just feels like ammo for your next "debate with your anti-AI ennemies" where you get to say "look, I proved with data that those people had a disproportionate reaction and have double standards, therefore anyone who dislikes LLMs or their impact are the same!". Like, sorry, I know that sounds really reductive, but this really is the vibe I get when reading this and your other replies where you repeatedly talk about "showing the hypocrisy and double standards".
The global LLM discourse has grown massive, it spans trillions of dollars in promises and investments and affects pretty much everyone, so it's the easiest thing in the world for both sides to just find some people being assholes in the other camp and say "look, here's how [other camp] behaves".
The irrational, extreme, and heinous reactions are partly bandwagoning, and you can go about your day thinking that anyone who reacts like that is evil. But if you wanna dig a bit further, you'll notice that the entire media sphere has been screaming in everyone's ears for a few years now, that they're expandable, low-value-human-capital. All the money in the world (exaggerating a little) is being spent on making sure to remind anyone who opens a computer, opens a website, looks at a billboard, or turns on his tv... that their boss really really really wants to replace them.
Now you'll say that the friendly rsync contributor has nothing to do with any of this and... well yeah he doesn't. You don't need to agree with an emotional response to understand where it's coming from, and even if you're still dead set on considering them "the enemy", then understanding why the anti-AI crowd reacts like that is STILL a positive for you.
I don't know how to word this in a non-confrontational, respectful way, but this article just feels like ammo for your next "debate with your anti-AI ennemies" where you get to say "look, I proved with data that those people had a disproportionate reaction and have double standards, therefore anyone who dislikes LLMs or their impact are the same!". Like, sorry, I know that sounds really reductive, but this really is the vibe I get when reading this and your other replies where you repeatedly talk about "showing the hypocrisy and double standards".
The global LLM discourse has grown massive, it spans trillions of dollars in promises and investments and affects pretty much everyone, so it's the easiest thing in the world for both sides to just find some people being assholes in the other camp and say "look, here's how [other camp] behaves".
The irrational, extreme, and heinous reactions are partly bandwagoning, and you can go about your day thinking that anyone who reacts like that is evil. But if you wanna dig a bit further, you'll notice that the entire media sphere has been screaming in everyone's ears for a few years now, that they're expandable, low-value-human-capital. All the money in the world (exaggerating a little) is being spent on making sure to remind anyone who opens a computer, opens a website, looks at a billboard, or turns on his tv... that their boss really really really wants to replace them.
Now you'll say that the friendly rsync contributor has nothing to do with any of this and... well yeah he doesn't. You don't need to agree with an emotional response to understand where it's coming from, and even if you're still dead set on considering them "the enemy", then understanding why the anti-AI crowd reacts like that is STILL a positive for you.
Why is the guy being rigorous worthy of criticism, but the guys being idiots aren't? Did you post any similar calm-down comments in either of the HN threads on the original attacks?
I am more inclined to be critical of AI boosters, so what? Am I supposed to crumble under the weight of immense cognitive dissonance because I have... a stance in the discourse?
These guys on the github thread aren't my friends, I have no concern for them embarrassing themselves or leaving a bad digital footprint by drawing ms paint gore. I also have no concern for OP, but it just so happened to be the post I found, and I just so happened to be in the mood to leave a comment.
Engaging in LLM discourse is already a waste of my time, I'm not going to waste more of it just to avoid fallacious accusations of double standards because I didn't "do the same for the other side".
These guys on the github thread aren't my friends, I have no concern for them embarrassing themselves or leaving a bad digital footprint by drawing ms paint gore. I also have no concern for OP, but it just so happened to be the post I found, and I just so happened to be in the mood to leave a comment.
Engaging in LLM discourse is already a waste of my time, I'm not going to waste more of it just to avoid fallacious accusations of double standards because I didn't "do the same for the other side".
Strange thing to say. The post does a good job of showing that there is no evidence Claude had anything to do with the regressions.
Your problem is that this was shown? You don't value epistemics -- you care about the ideology more than truth. Even if you don't like AI you should still do it in the right way.
Your comment comes across as more unself aware and more destructive. Lets keep this place truth first and ideology second.
Your problem is that this was shown? You don't value epistemics -- you care about the ideology more than truth. Even if you don't like AI you should still do it in the right way.
Your comment comes across as more unself aware and more destructive. Lets keep this place truth first and ideology second.
This article merely showed that one particular sample (two releases of rsync) or not statistically significantly different (p > 0.05) on one particular metric (bugs per commit). All that says is that you cannot generalize group difference over the population using that sample.
This still leaves the anecdotal evidence. And anecdotal evidence is still evidence, and in the absence of better evidence, it is perfectly rational to react based on the evidence you do have.
This still leaves the anecdotal evidence. And anecdotal evidence is still evidence, and in the absence of better evidence, it is perfectly rational to react based on the evidence you do have.
Yeah yeah the usual "look, these anti-ai people are so EMOTIONAL and HYSTERICAL while we're very logical and fact-based", I've lurked for a while so I read that one plenty of times.
OP spends a lot of time time doing statistics, but when another person replies "hey, 2 claude-authored features is not really statistically significant", the author literally agrees and says "my point was to show that you can't draw conclusions". Direct quote:
> I'm only trying to show there's no evidence for the anti-AI hypothesis
---
And here's another thing. How exactly is it self-unaware to say "Hey, I get your frustration with people being assholes, I'm not excusing it, but it can never hurt to understand why some of them have become so extreme in their vitriol, here's a few reasons for their feelings".
Is "feelings" a curse word or something? What's so wrong with understanding the emotional component of the AI discourse?
Talking about "emotions" is not destructive when the topic at hand is literally people being driven by emotion under a github thread.
What the article is saying is "these people are acting irrational because they're evil and the enemy, so here's how I prove them wrong with statistics!", and my comment to the article was "hey, you seem to go with the assumption that this is all based on pure evil, here are a few reasons why people might get tired, and then angry, about this whole thing".
You quite literally exemplify my point when I said that the analysis is mostly just >ammo for your next "debate with your anti-AI ennemies", it's a tool that allows you to not engage and dismiss any argument as "not on the side of truth because not on the side of the numbers".
All of this even though, and I need to state this again, I never once rejected the analysis or the results that OP came to, all I did was point out that OP is also engaging in "us vs them" think with the occasional "wink wink, CLASSIC AI hater amirite?" sprinkled in the article.
OP spends a lot of time time doing statistics, but when another person replies "hey, 2 claude-authored features is not really statistically significant", the author literally agrees and says "my point was to show that you can't draw conclusions". Direct quote:
> I'm only trying to show there's no evidence for the anti-AI hypothesis
---
And here's another thing. How exactly is it self-unaware to say "Hey, I get your frustration with people being assholes, I'm not excusing it, but it can never hurt to understand why some of them have become so extreme in their vitriol, here's a few reasons for their feelings".
Is "feelings" a curse word or something? What's so wrong with understanding the emotional component of the AI discourse?
Talking about "emotions" is not destructive when the topic at hand is literally people being driven by emotion under a github thread.
What the article is saying is "these people are acting irrational because they're evil and the enemy, so here's how I prove them wrong with statistics!", and my comment to the article was "hey, you seem to go with the assumption that this is all based on pure evil, here are a few reasons why people might get tired, and then angry, about this whole thing".
You quite literally exemplify my point when I said that the analysis is mostly just >ammo for your next "debate with your anti-AI ennemies", it's a tool that allows you to not engage and dismiss any argument as "not on the side of truth because not on the side of the numbers".
All of this even though, and I need to state this again, I never once rejected the analysis or the results that OP came to, all I did was point out that OP is also engaging in "us vs them" think with the occasional "wink wink, CLASSIC AI hater amirite?" sprinkled in the article.
If someone spouts untruths, that is a failing of theirs. Talking about how there's emotions of theirs that back this promulgation of untruths, emotions which you could understand if you tried, is putting the cart before the horse. First express things that are not trivially false, then I will try to understand what you mean by it.
You keep trying to twist what I'm saying into "guys you have to be nice to the ai haters and meet them at the middle, even when they insult you :))))". Once again you have this kneejerk reaction to the words "emotions" and "feelings" as if it would make you weak to acknowledge that "humans feeling a certain way act a certain way".
I'll once again have to remind you that all of what I said was just "yeah these guys are really pissed off, here's why they might be" and never once asked any of you to agree with them, or that their conclusions about rsync were right, or anything like that.
You act like "understanding" is only ever a gift/favor you do to someone else. While it's true that in many cases it is, "understanding" can also be something that just helps you and doesn't require interaction or agreement, it's when you think "oh there's no debating with these guys so it's probably not worth it to engage" or "oh these people are talking about X but it looks like the Y underlying issue is the actual problem, so talking about X might be a waste of time".
You completely disregard human emotions as if doing so makes you stronger, but all it does is make you more confused, surprised and angry whenever you're faced with irrational reactions you can't understand.
Now if you decide to interpret all of that as me saying "I am in moral agreement with the dogpiling and the witch hunts shown in that github thread", that is a failing of yours.
I'll once again have to remind you that all of what I said was just "yeah these guys are really pissed off, here's why they might be" and never once asked any of you to agree with them, or that their conclusions about rsync were right, or anything like that.
You act like "understanding" is only ever a gift/favor you do to someone else. While it's true that in many cases it is, "understanding" can also be something that just helps you and doesn't require interaction or agreement, it's when you think "oh there's no debating with these guys so it's probably not worth it to engage" or "oh these people are talking about X but it looks like the Y underlying issue is the actual problem, so talking about X might be a waste of time".
You completely disregard human emotions as if doing so makes you stronger, but all it does is make you more confused, surprised and angry whenever you're faced with irrational reactions you can't understand.
Now if you decide to interpret all of that as me saying "I am in moral agreement with the dogpiling and the witch hunts shown in that github thread", that is a failing of yours.
I don't think any of this relates to the post it's replying to. I appreciate the totally failed attempt to psychoanalyze my emotions, though. If you think I'm 'confused, surprised, and angry', you don't seem to be as good at the emotions thing as you keep acting like you are.
But it does relate, dunno what to tell you. You think it doesn't relate because your reasoning stops at "they're not right so there is literally nothing else that's relevant about any of this", and what I'm saying to OP is "they're not right, and here's why those people would go around being wrong on the internet, and why a big (statistically insignificant) debunk will mostly just be you preaching to the choir", I'm steering the conversation that way because I felt it would be more productive.
There's nothing more I can say that I haven't already said at this point, understanding that people have emotions (and once again, understanding!= agreement) literally has no downside for you when looking at any kind of discourse.
Call me preachy or whatever, I'm just trying to find a topic of discussion that's not just the usual yelling past eachother: "Large providers have ROI!!" vs "Large providers are in the red!!", not opening that can of worm.
There's nothing more I can say that I haven't already said at this point, understanding that people have emotions (and once again, understanding!= agreement) literally has no downside for you when looking at any kind of discourse.
Call me preachy or whatever, I'm just trying to find a topic of discussion that's not just the usual yelling past eachother: "Large providers have ROI!!" vs "Large providers are in the red!!", not opening that can of worm.
What in the AI discourse is trivially false? On one side we have almost all the money in the world, owners of giant social media companies, with easy access to media, who buy ads, etc. to tell you that half of us will be out of jobs in the next 6 months, that AI superinteligence is coming next year, that singularity is upon us, that Claude Mythos will cause so much harm it has to be contained, etc. etc.
I consider all of the above trivially false untruths. But on the other hand you have rsync users who are fed up with all the lies, all the propaganda, all the fear mongering brought up by literally the richest people the world has ever seen, who have experienced AI slop first hand, who have been tricked by AI music, are fed up by all the AI generated posters at their local coffee stand, have given up on trying to correct their coworkers AI generated code during peer review. And now these rsync users see AI slop has been pushed to their favorite tools, and see a new bugs popping up at the same time.
I consider the latter a completely understandable reaction. Yes they jumped to conclusion, but in doing so they have evidence. They have both anecdotal and circumstantial evidence for their conclusions. In comparison, the AI singularity people have nothing but vibes and science fiction behind their conclusions.
I consider all of the above trivially false untruths. But on the other hand you have rsync users who are fed up with all the lies, all the propaganda, all the fear mongering brought up by literally the richest people the world has ever seen, who have experienced AI slop first hand, who have been tricked by AI music, are fed up by all the AI generated posters at their local coffee stand, have given up on trying to correct their coworkers AI generated code during peer review. And now these rsync users see AI slop has been pushed to their favorite tools, and see a new bugs popping up at the same time.
I consider the latter a completely understandable reaction. Yes they jumped to conclusion, but in doing so they have evidence. They have both anecdotal and circumstantial evidence for their conclusions. In comparison, the AI singularity people have nothing but vibes and science fiction behind their conclusions.
> "look, here's how [other camp] behaves"
Motivated reasoning in both camps.
There are folks incentivized by AI: engineers and managers working for AI related companies who justify their beliefs with selective facts. And then there are engineers who are threatened by AI and are extra-sensitive to slop.
The article is missing the point that once camps have made up their minds - no amount of analysis is going to change that.
Motivated reasoning in both camps.
There are folks incentivized by AI: engineers and managers working for AI related companies who justify their beliefs with selective facts. And then there are engineers who are threatened by AI and are extra-sensitive to slop.
The article is missing the point that once camps have made up their minds - no amount of analysis is going to change that.
Why it is probably and regrettably true that few people in camps will change their mind, data analysis can help people who haven't been captured yet to either stay away from the camps or at least fall into the "more correct" one.
Stay out of camps, people!
Stay out of camps, people!
> It does not control for commit complexity, security intensity, or bug severity. It does not distinguish between a one-line typo fix and a CVE patch. It is a blunt instrument. But the critics' accusation is also blunt: "Claude is making things worse." A blunt instrument is the fairest response.
If by fairest you mean to say that this analysis and response is sufficient, then I'm sorry but I have to disagree. We really need to understand if the nature of the bugs are worse from a user's perspective. Even if the rate stayed unchanged, if the result is the perceived quality of the software declined then I would personally consider that worse, especially if I were a project maintainer.
That's not meant to be wholly dismissive either. But in general, I don't think quantitative analysis alone is enough to fully answer this type of question.
If by fairest you mean to say that this analysis and response is sufficient, then I'm sorry but I have to disagree. We really need to understand if the nature of the bugs are worse from a user's perspective. Even if the rate stayed unchanged, if the result is the perceived quality of the software declined then I would personally consider that worse, especially if I were a project maintainer.
That's not meant to be wholly dismissive either. But in general, I don't think quantitative analysis alone is enough to fully answer this type of question.
But it is fair. Up to this point I have yet to see anyone say they did an analysis of the code and found X regressions of Y severity. All they say is "there are more bugs because LLM". This analysis, which you can verify yourself if you wish, says "the bugs [number of] are pretty average even with LLM", which is a direct response to that. If you'd like a more nuanced analysis you're welcome to do one and share the result, if you're so inclined.
That which is asserted without evidence can be dismissed without evidence. This is more evidence, and of greater rigor, than was used to make the assertions. That's good enough for me. If someone wants to actually do the work to support the original claims with better evidence, great. I'd love to see it. Until then, I'm going to not worry about this issue.
The burden of proof is on the one making the claim?
This post just gives me more questions than answers and I'm unable to form a decision:
* Why was v3.4.1 the most buggy, right before the Claude commits? Why did "nobody notice"? It's way to strange to just say welp, it must be human error. * Why does v3.4.2 have 0 bugs, or 0 bug score. And why was such an outlier (no other commit seemingly has this??) allowed to mix into aggregate statistics and bring all the "is Claude buggy?" scores down. Tbh idk how that _wasn't_ a red flag in the author's analysis...
This article feels like half of an analysis presented as a highly complex finished product due all the advanced stats they're running.
* Why was v3.4.1 the most buggy, right before the Claude commits? Why did "nobody notice"? It's way to strange to just say welp, it must be human error. * Why does v3.4.2 have 0 bugs, or 0 bug score. And why was such an outlier (no other commit seemingly has this??) allowed to mix into aggregate statistics and bring all the "is Claude buggy?" scores down. Tbh idk how that _wasn't_ a red flag in the author's analysis...
This article feels like half of an analysis presented as a highly complex finished product due all the advanced stats they're running.
> Why was v3.4.1 the most buggy, right before the Claude commits? Why did "nobody notice"? It's way to strange to just say welp, it must be human error.
Why wouldn't it be except question begging priors assuming it couldn't be?
> Why does v3.4.2 have 0 bugs, or 0 bug score. And why was such an outlier (no other commit seemingly has this??) allowed to mix into aggregate statistics and bring all the "is Claude buggy?" scores down.
My original metrics which didn't filter out feature requests and questions had it at four bugs and prior to that it was even higher and it didn't make much of a difference to the overall analysis (fell well within the IQR, the lower end of it too). Also, removing one outlier just because it looks kind of funny to you, especially when we only have two Claude releases at all, would be worse in my opinion and more arbitrary.
Why wouldn't it be except question begging priors assuming it couldn't be?
> Why does v3.4.2 have 0 bugs, or 0 bug score. And why was such an outlier (no other commit seemingly has this??) allowed to mix into aggregate statistics and bring all the "is Claude buggy?" scores down.
My original metrics which didn't filter out feature requests and questions had it at four bugs and prior to that it was even higher and it didn't make much of a difference to the overall analysis (fell well within the IQR, the lower end of it too). Also, removing one outlier just because it looks kind of funny to you, especially when we only have two Claude releases at all, would be worse in my opinion and more arbitrary.
> Why wouldn't it be except question begging priors assuming it couldn't be?
A multitude of reasons? A change in maintainer. A change in the mental state of a maintainer. A sudden focus by the community on a given undesirable behavior. Someone else here suggested use of Claude AI before it was disclosured. The framing implies that it was human-produced coding error, but my point is it could be _any other human error_ or even just some odd benign human behavior (a stampede of bug submitters), affecting the data. Which does not lead to the conclusion that AI code > human code. Not looking at these potentials is so unsatisfying.
> My original metrics which didn't filter out feature requests...
It still feels like a lot of weight of the phrase "If that doesn't look like a red flag to you, you'd be right." hinges on the fact that one of the versions has 0 bugs and it really killed the weight of that statement for me, because the oddity of there being 0 bugs just wasn't explained.
---
Could you please post the duckdb file that has the raw bug -> severity + version mapping to the GitHub repo? I have a desire to dig into this myself
A multitude of reasons? A change in maintainer. A change in the mental state of a maintainer. A sudden focus by the community on a given undesirable behavior. Someone else here suggested use of Claude AI before it was disclosured. The framing implies that it was human-produced coding error, but my point is it could be _any other human error_ or even just some odd benign human behavior (a stampede of bug submitters), affecting the data. Which does not lead to the conclusion that AI code > human code. Not looking at these potentials is so unsatisfying.
> My original metrics which didn't filter out feature requests...
It still feels like a lot of weight of the phrase "If that doesn't look like a red flag to you, you'd be right." hinges on the fact that one of the versions has 0 bugs and it really killed the weight of that statement for me, because the oddity of there being 0 bugs just wasn't explained.
---
Could you please post the duckdb file that has the raw bug -> severity + version mapping to the GitHub repo? I have a desire to dig into this myself
I'll do that when I get a chance
> Tbh idk how that _wasn't_ a red flag in the author's analysis...
Because he didn't analyze shit, just asked a clanker to rationalize his "clankers are great" conclusion.
Because he didn't analyze shit, just asked a clanker to rationalize his "clankers are great" conclusion.
> The analysis uses a single metric: bugs per 10 commits (bugs/10c).
Bugs per commit as a metric papers over severity, both in terms of security severity as well as the effect on the user. A mislabeled button has the same weight as the entire app crashing in this framework.
Bugs per commit as a metric papers over severity, both in terms of security severity as well as the effect on the user. A mislabeled button has the same weight as the entire app crashing in this framework.
IMO "bugs per commit" is even worse than that, because, in addition to what you say, it also hides the extraordinary spike of commit activity of a project that had previously been stable. [0]
It is the exact metric you'd choose if you wanted to make the current situation of rsync look like not a big deal.
[0] https://github.com/RsyncProject/rsync/graphs/commit-activity
It is the exact metric you'd choose if you wanted to make the current situation of rsync look like not a big deal.
[0] https://github.com/RsyncProject/rsync/graphs/commit-activity
Yes, but we know why there was an "extraordinary spike," and it has nothing to do with rsync being "vibe coded." The maintained has directly addressed this.
> The maintained has directly addressed this.
Not sure if this is mentioned somewhere else, but looks like the maintainer has a blog post that addresses this: https://medium.com/@tridge60/rsync-and-outrage-d9849599e5a0
Not sure if this is mentioned somewhere else, but looks like the maintainer has a blog post that addresses this: https://medium.com/@tridge60/rsync-and-outrage-d9849599e5a0
Seems like this would be a good place to link to that.
I link to it multiple times in TFA and quote the specific thing I'm talking about here in there to explain that possible confounder. I think I've done more than the work I'm obligated to it.do to make all of the relevant information available to you. You are just refusing to use
I am not finding these links in TFA, I see a link to an issue #929 which (as mentioned in TFA) has over 350 replies, and and opinionated summary of what transpired, including some detailed description of specific posts there. However I did not find the maintainers response.
Of interest is this post here: https://github.com/RsyncProject/rsync/issues/929#issuecommen... which echos the same concern which was raised up thread, however, I failed to find the maintainers’ response.
EDIT: Found it! it is in the (untitled) discussion section (after the results).
https://lobste.rs/s/k1b0za/rsync_outrage#c_2iowov
EDIT 2 (and advice on design): The page design changes backgrounds after the results sections, which kind of conveys to the user that they have reached the end of what was is important and can just skim over the rest (usually pages have a radical change in typography like these when you’ve reached the comment section), however this is what is analogous to a discussion in a typical paper, and is arguably the most important part. I had simply assumed that you just left it at the result and skipped the discussion as a stylistic choice.
Of interest is this post here: https://github.com/RsyncProject/rsync/issues/929#issuecommen... which echos the same concern which was raised up thread, however, I failed to find the maintainers’ response.
EDIT: Found it! it is in the (untitled) discussion section (after the results).
https://lobste.rs/s/k1b0za/rsync_outrage#c_2iowov
EDIT 2 (and advice on design): The page design changes backgrounds after the results sections, which kind of conveys to the user that they have reached the end of what was is important and can just skim over the rest (usually pages have a radical change in typography like these when you’ve reached the comment section), however this is what is analogous to a discussion in a typical paper, and is arguably the most important part. I had simply assumed that you just left it at the result and skipped the discussion as a stylistic choice.
> EDIT: Found it! it is in the (untitled) discussion section (after the results).
I also paraphrase Tridge himself explicitly saying that this is why commits/releases have increased:
> Essentially, this isn't a "Claude" problem, it's a "more security work" problem, something that Tridge himself confirmed in his response, describing how a flood of AI-generated CVE reports forced rapid, extensive changes to rsync's attack surface.
> The page design changes backgrounds after the results sections, which kind of conveys to the user that they have reached the end of what was is important and can just skim over the rest (usually pages have a radical change in typography like these when you’ve reached the comment section), however this is what is analogous to a discussion in a typical paper, and is arguably the most important part. I had simply assumed that you just left it at the result and skipped the discussion as a stylistic choice.
Good point, I assumed everyone would read till the end, that's on me. I'll give it a heading.
I also paraphrase Tridge himself explicitly saying that this is why commits/releases have increased:
> Essentially, this isn't a "Claude" problem, it's a "more security work" problem, something that Tridge himself confirmed in his response, describing how a flood of AI-generated CVE reports forced rapid, extensive changes to rsync's attack surface.
> The page design changes backgrounds after the results sections, which kind of conveys to the user that they have reached the end of what was is important and can just skim over the rest (usually pages have a radical change in typography like these when you’ve reached the comment section), however this is what is analogous to a discussion in a typical paper, and is arguably the most important part. I had simply assumed that you just left it at the result and skipped the discussion as a stylistic choice.
Good point, I assumed everyone would read till the end, that's on me. I'll give it a heading.
Why don't you prove the bugs increased then?
Why is it that some unfounded claim is made and the onus is suddenly on the project maintainer to prove it beyond all doubt?
It should be on the person making the claim to prove it
Why is it that some unfounded claim is made and the onus is suddenly on the project maintainer to prove it beyond all doubt?
It should be on the person making the claim to prove it
I've now resolved this. The new version, which should be live on GH Pages soon, uses — what I think is — a pretty good methodology for assigning severity to each bug, normalizes it to 0.0-1.0, sums that, and treats that as the total severity weighted bugs, then does the analysis based on that. It did not change the analysis in any material way.
No Claude, it still makes zero sense as a metric.
A commit is a measure of nothing. Severity weighted bugs per unit of nothing? What does that even mean? In any repo it's trivial to achieve a sev/10c that's arbitrarily close to zero while completely ruining everything.
I suggest you practice some humility and update your conclusion instead of updating the mental gymnastics you used to arrive at the same conclusion.
A commit is a measure of nothing. Severity weighted bugs per unit of nothing? What does that even mean? In any repo it's trivial to achieve a sev/10c that's arbitrarily close to zero while completely ruining everything.
I suggest you practice some humility and update your conclusion instead of updating the mental gymnastics you used to arrive at the same conclusion.
Whether commits decrease the sev/10c depends on if there are a lot of small commits increasing the demoninator. In reality, we have the opposite: the post-Claude releases have way fewer commits than the pre-Claude ones.
Thus, if anything their sev/10c is inflated. If I changed it to lines of code changed, the relative bug ratios would be much smaller, and the conclusion wouldn't change. In fact, the conclusion would look "better" for Claude; if I was using "mental gymnastics" to come to this conclusion, I would have already used a metric other than adjusting per commits!
What different metric would you suggest that would change the conclusion?
Showing "humility", as you so moralistically and condescendingly put it, would require being wrong first.
Thus, if anything their sev/10c is inflated. If I changed it to lines of code changed, the relative bug ratios would be much smaller, and the conclusion wouldn't change. In fact, the conclusion would look "better" for Claude; if I was using "mental gymnastics" to come to this conclusion, I would have already used a metric other than adjusting per commits!
What different metric would you suggest that would change the conclusion?
Showing "humility", as you so moralistically and condescendingly put it, would require being wrong first.
> In reality, we have the opposite: the post-Claude releases have way fewer commits than the pre-Claude ones.
No, they don't, you just made that up.
> What different metric would you suggest that would change the conclusion?
What would be a lot more useful to know is whether or not the original prompt used to generate this post instructed you to do a fair and unbiased review of these bugs, and whether or not that prompt itself was framed in a fair and unbiased way. If you take a piece of paper and write "therefore Claude is not at fault" at the bottom, then nothing you write above that line is admissible, no matter how well-reasoned.
No, they don't, you just made that up.
> What different metric would you suggest that would change the conclusion?
What would be a lot more useful to know is whether or not the original prompt used to generate this post instructed you to do a fair and unbiased review of these bugs, and whether or not that prompt itself was framed in a fair and unbiased way. If you take a piece of paper and write "therefore Claude is not at fault" at the bottom, then nothing you write above that line is admissible, no matter how well-reasoned.
There was no analysis of severity in all of the rage posting that occurred. The single point being pushed was "use of an LLM led/leads to more bugs". The author specifically states that's what they're addressing (blunt accusation -> blunt response).
The specific problems mentioned were all reasonably severe. The original post itself described a show-stopping bug:
The second reply is similar:
Later in the comments, someone attempted to provide a list of issues that had been added: https://github.com/RsyncProject/rsync/issues/929#issuecommen.... The list included several failures to build or run rsync that appear to have resulted from broken backward compatibility. That seems reasonably severe. If intentional, I would have expected mention in the release notes about the removal of backwards compatibility, but none was made.
The issue comments already degraded into a lot of unnecessary vitriol even before the above mentioned comment and only gets worse from there, so I stopped. But, the fact remains that the whole issue started with a severe bug.
I applaud the attempt at dispassionately analyzing whether the recent LLM releases of rsync were normal or outliers as far as bugs are concerned, but I don't think you can do so properly without analyzing severity.
So my systems recently updated to rsync 3.4.3, and as soon as that happened my backup system - which does incremental backups using multiple --compare-dest= arguments - started to fail on anything but a full backup.
The second reply is similar:
i wondered why my 3d printers were running like sh*t and at 100% cpu; turns out log2ram uses rsync.
Later in the comments, someone attempted to provide a list of issues that had been added: https://github.com/RsyncProject/rsync/issues/929#issuecommen.... The list included several failures to build or run rsync that appear to have resulted from broken backward compatibility. That seems reasonably severe. If intentional, I would have expected mention in the release notes about the removal of backwards compatibility, but none was made.
The issue comments already degraded into a lot of unnecessary vitriol even before the above mentioned comment and only gets worse from there, so I stopped. But, the fact remains that the whole issue started with a severe bug.
I applaud the attempt at dispassionately analyzing whether the recent LLM releases of rsync were normal or outliers as far as bugs are concerned, but I don't think you can do so properly without analyzing severity.
To keep such an analysis fair and contextually relevant, it would have to be extended to the previous 928 issues as well (of course filtering for bug reports). I don't see anyone doing such an analysis, I think because they don't expect they'd find it useful (at least not as the rage fuel that many are seeking); what they'd be more likely to find is that there is a similar severity-mix going all the way back to v1.0.0, because these things inevitably happen whether coding is done by human or machine.
"A lot of claims in the wider discussion have treated every recent bug report as if it had the same cause. That is not accurate. Some reports were regressions from recent security hardening, some were missing historical test coverage, some were older bugs found because rsync suddenly had more eyes on it (especially by AI that can find issues quickly) and some were packaging or environment-specific failures. A Co-authored-by line is not enough by itself to establish root cause." - https://github.com/RsyncProject/rsync/issues/929#issuecommen...
"A lot of claims in the wider discussion have treated every recent bug report as if it had the same cause. That is not accurate. Some reports were regressions from recent security hardening, some were missing historical test coverage, some were older bugs found because rsync suddenly had more eyes on it (especially by AI that can find issues quickly) and some were packaging or environment-specific failures. A Co-authored-by line is not enough by itself to establish root cause." - https://github.com/RsyncProject/rsync/issues/929#issuecommen...
[deleted]
While I'm grateful for all Andrew has done to create and maintain rsync, I rely heavily on it for backing up files between machines on my home network, so I've spent the time to figure out how to pin the Homebrew version of rsync to 3.4.1 because the bugs in the subsequent two versions really scare me (as does the original report that triggered all this).
Here is the process I used to do it, which was way more complex than I thought it would be:
https://gist.github.com/e40/caa67c1b8d439a528695f996d0519d8e
Here is the process I used to do it, which was way more complex than I thought it would be:
https://gist.github.com/e40/caa67c1b8d439a528695f996d0519d8e
I've been coding for over 2 decades. I love it, I've always loved it and I likely always will.
I was an AI skeptic some months ago but truly Claude and Codex have changed my development style and velocity in a way I never imagined would ever be possible. With that, yes, I produce more code and am finding more bugs.
So looking over at comments in HN articles the amount of polarising hate to anything produced with AI is quite surprising. Just because some AI helped or even produced entirely doesn't suddenly make a project 'vibe coded' as if that's meant to be some insult levelled at users of LLMs.
It reminds me a lot of when offshore outsources started getting more software development work from the mid-90s with all the derogatory remarks made towards 'Indian developers'. Now we're in the mid 2020s and similar remarks are made towards AI.
I don't get it. I really don't. What I do know for sure is more and more code will be AI generated with or without the detractors.
I was an AI skeptic some months ago but truly Claude and Codex have changed my development style and velocity in a way I never imagined would ever be possible. With that, yes, I produce more code and am finding more bugs.
So looking over at comments in HN articles the amount of polarising hate to anything produced with AI is quite surprising. Just because some AI helped or even produced entirely doesn't suddenly make a project 'vibe coded' as if that's meant to be some insult levelled at users of LLMs.
It reminds me a lot of when offshore outsources started getting more software development work from the mid-90s with all the derogatory remarks made towards 'Indian developers'. Now we're in the mid 2020s and similar remarks are made towards AI.
I don't get it. I really don't. What I do know for sure is more and more code will be AI generated with or without the detractors.
I work with outsourced code all the time and it is a tyre fire without exception. I just spent a week scrubbing a codebase where some dev “did the needful” and committed an on-by-default flag to bypass authentication checks because he didn’t known how to set up his local work environment.
People report the same “took a shortcut” issue with AI vibe coding, and I can confirm that I’ve had to rewrite practically everything the AI generated for me, despite using a frontier model dialed up to 11 thinking levels.
Having said that, AI is very useful for other activities like PR review, security vulnerability analysis, typo hunting, reverse engineering, etc.
I’m probably going to have to increase my subscription to the next tier but at the same time I still can’t use any of the code it generates.
If even one person can simultaneously experience "very useful, need to pay more for it" and "useless output code quality" then of course you'd expect a variety of opinions amongst the general user base.
People report the same “took a shortcut” issue with AI vibe coding, and I can confirm that I’ve had to rewrite practically everything the AI generated for me, despite using a frontier model dialed up to 11 thinking levels.
Having said that, AI is very useful for other activities like PR review, security vulnerability analysis, typo hunting, reverse engineering, etc.
I’m probably going to have to increase my subscription to the next tier but at the same time I still can’t use any of the code it generates.
If even one person can simultaneously experience "very useful, need to pay more for it" and "useless output code quality" then of course you'd expect a variety of opinions amongst the general user base.
> I work with outsourced code all the time and it is a tyre fire without exception. I just spent a week scrubbing a codebase where some dev “did the needful” and committed an on-by-default flag to bypass authentication checks because he didn’t known how to set up his local work environment.
OP knows this but finds himself in the strange position of having to defend India slop in order to defend AI slop, totally unnecessarily and unprompted. It's baffling to you and me.
OP knows this but finds himself in the strange position of having to defend India slop in order to defend AI slop, totally unnecessarily and unprompted. It's baffling to you and me.
I was similarly an AI skeptic 3 years ago. When GPT-4 was the state of the art, I thought we're going to plateau soon because of context size limits (remember back when you had to pay insane money just to get 32K)?
Last year was the first time I saw an AI agent actually debug and fix a non-trivial bug in a satisfactory way. Even then, trying to use it on larger tasks made it clear that it wasn't something I could just hand over the issue tracker to.
Now? I've been using Codex for the past several months to work on a nontrivial project. Which was prototyped in C++ (for library reasons mostly), then had the initial version written in Haskell, and more recently I got it ported to Rust to keep memory use in check on mobile.
These things are not trouble-free, but the sheer amount of progress made in just the last year alone is astounding. Skepticism is well and good, but healthy skepticism ought to yield to tangible evidence.
Last year was the first time I saw an AI agent actually debug and fix a non-trivial bug in a satisfactory way. Even then, trying to use it on larger tasks made it clear that it wasn't something I could just hand over the issue tracker to.
Now? I've been using Codex for the past several months to work on a nontrivial project. Which was prototyped in C++ (for library reasons mostly), then had the initial version written in Haskell, and more recently I got it ported to Rust to keep memory use in check on mobile.
These things are not trouble-free, but the sheer amount of progress made in just the last year alone is astounding. Skepticism is well and good, but healthy skepticism ought to yield to tangible evidence.
Good code is a living document that shows intent, and allows ease of maintainability.
Most people feel more productive with chat bots, but often end up wasting more time chasing self-inflicted issues. Same clown-car of Dev-ops proponents no doubt billing by the hour. =3
Most people feel more productive with chat bots, but often end up wasting more time chasing self-inflicted issues. Same clown-car of Dev-ops proponents no doubt billing by the hour. =3
I've always noticed, within any subject involving tools, there are people who like the tools, and some people who like to use the tools to do something else.
With programming, I've always been in the later: it's a tool that allows me to do what I actually love, which is problem solving, system level thinking, and providing some nice solution to that problem, that happens to be through software.
So, I have an absolute blast with AI, because it helps do the more boring bits. And, seeing my non-programming colleagues get excited to see their vibe coded ideas become reality has been so much fun.
I'm genuinely curious to hear the perspective of someone anti-AI, who works in software. Perhaps the impending doom/skill shift of our profession?
With programming, I've always been in the later: it's a tool that allows me to do what I actually love, which is problem solving, system level thinking, and providing some nice solution to that problem, that happens to be through software.
So, I have an absolute blast with AI, because it helps do the more boring bits. And, seeing my non-programming colleagues get excited to see their vibe coded ideas become reality has been so much fun.
I'm genuinely curious to hear the perspective of someone anti-AI, who works in software. Perhaps the impending doom/skill shift of our profession?
I'm not anti-AI but something I've been thinking about is the discipline it requires.
As you said, it's a tool that allows you to rename a variable name on one end and do complete vibe coding on the other end. Developers may say that we should stay somewhere left on that spectrum, because that's where human's are more involved.
But developers also say good practices should be followed when talking to each other, and while some may do, reality is often very different.
It requires discipline, which varies a lot between developers, between projects, current mood, and so on.
In the beginning you might be careful doing small changes, but after a while you might get more tempted to accept the output for what it is, because ultimately that's much easier.
So the way I see it; the left side is harder work and potentially bigger but delayed dopamine hits, the right side is quick dopamine hits. How do we (at least those who struggle with discipline) resist just slipping to the right?
I started out carefully myself and slipped more into vibe coding, but I don't feel particularly proud of it for some reason.
But developers also say good practices should be followed when talking to each other, and while some may do, reality is often very different.
It requires discipline, which varies a lot between developers, between projects, current mood, and so on.
In the beginning you might be careful doing small changes, but after a while you might get more tempted to accept the output for what it is, because ultimately that's much easier.
So the way I see it; the left side is harder work and potentially bigger but delayed dopamine hits, the right side is quick dopamine hits. How do we (at least those who struggle with discipline) resist just slipping to the right?
I started out carefully myself and slipped more into vibe coding, but I don't feel particularly proud of it for some reason.
> It requires discipline, which varies a lot between developers, between projects, current mood, and so on.
In the beginning you might be careful doing small changes, but after a while you might get more tempted to accept the output for what it is, because ultimately that's much easier.
Counterpoint: how is this any different from how things were pre-LLMs? I have seen, in the same codebase, some throughly well-written and tested PRs that read like Shakespeare and some of the laziest slop that even no LLM would ever write because humans have an unlimited capacity for laziness.
You catch the bad stuff through oversight, process, automated and manual checks, and the ultimate threat that your job depends on your ability to deliver so you better allocate at least enough energy into this so that you can ship moderately working code.
In the beginning you might be careful doing small changes, but after a while you might get more tempted to accept the output for what it is, because ultimately that's much easier.
Counterpoint: how is this any different from how things were pre-LLMs? I have seen, in the same codebase, some throughly well-written and tested PRs that read like Shakespeare and some of the laziest slop that even no LLM would ever write because humans have an unlimited capacity for laziness.
You catch the bad stuff through oversight, process, automated and manual checks, and the ultimate threat that your job depends on your ability to deliver so you better allocate at least enough energy into this so that you can ship moderately working code.
I am anti-vibe coding if that meets your criteria?
Reviewing vibe-coded PRs and features has been utterly exhausting over the past few months.
I work on critical, mature software - a small change in behaviour can mean data loss or non-compliance with regulations for our customers. The biggest problem with AI PRs is the sheer amount of churn, extra code and lack of intent with the PRs it generates.
The only way I can describe the latter is that an AI-only PR feels to me like a painting where everything is high detail - and you have to comb over each part before you understand why it's there because so much is superfluous. A well written human PR on the other hand, is painted such that your eye naturally follows the thought process of the author so you can just nod along during the review, as if the solution was obvious.
Also when I'm _using_ the agent; at least 50 percent of my time is spent telling it to stop with it's approach so it doesn't go down a useless rabbit hole and waste tokens.
Reviewing vibe-coded PRs and features has been utterly exhausting over the past few months.
I work on critical, mature software - a small change in behaviour can mean data loss or non-compliance with regulations for our customers. The biggest problem with AI PRs is the sheer amount of churn, extra code and lack of intent with the PRs it generates.
The only way I can describe the latter is that an AI-only PR feels to me like a painting where everything is high detail - and you have to comb over each part before you understand why it's there because so much is superfluous. A well written human PR on the other hand, is painted such that your eye naturally follows the thought process of the author so you can just nod along during the review, as if the solution was obvious.
Also when I'm _using_ the agent; at least 50 percent of my time is spent telling it to stop with it's approach so it doesn't go down a useless rabbit hole and waste tokens.
> The biggest problem with AI PRs is the sheer amount of churn, extra code and lack of intent with the PRs it generates.
But this isn't an LLM problem; this is a problem of undisciplined engineers who feel they need to cram extra stuff in a PR. If an engineer doesn't look at the output of the LLM and generate extra work then it's still on them, right?
> The only way I can describe the latter is that an AI-only PR feels to me like a painting where everything is high detail - and you have to comb over each part before you understand why it's there because so much is superfluous
This just indicates that the engineer doesn't know how to use the tool. Hell they can ask the LLM to split the work into focused PRs and Claude will be happy to do it and the results might no even be half bad.
> Also when I'm _using_ the agent; at least 50 percent of my time is spent telling it to stop with it's approach so it doesn't go down a useless rabbit hole and waste tokens.
If this is happening often then the tool is probably not fit for the job.
But this isn't an LLM problem; this is a problem of undisciplined engineers who feel they need to cram extra stuff in a PR. If an engineer doesn't look at the output of the LLM and generate extra work then it's still on them, right?
> The only way I can describe the latter is that an AI-only PR feels to me like a painting where everything is high detail - and you have to comb over each part before you understand why it's there because so much is superfluous
This just indicates that the engineer doesn't know how to use the tool. Hell they can ask the LLM to split the work into focused PRs and Claude will be happy to do it and the results might no even be half bad.
> Also when I'm _using_ the agent; at least 50 percent of my time is spent telling it to stop with it's approach so it doesn't go down a useless rabbit hole and waste tokens.
If this is happening often then the tool is probably not fit for the job.
I don't believe so - it's not as though the original prompts asked for extra code churn (note that as soon as you look and edit the LLM code output extensively it ceases to be vibe coding, which I was talking about in my OP).
I'm not talking about extra feature s; I'm talking about for the same single feature the code is either convoulted because the algorithm is overly complicated or the abstractions are just wrong for the domain.
The PRs typically are already focused in that they address a single feature; or at least a single "usable" feature in a complex system which necessarily has a lot of connected parts and behaviors.
> then the tool is probably not fit for the job.
Perhaps; but with an LLM I haven't found which jobs it _does_ work for and which it doesn't. I already use planning mode extensively; and capture the major points, but then it makes a stupid decision mid implementation and just starts churning.
I'm not talking about extra feature s; I'm talking about for the same single feature the code is either convoulted because the algorithm is overly complicated or the abstractions are just wrong for the domain.
The PRs typically are already focused in that they address a single feature; or at least a single "usable" feature in a complex system which necessarily has a lot of connected parts and behaviors.
> then the tool is probably not fit for the job.
Perhaps; but with an LLM I haven't found which jobs it _does_ work for and which it doesn't. I already use planning mode extensively; and capture the major points, but then it makes a stupid decision mid implementation and just starts churning.
I started similarly with it. I'm of the opinion that its a tool that behaves like a tool - how well it works depends on who is using it and how.
I don't have a good analogy but the immediate one that comes to mind is treating AI like a junior developer that you're mentoring. If you know what you're doing you can iterate quickly; if you don't then its a whole other story.
Claude built me a Markdown editor - I designed it, set coding standards, etc. It coded it to my spec. The output is in my opinion not bad and is very usable (for me - I use it daily now). Probably would have cost me north of $50k to get a team of seasoned devs to build it to the current level of polish. https://github.com/emrul/md
I don't have a good analogy but the immediate one that comes to mind is treating AI like a junior developer that you're mentoring. If you know what you're doing you can iterate quickly; if you don't then its a whole other story.
Claude built me a Markdown editor - I designed it, set coding standards, etc. It coded it to my spec. The output is in my opinion not bad and is very usable (for me - I use it daily now). Probably would have cost me north of $50k to get a team of seasoned devs to build it to the current level of polish. https://github.com/emrul/md
> So, I have an absolute blast with AI, because it helps do the more boring bits.
So... you're vibing? Not looking at the code at all?
So... you're vibing? Not looking at the code at all?
Personally, it would still bother me if some lazy bro hit a code-generator and people end up dead.
For context search, I find LLM quite useful... still wrong 20% of the time... but it has some utility.
Here is a thought experiment: If "AI" will eventually generate your work, than what actual value do you bring to the table? =3
For context search, I find LLM quite useful... still wrong 20% of the time... but it has some utility.
Here is a thought experiment: If "AI" will eventually generate your work, than what actual value do you bring to the table? =3
I just really hate talking to the computer in human language.
> It reminds me a lot of when offshore outsources started getting more software development work from the mid-90s with all the derogatory remarks made towards 'Indian developers'.
What was the impetus of the derogatory remarks?
What was the impetus of the derogatory remarks?
Some of it was indeed driven by sub-par work from the outsourcing firms, as the style of work was new and people on both sides hadn't developed the right skill set and processes to do the work well.
Some of it was genuine cultural differences. It's hard to work with people and get the results you want when you don't understand their culture, and how they communicate. (For example, people from some cultures just can't say "no" or "I don't know"; you need to learn how to communicate with them in a different way to get the understanding you need.)
Some of it was certainly a form of jingoistic or xenophobic protectionism.
Some of it was genuine cultural differences. It's hard to work with people and get the results you want when you don't understand their culture, and how they communicate. (For example, people from some cultures just can't say "no" or "I don't know"; you need to learn how to communicate with them in a different way to get the understanding you need.)
Some of it was certainly a form of jingoistic or xenophobic protectionism.
LLM are good for context search, and template output.
However, you also get the lowest common salient answer guaranteed, uncopyrightable work (differs from public domain), and potential legal peril from copyright bleed-through.
We are in the golden Napster age of isomorphic plagiarism. =3
However, you also get the lowest common salient answer guaranteed, uncopyrightable work (differs from public domain), and potential legal peril from copyright bleed-through.
We are in the golden Napster age of isomorphic plagiarism. =3
Okay, I really have to point out to everyone: the numbers and report cards are TEMPLATED IN BY A SCRIPT. Hallucinations are a moot point. https://github.com/alexispurslane/rsync-analysis/blob/main/s...
> But the critics' accusation is also blunt: "Claude is making things worse." A blunt instrument is the fairest response.
So the criticism was bad, and that somehow makes it ok to use a bad metric?
So the criticism was bad, and that somehow makes it ok to use a bad metric?
That's not what I'm saying. What I'm saying is that if the criticism is referring to a broad set of metrics like bugs per release and number of commits that were made by Claude, then it's correct to look at precisely those things because that's what the claim is about.
[deleted]
AI + Interest != Expertise
I come to hn because I get very nuanced, informed information and glorious puns.
I come to hn because I get very nuanced, informed information and glorious puns.
What would be a better one?
Claude in general probably increases observed bugs in rsync, because it can churn out vulnerability reports that necessitate tons of changes to software that people are accustomed to working flawlessly in non-pathological use cases.
I don't have empirical evidence for this claim, but best I can tell, security patches are the principal source of observed bugs in software of a certain vintage, because they cause churn. (Just think of Windows updates that break drivers.)
I don't have empirical evidence for this claim, but best I can tell, security patches are the principal source of observed bugs in software of a certain vintage, because they cause churn. (Just think of Windows updates that break drivers.)
I agree that the bug report is not very good. I also agree that having commits written by Claude is not necessarily what caused the bug, although it might be (it is also possible that some of them introduced bugs and others didn't); whether or not it is in this case, is I don't know (some people think it is, but some think not). (Software without code written by generative AI will still have bugs too.)
However, the claim that "the original post was [...] no bug report" seems wrong; it does have a bug report, although not a very good one. It says that incremental backups using multiple --compare-dest arguments do not work, so it is a bug report. But, it should have been written differently, including by putting the text directly instead of a screenshot, giving a proper title, better details about the bug being reported, etc.
Their claims that they introduced deliberate bugs, are unlikely to be accurate, and not worth making those claims nor the violence that they involve.
I do have reasons for not wanting LLMs to commit code, so I agree with their opinion about that, but that does not justify making a bad bug report and the other stuff that they did. If it is FOSS, someone who disagrees with the project can fork it and make their own version, as has been done with other FOSS projects as well.
I think it is good that they are making statistical analysis. However, they used a language model to classify bug reports. They mentioned some things that might be missed, and they could be missed whether or not you are using a language model to classify bug reports, although there are some other possibilities e.g. whether or not a single report should count as multiple bugs in some cases, and mistakes in marking reports as duplicate.
However, the claim that "the original post was [...] no bug report" seems wrong; it does have a bug report, although not a very good one. It says that incremental backups using multiple --compare-dest arguments do not work, so it is a bug report. But, it should have been written differently, including by putting the text directly instead of a screenshot, giving a proper title, better details about the bug being reported, etc.
Their claims that they introduced deliberate bugs, are unlikely to be accurate, and not worth making those claims nor the violence that they involve.
I do have reasons for not wanting LLMs to commit code, so I agree with their opinion about that, but that does not justify making a bad bug report and the other stuff that they did. If it is FOSS, someone who disagrees with the project can fork it and make their own version, as has been done with other FOSS projects as well.
I think it is good that they are making statistical analysis. However, they used a language model to classify bug reports. They mentioned some things that might be missed, and they could be missed whether or not you are using a language model to classify bug reports, although there are some other possibilities e.g. whether or not a single report should count as multiple bugs in some cases, and mistakes in marking reports as duplicate.
Wait, how is any of this relevant if there were only 2 Claude commits? My statistics courses are far behind me, but don't you need at least 30 data points to conclude anything?
Depends on the methods you use. If you're trying to fit curves and so on, yes. The methods I use were designed for very low amounts of data, and are generally okay for that, specifically and especially when you're just trying to show a lack of evidence for some non-null hypothesis.
And again, that's kind of the point. There's exactly zero actual evidence, however you slice it, that "Claude broke rsync" except cherry-picked anecdata, and the whole point of my analysis is to demonstrate the total lack of any such trend/evidence at all, and just how in-distribution/normal these releases are, to show that if people hadn't known Claude was involved in them, they wouldn't have remarked on them.
And again, that's kind of the point. There's exactly zero actual evidence, however you slice it, that "Claude broke rsync" except cherry-picked anecdata, and the whole point of my analysis is to demonstrate the total lack of any such trend/evidence at all, and just how in-distribution/normal these releases are, to show that if people hadn't known Claude was involved in them, they wouldn't have remarked on them.
It wasn't 2 Claude commits. It's 2 releases where the (many) commits were largely co-authored by Claude.
> My statistics courses are far behind me, but don't you need at least 30 data points to conclude anything?
That cuts both ways. If we say that the author here can't claim any conclusion because there are only 2 Claude-authored releases, then we must also say that the people claiming "Claude broke rsync" have no statistical basis to draw that conclusion, either.
> My statistics courses are far behind me, but don't you need at least 30 data points to conclude anything?
That cuts both ways. If we say that the author here can't claim any conclusion because there are only 2 Claude-authored releases, then we must also say that the people claiming "Claude broke rsync" have no statistical basis to draw that conclusion, either.
> My statistics courses are far behind me, but don't you need at least 30 data points to conclude anything?
There is no fixed number. Sample size depends on the size of the set you're sampling, desired margin of error and confidence interval.
If your total set has a million items, you need ~16600 samples to draw conclusions with 99% ±1% certainty.
There is no fixed number. Sample size depends on the size of the set you're sampling, desired margin of error and confidence interval.
If your total set has a million items, you need ~16600 samples to draw conclusions with 99% ±1% certainty.
It's not uncommon to have small amounts of data come out of experiments. These are appropriate tests for the size of the data. These tests failed to disprove the null hypothesis.
This is a really cool post but I think one metric we may want to also look at is does using agentic coding tools in one domain impact your coding abilities in another domain? A lot of people I know have been talking about getting rusty on the fundamentals recently. This is not something I am particularly feeling as I do a mix of running agents in parallel and writing some code manually where it makes sense. But if people who have been prompt-only at work come home and work on rsync and are more "rusty" maybe that could also lead to more bugs?
This would be even harder to measure.
This would be even harder to measure.
It's always the most insufferable people that make the biggest hullabaloo about a project they have nothing to do with and have never contributed to. People with literally zero skin in the game using the AI boogeyman to push some agenda or some anti-agenda. OSS has become so incredibly toxic in the past decade, and consumers of OSS have become extremely entitled.
I run a smallish project with ~1k stars and I've stopped maintaining it last year because people feel like they're absolutely owed features or bug-fixes or whatever. It's tiring and a complete shame that author has to make such an insane deep dive into a random accusation that just caught on social media. I want to emphasize that this has nothing to do with AI, it's just tech tourists, consumers (as opposed to creators), and engagement farmers that have taken over. AI slop probably doesn't help, but the underlying issue has been brewing for at least a decade.
Also, the "making soup for the homeless & pissing in it" is not only an off-base analogy (software is pretty low on Maslow’s Hierarchy of Needs), but also somehow looks down on both people in need and the volunteers that help them. Just absolutely gross.
I run a smallish project with ~1k stars and I've stopped maintaining it last year because people feel like they're absolutely owed features or bug-fixes or whatever. It's tiring and a complete shame that author has to make such an insane deep dive into a random accusation that just caught on social media. I want to emphasize that this has nothing to do with AI, it's just tech tourists, consumers (as opposed to creators), and engagement farmers that have taken over. AI slop probably doesn't help, but the underlying issue has been brewing for at least a decade.
Also, the "making soup for the homeless & pissing in it" is not only an off-base analogy (software is pretty low on Maslow’s Hierarchy of Needs), but also somehow looks down on both people in need and the volunteers that help them. Just absolutely gross.
Absolutely agree. Quite a lot of judgement from people who benefited from this guy's software for over 20 years, probably without ever helping him pay his bills even once.
> It's always the most insufferable people that make the biggest hullabaloo about a project they have nothing to do with and have never contributed to.
Agreed, and similarly, as a hobbyist programmer who loves Rust and Go, I've always felt that the people who command others to "rewrite it in xyz" are not themselves developers, they're "ideas people." There's a mass of these people whose main interactions with the world are through the dramatic forcing of their correct opinions.
> I run a smallish project with ~1k stars and I've stopped maintaining it last year because people feel like they're absolutely owed features or bug-fixes or whatever.
That's a bummer and it's something I'm fearful of. I post some code on my website, not on a github type site, and don't interact with people about it. It's nice and plenty of people do it. Is that something you'd consider?
Agreed, and similarly, as a hobbyist programmer who loves Rust and Go, I've always felt that the people who command others to "rewrite it in xyz" are not themselves developers, they're "ideas people." There's a mass of these people whose main interactions with the world are through the dramatic forcing of their correct opinions.
> I run a smallish project with ~1k stars and I've stopped maintaining it last year because people feel like they're absolutely owed features or bug-fixes or whatever.
That's a bummer and it's something I'm fearful of. I post some code on my website, not on a github type site, and don't interact with people about it. It's nice and plenty of people do it. Is that something you'd consider?
haha, that analogy says more about whoever wrote it than it ever could to get the intended point across!
The discussions around this have devolved to excrement anyway, I feel tempted to invoke the meme where the goose asking a guy what his jacket is made of, asks “where is your reproducer case!?” instead.
Instead we have a shitstorm over presumably legit issue, for which the only source is some mastodon post.
One command that used to work in 3.4.1 and stopped working in 3.4.3. Just one! We could have already bisected the living shit out of this and go home, but no.
Instead we have a shitstorm over presumably legit issue, for which the only source is some mastodon post.
One command that used to work in 3.4.1 and stopped working in 3.4.3. Just one! We could have already bisected the living shit out of this and go home, but no.
> "Claude clearly made things worse" &emdash; the main claim
Even this report is full of claude-introduced bugs
Even this report is full of claude-introduced bugs
For those that don’t know the html entity is — not &emdash; although I think in modern codebases people usually just type — directly.
This mistake does exist in the wild though: https://github.com/search?q=%26emdash%3B&type=code
If I was more ambitious I would plot the dates of the blames of these results in a histogram and see if an there is a significant increase in these mistakes (over a baseline —) correlating with the release of some models.
This mistake does exist in the wild though: https://github.com/search?q=%26emdash%3B&type=code
If I was more ambitious I would plot the dates of the blames of these results in a histogram and see if an there is a significant increase in these mistakes (over a baseline —) correlating with the release of some models.
That one was on me. I always mess that up.
I think this writer kinda took the bait which is fine someone had to do this so we couldn't debate endlessly.
But the reality is that if you were already set enough to call rsync slop because of a single post, you aren't going to be more down now. Even in these responses I see everyone nitpicking and moving goalposts as if one more commit being actually claude-aided will tip the scales from stable project to "vibe coded slop".
Software has always been fuzzy, we have never come up with an objective way to handle software quality, and this Uber hatred of llm contributions lets the humans who make egregious bugs and mistakes off the hook.
Taking a step back, we need to have more empathy and thoughtfulness of one another in this space. Its new and people are experimenting and there will be nothing good coming from personal insults and DDOsing a good project just because someone got ragebaited on threads, x, mastodon or whatever else.
How do we determine bugs and increase quality? Its almost like we have been grappling with this question for decades and I still hear people fight on the best way forward. Simple design, test driven development, user surveys, all of the above have been used as a proxy for software and they all failed to capture everything. Back in the day we used that ambiguity to give each other grace, now we use that ambiguity to tear down other creators. Whatever, if open source software really is dying its because of this toxic shit just as much as the llms
But the reality is that if you were already set enough to call rsync slop because of a single post, you aren't going to be more down now. Even in these responses I see everyone nitpicking and moving goalposts as if one more commit being actually claude-aided will tip the scales from stable project to "vibe coded slop".
Software has always been fuzzy, we have never come up with an objective way to handle software quality, and this Uber hatred of llm contributions lets the humans who make egregious bugs and mistakes off the hook.
Taking a step back, we need to have more empathy and thoughtfulness of one another in this space. Its new and people are experimenting and there will be nothing good coming from personal insults and DDOsing a good project just because someone got ragebaited on threads, x, mastodon or whatever else.
How do we determine bugs and increase quality? Its almost like we have been grappling with this question for decades and I still hear people fight on the best way forward. Simple design, test driven development, user surveys, all of the above have been used as a proxy for software and they all failed to capture everything. Back in the day we used that ambiguity to give each other grace, now we use that ambiguity to tear down other creators. Whatever, if open source software really is dying its because of this toxic shit just as much as the llms
'this toxic shit' would not be occurring if we didn't invent a machine that can be used either as a firehose or a scalpel. I do acknowledge that behaving hurtfully towards somebody giving something away for free is unwarranted behaviour. perhaps a universally agreed quality control method does not exist - this does not suggest that ai slop is anything but low quality code. ai can indeed be used well, however you yourself mentioned letting humans off the hook for making egregious mistakes. pushing out ai slop IS an egregious mistake. when a release contains more commits than the previous N releases, slop likelihood increases, therefore further evidence is required to prove non sloppiness.
This is a neat post and I'm glad it got written and this is a little bit off-topic but:
Hey, 'logicprog, your writing is fine!
Use LLMs to critique your writing, check its structure, vet your choice of topic sentences, check flow from graf to graf and section to section, look for passive voice and overused words. LLMs are fantastic for that. But don't use a single word an LLM suggests in your actual writing. If it suggests something really fucking good, too bad, those words are disqualified. It's an easy red line to adhere to, easier than it sounds, and it'll keep your writing human.
(You ended up somewhere around here anyways, but that was after you posted something with LLM-written language because you weren't confident enough in your own writing. The things you do "worse" than an LLM are what make you you; be protective of them!)
Hey, 'logicprog, your writing is fine!
Use LLMs to critique your writing, check its structure, vet your choice of topic sentences, check flow from graf to graf and section to section, look for passive voice and overused words. LLMs are fantastic for that. But don't use a single word an LLM suggests in your actual writing. If it suggests something really fucking good, too bad, those words are disqualified. It's an easy red line to adhere to, easier than it sounds, and it'll keep your writing human.
(You ended up somewhere around here anyways, but that was after you posted something with LLM-written language because you weren't confident enough in your own writing. The things you do "worse" than an LLM are what make you you; be protective of them!)
Thank you!
I'm just curious about testing.
Is this a configuration that's not common and thus not tested?
If people think they can do better, I want to see their forks and them keeping up with it.
https://github.com/RsyncProject/rsync/graphs/contributors?fr...
Is this a configuration that's not common and thus not tested?
If people think they can do better, I want to see their forks and them keeping up with it.
https://github.com/RsyncProject/rsync/graphs/contributors?fr...
Thank you for (re)writing this in your own voice. Despite how much effort might be put into methodology, data collection, etc.. reading slop is unbearable, full stop. It's not intentional, but I have almost a nauseated reaction when the "AI tone" comes though, regardless of how good the data or how accurate the writing is.
Your verbosity and sentence structure are not a problem. I hope that publishing this gives you a bit more confidence in your writing, because it's legitimately good.
Your verbosity and sentence structure are not a problem. I hope that publishing this gives you a bit more confidence in your writing, because it's legitimately good.
Personally, I'm going to believe tridge, someone that has contributed more to software than 99.9(recurring) of the software development community over the last 3+ decades, than a bunch of brigaders jumping on the anti-AI backlash.
There was one regression bug apparently (related to multiple destinations and the way people do backups), but all the attention/anger has been about a test suite that makes the rsync development better, more rigorous and copes with the onslaught of both good and bad AI generated PRs as well as hardening something that has two decades of C code in it.
People need to grow up and appreciate what others in the community (especially people like tridge) have provided.
There was one regression bug apparently (related to multiple destinations and the way people do backups), but all the attention/anger has been about a test suite that makes the rsync development better, more rigorous and copes with the onslaught of both good and bad AI generated PRs as well as hardening something that has two decades of C code in it.
People need to grow up and appreciate what others in the community (especially people like tridge) have provided.
If the author is this concerned about security, I’m curious why rsync doesn’t just build with fil-c by default and skip the noise. Those who need the extra perf to do more than 1 gigabit/s can build it in “unsafe” mode.
Software is only as healthy as that of the mental-models of its "human" maintainers. This was implicit in the writings of Peter Naur (programming as theory building) and the Fred Brooks (mythical man month) ages ago. AI as a tool can assist just as IDEs and linters have assisted. But eventually the mental models of the human maintainers is the gate and bottleneck. Applying this to rsync, its maintainer would need to foster and grow other human contributors to eventually become maintainers so that the human mental models are carried forward.
The fact last few commits were attributed to claude doesn't mean previous ones didn't use it.
Also if you write a paper where you get statistical conclusions out of whole 2 datapoints you'd be laughed out of the room
Also if you write a paper where you get statistical conclusions out of whole 2 datapoints you'd be laughed out of the room
> Also if you write a paper where you get statistical conclusions out of whole 2 datapoints you'd be laughed out of the room
I'm using methods appropriate to that low amount of data, first of all. Second of all, since I'm only trying to show there's no evidence for the anti-AI hypothesis (not disprove it, or prove the null hypothesis), that's sufficient in itself. Also, I wonder why nobody said things like you're saying ("there's too little data to tell") in response to all the absolutist claims that AI caused rsync to get worse?
> The fact last few commits were attributed to claude doesn't mean previous ones didn't use it.
At this point, you're just positing Russel's Teapot: you'll keep assuming more and more of the code was "secretly" Claude when there's no evidence for it and no reason to think so, just because you've started with the assumption that Claude makes things worse and you want to find a way to prove it.
I'm using methods appropriate to that low amount of data, first of all. Second of all, since I'm only trying to show there's no evidence for the anti-AI hypothesis (not disprove it, or prove the null hypothesis), that's sufficient in itself. Also, I wonder why nobody said things like you're saying ("there's too little data to tell") in response to all the absolutist claims that AI caused rsync to get worse?
> The fact last few commits were attributed to claude doesn't mean previous ones didn't use it.
At this point, you're just positing Russel's Teapot: you'll keep assuming more and more of the code was "secretly" Claude when there's no evidence for it and no reason to think so, just because you've started with the assumption that Claude makes things worse and you want to find a way to prove it.
Why not? Claude marks its commit messages. That there were none, and then there were, seems a signal.
Especially since if the earlier commits were so clearly AI authored yet without the Claude marker, surely you or anyone would be able to spot them. You could say, X commit does not have the Claude commit marker yet was AI written. But for all the speculation on this thread, I haven’t seen anyone actually doing that. What may be possible is that the rsync maintainers used AI to assist yet reviewed and edited themselves, as many devs do, and if so then the stats in this article are still notable: there are no poor quality outliers that can reliably be attributed to AI and if one specific release (3.4.0) was, the subsequent releases which presumably also had as much AI as this speculative hidden AI release only show improvement and thus act as a pro-AI argument.
The blog has many more datapoints than two. It compares many releases. You’re looking at 2-vs, not 2.
Especially since if the earlier commits were so clearly AI authored yet without the Claude marker, surely you or anyone would be able to spot them. You could say, X commit does not have the Claude commit marker yet was AI written. But for all the speculation on this thread, I haven’t seen anyone actually doing that. What may be possible is that the rsync maintainers used AI to assist yet reviewed and edited themselves, as many devs do, and if so then the stats in this article are still notable: there are no poor quality outliers that can reliably be attributed to AI and if one specific release (3.4.0) was, the subsequent releases which presumably also had as much AI as this speculative hidden AI release only show improvement and thus act as a pro-AI argument.
The blog has many more datapoints than two. It compares many releases. You’re looking at 2-vs, not 2.
Not going to critique this survey. Must have taken a lot of time and required a lot of patience. Great work!
I think it will be up to some group in academia to make a real full blown study across several repositories.
There must be tons to learn on how LLMs have changed software development and perhaps the cleanest separation will simply be going by what repositories declare e.g. "No LLM involved" vs those that proudly do the opposite or are neutral.
Bugs is not the only variable of interest here. I am guessing someone is already doing this as we discuss it here...
I think it will be up to some group in academia to make a real full blown study across several repositories.
There must be tons to learn on how LLMs have changed software development and perhaps the cleanest separation will simply be going by what repositories declare e.g. "No LLM involved" vs those that proudly do the opposite or are neutral.
Bugs is not the only variable of interest here. I am guessing someone is already doing this as we discuss it here...
so the answer is: no. actaully less bugs. thanks
"fewer"
First rsync and now less? What comes next, cat?
[deleted]
Rsync is a highly trusted software, included in many distros. To move important, and high quantity
If several or critical lines of code get changes quickly, and keeps breaking things, with or without llms, there will be backlash
Rsync should rightly loose reputation if the project allows the release breaking changes to follow the latest hype trend
If several or critical lines of code get changes quickly, and keeps breaking things, with or without llms, there will be backlash
Rsync should rightly loose reputation if the project allows the release breaking changes to follow the latest hype trend
Trust is slowly gained and easily lost. The amount of apologia I hear from top-tier developers signals an inflection point downward.
[deleted]
I think we should start having ai versions like beta ora alpha versions and then consolidate them into human made versions with time, at least one is free to stay safe or on the bleeding edge as one likes and we all get a win-win best-of-all-worlds situation (hopefully)
Flagged. Article is as AI heavy as the commits that people are complaining about.
So a project is using a GPL licence, but instead of forking you harass the authors and you somehow think that you are the smart one and that you are doing anyone a favour?
I'm noticing more and more AI writing everywhere, from youtube to this article: From the subtitle: "Nothing complicated, answers only one question: " clearly LLM generated.
What’s the deal with anti ai people being so rude
I wonder if all the commits which involve adding tons more test are the basis for a rewrite in rust anthropic marketing event
I've got no love for AI, don't use it, but also after writing code for more than 40 years, keeping things in perspective helps. Whether it's you pecking away or some coding assistant helping, there will always be the potential for a regression or two to fly under the radar. (not like I've never done that before... nope.)
The issue the coding tools like Claude present is the sheer size and scope of changes and commits they generate that would take mere mortals months of careful coding to do.
That's an issue everyone using those tools will have to confront. I don't know Andrew personally, from a "let's go have a beer" standpoint, but I've known him from the samba list and his work with rsync for a very long time.
My take on the issue is less about the regressions and Claude screw-ups and more the lesson to all about the reliability of the coding tools and the diligence required to validate what they spit out.
It's an unfortunate black-eye, no doubt, but it's not a unique one. The takeaway is if something like this can slip by somebody like Andrew, then we all need to redouble the validation effort, lest we too are destined to share an unfortunate black-eye or two.
Never forget, "to err is human, but to really foul things up requires a computer."
AI just applies that adage at industrial-scale.
The issue the coding tools like Claude present is the sheer size and scope of changes and commits they generate that would take mere mortals months of careful coding to do.
That's an issue everyone using those tools will have to confront. I don't know Andrew personally, from a "let's go have a beer" standpoint, but I've known him from the samba list and his work with rsync for a very long time.
My take on the issue is less about the regressions and Claude screw-ups and more the lesson to all about the reliability of the coding tools and the diligence required to validate what they spit out.
It's an unfortunate black-eye, no doubt, but it's not a unique one. The takeaway is if something like this can slip by somebody like Andrew, then we all need to redouble the validation effort, lest we too are destined to share an unfortunate black-eye or two.
Never forget, "to err is human, but to really foul things up requires a computer."
AI just applies that adage at industrial-scale.
Unrelated, but this post has a level of rigor you rarely see nowadays. I think it deserves to be commended for that.
HN relatively, is a very intellectual part of the internet, yet even still, it's really common to see very uneducated opinions here. Not that everyone needs to be very educated, but posts with plainly wrong assumptions and biases shouldn't go completely unchecked so rampantly.
HN relatively, is a very intellectual part of the internet, yet even still, it's really common to see very uneducated opinions here. Not that everyone needs to be very educated, but posts with plainly wrong assumptions and biases shouldn't go completely unchecked so rampantly.
Reposting my previous comment because the post I commented on earlier was flagged to death:
This is kind of a sad situation. Tridge is an excellect programmer and a very respected member of the community, and I totally get it. rsync, like most old C projects, has a lot of accumulated cruft, and things that would be nice to fix, and bugs. And those bugs come in at least three classes: semantic bugs, improper interactions with the OS, and memory safety bugs. And the author and long-time maintainer has the same problem as every other maintainer and team: not enough time to deal with everything. And now LLMs come along, and they are so, so seductive. They will fix your bugs if you ask them to. They will even find your bugs. And they're right a remarkably large fraction of the time. It's magic! You can write an agent loop or magic harness or swarm and let them do this on their own if you want. And so you start getting through your backlog, and it's fun, and you feel good, and you let your guard down. And you start having problems: - Your favorite LLM does not have the context that lives in your head. I use rsync because Tridge wrote a fine piece of software, and he knows how to write serious software, and I'm willing to accept that it's in C and therefore almost certainly has a safety bug or three. If I wanted to use claude-ersatz-rsync, I'd use that instead, but I really don't, TYVM. - Remember how LLMs are right a remarkable fraction of the time? The fraction is remarkable, but it's nowhere close to 100%. (Yet? Who knows. Right now, it's DEFINITELY nowhere near 100%.) - The training process for the current crop of LLMs does not adequately reinforce long-term maintainability of the outputs. And, for all the LLMs seem magic, they seem to love a workload in which they write code with poorly named functions and no docs and sort of assume that they can parse their own code down the road and figure out WTF is going on, and they are AT BEST only a tiny bit right. Because every project has interfaces where one module touches another, and every LLM has very limited context (larger than humans' in straight up verbatim working memory but MUCH MUCH WORSE than humans' (for now, anyway) in actual broad picture retention), and this workload doesn't work. If it did, we could give up on structured programming and just have the LLMs vomit up uncommented asm. And so, where humans have conventions and decently named functions and ideas that you shouldn't churn your code just for funsies (at least not in a production context), LLMs do this: https://github.com/RsyncProject/rsync/commit/30656c5e358b1c6... Most of that is blindly changing calls do functions like do_foo(args) (which makes sense) to do_foo_at(the same args), which makes no sense. Sorry, but the world of POSIXish-targetting programers (including, presumably, Claude) knows what _at means, and it means "at" the specified directory fd. Which is not specified in the call sites. It makes no sense at all. Buried in all that mess [0] is the implementations, which are sloppy. Seriously: - There's a function called do_utimensat_at. Is Claude stuttering? - There's a lovely comment in syscall.c:1660-1673 that's quite bad. It's handling strings that contain "/../" and such. If there's some actual contract that the function makes to its callers (and there surely is -- this is critical security-sensitive code), then SAY WHAT THE CONTRACT IS. Don't bury a partial explanation in a comment in the middle. - There's a repeated pattern: In do_foobar_at(path), there is, in effect: if (!path) do_foobar(path); Nice NULL pointer handling. Is NULL a valid argument or not? Why handle it by forwarding it to the less secure variant? - Those nice, supposedly secure "at" variants check for paths that start with '/' and forward to the raw insecure syscall. And they don't check for .. in the middle. So what, exactly, is the special code for .. promising to do? (See above.) I don't think more details are needed. But my take is that this whole thing is a mistake. I personally work on the sort of code where messes like this are entirely unacceptable. And using an LLM while maintaining the kind of oversight that prevents it is mentally taxing and not exactly fun. If you want to fix all the gunk in a C program like rsync by LLM magic, go rewrite it in Rust or something -- you're already exposing yourself to a massive rewrite and all the risks that entails, and you're pretty much guaranteeing a high level of sloppiness, so at least use a language that is more resistant to slop.
[0] Which GitHub doesn't even render by default because their diff viewer is so bad.
[There were follow-ups. See https://news.ycombinator.com/item?id=48352182]
This is kind of a sad situation. Tridge is an excellect programmer and a very respected member of the community, and I totally get it. rsync, like most old C projects, has a lot of accumulated cruft, and things that would be nice to fix, and bugs. And those bugs come in at least three classes: semantic bugs, improper interactions with the OS, and memory safety bugs. And the author and long-time maintainer has the same problem as every other maintainer and team: not enough time to deal with everything. And now LLMs come along, and they are so, so seductive. They will fix your bugs if you ask them to. They will even find your bugs. And they're right a remarkably large fraction of the time. It's magic! You can write an agent loop or magic harness or swarm and let them do this on their own if you want. And so you start getting through your backlog, and it's fun, and you feel good, and you let your guard down. And you start having problems: - Your favorite LLM does not have the context that lives in your head. I use rsync because Tridge wrote a fine piece of software, and he knows how to write serious software, and I'm willing to accept that it's in C and therefore almost certainly has a safety bug or three. If I wanted to use claude-ersatz-rsync, I'd use that instead, but I really don't, TYVM. - Remember how LLMs are right a remarkable fraction of the time? The fraction is remarkable, but it's nowhere close to 100%. (Yet? Who knows. Right now, it's DEFINITELY nowhere near 100%.) - The training process for the current crop of LLMs does not adequately reinforce long-term maintainability of the outputs. And, for all the LLMs seem magic, they seem to love a workload in which they write code with poorly named functions and no docs and sort of assume that they can parse their own code down the road and figure out WTF is going on, and they are AT BEST only a tiny bit right. Because every project has interfaces where one module touches another, and every LLM has very limited context (larger than humans' in straight up verbatim working memory but MUCH MUCH WORSE than humans' (for now, anyway) in actual broad picture retention), and this workload doesn't work. If it did, we could give up on structured programming and just have the LLMs vomit up uncommented asm. And so, where humans have conventions and decently named functions and ideas that you shouldn't churn your code just for funsies (at least not in a production context), LLMs do this: https://github.com/RsyncProject/rsync/commit/30656c5e358b1c6... Most of that is blindly changing calls do functions like do_foo(args) (which makes sense) to do_foo_at(the same args), which makes no sense. Sorry, but the world of POSIXish-targetting programers (including, presumably, Claude) knows what _at means, and it means "at" the specified directory fd. Which is not specified in the call sites. It makes no sense at all. Buried in all that mess [0] is the implementations, which are sloppy. Seriously: - There's a function called do_utimensat_at. Is Claude stuttering? - There's a lovely comment in syscall.c:1660-1673 that's quite bad. It's handling strings that contain "/../" and such. If there's some actual contract that the function makes to its callers (and there surely is -- this is critical security-sensitive code), then SAY WHAT THE CONTRACT IS. Don't bury a partial explanation in a comment in the middle. - There's a repeated pattern: In do_foobar_at(path), there is, in effect: if (!path) do_foobar(path); Nice NULL pointer handling. Is NULL a valid argument or not? Why handle it by forwarding it to the less secure variant? - Those nice, supposedly secure "at" variants check for paths that start with '/' and forward to the raw insecure syscall. And they don't check for .. in the middle. So what, exactly, is the special code for .. promising to do? (See above.) I don't think more details are needed. But my take is that this whole thing is a mistake. I personally work on the sort of code where messes like this are entirely unacceptable. And using an LLM while maintaining the kind of oversight that prevents it is mentally taxing and not exactly fun. If you want to fix all the gunk in a C program like rsync by LLM magic, go rewrite it in Rust or something -- you're already exposing yourself to a massive rewrite and all the risks that entails, and you're pretty much guaranteeing a high level of sloppiness, so at least use a language that is more resistant to slop.
[0] Which GitHub doesn't even render by default because their diff viewer is so bad.
[There were follow-ups. See https://news.ycombinator.com/item?id=48352182]
Tips: In HN, You need blank line (i.e. Hit ENTER twice) to start a new paragraph. -- Everything jams into an incomprehensible wall of text if you use one new line.
Ugh. The source comment, which this was literally a copy and paste of, had newlines. I wish HN could roundtrip from itself via the clipboard correctly.
I haven't used this thing for like 10 years, when my modus operandi was googling my question and installing whatever stackoverflow suggested.
Can someone explain why one would ever use rsync (pre vibecode version) instead of cp and dd?
Can't we just 'apt remove rsync' and save ourselves the time even spent on evaluating this dependency?
Thanks
Can someone explain why one would ever use rsync (pre vibecode version) instead of cp and dd?
Can't we just 'apt remove rsync' and save ourselves the time even spent on evaluating this dependency?
Thanks
If you deal with large numbers of files, the ability to dynamically skip compressing media and zipped files for transfer can be extremely handy.
While stuff like sshfs is great for a few small files (and win11), it will be an order of magnitude slower than an rsync task.
Most smart folks automate backup/recovery scripts, and only sometimes edit them with a new OS install. =3
While stuff like sshfs is great for a few small files (and win11), it will be an order of magnitude slower than an rsync task.
Most smart folks automate backup/recovery scripts, and only sometimes edit them with a new OS install. =3
> rsync (remote sync) is a utility for transferring and synchronizing files between a computer and a storage drive and across networked computers by comparing the modification times and sizes of files.
https://wikipedia.org/wiki/Rsync
https://wikipedia.org/wiki/Rsync
Because cp will copy everything, while rsync will copy only the things that actually need copying, and also delete the things that should be gone?
What on earth is this. Literally the only thing that matters is are there more bugs after AI written code is allowed into the codebase at all. We all know the answer to that lol. But it's always nice to see "data" can be used to make any conclusion you need.
The data literally shows there aren't, there have been worse releases before. In what way did I manipulate the data?
Regardless of the claims made in this analysis, I've personally observed that there are indeed more bugs (or more subtle issues, like nonsensical error messages) being shipped when using LLMs, but not _really_ because LLMs suck, but because you're spending less time thinking about the problem, and you subsequently miss more edge cases, etc.
The best approach I've tried that actually increases quality (and _may_ speed up development) is to write ~80% of the code yourself and then ask LLM to review it thoroughly. While it's doing its thing you're also thinking about the code and reviewing it yourself in parallel. You then merge the findings and fix stuff worth fixing. At this point the authorship of the code is still mostly yours, you _understand_ the system and you ship fewer bugs, slightly faster than otherwise. It's a moderate improvement to the workflow, but it actually doesn't cost nearly as much either, and definitely doesn't produce rage at the machine from the slop. The only downside is that it requires lots of discipline, and it's a relatively rare commodity among software engineers these days.
The best approach I've tried that actually increases quality (and _may_ speed up development) is to write ~80% of the code yourself and then ask LLM to review it thoroughly. While it's doing its thing you're also thinking about the code and reviewing it yourself in parallel. You then merge the findings and fix stuff worth fixing. At this point the authorship of the code is still mostly yours, you _understand_ the system and you ship fewer bugs, slightly faster than otherwise. It's a moderate improvement to the workflow, but it actually doesn't cost nearly as much either, and definitely doesn't produce rage at the machine from the slop. The only downside is that it requires lots of discipline, and it's a relatively rare commodity among software engineers these days.
The peak of cascading effects from errant dependencies has yet to come
Do people enjoy interrogative headlines? Find out at 11.
> If anyone complains about my verbosity or sentence structure — as they usually do, which is the reason I originally let the AI write the prose, among other reasons obsoleted by templating — they can go fuck themselves.
You can write for an audience or you can write for yourself. Which is fine either way but you shouldn't pass the blame for bad results on to your audience.
> and recieving almost no substantive input, discussion, or response on the actual content of the article
Well did you write it for that purpose?
> "Just wait, more bugs will surface" -- v3.4.3 has been out long enough
Wait for _more releases_. As your own data shows the bug rate is not consistent between releases. So this is probably not a worthwhile metric. Perhaps systems touched, new features included, or attempted fixes would be a better way to contextualize releases and the goals of the author.
You can write for an audience or you can write for yourself. Which is fine either way but you shouldn't pass the blame for bad results on to your audience.
> and recieving almost no substantive input, discussion, or response on the actual content of the article
Well did you write it for that purpose?
> "Just wait, more bugs will surface" -- v3.4.3 has been out long enough
Wait for _more releases_. As your own data shows the bug rate is not consistent between releases. So this is probably not a worthwhile metric. Perhaps systems touched, new features included, or attempted fixes would be a better way to contextualize releases and the goals of the author.
Pretty cool site!
> v3.4.3 has been out long enough that its rate (5.00) is already comparable to historical releases. The "wait and see" argument is an appeal to an unknowable future that shifts the burden of proof away from the critics. If more bugs surface, they will enter the distribution like every other release. There is no reason to expect a regime break.
I mean, as someone who uses LLMs, it might be a good idea to consider how one might limit the amount of bugs that will appear in the future at least a little bit: parallel iterative code review loops would probably be the easiest and most applicable to LLMs, though I guess test coverage and other code analysis tools help too.
> v3.4.3 has been out long enough that its rate (5.00) is already comparable to historical releases. The "wait and see" argument is an appeal to an unknowable future that shifts the burden of proof away from the critics. If more bugs surface, they will enter the distribution like every other release. There is no reason to expect a regime break.
I mean, as someone who uses LLMs, it might be a good idea to consider how one might limit the amount of bugs that will appear in the future at least a little bit: parallel iterative code review loops would probably be the easiest and most applicable to LLMs, though I guess test coverage and other code analysis tools help too.
Another update: did an automated severity analysis on each bug report (~2000 of them!) using an LLM at temp=0 with a very strict rubric (and I checked to make sure that it rated things in a consistent, stable way using it). The rubric, LLM used, and some example ratings are included in the methodology section. For now, the information was just stored per-bug in the DuckDB and used to filter out non-bug bugs, to get a clearer signal. I'm going to try to use it to see if the post-Claude bugs were more severe in any way next.
In a scenario like this, where we can only know if the code has bugs, not if it doesn't, isn't survival analysis a more appropriate statistical technique? I.e. a technique where time is a first-class citizen.
By the way, I did find this a bit hard to read but, as instructed by OP, I'll go fuck myself.
For what it's worth, I find AI written prose easy to read, and am annoyed by all the constant HN comments which just point out the author was AI, without anything else substantive to add.
By the way, I did find this a bit hard to read but, as instructed by OP, I'll go fuck myself.
For what it's worth, I find AI written prose easy to read, and am annoyed by all the constant HN comments which just point out the author was AI, without anything else substantive to add.
The TLDR seems to be: needs more data.
needs a tldr; im not reading all that. maybe claude can summarize it for me.
And anti-AI people accuse people who use AI of being intellectually lazy. First of all, it's long because it's expanded to respond to all the criticisms. It seems that either something can be short, and dismissed as incomplete, or it can be complete, and dismissed as being long. Nice Kafka trap. Additionally, there's literally an Executive Summary section right there, for your TLDR.
Asked your Clanker what a joke is.
[deleted]
LLMs don't create bugs, people do
This is just narrow thinking. Say Claude did increase the bugs in rsync by a negligible factor.
So what? You've saved a significant amount of time for a decent number of humans, and if those humans are working on other projects, the overall net output for the world is net positive compared to without LLMs.
You have to broaden your perspective. It's not just about how rsync was affected.
So what? You've saved a significant amount of time for a decent number of humans, and if those humans are working on other projects, the overall net output for the world is net positive compared to without LLMs.
You have to broaden your perspective. It's not just about how rsync was affected.
Let me translate this comment:
> ok, so I was wrong and badly, but I will double down and say I was right anyway
> ok, so I was wrong and badly, but I will double down and say I was right anyway
What strikes me about the post is that it goes to great lengths to talk about proper statistical methods, but then is written in the most clearly biased language ("what stupid AI haters get wrong etc). If you want people to take your study seriously, why wreck it by coming across with such a strong prior bias? I stopped reading...
To be fair, the tone of the article is practically chill compared to the comments it is written in response to.
If they're the statistical methods and metrics hold up, or they don't. Also, if you don't want to read my opinion on things, then just grab the GitHub repo and run the end-to-end replication and look at the output data yourself.
What followed was extraordinary: 329 comments and counting, ranging from thoughtful concern to outright harassment.
The thread did not stop at words. One user posted My Little Pony drawings of themselves strangling the "project janitor that pushed vibecoded commits":
It spread to Hacker News and Lobsters, generating hundreds more comments.
Please correct your article.
It is neat that Lobsters has this feature (and HN should too), and I'm glad you took a beat to explain it. I think you didn't need the last sentence, though.
I have done so! that was a misremembering on my part. first mention of Lobsters is now here:
> On Lobste.rs, in response to the Medium essay Tridge himself posted in response, finally some users like boramalper begin to actually ask for evidence one way or another:
> On Lobste.rs, in response to the Medium essay Tridge himself posted in response, finally some users like boramalper begin to actually ask for evidence one way or another:
Thanks, I appreciate you sorting out the timeline on such a heated issue.
Is this an analysis made by/with Claude?
It very obviously is. "The Outlier Nobody Noticed" -_-"
FWIW, I asked ChatGPT to review the article just for my amusement. It's conclusion was:
"My honest assessment is that this is a competent calculation performed on a badly confounded measurement, followed by conclusions substantially stronger than the calculation warrants. It is useful as a rebuttal to “the Claude releases are obviously unprecedented disasters,” but not as evidence that Claude was harmless."
"My honest assessment is that this is a competent calculation performed on a badly confounded measurement, followed by conclusions substantially stronger than the calculation warrants. It is useful as a rebuttal to “the Claude releases are obviously unprecedented disasters,” but not as evidence that Claude was harmless."
[stub for offtopicness]
[see https://news.ycombinator.com/item?id=48416020 for how all this happened in the first place]
[see https://news.ycombinator.com/item?id=48416020 for how all this happened in the first place]
Some notes on this:
- I used GLM 5.1 to help with the coding and math for this.
- However, I explicitly dictated where the data should be pulled from (GitHub, Bugzilla, mailing list), how it should be tagged and grouped, and what data to look at (e.g. bugs instead of regressions)
- Additionally, I consulted with my wife, who has a master's degree in statistics from Penn State University for what sort of statistical methodology would be justified for this very limited data set, while still giving as much information as possible.
- I know the website looks like we stereotypically consider vibe-coded websites to look, but I actually explicitly asked for that. The original HTML design looked like a website from 1995, and I just prefer how this looks. It's pretty!
- I used GLM 5.1 to help with the coding and math for this.
- However, I explicitly dictated where the data should be pulled from (GitHub, Bugzilla, mailing list), how it should be tagged and grouped, and what data to look at (e.g. bugs instead of regressions)
- Additionally, I consulted with my wife, who has a master's degree in statistics from Penn State University for what sort of statistical methodology would be justified for this very limited data set, while still giving as much information as possible.
- I know the website looks like we stereotypically consider vibe-coded websites to look, but I actually explicitly asked for that. The original HTML design looked like a website from 1995, and I just prefer how this looks. It's pretty!
I really struggle to believe you wrote text like:
> A simple distributional analysis of every rsync release with bug data. No model. No assumptions. Just placement.
> A simple distributional analysis of every rsync release with bug data. No model. No assumptions. Just placement.
No, I didn't write the text itself. I'm typically significantly more verbose and elliptical, and more than that, the numbers and methodology changed often enough over the course of the last couple days I was working on this because I was trying to get it to be as accurate and fair as possible that trying to keep the whole thing up to date manually would have been problematic.
Sorry to say but I'm absolutely certain I would've preferred to read your worst attempt at a write-up over the grating utter shite LLMs output. It's not even a question, this is unreadable.
That's interesting; IME, most people get equally angry and are as likely to disengage with a superior tone over my autism-infodump verbose essay prose as with LLM output.
At least when I write an autistic info dump people know I wrote it. Why give your voice over to a corpo slop factory?
Heck, I use LLM assistance for coding and I’ve even coded up whole features with the clankers, but giving it the right to speak for me is too much.
I should also add that I read and understand every line of clanker output that I publish for others, so I’m not a vibe coder either, just adhd.
Heck, I use LLM assistance for coding and I’ve even coded up whole features with the clankers, but giving it the right to speak for me is too much.
I should also add that I read and understand every line of clanker output that I publish for others, so I’m not a vibe coder either, just adhd.
I read it perfectly fine. I see content, not style.
Style is also part of the content. Word choice, grammar, register, and tone all affect meaning and communication of that meaning. The medium is part of the message.
So your statement betrays a significant misunderstanding - there is no neat clean divide between style and content.
Also, LLMs often generate text that is plausible, but wrong, in ways big and small.
So your statement betrays a significant misunderstanding - there is no neat clean divide between style and content.
Also, LLMs often generate text that is plausible, but wrong, in ways big and small.
Well, I got the meaning in the article fine, and have no complaints.
> Also, LLMs often generate text that is plausible, but wrong, in ways big and small.
So do humans. Always have, always will.
> Also, LLMs often generate text that is plausible, but wrong, in ways big and small.
So do humans. Always have, always will.
Humans acting with intention do it a lot less. The difference is that LLMs don’t act with intention.
No, the difference is in the education/experience of any given human, which is mostly gated by age. Like you'd generally expect someone young to make a lot of mistakes, and as time went on they'd learn and make fewer. Pretty much the same with LLMs, which have been around for... a bit over 5 years now? What would you expect of a 5 year old acting with intention? Or 10? Or even a 15 year old?
[deleted]
When you say, "I see content, not style," you are separating what is being said from how it is being said. While it is great that you can extract the core message, you are missing a fundamental truth about writing: style and content are rarely completely separate. Writing involves both.
Poor prose does not just make writing ugly — it creates friction, obscures nuance, and introduces ambiguity.
You can eat a gourmet meal out of a dirty paper bowl. You still get the calories, but the delivery mechanism definitely impacts the experience and the perceived value of the food. Same food, different response.
See? I can write slop too, I don't even need to burn down a forest to do it. If you are OK with every fucking thing being written exactly like this, good for you. I am not.
Poor prose does not just make writing ugly — it creates friction, obscures nuance, and introduces ambiguity.
You can eat a gourmet meal out of a dirty paper bowl. You still get the calories, but the delivery mechanism definitely impacts the experience and the perceived value of the food. Same food, different response.
See? I can write slop too, I don't even need to burn down a forest to do it. If you are OK with every fucking thing being written exactly like this, good for you. I am not.
The internet is going to really suck for you if you keep that attitude, because LLM use will only increase. Though also maybe not too much as the LLM-isms will likely be fine-tuned out of them to the point that the only way you'll be sure something is done with one is if the author left a note saying such. But maybe that'll make it suck even more as then you'd be without a definite target most of the time, always wondering how much of the thing you're reading is by human and how much by LLM...
Uh... huh.
I waited a minute to make sure you weren't going to delete this post because frankly, if I had written it, I would have. Guess not, so... Here goes.
No. It is not the fault of my "attitude" that the Internet is going to suck. That is a complete reversal of the reality. The fact that even people without bad intent are already spreading slop everywhere should be enough evidence to essentially prove that there was never any hope. If this is what good actors are doing, what exactly do you expect from bad actors?
Also, to stress it yet again, I don't care if people use LLMs in general. I'll even say that I don't particularly care very much if people use them without disclosing it in most cases. If you're using it like a normal tool and not merely just dumping the output verbatim there is not any particular need to disclose it any more than you'd disclose other tools, though I think people would prefer if you did just for transparency.
My chief complaint is just how bad LLM slop writing is. It simply is not good at all. It would literally be much better for the Internet if they weren't so turboshit at writing. There is almost no writing style I don't prefer over garbage LLM writing. I'm dead serious. Early LLMs were worse at almost everything else, but they were a lot better at writing for sure. Something went wrong somewhere.
But I do also believe that it is inherently bad to dump prose as-if you are communicating as a human, but said prose isn't actually written by a human. If someone shows me a cool drawing that they made, that means that they sat there and went through the process of sketching, possibly multiple drafts, inking, coloring/shading/painting/etc. to create an expression. This involves many human skills that take years to hone, and every detail carries someone's explicit intention. I think that this is cool, and shows a great degree of skill and effort.
When you, of course, generate some crap from an image generator, it may very well look similar. It may emulate some actual defects that make it look like someone really drew it. But someone didn't. A model went directly from a text prompt and dumped out pixels on screen. No sketching. No layers. No thought processes about how to frame things or what details to include. That doesn't mean zero effort went in: I'm sure in many cases someone sat around and fudged with LoRas and inpainting for a couple hours and pulled the slot machine lever to get good seeds and etc. That doesn't mean that an AI model does not have some model for how to structure an appealing image: it does, that's obviously why the results can look decent to begin with. But when you dump out an image from an image generator and you wink wink nudge nudge present it as your own and people evaluate it as if you drew it, this is basically fraud. Everyone looking at it who doesn't know it is AI generated actually believes you went through the normal effort of drawing that image and all of the years of practicing skills and acquiring knowledge that takes. That's bullshit, and it takes away from the actual accomplishments of people who put in the work like cheating in sports does.
Like yeah, a lot of people are cheating at chess, by passing off engine play as their own, but does that really make it okay? When the entire point is using your brain and not just the raw outputs themselves, doesn't that hit you as a problem?
For generative AI, I personally draw this line at what I feel are expressions of creativity. If you use AI for drawing references, whatever. If you use AI to generate globs of repetitive code, whatever. Code can be creative but I do not view it as an expression of creativity and almost any tool is fair game. If you are using ML models for motion capture or some other data processing thing where humans had to do repetitive work before, whatever. Maybe these tools sometimes do devalue the work, but the LLMs are not doing the interesting part here, they're doing the boring part. (This is, in part, an admission that actually writing code is often pretty boring in and of itself, something that I realize programmers have been inconsistent with in an attempt to justify their value. But, I still believe it to be true.)
So okay fine. People are reluctant to disclose that they used AI to generate text because they fear the backlash that it will get them. This is understandable. What upsets me about this is that well-meaning people are apparently falling back to the idea that because LLM backlash is strong, what would be better than either trying to just simply write your own damn posts or be honest about your usage of LLMs... Is to just try to wink wink nudge nudge pass off more or less verbatim LLM writing as if it's a post that you wrote.
I am not ruining the Internet. There is literally nothing I or any group of angry mobs could do that would even remotely slow down the decay of the Internet even if we desperately wanted to.
So in fact, I'm not even trying to not ruin the Internet. I don't particularly care if my attitude is not helping or hurting. I'm not having an attitude as part of some grand strategy to save or destroy the internet. I'm having an attitude, because I am pissed off.
And I am pissed off because I am tired of reading posts the author probably only skimmed themselves.
I waited a minute to make sure you weren't going to delete this post because frankly, if I had written it, I would have. Guess not, so... Here goes.
No. It is not the fault of my "attitude" that the Internet is going to suck. That is a complete reversal of the reality. The fact that even people without bad intent are already spreading slop everywhere should be enough evidence to essentially prove that there was never any hope. If this is what good actors are doing, what exactly do you expect from bad actors?
Also, to stress it yet again, I don't care if people use LLMs in general. I'll even say that I don't particularly care very much if people use them without disclosing it in most cases. If you're using it like a normal tool and not merely just dumping the output verbatim there is not any particular need to disclose it any more than you'd disclose other tools, though I think people would prefer if you did just for transparency.
My chief complaint is just how bad LLM slop writing is. It simply is not good at all. It would literally be much better for the Internet if they weren't so turboshit at writing. There is almost no writing style I don't prefer over garbage LLM writing. I'm dead serious. Early LLMs were worse at almost everything else, but they were a lot better at writing for sure. Something went wrong somewhere.
But I do also believe that it is inherently bad to dump prose as-if you are communicating as a human, but said prose isn't actually written by a human. If someone shows me a cool drawing that they made, that means that they sat there and went through the process of sketching, possibly multiple drafts, inking, coloring/shading/painting/etc. to create an expression. This involves many human skills that take years to hone, and every detail carries someone's explicit intention. I think that this is cool, and shows a great degree of skill and effort.
When you, of course, generate some crap from an image generator, it may very well look similar. It may emulate some actual defects that make it look like someone really drew it. But someone didn't. A model went directly from a text prompt and dumped out pixels on screen. No sketching. No layers. No thought processes about how to frame things or what details to include. That doesn't mean zero effort went in: I'm sure in many cases someone sat around and fudged with LoRas and inpainting for a couple hours and pulled the slot machine lever to get good seeds and etc. That doesn't mean that an AI model does not have some model for how to structure an appealing image: it does, that's obviously why the results can look decent to begin with. But when you dump out an image from an image generator and you wink wink nudge nudge present it as your own and people evaluate it as if you drew it, this is basically fraud. Everyone looking at it who doesn't know it is AI generated actually believes you went through the normal effort of drawing that image and all of the years of practicing skills and acquiring knowledge that takes. That's bullshit, and it takes away from the actual accomplishments of people who put in the work like cheating in sports does.
Like yeah, a lot of people are cheating at chess, by passing off engine play as their own, but does that really make it okay? When the entire point is using your brain and not just the raw outputs themselves, doesn't that hit you as a problem?
For generative AI, I personally draw this line at what I feel are expressions of creativity. If you use AI for drawing references, whatever. If you use AI to generate globs of repetitive code, whatever. Code can be creative but I do not view it as an expression of creativity and almost any tool is fair game. If you are using ML models for motion capture or some other data processing thing where humans had to do repetitive work before, whatever. Maybe these tools sometimes do devalue the work, but the LLMs are not doing the interesting part here, they're doing the boring part. (This is, in part, an admission that actually writing code is often pretty boring in and of itself, something that I realize programmers have been inconsistent with in an attempt to justify their value. But, I still believe it to be true.)
So okay fine. People are reluctant to disclose that they used AI to generate text because they fear the backlash that it will get them. This is understandable. What upsets me about this is that well-meaning people are apparently falling back to the idea that because LLM backlash is strong, what would be better than either trying to just simply write your own damn posts or be honest about your usage of LLMs... Is to just try to wink wink nudge nudge pass off more or less verbatim LLM writing as if it's a post that you wrote.
I am not ruining the Internet. There is literally nothing I or any group of angry mobs could do that would even remotely slow down the decay of the Internet even if we desperately wanted to.
So in fact, I'm not even trying to not ruin the Internet. I don't particularly care if my attitude is not helping or hurting. I'm not having an attitude as part of some grand strategy to save or destroy the internet. I'm having an attitude, because I am pissed off.
And I am pissed off because I am tired of reading posts the author probably only skimmed themselves.
In general, it seems HN does not like to read llm-generated articles. I ran into this myself when using an llm to edit some stuff I wrote.
At the time, I found this a bit irritating, but with a few weeks time I see the merit. The informational content tends to fall into “derivative” territory when LLM’s write stuff. And people are here for novelty and some socialization.
Also LLM prose seems optimized for engagement rather than concise communication. Takes longer to sift through linguistic boilerplate to get to the point. (The quoted bit being a case in point)
At the time, I found this a bit irritating, but with a few weeks time I see the merit. The informational content tends to fall into “derivative” territory when LLM’s write stuff. And people are here for novelty and some socialization.
Also LLM prose seems optimized for engagement rather than concise communication. Takes longer to sift through linguistic boilerplate to get to the point. (The quoted bit being a case in point)
Why would anyone spend time reading something that someone couldn't even spend the time to write themselves?
I just find it to be utter dreck. It has one of the most agitating prose styles I've ever seen. I would legitimately rather read actual broken English than the cliché polished turds Claude pops out. I am not an LLM hater, I think these tools are pretty impressive and often even useful, but even if I didn't care about the fact that I want to read communication from humans and not robots (and I do care about that, FWIW) I just find the current LLMs are horrid at writing.
And while the comments are always flooded with people like me, the upvotes seem to tell a different story; clearly LLM writing really does appeal to some people. Or idk, maybe a lot of people who vote on stories and don't comment don't actually read them. Hard to say for sure.
And while the comments are always flooded with people like me, the upvotes seem to tell a different story; clearly LLM writing really does appeal to some people. Or idk, maybe a lot of people who vote on stories and don't comment don't actually read them. Hard to say for sure.
I think it’s just people don’t read before voting, they upvote on the headline and then come to discuss it here.
[deleted]
I don't even know what "just placement" is.
(I need a better model to translate from llmese.)
(I need a better model to translate from llmese.)
Sometimes the things word generators say just don’t make sense.
I'd suggest writing the lead-in yourself and boxing AI prose separately from your prose in the analysis for future articles. You can give the humanized summary/eli5/key points, then have "details according to AI" boxes that go into nitty-gritty. People seem to dislike AI ghostwriting, but most of these people still use AI, so perhaps keeping authorship clear and separate will avoid some of the flak.
This seems fair. Of course, now that I've posted this here once, I doubt it'll get constructive engagement again, but I can at least improve this for the future
Even if everything in the article is true you should not use AI to write this. A analogy would be tobacco company report on how smoking isn’t so bad for you.
So the original unfounded claim has 400+ comments because its perfect HN ragebait
The author provides evidence to the contrary and the HNers won't even engage with it instead just talking about the writing of the article in classic HN bikeshedding fashion.
How about after that we talk about the formatting of the website and the colors?
This site is really going down hill
Where is the accountability for your own opinions?
Are you guys only upvoting things that confirm your existing gripes?
The author provides evidence to the contrary and the HNers won't even engage with it instead just talking about the writing of the article in classic HN bikeshedding fashion.
How about after that we talk about the formatting of the website and the colors?
This site is really going down hill
Where is the accountability for your own opinions?
Are you guys only upvoting things that confirm your existing gripes?
Comments like this do more of what they complain about, only with an extra layer of judgment.
It would be preferable if someone would seed a better discussion by engaging with the article's claims/observations.
It would be preferable if someone would seed a better discussion by engaging with the article's claims/observations.
Why did you the admin allow such ragebait to stay on the front page then?
Is that the kind of low effort posts we want around here? Just a link to a github comment of a screenshot?
You're complicit here in fueling the harassment of an open source project
Is that the kind of low effort posts we want around here? Just a link to a github comment of a screenshot?
You're complicit here in fueling the harassment of an open source project
I don't have enough background info to understand what you're referring to here.
Even if you're right, though, you shouldn't be posting comments that break the site guidelines.
Even if you're right, though, you shouldn't be posting comments that break the site guidelines.
https://news.ycombinator.com/item?id=48342705
People opening issues just to rant against an open source project is acceptable content for HN?
How is that even allowed in the first place without getting flagged/removed?
And every time that happens the project gets brigaded from HN users
People opening issues just to rant against an open source project is acceptable content for HN?
How is that even allowed in the first place without getting flagged/removed?
And every time that happens the project gets brigaded from HN users
People have been ranting for, against, and about open source projects since open source began.
Please follow the site guidelines from now on.
Please follow the site guidelines from now on.
This ain’t a rant it’s literally a screenshot of a mastodon tweet posted on an issue page
How does low effort rage-bait spark curiosity?
The comment to upvote ratio clearly shows it’s inflammatory
How does low effort rage-bait spark curiosity?
The comment to upvote ratio clearly shows it’s inflammatory
> A simple distributional analysis of every rsync release with bug data. No model. No assumptions. Just placement.
If you want me to read your analysis, you are going to have to make it not read like Claude wrote it. What does "placement" even mean here?
If you want me to read your analysis, you are going to have to make it not read like Claude wrote it. What does "placement" even mean here?
Yeah, made me chuckle that an LLM— probably Claude— was used to write this.
The use of "regime shift" is what gave it away for me. I've never seen a human write that, but Claude does from time to time.
At least they removed occurrences of "load-bearing".
The use of "regime shift" is what gave it away for me. I've never seen a human write that, but Claude does from time to time.
At least they removed occurrences of "load-bearing".
"quietly" seems to be the new one recently
Ohhh, quietly load-bearing is the real just. No noise. Pure fact. Delivered robustly.
It's the ultimate product for marketers. It inserts itself as an advertisement into every conversation now and defends itself against criticism. Just crazy. There's no hope for the rest of us.
It's not defending itself here, both because I used GLM 5.1, not Claude, and because I was the one who decided to do this analysis, iterated through six or seven different methodologies to try to find the one that was most honest with the data that I had (all of the methodologies showed directionally and often in magnitude the exact same thing, but I wanted to do something that fit the purpose, in consultation with my wife, who, as I've mentioned elsewhere, has a master's degree in statistics), and, of course, I specifically chose all of the metrics and sources for the data.
If you don't want to read the LLM prose, you can just go to the GitHub of my project, grab the scripts, and run the full pipeline. It will gather the data, build the database, and run the analysis from scratch for you, and you can look at the numbers directly. It's all repeatable.
If you don't want to read the LLM prose, you can just go to the GitHub of my project, grab the scripts, and run the full pipeline. It will gather the data, build the database, and run the analysis from scratch for you, and you can look at the numbers directly. It's all repeatable.
Your rewritten post is far easier for me to read now, fwiw.
LLM output has conditioned in me a near reflex response to just close a tab as soon as I smell LLM-authored text. Like, I'm not mad or anything, I just frequently find most default LLM-voiced text very unpleasant to read so I just don't continue reading.
LLM output has conditioned in me a near reflex response to just close a tab as soon as I smell LLM-authored text. Like, I'm not mad or anything, I just frequently find most default LLM-voiced text very unpleasant to read so I just don't continue reading.
"Placement" as in where the Claude-driven releases exist within the existing distribution of bugs per 100 commits. If they're not OOD, then nothing is unusual.
Also, it wasn't written by Claude FWIW, GLM 5.1.
Also, it wasn't written by Claude FWIW, GLM 5.1.
This submission was heavily flagged, presumably because the article sounded like genai. But the article now says the following:
> After posting this on Hacker News and recieving almost no substantive input, discussion, or response on the actual content of the article, I decided to rewrite all of the prose in my own voice.
I've therefore turned off the flags and hopefully people can actually now discuss the claims/findings being reported.
> After posting this on Hacker News and recieving almost no substantive input, discussion, or response on the actual content of the article, I decided to rewrite all of the prose in my own voice.
I've therefore turned off the flags and hopefully people can actually now discuss the claims/findings being reported.
> I decided to rewrite all of the prose in my own voice.
Soo... it didn't just sound like genai but was genai?
___
Huh. From the article:
> If anyone complains about my verbosity or sentence structure — as they usually do, which is the reason I originally let the AI write the prose, among other reasons obsoleted by templating — they can go fuck themselves.
This is kinda sad, honestly. But also should show the author that doing what people try to bully you into doing will not stop them from bullying you.
Just stick with your unique voice man. If people don't want to read that that's fine. They do not have to. You're fine
.. what are those em-dashes doing there though?
Soo... it didn't just sound like genai but was genai?
___
Huh. From the article:
> If anyone complains about my verbosity or sentence structure — as they usually do, which is the reason I originally let the AI write the prose, among other reasons obsoleted by templating — they can go fuck themselves.
This is kinda sad, honestly. But also should show the author that doing what people try to bully you into doing will not stop them from bullying you.
Just stick with your unique voice man. If people don't want to read that that's fine. They do not have to. You're fine
.. what are those em-dashes doing there though?
The em dashes are fine.
If someone gives them shit about their writing, that's on the critic for being shitty. If they use AI to write, that's on them for being fake. But, to write online at all requires being ready to have people be shitty to you and ideally not reacting in a way that makes the situation worse. Sounds like they need work on that part.
Anyway it is basically always possible for someone to find something legitimately bad about anything a person does. The question is, how much of an issue is that? Not much actually. So you have flaws. Fine, just be flawed. It had no affect on your life beyond your reaction to the attack. And putting aside that reaction is a prerequisite for learning anything useful (or discerning that there is nothing to learn) from the experience.
Good people will trust good intentions through the flaws, while shitty people will write off your work and your intentions because of the flaws (and try to make sure you feel bad about it in the process). But it's always they're too weak to express disagreement maturely, or sometimes because they're bitter and threatened by your good intentions directly. Either way, it's their flaw, not yours.
If someone gives them shit about their writing, that's on the critic for being shitty. If they use AI to write, that's on them for being fake. But, to write online at all requires being ready to have people be shitty to you and ideally not reacting in a way that makes the situation worse. Sounds like they need work on that part.
Anyway it is basically always possible for someone to find something legitimately bad about anything a person does. The question is, how much of an issue is that? Not much actually. So you have flaws. Fine, just be flawed. It had no affect on your life beyond your reaction to the attack. And putting aside that reaction is a prerequisite for learning anything useful (or discerning that there is nothing to learn) from the experience.
Good people will trust good intentions through the flaws, while shitty people will write off your work and your intentions because of the flaws (and try to make sure you feel bad about it in the process). But it's always they're too weak to express disagreement maturely, or sometimes because they're bitter and threatened by your good intentions directly. Either way, it's their flaw, not yours.
I don't think that you can successfully dismiss an obvious AI writing marker with
"No these are fine, now look over there!! <lotsoftext>"
Pay no attention to the man behind the curtain?
"No these are fine, now look over there!! <lotsoftext>"
Pay no attention to the man behind the curtain?
Right so it’s gonna be a litmus test for knowledge workers going forward if they can separate style over substance. Genai tells are style. You have to be able to evaluate the ideas.
I doubt that you can separate style from substance in that way, because you can't separate writing from thinking.
I agree that it will be interesting to see how this develops going forward. One can imagine wildly varying scenarios.
I agree that it will be interesting to see how this develops going forward. One can imagine wildly varying scenarios.
Hm. Nah. Why?
Why should I care? If it's a good thought, chances are it appears without slop around it. If it doesn't re-appear, life will still go on regardless.
No need to shift through noise just to avoid FOMO.
Why should I care? If it's a good thought, chances are it appears without slop around it. If it doesn't re-appear, life will still go on regardless.
No need to shift through noise just to avoid FOMO.
> I decided to rewrite all of the prose in my own voice
"Claude, rewrite all of the prose in my own voice."
The funny part is that it probably works.
"Claude, rewrite all of the prose in my own voice."
The funny part is that it probably works.
This article reeks of LLM "assistance" at the very least.
Please, why can't people write stuff by hand themselves any more? It's a good analysis but how can I trust it without reviewing everything myself?!
Please, why can't people write stuff by hand themselves any more? It's a good analysis but how can I trust it without reviewing everything myself?!
I mean, you can literally clone my repo, run the Python that rebuilds the database and does the whole data analysis and to end from scratch, and verify that the numbers are accurate. I made the code for this analysis public for that exact reason. This wasn't just an LLM running unsupervised in a loop. I came up with the methodologies and metrics and data scraping strategies precisely myself, iterated on it to try to be as honest with what the data could show as possible.
I think the point people are making is that when the text has an "AI smell" (it does), we immediately lose trust in the veracity of any claim being made and feel like continuing to read what is possibly a hallucinated fiction is a complete waste of time.
At this point we're all used to skimming through thousands of AI-generated sentences every working day and constantly thinking "this is likely to be 20% bullshit", it's hard to turn that off even if I try.
At this point we're all used to skimming through thousands of AI-generated sentences every working day and constantly thinking "this is likely to be 20% bullshit", it's hard to turn that off even if I try.
Do you think it would help if I went through and manually rewrote all of the prose? If it would get people to listen, I'd be totally willing to do it. It's not like I don't like writing. I just was focused on something else when I was making this, namely trying to find a good methodology that isn't insane for this low amount of data.
When there's no discernable human filter on the text output, reading the text suggests it's what the LLM produced and not what a human considered.
This is low-quality--every single day I witness Codex and Claude misunderstand, mislead, and hallucinate responses based on "assumptions" and I have to fact-check them.
If I wanted a statistical analysis and to be the human in the loop, I would ask the LLM myself, and I would definitely NOT read an article that just dumps the LLM output as-is.
This is low-quality--every single day I witness Codex and Claude misunderstand, mislead, and hallucinate responses based on "assumptions" and I have to fact-check them.
If I wanted a statistical analysis and to be the human in the loop, I would ask the LLM myself, and I would definitely NOT read an article that just dumps the LLM output as-is.
Yes, that would help considerably.
(Also, I suggest clearly acknowledging where AI was/wasn’t used. I like CuriosityC’s suggestion: https://news.ycombinator.com/item?id=48411968)
(Also, I suggest clearly acknowledging where AI was/wasn’t used. I like CuriosityC’s suggestion: https://news.ycombinator.com/item?id=48411968)
Alright, I'll do that. Although, sadly, I already posted it here, so I won't be able to post it again — I'll be stuck with this trash comments section that doesn't deal with any of the actual claims, just the aesthetics.
Just reread the post — it’s much more pleasant to read now! Thank you!
(For what it’s worth, I think your own writing style is quite nice, now that I can see it.)
(For what it’s worth, I think your own writing style is quite nice, now that I can see it.)
I'm pretty sure more people would read it to the end if it didn't seem like AI output, yes.. At the very least you would have fewer (maybe not 0!) comments here saying it's AI slop.
> I mean, you can literally
You didn't care enough to make a good writeup, why should we believe that you cared enough to make a good analysis?
You didn't care enough to make a good writeup, why should we believe that you cared enough to make a good analysis?
You don't have to believe. The repository is there for anyone to attempt reproducing the results. Criticisms without proof when there's a pretty straightforward way toward that proof are pointless. Go run the experiment and rip that apart if it doesn't hold up. And until then, refrain from criticizing.
[deleted]
A lot of people criticizing because it's heavily written with LLM, but I mean, if someone produced this piece pre-LLM, would they criticize it? is the critique due to use of LLM or due to the content being truly hard to follow? I read it and I would say, there are some problems with the writing, but its not a bad piece.
Of course this is a bigger problem, as its now harder to distinguish content that is "AI slop" with "content co-authored with AI that is carefully reviewed" with a quick glimpse, and the "AI smell" is quite off-putting. My initial reaction was also negative, but after glimpsing it through and reading the summaries, I found it decent summary, which also... speaks of this thread, of the content of the blog post and everything about the discussion and the strong feelings people have developed around the use of LLMs.
Anyhow, it would be good to disclose the repo with the code for the statistics & use of LLM in the writing right up front. Which model, and why it was used to do the writing, etc. Its enough to say "I think it writes better than I do" or "I was in a hurry, sorry" or what ever, but it really should be disclosed. It reads more honest.
ps. really... that sideways scroll? plz fix it.
Of course this is a bigger problem, as its now harder to distinguish content that is "AI slop" with "content co-authored with AI that is carefully reviewed" with a quick glimpse, and the "AI smell" is quite off-putting. My initial reaction was also negative, but after glimpsing it through and reading the summaries, I found it decent summary, which also... speaks of this thread, of the content of the blog post and everything about the discussion and the strong feelings people have developed around the use of LLMs.
Anyhow, it would be good to disclose the repo with the code for the statistics & use of LLM in the writing right up front. Which model, and why it was used to do the writing, etc. Its enough to say "I think it writes better than I do" or "I was in a hurry, sorry" or what ever, but it really should be disclosed. It reads more honest.
ps. really... that sideways scroll? plz fix it.
> content co-authored with AI that is carefully reviewed
The problem I see is that this is indistinguishable to a reader at a glance.
Distancing the writing from the "AI smell" not only improves the quality by dropping the unnecessary ocean of rhetorical devices, it forces the human to have real weight and agency on what's being said.
I think that act of distancing from raw LLM output through refinement is a huge quality leap. Even if you're only doing the refinement with an LLM, it forces the writing to have more voice and ideas from the author.
I can see the work that went into the analysis here but again, as a casual reader, it's impossible to tell that there were any original ideas here expressed by the author.
The problem I see is that this is indistinguishable to a reader at a glance.
Distancing the writing from the "AI smell" not only improves the quality by dropping the unnecessary ocean of rhetorical devices, it forces the human to have real weight and agency on what's being said.
I think that act of distancing from raw LLM output through refinement is a huge quality leap. Even if you're only doing the refinement with an LLM, it forces the writing to have more voice and ideas from the author.
I can see the work that went into the analysis here but again, as a casual reader, it's impossible to tell that there were any original ideas here expressed by the author.
Thank you for your constructive input, you're one of only a few others here who had any. I'll definitely do that. I didn't think, since the output was templated directly from the numbers generated by a reproducible python script, that people would get so up in arms about the aesthetics, but I guess I forgot to say that.
The most quoted line here is "A simple distributional analysis of every rsync release with bug data. No model. No assumptions. Just placement." Not only is it cringe to read, it's also nonsensical ("placement" means what?)
If OP had said "here's an AI summary of the data" and generated a conscise summary, I think I would fine with it. But default AI writing is really verbose -- the opposite of a compression algorithm, spewing out cliched phrases that don't add information. It's exhausting to read, and it lacks the interesting noise of a human response.
If OP had said "here's an AI summary of the data" and generated a conscise summary, I think I would fine with it. But default AI writing is really verbose -- the opposite of a compression algorithm, spewing out cliched phrases that don't add information. It's exhausting to read, and it lacks the interesting noise of a human response.
Wow.
I am pretty insensitive to AI writing. I have never commented before about something sounding like AI, because mostly I don't notice. But this was so over the top that I spent the whole article trying to decide whether it was an intentional parody of AI writing style.
This article's language is not en-US. It's not en-BR. It's en-SLOP.
Yes, that was my clumsy attempt at AI parody. Here's another: this article doesn't just have AI tells. It is AI tells.
Every sentence is saturated with AI style. Perhaps the author so AI-indoctrinated that they can't see this? It doesn't read as even vaguely plausible human writing. Which is mightily ironic given the thesis of "AI generated stuff is just fine, m'kay?" The writing style does more to defeat its conclusion than the analysis itself.
As for the substance of the analysis, it seems pretty good to me but I see some flaws that weaken it a bit.
The presence of "The Outlier Nobody Noticed" proves nothing and deserves no more than a passing mention. A random release introduced way more bugs than the Claude-containing releases. That provides evidence that Claude doesn't introduce more bugs only if your hypothesis is a very naive "AI is the only thing that can ever increase bug introduction rates."
The whole analysis has very limited data. It's necessarily based off a single pair of releases at the very end of the chronological timeline. You would never be able to reject a null hypothesis based only on that, so it's even less sound to present it as proving the null hypothesis. (By the same token, it would be incorrect for critics to claim that it proves their point. Did anyone claim this, though? The heated complaints seemed more based on priors about AI code.)
"The critics' claim is a simple comparison: did the rate go up?" That's reductive. For one, these releases are known to be in reaction to a flood of (AI-discovered!) security reports, which is a novel situation and in fact is a huge confound to anyone arguing about what those two releases mean -- they're both heavily AI-written, but in response to an unusual situation. When the samples are only drawn from a distinct scenario, statistic analysis can only speak to the quality of code in that scenario.
Also, another reasonable hypothesis could be: AI-written code has bugs of a different flavor that bothers users more. It's optimized for passing tests and convincing people and AIs that security holes are closed, which means other considerations like preserving functionality can more easily be regressed as compared to if humans were doing it. (If true, it still doesn't support the claim that depending on AI code is a catastrophe, fwiw.)
I'm not arguing the conclusion is wrong. I'm saying the analysis proves far less than it claims to. As for whether it's a debacle for rsync to become dependent on AI code generation, I think that's a reasonable debate to have but it's not going to be resolved this reductively.
I am pretty insensitive to AI writing. I have never commented before about something sounding like AI, because mostly I don't notice. But this was so over the top that I spent the whole article trying to decide whether it was an intentional parody of AI writing style.
This article's language is not en-US. It's not en-BR. It's en-SLOP.
Yes, that was my clumsy attempt at AI parody. Here's another: this article doesn't just have AI tells. It is AI tells.
Every sentence is saturated with AI style. Perhaps the author so AI-indoctrinated that they can't see this? It doesn't read as even vaguely plausible human writing. Which is mightily ironic given the thesis of "AI generated stuff is just fine, m'kay?" The writing style does more to defeat its conclusion than the analysis itself.
As for the substance of the analysis, it seems pretty good to me but I see some flaws that weaken it a bit.
The presence of "The Outlier Nobody Noticed" proves nothing and deserves no more than a passing mention. A random release introduced way more bugs than the Claude-containing releases. That provides evidence that Claude doesn't introduce more bugs only if your hypothesis is a very naive "AI is the only thing that can ever increase bug introduction rates."
The whole analysis has very limited data. It's necessarily based off a single pair of releases at the very end of the chronological timeline. You would never be able to reject a null hypothesis based only on that, so it's even less sound to present it as proving the null hypothesis. (By the same token, it would be incorrect for critics to claim that it proves their point. Did anyone claim this, though? The heated complaints seemed more based on priors about AI code.)
"The critics' claim is a simple comparison: did the rate go up?" That's reductive. For one, these releases are known to be in reaction to a flood of (AI-discovered!) security reports, which is a novel situation and in fact is a huge confound to anyone arguing about what those two releases mean -- they're both heavily AI-written, but in response to an unusual situation. When the samples are only drawn from a distinct scenario, statistic analysis can only speak to the quality of code in that scenario.
Also, another reasonable hypothesis could be: AI-written code has bugs of a different flavor that bothers users more. It's optimized for passing tests and convincing people and AIs that security holes are closed, which means other considerations like preserving functionality can more easily be regressed as compared to if humans were doing it. (If true, it still doesn't support the claim that depending on AI code is a catastrophe, fwiw.)
I'm not arguing the conclusion is wrong. I'm saying the analysis proves far less than it claims to. As for whether it's a debacle for rsync to become dependent on AI code generation, I think that's a reasonable debate to have but it's not going to be resolved this reductively.
> The presence of "The Outlier Nobody Noticed" proves nothing and deserves no more than a passing mention. A random release introduced way more bugs than the Claude-containing releases. That provides evidence that Claude doesn't introduce more bugs only if your hypothesis is a very naive "AI is the only thing that can ever increase bug introduction rates."
It does not statistically prove anything, but as I thought I made extremely clear in the card where I discuss it, the point of bringing it up is different: to prove the hypocrisy of the anti-AI crowd.
> By the same token, it would be incorrect for critics to claim that it proves their point. Did anyone claim this, though? The heated complaints seemed more based on priors about AI code.
The entire outrage is because people noticed what they thought was an unusual number of bugs and/or regressions in the release, saw it had Claude in it, and assumed a causal link, not just "priors about AI code."
> You would never be able to reject a null hypothesis based only on that, so it's even less sound to present it as proving the null hypothesis.
The point I'm trying to make is that there is no evidence, based on these two releases, to think Claude made anything worse, whatsoever, and so the outrage is unfounded. This doesn't require me to prove Claude didn't cause any problems. If I ever made the latter claim, I should clean that up.
> It's optimized for passing tests and convincing people and AIs that security holes are closed, which means other considerations like preserving functionality can more easily be regressed as compared to if humans were doing it.
Tridge actually explicitly says he made that tradeoff on purpose, not the AI.
> Every sentence is saturated with AI style. Perhaps the author so AI-indoctrinated that they can't see this? It doesn't read as even vaguely plausible human writing. Which is mightily ironic given the thesis of "AI generated stuff is just fine, m'kay?" The writing style does more to defeat its conclusion than the analysis itself.
I've since rewritten nearly 100% of the prose in the analysis with my own, more inflammatory and verbose style. I also intentionally left in my natural mispellings and typos, to prove it was me.
It does not statistically prove anything, but as I thought I made extremely clear in the card where I discuss it, the point of bringing it up is different: to prove the hypocrisy of the anti-AI crowd.
> By the same token, it would be incorrect for critics to claim that it proves their point. Did anyone claim this, though? The heated complaints seemed more based on priors about AI code.
The entire outrage is because people noticed what they thought was an unusual number of bugs and/or regressions in the release, saw it had Claude in it, and assumed a causal link, not just "priors about AI code."
> You would never be able to reject a null hypothesis based only on that, so it's even less sound to present it as proving the null hypothesis.
The point I'm trying to make is that there is no evidence, based on these two releases, to think Claude made anything worse, whatsoever, and so the outrage is unfounded. This doesn't require me to prove Claude didn't cause any problems. If I ever made the latter claim, I should clean that up.
> It's optimized for passing tests and convincing people and AIs that security holes are closed, which means other considerations like preserving functionality can more easily be regressed as compared to if humans were doing it.
Tridge actually explicitly says he made that tradeoff on purpose, not the AI.
> Every sentence is saturated with AI style. Perhaps the author so AI-indoctrinated that they can't see this? It doesn't read as even vaguely plausible human writing. Which is mightily ironic given the thesis of "AI generated stuff is just fine, m'kay?" The writing style does more to defeat its conclusion than the analysis itself.
I've since rewritten nearly 100% of the prose in the analysis with my own, more inflammatory and verbose style. I also intentionally left in my natural mispellings and typos, to prove it was me.
My post wasn't written in a way to make friends, but:
> I've since rewritten nearly 100% of the prose in the analysis with my own, more inflammatory and verbose style. I also intentionally left in my natural mispellings and typos, to prove it was me.
Thank you thank you thank you. I would love to be able to describe how hard it was for me to think about the actual evidence you're presenting when reading about it through the AI writing, but I suspect it's one of those things where it bothers you or it doesn't. If you'd like to empathize, maybe I'll give it one try: imagine an otherwise solid PhD thesis written in crayon. The facts and evidence and reasoning are unaffected, but it's just so hard to take it seriously.
Anyway, with the rewrite I don't have to battle my kneejerk reactivity nearly as much.
I'm no expert like she is, but based on what I know, I agree with your wife on the statistics. That style of analysis is going to be the best you can do with the data available. It's an accepted way to stretch data without being too dependent on an assumed distribution. It's a good analysis. I still don't come away with the conclusion that concerns about AI code maintenance are necessarily overblown, but that's fine. I think your analysis project is a very solid contribution, and it's a hell of a lot more evidence-based than the rants people were posting.
> I've since rewritten nearly 100% of the prose in the analysis with my own, more inflammatory and verbose style. I also intentionally left in my natural mispellings and typos, to prove it was me.
Thank you thank you thank you. I would love to be able to describe how hard it was for me to think about the actual evidence you're presenting when reading about it through the AI writing, but I suspect it's one of those things where it bothers you or it doesn't. If you'd like to empathize, maybe I'll give it one try: imagine an otherwise solid PhD thesis written in crayon. The facts and evidence and reasoning are unaffected, but it's just so hard to take it seriously.
Anyway, with the rewrite I don't have to battle my kneejerk reactivity nearly as much.
I'm no expert like she is, but based on what I know, I agree with your wife on the statistics. That style of analysis is going to be the best you can do with the data available. It's an accepted way to stretch data without being too dependent on an assumed distribution. It's a good analysis. I still don't come away with the conclusion that concerns about AI code maintenance are necessarily overblown, but that's fine. I think your analysis project is a very solid contribution, and it's a hell of a lot more evidence-based than the rants people were posting.
This article is unfortunately unreadable because all of the prose is unfiltered LLM slop.
Tl;dr:
Yes, it did. Here is some math showing that you shouldn’t care about that.
Yes, it did. Here is some math showing that you shouldn’t care about that.
In what way did it create more bugs? It literally doesn't show up in the data. What are you talking about?
The only reason why people are talking about this is because of the bugs in the code that the chat bot generated OP. People updated to a version of rsync that didn’t work right, the one with all the bot commits in it. This blog post is about how claude didn’t create more bugs than usual if you think about it in one very specific way, not that it didn’t create more bugs at all.
It is like if your neighbor opens your door and a dog walks in, there’s no point in doing some weird analysis about all the times you yourself have let a dog walk in. He still did that.
It is like if your neighbor opens your door and a dog walks in, there’s no point in doing some weird analysis about all the times you yourself have let a dog walk in. He still did that.
That doesn't make any sense, what?
I am not sure what to tell you here, because somebody literally posted the code of a bug that Claude inserted here
https://news.ycombinator.com/item?id=48419197
And your response to someone pointing out that sloppy, buggy code that Claude introduced, was to just quote Tridge (which does not in any way refute the fact that you’re looking at a bug that Claude introduced to the code)
https://news.ycombinator.com/item?id=48419621
I’m not entirely sure what the purpose of this project is (maybe to “prove” Tridge’s opinions about LLMs and human intelligence that he made in the linked blog post to be right?), but it appears as though you are ignoring irrefutably true observations. You just asserted that “the data” doesn’t show Claude introducing any bugs (which is a bizarre claim) after previously responding to a documented bug with a… deferral? Do bugs not count if you can find a vague excuse for it?
There is nothing in the blog post that is evidence that Claude didn’t introduce bugs. It is a thought experiment that uses “increase bugs” and “increase bugs more than a given arbitrary statical amount that I selected” as interchangeable statements.
https://news.ycombinator.com/item?id=48419197
And your response to someone pointing out that sloppy, buggy code that Claude introduced, was to just quote Tridge (which does not in any way refute the fact that you’re looking at a bug that Claude introduced to the code)
https://news.ycombinator.com/item?id=48419621
I’m not entirely sure what the purpose of this project is (maybe to “prove” Tridge’s opinions about LLMs and human intelligence that he made in the linked blog post to be right?), but it appears as though you are ignoring irrefutably true observations. You just asserted that “the data” doesn’t show Claude introducing any bugs (which is a bizarre claim) after previously responding to a documented bug with a… deferral? Do bugs not count if you can find a vague excuse for it?
There is nothing in the blog post that is evidence that Claude didn’t introduce bugs. It is a thought experiment that uses “increase bugs” and “increase bugs more than a given arbitrary statical amount that I selected” as interchangeable statements.
I'm saying Claude didn't introduce any more bugs than the human maintainers of rsync had in the past, not that it introduced no bugs at all. That "arbitrary amount" is the historical amount of bugs. That's why I'm confused. You're completely missing my point and talking past me. Yes, Claude may have introduced a bug. That doesn't change that it doesn't really matter.
Additionally, I quoted Tridge in response to a comment about an increase in changes to rsync, not in response to the person pointing at one bug Claude introduced. If you actually looked at the thread, you'd see that. I didn't deny the Claude introduced bug at all.
Additionally, I quoted Tridge in response to a comment about an increase in changes to rsync, not in response to the person pointing at one bug Claude introduced. If you actually looked at the thread, you'd see that. I didn't deny the Claude introduced bug at all.
Claude is just a tool ? The developers who merged that code and didn't properly test increased the bugs.
"Did cars increase traveling deaths?"
"Cars are just a tool. The drivers who piloted the vehicles and weren't careful enough [are responsible for the deaths.]"
"Cars are just a tool. The drivers who piloted the vehicles and weren't careful enough [are responsible for the deaths.]"
If something's a bad tool that misleads people into doing bad work, it would be good to know that.
Please read the article.
The unsolicited security reports are the issue.
The unsolicited security reports are the issue.
This tool is claimed to be able to find and fix bugs.
Feels like something a bad (and potentially dangerous) tool would say.
Is there a non vibe coded fork of rsync?
Yes: https://news.ycombinator.com/item?id=48390931
So far it reintroduced several security issues and replaced the README.md.
So far it reintroduced several security issues and replaced the README.md.
There is openrsync, which the OpenBSD re-implementation of rsync
It's not a fork, but it's 8 years old, and is already shipped by default in OpenBSD and macOS.
It's not a fork, but it's 8 years old, and is already shipped by default in OpenBSD and macOS.
To quote Tridge:
> As to all the people saying “I’m going to package openrsync for platform XXX and we’ll use that!”. I find that rather amusing. If you do decide to go down that path I’d suggest you try the new rsync test suite on openrsync if you can stomach something that an AI has helped write. I tried it today and openrsync currently fails 85 of 98 tests, so I’m sure it won’t take you long to get it up to speed. You run it like this “./runtests.py — rsync-bin=../openrsync/openrsync — use-tcp”. Admittedly a lot of the failures are just features openrsync doesn’t have, but still, it’s not a great result.
> As to all the people saying “I’m going to package openrsync for platform XXX and we’ll use that!”. I find that rather amusing. If you do decide to go down that path I’d suggest you try the new rsync test suite on openrsync if you can stomach something that an AI has helped write. I tried it today and openrsync currently fails 85 of 98 tests, so I’m sure it won’t take you long to get it up to speed. You run it like this “./runtests.py — rsync-bin=../openrsync/openrsync — use-tcp”. Admittedly a lot of the failures are just features openrsync doesn’t have, but still, it’s not a great result.
I have already been using openrsync even before the recent AI drama.
Just like I have been using doas for several years.
All I need is `rsync -urvP` and I suspect the majority of users don't need the advanced features either.
The smaller code base also means less bugs and vulnerabilities. As an example doas is ~1k lines vs 160k for sudo. That surely means a smaller attack surface. The same is true for openrsync and rsync at approximately 18k vs 57k lines.
Just like I have been using doas for several years.
All I need is `rsync -urvP` and I suspect the majority of users don't need the advanced features either.
The smaller code base also means less bugs and vulnerabilities. As an example doas is ~1k lines vs 160k for sudo. That surely means a smaller attack surface. The same is true for openrsync and rsync at approximately 18k vs 57k lines.
Yeap and still has bugs (reports say that the base system bin in 7.9 doesn't work properly): https://marc.info/?l=openbsd-misc&m=178070682043839&w=2
And here: https://marc.info/?l=openbsd-tech&w=2&r=1&s=rsync&q=b you can quite a few other bugs...
I use OpenBSD for routers and love it :-) but it is software, hence it has bugs .-
And here: https://marc.info/?l=openbsd-tech&w=2&r=1&s=rsync&q=b you can quite a few other bugs...
I use OpenBSD for routers and love it :-) but it is software, hence it has bugs .-
It's also not feature-equivalent to rsync. But if it meets your needs, it's an option.
Ok.
$ apt-cache policy rsync | grep Installed
Installed: 3.4.1+ds1-7ubuntu0.2
$ sudo apt-mark hold rsync
rsync set on hold.That version has security fixes from the same day as the latest rsync release: https://ubuntu.com/security/notices/USN-8283-1
As usual, Ubuntu backported fixes and didn't upgrade to a new version. Whether or not they also backported regressions in edge cases that afflict the latest rsync, I don't know. Pinning the Ubuntu package may prevent getting further regressions, but is preventing you getting any future such backported security fixes.
As usual, Ubuntu backported fixes and didn't upgrade to a new version. Whether or not they also backported regressions in edge cases that afflict the latest rsync, I don't know. Pinning the Ubuntu package may prevent getting further regressions, but is preventing you getting any future such backported security fixes.
The Ubuntu backport did include regressions that an Ubuntu update a couple of days later addresses: https://ubuntu.com/security/notices/USN-8349-2
Did you face any actual bugs or regressions? Or are you doing this just because of the bandwagon that's going around right now? Because until you can actually present an argument for why this release is worse than any of the others, which is precisely the subject of my post, then this is not an argument against my post at all. This is just a self-referential appeal to authority.
> Did you face any actual bugs or regressions?
This is a terrible argument; I didn't need to have had secrets exfiltrated before applying row-hammer mitigations. If rsync is the cornerstone of my backup strategy, and has been for years, I need to trust that on its correctness, and for it to not lose my data. If I wait until I "face any actual bugs or regressions" - that will be far too late.
Stability is another issue not discussed. If the error rate holds steady, but number of significant PRs merged per release goes up from 5 to 200, that would be huge net-negative for my use case.
This is a terrible argument; I didn't need to have had secrets exfiltrated before applying row-hammer mitigations. If rsync is the cornerstone of my backup strategy, and has been for years, I need to trust that on its correctness, and for it to not lose my data. If I wait until I "face any actual bugs or regressions" - that will be far too late.
Stability is another issue not discussed. If the error rate holds steady, but number of significant PRs merged per release goes up from 5 to 200, that would be huge net-negative for my use case.
Nah, I skimmed TFA but then I went into the linked GH issues thread, and that's the one that scared me a bit. I just want to hold it for a while and not run into some of the things I'm reading since I'm on the latest ubuntu. Just a precaution.
I didn't have the time to actually think about any "arguments" at all tbh it's just a knee jerk reaction as I get ready to log off for the weekend. Not actually looking to argument for or against your post at all lol.
I didn't have the time to actually think about any "arguments" at all tbh it's just a knee jerk reaction as I get ready to log off for the weekend. Not actually looking to argument for or against your post at all lol.
I'm amazed that this is still being discussed.
It's open source, no one is forcing you to use it.
If you don't trust the newer versions; use the old versions.
If you no longer like the maintainer because of reasons, fork it/start your own.
It's not that hard.
Storm in a teacup.
It's open source, no one is forcing you to use it.
If you don't trust the newer versions; use the old versions.
If you no longer like the maintainer because of reasons, fork it/start your own.
It's not that hard.
Storm in a teacup.
I think there's evolution at play here - if you dislike AI enough to opt out of using any ai-generated code, you will likely suffer. I think there's definitely a conversation to be had about whether to disclose AI use or not but that's a separate issue if you assume that everyone is using it in some respect.
https://medium.com/@tridge60/rsync-and-outrage-d9849599e5a0
(Disclosure: while I haven't talked with him in years, Tridge was my colleague and mentor for many years. I feel it is worth considering his view before joining a crusade)