That depends on the model and the toolkit it uses. In my experience from using Claude Code (Max, Opus 4.5+) intensely for the past six months, I maybe had 3 instances where the implementation broke functionally. And all of these breaking changes were resolved by Claude.
Obviously, this won't apply to every context: I work primarily with well-known langs (e.g., Python, JS), small to medium codebases (<500k LoC, for sure), and relatively few co-developers.
I mean, not sure if everyone wants that for their project, and there will surely be plenty of trade-offs.
But it would be a very good compromise: You (the maintainer) get only human-generated PRs in the canonical project, and they (pro-AI contributors) get a lower-threshold sandbox to play with. Best case scenario, you cherrypick the pre-filtered golden nuggets to bring back to the canonical project.
Isn't that what I'm already doing? I assume that the author believes that he's right often enough to argue his position(s), but feels dejected without being proven wrong at the end of the argument.
But there's another important point here: the answer to the "am I really right?" question isn't always clear at the start of every argument.
Unless you believe there's room for (dis)proving your position, or getting some nuance on a topic [1], it's not a debate or an argument - it's a lecture. And lectures depend on other social dynamics which don't apply here.
[1] For example, maybe there are other reasons behind the position that the person can't express easily, or maybe you're actually arguing about different things.
Definite agree on having human-based redundant systems.
But I think the point the parent commenter made is: if there's not a functional difference in the result (i.e., job satisfies the definition of done), it doesn't matter if the AI generated the code or did the diagnosis.
But I think it's also fair to say that the process matters, even if the result is the same. If something exists for our benefit (e.g., there's no real alternative to learning-by-doing, and people need to know stuff for safety/security reasons), and we're fine with the trade-off, there's no reason to just give up the process to the AI.
Other than the obvious, self-reflective question that the author doesn't pose - "what if I'm the one who's wrong?" - I think it's worth arguing if the conditions are right.
Because I also like being correct, a debate to me has become something of a game where (ideally) we both win in both end scenarios: either my thinking was correct, and now I verified/validated it, and got you to think differently; or my thinking was incorrect, and you corrected it for me (or helped me get there).
However, I implicitly figured out that there are some qualifiers to actually getting the benefits:
- Can I be, and remain, polite and reflective? If not, my personality or knee-jerk responses will always get in the way of an argument's benefits.
- Is the subject sensitive to the person for whatever reason? If yes, any argument inadvertently becomes a signal of a person's worth.
- Are we in a competitive setting (e.g., corporate meeting, or larger social group)? If yes, any argument inadvertently becomes a social status competition.
- Do I know how to stick to the issue (instead of moving goalposts), and stop when the debate gets overwhelming (too long, too much difference)? If not, I'll overstep the boundary after which it isn't mutually beneficial anymore.
These are not easy to figure out, and sure, maybe stop arguing with most people if the conditions aren't right.
But unless you stop communicating altogether, I don't see how you can stop arguing with people in general.
Well, unless you needed those million copies for whatever reason, that is an example of spam or denial-of-service, regardless of how it's generated.
And I'm not disagreeing - it is hard to anticipate what needs verifying, regardless if it's functional or non-functional.
But if it's not a spam submission, you could probably design tests or static/dynamic analysis tools that can verify those million copies much faster than manual reviews.
Good point. Even if you submit small, verifiable and readable changes, you can still overload the review process by submitting too many of them (e.g., 100s of PRs).
But I'd argue that some projects [1] could benefit from the speed (and sometimes, quality) of AI code generation without filtering by something that's difficult to identify (i.e., is it truly human-generated).
One way could be to constrain the size of each commit and PR, and invest more heavily into the review process (e.g., tests, static/dynamic analysis, sandbox deployments), so even if you get 100s of contributions, you can knock each out quickly.
Obviously, easier said than done. And at that point, you may as well use the AI to make the commits yourself, instead of relying on community contributions.
[1] Of course, this is only the case if the project's only purpose is to be a tool, and not also an educational reason for humans to learn how to code - in which case, it makes sense to invest more into identifying the "cheaters".
There are also plenty of valid personal reasons for refusing to generate code with AI - learning-by-doing and ownership of the result being the main ones, IMO.
> everybody wants to let AI do their work for them, but nobody wants to be downstream of AI work.
This is also true in my experience. But in my work, I found that I don't care how the code or comment was generated, as long as it doesn't try to overload my brain with irrelevant and obfuscated things, and as long as the person is not pretending that it's true, verified or their own creation (when it isn't).
If you have some requirements/specifications, and the piece of code fits them, then it runs.
Alternatively, if you have some vague idea [1] about what you expect to see/have, and the running code satisfies that idea, then it also runs.
Obviously, there are plenty of non-functional specs (e.g., security, cleanness, readability) that a code should probably fulfill before one finds it acceptable, but these are also not somehow impossible for state-of-the-art models to satisfy.
[1] Vibe, if you prefer, tho I dislike the term. Another related term is eyeball estimation.
Interesting initiative. What are the guiding reasons behind these lists?
I can't think of a functional reason for a no-AI policy: if it runs, it runs, regardless of who or what made it.
Also, even if you avoid AI-generated slop, you can't really avoid the human-generated or human+AI-generated slop that passes your filters.
Still, I can definitely think of good non-functional reasons: provenance, accountability, proof-of-work, encouraging people to write code themselves, empirically tracking how humans develop codebases, etc.
It's a fair policy. Getting those verbose, AI-authored walls of text is very annoying, especially when you're expected to thoroughly review it. It's like a denial-of-service attack on the human mind. I can only imagine how frustrating this can get in open projects that get a lot of contributions.
However, I don't think this will discourage AI-based coding at all. In fact, I see two potential outcomes of these policies:
- Negative: Submitters just add stylistic markers to make their accounts and output seem human-generated. This is like syntactic sugar: the core content and the size of contributions stay the same, but the style gets quirkier.
- Positive: Submitters actually provide to-the-point, no-bullshit commits and comments - "here's the code, here's why I made that change, here are the effects of that change". Even if AI-generated, these small contributions may become much easier to verify & validate. We may even see some standardization in terms of what qualifies as an appropriately sized contribution, what requires more thorough review (e.g., adding unverified dependencies), etc.
I personally wouldn't care if it was AI-generated or not, as long as the content fit the latter category.
Actually, the replication crisis shows how difficult (or underinvested) the process of reviewing is.
Removing this (often very basic) peer review doesn't somehow fix the problem. The solution lies in more thorough reviews and replication studies, not in everyone deciding for themselves.
ArXiv is a good complement to the modern peer review, IMO. As long as someone "vouches" for you, and you adhere to its minimal standards, you're able to post a paper. Other readers can decide whether the paper is worth their attention, and whether the presented ideas or results are valuable.
It's also good that it doesn't gatekeep with the paywalls that you can pretty much only afford by affiliating yourself with a toll-paying institution.
Obviously, there are plenty of flaws with this system:
1. If you're associated with a brand (e.g., Google, MIT) or have a recognizable co-author (e.g., Yann LeCun), you'll get attention and citations no matter what.
2. "Vouching" can also just mean accepting someone's email request without ever having met or known them.
3. It puts the effort on the readers to decide whether each paper is valuable, and particularly scientifically valuable, for which most readers will be unequipped.
4. "Minimal standards" can be gamed by AI-generated submissions.
I'd love to see a synthesis of arXiv, open-access publishing and artifact reviews, like the following:
- Have a number of reviewers on retainer, or design a reward system similar to bug bounties. The reward mechanism probably shouldn't be based on money or allow a winner-takes-all strategy.
- Have a number of badges with respect to the quality and value of the paper. For example: validated by peers (i.e., reviewed by at least 3 peers with minimum borderline accept consensus), valuable (i.e., reviewed by at least 5 peers with a valuable indicator), etc.
- Allow vouched comments on the platform, and moderate for self-promotion, toxicity, etc. Obviously a big ask.
- Improve the "vouching" system, or add badges like "vouched by X people" or "vouched by established scientist".
Hope their new organization will implement some of these improvements.
Unfortunately, I don't think systems thinking alone would help much.
One could present the case in favor of Internet age verification to the nth-order effects, while downplaying the effects in the case against.
So, in addition to presenting the cases with foreseeable effects, we need ways to compare the impact of worst-case scenarios in the two cases, and make a decision or compromise based on that.
While I can understand being skeptical of non-experts' claims that such answers are enough, I don't understand why you call it "psychosis" and not simply naivety or lack of expertise.
At the same time, the new so-called "models" haven't been pure transformer-based LLMs, but entire systems with tools (with access to the Internet), data storage, and the options to trigger additional instances for different tasks.
I'm not always a fan of the squiggles, but I can appreciate the UI pattern. It's definitely one of the more intuitive and recognizable visual markers for "something's wrong with this word".
Obviously, this won't apply to every context: I work primarily with well-known langs (e.g., Python, JS), small to medium codebases (<500k LoC, for sure), and relatively few co-developers.