I think you first need to make an assumption of good faith on the submitter’s part. Yes, someone actively hostile to a repo and trying to sneak in changes will be successful, they can just prompt around the challenge.
I think the bigger picture is there won’t be one catch-all solution and we’ll need to embrace the Swiss cheese model from the world of aviation safety, this is just a suggestion for one layer.
It raises the stakes though. Getting challenged for an AI slop PR isn’t great, but ok, try and redeem yourself. Getting caught trying to cheat that challenge, you might as well just close down your account, like what is the point of even spending tokens to do this? These slop PRs are just people trying to pad their GitHub profiles.
I’m teasing a bit, yes your point is well made, you could just prompt around it. I think the bigger picture is there isn’t going to be some magical panacea that fixes this, we’ll just have to accept some sort of Swiss cheese model, like in the world of aviation safety. This “test” could be just be another layer in the stack.
The foundation points out something that has always been true, but AI has really brought to the forefront, that any contributor, including AI, could potentially not be relied on to maintain this patch in the future.
This is the core of the issue, not that someone uses AI, but that it’s just one of many smells a patch can have that indicates someone doesn’t understand what they’re submitting. You could be breaking variable naming conventions, changing APIs you shouldn’t, making amateur language mistakes, all indicate that yes, maybe the patch does work, but that there are other good reasons to reject it.
A way around this might be to mark a PR as rejected because of AI and then ask the author to point out some part of it they’re particularly proud of and explain in their own words, not a wall of AI text, what this does and why they like it. Just something where the author has to show that they have something an AI can’t, namely taste and an opinion.
Literally installing Arch right now as I read this. Same deal, been using windows as my daily driver all my life but have been running Linux servers since late 90s (and did daily drive RedHat back in the day).
My work env was just VS Code + WSL and I realized the most pain points came from using Explorer and trying to admin the machine with the fractured landscape of sys admin tools. For me it became very obvious that windows is only going to get more bloated and less “my” machine going forward, why stay on this platform if I’m already spending most of my day in a Linux environment (that I’m already familiar with).
Last "job" was a startup with two other people and it was so great having others to bounce ideas off of and share the ups and downs. Building something together is a fundamental human joy IMO. We sold that and I've been working solo for the last 4 years.
It's lonely, and these have helped me.
- Making a point to physically get out of the house and ideally meet up with someone for a coffee or lunch or something.
- Being part of an active group chat.
- Podcasts
- Working out of a library or bustling cafe, just physically being around other people.
- Working out and maintaining a good wake/sleep cycle.
I also have some great hobbies (flying, scuba, rock climbing) with great communities that keep me socialized.
For anyone about to quit their job and try the solo dev thing, I always say mental well being has to be a "remembered priority". You're never going to have work/life balance like you did at an office job and the first few years are incredibly tough, so you need to plan for that. Marathon, not a race etc etc.
I think we'll start to see AI as any other tool that can atrophy your natural faculties. You can use a wheelchair to get everywhere, but your leg muscles will start to wither, but a wheeled vehicle for going longer distances is a genuinely useful tool.
Reaching for AI as a _substitute_ for thinking is bad, but reaching for it as a tool to assist thinking is good; you just need to be honest about whether it's your brain in the driver's seat or the chat bot.
Design is one of those things that succeeds or fails in subtlety and both are difficult to quantify and back propagate through any sort of process, let alone training a model. The same way we figured out that the microwave can make approximations to good food quickly, so too shall we see that AI can do the same with tasks that rely heavily on a connection to people's aesthetics.
I've been using AI like this as well. The code-complete / 'randomly pop up a block of code while typing' feature was cool for a bit but soon became annoying. I just use it to generate a block of boilerplate code or to ask it questions, I do 90% of the 'typing the code' bit myself, but that's not where most programmers time is spent.
I think the bigger picture is there won’t be one catch-all solution and we’ll need to embrace the Swiss cheese model from the world of aviation safety, this is just a suggestion for one layer.
It raises the stakes though. Getting challenged for an AI slop PR isn’t great, but ok, try and redeem yourself. Getting caught trying to cheat that challenge, you might as well just close down your account, like what is the point of even spending tokens to do this? These slop PRs are just people trying to pad their GitHub profiles.