Quillx is an open standard for disclosing AI involvement in software projects(github.com)
github.com
Quillx is an open standard for disclosing AI involvement in software projects
https://github.com/QAInsights/AIx
36 comments
I would think the term 'vibe coded', 'vibed', '100% vibes', etc would be far more appropriate and well known, than 'lorem ipsum' when it comes to generating code without reviewing the output.
If I saw that badge on someones github I would think it had something to do with lorem ipsum text generation, rather than anything to do with AI.
If I saw that badge on someones github I would think it had something to do with lorem ipsum text generation, rather than anything to do with AI.
Nice idea, but the labels are a bit too opinionated for me.
Literally all my code has been ”ghostwritten” for the past 18 months. Does not sound like something enterprise customers would like to hear and try to understand what it means.
Literally all my code has been ”ghostwritten” for the past 18 months. Does not sound like something enterprise customers would like to hear and try to understand what it means.
Do you know all the code you're shipping? If no, as a customer that is a thing I'd certainly like to be informed of.
I think we really need hard liability for software engineering.
I think we really need hard liability for software engineering.
We should assume projects have AI/LLM development assistance unless stated otherwise.
You may have noticed the absolutely vast array of AI development tools and assistants and IDEs and integrations - this is a reasonable indicator that developers are actually doing AI/LLM development.
You may have noticed the absolutely vast array of AI development tools and assistants and IDEs and integrations - this is a reasonable indicator that developers are actually doing AI/LLM development.
In academia there has been a widespread practice to simply include a sentence about how AI has used in articles. It's simple and it works well.
A little ironic that the README, SPEC.md and the poster's comment here all smell of LLM writing!
They put a 3/5 for themselves, atleast they are honest.
Yes, that is why I have put 3/5 :)
How can you expect adoption of this scale if the AI side is so obviously negative? You might as well label the full AI option “I drown kittens” and go write a long post about how AI users won’t engage with your AI usage disclosure initiative in good faith.
To have any chance of adoption you have to be at least a little strategic. You may think AI is pure evil but you have to make some concessions to AI users to incentivise participation. Try making it sound neutral through out the spectrum, use neutral colour scheme. Yes, you’re not telegraphing your position on AI so obviously any more but you might get some useful information out of others.
To have any chance of adoption you have to be at least a little strategic. You may think AI is pure evil but you have to make some concessions to AI users to incentivise participation. Try making it sound neutral through out the spectrum, use neutral colour scheme. Yes, you’re not telegraphing your position on AI so obviously any more but you might get some useful information out of others.
The labels are not transparent - if you see this badge on GitHub readme, you won't be able to tell that it is about AI usage. I also don't find those labels to be particularly useful, when you are proposing an actual standard, you have to sit down and design it carefully and thoroughly, which I don't believe happened here. So, it looks cool, but I don't think it's super useful
(1) Why?
(2) The code I write with AI doesn’t fit on the scale.
(2) The code I write with AI doesn’t fit on the scale.
Ad 1: If my software vendor doesn't even know their own code, I'd like to know. This may be an issue of liability.
Considering the more AI you use the more red it is along with demeaning language for the scales I assume it is mainly for anti-AI people to virtue signal about not using it.
It looks a bit like that, but they gave their own repo a 3/5 rating and it's full of obvious LLMisms, so I think they're not totally anti-AI and are trying to be evenhanded.
To me, the metaphor doesn't really work, especially at level 5. Lorum Ipsum is literal placeholder text which is basically the same everywhere it's used; I don't see what that has to do with vibe code. (Also the verse/prose thing seems pretty wanky to me, but I admit that's just a matter of taste.)
To me, the metaphor doesn't really work, especially at level 5. Lorum Ipsum is literal placeholder text which is basically the same everywhere it's used; I don't see what that has to do with vibe code. (Also the verse/prose thing seems pretty wanky to me, but I admit that's just a matter of taste.)
Given the reality that there are a lot of people who [fairly or unfairly] judge anything that uses "AI" in a decisively negative way, what possible advantage is there in giving people a reason to dismiss your project without evaluating it on its own merits?
Is honesty an important quality to you? Does lying by omission concern you for the people and projects you choose to interact with?
I'm with you on honesty, and I've certainly seen people tacitly trying to pass off AI outputs as human written. But I think we've reached a point where, in lots of contexts, we can't reasonably assume human authorship by default any more. (We can reasonably want it and push for it! I just mean we can't literally expect it.) So even when we would prefer openness, I think 'lying by omission' is too harsh a characterisation for people who choose not to declare AI authorship but don't actively try to cover it up.
The replies to my comment kind of say everything that needs to be said about what I actually said; people can't even put aside their overt hostility to all things AI long enough to acknowledge that it makes no [good] sense to draw specific attention to LLM usage.
Ironically, it's precisely because of those folks that nobody with half a brain would acknowledge that they use LLMs to bring some or all of their ideas to life.
You can't demand that all people should disclose their bellybutton rings AND openly salivate for cancelling people with bellybutton rings.
Ironically, it's precisely because of those folks that nobody with half a brain would acknowledge that they use LLMs to bring some or all of their ideas to life.
You can't demand that all people should disclose their bellybutton rings AND openly salivate for cancelling people with bellybutton rings.
Honesty is the whole problem with ideas like this. If you're the kind of deluded idiot that considers LLM-generated crap "your code", stating exactly how little you had to do with it is not in your advantage. Far easier to maintain the lie.
If you review the code generated by the LLM in response to your specification and you own the responsibility for what that code will do in the real world... congratulations, it is now Your Code in every way that actually matters.
In a few years, your position will be revealed as getting angry for someone that called you with your number stored on their phone - instead of memorizing it and dialing the numbers manually - and didn't start the conversation by admitting that they haven't memorized your number.
In a few years, your position will be revealed as getting angry for someone that called you with your number stored on their phone - instead of memorizing it and dialing the numbers manually - and didn't start the conversation by admitting that they haven't memorized your number.
Nobody owes you any transparency about the way they develop their software.
They sure don't, but often insight into/alignment with the story and development process makes all the difference for which projects people choose to contribute to.
They do if they want me to use it.
AIx is an open standard for disclosing AI involvement in software projects - expressed through the language of authorship. Not a judgment. Just transparency.
Just one tiny issue:
"AIX®" is also a registered trademark of International Business Machines Corporation (IBM) and used for the AIX® operating system that is still in use today.
I would be careful to use that name.
"AIX®" is also a registered trademark of International Business Machines Corporation (IBM) and used for the AIX® operating system that is still in use today.
I would be careful to use that name.
Thanks. renamed it to https://github.com/QAInsights/Quillx
Color code is the other way around.
Red should meant manual human review without automated tools nor AI.
Green for proper AI review and tests verifying the expected input/outputs.
Red should meant manual human review without automated tools nor AI.
Green for proper AI review and tests verifying the expected input/outputs.
> Every line deliberate. Crafted like poetry. Human-authored entirely.
This is perhaps overly generous to pure-human authorship. These days, when I write code I like to think I know what it does. I still wouldn't call most of it "crafted like poetry". When I was just learning though, I wrote plenty of code 100% without AI (in fairness, it didn't exist) that I had little understanding of, and it was only "deliberate" in that I deliberately cajoled it into passing the tests.
Or put differently: don't conflate human authorship with quality; people can write garbage without needing AI help.
This is perhaps overly generous to pure-human authorship. These days, when I write code I like to think I know what it does. I still wouldn't call most of it "crafted like poetry". When I was just learning though, I wrote plenty of code 100% without AI (in fairness, it didn't exist) that I had little understanding of, and it was only "deliberate" in that I deliberately cajoled it into passing the tests.
Or put differently: don't conflate human authorship with quality; people can write garbage without needing AI help.
> don't conflate human authorship with quality
If you're thinking "crafted like poetry" implies any kind of existential "quality", I'd like to introduce you to William McGonagall[0] and you will swiftly and powerfully be disabused of any "poetry -> quality" confusions.
[0] https://en.wikipedia.org/wiki/William_McGonagall
If you're thinking "crafted like poetry" implies any kind of existential "quality", I'd like to introduce you to William McGonagall[0] and you will swiftly and powerfully be disabused of any "poetry -> quality" confusions.
[0] https://en.wikipedia.org/wiki/William_McGonagall
Regardless of whether poetry actually has anything to do with quality, I think it's reasonable to think that the author intended "Crafted like poetry" as a positive description.
> the author intended "Crafted like poetry" as a positive description.
Yes and it's a fine way to describe a process positively! Crafting something implies care, thought, etc.
It just doesn't have anything to say about the quality of the resultant output.
Yes and it's a fine way to describe a process positively! Crafting something implies care, thought, etc.
It just doesn't have anything to say about the quality of the resultant output.
Indeed. I like to call this form of code organic slop, and it predates the LLM era.
This is very similar to a project I created https://github.com/Entrpi/autonomy-golf and have been using as a gamified development process on active projects.
The key insight was to not just handwave or guess at how much is automated, but make evaluation and review part of the continuous development loop. I first implemented in https://github.com/Entrpi/autoresearch-everywhere where I used it to deliberately automate more, in the spirit of Karpathy's upstream (and to very good effect. I have some of the best autoresearch results anywhere, and the platform is far more robust than it started).
The key insight was to not just handwave or guess at how much is automated, but make evaluation and review part of the continuous development loop. I first implemented in https://github.com/Entrpi/autoresearch-everywhere where I used it to deliberately automate more, in the spirit of Karpathy's upstream (and to very good effect. I have some of the best autoresearch results anywhere, and the platform is far more robust than it started).
I like these efforts to neatly categorise the extent of AI usage in a project. I do think they need some kind of neutrally worded classification but this and the original post are fine attempts at this emerging niche. It's important to some of us and I look forward to what ends being adopted.
Neat idea. I like the five point scale
I find this idea that humans all of a sudden write beautiful code very funny. Most code produced by hand is dirty and filled with ugly hacks. The argument might work against AI art but falls flat for programming.