AI Might Be Lying to Your Boss(williamoconnell.me)
williamoconnell.me
AI Might Be Lying to Your Boss
https://williamoconnell.me/blog/post/ai-ide/
8 comments
Kudos on the analysis, the conclusion is right. I would go further and say even if the metric was completely fair and unbiased it would still not tell you anything useful, and any manager or executive that used it as part of any kind of headcount decision (firings, layoffs, which team to grow, etc) is a moron who probably should be facing tough questions from their own management chain.
AI is a tool, everything it does is attributable to the person who prompted it. Anything else is no different from the long-understood fallacy of counting lines of code.
AI is a tool, everything it does is attributable to the person who prompted it. Anything else is no different from the long-understood fallacy of counting lines of code.
so, what you're saying is that we can definitely expect this metric to be used to make hiring / firing decisions then
PCW clustering around ~85-95% regardless of usage is a measurement bias, not a real signal. In manufacturing, this would fail measurement system analysis by having a larger variation than you're trying to detect. Companies trying to make headcount and copyright decisions on that are doing the AI version of measuring with a broken ruler.
That's re-assuring. It'll be much easier to fake any AI usage metric my management sets.
You mean productivity? Sure — but speed isn’t the same as replacing humans.
Such lying statistics are not at all unexpected, but it is very good to have actual evidence about this practice.
This does make more sense to some of the big claims "95% of our code is ai generated" though. By the logic here a lot of my code would be considered generated for years. ide auto complete on variables, functions, closing brackets, class names etc. Lines changes by linters and formatters to add so much as a comma are not mine anymore. Add in refactoring and my use of ide templates and framework generators to make common files and increasingly small amounts would be considered my written code.