Is 30% of Microsoft's code AI-generated?(idiallo.com)
idiallo.com
Is 30% of Microsoft's code AI-generated?
https://idiallo.com/blog/is-30-percent-of-microsoft-code-ai-generated
16 comments
> Which you could argue is not a problem if it won’t be read by humans anyways anymore in the near future.
It's a problem right now for code that isn't being read by humans.
LLM-backed agents start by writing slightly bad code that's a little too verbose, too careful in error handling, writes too much fallback code, among other common minor LLM-ish flaws. And then it's next turn of the crank sees all that, both as an example but also as code it must maintain, and is slightly more bad in all those ways.
This is why vibing ends up so bad. It keeps producing code that does what you asked for a fairly long time, so you can get a long way vibing. By the time you hit a brick wall it will have been writing very bad code for a long while, and it's not clear that it's easier to fix it than start over and try not to accept any amount of slop.
It's a problem right now for code that isn't being read by humans.
LLM-backed agents start by writing slightly bad code that's a little too verbose, too careful in error handling, writes too much fallback code, among other common minor LLM-ish flaws. And then it's next turn of the crank sees all that, both as an example but also as code it must maintain, and is slightly more bad in all those ways.
This is why vibing ends up so bad. It keeps producing code that does what you asked for a fairly long time, so you can get a long way vibing. By the time you hit a brick wall it will have been writing very bad code for a long while, and it's not clear that it's easier to fix it than start over and try not to accept any amount of slop.
> too careful in error handling, writes too much fallback code
Is it possible that your code goes a little cowboy when it comes to error handling? I don't think I've ever seen code that was too careful when it came to error handling -- but I wrote GPU drivers, so perhaps the expectations were different in that context.
Is it possible that your code goes a little cowboy when it comes to error handling? I don't think I've ever seen code that was too careful when it came to error handling -- but I wrote GPU drivers, so perhaps the expectations were different in that context.
I’ve definitely seen agents add null checks to a computed value in a function, but then not change the return type to be non-null. Later, it adds a null checks at each call site, each with a different error message and/or behavior, but all unreachable.
For bonus points, it implements a redundant version of the same API, and that version can return null, so now the dozen redundant checks are sorta unreachable.
For bonus points, it implements a redundant version of the same API, and that version can return null, so now the dozen redundant checks are sorta unreachable.
When I'm writing web services I think I handle almost every error and I don't have this complaint there.
When I'm writing video games there's lots of code where missing assets or components simply mean the game is misconfigured and won't work and I would like it to loudly and immediately fail. I often like just crashing there. There are better options sometimes too, making a lot of noise but allowing continuation. But LLMs seem to be bad at using those too.
Actually to go back to web services, I do still hate the way I've had LLMs handle errors there too - too often they handle them silently or worse, provide some fallback behavior that masks the error. They just don't write code that looks like it was written by someone with 1) some assumptions about how the code is going to be used 2) some ideas about how likely their assumptions are to be wrong or 3) some opinions about how they'd like to learn their assumptions are wrong if so.
When I'm writing video games there's lots of code where missing assets or components simply mean the game is misconfigured and won't work and I would like it to loudly and immediately fail. I often like just crashing there. There are better options sometimes too, making a lot of noise but allowing continuation. But LLMs seem to be bad at using those too.
Actually to go back to web services, I do still hate the way I've had LLMs handle errors there too - too often they handle them silently or worse, provide some fallback behavior that masks the error. They just don't write code that looks like it was written by someone with 1) some assumptions about how the code is going to be used 2) some ideas about how likely their assumptions are to be wrong or 3) some opinions about how they'd like to learn their assumptions are wrong if so.
Up to 70% or more of statistics in sales calls are exaggerated, waffley or completely made up.
83% of people know that ...
I'm not sure I can trust this, 73.8% of statistics listed in HN comments are made up on the spot.
Even if true it's quite funny because code volume is not a good metric. It's why developer productivity generally should not measured in "LOC produced".
Or to paraphrase Blaise Pascal: "If I had more time, I would have written less code."
Or to paraphrase Blaise Pascal: "If I had more time, I would have written less code."
Well 90% of my test code is AI generated, and we have a lot of tests. Also Cursor is really good at generating all my documentation. So depending on how we spin it, I could say at least half the code (often more if I'm spinning up new stuff it can do based off existing stuff) in my PR was AI generated.
That said, AI wasn't very good until it had enough examples and guidance from us on our codebase. After that though, it definitely helps.
Caveat: I'm no rocket scientist. It's not difficult code. It's just web services and whatnot. The code is often the least difficult part of my job.
That said, AI wasn't very good until it had enough examples and guidance from us on our codebase. After that though, it definitely helps.
Caveat: I'm no rocket scientist. It's not difficult code. It's just web services and whatnot. The code is often the least difficult part of my job.
Ever used azure? Sure seems like there was an inhuman force behind it
It was bad before AI. Not saying AI vibe code is great, just that poor engineering culture existed before AI.
I'd be curious how much of MSFT's code is generated by simple auto-complete (intellisense) vs AI powered auto-complete vs generated from a prompt.
I wonder about ai-generated code with respect to copyright.
>I'd say maybe 20%, 30% of the code that is inside of our repos today and some of our projects are probably all written by software.
This sentence is carefully crafted. It's an opinion, not a statement of fact, so he can cover his ass. In other words, legal lying. Elon Musk does the same thing all the time. Somehow, the markets don't see through the obvious bullshit.
This sentence is carefully crafted. It's an opinion, not a statement of fact, so he can cover his ass. In other words, legal lying. Elon Musk does the same thing all the time. Somehow, the markets don't see through the obvious bullshit.
It’s not that it is wrong or anything - it’s just unnecessary verbose.
Which you could argue is not a problem if it won’t be read by humans anyways anymore in the near future.