ML code generation vs. coding by hand: what we think programming will look like(wasp-lang.dev)
wasp-lang.dev
ML code generation vs. coding by hand: what we think programming will look like
https://wasp-lang.dev/blog/2022/06/24/ML-code-gen-vs-coding-by-hand-future
133 comments
> Is this really the future of programming?
Not the one I want. I'd like to see better theorem proving, proof repair, refinement and specification synthesis.
It's not generating code that seems to be the difficulty, it's generating the correct code. We need to be able to write better specifications and have our implementations verified to be correct with respect to them. The future I want is one where the specification/proof/code are shipped together.
Not the one I want. I'd like to see better theorem proving, proof repair, refinement and specification synthesis.
It's not generating code that seems to be the difficulty, it's generating the correct code. We need to be able to write better specifications and have our implementations verified to be correct with respect to them. The future I want is one where the specification/proof/code are shipped together.
> It's not generating code that seems to be the difficulty, it's generating the correct code.
Also, it's not actually writing the code that is the difficulty when generating code changes for maintaining existing code, but figuring out what changes need to be made to add up to the desired feature, and where in the codebase controls each of those, and possibly rediscovering important related invariants.
For that matter, the really critical part of implementing a feature isn't implementing the feature -- it's deciding exactly what feature if any needs to be built! Even if many organizations try to split that among people who do and don't have the title of programmer (eg business analyst, product manager...), it's fundamentally programming and once you have a complex enough plan, even pushing around and working on the written English feature requirements resembles programming with some of the same tradeoffs. And we all know the nature of the job is that even the lowliest line programmer in the strictest agile process will still end up having to do some decision making about small details that end up impacting the user.
Copilot is soooooo far away from even attempting to help with any of this, and the part it's starting with is not likely to resemble any of the other problems or contribute in any way to solving them. It's just the only part that might be at all tractable at the moment.
Also, it's not actually writing the code that is the difficulty when generating code changes for maintaining existing code, but figuring out what changes need to be made to add up to the desired feature, and where in the codebase controls each of those, and possibly rediscovering important related invariants.
For that matter, the really critical part of implementing a feature isn't implementing the feature -- it's deciding exactly what feature if any needs to be built! Even if many organizations try to split that among people who do and don't have the title of programmer (eg business analyst, product manager...), it's fundamentally programming and once you have a complex enough plan, even pushing around and working on the written English feature requirements resembles programming with some of the same tradeoffs. And we all know the nature of the job is that even the lowliest line programmer in the strictest agile process will still end up having to do some decision making about small details that end up impacting the user.
Copilot is soooooo far away from even attempting to help with any of this, and the part it's starting with is not likely to resemble any of the other problems or contribute in any way to solving them. It's just the only part that might be at all tractable at the moment.
Can't +1 enough. This is clearly the future, as the increasing popularity of type systems shows.
As interesting as those systems may be, they have it exactly backwards. Don't let the computer generate code and have the human check it for correctness: let the human write code and have the computer check it for correctness.
As interesting as those systems may be, they have it exactly backwards. Don't let the computer generate code and have the human check it for correctness: let the human write code and have the computer check it for correctness.
Actually both don't work.
The human also needs to make sure the code does what the Ticket asked for. And sometimes not even what the Ticket asked for, but what the business and user want/need.
Honestly writing code is such a small part of a dev's job. It's figuring out how to translate the human (often non-tech) requirements into code that is also in a way that's also looking at requests that haven't been made yet.
The human also needs to make sure the code does what the Ticket asked for. And sometimes not even what the Ticket asked for, but what the business and user want/need.
Honestly writing code is such a small part of a dev's job. It's figuring out how to translate the human (often non-tech) requirements into code that is also in a way that's also looking at requests that haven't been made yet.
> let the human write code and have the computer check it for correctness.
Isn't that pretty much what ML does?
Isn't that pretty much what ML does?
That’s exactly the inverse of copilot and LLM code generation. The computer generates the code, the human checks if it’s correct
I bet most humans won't check. See the mindset of the author:
> ... I will not delve into the questions of code quality, security, legal & privacy issues, pricing, and others of similar character. ... Let’s just assume all this is sorted out and see what happens next.
> ... I will not delve into the questions of code quality, security, legal & privacy issues, pricing, and others of similar character. ... Let’s just assume all this is sorted out and see what happens next.
How would ML know what the correct code should be?
Nailed it. I see AI-coding is like hiring junior devs. It'll split out code fast, but unless carefully reviewed, it's just a means to push the tech debt forward. Which isn't bad in itself, as tech debt is good when companies understand the cost. But both are bad if there isn't a true understanding.
Ha ha. I almost spit out my coffee. "when companies understand the cost". OMG that's hilarious.
It's kind of like copying and pasting code from StackOverflow.. Except the code hasn't had as many eyes looking at it as something that was pasted on StackOverflow, and it has a higher likelihood of seeming like a correct solution to your specific use case.
Honestly, it seems pretty dangerous. Seemingly correct code generated by a neural network that we think probably understands your code but may not. Easy to just autocomplete everything, accept the code as valid because it compiles, and discover problems at run-time.
Honestly, it seems pretty dangerous. Seemingly correct code generated by a neural network that we think probably understands your code but may not. Easy to just autocomplete everything, accept the code as valid because it compiles, and discover problems at run-time.
Hit the nail on the head. The major problem with NN is that they are completely unexplainable black boxes. The algorithm spits out a piece of code (or recommends clickbait videos, or bans you from a website, or denies you credit) based on a mishmash of assorted examples and a few million/billion parameters, but can neither explain why it picked it, nor verify that it is correct.
Formal methods and theorem proving on the other hand, place a strong emphasis on proof production and verification.
Formal methods and theorem proving on the other hand, place a strong emphasis on proof production and verification.
The unsettling part is that something like Copilot requires a lot of the work that would need to create an amazing semantic-aware linter like no other on the market.
Instead, they decided to waste their work on that.
Instead, they decided to waste their work on that.
> amazing semantic-aware linter like no other on the market.
AWS has ML driven code analysis tools in their "CodeGuru" suite: https://aws.amazon.com/codeguru/
AWS has ML driven code analysis tools in their "CodeGuru" suite: https://aws.amazon.com/codeguru/
I think this requires a totally different approach to programming. The spec/proof/code cycle exists rarely in the real world. The AI generated code has the disadvantage of being trained on imperfect code, without having any context, so it's like an enthusiastic, very junior copy pasta coder.
Totally.
Yep 100% agree. And the good news is that we are moving closer every day to having tools that make proving code correct practical. Languages like Idris, Agda, Liquid Haskell, F* (F-star), LEAN etc. are spearheading this movement. If Rust had Refinement Types added it would turn it into a killer language for writing correct high performance code.
In all honesty, I typically spend almost twice as much time writing tests as I do writing the code that makes them pass.
More often than not, once I have the tests written, the code nearly writes itself. There are edge cases when I have to look up things, but most of the time it's I'm happily TDD'ing.
More often than not, once I have the tests written, the code nearly writes itself. There are edge cases when I have to look up things, but most of the time it's I'm happily TDD'ing.
Isn't a sufficiently complete specification + proof with which to check generated code against just going to mean that a human spent exactly the same amount of time writing code as they would without the AI generation side of things?
Sure, but you get probably correct code out of it. Not every productivity increase has to do with quantity.
Exactly!
A couple of years ago, we invested in using the libraries from Microsoft Code Contracts for a couple of projects. It was a really promising and interesting project. With the libraries you could follow the design-by-contract paradigm in your C# code. So you could specify pre-conditions, post-conditions and invariants. When the code was compiled you could configure the compiler to generate or not generate code for these. And next to the support for the compiler, the pre- and post-conditions and invariants were also explicitly listed in the code documentation, and there was also a static analyzer that gave some hints/warnings or reported inconsistencies at compile-time. This was a project from a research team at Microsoft and we were aware of that (and that the libraries were not officially supported), but still sad to see it go. The code was made open-source, but was never really actively maintained. [0]
Next to that, there is also the static analysis from JetBrains (ReSharper, Rider): you can use code annotations that are recognized by the IDE. It can be used for (simple) null/not-null analysis, but also more advanced stuff like indicating that a helper method returns null when its input is null (see the contract annotation). The IDE uses static analysis and then gives hints on where you can simplify code because you added null checks that are not needed, or where you should add a null-check and forgot it. I've noticed several times that this really helps and makes my code better and more stable. And I also noticed in code reviews bugs due to people ignoring warnings from this kind of analysis.
And finally, in the Roslyn compiler, when you use nullable reference types, you get the null/not-null compile-time analysis.
I wish the tools would go a lot further than this...
[0] https://www.microsoft.com/en-us/research/project/code-contra... [1] https://www.jetbrains.com/help/resharper/Reference__Code_Ann...
A couple of years ago, we invested in using the libraries from Microsoft Code Contracts for a couple of projects. It was a really promising and interesting project. With the libraries you could follow the design-by-contract paradigm in your C# code. So you could specify pre-conditions, post-conditions and invariants. When the code was compiled you could configure the compiler to generate or not generate code for these. And next to the support for the compiler, the pre- and post-conditions and invariants were also explicitly listed in the code documentation, and there was also a static analyzer that gave some hints/warnings or reported inconsistencies at compile-time. This was a project from a research team at Microsoft and we were aware of that (and that the libraries were not officially supported), but still sad to see it go. The code was made open-source, but was never really actively maintained. [0]
Next to that, there is also the static analysis from JetBrains (ReSharper, Rider): you can use code annotations that are recognized by the IDE. It can be used for (simple) null/not-null analysis, but also more advanced stuff like indicating that a helper method returns null when its input is null (see the contract annotation). The IDE uses static analysis and then gives hints on where you can simplify code because you added null checks that are not needed, or where you should add a null-check and forgot it. I've noticed several times that this really helps and makes my code better and more stable. And I also noticed in code reviews bugs due to people ignoring warnings from this kind of analysis.
And finally, in the Roslyn compiler, when you use nullable reference types, you get the null/not-null compile-time analysis.
I wish the tools would go a lot further than this...
[0] https://www.microsoft.com/en-us/research/project/code-contra... [1] https://www.jetbrains.com/help/resharper/Reference__Code_Ann...
There are tools and programming language that are already way ahead of what you are describing. Examples are Idris, F* (F-Star), Dafny etc. They use Dependent Types and/or Refinement Types to make it possible to prove your code correct. There is now proven correct code in the Windows Kernel and other large projects implemented with those tools.
Thank you for the references to those languages.. interesting!
>> I'd like to see better theorem proving, proof repair, refinement and specification synthesis.
Thanks for making me smile. Some of us are working on exactly that kind of thing :)
Thanks for making me smile. Some of us are working on exactly that kind of thing :)
Machine readable specs?
Something along lines of this perhaps...
https://www.fixtrading.org/standards/fix-orchestra/
Something along lines of this perhaps...
https://www.fixtrading.org/standards/fix-orchestra/
Aren't "Machine Readable Specs" a programming language?
I would argue a programmer who doesn't notice the encoding issue when looking at this code would be the same kind of programmer who would write this kind of encoding issue. You definitely need to manage copilot's generated code to meet your requirements, the key is that massaging takes way less time than writing it all from 0. Copilot is, as the name suggests, a human-in-the-loop system.
For me, this is definitely a piece of the future of coding, and it doesn't change coding from a creative endeavour. It's just the difference between "let me spend five minutes googling this canvas API pattern I used 5 years ago and forgot, stumble through 6 blogs that are garbage, find one that's ok, ah right that's how it works. Now what was I doing?" To just writing some prompt comment like "// fn to draw an image on the canvas" and then being like "ah that's how it works. Tweak tweak write write". For me the creativity is in thinking up the high level code relationships, product decisions, etc. I don't find remembering random APIs or algos I've written a million times before to be creative.
For me, this is definitely a piece of the future of coding, and it doesn't change coding from a creative endeavour. It's just the difference between "let me spend five minutes googling this canvas API pattern I used 5 years ago and forgot, stumble through 6 blogs that are garbage, find one that's ok, ah right that's how it works. Now what was I doing?" To just writing some prompt comment like "// fn to draw an image on the canvas" and then being like "ah that's how it works. Tweak tweak write write". For me the creativity is in thinking up the high level code relationships, product decisions, etc. I don't find remembering random APIs or algos I've written a million times before to be creative.
Let’s take the security aspect one step farther: could a bad actor introduce popular code into copilot, which is full of subtle back-doors? Asking for a befriended state actor.
Why not put copilot code into a SAST tool as a training layer?
Given our experience with Stack Overflow we probably know the answer to that question.
Why would you need a bad actor introducing a bug when Copilot already generates code that is indistinguishable from buggy (and possible backdoored) stuff? It scraped random stuff on Github and was trained on it, nobody knows what the quality or content of that training set was.
Is it cost effective? If companies think that this way of working is cutting development time and/or cost then it's the future. At least in the professional world.
Even bugs are acceptable if the advantages of auto generated code are large enough.
Even bugs are acceptable if the advantages of auto generated code are large enough.
Depends on the bug and how much damage it causes, I guess. The more nefarious and subtle the bug, the more intimate you have to be with the code to understand it.
For developers, this intimacy comes with actually writing the code. I know that when I write code manually, I'm not just tapping keys on a keyboard but building a mental model in my brain. When there's a bug, I can usually intuit where it's located if that model is detailed enough. Maybe developers of the future will just be code reviewers for ML models. Yuck, who wants to do that? So then will we just have ML models reviewing ML generated code?
My concern is that ML code generation will create very shallow mental models of code in developers' minds. If everyone in the organization is building code this way, then who has the knowledge and skills to debug it? I foresee showstopping bugs from ML-generated that bring entire organizations to a grinding halt in the future. I remember back in the day when website generators were in vogue, it became a trend for many webdevs to call their sites "handmade" if they typed all the code themselves. I predict we'll be seeing the same in the future in other domains to contrast "handwritten" source with ML generated source.
What's that famous quote? "Debugging is twice as hard as writing the code in the first place." Well where does that leave us when you don't even have to write the code in the first place? I guess then we will have to turn to ML to handle that for us as well.
My concern is that ML code generation will create very shallow mental models of code in developers' minds. If everyone in the organization is building code this way, then who has the knowledge and skills to debug it? I foresee showstopping bugs from ML-generated that bring entire organizations to a grinding halt in the future. I remember back in the day when website generators were in vogue, it became a trend for many webdevs to call their sites "handmade" if they typed all the code themselves. I predict we'll be seeing the same in the future in other domains to contrast "handwritten" source with ML generated source.
What's that famous quote? "Debugging is twice as hard as writing the code in the first place." Well where does that leave us when you don't even have to write the code in the first place? I guess then we will have to turn to ML to handle that for us as well.
It's the old "C vs. Assembly" all over again.
Not quite - C is predicable(ish). It would be like C vs Asm if there was no stable language spec, platform spec, or compiler.
I think it’s more like WYSIWYG website generators (anyone remember Cold Fusion? The biz guys LOVED that - for like a year).
I think it’s more like WYSIWYG website generators (anyone remember Cold Fusion? The biz guys LOVED that - for like a year).
Except that it's not. I can be reasonably sure that my C compiler will generate correct machine code if my C code is correct. I can also treat the generated machine code as a build artifact which can be thrown away and re-generated on demand; I can treat my C code as my only source of truth. None of these things are true for Copilot, you need to keep around and maintain the generated code, and you can't just assume that the generated code is correct if your input is correct.
I don't disagree with that. My comment is more intended to be read as, "Is this the future we want?" rather than, "This doesn't seem like something which will take off".
Except for companies that are ‘Software Eng is the core business’ anyway, which is super rare, I doubt anyone on the business side cares if we want it. They care if it is cost effective/works.
Which I personally suspect this will end up about as well as WYSIWYG HTML editors/site generators. Which do have a niche.
That would be the ‘makes a lot of cheap crap quickly, but no one uses it for anything serious or at scale’ niche.
Because what comes out the other end/under the hood is reprocessed garbage that only works properly in a very narrow set of circumstances.
I suspect with this market correction however we’ll see a decrease in perceived value for many of these tools, as dev pay is going to be going down (or stay the same despite inflation), and dev availability is going to go up.
Which I personally suspect this will end up about as well as WYSIWYG HTML editors/site generators. Which do have a niche.
That would be the ‘makes a lot of cheap crap quickly, but no one uses it for anything serious or at scale’ niche.
Because what comes out the other end/under the hood is reprocessed garbage that only works properly in a very narrow set of circumstances.
I suspect with this market correction however we’ll see a decrease in perceived value for many of these tools, as dev pay is going to be going down (or stay the same despite inflation), and dev availability is going to go up.
You can't have cost effective when the code doesn't work because it has been generated by a monkey that doesn't understand neither the logic nor the requirements.
That the monkey is actually a computer and not an actual banana eating primate doesn't change anything there.
Any business fool that tries this will discover this very quickly - that's like claiming we don't need drivers because we have cruise control in our cars! It is not perfect but it is cost effective/works (some of the time ...)!
That the monkey is actually a computer and not an actual banana eating primate doesn't change anything there.
Any business fool that tries this will discover this very quickly - that's like claiming we don't need drivers because we have cruise control in our cars! It is not perfect but it is cost effective/works (some of the time ...)!
I agree - that wouldn’t stop a bunch of sales folks from selling it though, or some folks maybe trying it and claiming it worked.
Do it enough times, and I’m sure it will eventually compile - might even complete a regression test suite!
Do it enough times, and I’m sure it will eventually compile - might even complete a regression test suite!
Why are you comparing Indians to monkeys? That's totally not ok.
I don't think "we" have any say in this---it's a question of whether this is what the people with money want.
If companies think that this way of working is cutting development time and/or cost then it's the future.
I think most companies' management would still leave decisions like that in the hands of developers/development-teams. At the level of management, I think companies aren't asking for "code" but for results (a website, an application, a feature). Results with blemishes or insecurities are fine but a stream of entirely broken code wouldn't be seen as cost effective.
I think most companies' management would still leave decisions like that in the hands of developers/development-teams. At the level of management, I think companies aren't asking for "code" but for results (a website, an application, a feature). Results with blemishes or insecurities are fine but a stream of entirely broken code wouldn't be seen as cost effective.
If shortsighted management can offshore to the lowest bidder, they can believe that a few junior devs and ML code generation can replace a team of seniors. Sure, they can leave the decisions of frameworks in the hands of developers and development teams, but if those people are asked to justify the budgetary costs when compared to ML code generation it is no longer a technical argument.
The problem always comes later (often after the manager has moved on and is no longer aware of the mistakes that were made and the repercussions can't follow) where the maintenance costs of poorly written initial code come back to haunt the origination for years to come.
The problem always comes later (often after the manager has moved on and is no longer aware of the mistakes that were made and the repercussions can't follow) where the maintenance costs of poorly written initial code come back to haunt the origination for years to come.
For the benefit of others wondering how it should be done instead: it’s best to use a proper type that will do the encoding, instead of passing a raw body. Sure, you can perform the encoding yourself (e.g. `text=${encodeURIComponent(text)}`) and pass a raw body, but as things scale that’s much more error-prone; it’s better to let types do the lifting from the start (kinda like lifting heavy things by squatting rather than bending at the waist).
For application/x-www-form-urlencoded as shown, that’d be using URLSearchParams:
As a fun additional aside, you’ve made a tiny error in your transcription: the URL was actually surrounded in backticks (a template literal), not single quotation marks. Given the absence of a tag or placeholders (which would justify a template literal) and the use of double-quoted strings elsewhere, both of these would be curious choices that a style linter would be very likely to complain about. So yeah, just another small point where it’s generating weird code.
For application/x-www-form-urlencoded as shown, that’d be using URLSearchParams:
fetch("https://text-processing.com/api/sentiment", {
method: "POST",
body: new URLSearchParams({ text }),
})
const body = new FormData();
body.append("text", text);
fetch("https://text-processing.com/api/sentiment", { method: "POST", body })
As a fun additional aside, you’ve made a tiny error in your transcription: the URL was actually surrounded in backticks (a template literal), not single quotation marks. Given the absence of a tag or placeholders (which would justify a template literal) and the use of double-quoted strings elsewhere, both of these would be curious choices that a style linter would be very likely to complain about. So yeah, just another small point where it’s generating weird code.
The thing is, devs already write this kind of bug on a regular basis. I saw it firsthand as a pentester. So at worst, it's still matching the status quo.
The programmers that consistently create this kind of code are what is normally called "incompetent".
Yes, many companies prefer to hire them, or can only retain them for a variety of reasons. None of this is good in any way.
Anyway, those programmers have really good odds to stop creating this kind of code given some learning. While adding Copilot as a "peer" just makes them less likely to learn, and all programmers more likely to act like them. That's not matching the status-quo, that's a very real worsening of it.
Yes, many companies prefer to hire them, or can only retain them for a variety of reasons. None of this is good in any way.
Anyway, those programmers have really good odds to stop creating this kind of code given some learning. While adding Copilot as a "peer" just makes them less likely to learn, and all programmers more likely to act like them. That's not matching the status-quo, that's a very real worsening of it.
That's kind of fallacious. First it assumes that humans and Copilot are about on par just because humans sometimes write that kind of bug; in reality we need statistics about the relative rates, where I would assume that experienced programmers write that kind of bug less frequently than Copilot, since it seems to be in virtually every Copilot show-case. Second, it categorizes humans as a whole into one group and Copilot into another; in reality, there are better and worse human programmers, while there's just the one Copilot which seems to be on par with a pretty shitty programmer.
Well, except humans don’t do it as consistently at scale! Hah
Not to excuse security holes, but at least they're adversarial - I would expect this code to generate all kinds of error logs if tested against any reasonable corpus along the "happy path" - if you ran your code against a sample of extremely representative input you'd find this.
Security holes are more excusable because someone who didn't realize the above could happen maybe never tested it... given the use case, this is more like "did you even run your code?"
Security holes are more excusable because someone who didn't realize the above could happen maybe never tested it... given the use case, this is more like "did you even run your code?"
>> This isn't the first time I've seen this kind of bug either -- and this class of bug is always shown by people trying to showcase how amazing Copilot is, so it seems like an inherent flaw.
I think it's because people copy/paste generated code without reading it carefully. They eyball it, it makes sense, they go tweet about it.
I don't know if this predicts how people will mostly use generated code. I note however that this is probably too much code to expect CoPilot to generate correctly: about 10 LoCs is too much for a system that can generate code, but can't check it for correctness of some sort. It's better to use it for small code snippets of a couple of lines, like loops and branches etc, than to ask it to genrate entire functions. The latter is asking for trouble.
I think it's because people copy/paste generated code without reading it carefully. They eyball it, it makes sense, they go tweet about it.
I don't know if this predicts how people will mostly use generated code. I note however that this is probably too much code to expect CoPilot to generate correctly: about 10 LoCs is too much for a system that can generate code, but can't check it for correctness of some sort. It's better to use it for small code snippets of a couple of lines, like loops and branches etc, than to ask it to genrate entire functions. The latter is asking for trouble.
I don't think you're right here frankly, since the buggy snippet is taken from the Copilot marketing page (https://github.com/features/copilot). The examples on that page which could conceivably have missing escape bugs are the sentiment analysis example (sentiments.ts), the tweet fetcher examples (fetch_tweets.js, fetch_tweets.ts, fetch_tweets.go) and the goodreads rating examples (rating.js, rating.py, rating.ts, rating.go). Of all of them, only the rating.go example is without a serious escaping bug, and only because Copilot happened to use a URL string generation library for rating.go.
These are the examples which GitHub itself uses to demonstrate what Copilot is capable of, so it's not just a matter of people tweeting without reading through the code properly. It also suggests that the people behind Copilot do believe that one primary use-case for Copilot is to generate entire functions.
These are the examples which GitHub itself uses to demonstrate what Copilot is capable of, so it's not just a matter of people tweeting without reading through the code properly. It also suggests that the people behind Copilot do believe that one primary use-case for Copilot is to generate entire functions.
Hm, thanks, I wasn't aware of that.
Well, OpenAI are certainly trying to sell copilot as more capable than it is, or anyway they haven't done much to explain the limitations of their systems. But they're not alone in that. I can't think of many companies with a product they sell that tell you how you _can't_ use it.
Not to excuse misleading advertisment. On the contrary.
Well, OpenAI are certainly trying to sell copilot as more capable than it is, or anyway they haven't done much to explain the limitations of their systems. But they're not alone in that. I can't think of many companies with a product they sell that tell you how you _can't_ use it.
Not to excuse misleading advertisment. On the contrary.
> Is programming going to go from a creative endeavor to make the machine do what you want, to a job which mostly consists of reviewing and debugging auto-generated code?
Maybe, in some organizations, sure. However, there are still people hand-crafting things in wood, metal, and other materials even though we have machines that can do almost anything. Maybe career programmers will turn into "ML debuggers", so perhaps all of us who enjoy building things ourselves will just stop working as programmers? I certainly won't work in the world where I'm just a "debugger" for machine-generated crap.
Maybe, in some organizations, sure. However, there are still people hand-crafting things in wood, metal, and other materials even though we have machines that can do almost anything. Maybe career programmers will turn into "ML debuggers", so perhaps all of us who enjoy building things ourselves will just stop working as programmers? I certainly won't work in the world where I'm just a "debugger" for machine-generated crap.
Written by GPT-3:
function evil(code) {
eval(code);
}
evil("alert('hi');");So essentially Copilot is very dull and boring (yet).
Taking your comment as inspiration, I would like to add OWASP/Unit Testing thinking to Copilot. If Copilot considers your remarks and maybe others, it becomes helpful. Something like security checking on the fly, what would normally be considered by colleagues during code reviews or checks with SonaCube.
Taking your comment as inspiration, I would like to add OWASP/Unit Testing thinking to Copilot. If Copilot considers your remarks and maybe others, it becomes helpful. Something like security checking on the fly, what would normally be considered by colleagues during code reviews or checks with SonaCube.
Also can't avoid to see the bug in the reasoning:
> Let’s just assume all this (code quality, security, legal & privacy issues, pricing, and others of similar character) is sorted out and see what happens next.
What could go wrong next in the systems where they employ this AI with this mindset?
> Let’s just assume all this (code quality, security, legal & privacy issues, pricing, and others of similar character) is sorted out and see what happens next.
What could go wrong next in the systems where they employ this AI with this mindset?
This type of thing is currently useful for rapid prototyping, or building a quick framework. At the end of the day though, the experienced coders need to fix bugs, modify the features, enhance the efficiencies, and review if the program is doing what it is truly intended to do.
>This code doesn't escape the text, so if the text contains the letter '&' or other characters with special meanings in form URL encoding, it will break
Why? POST data doesn't need to be URL encoded.
Why? POST data doesn't need to be URL encoded.
Yes, it does. POST data with `Content-Type: application/x-www-form-urlencoded` uses the same encoding format as query parameters in a URL -- `foo=a&bar=b` -- hence the name.
Machine Learning algorithms are only as good as the data they are trained on.
For tasks like self-driving or spotting cancer in x-rays, they are producing novel result because these kinds of tasks are amenable to reinforcement. The algorithm crashed the car, or it didn't. The patient had cancer, or they didn't.
For tasks like reproducing visual images or reproducing text, it _seems_ like these algorithms are starting to get "creative", but they are not. They are still just regurgitating versions of the data they've been fed. You will never see a truly new style or work of art from DALL-E, because DALL-E will never create something new. Only new flavors of something old or new flavors of the old relationships between old things.
Assuming that it is even possible to describe novel software engineering problems in a way that a machine could understand (i.e. in some complex structured data format), software engineering is still mostly a creative field. So software engineering isn't going to performed by machine learning for the same reason that truly interesting novels or legitimately new musical styles won't be created by machine learning.
Creating something new relies on genuine creativity and new ideas and these models can only make something "new" out of something old.
For tasks like self-driving or spotting cancer in x-rays, they are producing novel result because these kinds of tasks are amenable to reinforcement. The algorithm crashed the car, or it didn't. The patient had cancer, or they didn't.
For tasks like reproducing visual images or reproducing text, it _seems_ like these algorithms are starting to get "creative", but they are not. They are still just regurgitating versions of the data they've been fed. You will never see a truly new style or work of art from DALL-E, because DALL-E will never create something new. Only new flavors of something old or new flavors of the old relationships between old things.
Assuming that it is even possible to describe novel software engineering problems in a way that a machine could understand (i.e. in some complex structured data format), software engineering is still mostly a creative field. So software engineering isn't going to performed by machine learning for the same reason that truly interesting novels or legitimately new musical styles won't be created by machine learning.
Creating something new relies on genuine creativity and new ideas and these models can only make something "new" out of something old.
Kinda - if you have a problem amenable to some kind of evolutionary training model/feedback, you could certainly improve it via ML. Anything from compression to data processing can and has been improved through evolutionary training processes.
If you trained an ML model on recognizing patterns of older less efficient algorithms and swapping in new ones, that could be useful. It probably isn’t that much harder of a problem than knowledge graph building and objecg recognition.
Also, having an ML model trained to recognize certain types of problem patterns in whatever source product/model/codebase or whatever and then spit out code to solve them is certainly not a theoretically impossible or low value task.
That said, ML as an art or science is no where near being able to do that usefully near as I can tell.
Hell even basic chatbots still suck.
Hell, even getting Hibernate to usefully reverse engineer a pretty normal schema still sucks.
If you trained an ML model on recognizing patterns of older less efficient algorithms and swapping in new ones, that could be useful. It probably isn’t that much harder of a problem than knowledge graph building and objecg recognition.
Also, having an ML model trained to recognize certain types of problem patterns in whatever source product/model/codebase or whatever and then spit out code to solve them is certainly not a theoretically impossible or low value task.
That said, ML as an art or science is no where near being able to do that usefully near as I can tell.
Hell even basic chatbots still suck.
Hell, even getting Hibernate to usefully reverse engineer a pretty normal schema still sucks.
Machine Learning algorithms are only as good as the data they are trained on.
Machine Learning algorithms are never better than the data they're trained-on. But they can easily be "worse".
Specifically, an ML algorithm trained on one data set can have a hard time operating on a different data set with some similarities and some fundamental differences. This is related to algorithms generally not having an idea how accurate their predictions/guesses are. This in turn relates to current ML as being something like statistical prediction with worries about bias tossed-out (which isn't to dismiss it but to illuminate it's limitations).
For tasks like self-driving or spotting cancer in x-rays, they are producing novel result because these kinds of tasks are amenable to reinforcement. The algorithm crashed the car, or it didn't. The patient had cancer, or they didn't.
Ironically, both those applications have been failures so-far. Self-driving is far more complex than a binary crash-or-not scenario (usually the road is predictable but just about anything can wander into the roadway occasionally and you need to deal with this "long tail"). It also requires extreme safety to reach human levels. Diagnosis by image has problems of consistence, of doctors considering more than just an image and other things possibly not understood.
Machine Learning algorithms are never better than the data they're trained-on. But they can easily be "worse".
Specifically, an ML algorithm trained on one data set can have a hard time operating on a different data set with some similarities and some fundamental differences. This is related to algorithms generally not having an idea how accurate their predictions/guesses are. This in turn relates to current ML as being something like statistical prediction with worries about bias tossed-out (which isn't to dismiss it but to illuminate it's limitations).
For tasks like self-driving or spotting cancer in x-rays, they are producing novel result because these kinds of tasks are amenable to reinforcement. The algorithm crashed the car, or it didn't. The patient had cancer, or they didn't.
Ironically, both those applications have been failures so-far. Self-driving is far more complex than a binary crash-or-not scenario (usually the road is predictable but just about anything can wander into the roadway occasionally and you need to deal with this "long tail"). It also requires extreme safety to reach human levels. Diagnosis by image has problems of consistence, of doctors considering more than just an image and other things possibly not understood.
Agreed. ML (a component of AI) faces the same issue today as it always has, outliers, i.e. the real world.
Alphago, and similar, are all examples of ML systems exceeding their training data and human abilities. It's clearly possible to beat human capability with ML.
I should have add "in unstructured interaction with the 'real world'", 'cause that's the context here.
Computers exceeded human chess capacities without any AI as such in 1999 or so. Computer exceeded human addition and multiplication capacities as soon as they were built - that's what they were built for.
Computers exceeded human chess capacities without any AI as such in 1999 or so. Computer exceeded human addition and multiplication capacities as soon as they were built - that's what they were built for.
>> Computers exceeded human chess capacities without any AI as such in 1999 or so.
To be pedantic, that was without any machine learning. Deep Blue was absolutely considered an AI system at the time. Tree search as used in Deep Blue is also still central to Alpha* family of game-playing systems.
To be pedantic, that was without any machine learning. Deep Blue was absolutely considered an AI system at the time. Tree search as used in Deep Blue is also still central to Alpha* family of game-playing systems.
>Alphago, and similar, are all examples of ML systems exceeding their training data
Alphago basically generated its own training data (hence, "exceeding" is not correct) because the domain was amenable to a specific form of reinforcement learning. Programming isn't such a domain.
Alphago basically generated its own training data (hence, "exceeding" is not correct) because the domain was amenable to a specific form of reinforcement learning. Programming isn't such a domain.
An early version of Alphago was trained on expert human games and did go on to defeat the human champion. Later versions of Alphago are stronger still and developed entirely through selfplay to become superhuman. I would say these versions did exceed their training data because they started training on basically random games and got increasingly better as they learned. What is that if not exceeding what they started from?
Alphago is proof that ML systems can exceed human abilities in at least some domains. Other domains, like identifying animal breeds/species by sight, I believe ML models are also super human. It's just not correct to say models are limited by their training data to something less than human quality. We have multiple counterexamples already.
Alphago is proof that ML systems can exceed human abilities in at least some domains. Other domains, like identifying animal breeds/species by sight, I believe ML models are also super human. It's just not correct to say models are limited by their training data to something less than human quality. We have multiple counterexamples already.
>I would say these versions did exceed their training data
Well, this is obviously incorrect because the training data included all the games it played against itself (good luck replicating that in the domain of programming though).
We were not talking about "exceeding human abilities in at least some domains", we were talking about whether ML (DL really) systems were able to "go beyond" their training data - but your deflection and going off point is noted.
Well, this is obviously incorrect because the training data included all the games it played against itself (good luck replicating that in the domain of programming though).
We were not talking about "exceeding human abilities in at least some domains", we were talking about whether ML (DL really) systems were able to "go beyond" their training data - but your deflection and going off point is noted.
Their training data is also part of their output. The training games produced last exceed the earlier games. So, yes, obviously the model can exceed its training data.
If you're trying to argue that the model can't be better than it is, i.e. the last training data it produces, then that's true but tautologically so. ML has conquered some domains, like Go or Chess or birdwatching, and is making progress in others - like programming. There is no reason to assume ML systems can't exceed human ability at programming or any other task.
If you're trying to argue that the model can't be better than it is, i.e. the last training data it produces, then that's true but tautologically so. ML has conquered some domains, like Go or Chess or birdwatching, and is making progress in others - like programming. There is no reason to assume ML systems can't exceed human ability at programming or any other task.
>Their training data is also part of their output.
More deflection and going off point.
>is making progress in others - like programming
I already explained that programming is not amenable to the approach that allowed alphago to learn from the games it played against itself.
This is already getting pointless and tedious, should in the future know better than to respond to people that are obviously in the "scale is all you need" nonsense camp, nothing meaningful comes out of it.
More deflection and going off point.
>is making progress in others - like programming
I already explained that programming is not amenable to the approach that allowed alphago to learn from the games it played against itself.
This is already getting pointless and tedious, should in the future know better than to respond to people that are obviously in the "scale is all you need" nonsense camp, nothing meaningful comes out of it.
I don't see any reason why an ML algorithm wouldn't be able to extrapolate to novel ideas. Once it has learned some high dimensional subspace based on the examples given to it, I suppose it could search for areas outside that boundary which are in some way likely to be interesting.
That's a super interesting observation; you've convinced me that ML algorithms could generate novel things.
This is why I'm gonna commit a few bugs to the main branch today.
Trying to solve the lack of progress in simplifying the complexity of standard development scenarios needs to be the next step in framework development. I just don't understand how or why it needs to be polluted by the unnecessary added hidden complexity of ML.
You're just shifting your unknowns into a black box and it's magic output.
ML is not a “black box”.
It’s formalized statistics translated to code by community.
Regex and grep are just as “black box magic” to those who read a blog post on how to scan some text files.
ML is statistics on arrays, whereas Unix tools are stats for a Unix system.
It’s the same abstraction at its core, new implementation.
It’s formalized statistics translated to code by community.
Regex and grep are just as “black box magic” to those who read a blog post on how to scan some text files.
ML is statistics on arrays, whereas Unix tools are stats for a Unix system.
It’s the same abstraction at its core, new implementation.
In a recently reported issue with an Apple Car almost running over a pedestrian, someone on HN asked why can't we simply add the code:
This is what people mean when they say it is a "black box." There is no source code to edit, only training data that outputs an as-is model. It's similar to getting binary blobs, for eg Linux drivers.
In contrast, I could fork regex if I wanted and add an `if` statement in the middle of it, and re-compile it. It is pure source code. Same with grep.
if (personInFrontOfCar) { halt(); }
This is what people mean when they say it is a "black box." There is no source code to edit, only training data that outputs an as-is model. It's similar to getting binary blobs, for eg Linux drivers.
In contrast, I could fork regex if I wanted and add an `if` statement in the middle of it, and re-compile it. It is pure source code. Same with grep.
From my admittedly limited knowledge of how self driving policy models are implemented, I believe the main black box AI is determining the part
`if personInFrontOfCar:`
(using object detection, masking, 3D pose estimation, et/or cetera)
And actually defined policies on how to handle certain situations given that inference are possible, in the case specifically of self driving vehicles.
My assumptions come from when I'd asked someone at NeurIPS in 2019 about what reinforcement learning methods they use as they said "I don't think any self driving companies are using reinforcement learning, it is too risky". I don't mean to imply this is hard to find information either, reading a few papers would be all it takes to clear up the degree to which most self driving policies are "controllable" in the way you describe, or at least to what degree.
My main point is that I think it is the inferring of the environment (current state) rather than the chosen policy at each time step which is more of an error prone black box.
`if personInFrontOfCar:`
(using object detection, masking, 3D pose estimation, et/or cetera)
And actually defined policies on how to handle certain situations given that inference are possible, in the case specifically of self driving vehicles.
My assumptions come from when I'd asked someone at NeurIPS in 2019 about what reinforcement learning methods they use as they said "I don't think any self driving companies are using reinforcement learning, it is too risky". I don't mean to imply this is hard to find information either, reading a few papers would be all it takes to clear up the degree to which most self driving policies are "controllable" in the way you describe, or at least to what degree.
My main point is that I think it is the inferring of the environment (current state) rather than the chosen policy at each time step which is more of an error prone black box.
It's what they call 'democratization of code' imo.
ML generated code won't be used for high tech software at first, but for the mass of not so critical software like web frontend scripting.
I have zero hopes for ML getting web frontend right when real life humans have such a hard time doing so.
What you said actually sounds pretty similar to what I can imagine was said 'back in the day' when compilers were first developed and languages like C were invented.
> Trying to solve the lack of progress in unifying computer architectures needs to be the next step in computer science. I just don't understand how or why it needs to be polluted by the unnecessary added hidden complexity of GCC. You're just shifting your assembly language generation into a black box and its magic output.
(I don't actually know how to write assembly, so pardon any technical faux pas. But the comparison seemed apt. ML as an intent->code compiler just seems like an inevitable step forward.)
> Trying to solve the lack of progress in unifying computer architectures needs to be the next step in computer science. I just don't understand how or why it needs to be polluted by the unnecessary added hidden complexity of GCC. You're just shifting your assembly language generation into a black box and its magic output.
(I don't actually know how to write assembly, so pardon any technical faux pas. But the comparison seemed apt. ML as an intent->code compiler just seems like an inevitable step forward.)
I can see why you would make this analogy but I personally don't think of code generation and code compilation to be analogous.
GCC is less opaque (less of a black box) than these language models.
My work is 90% business and 10% code. I do not see how we can have AI writing useful code on its own without also understanding lots of context about human society.
When I code something out, I have to first know which problem to solve, and why it’s a problem. I then have to understand the problem in minute detail. This usually involves a slew of human factors, desires, and interfacing with idiosyncratic systems that evolved a certain way because of human factors and desires.
When I code something out, I have to first know which problem to solve, and why it’s a problem. I then have to understand the problem in minute detail. This usually involves a slew of human factors, desires, and interfacing with idiosyncratic systems that evolved a certain way because of human factors and desires.
This is a fair point, but you may lose 10% of your job. By the time one of our designers hands off their interactive Figma mocks to me this has already been figured out. I think there is plenty of specificity that is required to build things of value, but sufficient specificity is available from the people who don't code to allow a slightly better version of Copilot to do the implementation.
If we imagine that at some point in the future, the majority of code would be produced by ML code generation, then where will the training data for future ML models come from?
With the current way of ML code generation, it seems that there will always need to be a critical mass of coders who produce code on their own, to be able to serve as new input for the ML models, to be able to learn about new use cases, new APIs, new programming languages, and so on.
ML code generation may serve as a multiplier, but it raises questions about the creation and flow of new knowledge and best practices.
With the current way of ML code generation, it seems that there will always need to be a critical mass of coders who produce code on their own, to be able to serve as new input for the ML models, to be able to learn about new use cases, new APIs, new programming languages, and so on.
ML code generation may serve as a multiplier, but it raises questions about the creation and flow of new knowledge and best practices.
The problem with all of these arguments is we are just shooting in the dark.
The day before Copilot launched, if someone had told me code generation at the fairly decent quality Copilot achieves was already possible I probably wouldn't believe it. I could happily rattle off a couple of arguments for why it might be better than autocomplete but could never write a full function. Then it did.
Who can say how far it's come since then? I think only the Copilot team knows. I wish we could hear from them, or some ML experts who might know?
The day before Copilot launched, if someone had told me code generation at the fairly decent quality Copilot achieves was already possible I probably wouldn't believe it. I could happily rattle off a couple of arguments for why it might be better than autocomplete but could never write a full function. Then it did.
Who can say how far it's come since then? I think only the Copilot team knows. I wish we could hear from them, or some ML experts who might know?
Generating entire functions has been possible for a long time before Copilot. The discipline that studies algorithms that generate code or programs is called Program Synthesis. For a recent review see Gulwani et al:
Program Synthesis
https://www.semanticscholar.org/paper/Program-Synthesis-Gulw...
Copilot and similar systems (like DeepMind's AlphaCode) are a step backwards compared with earlier systems most of which are capable of not only generating code, but generating correct programs, where correctness is measured with respect to a specification, which can be complete or incomplete (as in input/output examples etc).
Systems like Copilot instead use a large language model to generate code by completing an initial "prompt" and have no way to control what kind of code is generated or whether it does what the programmer expects it to do. The task of checking correctness is left to the user. The result is code that may or may not contain bugs to an extent difficult to predict beforehand.
Specifically about Copilot, it's based on Codex, a large language model that's a variant of GPT-3 fine-tuned on all of github. In formal testing, OpenAI found that Codex could generate the correct program about 30% of the time, which is pretty low:
Evaluating Large Language Models Trained on Code
https://arxiv.org/abs/2107.03374
Interestingly, if Codex was allowed 100 "guesses", the correct program was in there 75% of the time, instead. But that only goes to show that these kinds of systems are code firehoses without any ability to tell a correct program from garbage.
Program Synthesis
https://www.semanticscholar.org/paper/Program-Synthesis-Gulw...
Copilot and similar systems (like DeepMind's AlphaCode) are a step backwards compared with earlier systems most of which are capable of not only generating code, but generating correct programs, where correctness is measured with respect to a specification, which can be complete or incomplete (as in input/output examples etc).
Systems like Copilot instead use a large language model to generate code by completing an initial "prompt" and have no way to control what kind of code is generated or whether it does what the programmer expects it to do. The task of checking correctness is left to the user. The result is code that may or may not contain bugs to an extent difficult to predict beforehand.
Specifically about Copilot, it's based on Codex, a large language model that's a variant of GPT-3 fine-tuned on all of github. In formal testing, OpenAI found that Codex could generate the correct program about 30% of the time, which is pretty low:
Evaluating Large Language Models Trained on Code
https://arxiv.org/abs/2107.03374
Interestingly, if Codex was allowed 100 "guesses", the correct program was in there 75% of the time, instead. But that only goes to show that these kinds of systems are code firehoses without any ability to tell a correct program from garbage.
The problem with this is that, just like visual programming, it doesn’t really do the hard parts.
I remember hearing “a million monkeys with a million type writers writing for a million years could write Shakespeare”. Sure, but how would they know it? How could they recognize when they have achieved their goal?
ML will lower the bar for the trivial tedious things and let people believe they are more capable than they are. Even with ML you will have to know what to ask and what the answer should be. That is always the hard part.
I remember hearing “a million monkeys with a million type writers writing for a million years could write Shakespeare”. Sure, but how would they know it? How could they recognize when they have achieved their goal?
ML will lower the bar for the trivial tedious things and let people believe they are more capable than they are. Even with ML you will have to know what to ask and what the answer should be. That is always the hard part.
Automating the easy parts does give you more time to work on hard things, though.
I've been using a Copilot for a while. It's neat, saves me some typing sometimes, but it's not even remotely close to doing even a small portion of my work for me.
I couldn't find any use for it. It just bloated my IDE and added latency, and never helped me accomplish a single thing. I deleted the plugin.
I guess if it did one thing it was reinforce something I already knew about my job - writing code is always the easy part.
I guess if it did one thing it was reinforce something I already knew about my job - writing code is always the easy part.
Yeah Copilot has been pretty useless for me. I'm not really sure what this is indicative of. Maybe my coding style is not that common? I'm unsure. Mostly used it with Go and configuration files.
“Although ML code generation helps with getting the initial code written, it cannot do much beyond that - if that code is to be maintained and changed in the future (and if anyone uses the product, it is), the developer still needs to fully own and understand it.”
I think this is a misconception. It’s true for these first prototype code generation tools, but there’s no reason to think that in the future these models won’t be adapted to modify/maintain code too.
I think this is a misconception. It’s true for these first prototype code generation tools, but there’s no reason to think that in the future these models won’t be adapted to modify/maintain code too.
The article presupposes that ML is only for code generation but not maintenance. Why couldn't this change in the future? Perhaps you'd have an english language document that describes the functionality, and as you update it, the code gets re-generated or updated.
Wouldn't that be strictly worse than specifying the functionality in code? The reason we've invented programming languages is because natural language is ambiguous and imprecise.
Consider the instructions "take a list, add the number three and five, return the list"
This compiles to both
Consider the instructions "take a list, add the number three and five, return the list"
This compiles to both
f(list):
3+5;
return list;
f(list):
list.add(3&5);
return list;
f(list):
list.add(3);
list.add(5);
return list;
> This compiles to both
[three code segments]
Did you intentionally do an off-by-one error?
[three code segments]
Did you intentionally do an off-by-one error?
Not really, but it furthers my point about natural language being imprecise. Even the grammatical rules are ambiguous and frequently broken.
It's as they say: Time flies like an arrow, fruit flies like a banana.
It's as they say: Time flies like an arrow, fruit flies like a banana.
I agree with you, and in fact I think that initial code generation like CoPilot is strictly worse as well. But I'm discussing something orthogonal to code quality here.
This is an interesting thought. Kinda analogous to when you e.g. change your C++ code and the new assembly code is generated.
If there was high-level intent (e.g. something like spec language) that was preserved and versioned, that might be possible I guess.
If there was high-level intent (e.g. something like spec language) that was preserved and versioned, that might be possible I guess.
There's only 2 options, either you train on the full mediocrity of Github, or you over-train on highly rated projects, effectively just copy/pasting those projects and riding roughshod over any licensing the authors think they have. At least the mediocrity of the suggested code suggests they're trying to do the former.
The momentum behind this whole thing feels strongly like a handwavy attempt to subvert the GPL and similar.
How does one differentiate? Say I spend some time looking at GPL code, learning the lessons presented, then writing my own code using what I’ve learned. Is this OK if I am working on non-GPL software? Is the answer different if an algorithm does the same thing?
Yes, because an algorithm isn't a person, and necessarily got to where it got to through what the law clearly defines as "copying," whereas what the human does is expressly not "copying."
Again, always good to remember the thing in the background: This is not a philosophical debate, it's a legal one. Intellectual property, for better or worse, is a made-up concept by humans to try to encourage more creation.
Again, always good to remember the thing in the background: This is not a philosophical debate, it's a legal one. Intellectual property, for better or worse, is a made-up concept by humans to try to encourage more creation.
Intellectual Property is marketed as encouragement for more creation. At this point, its primary purpose is rent-seeking, like most old institutions, I guess.
I agree with you, but as long as megacorps are abusing it to protect their pseudo-monopolies, they should be held up to their own standards. Look at how pissy they get when people reverse-engineer their proprietary systems and then implement unofficial replacements.
I mostly agree, but when I begin to game theory it out* in my own head, I still think it's something worth reforming, not abolishing.
*for the internet, it would simply transfer literally all the power to the youtubes and spotifys.
*for the internet, it would simply transfer literally all the power to the youtubes and spotifys.
I think the biggest misconception with a lot of the comments here is that they are assuming that the code generated by copilot is final. Copilot, as the name suggests, is a human-in-the-loop system. It requires a developer to give it prompts and to massage the code it generates. It's not intended to or will it ever feasibly be able to generate code entirely independently.
The point is that this massaging is a lot faster than writing everything from 0 in most cases. It doesn't have to be bug free. It doesn't have to be perfect quality. It doesn't have to do "the hard parts" of coding. It just has to be good enough to be sufficiently faster for the developer than starting from 0, and it is.
The point is that this massaging is a lot faster than writing everything from 0 in most cases. It doesn't have to be bug free. It doesn't have to be perfect quality. It doesn't have to do "the hard parts" of coding. It just has to be good enough to be sufficiently faster for the developer than starting from 0, and it is.
I think it’s hard to view ML generated code as the future when you see how powerful type-driven development can be with the likes of Idris and others.
Perhaps the best approach could use ML within the constraints provided by a type system. Does research like this exist?
Perhaps the best approach could use ML within the constraints provided by a type system. Does research like this exist?
So far my most practical use of Copilot is prompting it with `// Handle errors`.
Garbage in garbage out.
No, it's still a terrible idea even if it's used as an auto-complete. As the article correctly states, the primary job of a software engineer is to maintain existing code, not to write new code. You're not going to be doing yourself any favors by committing code that you didn't even read (unless you're one of those people who can get away with shoving all your bugs onto coworkers, in which case you're an asshole).
So devils advocate:
- the average developer tenure now is very short
- most software companies just want features, features and more features due to market pressure
- writing new systems is much more popular than maintaining or upgrading old ones.
So maybe this is the future. But it's kind of a sad one.
- the average developer tenure now is very short
- most software companies just want features, features and more features due to market pressure
- writing new systems is much more popular than maintaining or upgrading old ones.
So maybe this is the future. But it's kind of a sad one.
> For the purposes of this post, I will not delve into the questions of code quality, security, legal & privacy issues, pricing, and others of similar character that are often brought up in these early days of ML code generation. Let’s just assume all this is sorted out and see what happens next.
"Let's assume magic ponies exist, are commonplace and would love to give us all magic rainbow rides"
"Let's assume magic ponies exist, are commonplace and would love to give us all magic rainbow rides"
It's kind of sad that the future of programming isn't more expressive languages. It's auto-generating code in the languages we have.
AI coding is not constrained to javascript, which is a function of human knowledge inertia and hiring practices, all irrelevant to AI-assisted coding. Ideally syntax becomes a non issue and we can start valuing correctness and maximizing business leverage per LOC. However we don’t know how long the transition window will be. Everything is accelerating, so quickly I hope!
I think it is strongly likely that Copilot will be 1 million times better by the end of the decade. Increasing compute, new techniques, etc
The top comment on this post is complaining about a bug in Copilot generated code. 1 million times better Copilot won't do that.
The top comment on this post is complaining about a bug in Copilot generated code. 1 million times better Copilot won't do that.
Given the fact that those models seem to improve logarithmically and that they pretty much trained codex on ~all available public code (maybe divide by 2) meaning the training data will at most scale with the growth rate of public repos in Github, I doubt that whole "1 million times better".
Even the number of parameters won't improve with a factor of 1 million (it will not improve by a factor of a 1000 and may even not reach a factor of 100) by the end of the decade.
If you use a formal system with a checker/compiler, there are certain properties that you can guarantee with essentially total certainty. I’m not sure a stochastic system like ML can ever do that.
Perhaps there is research on providing mathematical guarantees and constraints on complex models like neural networks? If not, it feels like it would be harder to give a model a high degree of control. Although embedding the model in a system that did provide guarantees (e.g. using a compiler) might be a pragmatic solution?
Perhaps there is research on providing mathematical guarantees and constraints on complex models like neural networks? If not, it feels like it would be harder to give a model a high degree of control. Although embedding the model in a system that did provide guarantees (e.g. using a compiler) might be a pragmatic solution?
Stochastic methods may not be able to do so in isolation, but they can be used in tandem with other approaches such as in
“A Deep Reinforcement Learning Approach to First-Order Logic Theorem Proving”
https://ojs.aaai.org/index.php/AAAI/article/view/16780/16587
yep that would be really interesting! If e.g. the target generated code was some language with high type safety or a DSL whose compilation already guarantees certain properties.
>[...] thank you for your generous comments, ideas and suggestions! You made this post better and made sure I don't go overboard with memes :).
Yeah man, I'm not sure about the latter. Not sure...
Yeah man, I'm not sure about the latter. Not sure...
Haha you should have seen the first draft! Still appreciate the feedback
[deleted]
Wonder if anyone is working on using ML for machine code generation.
> a huge win for the developer community
> allows us to write even less code and care about fewer implementation details
Remember Bjarne Stroustrup: “I Did It For You All…”? More code, more complexity — more job security.
> allows us to write even less code and care about fewer implementation details
Remember Bjarne Stroustrup: “I Did It For You All…”? More code, more complexity — more job security.
it's a lot harder to fix code you didn't write, seems like even more job security for those who can clean up after the "ai"
Early in my career I worked for a contracting firm that established a secure position in a host company by being willing to take on maintenance of a code base that had been created by a code generator. It wasn't AI-generated, but it was a similar situation where none of the captive employees wanted to slog through the generated code, and my bosses were more than happy to take huge wads of cash to do it. So, the crusty old contractor in me kind of says, "Bring it on."
Great attitude! As I'm working on a non-AI code generator, I will be relieved to know you, and those like you, are out there to clean up my downstream messes ;]
This code doesn't escape the text, so if the text contains the letter '&' or other characters with special meanings in form URL encoding, it will break. Moreover, these kinds of errors can cause serious security issues; probably not in this exact case, the worst an attacker could do is change the sentiment analysis language, but this class of bug in general is rife with security implications.
This isn't the first time I've seen this kind of bug either -- and this class of bug is always shown by people trying to showcase how amazing Copilot is, so it seems like an inherent flaw. Is this really the future of programming? Is programming going to go from a creative endeavor to make the machine do what you want, to a job which mostly consists of reviewing and debugging auto-generated code?