On Learning to Program with LLMs(se-radio.net)
se-radio.net
On Learning to Program with LLMs
https://www.se-radio.net/2023/09/se-radio-582-leo-porter-and-daniel-zingaro-on-using-llms-in-the-classroom/
79 comments
The thing about LLMs as a teacher is that the feedback loop is so short that progress can be made quite fast. Being able to ask a question, regardless of how stupid or trivial it may seem, without fear of judgement is a huge learning hurdle that is overcome. Not only that but reverse engineering other code “if I have this code and wanted it to do this instead - what and how would we change? Etc”, and a short feedback loop for errors where you might have to copy and paste something a lot before it works, makes LLMs the ultimate teachers.
I agree about short feedback loops, but this was there long before LLMs, and got me into programming initially.
I love that I could compile my program and try it without needing to check it with a teacher or superior. It cost literally nothing to try, and if it failed the computer would tell you it was wrong.
In a sense you are constantly bouncing off of reality (or the limits of the compiler) and that allows you to learn really quickly by experiment. LLMs with their tendency to hallucinate or just provide you with the solution seem like they could be both a blessing and a curse (for learning).
I love that I could compile my program and try it without needing to check it with a teacher or superior. It cost literally nothing to try, and if it failed the computer would tell you it was wrong.
In a sense you are constantly bouncing off of reality (or the limits of the compiler) and that allows you to learn really quickly by experiment. LLMs with their tendency to hallucinate or just provide you with the solution seem like they could be both a blessing and a curse (for learning).
Providing the solution is one thing, but being able to tell you more about the solution, why it is a certain way, what other examples are, what to look out for, etc. having an infinite dialog about the problem space, that's where it really shines. Banging my head against compiler output and combing through Google to figure out _why_ it failed isn't nearly as productive, imo. Otherwise yes, hallucinations are an issue, though I find them not nearly as detrimental as I thought I would, then again I'm not learning to program, I'm merely learning new frameworks or techniques, I already have the underlying foundation and can sniff out hallucinations probably better than a total novice.
I don't know about this take. A lot of what you say makes sense but your conclusion seems like a bit of a stretch.
LLMs could be very useful for learning and probably are, but they also allow you to ask the wrong questions and circumvent the learning you were supposed to be doing.
I wouldn't be surprised to see people using it to simply solve exercises for them, and those people will have a hard time trying to do things on their own.
LLMs could be very useful for learning and probably are, but they also allow you to ask the wrong questions and circumvent the learning you were supposed to be doing.
I wouldn't be surprised to see people using it to simply solve exercises for them, and those people will have a hard time trying to do things on their own.
It's actually worse than that because although LLMs are capable of a lot, they still hallucinate quite frequently and sometimes are just plain stupid.
Recently I was trying to manipulate data in Google Sheets and was using ChatGPT to help. In the beginning is was fantastic, I was very productive because I didn't have to stop and think about formulas, read crappy documentation, or analyze data transformation. ChatGPT just gave me the right answer in a split second, as long as I kept asking the right questions.
Then I stumbled upon a particular issue that wasn't really too complicated but ChatGPT could not give me a correct answer. Unknowingly I spent 3 days trying to fix problems with the solution and every time I got an answer that was slightly wrong, not in subtle ways.
I have 20 years experience as a software engineer and still, I continued to waste time in this loop. After 3 days I decided to apply my engineering skills and solved the matter in 30 min. Now I know the solution and it was way simpler than I thought.
What surprised me was how dumb the whole process was. My questions certainly weren't the problem - as bad as they possibly were, the solution wasn't too far off. Not only ChatGPT had become a crutch but it put me in a situation that no human-tutor would ever put me.
So removing the pain of quick interactions with a tutor has benefits but the technology is not quite ready to be considered as true guidance in forming (or even helping) someone's understanding of a subject.
I've been using ChatGPT a lot for general language but when it requires logical thinking it falls pretty flat.
Recently I was trying to manipulate data in Google Sheets and was using ChatGPT to help. In the beginning is was fantastic, I was very productive because I didn't have to stop and think about formulas, read crappy documentation, or analyze data transformation. ChatGPT just gave me the right answer in a split second, as long as I kept asking the right questions.
Then I stumbled upon a particular issue that wasn't really too complicated but ChatGPT could not give me a correct answer. Unknowingly I spent 3 days trying to fix problems with the solution and every time I got an answer that was slightly wrong, not in subtle ways.
I have 20 years experience as a software engineer and still, I continued to waste time in this loop. After 3 days I decided to apply my engineering skills and solved the matter in 30 min. Now I know the solution and it was way simpler than I thought.
What surprised me was how dumb the whole process was. My questions certainly weren't the problem - as bad as they possibly were, the solution wasn't too far off. Not only ChatGPT had become a crutch but it put me in a situation that no human-tutor would ever put me.
So removing the pain of quick interactions with a tutor has benefits but the technology is not quite ready to be considered as true guidance in forming (or even helping) someone's understanding of a subject.
I've been using ChatGPT a lot for general language but when it requires logical thinking it falls pretty flat.
I've experienced this exact frustration loop before, but I've also got a major counter-example.
I've been using GPT4 as a tool to interrogate lesson transcripts for a language I'm learning and mention in the prompt to specifically focus on things mentioned in the transcript, if it's not in the transcript, check the helper script I update as I move on through the process (which does sadly take up more and more context window) and figure out if my answer is in one of those previous lessons, and to not guess. Hallucinations are quite rare, I don't think I can name an egregious instance of it in the 25 lesson I've done of approximately 20 minutes in length each, though I'm sure it's happened.
It's also pretty good at suggesting drills based on the contents of the lesson, there are probably a whole bunch of lesson plans in the training data.
The end result has been progress at a pace I could only dream of previously, and it doesn't matter if a question is too basic because I'm asking a computer. There is zero concern of any question being embarrassing because it's only between me, GPT4, and the OAI engineer who happens across the conversaiton.
I've been using GPT4 as a tool to interrogate lesson transcripts for a language I'm learning and mention in the prompt to specifically focus on things mentioned in the transcript, if it's not in the transcript, check the helper script I update as I move on through the process (which does sadly take up more and more context window) and figure out if my answer is in one of those previous lessons, and to not guess. Hallucinations are quite rare, I don't think I can name an egregious instance of it in the 25 lesson I've done of approximately 20 minutes in length each, though I'm sure it's happened.
It's also pretty good at suggesting drills based on the contents of the lesson, there are probably a whole bunch of lesson plans in the training data.
The end result has been progress at a pace I could only dream of previously, and it doesn't matter if a question is too basic because I'm asking a computer. There is zero concern of any question being embarrassing because it's only between me, GPT4, and the OAI engineer who happens across the conversaiton.
I absolutely have the same overall feeling, when the task at hand is related to text processing (including understanding and spinning off new takes or ideas from it).
But when the task at hand involves logical thinking, that's when I believe the LLMs of today are still a very much work in progress.
So I'm skeptical of trying to use them as tutors for now. I'm sure things will evolve quite quickly from now.
But when the task at hand involves logical thinking, that's when I believe the LLMs of today are still a very much work in progress.
So I'm skeptical of trying to use them as tutors for now. I'm sure things will evolve quite quickly from now.
I'm skeptical of this takeaway.
"I spent days trying to solve X by doing Y, then it turned out I could have solved it by doing Z instead" is an experience I've had countless times before LLMs were a thing. Sometimes you really do need these three days of stumbling before you can build up the confidence to do the easy solution.
(Then again, I don't know the specifics of your case.)
"I spent days trying to solve X by doing Y, then it turned out I could have solved it by doing Z instead" is an experience I've had countless times before LLMs were a thing. Sometimes you really do need these three days of stumbling before you can build up the confidence to do the easy solution.
(Then again, I don't know the specifics of your case.)
In addition, I feel like you build a sort of intuition after a while and detect when the conversation with the LLM has hit a dead end. When to stop and take a step back, think what it is you're trying to do and try to go down a different path. The LLM can even support with that, it's on the human to kick that off, at least at this point in time.
I can't really disagree with you. Although I want to believe this problem is more exacerbated with LLMs than with human-tutors.
I have no evidence other than hundreds of hours using ChatGPT.
I have no evidence other than hundreds of hours using ChatGPT.
> Although I want to believe this problem is more exacerbated with LLMs than with human-tutors.
Well, yeah, but the central insight here is that LLMs enable a worse-is-better approach with a tighter feedback loop. They're not as good as a regular tutor, but a regular tutor costs money, gets impatient, is only available at set hours, etc.
Part of the appeal of learning-by-LLM is that you can get a flash of motivation at 2 AM and go "hey, I should totally learn about X, that would be cool!", open up ChatGPT, ask some very naive questions, and get just enough to get started.
Well, yeah, but the central insight here is that LLMs enable a worse-is-better approach with a tighter feedback loop. They're not as good as a regular tutor, but a regular tutor costs money, gets impatient, is only available at set hours, etc.
Part of the appeal of learning-by-LLM is that you can get a flash of motivation at 2 AM and go "hey, I should totally learn about X, that would be cool!", open up ChatGPT, ask some very naive questions, and get just enough to get started.
(It's funny, when I wrote this yesterday I thought "this is an unrealistic example", and yet here I am, at 1 AM, asking ChatGPT a bunch of questions about Google's XLS. I'm pretty sure the answers I got were hallucinations, but at least it helped me formulate the questions for when I go to the mailing list.)
> I wouldn't be surprised to see people using it to simply solve exercises for them, and those people will have a hard time trying to do things on their own.
This is a more recent version of the argument against most machines and similar automation tools. There is usually some validity to it, but we saw the same arguments against MOOCs, calculators, computers, type writers, smart phones, spell checkers, etc.
This is a more recent version of the argument against most machines and similar automation tools. There is usually some validity to it, but we saw the same arguments against MOOCs, calculators, computers, type writers, smart phones, spell checkers, etc.
They should be smart enough to know when you ask the wrong questions. Perhaps they aren't there yet, but that's likely where they are going.
Which is also an interesting problem isn't it? The majority of people will only be doing what the LLM suggests...
The LLM's best role is this context is as a trusted, knowledgeable advisor/teacher. It works so well because there's no negative psychology that can be introduced, because you know the LLM isn't a human.
Like the commenter for this thread was saying, there's no fear of asking a stupid question. Or going to slowly or too quickly.
Like the commenter for this thread was saying, there's no fear of asking a stupid question. Or going to slowly or too quickly.
Absolutely. Fail fast, fail often. It makes it very easy to quickly try out ideas before you go down a rabbit hole. However I think for people that are completely new to programming it could quickly become a crutch.
Seconded. And it's not just programming. Any subject that is easy to moderately verifiable like most Math and Science in school can also benefit from this. We're one step closer to personal tutors and autodidactism for everyone. The other half of the learning journey would be arranging the curriculum for yourself.
If only more people would focus on cultivating learning instead of locking down tech like this because they're afraid (and perhaps lazy to create new teaching methods).
If only more people would focus on cultivating learning instead of locking down tech like this because they're afraid (and perhaps lazy to create new teaching methods).
I still have very mixed feelings about using LLMs to learn about things. When it works and produces valid information, it's absolutely amazing, and I find it much easier to find and grasp things than simply googling and reading through various pages hoping to piece together what I'm looking for. The problem is that a lot of the time it doesn't, and it's such a convincing liar that it's hard to tell, especially when you don't already have comprehensive knowledge about the subject at hand.
I'm finding that the stronger the LLM, the farther the 'frontier' of obscurity before it hallucinates gets. Mistral-7B-instruct is nice to talk to, but it's pretty easy to come up with a topic that it knows nothing about. ChatGPT4 is a lot better, but past a certain point it'll start going wonky too.
For learning to code I suspect that with a strong enough version of GPT a new user won't hit the limits, and if it goes wrong having to debug isn't a bad thing ;)
For learning to code I suspect that with a strong enough version of GPT a new user won't hit the limits, and if it goes wrong having to debug isn't a bad thing ;)
I'm having this issue currently with Elasticsearch and Opensearch. I use libraries for both (since we're in the middle of migration) and not only does it conflate the two, but it even confuses versions of Elasticsearch that have breaking changes, so it ends up recommending all this convincing code that is based on non-existent or wrong-version library APIs. Drives me nuts.
Anything to do with Powershell/Azure is this x1000. It constantly recommends libraries that do not exist, to call API endpoints that were deprecated or never existed. It's maddening.
I wish more interfaces supported intake of PDFs/URLs/HTML content as domain material. Then at least we'd be able to point the bot at the most current docs and have it most-heavily weight that information.
I wish more interfaces supported intake of PDFs/URLs/HTML content as domain material. Then at least we'd be able to point the bot at the most current docs and have it most-heavily weight that information.
I believe, that when people learn programming mainly from LLMs, they will navigate themselves into a corner faster than with regular learning. i.e. they learn anti-patters and won't be able to get out of they local optimum.
On the other hand: that's happening today already en masse, so I am not sure if it can actually get worse.
I agree, that LLMs as an addition to the established resources (teachers, books, youtubers, etc.) will have lots of value.
On the other hand: that's happening today already en masse, so I am not sure if it can actually get worse.
I agree, that LLMs as an addition to the established resources (teachers, books, youtubers, etc.) will have lots of value.
Interestingly this assumes that the goal is learning how to manually write code, rather than learning how to get a LLM to generate code for you, or learning how to build a product.
If you believe the future of coding sits in AI/LLMs/ML, this next generation of learners might just be using the 'beta' tools-of-tomorrow, rather than just being people that need to learn to manually code.
(My personal view is that manual coding will be changed almost unrecognisably within 5-10 years, but lets see! So much of learning to program is about learning syntax, and I can see a world of AI-powered-future-programming-languages where syntax is less important than clarity)
If you believe the future of coding sits in AI/LLMs/ML, this next generation of learners might just be using the 'beta' tools-of-tomorrow, rather than just being people that need to learn to manually code.
(My personal view is that manual coding will be changed almost unrecognisably within 5-10 years, but lets see! So much of learning to program is about learning syntax, and I can see a world of AI-powered-future-programming-languages where syntax is less important than clarity)
How many years until manual programming, as we know it, will be obsolete?
Meaning that there won't be coders that sit around punching code all day, but rather just generate diagrams and/or feed models with descriptions of systems, and in return get fully functional systems.
5 years? 10 years? 15 years? 20?
Meaning that there won't be coders that sit around punching code all day, but rather just generate diagrams and/or feed models with descriptions of systems, and in return get fully functional systems.
5 years? 10 years? 15 years? 20?
Bug fixing via gpt interface is going to be fun! It needs to be far better than humans to avoid the following situation, and at that point these models have taken over basically every office job, so at that point you wont have to go through this process anyway.
You: Here is a bug: [explanation]
GPT: Ok, fixed it!
You: It is still there.
Gpt: Ok, now fixed for real!
You: It is still there...
You: Here is a bug: [explanation]
GPT: Ok, fixed it!
You: It is still there.
Gpt: Ok, now fixed for real!
You: It is still there...
You're right, and I like the joke, but we should be clar that LLM != GPT != ChatGPT.
There's a whole field emerging of prompt engineers, but I think what's desperately missing is good quality integrated UX engineering.
Being able to write a comment and/or a method signature and having an LLM spit out a generated piece of code is kinda cool, but the interface for showing that the code is buggy needs refining. I think that's likely to be quickly showing a test case: given these inputs I'm expecting this output. LLM refactors against all tests, and you iterate together towards the correct solution.
That's one small example of how we can start to think about the future UX that isn't about having a "chat" about your code.
Similar UX thought is needed in other disciplines, it'll be interesting to see how this domain evolves.
There's a whole field emerging of prompt engineers, but I think what's desperately missing is good quality integrated UX engineering.
Being able to write a comment and/or a method signature and having an LLM spit out a generated piece of code is kinda cool, but the interface for showing that the code is buggy needs refining. I think that's likely to be quickly showing a test case: given these inputs I'm expecting this output. LLM refactors against all tests, and you iterate together towards the correct solution.
That's one small example of how we can start to think about the future UX that isn't about having a "chat" about your code.
Similar UX thought is needed in other disciplines, it'll be interesting to see how this domain evolves.
We should probably also be clear that LLMs aren’t the only ML models that can learn to code. If hands-on programming is automated away, it’s not obvious that it’ll be an LLM that does it.
I have already built a system that puts ChatGPT in a loop with the output of programs and running commands. It can be pretty effective.
It's even more effective when you can limit the domain somewhat to things that can fit in the context window, which eliminates out of date API information.
We will now shortly be able to add visual feedback of the applications into the loop.
It's even more effective when you can limit the domain somewhat to things that can fit in the context window, which eliminates out of date API information.
We will now shortly be able to add visual feedback of the applications into the loop.
[deleted]
My guess is it's sooner than later. First to go would be UI programmers and automation testers. Last would probably be the hardware interfacing engineers like embedded etc. May be writing code manually would be equivalent of using punch cards or artisan bread :)
Frankly I feel like we are looking at this the wrong way. In the future we might not even have a way to program some things manually. There might be a llm for each device that generates necessary hardware instructions on the fly like today's jit based on natural language or environment. May be there would be a inter llm spec/language that would felicitate cross llm functionality. All short circuiting out the programmers between the user and the utility.
Frankly I feel like we are looking at this the wrong way. In the future we might not even have a way to program some things manually. There might be a llm for each device that generates necessary hardware instructions on the fly like today's jit based on natural language or environment. May be there would be a inter llm spec/language that would felicitate cross llm functionality. All short circuiting out the programmers between the user and the utility.
UI programmers have to deal with a ton of ambiguity and their job is literally to understand how humans will use a program. I have no idea how the hell you can come to the conclusion that this would be easier to automate than embedded.
Like the other comments say, most likely the product owner describes his requirement in as much detail as possible and the phase-1 LLM will convert that to designs. Based on the feedback, it will be iterated and may be 3-4 designs selected for A/B testing.
The phase-2 LLM converts the designs to code in either react/flutter (both of which current LLMs are already quite good at) and deploy for a certain set of users. Basically same flow as today minus the engineers. Most of this is even possible today. The only thing saving engineering necks is the context window size which prevents fitting the whole repo in memory. Hopefully we (the programmers) will build our savings to a sustainable level in the next 5 years before signing off on manual coding or finding another thing to do than typing on keyboards to make computers dance.
The phase-2 LLM converts the designs to code in either react/flutter (both of which current LLMs are already quite good at) and deploy for a certain set of users. Basically same flow as today minus the engineers. Most of this is even possible today. The only thing saving engineering necks is the context window size which prevents fitting the whole repo in memory. Hopefully we (the programmers) will build our savings to a sustainable level in the next 5 years before signing off on manual coding or finding another thing to do than typing on keyboards to make computers dance.
> the product owner describes his requirement in as much detail as possible
This is basically science fiction
This is basically science fiction
The current LLMs were science fiction 10 years ago. It's difficult enough to imagine what the tech will be like 10 years from now, let alone 20 years from now.
> > the product owner describes his requirement in as much detail as possible
> This is basically science fiction
Product owners describing requirements in detail! More like fantasy, I think.
> This is basically science fiction
Product owners describing requirements in detail! More like fantasy, I think.
It’s not just science fiction, it’s a software engineering deus ex machina.
Considering speech is becoming the primary interface for HCI via LLMs, that alone may reduce the need for UI design.
Isn’t that the designers job? You don’t need to actually write code to do UX
Next generation of programmers will be incredible. I don't know if it will make better programmers but there definitely will be more now that there is even less friction to learn. Similar to the big boom in interactive coding tutorials allowing more self learning devs.
I'm not sure about that. Some amount of devils-advocacy in the following but here goes:
If you're learning to code now, using an LLM, who's to say that the LLMs won't improve faster than you do? In 10 years will there be a recognisable programming job for most people learning programming now? We're standing on a cliff watching the tide rise, wondering if it'll reach us. People learning programming are down in the sea, trying to climb up out of it.
Years ago taxi drivers actually needed to know their way around their local area. Then we got GPS and mapping apps, meaning Uber could exist and any random dude could be a taxi driver instantly. In future* the drivers will be unnecessary and the cars will be the drivers too. [*timeline 5-50 years]
In the early 2000s there were plenty of jobs for mediocre developers making websites for small companies. Now we've got Facebook, Squarespace & Shopify, that role is gone.
Lots of observations around here recently about how kids are less tech-savvy now, they know how to use appliances - not general purpose computers. Make stuff easy enough that anyone can do it, and nobody will know how it works.
If you're learning to code now, using an LLM, who's to say that the LLMs won't improve faster than you do? In 10 years will there be a recognisable programming job for most people learning programming now? We're standing on a cliff watching the tide rise, wondering if it'll reach us. People learning programming are down in the sea, trying to climb up out of it.
Years ago taxi drivers actually needed to know their way around their local area. Then we got GPS and mapping apps, meaning Uber could exist and any random dude could be a taxi driver instantly. In future* the drivers will be unnecessary and the cars will be the drivers too. [*timeline 5-50 years]
In the early 2000s there were plenty of jobs for mediocre developers making websites for small companies. Now we've got Facebook, Squarespace & Shopify, that role is gone.
Lots of observations around here recently about how kids are less tech-savvy now, they know how to use appliances - not general purpose computers. Make stuff easy enough that anyone can do it, and nobody will know how it works.
We somehow have more programming jobs now than ever. Those kids learning programming with LLMs may not end up writing code by hand but they will still be building stuff with computers.
We shouldn't take for granted that will continue for very long because it’s obviously as unsustainable as infinite economic growth in general. At some point that growth has to stop.
Up till now software was mostly an arms race between competitors because of software’s low marginal cost and high utility. Now with ZIRP over and many of parts of the industry consolidating, I fear enshittification is a sign that the good times are over and we will now be milked to make up for all the profit lost the industrialists lost to the plebs.
Up till now software was mostly an arms race between competitors because of software’s low marginal cost and high utility. Now with ZIRP over and many of parts of the industry consolidating, I fear enshittification is a sign that the good times are over and we will now be milked to make up for all the profit lost the industrialists lost to the plebs.
Absolutely not. There is a point where ease of use decrease the knowledge of the users. Most "digital natives" are worse with technology than Gen X and early millenial, simply because they didn't have to learn how it worked to use it
My biggest fear is what has happened with Gen Z. The UX becomes so user friendly that you end up never needing to learn how it actually works. Great for happy paths, but terrible if you need to dig into edge cases and work under the hood. We all know the classic case of the junior dev who just brute forces stack overflow into their code and prays it works; this is the next iteration of that.
Will it? I mean there is stack overflow already. All it is is a better UI for stack overflow.
Exactly. Stackoverflow can be kind of a mess though. You'll often be sifting through tons of answers or might still be confused and asking follow up questions can take time to get an answer. Now that is all immediate.
Ah yes, Stack Closed. This question needs to be more focused. It is not currently accepting answers. duplicate, deleted Overflow. Such a wonderful site, really useful for a beginner.
Would be funny if chatgpt starts answering that to certain questions.
Well, that may be true but because it's a 'better UI' while also being completely independent of Stack Overflow, it has been hammering SO's traffic recently.
The reason stack overflow is such a useful resource is that people actually ask questions on it and receive high quality answers. I wonder, how long until the website becomes obsolete because people are asking LLMs instead, and how long after that do LLMs themselves consequently become less useful?
> that people actually ask questions on it and receive high quality answers
That really hasn't been my experience with StackOverflow for the last 10 years.
That really hasn't been my experience with StackOverflow for the last 10 years.
Fair enough. Regardless, as I understand it LLMs can only be as good as their training data. Therefore they are vulnerable to that data losing its freshness.
Wouldn't it seem reasonable to assume that any new technology will train an LLM with extensive documentation and tutorials before being released. GPT4 can browse the web now too and open source LLMa can be trained on custom data.
That is reasonable and I believe it would greatly help. However, docs and tutorials rarely anticipate all the ways in which people need to use that technology.
But LLMs are getting the "upvote" as feedback...more the usage, more they improve as well?
I don't know how LLM upvotes work, but I'm guessing it's typically only the direct user that will upvote or downvote. The value of a public forum voting system is that a pool of experienced users can verify whether an answer is of good quality.
If an inexperienced engineer asks for code and the LLM produces something that has sub-par performance, is overly complex, and botches various edge-cases, the engineer may not realise until years later if at all that they could have produced something much better.
Combined with the fact that there are plenty of things that mostly only junior engineers will ask about, and I think you have a problem.
If an inexperienced engineer asks for code and the LLM produces something that has sub-par performance, is overly complex, and botches various edge-cases, the engineer may not realise until years later if at all that they could have produced something much better.
Combined with the fact that there are plenty of things that mostly only junior engineers will ask about, and I think you have a problem.
How long? It started months ago.
That's the fundamental reason why there was such a falling out between Stack Overflow executives and the moderators.
People will say it was just the way they handled the interactions, which was part of it, but I don't believe there was any way to do it in a way that the users wouldn't hate.
Because the basic task for Stack Overflow IS to replace the users answering the questions with AI. They don't have a choice. Because now they are already competing with ChatGPT etc. doing exactly that.
It's true that taking away human answers from Stack Overflow will make it impossible for AI to answer many of those types of questions. But it will still retain fundamental knowledge and skills from other sources. And can be trained on its own debugging experiences as it gets deployed for building software. So although that presents a challenge, it's not insurmountable at all.
That's the fundamental reason why there was such a falling out between Stack Overflow executives and the moderators.
People will say it was just the way they handled the interactions, which was part of it, but I don't believe there was any way to do it in a way that the users wouldn't hate.
Because the basic task for Stack Overflow IS to replace the users answering the questions with AI. They don't have a choice. Because now they are already competing with ChatGPT etc. doing exactly that.
It's true that taking away human answers from Stack Overflow will make it impossible for AI to answer many of those types of questions. But it will still retain fundamental knowledge and skills from other sources. And can be trained on its own debugging experiences as it gets deployed for building software. So although that presents a challenge, it's not insurmountable at all.
> The basic task for Stack Overflow IS to replace the users answering the questions with AI. They don't have a choice.
By going in that direction, I feel like they put the nail in the coffin. Why would I use their website when I can just ask ChatGPT directly?
I think they had one other option, which was to identify the fundamental limitations of LLMs, and find a way to demonstrate value over and above.
By going in that direction, I feel like they put the nail in the coffin. Why would I use their website when I can just ask ChatGPT directly?
I think they had one other option, which was to identify the fundamental limitations of LLMs, and find a way to demonstrate value over and above.
People were already just asking ChatGPT.
I don't believe there is any such fundamental limitation, especially when you look say two years down the road and don't limit yourself to language-only models or the AI models of today. Already GPT-4 has audio and image input.
Good human programmers today have better abilities and more current knowledge overall than GPT-4. But it's a matter of degree, and a matter of time.
What types of limitations do you suppose? And again, we have to anticipate that these technologies evolve rapidly. LLM is a misleading term in my opinion for a fundamentally multimodal model. And totally new types of models or model integrations will be likely be coming out over the next few years that will make current systems obsolete. Such as combining AlphaZero-type systems with language models.
I don't believe there is any such fundamental limitation, especially when you look say two years down the road and don't limit yourself to language-only models or the AI models of today. Already GPT-4 has audio and image input.
Good human programmers today have better abilities and more current knowledge overall than GPT-4. But it's a matter of degree, and a matter of time.
What types of limitations do you suppose? And again, we have to anticipate that these technologies evolve rapidly. LLM is a misleading term in my opinion for a fundamentally multimodal model. And totally new types of models or model integrations will be likely be coming out over the next few years that will make current systems obsolete. Such as combining AlphaZero-type systems with language models.
> What types of limitations do you suppose?
I don't see the capacity for LLMs to perform novel and creative reasoning tasks. They're limited by virtue of being a pattern-matching system without the ability to make genuine causal or logical inferences.
I wrote here about my recent experiments with ChatGPT - https://news.ycombinator.com/item?id=37583333
I'm vaugely interested to know how GPT4 deals with questions like these, but I'm not willing to pay to find out.
I don't see the capacity for LLMs to perform novel and creative reasoning tasks. They're limited by virtue of being a pattern-matching system without the ability to make genuine causal or logical inferences.
I wrote here about my recent experiments with ChatGPT - https://news.ycombinator.com/item?id=37583333
I'm vaugely interested to know how GPT4 deals with questions like these, but I'm not willing to pay to find out.
Lol. So even the dumb version actually did your task, just not in a very convincing or "genuine" way. Pattern matching can be used for inference and GPT-4 does it very effectively. 3.5 turbo is relatively brittle and maybe 20 IQ points lower. But still, it did your task.
Anyway, it will only be a year or two before things like tree search over problems spaces are explicitly integrated into models or systems similar to this. Deep Mind may have already done something like that.
Anyway, it will only be a year or two before things like tree search over problems spaces are explicitly integrated into models or systems similar to this. Deep Mind may have already done something like that.
Plus where will chatgpt take new stuff from?
Whatever is new will probably be designed for chatgpt to understand first. Maybe people won't bother using your tool if it can't be used without a AI chat interface helping. The same way I won't learn a programming language if the documentation is bad. Tool makers knowing this will make chatgpt specific technical documents that can be easily parsed by an LLM.
Stackoverflow is about expert programmer answering questions from novice programmers (it can be novice on a specific tech, not necessarily junior devs). How will chatgpt learn from expert programmers, who don't need chatgpt to code something up.
My theory is that experts will be hired to train specific models. Documentation will be written to be understood my an LLM chat interface. Edge cases and tutorials will be pre-made and then fed to the model. Technical writing might be skill that will be in huge demand.
Looking forward to the controlled experiments comparing groups learning with and without.
They are also pretty good in teaching you human languages. There is a saying that the best way to learn a new language is to fall in love with someone who speaks it. ChatGPT speaks them all. Start with "Show me some simple phrases in Chinese and in English..."
What is programming if not a structured language for describing tasks to a machine. We are not there yet, but outside of the world of core technology, most of the programming we will see in the near future will be essentially telling the machine what to do in natural language. So if you want to have some form of job security my recommendation is to go lower level and work on the fundamental pieces of tech because the higher level stuff will be automated away.
Right. It seems like LLMs can be both a huge advantage for one-on-one instruction, but at the same time could just do many tasks for you.
Code generation AI systems will continue to get better rapidly.
They are not just language models anymore but now can include image input and output.
There will be systems trained (or context "trained") for software platforms that put a wide variety of task configurations within easy reach by translating to relatively small amounts of DSL code.
The next revolution may come from applying things like Monte Carlo search to concrete domains defined on the fly by LLMs. Or just a tight integration between implementation and testing within the platform and AI loop.
Also consider the programming ability increases when models are trained to be able to unroll state very accurately within their platform domains as part of the software design process.
Looking anywhere from a few months out to say five years, we should anticipate increasingly capable and more general AI programming systems.
Who knows how long it will take for these systems to be fully deployed in all areas. But beyond five years or so, it is kind of hard to imagine what the leading-edge capabilities of these types of systems might be.
Maybe no one will want to buy a particular computer game or application anymore. Instead, you buy an AI programmer that comes with a kit for configuring and even coding some custom parts of software specifically tailored for you.
Maybe the only thing that really sells at some point are AI asset streaming services which allow your existing AI swarms to on demand pull in new neural radiance models, behavior models, neural world state models, neural avatar models, domain experts, document drafters, etc.
These will be integrated with mixed reality. So you are looking at basically portable Holodeck providers. The commodification of all types of skills and knowledge, manifested in person on demand.
Code generation AI systems will continue to get better rapidly.
They are not just language models anymore but now can include image input and output.
There will be systems trained (or context "trained") for software platforms that put a wide variety of task configurations within easy reach by translating to relatively small amounts of DSL code.
The next revolution may come from applying things like Monte Carlo search to concrete domains defined on the fly by LLMs. Or just a tight integration between implementation and testing within the platform and AI loop.
Also consider the programming ability increases when models are trained to be able to unroll state very accurately within their platform domains as part of the software design process.
Looking anywhere from a few months out to say five years, we should anticipate increasingly capable and more general AI programming systems.
Who knows how long it will take for these systems to be fully deployed in all areas. But beyond five years or so, it is kind of hard to imagine what the leading-edge capabilities of these types of systems might be.
Maybe no one will want to buy a particular computer game or application anymore. Instead, you buy an AI programmer that comes with a kit for configuring and even coding some custom parts of software specifically tailored for you.
Maybe the only thing that really sells at some point are AI asset streaming services which allow your existing AI swarms to on demand pull in new neural radiance models, behavior models, neural world state models, neural avatar models, domain experts, document drafters, etc.
These will be integrated with mixed reality. So you are looking at basically portable Holodeck providers. The commodification of all types of skills and knowledge, manifested in person on demand.
The ability to use natural language as an input to produce code isn't going to eliminate the profession. It will provide an onramp for learning, and allow civilians to learn easier (not faster necessarily) and automate some things themselves, which is a good thing.
An LLM is never going to be able to figure out what you need written, and bug-free code is a big ask, putting it mildly. Even structurally correct code is a bug if it doesn't actually do what you want. Someone still has to understand the problem domain, design an architecture to solve it, and verify that it does what it's supposed to do, this is a professional position, no way around it.
Where we are now, Copilot and friends are a nice assist for drudge work, a way to learn new libraries/APIs/languages, but a danger to beginners (you have to be able to proof the code it produces, it will still frequently make straightforward errors) and for experts it's less useful. As they improve I expect them to become an essential part of the profession, but they aren't actually able to replace it.
An LLM is never going to be able to figure out what you need written, and bug-free code is a big ask, putting it mildly. Even structurally correct code is a bug if it doesn't actually do what you want. Someone still has to understand the problem domain, design an architecture to solve it, and verify that it does what it's supposed to do, this is a professional position, no way around it.
Where we are now, Copilot and friends are a nice assist for drudge work, a way to learn new libraries/APIs/languages, but a danger to beginners (you have to be able to proof the code it produces, it will still frequently make straightforward errors) and for experts it's less useful. As they improve I expect them to become an essential part of the profession, but they aren't actually able to replace it.
What's "lower level" in this context? And why shouldn't we assume that assembly language or even circuit design will also be affected by describing in natural language?
We've heard variations on this argument for decades and I see no difference why LLMs, despite their immense capabilities, will be massively different. They still take guidance and produce garbage output which requires knowledge and expertise to turn into a cohesive product. Will they replace some script writing, or some business logic translation? Of course. But so did SQL, no-code, and plain-English testing frameworks. LLMs are an incredibly powerful tool but they're not a silver bullet.
We've heard variations on this argument for decades and I see no difference why LLMs, despite their immense capabilities, will be massively different. They still take guidance and produce garbage output which requires knowledge and expertise to turn into a cohesive product. Will they replace some script writing, or some business logic translation? Of course. But so did SQL, no-code, and plain-English testing frameworks. LLMs are an incredibly powerful tool but they're not a silver bullet.
There's nothing stopping LLMs from being good at low-level stuff.
I feel LLMs are not at the point to tell me I'm wrong yet, it feels like it avoids to do it because it doesn't want to be rude.
And that sucks, big part of the teaching process is the feedback loop between students and teachers doing exactly that.
I still wonder why there is not more emphasis in these new times on writing requirements.