The age of machine learning as code has arrived(huggingface.co)
huggingface.co
The age of machine learning as code has arrived
https://huggingface.co/blog/the-age-of-ml-as-code
56 comments
To add to that my current struggle: Most stuff is geared towards conda and terrible engineering practices (requirements.txt and not even locked versions). If you want to actually build and deploy something on actual hardware, you are in a world of hurt. And even for training the dependency situation is so ridiculous that most repos come with their own Dockerfile.
I marvel at the cool machine learning demos but I'm kind of sick working on this stuff, tbh.
I marvel at the cool machine learning demos but I'm kind of sick working on this stuff, tbh.
Also, nvidia-docker...
I have tried using Google Cloud - I spent a lot of time finding a location and getting the permissions to use GPUs. Then one time after I switched off the vm to economize on costs I could not switch it on again because of lack of resources. I don't know if I could migrate it somewhere else - but most probably not, because the GPUs setups are pretty unique. After a few weeks I gave up. I bought a laptop with nvidia. I switched off everything that I thought was to switch off. Then for 4 months I was constantly being reminded about something more to switch off. Now I still have a static IP address and I don't know how to switch it off because the VM it was attached to is deleted.
By the way as a seasoned (and actually kind of retired) programmer I started learning ML by myself after deciding that I don't like the code I was getting from hired companies. It is a strange land with many software writing practicies surprising someone coming from different programming culture. I wish I could find a community of people who made a similar path - here is my blog: https://zby.github.io/.
By the way as a seasoned (and actually kind of retired) programmer I started learning ML by myself after deciding that I don't like the code I was getting from hired companies. It is a strange land with many software writing practicies surprising someone coming from different programming culture. I wish I could find a community of people who made a similar path - here is my blog: https://zby.github.io/.
Can you name some breaking changes PyTorch made?
Not off-hand because it's been a while, but IIRC the names of some methods were changed, arguments were added/removed. I will say that was before version 1.0, so it may be more stable now.
Last I saw the nvidia driver situation did not improve though. I wanted to come back to some old code, installed a new version of PyTorch. It complained that my GPU drivers were not recent enough, had to do this long tedious process that took over an hour to get newer drivers working.
Last I saw the nvidia driver situation did not improve though. I wanted to come back to some old code, installed a new version of PyTorch. It complained that my GPU drivers were not recent enough, had to do this long tedious process that took over an hour to get newer drivers working.
[deleted]
The trick is to get a disk that works; wire it up to boot up, run your job, then shut down, on whatever schedule you need; then never touch it for anything besides business logic changes.
Brownie points are awarded for persisting the setup of the image in the form of a dockerfile, shell script, etc., but such documents are to be considered non-authoritative sources, and must be expected to break at a rate roughly proportional to their size.
Brownie points are awarded for persisting the setup of the image in the form of a dockerfile, shell script, etc., but such documents are to be considered non-authoritative sources, and must be expected to break at a rate roughly proportional to their size.
> A few years ago, Forbes wrote that "Software ate the world, now AI is eating Software"
As far as I know, AI is Software, so this type of reasoning really confuses me.
I think it is possible that "developers" might have some of their work find automation using AI and ML, but Computer Science will obviously stick around as long as computers are around.
For example, someone has to code the algorithms which are used to teach the computer how to learn. Someone has to code the pipelines for capturing data from the physical world into the digital world. Someone has to code the storage and exchange of that data to the ML models, and someone has to plug the inference into the production processes and systems. Someone has to put alarms and monitors around all these things to assert their operation. Someone has to code the UIs and interfaces that users will use to interact with all those systems. Finally, someone needs to code compatibility and optimizations of all these models and their implementation to continue to work and leverage new hardware capabilities.
If people think that developers were spending 90% of their time hand tuning business rules and coming up with manually implemented inference and decision models they are highly mistaken. That could have been 5% of the job in some circles, but 95% of the job has always been all these peripheral tasks.
If anything, there will be even more work now, since ML models require a lot more periphery to develop, train and ship.
As far as I know, AI is Software, so this type of reasoning really confuses me.
I think it is possible that "developers" might have some of their work find automation using AI and ML, but Computer Science will obviously stick around as long as computers are around.
For example, someone has to code the algorithms which are used to teach the computer how to learn. Someone has to code the pipelines for capturing data from the physical world into the digital world. Someone has to code the storage and exchange of that data to the ML models, and someone has to plug the inference into the production processes and systems. Someone has to put alarms and monitors around all these things to assert their operation. Someone has to code the UIs and interfaces that users will use to interact with all those systems. Finally, someone needs to code compatibility and optimizations of all these models and their implementation to continue to work and leverage new hardware capabilities.
If people think that developers were spending 90% of their time hand tuning business rules and coming up with manually implemented inference and decision models they are highly mistaken. That could have been 5% of the job in some circles, but 95% of the job has always been all these peripheral tasks.
If anything, there will be even more work now, since ML models require a lot more periphery to develop, train and ship.
I don't think it means anything really, just the author though it sounds clever.
Software is also part of the world, just a very different part.
True, though I'm assuming by "the world", they mean that traditional processes which would have involved pen and paper and other non computer based tools and systems are all being redesigned to make use of computers, thus Software is eating them.
But AI is not replacing processes that rely on software with something that no longer relies on software. What AI is doing is it lets you replace even more processes that were not yet able to be handled by a computer with one that a computer can do, and thus it just helps Software eat even more of the world. Decision making and judgement tasks were not able to be successfully moved to computers prior to the recent advancements in AI, now some of those can be.
But AI is not replacing processes that rely on software with something that no longer relies on software. What AI is doing is it lets you replace even more processes that were not yet able to be handled by a computer with one that a computer can do, and thus it just helps Software eat even more of the world. Decision making and judgement tasks were not able to be successfully moved to computers prior to the recent advancements in AI, now some of those can be.
I think this is a reference class problem. One could also say software replaced work done in offices with work still being done in offices. (until recently) Most of the work was still done on literal desktops. but significant changes have been happening within the subset of the office and within the use of the desk.
Once upon a time you could make a good living building business websites (i.e. the local florist) but even without AI and ML frameworks have already killed that dead - your florist now probably uses instagram as their website, they might use Squarespace - and if they're hardcore they use Joomla, Wordpress or Drupal. I think a lot of the media thinks that developer hours are spent proportional to the stuff they see and this is the crux of the matter. Ford isn't retaining a team of two hundred devs to maintain their sales pages - it might be like ten or so? While an insurer managing a claims system is probably retaining a boatload of FT devs and also shipping out a bunch of hours to consulting firms. Software programming labour is very much an iceberg problem - the things that look expensive and difficult are trivial and those things are easy to automate away - but we're all working on the hard shit.
There's nice money to be made configuring/writing custom modules for Wordpress and Drupal. Your business won't become a unicorn but there's a living to be made on top of those technologies. The same thing when it comes to Shopify and Magento.
The funny thing is that now, many smaller shops are catching up to what Shopify, Lodify etc do, and offering niche solutions to problems which people are actually interested in paying for, so while a lot of these players were first to take over many smaller players / individuals. People had a lot of time to sit around and tinker and build alternatives.
> Once upon a time you could make a good living building business websites (i.e. the local florist)
Hum, back that up, because I don't think you could ever make a good living with that. No florists is going to pay you the same salaries that going to work for Shopify pays you, small local business just didn't have the means to hire someone to build a website, that's why there was a market for things like Drupal, WordPress and Shopify in the first place. So in reality, those frameworks created jobs, because now you can actually be paid by a local florist to setup and manage a WordPress for them, and the equation works out that for what they pay you and the time it takes you is now profitable. And you have all the jobs opened up from the company developing the frameworks as well.
Hum, back that up, because I don't think you could ever make a good living with that. No florists is going to pay you the same salaries that going to work for Shopify pays you, small local business just didn't have the means to hire someone to build a website, that's why there was a market for things like Drupal, WordPress and Shopify in the first place. So in reality, those frameworks created jobs, because now you can actually be paid by a local florist to setup and manage a WordPress for them, and the equation works out that for what they pay you and the time it takes you is now profitable. And you have all the jobs opened up from the company developing the frameworks as well.
I've got a friend who coasted through the early 2000s plopping out websites for random businesses for around 1k a pop - for a weeks work that's a pretty good take home.
The florist example was probably a poor choice since they tend to have pretty thin margins - but restaurants (while also not having a bunch of money) tend to have advertising budgets that can float a relatively modest cost for plopping out a website.
The florist example was probably a poor choice since they tend to have pretty thin margins - but restaurants (while also not having a bunch of money) tend to have advertising budgets that can float a relatively modest cost for plopping out a website.
Sounds like an ok gig, until you have to support them. I used to throw together and host little sites for friends for free, now I'd rather keep the friendships intact and tell people to just use Squarespace or some other builder.
> 1k a pop - for a weeks work that's a pretty good take home
52k a year with no time off / benefits?
52k a year with no time off / benefits?
That wasn't bad in the early 2000s. Median U.S. household income was $42K in 2000, and you didn't have these $half-mil/year FAANG/unicorn jobs back then.
44k with 8 weeks off sounds better.
Glassdoor says a web dev freelancer today in the US averages 30$ per hour and it can range from 60k to 80k a year.
So I don't, know, maybe their data is wrong, but it seems to be an even better gig today.
So I don't, know, maybe their data is wrong, but it seems to be an even better gig today.
I don't think it is wrong at all. It may be a niche – specially in the US –, but with 46.3% of the world population still without internet access, the long tail is indeed long.
I've heard that software engineers are the automobile workers of the 50s-70s: Extremely well-compensated professionals who were ultimately thrown to the wayside when technology came. I wonder if data scientists and AI will do a similar thing to software engineers.
We have spent the last 50 years trying to automate our job away. Many projects that 20 years ago took an entire team a year or more are now weekend projects for a single person, because of how much better our programming languages, libraries, APIs, IDEs, linters, etc have become. But demand turns out to be incredibly elastic. A 50% increase in productivity just means that there's 400% more work, because so many new things have now become cost effective to try.
Of course there is bound to be an end to this, a peak-software if you so will, where productivity increases will stop being offset by demand increases. I might live to see that day, but this AI wave is unlikely to be it. Yes, it brings some new tools that make programming easier, but it stimulates more than enough demand to offset that.
Of course there is bound to be an end to this, a peak-software if you so will, where productivity increases will stop being offset by demand increases. I might live to see that day, but this AI wave is unlikely to be it. Yes, it brings some new tools that make programming easier, but it stimulates more than enough demand to offset that.
> how much better our programming languages, libraries, APIs, IDEs, linters, etc have become
It's just libraries, really. Or libraries are 80% of the gain. So it's less about automation and tools and more about code reuse (sometimes it's even just binary reuse - you surely do that everyday by calling dynamic libraries).
The progress you note in the last 20-30 years is just Internet that became widely available. First at work and Unis, and then at home. We switched from sending disks by mail to downloading stuff from FTP servers. Recently WWW Git front-ends gave it another serious boost. That's what happened.
> But demand turns out to be incredibly elastic. A 50% increase in productivity just means that there's 400% more work, because so many new things have now become cost effective to try.
I would credit Moore's law for that instead. Really, if one can do e.g. affordable, real-time machine vision stuff today, it has more to do with CPUs and GPUs gaining power every year, than software getting better.
It's just libraries, really. Or libraries are 80% of the gain. So it's less about automation and tools and more about code reuse (sometimes it's even just binary reuse - you surely do that everyday by calling dynamic libraries).
The progress you note in the last 20-30 years is just Internet that became widely available. First at work and Unis, and then at home. We switched from sending disks by mail to downloading stuff from FTP servers. Recently WWW Git front-ends gave it another serious boost. That's what happened.
> But demand turns out to be incredibly elastic. A 50% increase in productivity just means that there's 400% more work, because so many new things have now become cost effective to try.
I would credit Moore's law for that instead. Really, if one can do e.g. affordable, real-time machine vision stuff today, it has more to do with CPUs and GPUs gaining power every year, than software getting better.
>I would credit Moore's law for that instead. Really, if one can do e.g. affordable, real-time machine vision stuff today, it has more to do with CPUs and GPUs gaining power every year, than software getting better.
Not really. AlexNet was a watershed moment for computer science, and the kind of things that were possible by 2015 with AI that weren't possible in 2010 far outstrip the gains in hardware advancement during that time.
Not really. AlexNet was a watershed moment for computer science, and the kind of things that were possible by 2015 with AI that weren't possible in 2010 far outstrip the gains in hardware advancement during that time.
True. VGG models halved the ImageNet error rate of AlexNet two years after AlexNet, followed by ResNet models a little over a year later which halved the error rate again. Recent NAS generated models are up to 20x more computationally efficient than ResNet models of the same accuracy.
> I would credit Moore's law for that instead. Really, if one can do e.g. affordable, real-time machine vision stuff today, it has more to do with CPUs and GPUs gaining power every year, than software getting better.
How can you say that seriously when, for the vast majority of people, a computer vision project consists of `import cv2`?
How can you say that seriously when, for the vast majority of people, a computer vision project consists of `import cv2`?
>We have spent the last 50 years trying to automate our job away.
Yes I've heard it for 25 years. It's mostly low hanging fruit and it hasn't been done with auto-coding per-se, but more data drive architecture where a layman can do a basic website. Anything custom or sophisticated requires programming knowledge though, and since they are proprietary systems, it's a lot more expensive.
Yes I've heard it for 25 years. It's mostly low hanging fruit and it hasn't been done with auto-coding per-se, but more data drive architecture where a layman can do a basic website. Anything custom or sophisticated requires programming knowledge though, and since they are proprietary systems, it's a lot more expensive.
In my opinion, unless we develop AGI, not at all... And if we did develop AGI, then data scientists would find themselves out of a job too.
I think you'd be very hard pressed to automate systems programming, and most forms of programming for that matter. So, as long as we develop new hardware, new products, new platforms, new programming languages, then there will be lots of programming work to be done.
The jobs that machine learning is replacing right now (if any?) are fairly repetitive jobs that don't really require any reasoning. Maybe machine learning is also helping automate phishing schemes, but, is there any programming job out there that's really repetitive and thoughtless, where the work is predictable and akin to factory work? Do you think a transformer model is going to be able to handle debugging?
I think you'd be very hard pressed to automate systems programming, and most forms of programming for that matter. So, as long as we develop new hardware, new products, new platforms, new programming languages, then there will be lots of programming work to be done.
The jobs that machine learning is replacing right now (if any?) are fairly repetitive jobs that don't really require any reasoning. Maybe machine learning is also helping automate phishing schemes, but, is there any programming job out there that's really repetitive and thoughtless, where the work is predictable and akin to factory work? Do you think a transformer model is going to be able to handle debugging?
You obviously have more knowledge than me on this subject, so take what I say with that in mind. But I look at the march of progress and can't help but draw parallels.
Websites used to be made by hand. Then came frameworks and then came code-free e-commerce sites like Shopify. Github is testing the Copilot AI. How much longer before some architect tells an AI to stitch together code that fits some sort of functional spec. REST API's are already defined by their model and have code generated to match it. Is it that much a stretch to say that in the near future (15-25 years), AI might take over a significant part of software engineering?
Websites used to be made by hand. Then came frameworks and then came code-free e-commerce sites like Shopify. Github is testing the Copilot AI. How much longer before some architect tells an AI to stitch together code that fits some sort of functional spec. REST API's are already defined by their model and have code generated to match it. Is it that much a stretch to say that in the near future (15-25 years), AI might take over a significant part of software engineering?
However, despite the existence of products like Shopify and Wordpress, web developers have no trouble finding jobs, as far as I know.
> How much longer before some architect tells an AI to stitch together code that fits some sort of functional spec.
Would that person not be, essentially, a programmer? And what would they do if they needed to interface with external hardware that's not yet supported, or some network protocol or piece of software that's not built into AI-code-generator-3000?
> Is it that much a stretch to say that in the near future (15-25 years), AI might take over a significant part of software engineering?
In my opinion, people who probably think little of software developers, keep imagining that software engineering is this primitive, simple, reducible thing, and they are simply wrong.
Can there be a use for AI-generated code, and can that take over things like generating user-facing forms and website? Sure. However, what I think is likely to happen is that in 15-25 years, the world will be changing just as fast as it is now, and the software engineering discipline will just expand. Companies that hire "architects" who only know how to use AI-code-generator-3000 will be at a disadvantage compared to companies that hire more fully fledged software engineers.
Imagine you run a business and you only have architects who know how to use some code generator code that runs on the Google cloud. You pay some monthly bill to Google and hefty fees to use that tool. You're locked-in to that ecosystem. If you need a new feature or access to a new API, you may need to wait for Google to add support. In contrast, your competitors who hired pesky software engineers are paying higher salaries but save on Google cloud fees, and they don't have to wait for Google to fix problems to implement new features.
> How much longer before some architect tells an AI to stitch together code that fits some sort of functional spec.
Would that person not be, essentially, a programmer? And what would they do if they needed to interface with external hardware that's not yet supported, or some network protocol or piece of software that's not built into AI-code-generator-3000?
> Is it that much a stretch to say that in the near future (15-25 years), AI might take over a significant part of software engineering?
In my opinion, people who probably think little of software developers, keep imagining that software engineering is this primitive, simple, reducible thing, and they are simply wrong.
Can there be a use for AI-generated code, and can that take over things like generating user-facing forms and website? Sure. However, what I think is likely to happen is that in 15-25 years, the world will be changing just as fast as it is now, and the software engineering discipline will just expand. Companies that hire "architects" who only know how to use AI-code-generator-3000 will be at a disadvantage compared to companies that hire more fully fledged software engineers.
Imagine you run a business and you only have architects who know how to use some code generator code that runs on the Google cloud. You pay some monthly bill to Google and hefty fees to use that tool. You're locked-in to that ecosystem. If you need a new feature or access to a new API, you may need to wait for Google to add support. In contrast, your competitors who hired pesky software engineers are paying higher salaries but save on Google cloud fees, and they don't have to wait for Google to fix problems to implement new features.
> Can there be a use for AI-generated code, and can that take over things like generating user-facing forms and website?
Sure, but...I'm not convinced there's any advantage over writing deterministic generators for things like that.
On one hand, you have something that produces a predictable output for a given input (modulo bugs in the generator, which can be corrected over time).
On the other hand, you have a black box that could misinterpret some non-obvious aspect of its input and produce completely unexpected output. Its training data will need to be carefully curated and maintained over time, similar to how a deterministic generator would need to be maintained, but, again, without the same degree of certainty as to what it's going to produce.
Honestly, the black box approach just seems like a nightmare to me.
Sure, but...I'm not convinced there's any advantage over writing deterministic generators for things like that.
On one hand, you have something that produces a predictable output for a given input (modulo bugs in the generator, which can be corrected over time).
On the other hand, you have a black box that could misinterpret some non-obvious aspect of its input and produce completely unexpected output. Its training data will need to be carefully curated and maintained over time, similar to how a deterministic generator would need to be maintained, but, again, without the same degree of certainty as to what it's going to produce.
Honestly, the black box approach just seems like a nightmare to me.
>Github is testing the Copilot AI. How much longer before some architect tells an AI to stitch together code that fits some sort of functional spec.
You should register for the Copilot beta. It's pretty eye opening.
Basically it generates bog standard terrible code, because thats the majority of code. It can "Hello, world!" in most languages. Maybe it's useful for particularly boilerplate heavy languages or librairies, but for the most part it's more like StackOverflow copy-pasting except the code is worse.
So quite a long way to go.
You should register for the Copilot beta. It's pretty eye opening.
Basically it generates bog standard terrible code, because thats the majority of code. It can "Hello, world!" in most languages. Maybe it's useful for particularly boilerplate heavy languages or librairies, but for the most part it's more like StackOverflow copy-pasting except the code is worse.
So quite a long way to go.
Pretty much what should be expected by applying GPT-3 to code or whatever.
> Is it that much a stretch to say that in the near future (15-25 years), AI might take over a significant part of software engineering?
Not much of a stretch at all. But we've seen this sort of transition before: lower-level languages (microcode, assembly) are now largely generated by other software yet the amount of software development work needing to be done by humans has only expanded, and I don't expect there to be much more than a bump in the road in terms of demand for software developers this time around either.
Though I will say that the bump will be larger than the non-event that 4GLs turned out to be: https://en.wikipedia.org/wiki/Fourth-generation_programming_...
The most disruptive scenario I can imagine will result in a slew of businesses hit with absolutely massive cloud computing bills (due to naive generated code that works, but very inefficiently, or perhaps just code running for naively specified tasks that HAVE no efficient solution as stated), leading to a standard requirement of having an expensive human software developer sign off on it before deployment like architects signing off on blueprints (which might be the thin edge of the wedge for professional licensing, but that's a different conversation).
As a secondary effect, there might also be a proliferation of new (or newly popular) languages (for which insufficient public code exists for ML systems to learn from), as developers focus on domain+skill+technology combinations that haven't been automated YET.
Not much of a stretch at all. But we've seen this sort of transition before: lower-level languages (microcode, assembly) are now largely generated by other software yet the amount of software development work needing to be done by humans has only expanded, and I don't expect there to be much more than a bump in the road in terms of demand for software developers this time around either.
Though I will say that the bump will be larger than the non-event that 4GLs turned out to be: https://en.wikipedia.org/wiki/Fourth-generation_programming_...
The most disruptive scenario I can imagine will result in a slew of businesses hit with absolutely massive cloud computing bills (due to naive generated code that works, but very inefficiently, or perhaps just code running for naively specified tasks that HAVE no efficient solution as stated), leading to a standard requirement of having an expensive human software developer sign off on it before deployment like architects signing off on blueprints (which might be the thin edge of the wedge for professional licensing, but that's a different conversation).
As a secondary effect, there might also be a proliferation of new (or newly popular) languages (for which insufficient public code exists for ML systems to learn from), as developers focus on domain+skill+technology combinations that haven't been automated YET.
i think you're focusing too much on how many programmers will be needed in the future without considering what kind of pay they'll be able to get.
I get that we're basically arguing over the Christensen Innovator's Dilemma, and I acknowledge that at some point ML-driven solutions will cross the 'good enough' threshold that eliminates certain swaths of work wholesale, but I don't think that an increase in the demand for software development and automating away most of today's software development related tasks spells a decrease in software development pay, any more than desktop publishing was a harbinger for fewer graphic designers or lower pay for graphic design work.
Progress != destruction of jobs.
Farming, auto, journalism, and marketing fits your model. But medicine, finance, and tech would suggest otherwise. I've been getting advice about how there is no money in software and I should study hardware since 90s.
Yes, maybe this is a freefall and we haven't made impact yet. But drawing parallel without further evidence seems to be too simple of a analysis.
What cause the jobs in Detroit to be outsourced? What skills was replaced? How easy was it to train those skills? What was the potential pool of workers? What was the demand of workers?
All I'm saying is that this question is a lot more nuanced. I'm a software engineer, so I'm biased. But I'm guessing that you might be biased in some ways too. The real answer, perhaps boring, is that we really don't know. Even if you're right - you could easily be off by 200 years.
Farming, auto, journalism, and marketing fits your model. But medicine, finance, and tech would suggest otherwise. I've been getting advice about how there is no money in software and I should study hardware since 90s.
Yes, maybe this is a freefall and we haven't made impact yet. But drawing parallel without further evidence seems to be too simple of a analysis.
What cause the jobs in Detroit to be outsourced? What skills was replaced? How easy was it to train those skills? What was the potential pool of workers? What was the demand of workers?
All I'm saying is that this question is a lot more nuanced. I'm a software engineer, so I'm biased. But I'm guessing that you might be biased in some ways too. The real answer, perhaps boring, is that we really don't know. Even if you're right - you could easily be off by 200 years.
>What cause the jobs in Detroit to be outsourced?
Higher profit margins at the cost of lower quality and enabled with NAFTA. In the 90s until the mid 2000s, US cars were mostly top notch, now they're not that great.
You can read what the market thinks of the quality of a vehicle by its resale price relative to its new price. Used Honda and Toyotas are pretty close to a new price. Chryslers, not so much. This isn't exclusively a US problem though. You can get a used BMW or Mercedes for cheap, but you probably shouldn't if you want to keep it for a while and not spend a lot of money. Back in the 90s, you could expect to get 10 years out of a vehicle relatively hassle free, now it's about 3. YMMV.
Higher profit margins at the cost of lower quality and enabled with NAFTA. In the 90s until the mid 2000s, US cars were mostly top notch, now they're not that great.
You can read what the market thinks of the quality of a vehicle by its resale price relative to its new price. Used Honda and Toyotas are pretty close to a new price. Chryslers, not so much. This isn't exclusively a US problem though. You can get a used BMW or Mercedes for cheap, but you probably shouldn't if you want to keep it for a while and not spend a lot of money. Back in the 90s, you could expect to get 10 years out of a vehicle relatively hassle free, now it's about 3. YMMV.
yeah, a lot of people are grasping at patterns they see in their understanding of the past and treating them like rules that will determine the future.
These are the same arguments that people have made about outsourcing for thirty years, and we've only got more programmers who are paid vastly more money because of it.
because of it?
To some extent, yes. There's an awful lot of work that comes back to be done correctly.
It's how I got my job originally.
It's how I got my job originally.
>I wonder if data scientists and AI will do a similar thing to software engineers.
I don't really see how. Software development is about translating arbitrary requirements into a rigorous form without ambiguity (ie the developer has to fill in the blanks). Unless we invent AGI we would need the 'clients' of software development to be able to unambiguously list their requirements (without there being any blanks that need to be filled in). Effectively, they'd have to become devs themselves. It might not look like the software development of today of writing text, but they'd still be doing a similar type of job.
I don't really see how. Software development is about translating arbitrary requirements into a rigorous form without ambiguity (ie the developer has to fill in the blanks). Unless we invent AGI we would need the 'clients' of software development to be able to unambiguously list their requirements (without there being any blanks that need to be filled in). Effectively, they'd have to become devs themselves. It might not look like the software development of today of writing text, but they'd still be doing a similar type of job.
Depends.
If you mean people who's work involves developing software by typing instructions into a computer via a keyboard interface... and this is what entirely defines a software engineer. Then possibly.
If you mean people who's work involves developing software by instructing computers in various other ways (e.g. via talking, feeding it pictures, videos, audio, or other data)... and this is what defines a soft-... er I mean data scientist. Then I'd argue data scientists are just a newer software engineer.
If you mean engineers in the true and general sense? Never. Devising new technology & solving problems are skills forever employable.
If you mean people who's work involves developing software by typing instructions into a computer via a keyboard interface... and this is what entirely defines a software engineer. Then possibly.
If you mean people who's work involves developing software by instructing computers in various other ways (e.g. via talking, feeding it pictures, videos, audio, or other data)... and this is what defines a soft-... er I mean data scientist. Then I'd argue data scientists are just a newer software engineer.
If you mean engineers in the true and general sense? Never. Devising new technology & solving problems are skills forever employable.
I just got my email invite to the Github Copilot tech preview. So I'm definitely thinking of "Code as ML" ;)
I think we begin to see software construction design oriented towards "agents". Analogous to previous abstractions like "daemons" or "running programs". They have permission to "self-assemble" intelligently.
I think we begin to see software construction design oriented towards "agents". Analogous to previous abstractions like "daemons" or "running programs". They have permission to "self-assemble" intelligently.
It depends, because you still need people even in car factories.
Also, automation really changes the nature of a product, and the quality is often poor.
Also, automation really changes the nature of a product, and the quality is often poor.
ML and data-science relies on software to work. Nobody is going to do data-science and ML by hand, so already you have a demand for software engineers and computer scientists to build tools and implement algorithms and information pipelines for the data-scientists and businesses who want to leverage ML.
I think there might be a bit of a misconception of what existing software engineers did. Very rarely were they focused on designing inference models or performing data analysis. Most of the time the business side of things, like a business analysts, or people working on the business problem would perform analysis manually, and come up with business rules and logic. Software engineers could help them leverage more powerful techniques by building them tools that help for data analysis, like BI platforms, or data stores that can do bigger and more complex queries faster. Or they could help them with letting them know of techniques, generally known as old-school AI, such as edit distance algorithms, graph algorithms, logic rule engines, expert systems, etc.
So nothing has really changed, except that there is even more demand from businesses who want software engineers to help them setup tools and pipelines for them to experiment with ML and have data scientists analyze their data.
What it would take to get rid of the demand for software engineers is to be able to automate their work, and ML is far from able to do that. It could one day, but we are talking about a computer that can reprogram itself successfully, and that can also understand the requests of the user in how it should reprogram itself. And we're talking not just reprogramming, but it should also be able to replicate itself to other machines and all that, since most use case today involve distributed system, as no one computer has the compute needed for the scale we operate at. Things like GPT-3 show some promise, but as it stands today, you'd need a software engineer to inspect and review everything it generates, which would take almost as much time if not more.
I think what is far more likely is that the offer increases, that is, that there eventually is just a lot more people with the ability to perform the work of software engineers. This will probably happen as a combination of more people joining the field, and the barrier to entry becoming smaller. The latter could be helped a little bit by ML, if it can deliver better auto-complete for example, but I think it largely is just driven by frameworks, PAAS, SAAS and IAAS, library, and all other type of code reuse we've been building for years now.
I think there might be a bit of a misconception of what existing software engineers did. Very rarely were they focused on designing inference models or performing data analysis. Most of the time the business side of things, like a business analysts, or people working on the business problem would perform analysis manually, and come up with business rules and logic. Software engineers could help them leverage more powerful techniques by building them tools that help for data analysis, like BI platforms, or data stores that can do bigger and more complex queries faster. Or they could help them with letting them know of techniques, generally known as old-school AI, such as edit distance algorithms, graph algorithms, logic rule engines, expert systems, etc.
So nothing has really changed, except that there is even more demand from businesses who want software engineers to help them setup tools and pipelines for them to experiment with ML and have data scientists analyze their data.
What it would take to get rid of the demand for software engineers is to be able to automate their work, and ML is far from able to do that. It could one day, but we are talking about a computer that can reprogram itself successfully, and that can also understand the requests of the user in how it should reprogram itself. And we're talking not just reprogramming, but it should also be able to replicate itself to other machines and all that, since most use case today involve distributed system, as no one computer has the compute needed for the scale we operate at. Things like GPT-3 show some promise, but as it stands today, you'd need a software engineer to inspect and review everything it generates, which would take almost as much time if not more.
I think what is far more likely is that the offer increases, that is, that there eventually is just a lot more people with the ability to perform the work of software engineers. This will probably happen as a combination of more people joining the field, and the barrier to entry becoming smaller. The latter could be helped a little bit by ML, if it can deliver better auto-complete for example, but I think it largely is just driven by frameworks, PAAS, SAAS and IAAS, library, and all other type of code reuse we've been building for years now.
This feel like a slow end of an era. I work in a mixed ML/dev team, on the dev side (distributed systems) and, looking at real-world ML work (and having tried it a bit at home), I really don't want to do ML unless that becomes the only choice... I hope I can get standard dev gigs till I can retire ;)
There was a good quote either from Joel or someone like that (or someone he was interviewing on a podcast), along the lines of "I went into software because I like tinkering with intricate clockwork; not because I like training a puppy to not pee on the carpet". On top of that, you tend to not even spend most of the time on the puppy, most of the time is spent collecting and organizing carpet samples.
There was a good quote either from Joel or someone like that (or someone he was interviewing on a podcast), along the lines of "I went into software because I like tinkering with intricate clockwork; not because I like training a puppy to not pee on the carpet". On top of that, you tend to not even spend most of the time on the puppy, most of the time is spent collecting and organizing carpet samples.
In the relentless pursuing of fame and profit, the knowledge output from companies are outpacing serious academic research institutions. Because of the company's inherent goal for profit, knowledge is also more and more being sugar-coated and pushed out prematurely (which had nothing to do with the morality of the people involved, as the environment of operation are different). Weaker and weaker knowledge is being used nowadays across the society in easier and easier accessible fashion. Skills are replace by automated answers both the asker and answerer might not have the mental capacity and tenacity to truly understand and verify. Cracks are forming between all corners of the intellectual babel tower that is being building and deteriorating at the same time.
Of course, we have not figure out how to reliably writing dumb code yet, but we are going to replace that with ML. Now everyone involved can truly shed the responsibility of program failures. After all, no one understand ML anyway...
Of course, we have not figure out how to reliably writing dumb code yet, but we are going to replace that with ML. Now everyone involved can truly shed the responsibility of program failures. After all, no one understand ML anyway...
"AI is increasingly being applied to mission critical infrastructure like national electric grids and automated supermarket warehousing calculations during pandemics. However, there are questions about whether the maturity of the industry has caught up with the enormity of its growing deployment."
I surmise "being applied" means outcome of an ML model is used to augment the decision taken by a human. I doubt they are used to handle everything automatically for critical infra, since ML models are neither 100% accurate and fail miserably when they fail. But the insinuation is that the field has already been won by AI.
I surmise "being applied" means outcome of an ML model is used to augment the decision taken by a human. I doubt they are used to handle everything automatically for critical infra, since ML models are neither 100% accurate and fail miserably when they fail. But the insinuation is that the field has already been won by AI.
This is the best quote I've seen in a while: "Do machine learning like the great engineer you are, not like the great machine learning expert you aren't."
[deleted]
You're generally tied to proprietary nvidia drivers that are a pain to install, and when they change version, it breaks software such as PyTorch, you're forced to update. PyTorch itself often has breaking changes. Then, most of the code is written in Python, often by academics who know and care very little about good software engineering practices. That also leads to fragile and breakable code. You'll often find yourself trying to install code that's just DOA because of multiple broken dependencies.
The quality of the code can be terrible, because academics are in a constant rush to publish the next thing, and tend to view code as throwaway. Worse yet, it's often written by students, and these students are taught by professors who view software engineering as an inferior, unscientific discipline. I've seen many github repos with no open source license, and no README, just a random code dump with no comments.
I'm personally not super enthusiastic by the idea of replacing a lot of code by machine learning just yet, because not only is the software ecosystem fragile, but the models can be as well. You're likely to introduce weird, unpredictable modes of failure. I can see that there's clearly a lot of people who dream of being able to basically create software without coding, and not have to deal with pesky software developers, but IMO, we're not there yet.
This guy talks about cloud-based machine learning solutions. You won't have to deal with drivers and general "IT" concerns there. The problem is that, besides being very expensive, these cloud-based solution will introduce extra layers of vendor lock-in. If you're training on the google cloud and using TPUs that you can't even physically buy/own, you might have a hard time porting your code to run without a TPU. If Google revokes your access for some reason, because they claim you've violated their TOS, you are fucked.