What Should We Do to Prevent Software from Failing?(sloanreview.mit.edu)
sloanreview.mit.edu
What Should We Do to Prevent Software from Failing?
https://sloanreview.mit.edu/article/what-should-we-do-to-prevent-software-from-failing/
127 comments
None of the above changes the discussion. It's a very elaborate way of saying "writing correct software is hard". You know what's also hard to do correctly? Designing airplanes. Building skyscrapers. It's not that their designs are made flawless, it's that (in theory) the sheer volume of man-hours spent pouring over every minute detail of them, incorporating lessons learned from every failure that's ever happened in the field, amount to a very very high success rate.
In software the mentality is that tests, or formal specs, or even thoughtful design, are luxuries. That it's better to just hire someone who can make the computer do things than spend twice as much on someone who can write "good" code and take twice as long to do so (whatever "good" means, because we have no standard). Of course for "the Uber for X" those standards may be fine. But for autonomous vehicles, or credit bureaus, or the nascent private space industry, they are absolutely not.
In software the mentality is that tests, or formal specs, or even thoughtful design, are luxuries. That it's better to just hire someone who can make the computer do things than spend twice as much on someone who can write "good" code and take twice as long to do so (whatever "good" means, because we have no standard). Of course for "the Uber for X" those standards may be fine. But for autonomous vehicles, or credit bureaus, or the nascent private space industry, they are absolutely not.
A key difference between physical engineering and software engineering is that the economics of over-engineering to improve safety margins are wildly different. For physical engineering, the costs of substantially decreasing failure risk (e.g. via increasing strength, different materials, more complex design, etc) is approximately linear. For software engineering, that cost is something closer to quadratic or worse.
This is intrinsic to the differing nature of the problems. Verifying the safety of a bridge design is computationally trivial, verifying the safety of non-trivial software is NP-Hard. Safety is so inexpensive to add in physical engineering designs that we invest a lot of effort removing incidental unnecessary safety.
This is intrinsic to the differing nature of the problems. Verifying the safety of a bridge design is computationally trivial, verifying the safety of non-trivial software is NP-Hard. Safety is so inexpensive to add in physical engineering designs that we invest a lot of effort removing incidental unnecessary safety.
> verifying the safety of non-trivial software is NP-Hard.
While true, this depends very much on the design (or lack thereof) of the software system. Many of our software systems are more complex than necessary because they weren't designed or the designs weren't considered carefully enough. When the software is allowed to grow and accumulate masses of code virtually unchecked, it becomes computationally intractable to continue to verify the system.
Much of the software involved in aircraft, specifically, can be reduced to very simple designs and implementations where the necessary safety analysis is feasible.
The present complexity is often in the accumulation of legacy defects and corrections (fixing the right problem in the wrong place) or trying to fix hardware problems with software. I'm not 100% pro-rewrite, but many avionics software groups (that I've encountered) refuse any rewrites on certification grounds but end up with something unbearably complex as a result (simulating hardware databuses from the 1980s in software on a 1 GHz processor when the system was initially targetting a 16 MHz processor).
While true, this depends very much on the design (or lack thereof) of the software system. Many of our software systems are more complex than necessary because they weren't designed or the designs weren't considered carefully enough. When the software is allowed to grow and accumulate masses of code virtually unchecked, it becomes computationally intractable to continue to verify the system.
Much of the software involved in aircraft, specifically, can be reduced to very simple designs and implementations where the necessary safety analysis is feasible.
The present complexity is often in the accumulation of legacy defects and corrections (fixing the right problem in the wrong place) or trying to fix hardware problems with software. I'm not 100% pro-rewrite, but many avionics software groups (that I've encountered) refuse any rewrites on certification grounds but end up with something unbearably complex as a result (simulating hardware databuses from the 1980s in software on a 1 GHz processor when the system was initially targetting a 16 MHz processor).
Try to solve an aerospace problem with this "simple" approach. No, you don't get to add strength willy-nilly, or just swap materials on a hunch. You're working with safety factors which sometimes are less than 1 (come back for references), and those physical properties of materials are notoriously stubborn to changes. You know, some Richard Feynman once quipped "nature can't be fooled, dammit" - that extends to "water boils at 273.16K full stop (standard conditions), no, you're not going to change that".
Yes, you can often do more complex design. Sometimes though it costs you centuries to choose a good combination of existing physical properties (architecture), and sometimes it's "just" decades (Gravity Probe B). Wanna propose similar approach to software? :)
Yes, you can often do more complex design. Sometimes though it costs you centuries to choose a good combination of existing physical properties (architecture), and sometimes it's "just" decades (Gravity Probe B). Wanna propose similar approach to software? :)
safety factor less than 1? you're planning that the thing will break early?
Liquid fuel rocket engines sometimes could be planned to work under the plastic deformation stresses, not elastic stresses. Given that their life-cycle is measured in seconds, such an approach - which brings some admirable weight savings - could produce results.
Writing software is not engineering - engineering is building the same thing over and over again and you slowly learn how to do it the right way. There are incremental improvements along the way - stronger lighter materials, better tools, etc. But the folks who built the Roman aqueducts could get up to speed pretty quickly on the Golden Gate Bridge.
The principles of flight don't change. The fundamentals of building a skyscraper or a bridge don't change. There are hundreds of years of learning behind those things that are still relevant today.
Building airplanes and bridges and skyscrapers is not hard at all. People do it all the time with a high success rate. The hardest part is scraping up the capital and managing the logistics. I'm sure none of that knowledge is required to get any kind of engineering certification.
If you redesigned every bridge and airplane from scratch, from first principles, you would have a tremendous failure rate no matter how many certifications you held. That's what software is - every project is a brand new thing that has never been done before - there isn't 200 years of learning to fall back on.
One of the worst things that could happen to the software industry is for bureaucrats and other people who know nothing about software to really believe it is an engineering discipline and try and regulate it as such. We'll all be forced to do a fantastic amount of busy work to meet arbitrary and archaic "engineering standards", and the software will actually decline in quality. You'll be fined $50,000 if you use tabs rather than spaces, or don't use yoda compares in your C. But as long as you are using spaces and yoda compares you won't be liable for a failure. Stuff basically like that.
Maybe when we have 200 years of experience writing software, rather than 50 or so, we can start to think about formalizing it as an engineering job.
The principles of flight don't change. The fundamentals of building a skyscraper or a bridge don't change. There are hundreds of years of learning behind those things that are still relevant today.
Building airplanes and bridges and skyscrapers is not hard at all. People do it all the time with a high success rate. The hardest part is scraping up the capital and managing the logistics. I'm sure none of that knowledge is required to get any kind of engineering certification.
If you redesigned every bridge and airplane from scratch, from first principles, you would have a tremendous failure rate no matter how many certifications you held. That's what software is - every project is a brand new thing that has never been done before - there isn't 200 years of learning to fall back on.
One of the worst things that could happen to the software industry is for bureaucrats and other people who know nothing about software to really believe it is an engineering discipline and try and regulate it as such. We'll all be forced to do a fantastic amount of busy work to meet arbitrary and archaic "engineering standards", and the software will actually decline in quality. You'll be fined $50,000 if you use tabs rather than spaces, or don't use yoda compares in your C. But as long as you are using spaces and yoda compares you won't be liable for a failure. Stuff basically like that.
Maybe when we have 200 years of experience writing software, rather than 50 or so, we can start to think about formalizing it as an engineering job.
>That's what software is - every project is a brand new thing that has never been done before - there isn't 200 years of learning to fall back on.
Sorry but this is just gatekeeping bullshit. Most apps being built today by companies with the budget to build them are TODO apps, report apps, storefronts, big data mungers, CRMs, etc. They all just have different business requirements and domain specific knowledge. To think that every software project can't be categorized is ridiculous.
Sorry but this is just gatekeeping bullshit. Most apps being built today by companies with the budget to build them are TODO apps, report apps, storefronts, big data mungers, CRMs, etc. They all just have different business requirements and domain specific knowledge. To think that every software project can't be categorized is ridiculous.
I honestly don't know what gatekeeping means in this context.
> all just have different business requirements and domain specific knowledge
Yeah - that's what I said. You realize that's where all the complexity is, right?
> all just have different business requirements and domain specific knowledge
Yeah - that's what I said. You realize that's where all the complexity is, right?
>I honestly don't know what gatekeeping means in this context.
you're trying to create a barrier to standardization. You don't want to codify successful patterns to keep your job coveted.
>Yeah - that's what I said. You realize that's where all the complexity is, right?
That's not what you said.
you're trying to create a barrier to standardization. You don't want to codify successful patterns to keep your job coveted.
>Yeah - that's what I said. You realize that's where all the complexity is, right?
That's not what you said.
You're being extremely uncharitable.
Look, business processes are different at every single company. Especially between companies doing the exact same thing. Which means that the software ostensibly doing the exact same thing at each of these companies is going to vary dramatically. Because the software at a company aligns to the business processes of that company. This is the true heart of Conway's law: "organizations which design systems ... are constrained to produce designs which are copies of the communication structures of these organizations."
And there is no fixing this. It's baked directly into the axioms. If every company in a domain ran exactly the same way, there would be no such thing as a competitive advantage.
Look, business processes are different at every single company. Especially between companies doing the exact same thing. Which means that the software ostensibly doing the exact same thing at each of these companies is going to vary dramatically. Because the software at a company aligns to the business processes of that company. This is the true heart of Conway's law: "organizations which design systems ... are constrained to produce designs which are copies of the communication structures of these organizations."
And there is no fixing this. It's baked directly into the axioms. If every company in a domain ran exactly the same way, there would be no such thing as a competitive advantage.
> you're trying to create a barrier to standardization. You don't want to codify successful patterns to keep your job coveted.
Ugh. I use as many libraries and services as possible so I can focus on writing code when no standard tools exist. That will keep me busy every day forever. If you think I'm out there sabotaging standardization efforts you're delusional. I wouldn't even know how to do that. Slash tires at a W3C convention or something?
Ugh. I use as many libraries and services as possible so I can focus on writing code when no standard tools exist. That will keep me busy every day forever. If you think I'm out there sabotaging standardization efforts you're delusional. I wouldn't even know how to do that. Slash tires at a W3C convention or something?
Yeah, and a lot of that software sucks because the companies who pay for it to be developed want the cheapest possible thing that just about works and not some ultra-engineered unkillable concrete bunker of an app that costs them 50x more. That was the entire point of the agile revolution - get rid of waterfall methods inherited from classical physical engineering and switch to constant iteration and shipping, with fixes to not-quite-right work being done whilst workers are in the building. Software engineering became less like regular engineering because that's what the customers wanted, and who's to say they were wrong?
The firmware in all the electronic gizmos in your life aren't running "just apps". Much of it is poorly developed shoot from the hip code that is riddled with bugs. Some of it is running in cars and planes that manage to kill people because of hubris.
It's a very elaborate way of saying "writing correct software is hard". You know what's also hard to do correctly? Designing airplanes. Building skyscrapers... the sheer volume of man-hours spent pouring over every minute detail of them, incorporating lessons learned from every failure that's ever happened in the field, amount to a very very high success rate.
It's not only hard, but the cost/benefit (or the large penalties of catastrophic failure) works out so that it's worth it to go to that amount of effort. The same amount of effort doesn't go into building a backyard garden shed. I'll grant that we've gotten to the point where the cost/benefit analysis demands that we go to that amount of effort in critical situations.
It's not only hard, but the cost/benefit (or the large penalties of catastrophic failure) works out so that it's worth it to go to that amount of effort. The same amount of effort doesn't go into building a backyard garden shed. I'll grant that we've gotten to the point where the cost/benefit analysis demands that we go to that amount of effort in critical situations.
You're ignoring cost. Want to build software like we (were supposed to) build 737s -- to always work? Watch your costs go up by (rough guess) 10-100x.
We've generally made a decisive choice for functionality and cost over bugs.
Which isn't to say that I don't agree that there are areas: self-driving cars, airplanes (required here), human diagnostic software (already required here), etc that should have much tougher standards. But across the board ... I don't think people are willing to pay for that.
We've generally made a decisive choice for functionality and cost over bugs.
Which isn't to say that I don't agree that there are areas: self-driving cars, airplanes (required here), human diagnostic software (already required here), etc that should have much tougher standards. But across the board ... I don't think people are willing to pay for that.
Perhaps one of the reasons for this is that software doesn't allow humans any way of leveraging our evolved "danger detectors".
It's hard to make a safe building, and it's hard to make safe software. The difference is it's (often) incredibly obvious that a building isn't safe the moment you step into it. In software it's sometimes pretty much impossible to tell the difference between a stone temple and a house of cards.
It's hard to make a safe building, and it's hard to make safe software. The difference is it's (often) incredibly obvious that a building isn't safe the moment you step into it. In software it's sometimes pretty much impossible to tell the difference between a stone temple and a house of cards.
It is incredibly obvious to people with the knowledge and skills. They can look at code and just by glancing at it can have a gut reaction that triggers red flags. Sometimes it can even be an emotional disgust. It's just not common knowledge. We as a community need to do a better job at transferring this knowledge.
Proper strong typing, not just the weak typing that is more common, provides these "danger detectors". It makes it a compile error. Compiler errors make things immediately obvious.
Type systems are very rich and most statically typed languages that are commonly used now can barely count as type systems when compared against what we know from mathematics and computer science. They merely describe the data format (or not in the case of nullable) and not acceptable inputs. They don't allow types for scenarios like requiring a function that exhibits these behaviors but not these other ones.
Pattern matching in combination with strong typing requires code that handles all possible inputs or you get a compiler error.
Dependent typing requires inputs to be within certain tolerances (not just adhering to a data format).
Type systems can also make certain race conditions impossible to create.
Proper strong typing, not just the weak typing that is more common, provides these "danger detectors". It makes it a compile error. Compiler errors make things immediately obvious.
Type systems are very rich and most statically typed languages that are commonly used now can barely count as type systems when compared against what we know from mathematics and computer science. They merely describe the data format (or not in the case of nullable) and not acceptable inputs. They don't allow types for scenarios like requiring a function that exhibits these behaviors but not these other ones.
Pattern matching in combination with strong typing requires code that handles all possible inputs or you get a compiler error.
Dependent typing requires inputs to be within certain tolerances (not just adhering to a data format).
Type systems can also make certain race conditions impossible to create.
That's not really true. Much of what makes a modern construct safe is extremely unintuitive; material science being a big example. Some modern buildings are even designed to look precarious on purpose, for aesthetic effect, despite being very sound.
> The difference is it's (often) incredibly obvious that a building isn't safe the moment you step into it.
This is just not true. If anything, our "danger detectors" probably give off an equal amount of false positives and false negatives when judging the structural integrity of objects we're unfamiliar with. And the same holds true for code bases, which is exactly why it makes sense to hold software to a similar standard as physical engineering.
This is just not true. If anything, our "danger detectors" probably give off an equal amount of false positives and false negatives when judging the structural integrity of objects we're unfamiliar with. And the same holds true for code bases, which is exactly why it makes sense to hold software to a similar standard as physical engineering.
> it's that (in theory) the sheer volume of man-hours spent pouring over every minute detail of them, incorporating lessons learned from every failure that's ever happened in the field, amount to a very very high success rate
and this is what company executives need to understand. Given the fleeting nature of software and the speed in which one can write working code (not strictly correct or bug-free, but something that apparently works) people are convinced that software development should be cheap -- or cheaper than say airplane design. Software design is thus seen as cost not investment and those flaws in specifying and writing correct core is then shown in not treated edge cases, sometimes catastrofically
and this is what company executives need to understand. Given the fleeting nature of software and the speed in which one can write working code (not strictly correct or bug-free, but something that apparently works) people are convinced that software development should be cheap -- or cheaper than say airplane design. Software design is thus seen as cost not investment and those flaws in specifying and writing correct core is then shown in not treated edge cases, sometimes catastrofically
The problem isn't that executives don't understand, it's that they don't care. Capitalism's incentives do not encourage the practice that's best for society. The only industries where care is taken are the ones that have been regulated to be as such. And then there are cases like Boeing, where that regulatory structure decayed and the safety practices decayed in tandem.
This reminds me of one of the first conversations I had with my manager when I started my current job as a software engineer.
I used to work at a large financial company who had an intensely rigorous approach to verifying it's software in both manual and automated ways, as well as a much more careful approach to writing code to begin with.
When I started here (a much smaller startup delivering marketing software for small business clients), I brought up that I was concerned about going to somewhere with a less comprehensive testing pipeline in place, to which I was told:
"It's not particularly important to be able to find bugs before they're released, it's more important to optimize the deployment speed so that we can release the fixes quickly"
Sounded crazy to me at the time, but I later realized it's really all about what your industry is optimizing for, and what would cost you more money.
I used to work at a large financial company who had an intensely rigorous approach to verifying it's software in both manual and automated ways, as well as a much more careful approach to writing code to begin with.
When I started here (a much smaller startup delivering marketing software for small business clients), I brought up that I was concerned about going to somewhere with a less comprehensive testing pipeline in place, to which I was told:
"It's not particularly important to be able to find bugs before they're released, it's more important to optimize the deployment speed so that we can release the fixes quickly"
Sounded crazy to me at the time, but I later realized it's really all about what your industry is optimizing for, and what would cost you more money.
> it's really all about what your industry is optimizing for, and what would cost you more money
Sure, those were not critical systems, they could be patched with minimal hasse to clients. However you can't really predict catastrophe costs. It could range from minor usability issues to hard to trace calculation errors that could jeopardise the whole production operation. If the company doesn't work work with prototypes it should invest heavily in test coverage, integration test and delivery pipelines
Sure, those were not critical systems, they could be patched with minimal hasse to clients. However you can't really predict catastrophe costs. It could range from minor usability issues to hard to trace calculation errors that could jeopardise the whole production operation. If the company doesn't work work with prototypes it should invest heavily in test coverage, integration test and delivery pipelines
I think there is agreement, or at least slowly developing consensus, that some software should be "engineered" and some should just be "developed". But there is not yet really a structure (that I have seen) for declaring which thing your project is doing and setting up your practices accordingly. One reason this is tricky is that many (most?) projects start in the should-be-developed category and only transition to the should-be-engineered category later after they become mission critical in some way. The answer is often to build an "engineered" version as soon as you can figure out you need it. But then that's even more expensive than doing it that way to start, and it's also a really tough call to make because everybody knows rewrites are a bad idea. It's a tough bind for all but the most obvious cases. And then because those obvious cases are fairly few and far between, the best practices and the experiences of most of the workforce are skewed toward software that merely needs to be "developed".
To make this concrete: a couple times now in my career I've embarked on projects where setting up any level of formal verification at the beginning would have clearly been overkill, but where a few years later I really wished I had it for certain important pieces. And now since formal verification is so uncommon in my experience, I would have to play a lot of catch up to do it, even for a project that is clearly important enough for it even from the beginning. I think this is a pretty widespread experience.
To make this concrete: a couple times now in my career I've embarked on projects where setting up any level of formal verification at the beginning would have clearly been overkill, but where a few years later I really wished I had it for certain important pieces. And now since formal verification is so uncommon in my experience, I would have to play a lot of catch up to do it, even for a project that is clearly important enough for it even from the beginning. I think this is a pretty widespread experience.
> there is no magic formula for ensuring your ideas are always sound. If the idea is wrong you can't even write tests to ensure it's correct.
There are two parts to this. A wrong idea as in building the wrong product, and a wrong idea as in building the product wrong. There aren't any mechanical processes to protect against the first, but building the wrong product is rarely catastrophic (maybe only from a business perspective), and even if it is, it is usually not the software developer's fault. But as to building the product wrong, there are processes and tools that can help -- a lot -- like TLA+, which is slowly gaining momentum and has completely changed the way I think about software. Tools like TLA+ are extremely powerful at tackling those "complex, difficult to predict side-effects when composed into complex systems", but they don't replace careful thinking, they only help you with attacking complexity once you have decided to think hard about the problem. Getting developers to think hard before or during the development process -- but not while typing code -- is a cultural shift.
There are two parts to this. A wrong idea as in building the wrong product, and a wrong idea as in building the product wrong. There aren't any mechanical processes to protect against the first, but building the wrong product is rarely catastrophic (maybe only from a business perspective), and even if it is, it is usually not the software developer's fault. But as to building the product wrong, there are processes and tools that can help -- a lot -- like TLA+, which is slowly gaining momentum and has completely changed the way I think about software. Tools like TLA+ are extremely powerful at tackling those "complex, difficult to predict side-effects when composed into complex systems", but they don't replace careful thinking, they only help you with attacking complexity once you have decided to think hard about the problem. Getting developers to think hard before or during the development process -- but not while typing code -- is a cultural shift.
TLA for the curious.
https://learntla.com/introduction/
https://learntla.com/introduction/
I've heard the analogy of a software developer to a mechanic, and I like it much more.
Developer = Mechanic
Computer = Car
Make = Language
Model = Platform
Year = Version
A mechanic is licensed by ASE to work on cars in the USA and there are 50 different certifications [1], depending on the type of car/truck/bus. I don't want to see software development go this route, where developers have to get multiple certifications to prove they can work on a language or platform. It would be draining and force devs into silos, even more than they already are. And as the parent comment mentions, software is an "idea" problem and not a structural/foundational one. Unless your developer is a certified genius, what good is his certification?
[1] https://en.wikipedia.org/wiki/Automotive_Service_Excellence
Developer = Mechanic
Computer = Car
Make = Language
Model = Platform
Year = Version
A mechanic is licensed by ASE to work on cars in the USA and there are 50 different certifications [1], depending on the type of car/truck/bus. I don't want to see software development go this route, where developers have to get multiple certifications to prove they can work on a language or platform. It would be draining and force devs into silos, even more than they already are. And as the parent comment mentions, software is an "idea" problem and not a structural/foundational one. Unless your developer is a certified genius, what good is his certification?
[1] https://en.wikipedia.org/wiki/Automotive_Service_Excellence
I don't like car analogies.
A software engineer is like a plumber you link up lots of small components, weld them together and then the data flows through the application (at least that is the idea).
A software engineer is like a plumber you link up lots of small components, weld them together and then the data flows through the application (at least that is the idea).
I don't like either of these analogies. Work of SE is much more complicated. I guess a developer who just mashes things together on the front end is kind of like a plumber.
However, if you work with the backend, you need to worry about data access, any concurrency issues, performance problems. Generally tradesman follow relatively simple guidelines, which may change at a slower pace....tech on the other hand changes insanely quickly.
However, if you work with the backend, you need to worry about data access, any concurrency issues, performance problems. Generally tradesman follow relatively simple guidelines, which may change at a slower pace....tech on the other hand changes insanely quickly.
You don't "need" to do any of that on the back-end either to launch a product. Not sure what distinction between FE and BE you think there is here. One messes up an application on the server, the other messes up an application on the client. And there are plenty of avenues for being a clueless plumber on the backend, like using Ruby on Rails + "gem install devise" for authn + "gem install carrierwave" when the boss tells you they want avatars, without having much idea how anything actually works.
> I don't like either of these analogies.
I think it's hard to find a good analogy because software is an incredibly organic and artful thing. It can be highly technical but also playful.
I think it's a matter of finding the right people for the right job, and making sure those people have some skin in the game. If there is no incentive to write quality code, or the dev's don't care to write quality code you probably won't get quality code.
Then again maybe we still haven't come across the correct tools for the job that make it easier to produce quality software.
The fundamental architecture of a car has been relatively the same for many years now. To a lesser extent we see that in computer hardware however in software it so easy to invent new tools that maybe the good tools get lost in the mess of it all.
I think it's hard to find a good analogy because software is an incredibly organic and artful thing. It can be highly technical but also playful.
I think it's a matter of finding the right people for the right job, and making sure those people have some skin in the game. If there is no incentive to write quality code, or the dev's don't care to write quality code you probably won't get quality code.
Then again maybe we still haven't come across the correct tools for the job that make it easier to produce quality software.
The fundamental architecture of a car has been relatively the same for many years now. To a lesser extent we see that in computer hardware however in software it so easy to invent new tools that maybe the good tools get lost in the mess of it all.
That happens already. As things stabilize you tend to value predictability. You can have 6-10 smart people do the architecture and design work that drives the work of hundreds of developers.
Programmers want programming to be about ideas, but many/most problems are applying patterns to process.
Programmers want programming to be about ideas, but many/most problems are applying patterns to process.
It is already like that in the enterprise, where certifications are the entry point for certain partnerships.
As engineer I find there should be a baseline.
I also don't expect weekend cooks to keep a Michelin restaurant running.
As engineer I find there should be a baseline.
I also don't expect weekend cooks to keep a Michelin restaurant running.
There is another problem, that software developers in general hate maintenance work. People should be working on keeping existing stuff working and well-oiled. We want exciting work where we get to learn some new tech. Nobody wants the janitorial jobs where you're just keeping things ticking along and running smoothly. Ofcource there are people who work on legacy systems - banking systems, old mainframes, etc. We already have so much software thats out there, that just goes unused, and then people "re-invents" it, and often times makes it worse - with DRM, subscription, micro-transactions, "but now its on the web", etc.
I'd argue that there are an awful lot of us who would like nothing better than to spend our time cleaning up the codebase, refactoring risky code, getting the whole thing under test, etc.
Unfortunately, most organisations only grudgingly fund such work, and if I were to spend a significant portion of my time on such work (without being on a team dedicated to a critical modernisation effort), my performance reviews and compensation would suffer accordingly.
Unfortunately, most organisations only grudgingly fund such work, and if I were to spend a significant portion of my time on such work (without being on a team dedicated to a critical modernisation effort), my performance reviews and compensation would suffer accordingly.
Bingo. Hell, I'd love to do nothing but maintenance work, but I'd have a hard time paying the bills that way.
[deleted]
Agreed. Also, there's a big difference in building a doghouse, tent, home and skyscraper or bridge. Not all of them require the same rigor and planing. Beyond this, a lot of organizations don't allow for pushback from development to cover for clearing technical debt.
None of this accounts for the fact that over half the software industry has less than 5 years of experience and less than 25% has over 10. People with less than 5-10 years of experience can be brilliant, but there are things you learn with experience. First and foremost is the ability to push back and say, "No, it can't reasonably be done in X timeframe." Actually using scrum process to break things down in to chunks that can be swallowed. Not allowing mid-level developers to use "enterprise patterns" everywhere, especially when they increase complexity and fail to provide real value to the project.
Preplanning and the level it takes will depend on the scope of the project and the use case. A heart monitor or guidance system isn't the same as an eCommerce or entertainment website. Engineering discipline isn't required for the latter, but is for the former examples.
None of this accounts for the fact that over half the software industry has less than 5 years of experience and less than 25% has over 10. People with less than 5-10 years of experience can be brilliant, but there are things you learn with experience. First and foremost is the ability to push back and say, "No, it can't reasonably be done in X timeframe." Actually using scrum process to break things down in to chunks that can be swallowed. Not allowing mid-level developers to use "enterprise patterns" everywhere, especially when they increase complexity and fail to provide real value to the project.
Preplanning and the level it takes will depend on the scope of the project and the use case. A heart monitor or guidance system isn't the same as an eCommerce or entertainment website. Engineering discipline isn't required for the latter, but is for the former examples.
Reminds me of Michael A. Jackson's paper [0]. He specifically discusses this very same issue. I think it was posted on HN a while back but can't dig up the link.
0. http://mcs.open.ac.uk/mj665/FoSEZurich2010.pdf
0. http://mcs.open.ac.uk/mj665/FoSEZurich2010.pdf
Typing systems and language choices are really small potatoes in this field. There are much more scary problems to solve to make software work right.
When you can side channel attack any CPU with a branch predictor and bits get flipped regularly by cosmic rays at high altitude then you need to rethink what software engineering really means. The big problems are rarely actually about the software or proving some algorithm is correct.
When you can side channel attack any CPU with a branch predictor and bits get flipped regularly by cosmic rays at high altitude then you need to rethink what software engineering really means. The big problems are rarely actually about the software or proving some algorithm is correct.
Ah yes, real engineers anticipate the side channel attack that can happen when someone releases a butterfly at precisely the right location and precisely the right time so that it flaps its wings that affects the climate 10 years later that affects the cosmic rays that cause electrons to misbehave in all the CPUs across all the Docker containers on all the hosts in all the data centers and availability zones that causes all the right double entry debits and credits across multiple bank accounts and across multiple banking companies to steal some money.
This is a much bigger problem than type safety.
But luckily that is a hardware problem.
This is a much bigger problem than type safety.
But luckily that is a hardware problem.
[deleted]
[deleted]
In my country, all engineers are protected titles and basically just says you spent half of your 3 or 5 years studying math. Not even algorithms, just math. Makes it very confusing when teens calls themselves SEs
We could start by not allowing pseudo-engineers out of bootcamps.
Yes, specs and tests don't cover everything, however without them software is much worse.
I guess only an increase in lawsuits and product returns will eventually change the status quo.
Yes, specs and tests don't cover everything, however without them software is much worse.
I guess only an increase in lawsuits and product returns will eventually change the status quo.
Why? Because those with CS degrees don’t make mistakes either? Or how about self taught engineers, do they not count?
A problem is software is so easily created, destroyed and refactored. Combined with business interests pushing for all sorts of things midway through. How often are bridges built halfway when their entire spec changes, oh just convert his half done 2 lane bridge to a 4 lane double decker, should be no problem right? That’s how we treat software.
A problem is software is so easily created, destroyed and refactored. Combined with business interests pushing for all sorts of things midway through. How often are bridges built halfway when their entire spec changes, oh just convert his half done 2 lane bridge to a 4 lane double decker, should be no problem right? That’s how we treat software.
Engineering is a professional title, with a set of requirements that express a certain level of skill, moral and ethics.
Self taught, yes if they can attain the expected level of the Engineering Order.
Self taught, yes if they can attain the expected level of the Engineering Order.
> the misapplication of the word "engineer" to software developers.
Is it? you didn't say how it's misapplied.
Is it? you didn't say how it's misapplied.
Speaking as someone who builds mission critical software, licensing is not the answer to this problem. Occupational licensing only serves to raise barriers to entry and crowd out competition without raising quality. To put my opinion bluntly, Bernie Madoff had a license.
The things you should do to prevent catastrophic failure:
- Reduce your attack surface as much as possible.
- Automate your infrastructure, human input should be nonexistent
- Write a lot of automated tests
- Take the time to learn attack methods and actively try to break your own systems
- Have an external firm that will try to break your systems
- Add additional redundancy for hot spots in codebase
- Code reviews are mandatory and you should follow an internal style guide.
- Adopt a maintenance first culture, it should take less than 48 hours for you to be on the most recent version of all your dependencies. Bug fixes are your #1 priority.
- Educate your customers on what your potential failure conditions are, your recovery point objectives, and your recovery time objectives.
- If you do have a failure, fail loudly in a way that a customer can understand, report the failure and hopefully have a backup plan that users know and can run through.
- Customer support is a responsibility of the engineering department.
- Learn from others and contribute back with your tooling.
You can write mission critical code in any language. (Some are definitely better than others) You can't manage a mission critical application without investing in great operations.
Read google's SRE books for more practical information: https://landing.google.com/sre/books/
The things you should do to prevent catastrophic failure:
- Reduce your attack surface as much as possible.
- Automate your infrastructure, human input should be nonexistent
- Write a lot of automated tests
- Take the time to learn attack methods and actively try to break your own systems
- Have an external firm that will try to break your systems
- Add additional redundancy for hot spots in codebase
- Code reviews are mandatory and you should follow an internal style guide.
- Adopt a maintenance first culture, it should take less than 48 hours for you to be on the most recent version of all your dependencies. Bug fixes are your #1 priority.
- Educate your customers on what your potential failure conditions are, your recovery point objectives, and your recovery time objectives.
- If you do have a failure, fail loudly in a way that a customer can understand, report the failure and hopefully have a backup plan that users know and can run through.
- Customer support is a responsibility of the engineering department.
- Learn from others and contribute back with your tooling.
You can write mission critical code in any language. (Some are definitely better than others) You can't manage a mission critical application without investing in great operations.
Read google's SRE books for more practical information: https://landing.google.com/sre/books/
> Occupational licensing only serves to raise barriers to entry and crowd out competition without raising quality. To put my opinion bluntly, Bernie Madoff had a license.
Almost every country in the world established at one point or another some kind of licensing for various professions. I think the barrier of "this is all bullshit" is a bit higher than one bad apple for something found to be a good idea by so many people. Do you have some supporting arguments for your position?
Almost every country in the world established at one point or another some kind of licensing for various professions. I think the barrier of "this is all bullshit" is a bit higher than one bad apple for something found to be a good idea by so many people. Do you have some supporting arguments for your position?
Look at teaching in the United States. All states require teacher licensing at public schools. However private schools do not and get better outcomes. Same thing with many unions for trades. I have yet to see any evidence a union electrician or plumber is better than a non-union worker, however unions do all they can to keep competition off the market.
> All states require teacher licensing at public schools. However private schools do not and get better outcomes.
Some private schools get better outcomes than some public schools, but that is not universally true. The factors involved in outcomes in education are a whole lot more complicated than teacher licensing.+
Some private schools get better outcomes than some public schools, but that is not universally true. The factors involved in outcomes in education are a whole lot more complicated than teacher licensing.+
Would you be willing to be treated by a Doctor without a medical license or be represented by an attorney who hasn’t taken or passed the bar exam?
Me? In a world where such licenses were not mandatory, i.e. the doctor in question wasn't actually breaking any laws? Sure.
I'd probably want some alternative evidence of competence of course, like a long stream of happy, healthy, cured patients who could vouch for him. Or alternatively some sort of private sector approval scheme, based on trademark licensing. Sort of like how you can't claim a phone supports Bluetooth unless it's been licensed and approved.
But in general I don't see any reason to assume the specific licensing regimes in place today are so grea.t
I'd probably want some alternative evidence of competence of course, like a long stream of happy, healthy, cured patients who could vouch for him. Or alternatively some sort of private sector approval scheme, based on trademark licensing. Sort of like how you can't claim a phone supports Bluetooth unless it's been licensed and approved.
But in general I don't see any reason to assume the specific licensing regimes in place today are so grea.t
Sure, this report does a good job summarizing the tradeoffs of occupational licensure. If I were to highlight a section to read it would be pages 13 and 14.
https://obamawhitehouse.archives.gov/sites/default/files/doc...
https://obamawhitehouse.archives.gov/sites/default/files/doc...
"one bad apple destroys the whole batch"
you need one bad example to disprove the validity of this kind of approach, unless standards are enforced to get the bad apples out as early and as quickly as possible.
The given example was send to jail until the end of his life. Looks like standards enforced to me.
It's also the whole point of licensing: You have a standard you can enforce and if people fail to fulfill it there are consequences.
It's also the whole point of licensing: You have a standard you can enforce and if people fail to fulfill it there are consequences.
This is all very reasonable, but assumes that avoiding catastrophic failure is the most profitable course of action.
Still a big difference between mission critical (money) and safety critical (lives).
Safety critical encourages use of low variability languages, not just any language will do.
Also expects that software will fail at some point, so encourages architecture that is tolerant of this where the consequences warrant, eg redundancy and diversity such as 2oo3 voting a la airbus and space shuttle.
So many other things, but it always annoys me when the "IT" guys get involved in the safety software engineering just because there is a PC used to program the device - they just don't get that there is no suitable attitude of "don't worry how or why it got lost, just send that email again". Tell that to the widow.
Safety critical encourages use of low variability languages, not just any language will do.
Also expects that software will fail at some point, so encourages architecture that is tolerant of this where the consequences warrant, eg redundancy and diversity such as 2oo3 voting a la airbus and space shuttle.
So many other things, but it always annoys me when the "IT" guys get involved in the safety software engineering just because there is a PC used to program the device - they just don't get that there is no suitable attitude of "don't worry how or why it got lost, just send that email again". Tell that to the widow.
> Have an external firm that will try to break your systems
This is by far the most important one, and in fact, I would argue that having external code or design audit would be incredibly useful as well.
I work in a regulated area and it is amazing how well we do the things that are audited compared to the things that are not. In fact, all our systems are neatly divided into those that get audited and those that don't specifically so that we can apply all our higher standards to the audited ones. It is also important to recognise how empowering it is especially for less senior staff to be able to reference an standard or guideline and the point out that a flaw is vulnerable to discovery by audit.
This is by far the most important one, and in fact, I would argue that having external code or design audit would be incredibly useful as well.
I work in a regulated area and it is amazing how well we do the things that are audited compared to the things that are not. In fact, all our systems are neatly divided into those that get audited and those that don't specifically so that we can apply all our higher standards to the audited ones. It is also important to recognise how empowering it is especially for less senior staff to be able to reference an standard or guideline and the point out that a flaw is vulnerable to discovery by audit.
My favorite from your insightful recommendations: actively try to break your own systems.
"Chaos monkeys" is a wonderful phrase and concept I picked up somewhere, as a strategy for testing and improving the resilience of systems. Since I learned of it, I've been introducing chaotic elements to test a wider range of inputs/behaviors, and indeed it has uncovered failing edge cases and helped to make application logic more robust.
"Chaos monkeys" is a wonderful phrase and concept I picked up somewhere, as a strategy for testing and improving the resilience of systems. Since I learned of it, I've been introducing chaotic elements to test a wider range of inputs/behaviors, and indeed it has uncovered failing edge cases and helped to make application logic more robust.
It's the name of Netflix's tool.
https://netflix.github.io/chaosmonkey/
https://netflix.github.io/chaosmonkey/
And don't forget:
- Impose civil and criminal liability for failures that can be traced to not following these standards
Gotta have teeth.
- Impose civil and criminal liability for failures that can be traced to not following these standards
Gotta have teeth.
"Imagine joining an engineering team...You start by meeting Mary, project leader for a bridge in a major metropolitan area. Mary introduces you to Fred, after you get through the fifteen security checks installed by Dave because Dave had his sweater stolen off his desk once and Never Again. Fred only works with wood, so you ask why he’s involved because this bridge is supposed to allow rush-hour traffic full of cars full of mortal humans to cross a 200-foot drop over rapids. Don’t worry, says Mary, Fred’s going to handle the walkways. What walkways? Well Fred made a good case for walkways and they’re going to add to the bridge’s appeal. Of course, they’ll have to be built without railings, because there’s a strict no railings rule enforced by Phil, who’s not an engineer. Nobody’s sure what Phil does, but it’s definitely full of synergy and has to do with upper management, whom none of the engineers want to deal with so they just let Phil do what he wants. Sara, meanwhile, has found several hemorrhaging-edge paving techniques, and worked them all into the bridge design, so you’ll have to build around each one as the bridge progresses, since each one means different underlying support and safety concerns. Tom and Harry have been working together for years, but have an ongoing feud over whether to use metric or imperial measurements, and it’s become a case of “whoever got to that part of the design first.” This has been such a headache for the people actually screwing things together, they’ve given up and just forced, hammered, or welded their way through the day with whatever parts were handy. Also, the bridge was designed as a suspension bridge, but nobody actually knew how to build a suspension bridge, so they got halfway through it and then just added extra support columns to keep the thing standing, but they left the suspension cables because they’re still sort of holding up parts of the bridge. Nobody knows which parts, but everybody’s pretty sure they’re important parts. After the introductions are made, you are invited to come up with some new ideas, but you don’t have any because you’re a propulsion engineer and don’t know anything about bridges.
Would you drive across this bridge? No. If it somehow got built, everybody involved would be executed. Yet some version of this dynamic wrote every single program you have ever used, banking software, websites, and a ubiquitously used program that was supposed to protect information on the internet but didn’t."
https://www.stilldrinking.org/programming-sucks
Would you drive across this bridge? No. If it somehow got built, everybody involved would be executed. Yet some version of this dynamic wrote every single program you have ever used, banking software, websites, and a ubiquitously used program that was supposed to protect information on the internet but didn’t."
https://www.stilldrinking.org/programming-sucks
Licensing isn't the solution here. Software changes too quickly, while institutionalized tests and textbooks, and certifications move incredibly slowly. Treating this industry the same as others will hopelessly slow down the already glacial pace of government software, while innovation will continue outside the system.
Software is a different realm, as other commenters point out. When faced with a problem in the software world, what's the solution? More software!
The accrediting bodies that regulators might want to form should focus on free test suites and tools for gauging security. Static code analyzers, auto penetration testers, something along these lines.
Regulate the quality of the software, not the software developers. It's easier to objectively test.
Software is a different realm, as other commenters point out. When faced with a problem in the software world, what's the solution? More software!
The accrediting bodies that regulators might want to form should focus on free test suites and tools for gauging security. Static code analyzers, auto penetration testers, something along these lines.
Regulate the quality of the software, not the software developers. It's easier to objectively test.
The math doesn't change. High reliability software has the same quality as high reliability knowledge, namely it has been proven logically to be correct.
The problem with software is that, for almost all software, the only specification for what it is supposed to do is discovered as the software is written. Nobody knows all the data, the data semantics, its dependencies, what output is really required and how that it to be computed from that data. Virtually all cases, and certainly all software I've ever been involved with, all this is discovered on the fly during the development effort. We even encode this into our processes, after all one of the main reason agile and similar processes exist is the understanding that we need to iterate to discover what the spec should be.
If a full specification is truly known up front and validated, then building software that works correctly to that specification is a well understood problem with well understood processes (sure, much more time consuming and expensive than regular software engineering, but that is a different category of issue)
But look at the Boeing 737-Max8 problem: the issue doesn't appear to have been the implementation of the software, the problem seems to have been that the specification itself was poorly constructed for business reasons. And now software gets the blame? Sigh.
If a full specification is truly known up front and validated, then building software that works correctly to that specification is a well understood problem with well understood processes (sure, much more time consuming and expensive than regular software engineering, but that is a different category of issue)
But look at the Boeing 737-Max8 problem: the issue doesn't appear to have been the implementation of the software, the problem seems to have been that the specification itself was poorly constructed for business reasons. And now software gets the blame? Sigh.
Civil engineers aren't asked to design a small 2-story building that can be retrofitted as a 300-story skyscraper when the time comes. Or connect two buildings 15km apart over the weekend.
Design: "A bridge that can deploy itself on any river, in any city, any climate, safe in any whether, handle any vehicle (even ones designed after the bridge was deployed), with guaranteed time to cross (99th percentile under 5s), and regularly replace itself with newer version (while people are using it). It also has to be nuclear-attack resistant, and prevent 99% suicides. All built from readily available, standard components. On $100k budget, in 3 months."
Or make all the windows suddenly round, two days before the tenants move in...
Exactly. I could make my stuff super rock solid if I could focus on that for the next few years. I am sure we can improve but the main reason for failing software is the speed we are developing at. And the fact that organizational learning gets prevented by constantly hiring “fresh” people who have no knowledge of the work that was done previously.
Certification already exists - called Functional Safety Engineer, issued by people like Exida and TUV.
I hold this Certification. It is painfully slow to use performance based standards to extract users risks and needs and then produce software designed and tested to standards like IEC 61511 and IEC 61508.
They are basically Q/A standards, encourage use of the V-model, defined life cycle activities etc etc in an attempt to produce critical software to an estimated known quality.
When you do the calcs the equipment is usually the easier part, it is almost always the humans that end up causing the problems. It is amazing how creative they can be to do a task, or to not do a task.
I hold this Certification. It is painfully slow to use performance based standards to extract users risks and needs and then produce software designed and tested to standards like IEC 61511 and IEC 61508.
They are basically Q/A standards, encourage use of the V-model, defined life cycle activities etc etc in an attempt to produce critical software to an estimated known quality.
When you do the calcs the equipment is usually the easier part, it is almost always the humans that end up causing the problems. It is amazing how creative they can be to do a task, or to not do a task.
Kind of sad that out of 83 comments, only 1 mentions "functional safety".
If you want to write SW for, say, cars, ISO safety standards exist, and you generally get your work and development audited by a 3rd party agency who will confirm whether you've conformed to the safety standards.
Conforming to those standards is not fun. And obviously, conforming to them doesn't mean your SW is actually "safe". If most engineers, managers, and auditors follow the spirit of the standards, it probably is safe.
You really don't want such rigorous standards for your average SW product.
If you want to write SW for, say, cars, ISO safety standards exist, and you generally get your work and development audited by a 3rd party agency who will confirm whether you've conformed to the safety standards.
Conforming to those standards is not fun. And obviously, conforming to them doesn't mean your SW is actually "safe". If most engineers, managers, and auditors follow the spirit of the standards, it probably is safe.
You really don't want such rigorous standards for your average SW product.
Yeah ISO 26262 is a derived standard from the parent 61508, as is 61511 for process sector.
It is all about risk management, probabilistic outcomes and not spending all your money on a highly visible/emotive risk and ignoring an equivalent risk that is pervasive but lower consequences.
A day to day example of how humans can be poor at intuitively allocating risk and avoidance is the immense fear some people have of the shark attack or plane crash, when they are statistically at way more risk driving to the beach or the airport, but think nothing of it. (Cars can be mighty convenient though).
In my case risk decisions are usually driven by the company risk matrix as to tolerable risk, the companies way of conveying the enterprise wide appetite for risk.
The risk matrix is also known as "the death quota" because it actually says how much is the maximum they are prepared to spend to avoid a fatality, or multiple fatalities, but you are never popular with the client when you frame their risk aversion guidance in that manner, not at all.
Usual guidance is that the company wants no real increase of risk of harm to employees as they experience in general public day to day life, around 1x10-5 chance of death per year in western countries.
Not thinking this sort of risk is part of the day to day thinking of most SV software engineers, because the consequences of your Uber going to the right street in the wrong suburb are undoubtedly significant to someone at the time, hardly likely to be life ending.
It is all about risk management, probabilistic outcomes and not spending all your money on a highly visible/emotive risk and ignoring an equivalent risk that is pervasive but lower consequences.
A day to day example of how humans can be poor at intuitively allocating risk and avoidance is the immense fear some people have of the shark attack or plane crash, when they are statistically at way more risk driving to the beach or the airport, but think nothing of it. (Cars can be mighty convenient though).
In my case risk decisions are usually driven by the company risk matrix as to tolerable risk, the companies way of conveying the enterprise wide appetite for risk.
The risk matrix is also known as "the death quota" because it actually says how much is the maximum they are prepared to spend to avoid a fatality, or multiple fatalities, but you are never popular with the client when you frame their risk aversion guidance in that manner, not at all.
Usual guidance is that the company wants no real increase of risk of harm to employees as they experience in general public day to day life, around 1x10-5 chance of death per year in western countries.
Not thinking this sort of risk is part of the day to day thinking of most SV software engineers, because the consequences of your Uber going to the right street in the wrong suburb are undoubtedly significant to someone at the time, hardly likely to be life ending.
Only when the cost of failure exceeds the cost of quality will we see real improvements. The company producing the software should bear this burden.
Licensure for software developers will only benefit the professional liability insurance industry.
Licensure for software developers will only benefit the professional liability insurance industry.
We can also reduce the effort to write better code. Certain language and tooling features make quality easier.
I think that's the key. There was ADA now there's Rust. If you can have GC on your system there are plenty of other languages that make strong static typing and other compile time guarantees that beat any analyzed C++.
Is static typing really what you want? I would think that if you want safety against failure, you really want redundant systems that live in separate failure domains and can coordinate in the event of failure.
Why can't you have that with static typing? Btw I think only strong static typing really helps (C is statically typed but allows way too many implicit conversions).
There's also the question of what you regard as safety. Fault tolerance (like in elixir/erlang), or a design for fewer faults (e.g. ADA/Rust)?
Fault tolerance is good for high availability, which can be a safety goal (e.g. What does your airplane do when all the computers are off?) it doesn't help you however if yoir software didn't crash but does the wrong thing (like the boeing).
For the second part, strong static type systems definitely help, they provide clear documentation what can and can't be done, and they cause as many compile time errors if you do the wrong thing as possible. It doesn't rule everything out, but it gets you closer to the goal of correctness.
Fault tolerance is still needed. Usually you want to have redundancy (I think most airplanes use 5 controllers calculating at the same time to prevent atmospheric radiation causing bit flips to cause issues). And you want the system to go into safe states (e.g. Detect a fault and reinitialize, if that doesn't work do your best to stay operational until you've reached safety).
I think we could probably take over some of the supervision ideas from elixir, but combine it with strong static typing guarantees.
There's also the question of what you regard as safety. Fault tolerance (like in elixir/erlang), or a design for fewer faults (e.g. ADA/Rust)?
Fault tolerance is good for high availability, which can be a safety goal (e.g. What does your airplane do when all the computers are off?) it doesn't help you however if yoir software didn't crash but does the wrong thing (like the boeing).
For the second part, strong static type systems definitely help, they provide clear documentation what can and can't be done, and they cause as many compile time errors if you do the wrong thing as possible. It doesn't rule everything out, but it gets you closer to the goal of correctness.
Fault tolerance is still needed. Usually you want to have redundancy (I think most airplanes use 5 controllers calculating at the same time to prevent atmospheric radiation causing bit flips to cause issues). And you want the system to go into safe states (e.g. Detect a fault and reinitialize, if that doesn't work do your best to stay operational until you've reached safety).
I think we could probably take over some of the supervision ideas from elixir, but combine it with strong static typing guarantees.
Static typing doesn't help you if your software doesn't crash but does the wrong thing (like the Boeing). In that case not even dimensional analysis helps you, because by my understanding that code was really stupid.
I find that if you're only documenting via types, that's a problem. You should also document by writing documentation.
I find that if you're only documenting via types, that's a problem. You should also document by writing documentation.
I dont think anybody argues that types replace documentation. However, documentation without types tends to get repetitive and requires you to come up with ways to structure it properly. Turns out, you can encode a lot of information into a type and now it can be automatically verified and frees you to write documentation for things that can not be expressed in types.
Our safety critical languages need to be simpler, so they are amenable to mechanical proofs and exhaustive model checking. Dimensional units would be a good start.
https://gmpreussner.com/research/dimensional-analysis-in-pro...
https://gmpreussner.com/research/dimensional-analysis-in-pro...
F# has those built in (not mission critical embedded ready, but a very very nicely designed language).
ADA hat strongly typed fixed point built in. Rust I think has a strong enough type system to implement this as a library, at least there seem to be a few.
Idris also looks interesting. Maybe something like this can be done with dependent types.
ADA hat strongly typed fixed point built in. Rust I think has a strong enough type system to implement this as a library, at least there seem to be a few.
Idris also looks interesting. Maybe something like this can be done with dependent types.
Nothing new under the sun... MISRA C and Autosar were introduced around 1998, so we can also say standards for safe software are there for decades (around 2 decades but always). You still get companies skipping best practices, guidlines, etc.
What is funny you get the same stuff in construction industry. Companies take shortcuts, architects take shortcuts and there are buildings, bridges collapsing.
I just don't like dichotomy "proper engineers, mechanical, construction", "kids in the fog, software devs". There are tons of reliable software, and tons of buildings that might collapse tomorrow because of wind/temperature/vibrations which never were considered by any builder or architect...
What is funny you get the same stuff in construction industry. Companies take shortcuts, architects take shortcuts and there are buildings, bridges collapsing.
I just don't like dichotomy "proper engineers, mechanical, construction", "kids in the fog, software devs". There are tons of reliable software, and tons of buildings that might collapse tomorrow because of wind/temperature/vibrations which never were considered by any builder or architect...
Even in the mechanical world, chicken coops aren't engineered to the same standards as skyscrapers. In software, we can build to skyscraper standards when we need to. But most of our software is done like we're building a chicken coop. And that's actually appropriate for some of it - your web app probably isn't the software equivalent of a skyscraper.
I don't know how Boeing is doing their software but 737 Max is nowhere near chicken coop. I kind of believe they use something like MISRA.
Article also is not saying anything about web applications.
What I have seen on my own eyes was for example airport parking lot in Eindhoven which is also not a chicken coop: https://nltimes.nl/2017/09/25/eindhoven-airport-parking-gara...
On the other hand I have never seen collapsed chicken coop.
Article also is not saying anything about web applications.
What I have seen on my own eyes was for example airport parking lot in Eindhoven which is also not a chicken coop: https://nltimes.nl/2017/09/25/eindhoven-airport-parking-gara...
On the other hand I have never seen collapsed chicken coop.
The problem is that you can be nearly 100% Misra compliant and still have shit software.
It takes away about 90% of the footguns you have in C and C++, but those 10% are just never going to be covered by a coding standard.
Furthermore it's not only about using the worst language for safety in a corset. The disagreeing AoAs, the lack of a warning signal, the weird UI to disable Mcas. Nothing in MISRA or similar coding standards prevents these.
It's good old common sense that you sadly can't teach (although you should maybe pay for a design course for your software developers). If not at least read the design of everyday things. Most of the design problems are the same that downed these two planes.
It takes away about 90% of the footguns you have in C and C++, but those 10% are just never going to be covered by a coding standard.
Furthermore it's not only about using the worst language for safety in a corset. The disagreeing AoAs, the lack of a warning signal, the weird UI to disable Mcas. Nothing in MISRA or similar coding standards prevents these.
It's good old common sense that you sadly can't teach (although you should maybe pay for a design course for your software developers). If not at least read the design of everyday things. Most of the design problems are the same that downed these two planes.
I would hope they're using Ada.
Licensing is not the solution.
Here are some things that can help:
1) Limit mutations, shared state, and side effects to very small sections of code. The majority of code should be pure functions and immutable code.
2) Tooling and IDE's should have "code coverage" to show where code is pure and referentially transparent and where it is not.
3) Learn from mathematics / computer science. Adopt formal methods and proof based techniques.
4) Keep code simple and understandable. Category theory has many insights on how to better compose and structure code. Code that is in harmony with certain laws will be more predictable, simpler, and more reusable. Constraints == freedom.
5) Use strong typing. And by typing I mean types that limit what values can be and even what code can do. Types that are nullable are not sufficient for safety. Types that only describe the data format and not the allowed functionality are not sufficiently safe or descriptive. Dependent typing allows for even greater safety because it can specify acceptable values ("tolerances") not just the data format of the value.
6) Adopt metrics to measure complexity and other factors that affect reliability of code. Once we have metrics we can have better discussions different ways to write the same code and why certain ways might be better under certain circumstances.
7a) Expand your vocabulary of concepts. Words affect the very thoughts you are capable of thinking about (linguistic relativity). We went from machine code to assembly to higher level languages. Each requires more concepts to know and yet it gets simpler and less error prone each time. The more words you have the easier it is to describe, write, and understand code. Category theory has many new concepts that are not in common use that improve communication and understanding. What other fields can we borrow from?
7b) Keep code short and simple to maximize understanding. More code means more surface area for software defects. 'map' is easier to understand and less error prone than 2 different arrays, counters, conditionals, increments, and mutations.
8) Typing and pattern matching provide a framework where the compiler can determine if you have handled all the cases for all possible inputs (total function). Failure to do so is a compile time error.
Here are some things that can help:
1) Limit mutations, shared state, and side effects to very small sections of code. The majority of code should be pure functions and immutable code.
2) Tooling and IDE's should have "code coverage" to show where code is pure and referentially transparent and where it is not.
3) Learn from mathematics / computer science. Adopt formal methods and proof based techniques.
4) Keep code simple and understandable. Category theory has many insights on how to better compose and structure code. Code that is in harmony with certain laws will be more predictable, simpler, and more reusable. Constraints == freedom.
5) Use strong typing. And by typing I mean types that limit what values can be and even what code can do. Types that are nullable are not sufficient for safety. Types that only describe the data format and not the allowed functionality are not sufficiently safe or descriptive. Dependent typing allows for even greater safety because it can specify acceptable values ("tolerances") not just the data format of the value.
6) Adopt metrics to measure complexity and other factors that affect reliability of code. Once we have metrics we can have better discussions different ways to write the same code and why certain ways might be better under certain circumstances.
7a) Expand your vocabulary of concepts. Words affect the very thoughts you are capable of thinking about (linguistic relativity). We went from machine code to assembly to higher level languages. Each requires more concepts to know and yet it gets simpler and less error prone each time. The more words you have the easier it is to describe, write, and understand code. Category theory has many new concepts that are not in common use that improve communication and understanding. What other fields can we borrow from?
7b) Keep code short and simple to maximize understanding. More code means more surface area for software defects. 'map' is easier to understand and less error prone than 2 different arrays, counters, conditionals, increments, and mutations.
8) Typing and pattern matching provide a framework where the compiler can determine if you have handled all the cases for all possible inputs (total function). Failure to do so is a compile time error.
> Licensing is not the solution.
> Here are some things that can help:
Maybe license people who can demonstrate that they really understand this stuff then?
Maybe license people who can demonstrate that they really understand this stuff then?
Certification might be less heavy handed and flexible. It keeps control within the industry and out of gov't where there can be lots of other inefficiency and conflicting interests.
Laws also tend to be very slow to change (software moves too fast). If colleges are struggling to keep up with relevant skills how much worse will it be for laws?
Licensing typically also creates a paywall (student loan debt) and denies access to otherwise bright and capable people who don't have the resources to go through a college / licensing system.
Laws also tend to be very slow to change (software moves too fast). If colleges are struggling to keep up with relevant skills how much worse will it be for laws?
Licensing typically also creates a paywall (student loan debt) and denies access to otherwise bright and capable people who don't have the resources to go through a college / licensing system.
Hold executives and VC personally liable for the failures.
Crunch; poor office environments; PMs, sales, marketing, and so on, lying to customers/stakeholders; bad tools; technical debt; or the like; are beyond the control of software developers and contribute greatly to software failure.
Crunch; poor office environments; PMs, sales, marketing, and so on, lying to customers/stakeholders; bad tools; technical debt; or the like; are beyond the control of software developers and contribute greatly to software failure.
Licensing will stifle innovation and inflate cost. Licensing is the old guild approach to control the labor pool and control the market. It's really the wrong approach to the problem.
Without getting too philosophical about _why_ things fail (yes I'm cheating), I reckon there are only two practical generic things you can do:
1. Make it very small (how small is subjective and language dependent of course).
2. Make the large part fault tolerant and recoverable, e.g triple redundancy. (or make the lower level part fault tolerant).
Leaving a third option of making perfect software in the face of immense complexity sounds more like magic to me, I doubt any solution in this area could be generalised.
1. Make it very small (how small is subjective and language dependent of course).
2. Make the large part fault tolerant and recoverable, e.g triple redundancy. (or make the lower level part fault tolerant).
Leaving a third option of making perfect software in the face of immense complexity sounds more like magic to me, I doubt any solution in this area could be generalised.
Don’t make excuses to yourself when you know there is a problem with your code. Don’t say “well, it will only fail in these edge cases and I know not to use it that way.” You’ll forget, or someone else will use it the wrong way and it will break. The easiest time to fix it is when you are already working on the code and first notice the problem, not later.
I think software systems whose failure directly jeopardizes human lives should be engineered, and should be defined and enforced legally. Systems that don't directly risk human life should not be so constrained.
Defining that standard and enforcing it is a huge task which could easily go very wrong.
Businesses would of course be free to apply the "engineered" standard to any systems they choose to. Outside of mission-critical systems operating at gigantic scale (FAAMANG-level) I doubt it would often be profitable to do that.
The hypothetical standard would also be less relevant when building proofs-of-concept, so long as the POC is not usefully deployable (if it is there's risk of someone deploying it despite it being unsafe).
In the end, most software does not put human lives directly at risk. Acting like it does would waste resources and doom many small businesses whose profit margins couldn't absorb the costs of genuine engineering (it could kill several companies I have worked for).
Even software systems that do risk human lives don't do so in all subsystems - as far as I know, painting a mural in your skyscraper's lobby requires no engineers to be involved. Similar distinctions may be reasonable in sufficiently-isolated components of software packages that do some life-protection tasks. Maybe allow a formal verification to show that subsystem X cannot impact the critical subsystems and therefore does not need the same level of rigor?
Defining that standard and enforcing it is a huge task which could easily go very wrong.
Businesses would of course be free to apply the "engineered" standard to any systems they choose to. Outside of mission-critical systems operating at gigantic scale (FAAMANG-level) I doubt it would often be profitable to do that.
The hypothetical standard would also be less relevant when building proofs-of-concept, so long as the POC is not usefully deployable (if it is there's risk of someone deploying it despite it being unsafe).
In the end, most software does not put human lives directly at risk. Acting like it does would waste resources and doom many small businesses whose profit margins couldn't absorb the costs of genuine engineering (it could kill several companies I have worked for).
Even software systems that do risk human lives don't do so in all subsystems - as far as I know, painting a mural in your skyscraper's lobby requires no engineers to be involved. Similar distinctions may be reasonable in sufficiently-isolated components of software packages that do some life-protection tasks. Maybe allow a formal verification to show that subsystem X cannot impact the critical subsystems and therefore does not need the same level of rigor?
The problem is: most of the worst software disasters have been perpetrated by exactly the kind of people who would get licensed under such a system, while much of the most important software in history has been written by people who would never get licensed (Linux, Python, etc etc). As appealing as it is to try and solve the software quality challenge through regulation, all the evidence seems to be that it just wouldn't work and would probably be highly counter productive.
There's only one answer: exhaustive testing and attention to detail, NASA style. With NASA it's probably worth it. The rest of us, not so much.
Like efficax says, there's an oft-used analogy between software "engineering" and structural engineering that I think is way overplayed. The better analogy, IMO, is between software and business.
What "CEO" means is basically "authority and responsibility." We have some guardrails called laws, but outside of that we acknowledge that "CEO'ing" is a complex, ambiguous task requiring personal judgment, and is not reducible beyond that.
"How careful should we be?" is a question with some interesting characteristics:
- it's a matter of opinion. How risk-averse are the shareholders?
- it's a matter of context. On the CSS for the company promo site: eh. On the embedded code running the flight-control system---very. But what if that stems from MVP code that originally ran in a simulation on the company promo site?
I mean, what you really need here, is for your board members to write the code.
I'm speaking tongue-in-cheek a bit, but I'm not sure it's 100% wrong, maybe just 75%. In a very real way, the authority-responsibility-reward lineups are so out of whack in most organizations. Which is not to say I know what they should look like.
(Two words I need to think about more: "Software insurance")
Like efficax says, there's an oft-used analogy between software "engineering" and structural engineering that I think is way overplayed. The better analogy, IMO, is between software and business.
What "CEO" means is basically "authority and responsibility." We have some guardrails called laws, but outside of that we acknowledge that "CEO'ing" is a complex, ambiguous task requiring personal judgment, and is not reducible beyond that.
"How careful should we be?" is a question with some interesting characteristics:
- it's a matter of opinion. How risk-averse are the shareholders?
- it's a matter of context. On the CSS for the company promo site: eh. On the embedded code running the flight-control system---very. But what if that stems from MVP code that originally ran in a simulation on the company promo site?
I mean, what you really need here, is for your board members to write the code.
I'm speaking tongue-in-cheek a bit, but I'm not sure it's 100% wrong, maybe just 75%. In a very real way, the authority-responsibility-reward lineups are so out of whack in most organizations. Which is not to say I know what they should look like.
(Two words I need to think about more: "Software insurance")
ah, here we go
fast forward 25 years and we'll need a government approved license to open dev tools in a browser.
I hope I am wrong.
fast forward 25 years and we'll need a government approved license to open dev tools in a browser.
I hope I am wrong.
Sure there will be a bypass available on the black market which every kid will pass around to each other as a matter of course. software isn't the same as hardware, it's almost impossible to limit access unless you go full on draconian government - which might happen,granted.
Yeah i think this is the 'software engineers are not REAL engineers'. There is a fair argument on both sides and the article raises an interesting point. Perhaps the root cause is how new & rapid all this tech has come across our world. Folks have been building things thousands of years right?
We've only been building scalable docker microservice agile jira lambda functions for a few years now. Curious to see where time takes us. Change takes a while to come along, companies and people have been getting hacked for many years but i havent heard any whispers of change.
We've only been building scalable docker microservice agile jira lambda functions for a few years now. Curious to see where time takes us. Change takes a while to come along, companies and people have been getting hacked for many years but i havent heard any whispers of change.
I love how in this article "If industry fails to self-regulate, governments might seize the opportunity." is considered a bad outcome.
But indeed no one forbids anyone from providing certifications or imposing them. I actually have a diploma that says (rightly) that I know how to write provable code and good software. Only once have I been asked to do so by my clients.
I, personally and unoriginally, blame Microsoft for the current situation. "Runs on Windows" is a common requirement and when it is present, you know your software won't be held to a higher standard than Microsoft's is.
If we had a certification that for instance required that we are 100% certain that a given process can not be interrupted for a software reason, we would need to examine Windows code or blindly trust Microsoft, rendering the first point of the certification moot.
I know how to write realtime code where I can give hard guarantees on the time it takes to finish a task. But it requires an explicit and transparent scheduling system, a known IRQ system and a precise account of the number of cycles necessary to execute a given procedure.
Maybe it can happen now, but until recently, a certification procedure that would require an open OS would have been seen as ideological.
But indeed no one forbids anyone from providing certifications or imposing them. I actually have a diploma that says (rightly) that I know how to write provable code and good software. Only once have I been asked to do so by my clients.
I, personally and unoriginally, blame Microsoft for the current situation. "Runs on Windows" is a common requirement and when it is present, you know your software won't be held to a higher standard than Microsoft's is.
If we had a certification that for instance required that we are 100% certain that a given process can not be interrupted for a software reason, we would need to examine Windows code or blindly trust Microsoft, rendering the first point of the certification moot.
I know how to write realtime code where I can give hard guarantees on the time it takes to finish a task. But it requires an explicit and transparent scheduling system, a known IRQ system and a precise account of the number of cycles necessary to execute a given procedure.
Maybe it can happen now, but until recently, a certification procedure that would require an open OS would have been seen as ideological.
The first question when anyone proposes licensing as a solution to anything in this field should always be what the objective criteria will be for granting a licence. Until there is some authority qualified to answer that question on a credible technical basis, any legally mandated licensing scheme will just be a tool for lawyers, "thought leaders", insurance firms and other people who don't actually make useful software to beat up those who do.
"Consider the construction industry, which has had formal standards in place for decades."
The construction industry has existed, in some form, for centuries. It has had time to develop these formal standards. Software is much younger, and the "formal standards" it should follow are still in flux.
The construction industry has existed, in some form, for centuries. It has had time to develop these formal standards. Software is much younger, and the "formal standards" it should follow are still in flux.
A sharper line between critical and non-critical software would help. Only some software needs to work well.
"All of these professionals have years of schooling and relevant work experience and have passed rigorous certification exams. [...] To start, coders who work on critical infrastructure should have a professional accreditation framework that issues licenses."
"mit.edu"
Uh, huh.
"mit.edu"
Uh, huh.
This is the MIT Sloan Management Review, not the university itself. The author never attended the university (based on his LinkedIn profile), rather he attended the University of Waterloo.
As i see, without failure, there'll be no improvement in software.
It's fact. Just look at frontend ecosystem. The next big thing is mostly the one that "fixes" previous failures.
You can't have both "improvement" and "without failure" though.
It's fact. Just look at frontend ecosystem. The next big thing is mostly the one that "fixes" previous failures.
You can't have both "improvement" and "without failure" though.
That question was asked, but the actual question that appears to have been answered is "What can we do to get all software development to move offshore?"
I certainly don't disagree but I'd be interested in discussing the nature of the licensing.
Don't try to handle all the edge cases, instead in the else give control to the user.
we can do better than licensing. we can have formal verification. soon
Formal verification is the future, and always will be.
The gap between what's doable and writing all software that way is epic, and it's not at all clear to me that its the best way forward.
Software is (potentially) about creating what didn't exist before, nailing everything to the floor based on past experience isn't going to lead anywhere worth going.
Maybe part of the solution is writing software that's not verifiable or rejected. Probably, since it has to be something we didn't try yet.
The gap between what's doable and writing all software that way is epic, and it's not at all clear to me that its the best way forward.
Software is (potentially) about creating what didn't exist before, nailing everything to the floor based on past experience isn't going to lead anywhere worth going.
Maybe part of the solution is writing software that's not verifiable or rejected. Probably, since it has to be something we didn't try yet.
[deleted]
wouldn't develop software like airplane takes us back to something very similar to waterfall process?
[deleted]
Nothing. Capitalism solves all problems. If not then wed be socialist.
We have already solved large areas of common classes of bugs at the language level, through strong static typing, memory safety and safe concurrency (Rust is the vanguard here but there are lots of languages that tackle these problems in effective ways).
Institutional inertia (technical debt) is the only reason why we can't apply these solutions to all software. I work on a PHP app with code that goes back 15 years. To convert it to rust we'd have to invest basically the same as the operating cost of the company. That'll never happen.
Once "safe" languages are used, there are still bugs of course. But what are those bugs? Failures of thought, plain and simple. Structural engineers work within the limits of physical reality and these constraints greatly limit the range of possible creation, thus also the range of possible failure.
Software has limits too, in memory, storage and processing capacity. But otherwise it works entirely in the realm of pure thought.
Well defined specifications that can be turned into concrete tests can solve a lot of these problems, too. (But this is also very expensive).
But even if you have specs and tests, there is no magic formula for ensuring your ideas are always sound. If the idea is wrong you can't even write tests to ensure it's correct. "Correctly" functioning components can have complex, difficult to predict side-effects when composed into complex systems.
We do have everything we need to write highly reliable software. But solving the software failure problem is the same task as solving the erroneous thinking problem. I don't know that we meat-sacks can "solve" that problem.