Responding to “Are bugs and slow delivery ok?”(uselessdevblog.wordpress.com)
uselessdevblog.wordpress.com
Responding to “Are bugs and slow delivery ok?”
https://uselessdevblog.wordpress.com/2023/07/03/responding-to-are-bugs-and-slow-delivery-ok-the-blog-post-that-ive-hated-the-most-ever/
127 comments
Thank you for (1). I was trying to explain to someone the other day how unpredictable coding is, but you're right, it's wildly unpredictable. Even for tiny projects they sometimes go so far off the rails.
I remember working on this one major project - an online music store - and one of the execs hired his very lovely wife to project manage a team of developers. She would create these fantastic charts in Microsoft Project and every day try to somehow - somehow - get the developers to either (a) give her timescales for each component, or (b) explain why previous component X took twice as long as they had scheduled.
I've been writing code for 40 years and I still get caught out myself. I try to give realistic deadlines and then some fucking bug in the compiler or other such insanity will come along and make me look like a fool.
I remember working on this one major project - an online music store - and one of the execs hired his very lovely wife to project manage a team of developers. She would create these fantastic charts in Microsoft Project and every day try to somehow - somehow - get the developers to either (a) give her timescales for each component, or (b) explain why previous component X took twice as long as they had scheduled.
I've been writing code for 40 years and I still get caught out myself. I try to give realistic deadlines and then some fucking bug in the compiler or other such insanity will come along and make me look like a fool.
> I've been writing code for 40 years and I still get caught out myself. I try to give realistic deadlines [...]
Call it immaturity, or perhaps just not being a "team player", but I've been coding for 15 years and stopped giving time estimates about 5 years ago.
I simply refuse, and instead, explain that I would just be making something up, and I'm not interested in taking responsibility for an arbitrary guess, since I know from experience how it can come back to haunt me.
I try to suggest saving such speculation for a time, hopefully in the near future, where the initial research has been completed and _some_ progress has been made on the foundation of the requested work, so that my speculation can have some basis in reality.
If all else fails, I give a wild estimate that I don't expect anyone to agree to, such as 2 years for work that might not even take 2 or 3 months.
Call it immaturity, or perhaps just not being a "team player", but I've been coding for 15 years and stopped giving time estimates about 5 years ago.
I simply refuse, and instead, explain that I would just be making something up, and I'm not interested in taking responsibility for an arbitrary guess, since I know from experience how it can come back to haunt me.
I try to suggest saving such speculation for a time, hopefully in the near future, where the initial research has been completed and _some_ progress has been made on the foundation of the requested work, so that my speculation can have some basis in reality.
If all else fails, I give a wild estimate that I don't expect anyone to agree to, such as 2 years for work that might not even take 2 or 3 months.
[deleted]
At some point I ran across this strategy for estimating when you’re the one doing it: Take whatever time you think it’ll take, then double it. Then add 20%*
Things still go off the rails of course making an an imperfect solution, but it ends up being more right than any other method I’ve tried.
*If there are any unknown unknowns I might start multiplying by the number of big ones.
Things still go off the rails of course making an an imperfect solution, but it ends up being more right than any other method I’ve tried.
*If there are any unknown unknowns I might start multiplying by the number of big ones.
I always double it. I think I'm missing the "add 20%" bit...
If the code you write is uncovering bugs in compiler, you literally are too smart for your own good. Fwiw, I have seen how compilers are made. Makes me amazed anything actually compiles and runs.
> If the code you write is uncovering bugs in compiler, you literally are too smart for your own good.
Nah, it can happen through sheer chance. I remember hitting a compiler bug when I was young (the only time I hit a compiler bug). I was certainly not particularly smart, just unlucky.
The interesting thing is that it had been drilled into me that "it's never a compiler bug" and so I banged my head against that wall for a couple of weeks before a more experienced programmer took interest and showed that it was actually a compiler bug.
Nah, it can happen through sheer chance. I remember hitting a compiler bug when I was young (the only time I hit a compiler bug). I was certainly not particularly smart, just unlucky.
The interesting thing is that it had been drilled into me that "it's never a compiler bug" and so I banged my head against that wall for a couple of weeks before a more experienced programmer took interest and showed that it was actually a compiler bug.
> If the code you write is uncovering bugs in compiler, you literally are too smart for your own good.
Definitely not in my case -- sometimes we just stumble over these things. As an inexperienced programmer (around 1989 or 1990?) I found a bug in the beta of microsoft's first C++ compiler, shipped on a dozen 5.25" floppy discs. All the other developers around me said "Don't be stupid. It's your code, it's not the compiler!" but I distilled it down to a minimal test case and proved it. The bug?
i=i+i;
That's it. It would give random results based on prior instructions unrelated to i.
I reported it and got a nice note back from the MSFT engineering team. Something like "Wow! I've no idea how we missed this. Good catch!"
Definitely not in my case -- sometimes we just stumble over these things. As an inexperienced programmer (around 1989 or 1990?) I found a bug in the beta of microsoft's first C++ compiler, shipped on a dozen 5.25" floppy discs. All the other developers around me said "Don't be stupid. It's your code, it's not the compiler!" but I distilled it down to a minimal test case and proved it. The bug?
i=i+i;
That's it. It would give random results based on prior instructions unrelated to i.
I reported it and got a nice note back from the MSFT engineering team. Something like "Wow! I've no idea how we missed this. Good catch!"
> If the code you write is uncovering bugs in compiler, you literally are too smart for your own good.
It's been a while so I don't remember all the details but I remember an ancient version of gcc refusing to accept perfectly valid zero initialisation of structs. Not terribly exciting but you don't have to be too clever to hit compiler bugs.
It's been a while so I don't remember all the details but I remember an ancient version of gcc refusing to accept perfectly valid zero initialisation of structs. Not terribly exciting but you don't have to be too clever to hit compiler bugs.
I have run into bugs that only happened in specific linux kernel versions when used with specific java runtimes
In my experience, one can accurately predict development time by correctly working around the planning fallacy.
The planning fallacy is something like "Humans are hopelessly optimistic. Even if you think you're planning for the worst case, unexpected things will happen, and your estimate is more like the best case scenario. This is true even if you try to compensate for the planning fallacy by being more pessimistic".
So you can't fix it by padding more weeks, trying to think of all the things that might happen, etc.
The only way to compensate for this is detailed in Kahneman's "Thinking Fast and Slow" (Must-read for many reasons). That is to start with prior evidence. Rather than making your estimate, start with "How long did it ACTUALLY take the last time someone did something like this?" and only then modifying the estimate for circumstances.
There is a huge gap between these two methods:
1. "How long does it take an average team to create a SAS product? Maybe a year?" Then modify for "but our team is extra smart, so maybe I'll shorten that to 10 months"
vs
2. "How long will this take? It's not a complicated product, and our team is smart: maybe 4 weeks for feature X, 6 for feature Y, 2 to deploy. I'll even throw in an extra 4 weeks of padding for unknowns, so 16 weeks".
The planning fallacy is something like "Humans are hopelessly optimistic. Even if you think you're planning for the worst case, unexpected things will happen, and your estimate is more like the best case scenario. This is true even if you try to compensate for the planning fallacy by being more pessimistic".
So you can't fix it by padding more weeks, trying to think of all the things that might happen, etc.
The only way to compensate for this is detailed in Kahneman's "Thinking Fast and Slow" (Must-read for many reasons). That is to start with prior evidence. Rather than making your estimate, start with "How long did it ACTUALLY take the last time someone did something like this?" and only then modifying the estimate for circumstances.
There is a huge gap between these two methods:
1. "How long does it take an average team to create a SAS product? Maybe a year?" Then modify for "but our team is extra smart, so maybe I'll shorten that to 10 months"
vs
2. "How long will this take? It's not a complicated product, and our team is smart: maybe 4 weeks for feature X, 6 for feature Y, 2 to deploy. I'll even throw in an extra 4 weeks of padding for unknowns, so 16 weeks".
As far as I can tell there are two major problems with this.
The first is that you can predict that there will be unforeseen problems, but not how many of them nor how long each will take to resolve. The result is high variance. Then if the average such project takes 16 months, and you predict each such project will take 16 months, then some take 7 months and some take 17 months and some take 25 months and people get mad at you for underestimating some of them. But if you predict at the upper end of the range then people will get mad at you for giving 25 month estimates for projects that only take 16 months on average.
The second is that people know what the estimate is and work expands to fill all available time. When you know you have 16 months, you set a pace that will have the project finished in 16 months. Then in month 14 you encounter some unexpected problem that takes six months to resolve. If you'd found it in month 2 you'd have been fine, because you'd have cut some less critical features and spent the time on that, but now that's in the past and your project is five months late.
The first is that you can predict that there will be unforeseen problems, but not how many of them nor how long each will take to resolve. The result is high variance. Then if the average such project takes 16 months, and you predict each such project will take 16 months, then some take 7 months and some take 17 months and some take 25 months and people get mad at you for underestimating some of them. But if you predict at the upper end of the range then people will get mad at you for giving 25 month estimates for projects that only take 16 months on average.
The second is that people know what the estimate is and work expands to fill all available time. When you know you have 16 months, you set a pace that will have the project finished in 16 months. Then in month 14 you encounter some unexpected problem that takes six months to resolve. If you'd found it in month 2 you'd have been fine, because you'd have cut some less critical features and spent the time on that, but now that's in the past and your project is five months late.
The third major problem:
Project Manager/Scheduler/Sales/Customer doing one/any/all of: not believing the input of the SME (you) when they ask for input developing the schedule, overpromising, expecting the world, or so many other problems.
Too many times in my own field (heavy construction) the scheduler has asked me how long it will take to do a given task, and it is actually reasonably quantifiable to say "It will take my crew X days to move Y volume of material at Z/shift" only to be told "Well our proposal only allocated X-5 days and budgeted a Crew of <your chosen size - 4> for that task. Figure it out."
Project Manager/Scheduler/Sales/Customer doing one/any/all of: not believing the input of the SME (you) when they ask for input developing the schedule, overpromising, expecting the world, or so many other problems.
Too many times in my own field (heavy construction) the scheduler has asked me how long it will take to do a given task, and it is actually reasonably quantifiable to say "It will take my crew X days to move Y volume of material at Z/shift" only to be told "Well our proposal only allocated X-5 days and budgeted a Crew of <your chosen size - 4> for that task. Figure it out."
> How long did it ACTUALLY take
Haha, I came to that same conclusion decades ago - I don't know if it would actually produce meaningful estimates or not, though, because every time I try to estimate that way, somebody more "important" than me comes along and throws out that estimate for one that fits the timeline he already had in mind.
Haha, I came to that same conclusion decades ago - I don't know if it would actually produce meaningful estimates or not, though, because every time I try to estimate that way, somebody more "important" than me comes along and throws out that estimate for one that fits the timeline he already had in mind.
This is the only thing that ever works for me, but a) you have to be doing repeated projects or high overlap, and b) it's hard to defend when a PM challenges. My response is usually "<shrug> this is how long this kind of thing has taken in the past". I do a lot of the same projects at different customers, so my gut feel is usually pretty close (and I have to give a ballpark quote, so there's some penalty to being over or under). But if it's something completely new, breaking it down in to detail tasks almost never results in a good estimate.
> That is to start with prior evidence. Rather than making your estimate, start with "How long did it ACTUALLY take the last time someone did something like this?" and only then modifying the estimate for circumstances.
This requires to have prior evidence within your team to start with as not all teams have the same velocity.
This requires to have prior evidence within your team to start with as not all teams have the same velocity.
Let me add more nuance here. We are always balancing a set of constraints: total cost, speed of delivery, certainty of cost, and certainty of delivery for a given date are but four.
Agile, for example, suggests that decreasing the cost of change at the expense of certainty of cost will increase the total chances of delivery within cost and date. Outsources are (implicitly) arguing that the decrease of total costs are worth the other friction in the organization.
This all goes back to return on investment. In the most simplistic case, if I have ten people working on this task now, and this software will allow me to reduce that to six, then that's four people I don't have to pay for that task. If those four people are making $100,000 a year, and the software lasts 10 years before having to be replaced, that's 4 million dollars. The software plus maintenance needs to cost less than 4 million dollars to make sense. We all know the simplistic case.
But if I can trade $100,000 dollars in guaranteed cost for a reduction of $500,000 in risk in making that goal, I can trade cost for certainty and that's going to be a good trade. Now, as developers we know that these numbers are often fudged, but everyone realizes this is inexact and just the best we all can do. We need to know how to think about these in terms of arguing against bad investments and steer toward good investments by knowing how the tradeoffs works.
Agile, for example, suggests that decreasing the cost of change at the expense of certainty of cost will increase the total chances of delivery within cost and date. Outsources are (implicitly) arguing that the decrease of total costs are worth the other friction in the organization.
This all goes back to return on investment. In the most simplistic case, if I have ten people working on this task now, and this software will allow me to reduce that to six, then that's four people I don't have to pay for that task. If those four people are making $100,000 a year, and the software lasts 10 years before having to be replaced, that's 4 million dollars. The software plus maintenance needs to cost less than 4 million dollars to make sense. We all know the simplistic case.
But if I can trade $100,000 dollars in guaranteed cost for a reduction of $500,000 in risk in making that goal, I can trade cost for certainty and that's going to be a good trade. Now, as developers we know that these numbers are often fudged, but everyone realizes this is inexact and just the best we all can do. We need to know how to think about these in terms of arguing against bad investments and steer toward good investments by knowing how the tradeoffs works.
> Agile, for example, suggests that decreasing the cost of change at the expense of certainty of cost will increase the total chances of delivery within cost and date
Yes, that's the promise. I've seen it tried as earnestly as possible over the past 20 or so years since the term became popular, but I've never seen it deliver on its promises either. I suspect that a lot of that is psychological - risk-averse management hears "on time and on budget" and consciously or unconsciously starts steering the development process back to a more predictable seeming waterfall style that negates the supposed benefits of "agile" development.
I assert, however, based on 30 years of observation, that no matter how carefully you follow an agile delivery framework, whatever that may be, actual delivery and cost will have no relationship whatsoever on your estimated cost and date.
But I also predict that you and I will continue to smile and nod when they insist that it must be finished by such and such date.
Yes, that's the promise. I've seen it tried as earnestly as possible over the past 20 or so years since the term became popular, but I've never seen it deliver on its promises either. I suspect that a lot of that is psychological - risk-averse management hears "on time and on budget" and consciously or unconsciously starts steering the development process back to a more predictable seeming waterfall style that negates the supposed benefits of "agile" development.
I assert, however, based on 30 years of observation, that no matter how carefully you follow an agile delivery framework, whatever that may be, actual delivery and cost will have no relationship whatsoever on your estimated cost and date.
But I also predict that you and I will continue to smile and nod when they insist that it must be finished by such and such date.
We are always balancing a set of constraints: total cost, speed of delivery, certainty of cost, and certainty of delivery for a given date are but four.
The real problem is that management think they can directly constrain those kinds of variables but in reality they are not inputs they're outputs. The real inputs are things you can actually control like what you're building and which of your team you assign to build which parts of it at which times. That might come in on time and under budget or it might not but shortening a timeline or decreasing a budget won't change how long it takes or what it costs.
Of course that sucks for managers who want to make informed decisions like whether to go ahead with a project that will require an uncertain amount of investment to complete but needs to generate more return than some threshold to be worth doing. Now the managers have to find a way to guess what the important outputs like cost and timescales will be as some sort of probability curve so they can make a rational decision based on the best but still incomplete information they have available. And even then a lot of managers insist on trying to be absolute about things like time and money when the actionable decisions they need to make are more often whether to start some project at all or which ongoing projects currently have the greatest need for any additional resources that are available.
The real problem is that management think they can directly constrain those kinds of variables but in reality they are not inputs they're outputs. The real inputs are things you can actually control like what you're building and which of your team you assign to build which parts of it at which times. That might come in on time and under budget or it might not but shortening a timeline or decreasing a budget won't change how long it takes or what it costs.
Of course that sucks for managers who want to make informed decisions like whether to go ahead with a project that will require an uncertain amount of investment to complete but needs to generate more return than some threshold to be worth doing. Now the managers have to find a way to guess what the important outputs like cost and timescales will be as some sort of probability curve so they can make a rational decision based on the best but still incomplete information they have available. And even then a lot of managers insist on trying to be absolute about things like time and money when the actionable decisions they need to make are more often whether to start some project at all or which ongoing projects currently have the greatest need for any additional resources that are available.
Software does not have to be wildly unpredictable if you do good estimation work up front. Being good at this is very important if you work for clients that have a specific budget or are doing fixed bids for software. The main thing is that you have to take estimation very seriously. You have to put experienced people on the estimation task, and you have to spend a lot of time on it to get it right, but doing it right can pay huge dividends. A few strategies that have worked very well for me:
1. When doing estimates, estimate hierarchically and break down every task until the leaf nodes are 4 hours or less. If the leaf nodes are a week of effort, that means that you don't yet understand fully what the task entails. If you don't know what the task entails, it is very difficult to get an accurate estimate.
2. Maintain a database of various projects and features, and use them for comparison purposes. If you know how long something actually took in the past, you can much more accurately how long it will take to do something similar again.
3. Include specific time for various types of "overhead": meetings, code review, writing tests, debugging, making presentations, etc.
4. Derisk ahead of time things that are truly unpredictable. Doing something in a brand new programing language or framework, training an ML model on a new type of data, working on datasets of unknown quality, etc. For those you need to estimate out a project to determine feasibility and generate an estimate.
The main thing is that this is clearly not impossible. Years ago I got really interested in the Apollo program as an example of bringing in perhaps the most complex and risk-laden project ever attempted, on time. One of the things that clearly stands out about that project was that there was huge derisking happening first. At one level you had the Mercury and Gemini programs that were figuring out human spaceflight generally, and docking with Gemini, and also things like the first stage engines of the Saturn V were already in development in the mid fifties.
1. When doing estimates, estimate hierarchically and break down every task until the leaf nodes are 4 hours or less. If the leaf nodes are a week of effort, that means that you don't yet understand fully what the task entails. If you don't know what the task entails, it is very difficult to get an accurate estimate.
2. Maintain a database of various projects and features, and use them for comparison purposes. If you know how long something actually took in the past, you can much more accurately how long it will take to do something similar again.
3. Include specific time for various types of "overhead": meetings, code review, writing tests, debugging, making presentations, etc.
4. Derisk ahead of time things that are truly unpredictable. Doing something in a brand new programing language or framework, training an ML model on a new type of data, working on datasets of unknown quality, etc. For those you need to estimate out a project to determine feasibility and generate an estimate.
The main thing is that this is clearly not impossible. Years ago I got really interested in the Apollo program as an example of bringing in perhaps the most complex and risk-laden project ever attempted, on time. One of the things that clearly stands out about that project was that there was huge derisking happening first. At one level you had the Mercury and Gemini programs that were figuring out human spaceflight generally, and docking with Gemini, and also things like the first stage engines of the Saturn V were already in development in the mid fifties.
> huge derisking happening first
How long did that huge derisking take? What was the budget for the time spent doing the derisking? I'm willing to bet the "huge derisking" ended up taking longer than the actual project (and was itself open ended and unpredictable).
How long did that huge derisking take? What was the budget for the time spent doing the derisking? I'm willing to bet the "huge derisking" ended up taking longer than the actual project (and was itself open ended and unpredictable).
splitting a project into an open ended "exploratory phase" of undefined length followed by a "main phase" of defined length doesn't seem like it would make any change to the total time taken, just shifting around the unpredictability
> shifting around the unpredictability
AFAICT it's actually worse - the exploratory phase always seems to get bogged down by theoretical discussions and imaginary scenarios that don't end up being very important (and putting off how long it takes to find the scenarios that actually do).
AFAICT it's actually worse - the exploratory phase always seems to get bogged down by theoretical discussions and imaginary scenarios that don't end up being very important (and putting off how long it takes to find the scenarios that actually do).
Right, and that’s Tesler’s law restated. You’re really just shifting complexity and not reducing it.
Where it makes sense is from a sociocultural perspective. Under Taylorist management philosophy, management is to do the thinking, and workers are to do the doing. Structuring a project where managers take on the exploratory phase fits well within the set of assumptions taught in project management programs.
Where it makes sense is from a sociocultural perspective. Under Taylorist management philosophy, management is to do the thinking, and workers are to do the doing. Structuring a project where managers take on the exploratory phase fits well within the set of assumptions taught in project management programs.
Everything you describe works well for estimating tasks that can be automated.
If you’re doing that kind of work, your employer will be happy to replace you with an AI system.
If you’re doing that kind of work, your employer will be happy to replace you with an AI system.
> replace you with an AI system
Or a couple of Excel macros.
Or a couple of Excel macros.
> Software does not have to be wildly unpredictable if you do good estimation work up front.
You also need someone with 1) strong technical and/or product skills, 2) good communication skills and decisiveness, and 3) explicit executive authority. That's the only way to avoid scope creep and bikeshedding.
When you started working on a project without knowing what you're building, or end up delivering a very different project than what was originally planned, it's hard to even say "that project was late." No it wasn't: you just never built whatever you had promised to build by the original deadline.
You also need someone with 1) strong technical and/or product skills, 2) good communication skills and decisiveness, and 3) explicit executive authority. That's the only way to avoid scope creep and bikeshedding.
When you started working on a project without knowing what you're building, or end up delivering a very different project than what was originally planned, it's hard to even say "that project was late." No it wasn't: you just never built whatever you had promised to build by the original deadline.
This is a great point. This works in my company because we have this, but without it I agree accurate estimation is impossible.
You can make some good guesses about levels of unpredictability based on previous experience though.
For example:
If the job involves dealing with anyone from Samsung or LG, it will take 2-3 times as long as you think.
If the job involves adding in a commercial third-party component which has a website about it, and that website doesn't clearly explain what the benefit of the thing actually is, the job will take 10-20 times as long as it should.
If the job involves a new feature Amazon or Google have just launched on one of their devices, and you are being CC'd on email chains involving representatives of Amazon or Google who are claiming implementation is very easy, the job will take a minimum of 6 months and may never be completed because the feature will be dropped or replaced with V2 before V1 ever actually works.
For example:
If the job involves dealing with anyone from Samsung or LG, it will take 2-3 times as long as you think.
If the job involves adding in a commercial third-party component which has a website about it, and that website doesn't clearly explain what the benefit of the thing actually is, the job will take 10-20 times as long as it should.
If the job involves a new feature Amazon or Google have just launched on one of their devices, and you are being CC'd on email chains involving representatives of Amazon or Google who are claiming implementation is very easy, the job will take a minimum of 6 months and may never be completed because the feature will be dropped or replaced with V2 before V1 ever actually works.
> the job will take 10-20 times as long as it should.
But your CTO will think it will take 10-20 times less time than it should, because that's what the vendor promised.
But your CTO will think it will take 10-20 times less time than it should, because that's what the vendor promised.
CTO's can't all be idiots, some of them will only be pretending to think that.
> but what they delivered always had "bugs" (either actual incorrect behavior, or edge cases that nobody thought of until they actually used the software)
This is why they are not only inevitable but unending.
Most real world problems are so large and messy that they do not have closed form solutions. In this context "bug" more often means a deficiency, a reflections of changing or competing and incompatible requirements, than abstract mistakes.
In this context, I've learned to appreciate the choice becomes "which bugs", because "fixing" such bugs is not necessarily a net positive, especially when the fix is not a natural fit for the design or architecture of the overarching solution. The mindless, incessant fixing of such bugs on a case-by-case basis can result in accumulation of so much complexity and LoC it becomes self perpetuating in bug fixing alone. When you are conscious of this, and have enough respect and autonomy, it's possible to decide whether it's worth this intangible cost, or whether to put it into the pile of "wont fix", this kind of decision requires cross domain collaboration. But that pile is not worthless, you can periodically revisit it and see opportunities to a fundamentally different approach that may tick more boxes (or more preferable boxes) without being ad-hock.
This is why they are not only inevitable but unending.
Most real world problems are so large and messy that they do not have closed form solutions. In this context "bug" more often means a deficiency, a reflections of changing or competing and incompatible requirements, than abstract mistakes.
In this context, I've learned to appreciate the choice becomes "which bugs", because "fixing" such bugs is not necessarily a net positive, especially when the fix is not a natural fit for the design or architecture of the overarching solution. The mindless, incessant fixing of such bugs on a case-by-case basis can result in accumulation of so much complexity and LoC it becomes self perpetuating in bug fixing alone. When you are conscious of this, and have enough respect and autonomy, it's possible to decide whether it's worth this intangible cost, or whether to put it into the pile of "wont fix", this kind of decision requires cross domain collaboration. But that pile is not worthless, you can periodically revisit it and see opportunities to a fundamentally different approach that may tick more boxes (or more preferable boxes) without being ad-hock.
We are all at the mercy of Hofstadter's Law.
https://en.wikipedia.org/wiki/Hofstadter%27s_law
> Hofstadter's Law: It always takes longer than you expect, even when you take into account Hofstadter's Law.
> Hofstadter's Law: It always takes longer than you expect, even when you take into account Hofstadter's Law.
> You know what I haven’t seen? not once in 15 years?
> A company going under.
What a wild assertion: The OP hasn’t personally seen a company fail, and therefore software quality doesn’t matter? Bugs and slow delivery are fine?
It’s trivially easy to find counterexamples of companies failing because their software products were inferior to newcomers who delivered good results, fast development, and more stable experience. Startups fail all the time because their software isn’t good enough or isn’t delivered before the runway expires. The author is deliberately choosing to ignore this hard reality.
I think the author may have been swept up in big, slow companies that have so much money that they can afford to have terrible software development practices and massive internal bloat. Stay in this environment long enough and even the worst software development practices start to feel “normal” because you look around and nothing bad has happened yet.
> A company going under.
What a wild assertion: The OP hasn’t personally seen a company fail, and therefore software quality doesn’t matter? Bugs and slow delivery are fine?
It’s trivially easy to find counterexamples of companies failing because their software products were inferior to newcomers who delivered good results, fast development, and more stable experience. Startups fail all the time because their software isn’t good enough or isn’t delivered before the runway expires. The author is deliberately choosing to ignore this hard reality.
I think the author may have been swept up in big, slow companies that have so much money that they can afford to have terrible software development practices and massive internal bloat. Stay in this environment long enough and even the worst software development practices start to feel “normal” because you look around and nothing bad has happened yet.
For what it's worth, Mozilla was nearly killed at least twice by code quality.
Once upon startup. The code inherited from Netscape was... let's say charitably difficult to maintain. Turning it into something that could actually be improved upon and tested took years without any release of the Mozilla Suite.
Once when Chrome emerged. Not because the code was particularly bad, but because its architecture didn't reflect more modern requirements in terms of responsiveness or security. See https://yoric.github.io/post/why-did-mozilla-remove-xul-addo... for a few more details.
Once upon startup. The code inherited from Netscape was... let's say charitably difficult to maintain. Turning it into something that could actually be improved upon and tested took years without any release of the Mozilla Suite.
Once when Chrome emerged. Not because the code was particularly bad, but because its architecture didn't reflect more modern requirements in terms of responsiveness or security. See https://yoric.github.io/post/why-did-mozilla-remove-xul-addo... for a few more details.
> For what it's worth, Mozilla was nearly killed at least twice by code quality.
> That’s not to say that I now think that “High quality good! Low quality better!”. > As many commenters on Cupać’s post observed, while high quality isn’t necessary for success, it improves the chances of it.
_Nearly_ killed. But Mozilla is still kicking around. Maybe not doing the kind of internet-shaping work they once did (here's hoping they get back there), but they continue to exist.
> That’s not to say that I now think that “High quality good! Low quality better!”. > As many commenters on Cupać’s post observed, while high quality isn’t necessary for success, it improves the chances of it.
_Nearly_ killed. But Mozilla is still kicking around. Maybe not doing the kind of internet-shaping work they once did (here's hoping they get back there), but they continue to exist.
I was in a successful company that nearly died in 2017, where our entire production system corrupted itself due to a sneaking scale bug we had ported the system into. The problem was, the system with data had been running migrated for 3 months with the bug in it, so it was no longer possible to revert to the earlier working design. We were down for a week where no clients could run, and we spent the next 12 months purely digging ourselves out of that hole, with all new development paused, and all hands on limping the ship along. I would say, that bug was very close to ending us. Luckily, we never disappointed in a similar way since.
What I’m hearing here is that even a software problem so bad it forced you to focus on nothing except trying to fix it, for a full year, wasn’t bad enough to make the company go under.
“My company didn’t go under” is pretty much the lowest bar you can shoot for.
This blog post is saying “Staying healthy doesn’t matter because neither I nor anyone I know died so far.”
This blog post is saying “Staying healthy doesn’t matter because neither I nor anyone I know died so far.”
This analogy is a bit tortured but it's honestly pretty good anecdata that something probably doesn't generally kill you if you've been around people doing it for 15 years and none of them have died yet. It can kill you, but that doesn't mean it generally does.
I'd love to come to the conclusion that it's wrong, but it's right. Most companies absolutely 100% can afford to ship bugs: the proof? They're doing it, they have been doing it, they will continue to do it. That doesn't mean that every company can, always, forever. A single really bad bug can tank a company. In the right conditions, simply being buggier to a superior competitor can tank a company. However, those are mostly just things developers fantasize about. The market is by and large not decided by how elegant and well-designed your software is, moreso in some verticals than others. In fact, this is basically true for almost anything that isn't explicitly developer oriented in the first place.
Just look at enterprise software. Jesus Christ.
I'd love to come to the conclusion that it's wrong, but it's right. Most companies absolutely 100% can afford to ship bugs: the proof? They're doing it, they have been doing it, they will continue to do it. That doesn't mean that every company can, always, forever. A single really bad bug can tank a company. In the right conditions, simply being buggier to a superior competitor can tank a company. However, those are mostly just things developers fantasize about. The market is by and large not decided by how elegant and well-designed your software is, moreso in some verticals than others. In fact, this is basically true for almost anything that isn't explicitly developer oriented in the first place.
Just look at enterprise software. Jesus Christ.
People smoke from 15-20 years old but still don't die in their 30s. It doesn't mean it's not bad for your health. This is the same thing, it's bad for the health of the company. High dev turnover, poor working conditions, and productivity issues can all lead to a death from a thousand cuts, and even if the company doesn't actually go under, you can't say that a company didn't suffer because of it.
I'm going to be honest, this analogy is just not that good. You can draw some parallels to cancer and bad workplace culture, but it's a very skin-deep/awkward comparison in my opinion.
Well, what is your goal?
Sure, healthy and happy chickens are nice, but if you are into industrial farming and your goal is to make money, you will make their life as miserable as necessary to extract profit. You don't care if they are barely alive as long as the health standards are met.
It's not about you being happy as an individual, it's about the company making money. If your lawyers and sales are good enough to build a captive market of miserable customers, your company can still make a ton of money and be very successful.
Sure, healthy and happy chickens are nice, but if you are into industrial farming and your goal is to make money, you will make their life as miserable as necessary to extract profit. You don't care if they are barely alive as long as the health standards are met.
It's not about you being happy as an individual, it's about the company making money. If your lawyers and sales are good enough to build a captive market of miserable customers, your company can still make a ton of money and be very successful.
From personal experience with early stage startups, it is quite a low bar. I've seen companies "acquired" for pennies on the dollar, after almost the entire staff is laid off. Somehow, this is considered a success even though you would've had an better outcome investing your time and money in almost anything else. Employees lost, investors lost. The founder often gets a cushy job they could've had anyway.
"and I've never met any dead people either"
I think that it is (a little bit) more subtle: the importance of quality OF A PRODUCT (projects delivery is another beast) is relative to:
- the customer: either B2B or B2C
- the market share: minimal (< 1%... including all startups) or dominant (> 30%)
- B2C is really dynamic and few bad versions/products can make the customers fly away (except when strong dominance - like Windows - or no equivalent product) and shutdown a company. Price can be a strong factor and cost of migration/switching is usually not considered
- B2B is more conservative: hard to enter the market (so small market shares will need a lot of time to take off... if there's no competitor) but once you're in, the cost of change for a company is usually high enough to tolerate more bad versions (and more if there's few competitors, and incompatibilities between products, and legal requirements to keep records, and a lot of "configuration", and requiring a strong training for a lot of people...). Companies as customer dont see the switch of software as a technical problem (replacing on editor by another one) but as a management problem (training, cost to switch, data availability, availability of people already trained, cost of multi-year/multi-instance licences...)
- the customer: either B2B or B2C
- the market share: minimal (< 1%... including all startups) or dominant (> 30%)
- B2C is really dynamic and few bad versions/products can make the customers fly away (except when strong dominance - like Windows - or no equivalent product) and shutdown a company. Price can be a strong factor and cost of migration/switching is usually not considered
- B2B is more conservative: hard to enter the market (so small market shares will need a lot of time to take off... if there's no competitor) but once you're in, the cost of change for a company is usually high enough to tolerate more bad versions (and more if there's few competitors, and incompatibilities between products, and legal requirements to keep records, and a lot of "configuration", and requiring a strong training for a lot of people...). Companies as customer dont see the switch of software as a technical problem (replacing on editor by another one) but as a management problem (training, cost to switch, data availability, availability of people already trained, cost of multi-year/multi-instance licences...)
> It’s trivially easy to find counterexamples of companies failing because their software products were inferior to newcomers who delivered good results, fast development, and more stable experience.
Is it? What I've seen is the opposite.
Businesses can be terrible top to bottom, slow, inefficient, and painful for customers, and still keep going for years and years. It's more about income/funding than product.
> I think the author may have been swept up in big, slow companies that have so much money that they can afford to . . .
That's what I'm talking about. They are legion! They could be companies that serve a niche that no one else does or with prohibitive switching costs (training is expensive). They could also be companies that somehow got enough market share that "no one gets fired for buying IBM."
Also, you know what those "big, slow companies" have in common? They are successful businesses. Unlike most startups.
Is it? What I've seen is the opposite.
Businesses can be terrible top to bottom, slow, inefficient, and painful for customers, and still keep going for years and years. It's more about income/funding than product.
> I think the author may have been swept up in big, slow companies that have so much money that they can afford to . . .
That's what I'm talking about. They are legion! They could be companies that serve a niche that no one else does or with prohibitive switching costs (training is expensive). They could also be companies that somehow got enough market share that "no one gets fired for buying IBM."
Also, you know what those "big, slow companies" have in common? They are successful businesses. Unlike most startups.
> It’s trivially easy to find counterexamples of companies failing because their software products were inferior to newcomers who delivered good results, fast development, and more stable experience. Startups fail all the time because their software isn’t good enough or isn’t delivered before the runway expires. The author is deliberately choosing to ignore this hard reality.
While I personally haven't seen a company going under due to bad code, one can also definitely make the argument that software that is buggy or doesn't scale will lead to lost profits, which could also eventually become a problem, or present risks to business continuity.
I still recall working on critical performance issues for a near-real-time auction system way past the end of my working day, because there was an auction scheduled the next day and a fix was needed. I also recall visiting a healthcare business which could not provide services, because a system of theirs kept crashing and there was a long queue of people just sitting around and being miserable.
Whether private or public sector, poor code has a bad impact on many different things.
However, once can also definitely make the distinction between keeping the lights on (KTLO) and everything else. If an auction system cannot do auctions for some needed amount of users, that's a KTLO issue. If an e-commerce system calculates prices/taxes/discounts wrong and this leads to direct monetary losses, that's possibly a KTLO issue. If users occasionally get errors or weird widget sizing in some CRUD app or blog site, nobody really cares that much, at least as far as existential threats go.
Outside of that, bugs and slow delivery can be an unfortunate reality, yet one that can mostly be coped with.
While I personally haven't seen a company going under due to bad code, one can also definitely make the argument that software that is buggy or doesn't scale will lead to lost profits, which could also eventually become a problem, or present risks to business continuity.
I still recall working on critical performance issues for a near-real-time auction system way past the end of my working day, because there was an auction scheduled the next day and a fix was needed. I also recall visiting a healthcare business which could not provide services, because a system of theirs kept crashing and there was a long queue of people just sitting around and being miserable.
Whether private or public sector, poor code has a bad impact on many different things.
However, once can also definitely make the distinction between keeping the lights on (KTLO) and everything else. If an auction system cannot do auctions for some needed amount of users, that's a KTLO issue. If an e-commerce system calculates prices/taxes/discounts wrong and this leads to direct monetary losses, that's possibly a KTLO issue. If users occasionally get errors or weird widget sizing in some CRUD app or blog site, nobody really cares that much, at least as far as existential threats go.
Outside of that, bugs and slow delivery can be an unfortunate reality, yet one that can mostly be coped with.
Agree.
For example, pg said that ViaWeb was successful because they had put care into their code, which allowed them to iterate quickly and integrate new features that customers requested. Whereas competitors were held back by their cumbersome code and slow cadence of releasing features.
For example, pg said that ViaWeb was successful because they had put care into their code, which allowed them to iterate quickly and integrate new features that customers requested. Whereas competitors were held back by their cumbersome code and slow cadence of releasing features.
pg would say that, though, because it was his company and it worked out for him.
But maybe he was actually wrong about quality being a differentiator, and his competitors could've shipped features with shitty code but had other problems.
But maybe he was actually wrong about quality being a differentiator, and his competitors could've shipped features with shitty code but had other problems.
Friendster is memorable example. It did not scale well, and fed up users flocked to MySpace and Facebook, which came later.
More generally it depends on how competitive the space the product operates in, whether quality is something the buyer values and is able to evaluate! Enterprises for example infamously don't appreciate quality as much consumers because the economic buyer does not use the product.
More generally it depends on how competitive the space the product operates in, whether quality is something the buyer values and is able to evaluate! Enterprises for example infamously don't appreciate quality as much consumers because the economic buyer does not use the product.
It should have been "going under due to bugs / low quality and slow delivery".
So, here’s a simple game:
- take a blank piece of paper (this is “the software”)
- pick two random points on the paper roughly 3 inches apart (“a requirement”)
- draw a line between the two points; the line cannot cross any other line (“the implementation”).
Repeat the exercise multiple times.
You’ll quickly learn that unless you have a “system”, drawing a three inch line goes from taking about 2 seconds, to taking 10, 15 or 30 seconds, despite “the requirement complexity” being exactly the same every time.
Now try playing with people taking turns. :)
I like playing this game with people who think that writing bad code is fine, or that they can work by themselves and not worry about what other people do as long as “their own” code is good.
You can still solve pretty much any problem with enough time and effort; but if you don’t have a “system” for good organisation, eventually, you’ll be struggling to solve basic problems because of layered unmanageable requirements, no matter how smart or capable you are.
…it’s not about shipping bugs, it’s about fundamentally reducing the speed of delivery over time, incrementally, in an unbounded fashion that eventually makes systems impossible to make changes to without compromising on requirements.
(Obviously the game is contrived, but it works very well in explaining why past requirements some times have to go away to business people in my experience).
- take a blank piece of paper (this is “the software”)
- pick two random points on the paper roughly 3 inches apart (“a requirement”)
- draw a line between the two points; the line cannot cross any other line (“the implementation”).
Repeat the exercise multiple times.
You’ll quickly learn that unless you have a “system”, drawing a three inch line goes from taking about 2 seconds, to taking 10, 15 or 30 seconds, despite “the requirement complexity” being exactly the same every time.
Now try playing with people taking turns. :)
I like playing this game with people who think that writing bad code is fine, or that they can work by themselves and not worry about what other people do as long as “their own” code is good.
You can still solve pretty much any problem with enough time and effort; but if you don’t have a “system” for good organisation, eventually, you’ll be struggling to solve basic problems because of layered unmanageable requirements, no matter how smart or capable you are.
…it’s not about shipping bugs, it’s about fundamentally reducing the speed of delivery over time, incrementally, in an unbounded fashion that eventually makes systems impossible to make changes to without compromising on requirements.
(Obviously the game is contrived, but it works very well in explaining why past requirements some times have to go away to business people in my experience).
What "system" would you use for random "requirements" (dots on a piece of paper)? The only reasonable thing to do is to push back on "requirements" that don't fit into the existing requirements/software/implementation.
The thing about this game is that there is no requirement that is technically impossible.
That's provable; with no closed loops, any pair of points on the paper can be joined by a long enough line. That's why it's a good analogy for code.
...but if you, eg. a system for this; Use straight lines only and space the lines equidistant from other lines.
I mean, it's a game. Literally play the game a few times (grab a friend and do it together) and you'll very quickly figure out things that do work (leaving space) and dont work (draw crazy squiggly lines and lines that are too close together).
This is just a contrived example; if you want to make it more realistic allow 'you can erase a line and redraw it as long as the requirement is maintained', and then you can structure lines into groups, etc.
That's provable; with no closed loops, any pair of points on the paper can be joined by a long enough line. That's why it's a good analogy for code.
...but if you, eg. a system for this; Use straight lines only and space the lines equidistant from other lines.
I mean, it's a game. Literally play the game a few times (grab a friend and do it together) and you'll very quickly figure out things that do work (leaving space) and dont work (draw crazy squiggly lines and lines that are too close together).
This is just a contrived example; if you want to make it more realistic allow 'you can erase a line and redraw it as long as the requirement is maintained', and then you can structure lines into groups, etc.
Ah, so you are pushing back on the requirements. "I'm not going to implement the requirement (two random dots added) as it would break my requirement to only have straight lines in my implementation."
There's no part of anything I posted that required straight lines.
Whether you use structured code (straight lines and spacing) or squiggles (spaghetti code) is up to you as the developer implementing the requirement.
Whether you use structured code (straight lines and spacing) or squiggles (spaghetti code) is up to you as the developer implementing the requirement.
In practice, the implementations will end up crossing a few times anyway (bugs) and noone cares much because software bugs are a part of life.
Speed only goes to zero if you insist on bug free software.
Speed only goes to zero if you insist on bug free software.
This blog post is a response to an advertisement for coaching services. The article it responds to was so content-free and sales-focused that it actually got flagged here on HN.
https://news.ycombinator.com/item?id=36615325
I don’t think anyone needs to waste their time arguing with a strawman.
https://news.ycombinator.com/item?id=36615325
I don’t think anyone needs to waste their time arguing with a strawman.
If I scribble a random time stamp on a piece of paper and frame it and call it a clock, it's still right at least once a day (24 hour notation or bust :) ) The original article was just a garbage native advertisement with the wrong conclusion, I agree, but the ideas it stated weren't really wrong; Microsoft is a pretty good example of a company that constantly leaves its products in very buggy states for long periods of time, and it expects you to use the products in that state. (read: pay Microsoft) They're still dominating. While I admire the ReFS team for what they pulled off, the first 2 years it was virtually unusable if you tried to use it like they told you to, either due to memory leak bugs or outright corrupting your data silently.
It still got used and was one of the common deployments for MS SQL and Exchange during that time.
The original article doesn't sell a solution, it sells some spin from the author, but the original author wasn't wrong that there is a ridiculously high tolerance for buggy, low quality software, and the reason isn't super clear. These aren't cheap services either, they're billion dollar companies that can barely keep the services running sometimes or are incredibly slow to react to missing features or features not working well/correctly.
I think this situation merits discussion.
It still got used and was one of the common deployments for MS SQL and Exchange during that time.
The original article doesn't sell a solution, it sells some spin from the author, but the original author wasn't wrong that there is a ridiculously high tolerance for buggy, low quality software, and the reason isn't super clear. These aren't cheap services either, they're billion dollar companies that can barely keep the services running sometimes or are incredibly slow to react to missing features or features not working well/correctly.
I think this situation merits discussion.
> the original author wasn't wrong that there is a ridiculously high tolerance for buggy, low quality software, and the reason isn't super clear
I disagree. I’d say the reason is clear: the market decides; and the decision is that current amount of bugs is ok. Most software is fine for the end user.
We (devs) notice the bugs because we’re familiar with the process; but in the end we’re just people who know how the sausage is made.
I disagree. I’d say the reason is clear: the market decides; and the decision is that current amount of bugs is ok. Most software is fine for the end user.
We (devs) notice the bugs because we’re familiar with the process; but in the end we’re just people who know how the sausage is made.
Well, I get where you're coming from but I don't think that the _market_ tells this. Not really. Maybe it's okay for businesses and the market, but it affects individual users. The market/business maybe pays the bills, but the users pay the market.
When I wrote the reason isn't super clear, I understood that as developers, you're just the sausage makers so to speak. (but let's not think of this in a bad way! Sausages are tasty and a stable of many cuisines!) The loss of agency (or lack of it from the beginning) for developers is not unknown and it's a point I'm extremely sympathetic to. Where I say it's unclear is that:
- Developers don't like bugs
- Users don't like bugs
- When it loses them money, businesses don't like bugs
Every participant in the software chain basically doesn't like bugs and wants to make it better. A bug that does a particularly bad thing at a bad time can be fatal to any business; cosmetic bugs that require constant workarounds can be costly to the users and businesses using the software, and those issues that "only take a second to workaround" add up.
Basically, as I see it all bugs are costing us a ton and chipping away at our time and expenses, but universally we still seem to be pretty okay with even fairly large blocking bugs sitting in production systems for months or longer.
That is the part that is unclear for me; it's well documented how impactful the bugs are, we can measure the costs for everyone involved, lost renewals or contracts because bugs couldn't be fixed fast enough or were deprioritized for something else; but we're still pretty okay with having these sit around.
The only simple answer I can really come up with is that the market is okay with it only because short term profit can still be derived or there is capture, or both and possibly more.
It's just a really bad situation for everyone except a select few who still profit despite everyone hating the situation.
When I wrote the reason isn't super clear, I understood that as developers, you're just the sausage makers so to speak. (but let's not think of this in a bad way! Sausages are tasty and a stable of many cuisines!) The loss of agency (or lack of it from the beginning) for developers is not unknown and it's a point I'm extremely sympathetic to. Where I say it's unclear is that:
- Developers don't like bugs
- Users don't like bugs
- When it loses them money, businesses don't like bugs
Every participant in the software chain basically doesn't like bugs and wants to make it better. A bug that does a particularly bad thing at a bad time can be fatal to any business; cosmetic bugs that require constant workarounds can be costly to the users and businesses using the software, and those issues that "only take a second to workaround" add up.
Basically, as I see it all bugs are costing us a ton and chipping away at our time and expenses, but universally we still seem to be pretty okay with even fairly large blocking bugs sitting in production systems for months or longer.
That is the part that is unclear for me; it's well documented how impactful the bugs are, we can measure the costs for everyone involved, lost renewals or contracts because bugs couldn't be fixed fast enough or were deprioritized for something else; but we're still pretty okay with having these sit around.
The only simple answer I can really come up with is that the market is okay with it only because short term profit can still be derived or there is capture, or both and possibly more.
It's just a really bad situation for everyone except a select few who still profit despite everyone hating the situation.
I’d say the reason is clear: the market decides; and the decision is that current amount of bugs is ok.
Although when it comes to software that market is quite heavily skewed by the VC funding effect. If "the market" is getting something free or cheap as a loss leader because it's really the VC money that is paying the bills for a few years then end users are going to be much more tolerant of junk for a while.
In the VC model it's boom or bust so quality and coding for the long term might not be business priorities. In the short term unless you're so bad that it's outweighing the sales and marketing push to drive that rapid growth you can get away with a lot. In the long term you don't care so much about maintaining the early code because by the time a big exit is a realistic outcome you have the resources to throw things out and rebuild from the ground up if you need to.
Of course none of that means you're actually providing a good experience to your users during those early years or that they'd be willing to pay enough for what you're offering or stay with you for long enough to sustain and grow your business independently. You're still shipping junk and your users still don't like it. You just don't care because your bills are really being paid by someone whose interests barely align with your users in those early years and the bill payer also doesn't care because they're only interested in the big exits and expecting to write off all their other investments anyway.
If you're a software developer who does want to build good products and have happy users benefitting from them then this kind of market is toxic. If you're a user who just wants good software to do its job so you can get on with yours then this kind of market is toxic. Eventually maybe the potentially paying customers will get wise to the startup game and start avoiding it in favour of alternatives made by smaller companies that bootstrap and aim to build steady and sustainable products and businesses. But there's been so much money splashing around in VC world for so long now that a lot of people just can't think outside the box.
Although when it comes to software that market is quite heavily skewed by the VC funding effect. If "the market" is getting something free or cheap as a loss leader because it's really the VC money that is paying the bills for a few years then end users are going to be much more tolerant of junk for a while.
In the VC model it's boom or bust so quality and coding for the long term might not be business priorities. In the short term unless you're so bad that it's outweighing the sales and marketing push to drive that rapid growth you can get away with a lot. In the long term you don't care so much about maintaining the early code because by the time a big exit is a realistic outcome you have the resources to throw things out and rebuild from the ground up if you need to.
Of course none of that means you're actually providing a good experience to your users during those early years or that they'd be willing to pay enough for what you're offering or stay with you for long enough to sustain and grow your business independently. You're still shipping junk and your users still don't like it. You just don't care because your bills are really being paid by someone whose interests barely align with your users in those early years and the bill payer also doesn't care because they're only interested in the big exits and expecting to write off all their other investments anyway.
If you're a software developer who does want to build good products and have happy users benefitting from them then this kind of market is toxic. If you're a user who just wants good software to do its job so you can get on with yours then this kind of market is toxic. Eventually maybe the potentially paying customers will get wise to the startup game and start avoiding it in favour of alternatives made by smaller companies that bootstrap and aim to build steady and sustainable products and businesses. But there's been so much money splashing around in VC world for so long now that a lot of people just can't think outside the box.
The problem is that anyone with a semblance of professional pride will find it mentally quite difficult to knowingly ship bad software.
Like, sure, even if we make best efforts to catch both obvious and less obvious bugs, all software we ship will still be full of bugs. But knowingly shipping software full of obvious bugs... it feels unprofessional. And makes you really feel the weight of all the inevitable bug reports that come in.
The alternative often becomes to warn about it, have the organization respond (whether explicitly or otherwise) "we don't care", and then, to preserve your sanity, say to yourself "well, then all bets are off, and I'm not responsible" which also isn't healthy, because it leaves you jaded, disengaged and robs you of your sense of professionalism.
I don't have a solution here lol other than find an org that cares about quality to an acceptable degree.
Like, sure, even if we make best efforts to catch both obvious and less obvious bugs, all software we ship will still be full of bugs. But knowingly shipping software full of obvious bugs... it feels unprofessional. And makes you really feel the weight of all the inevitable bug reports that come in.
The alternative often becomes to warn about it, have the organization respond (whether explicitly or otherwise) "we don't care", and then, to preserve your sanity, say to yourself "well, then all bets are off, and I'm not responsible" which also isn't healthy, because it leaves you jaded, disengaged and robs you of your sense of professionalism.
I don't have a solution here lol other than find an org that cares about quality to an acceptable degree.
Perhaps one solution is to divorce your own self worth from the quality of the software you happen to write and to realize that you are an employee first and a software engineer second - make decisions that increase the amount of money the company makes and look for self worth in that itself or in hobbies/family/friends/something else.
That’s was the main reason that I do my pet projects: to detach myself from some corporate persona that is a completely different from my professional self.
I still have a lot of professional pride, but it comes more from my personal projects and sharpening my craft than to some corporate version that could be dragged down by structural issues.
I still have a lot of professional pride, but it comes more from my personal projects and sharpening my craft than to some corporate version that could be dragged down by structural issues.
You are not professional if you are not aligning with the values that are being optimized for the context. There should be an explicit decision tree setup for your work context that informs you what those values are. If not have some conversations and align.
yeah. and not just "not professional". It's basically giving the finger to a whole bunch of (generally) lower-paid support staff answering phone calls from irate customers.
Not to mention it's giving the finger to the customers.
> Software doesn’t make or break an organization.
This is true... but it's also true of ≈everything else in the organization. An organization that could be broken by any single thing would not survive for long! This includes the things people tediously insist to be "more important" than software quality. It certainly includes the usual suspects that push people to cut software corners: companies don't break by missing OKRs or slipping schedules or not "focusing" or any other management pablum any more than they break due to software quality.
But, of course, all of those can contribute to a company failing—including bugs, slow delivery and poor quality code. The dynamics and the extent to which different factors matter is inherently context-specific.
Companies can fail despite doing the right thing and succeed despite doing the wrong thing. This doesn't make the wrong thing effective or reasonable, it just means we're operating in the domain of complex, stochastic systems, so our simple mental models of cause and effect become misleading.
The fact that low-quality software doesn't directly cause a company to fail does not imply that investing in software quality is not right. I am firmly convinced that maintaining high quality software will have better outcomes in expectation than cutting corners pretty much universally, if only because it simply doesn't cost much—it's a matter of setting the right culture more than anything else.
There are interesting observations to be made here. Variations on "software quality doesn't matter because it won't destroy the company" are not it.
This is true... but it's also true of ≈everything else in the organization. An organization that could be broken by any single thing would not survive for long! This includes the things people tediously insist to be "more important" than software quality. It certainly includes the usual suspects that push people to cut software corners: companies don't break by missing OKRs or slipping schedules or not "focusing" or any other management pablum any more than they break due to software quality.
But, of course, all of those can contribute to a company failing—including bugs, slow delivery and poor quality code. The dynamics and the extent to which different factors matter is inherently context-specific.
Companies can fail despite doing the right thing and succeed despite doing the wrong thing. This doesn't make the wrong thing effective or reasonable, it just means we're operating in the domain of complex, stochastic systems, so our simple mental models of cause and effect become misleading.
The fact that low-quality software doesn't directly cause a company to fail does not imply that investing in software quality is not right. I am firmly convinced that maintaining high quality software will have better outcomes in expectation than cutting corners pretty much universally, if only because it simply doesn't cost much—it's a matter of setting the right culture more than anything else.
There are interesting observations to be made here. Variations on "software quality doesn't matter because it won't destroy the company" are not it.
> This is true... but it's also true of ≈everything else in the organization.
Except for sales?
Except for sales?
This is called software philistinism.
There are many blog posts like this one that deny (often deride) the value of software craftmanship, including the skillset of planning.
They almost always have this same conclusiveness to them as well ("she was right"), with no tangible backing other than personal anecdata ("everyone sucked who I worked with, so everyone must suck").
If you zoom out, the reality is that people in the software industry are spoiled AF. A majority of practitioners get paid a ridiculous amount and get away with murder.
It is true that current market dynamics give people in this industry pretty much zero incentive to get good at the craft. So people don't and then to rhetorically justify their philistinism, they write blog posts like this.
It is also true that many businesses get away with writing absolutely shitty software and stay in business because of these dynamics. But that doesn't make it "ok" and doesn't say anything about the enormous amount lost to such apathy.
There are many blog posts like this one that deny (often deride) the value of software craftmanship, including the skillset of planning.
They almost always have this same conclusiveness to them as well ("she was right"), with no tangible backing other than personal anecdata ("everyone sucked who I worked with, so everyone must suck").
If you zoom out, the reality is that people in the software industry are spoiled AF. A majority of practitioners get paid a ridiculous amount and get away with murder.
It is true that current market dynamics give people in this industry pretty much zero incentive to get good at the craft. So people don't and then to rhetorically justify their philistinism, they write blog posts like this.
It is also true that many businesses get away with writing absolutely shitty software and stay in business because of these dynamics. But that doesn't make it "ok" and doesn't say anything about the enormous amount lost to such apathy.
I don't think the blog post derides the value. Rather, it points out that, while fixing bugs is valuable to the dev (professional self-respect shipping quality products) and the user (obvious), the way software market works in practice is that it's not particularly valuable to the company - at least, not valuable enough to justify the expense. Speaking as someone with 20 years of software engineering experience, I think this is an accurate assessment of things. But at the same time you're absolutely correct that it doesn't make it okay, and it is actually detrimental to both users and developers for the aforementioned reasons.
But, well, this is capitalism for you - whatever is the most profitable thing to do is what gets done regardless of how not okay it really is. I think the question we should be asking is: if users hate using crappy buggy software, and devs hate writing it, but it still gets written, aren't our basic economic incentives clearly out of tune?
But, well, this is capitalism for you - whatever is the most profitable thing to do is what gets done regardless of how not okay it really is. I think the question we should be asking is: if users hate using crappy buggy software, and devs hate writing it, but it still gets written, aren't our basic economic incentives clearly out of tune?
Well said — and yes.
It’s definitely market dynamics, but I think something could change those market dynamics.
Maybe a craftsmen’s guild that unionizes skilled software practitioners.
It’s definitely market dynamics, but I think something could change those market dynamics.
Maybe a craftsmen’s guild that unionizes skilled software practitioners.
The core of this argument is really about the emotional health of the workers.
An emotionally healthy person tends to want to maintain their environment. It doesn't do anything objective to have nice furniture and pretty paintings, but you like it in your house. You want to feel at peace, so you adjust little things...the kitchen would be nicer with the plates in a pile over here instead of just pulling them out of the dishwasher.
At work, you're the householder of a million technical details, and to feel good about it you want to maintain your environment! If that window has an ugly redraw you want to recode it, if the ribbon has big ugly icons you want to collapse it into a strip menu, if the email client freezes whenever WiFi gets spotty you get annoyed because the code is still from 1995 under the surface. The software you're making and using, in service to your own craft, has an effect on you. If it makes you feel ugly, you want to fix it. This blog post -- which probably wasn't written on company time -- is here because someone needed to reorganize their thinking to feel emotional health. I think we should insist on doing a better job than just sacrificing our work to feed the money machine, even if our self-described jobs are just to turn funding into paychecks.
An emotionally healthy person tends to want to maintain their environment. It doesn't do anything objective to have nice furniture and pretty paintings, but you like it in your house. You want to feel at peace, so you adjust little things...the kitchen would be nicer with the plates in a pile over here instead of just pulling them out of the dishwasher.
At work, you're the householder of a million technical details, and to feel good about it you want to maintain your environment! If that window has an ugly redraw you want to recode it, if the ribbon has big ugly icons you want to collapse it into a strip menu, if the email client freezes whenever WiFi gets spotty you get annoyed because the code is still from 1995 under the surface. The software you're making and using, in service to your own craft, has an effect on you. If it makes you feel ugly, you want to fix it. This blog post -- which probably wasn't written on company time -- is here because someone needed to reorganize their thinking to feel emotional health. I think we should insist on doing a better job than just sacrificing our work to feed the money machine, even if our self-described jobs are just to turn funding into paychecks.
Precisely. I get not going overboard and being a perfectionist. One has to ship. But also, you should have SOME pride in your work. You should want to not make total shit.
Both this article and the one it references only seem to provide more evidence that the descent into mediocrity of the software industry is certainly happening. That said, there's a difference between actual quality, and "quality" as popularised by metrics-driven dogma.
Example for the last point: people seem to have forgotten how much of a "quality" shitshow was Twitter. The thing was written in Ruby on Rails, downtime was normal and expected by everyone. Yet the platform thrived and the username grew. Or Facebook, the page didn't fully load like 30% of the time, meaning that at least one panel was broken, some picture was missing, etc.
Always worth remembering the adage that you can't inspect quality into a product.
Which, interestingly, wasn't something a software engineer came up with.
Which, interestingly, wasn't something a software engineer came up with.
It really saddens me to know that the hacker culture I once so much loved to be part of is rapidly disappearing from the industry.
Is it? Or is a massive industry forming around a still small group of hackers?
Maybe the two are groups are not the same. Someone getting into software to make money is different to the kid who couldn't put the computer down. There are people that want to hack on software and others that want to solve business problems with computers for profit.
Maybe the two are groups are not the same. Someone getting into software to make money is different to the kid who couldn't put the computer down. There are people that want to hack on software and others that want to solve business problems with computers for profit.
Maybe, just maybe, the best people to build complex software systems that serve millions of people aren't the ones who can do the most leetcode problems.
A blog with ads every few paragraphs isn’t where we’ll find writers with standards.
This is how I feel about most "things I hate". Despite my strong feelings towards them, there's a reason why they are the way that they are. Almost always it comes down to my interests & priorities not aligning with reality/everyone else.
In this case I would argue that the interests of most people don't align with the "reality", but it is the way it is because it aligns with the interests of those people who have the most power to affect it - in this case, top executives and major shareholders.
To some extent hating stuff that frankly has no reason to exist is by definition temporal, it only lasts until the thing rightfully stops existing.
To really grind someone's gears they'd need to hate something that cannot simply go away.
To really grind someone's gears they'd need to hate something that cannot simply go away.
And even though the points raised are mostly valid I think there is a lot of nuance to this.
If you write control software for medical devices this is simply not true. Same goes if you handle incredibly sensitive data or critical infrastructure or assistance systems for air travel and you can probably come up with a a bunch more cases where bugs are not ok.
And also I think this sentence
> Where high quality is nice to have, but is not the be-all-and-end-all.
while being mostly true in all other cases I don't think it does it justice what mediocrity actually means for everyone involved. I have seen very varying levels of quality requirements/enforcement, testing, delivery speed and eventual bugs at different costs at different companies. And I in general found that higher quality software and less bugs were associated with much happier and more productive developers. And I would make the argument that less happy developers => more turnover => more second order costs. So while it might not make or break the company it certainly has a negative impact on profits downstream. And for customers too. Vendor lock in is a great thing for a company to profit from but if you really make a shitty product it opens up a lot of venues for competitors to eventually canibalize your market or from users to jump ship the next time they can. Take MS Teams for example. Lots of lock in but trust me the second better competitors are on the table I vigorously fight to switch. It's a slow burn but a burn nonetheless.
> Where high quality is nice to have, but is not the be-all-and-end-all.
while being mostly true in all other cases I don't think it does it justice what mediocrity actually means for everyone involved. I have seen very varying levels of quality requirements/enforcement, testing, delivery speed and eventual bugs at different costs at different companies. And I in general found that higher quality software and less bugs were associated with much happier and more productive developers. And I would make the argument that less happy developers => more turnover => more second order costs. So while it might not make or break the company it certainly has a negative impact on profits downstream. And for customers too. Vendor lock in is a great thing for a company to profit from but if you really make a shitty product it opens up a lot of venues for competitors to eventually canibalize your market or from users to jump ship the next time they can. Take MS Teams for example. Lots of lock in but trust me the second better competitors are on the table I vigorously fight to switch. It's a slow burn but a burn nonetheless.
The classic case study.
https://en.wikipedia.org/wiki/Therac-25
https://en.wikipedia.org/wiki/Therac-25
The examples you give have regulations for software. Even if companies want to "move fast and break things," with medical devices there are required tests and the testing and results are audited by the feds (here in the US), so you're forced to move slowly and not be broken (in ways that are regulated).
That does not mean "high quality." You can still have an architecture that's tightly-coupled spaghetti code that resists all human efforts at debugging. It just means "we threw enough paperwork at the auditors that they're not asking for more."
That does not mean "high quality." You can still have an architecture that's tightly-coupled spaghetti code that resists all human efforts at debugging. It just means "we threw enough paperwork at the auditors that they're not asking for more."
You'd be surprised how many businesses are run by completely incompetent people and still manage to survive.
A relative of mine who was an accountant for a small manufacturing plant and the systematic problems from top to bottom were shocking. They worked there for well over a decade and also managed a lot of other business aspects. Nothing got shipped, billed, ordered, or paid without them knowing. The company always had just enough income to keep going. Somehow, it was sustainable without any hidden external sources of money.
A relative of mine who was an accountant for a small manufacturing plant and the systematic problems from top to bottom were shocking. They worked there for well over a decade and also managed a lot of other business aspects. Nothing got shipped, billed, ordered, or paid without them knowing. The company always had just enough income to keep going. Somehow, it was sustainable without any hidden external sources of money.
Well great, I hate it!
Yesterday somebody shared this with me and it made me reflect about what things could we learn from other engineering disciplines. It might be just me but I have this gnawing impression that a lot of the pains we have in software are self imposed.
https://web.cs.wpi.edu/~gogo/humor/hum_toast.html
Yesterday somebody shared this with me and it made me reflect about what things could we learn from other engineering disciplines. It might be just me but I have this gnawing impression that a lot of the pains we have in software are self imposed.
https://web.cs.wpi.edu/~gogo/humor/hum_toast.html
> Software doesn’t make or break an organization.
As someone who works for an organization that is on the receiving end of bad software, I can assure you this is not the case. There is lost opportunity, due to decreased productivity and outright lost clients. There is a souring of the work environment, an environment where people are constantly putting out fires and there is constant churn in staff because of that.
Perhaps the organization I work for is unique in that it is public sector and cannot go out of business (which appears to be the author's definition of broken). Perhaps the author has never dealt with software where edge cases involving lost and phantom data is a daily norm. Clients, real people with real lives, are literally lost by the system.
Things are so bad that losing several years of business records (or, more likely, paying to have them maintained by a retired system for some legally defined period) is considered as a viable option.
So yes, buggy software does break organizations.
As someone who works for an organization that is on the receiving end of bad software, I can assure you this is not the case. There is lost opportunity, due to decreased productivity and outright lost clients. There is a souring of the work environment, an environment where people are constantly putting out fires and there is constant churn in staff because of that.
Perhaps the organization I work for is unique in that it is public sector and cannot go out of business (which appears to be the author's definition of broken). Perhaps the author has never dealt with software where edge cases involving lost and phantom data is a daily norm. Clients, real people with real lives, are literally lost by the system.
Things are so bad that losing several years of business records (or, more likely, paying to have them maintained by a retired system for some legally defined period) is considered as a viable option.
So yes, buggy software does break organizations.
It really, really depends on the use-case. You certainly don't want bugs in your MMU, process scheduler, sensors, security libraries, etc.
But a piece of line-of-business software that sends out the wrong email once because someone made a bug in a batch script? Sure. Not the end of the world.
Users will tolerate some level of errors. Even if that level is sometimes expected to be 0.
But a piece of line-of-business software that sends out the wrong email once because someone made a bug in a batch script? Sure. Not the end of the world.
Users will tolerate some level of errors. Even if that level is sometimes expected to be 0.
I disagree with this conclusion because it's taking as implicit the assumption that the organization is the thing to be victorious, which is a misunderstanding of organizational politics.
There is no organization. There is only people. Some of those people, like your boss or your investors, may want you to burn yourself out or do unethical things to your customers so they can make more money. It's your prerogative to tell them to fuck off, and go work somewhere else if you don't get your way. Some people don't have that capacity. Software developers generally do.
Perverse organizational incentives will pressure you to be bad. Refuse.
There is no organization. There is only people. Some of those people, like your boss or your investors, may want you to burn yourself out or do unethical things to your customers so they can make more money. It's your prerogative to tell them to fuck off, and go work somewhere else if you don't get your way. Some people don't have that capacity. Software developers generally do.
Perverse organizational incentives will pressure you to be bad. Refuse.
"Software quality" is such ambigously defined term that all of these can simultaneously be true (for different products):
* software quality is (more) important
* software quality is not (very) important
* bugs are not important (aka "i don't test, but when I do, it's in production")
* the less bugs you have the better, but they're not gonna kill you
* bugs are safety-critical
* slow delivery is fine
* slow delivery is not fine
In the last 20 or so years working as software developer professionaly, in various teams and quite different projects, I've noticed that:
* people (decision makers, software buyers/clients) always say they want higher quality
* same people very rarely want to back that up with their budget
* people almost always prefer more features to more quality (even when explicitly told that more feature is not a good thing for eg. MVP stage of a startup)
* most people except dedicated QA engineers can only think of a happy path (if everything works the ideal way); this includes not only other engineers (who try to think up failure scenarios but often miss important ones), but also designers (when have you ever seen a design mockup for "DNS lookup failure connecting to the server" case?), and definitely product owners or clients (in context of a dev agency)
* I have never worked on a nontrivial software project that didn't change requirements/scope
Myself, as with many other software developers who chose this field out of passion and not stratospheric wages, was for a long time appalled by this apparent lax approach to quality? Don't you want to have the best possible product?
But working with a lot of diverse clients while doing dev agency work, and running several startups myself, has taught me that software quality is not, and (it pains me to write this) should not be, the top priority. Product-market fit, treating your customers well, having a sustainable monetization strategy, marketing and sales ... if you don't execute well on those, nobody will notice the (lack of) quality[0].
In most organizations (except the ones that are swimming in cash) it's a tough act to balance all of the above. While I wish for all the projects I work on to be the best they can be, a programming work of art if you will, I can certainly empathize for people who must prioritize otherwise.
[0] unless the quality is safety critical, in case I need to point this out explicitly.
* software quality is (more) important
* software quality is not (very) important
* bugs are not important (aka "i don't test, but when I do, it's in production")
* the less bugs you have the better, but they're not gonna kill you
* bugs are safety-critical
* slow delivery is fine
* slow delivery is not fine
In the last 20 or so years working as software developer professionaly, in various teams and quite different projects, I've noticed that:
* people (decision makers, software buyers/clients) always say they want higher quality
* same people very rarely want to back that up with their budget
* people almost always prefer more features to more quality (even when explicitly told that more feature is not a good thing for eg. MVP stage of a startup)
* most people except dedicated QA engineers can only think of a happy path (if everything works the ideal way); this includes not only other engineers (who try to think up failure scenarios but often miss important ones), but also designers (when have you ever seen a design mockup for "DNS lookup failure connecting to the server" case?), and definitely product owners or clients (in context of a dev agency)
* I have never worked on a nontrivial software project that didn't change requirements/scope
Myself, as with many other software developers who chose this field out of passion and not stratospheric wages, was for a long time appalled by this apparent lax approach to quality? Don't you want to have the best possible product?
But working with a lot of diverse clients while doing dev agency work, and running several startups myself, has taught me that software quality is not, and (it pains me to write this) should not be, the top priority. Product-market fit, treating your customers well, having a sustainable monetization strategy, marketing and sales ... if you don't execute well on those, nobody will notice the (lack of) quality[0].
In most organizations (except the ones that are swimming in cash) it's a tough act to balance all of the above. While I wish for all the projects I work on to be the best they can be, a programming work of art if you will, I can certainly empathize for people who must prioritize otherwise.
[0] unless the quality is safety critical, in case I need to point this out explicitly.
Ultimately it all comes down to business case. If there isn't one to invest in CI, developer time to build a test suite with 100% coverage, and then ongoing upkeep of that, then it's not going to happen.
We are too often religious about these things, and may not have visibility within the business to see why there isn't a case for it.
Having said that it can also be very hard to convince management of the business case. That may have had a bad experience of these tool and practices before and see them as a waste. Thay may also, in that specific case, be right.
We are too often religious about these things, and may not have visibility within the business to see why there isn't a case for it.
Having said that it can also be very hard to convince management of the business case. That may have had a bad experience of these tool and practices before and see them as a waste. Thay may also, in that specific case, be right.
[deleted]
Maybe this is my years of experience talking, but curious if anyone else sees a connection between a slow delivery (aka slower development time) and less bugs? The idea is that developers have more time to think about the solution, write better code and overall not feel so rushed just to get something out the door. In my experience, the more pressure put on developers to get the thing done, the more bugs are introduced.
curious if anyone else sees a connection between a slow delivery (aka slower development time) and less bugs?
I certainly see a connection (several actually) between having more bugs and following development processes that emphasise metrics like time between starting work on a job and running the code in production. The advocates of Accelerate and the annual Google DORA reports seem to have taken over the web development asylum lately but the underlying arguments and data are still dubious at best.
I certainly see a connection (several actually) between having more bugs and following development processes that emphasise metrics like time between starting work on a job and running the code in production. The advocates of Accelerate and the annual Google DORA reports seem to have taken over the web development asylum lately but the underlying arguments and data are still dubious at best.
I've seen bad software and slow delivery lose companies important clients, causing job losses and severe financial problems.
I've also seen companies go out of business because they tried to make perfect software and never actually delivered it to anyone.
The right approach, I think, is to do the minimum necessary and do it very well. Easier said than done though.
I've also seen companies go out of business because they tried to make perfect software and never actually delivered it to anyone.
The right approach, I think, is to do the minimum necessary and do it very well. Easier said than done though.
I'd like to add: Do the minimum that is necessary. If it's well received, take the time to refine it. If it's not, scrap it.
This does seem like a bit of a fishing article, but I'll bite.
I've personally been involved in 2-3 software projects where the immediate ramifications related to software quality and delivery led to project failure.
The idea that bad (or no) architecture and terrible code won't have business impacts is just wrong. Perhaps he was lucky, perhaps we have collective survivorship bias, but I strongly disagree on the main premise here.
In one case, some bad decisions related to API design and customer extensibility led to a product that was simply impossible to change or evolve. That company did some custom work for a few clients on a shared framework, but after ~5-8 customers, the product was basically unchangeable. Some of the customers left, but we still couldn't change it, and the result was pretty predictable.
That you can limp along with terrible practices doesn't mean that better practices wouldn't improve the situation.
I've personally been involved in 2-3 software projects where the immediate ramifications related to software quality and delivery led to project failure.
The idea that bad (or no) architecture and terrible code won't have business impacts is just wrong. Perhaps he was lucky, perhaps we have collective survivorship bias, but I strongly disagree on the main premise here.
In one case, some bad decisions related to API design and customer extensibility led to a product that was simply impossible to change or evolve. That company did some custom work for a few clients on a shared framework, but after ~5-8 customers, the product was basically unchangeable. Some of the customers left, but we still couldn't change it, and the result was pretty predictable.
That you can limp along with terrible practices doesn't mean that better practices wouldn't improve the situation.
I think there's a few things at play when people talk about "software quality".
Some people hear it and think it means highly-structured abstraction towers vs low-structure imperative-heavy code. Some think about horrible spaghetti with 16 levels of indentation and 1-letter variable names (moving on to 2-letter once you exhaust the first 26) and if-statements whose predicates span twelve lines. Some people think about feature completeness and bugginess (or, rather, lack thereof).
I think they're dangerous over different time horizons and you can't just prioritize over all time horizons.
Spaghetti is dangerous in the short-term - it makes small changes difficult and unsafe. You'll end up tied by your own shoestrings and not even finish the project. Luckily if this is the only problem, it's also the easiest to clean up. (But it's not usually the only problem).
Less-structured code is good for when you don't understand your requirements well. (Which you typically never will a-priori, even if you think you do). It's easier to add more layers of abstraction than it is to change or remove them. If you prematurely build an abstraction tower, you will suddenly find yourself having to plumb information up and down the entire tower as the abstractions prove to leak or not be completely perpendicular to the kinds of features you need to add. But if you wait too long, you have too much unstructured code to refactor and you're now "chasing" people adding new features to the system while you try to refactor the system underneath them. So there's just some engineering gut-sense that's important to know when to stop and codify the patterns you've seen emerge into a useful, tight abstraction.
And then of course there's features and bugginess. It's possible for the shittiest code to have very few bugs and please users. The worst thing about the worst code I've ever seen in my career is that it worked - so it was hard to want to mess with it when it was so easy to ignore! And it doesn't help that what is intractable code to some people is only a little complicated to others. Especially if you use a big language and are combining a lot of esoteric/complex features, or have a complicated logical expression and you are a logician and others are not.
I've seen projects become unchangeable because the code was spaghetti. I've seen them become unchangeable because there was too much abstraction. I've seen them become unchangeable because there was too little abstraction. I've seen awful code make big impact and I've seen excellent code flop because it wasn't competitive with "worse" but more complete offerings.
(All of this reinforcing, of course, that it's impossible to estimate how long software will take..... all of it relies far too heavily on the intuition of your mid+senior engineering talent).
Some people hear it and think it means highly-structured abstraction towers vs low-structure imperative-heavy code. Some think about horrible spaghetti with 16 levels of indentation and 1-letter variable names (moving on to 2-letter once you exhaust the first 26) and if-statements whose predicates span twelve lines. Some people think about feature completeness and bugginess (or, rather, lack thereof).
I think they're dangerous over different time horizons and you can't just prioritize over all time horizons.
Spaghetti is dangerous in the short-term - it makes small changes difficult and unsafe. You'll end up tied by your own shoestrings and not even finish the project. Luckily if this is the only problem, it's also the easiest to clean up. (But it's not usually the only problem).
Less-structured code is good for when you don't understand your requirements well. (Which you typically never will a-priori, even if you think you do). It's easier to add more layers of abstraction than it is to change or remove them. If you prematurely build an abstraction tower, you will suddenly find yourself having to plumb information up and down the entire tower as the abstractions prove to leak or not be completely perpendicular to the kinds of features you need to add. But if you wait too long, you have too much unstructured code to refactor and you're now "chasing" people adding new features to the system while you try to refactor the system underneath them. So there's just some engineering gut-sense that's important to know when to stop and codify the patterns you've seen emerge into a useful, tight abstraction.
And then of course there's features and bugginess. It's possible for the shittiest code to have very few bugs and please users. The worst thing about the worst code I've ever seen in my career is that it worked - so it was hard to want to mess with it when it was so easy to ignore! And it doesn't help that what is intractable code to some people is only a little complicated to others. Especially if you use a big language and are combining a lot of esoteric/complex features, or have a complicated logical expression and you are a logician and others are not.
I've seen projects become unchangeable because the code was spaghetti. I've seen them become unchangeable because there was too much abstraction. I've seen them become unchangeable because there was too little abstraction. I've seen awful code make big impact and I've seen excellent code flop because it wasn't competitive with "worse" but more complete offerings.
(All of this reinforcing, of course, that it's impossible to estimate how long software will take..... all of it relies far too heavily on the intuition of your mid+senior engineering talent).
No, both this and the original blog entry are myopic.
There have been companies that went out of business, not only the software publisher but companies that relied on such software.
If the assertion is that "known unknown" bugs exist in code, and the publishers are aware that there might be bugs but in their QA/QC could not identify them, I can accept it. But, if the assertion is that there are "known known" bugs exists, I find it bordering unethical. The only way I can accept this if the publishers informs the buyer, and let them know what is the impact, the plan of action to remediate or mitigate the bug, and in what time frame.
"Known known bugs" (the publisher knew about the bug for extended period of time, and never informed the users) have destroyed multitude of businesses.
There have been companies that went out of business, not only the software publisher but companies that relied on such software.
If the assertion is that "known unknown" bugs exist in code, and the publishers are aware that there might be bugs but in their QA/QC could not identify them, I can accept it. But, if the assertion is that there are "known known" bugs exists, I find it bordering unethical. The only way I can accept this if the publishers informs the buyer, and let them know what is the impact, the plan of action to remediate or mitigate the bug, and in what time frame.
"Known known bugs" (the publisher knew about the bug for extended period of time, and never informed the users) have destroyed multitude of businesses.
The author's insight is fair on a personal level, but the thesis is not very useful as a generalization. The reality is that bugginess and velocity are all relative, and every situation is different. It's good to understand the leadership viewpoint and be realistic about the business that employs you so you can decide if it fits your values and look for a change if not. If you care a lot about the quality of what you produce, there are plenty of companies that need and value that. On the other hand, many people just want a steady job where expectations are clear and ambiguity isn't too high; mediocrity is one way to achieve this, and it can work fine (for a while at least) with sufficient moats.
Valentina's original blog post is easily misinterpreted. She is correct that quality is optional. Obviously so, many companies do fine with very poor quality software.
But something being mandatory or not doesn't influence the ROI. Something can be optional and high ROI or mandatory and low ROI. Low quality software is quite destructive over the long term and it is a horrible mistake to let coders go wild writing buggy software slowly.
That being said, the post is probably clickbait aimed at promoting her services. I doubt many people would make the mistake of thinking it a good idea to purposefully low quality written at slow speed.
But something being mandatory or not doesn't influence the ROI. Something can be optional and high ROI or mandatory and low ROI. Low quality software is quite destructive over the long term and it is a horrible mistake to let coders go wild writing buggy software slowly.
That being said, the post is probably clickbait aimed at promoting her services. I doubt many people would make the mistake of thinking it a good idea to purposefully low quality written at slow speed.
Most enterprises that fail do so because they run out of money. In software business where the dominant cost is engineering salaries [1], poor tech practices with expensive fixes are a major factor in running out of money.
More abstractly, high productivity engineers help you to exhaust the idea space through delivering products into real world before you run out of money. It's better to fail because you discovered your supposed gold mine had no gold rather because you ran of mining resources half way through.
[1] Notably this doesn't apply to capital intensive business with high marketing or infrastructure costs
More abstractly, high productivity engineers help you to exhaust the idea space through delivering products into real world before you run out of money. It's better to fail because you discovered your supposed gold mine had no gold rather because you ran of mining resources half way through.
[1] Notably this doesn't apply to capital intensive business with high marketing or infrastructure costs
Bugs are never OK. Slow delivery, though? That depends on what you mean by "slow delivery". I happen to think that the trend of rapid release has been pretty awful for software product and service quality, so if "slow delivery" is the absence of that, I think it's desirable.
Bugs are inevitable, but that doesn't make them OK.
But that's not the point the article is making. They're talking about whether or not bugs and slow delivery will break a company. Obviously it won't, because both have been a part of what the industry releases since the start.
Bugs are inevitable, but that doesn't make them OK.
But that's not the point the article is making. They're talking about whether or not bugs and slow delivery will break a company. Obviously it won't, because both have been a part of what the industry releases since the start.
I hate to be that guy, but this is an artifact of ZIRP. Watch what happens when interest rates go up and people start demanding more value for their SaaS subscription. It'll be a return to software development in the 1980s when most of our current development processes were developed (which most companies talk about but don't follow). Most companies aren't gonna make it out of their technical debt holes...
I have, for the record, seen companies lose market share, commercial opportunities, and once have to go through financial restructuring due to missing product goals.
Technical debt / quality / boneheadedness was at least a factor in each case.
I've also worked in financial services where a bad enough bug can and will lose you your (jurisdiction-appropriate) license to to do business.
Technical debt / quality / boneheadedness was at least a factor in each case.
I've also worked in financial services where a bad enough bug can and will lose you your (jurisdiction-appropriate) license to to do business.
Bug is ok, if you afford to crash every Mars rovers onto the surface, or fucked up every ignition moment and blow up every space shuttles, etc.
If you don't care about anything that you should be care about, of course nothing will be an issue.
If you don't care about anything that you should be care about, of course nothing will be an issue.
The meme with the men at the gallows, nooses around their necks and one looking at the other saying "First time?" comes to mind. That is pretty much software development in a nutshell.
It's hard to turn "bug free" into a selling point. Nobody has "bug free" as a checkbox on their decision matrix.
[deleted]
I got really surprised when I read the “original blog post” because it's less a blog post than is it some bland advertisement to “some online coaching that will make you a 10x engineer”. So when I see the OP qualifying as ”short and well-written, well worth the read“, I suspect all this is just a PR stun to attract attention to Cupać's paid online training program… (and this seems to be working pretty well since it's now on the top of the front page)
Bugs in cybersecurity products are not okay at all. We call those zero-days.
Working rapidly without bugs is preferable to working slowly with bugs.
Wise men from a prior generation said: worse is better.
Original title of this HN post was:
"Responding to “Are bugs and slow delivery ok?”: The blog post that I’ve hated the most, ever"
But if you scroll down to the end of the post:
> So thank you, Valentina, for your thought-provoking post. I loved to hate it.
So the author does not actually hate the post they say in the title they hated.
"Responding to “Are bugs and slow delivery ok?”: The blog post that I’ve hated the most, ever"
But if you scroll down to the end of the post:
> So thank you, Valentina, for your thought-provoking post. I loved to hate it.
So the author does not actually hate the post they say in the title they hated.
In the 30 years I've been coding, languages have come and gone, platforms have come and gone, frameworks have come and gone, methodologies have come and gone, but two things have never changed:
1) Software development is wildly unpredictable. I've never met anybody who could reliably predict how long something would take to develop, even if the requirements were "frozen" and even if they were the person delivering it. I have met people who could predict ahead of time how long it would take them to deliver something, but what they delivered always had "bugs" (either actual incorrect behavior, or edge cases that nobody thought of until they actually used the software) that themselves took an unpredictable amount of time to fix.
2) No organization can ever accept any amount of unpredictability. Every "advance" in software development I've ever seen has claimed to (implicitly or explicitly) remove the inherent unpredictability in software development, although none have actually delivered. They're fine with spending months in planning meetings while not actually developing anything tangible as long as those planning meetings provide the illusion of predictability.