Ask HN: What would happen if we prioritised all bugs over all new features?
76 comments
This is known as a zero bug policy [1]. I've had decent success implementing it with my team. The main advantages are:
- Prioritizing is hard, so avoid wasting brain cycles deciding how important your bugs are
- It encourages all of your team to get things right the first time, because if they don't they know they will be going back to fix it immediately.
- If you want to create a culture of quality then it's an obvious first step.
- It saves time in the long term by addressing problems when you have the most context and by avoiding building hack upon hack upon hack
In extreme cases you can relax the policy, but be aware that if you don't quickly correct course then things will be permanently worse. Also accept that hard external deadlines are not suited to this approach, but using triage some of that can be mitigated.
[1] https://sookocheff.com/post/process/zero-bug-policy/
- Prioritizing is hard, so avoid wasting brain cycles deciding how important your bugs are
- It encourages all of your team to get things right the first time, because if they don't they know they will be going back to fix it immediately.
- If you want to create a culture of quality then it's an obvious first step.
- It saves time in the long term by addressing problems when you have the most context and by avoiding building hack upon hack upon hack
In extreme cases you can relax the policy, but be aware that if you don't quickly correct course then things will be permanently worse. Also accept that hard external deadlines are not suited to this approach, but using triage some of that can be mitigated.
[1] https://sookocheff.com/post/process/zero-bug-policy/
https://www.joelonsoftware.com/2000/08/09/the-joel-test-12-s...
Article from back in 2000, based on info from ~1990.
> 5. Do you fix bugs before writing new code?
> To correct the problem, Microsoft universally adopted something called a “zero defects methodology”. Many of the programmers in the company giggled, since it sounded like management thought they could reduce the bug count by executive fiat. Actually, “zero defects” meant that at any given time, the highest priority is to eliminate bugs before writing any new code.
Please read the entire article, it's worth it ;-)
Article from back in 2000, based on info from ~1990.
> 5. Do you fix bugs before writing new code?
> To correct the problem, Microsoft universally adopted something called a “zero defects methodology”. Many of the programmers in the company giggled, since it sounded like management thought they could reduce the bug count by executive fiat. Actually, “zero defects” meant that at any given time, the highest priority is to eliminate bugs before writing any new code.
Please read the entire article, it's worth it ;-)
I was there a little after and his division nor mine still had any such policy.
Joel worked on Office. Have you heard the hair-on-fire customer stories about Office from the 90s? It was a draconian response to what was probably seen as a serious company-wide problem at the time.
Joel worked on Office. Have you heard the hair-on-fire customer stories about Office from the 90s? It was a draconian response to what was probably seen as a serious company-wide problem at the time.
Knowing that essay is 20 years old makes me feel old.
One slightly different approach I've seen at some more old school industry customer: A bug with a semi-doable workaround is usually not urgent. This is less a problem with a more continuous deployment, but if your release cycles are measured in weeks or low-digit months, it's just often not feasible to roll back or hotfix, if the (even breaking) bug can be worked around somehow.
That made me rethink my usual "prioritize urgent bugfixes over features" stance, but it works better if you have a lower number of (known) users who communicate, over a mass audience.
That made me rethink my usual "prioritize urgent bugfixes over features" stance, but it works better if you have a lower number of (known) users who communicate, over a mass audience.
That's a really good point. If your userbase is feels listened to (which is admittedly easier in B2B than B2C) then these decisions become much easier.
We did releases that were only bug fixes and maintenance. There can be a long tail of bugs in a complex system, most of which are relatively harmless and only occur in very rare situations; it’s hard to justify fixing these when paying customers are asking for a new feature.
Also you’d be surprised at what gets classified as a bug when you apply this.
Ultimately though, everybody does this. Critical bugs get fixed first. Anything too far down the backlog de-facto doesn’t get fixed. I like the intellectual honesty that this approach brings, in that it forces you to set a bar for bugs not worth fixing and consequently marking them as won’t fix.
Final note: whether a bug is worth fixing changes over time. Maybe your best engineer can’t find it after two weeks. Maybe your biggest customer just ran into it. Maybe you can’t reproduce it. Maybe the platform causing it got acquired by Google.
Can it work? Yes, but it doesn’t look very different to what you’re probably already doing in practice.
Also you’d be surprised at what gets classified as a bug when you apply this.
Ultimately though, everybody does this. Critical bugs get fixed first. Anything too far down the backlog de-facto doesn’t get fixed. I like the intellectual honesty that this approach brings, in that it forces you to set a bar for bugs not worth fixing and consequently marking them as won’t fix.
Final note: whether a bug is worth fixing changes over time. Maybe your best engineer can’t find it after two weeks. Maybe your biggest customer just ran into it. Maybe you can’t reproduce it. Maybe the platform causing it got acquired by Google.
Can it work? Yes, but it doesn’t look very different to what you’re probably already doing in practice.
I think you can have alternative: bugs are either critical or "non-critical". Non-critical end up in the backlog, but are classed before improvements and before "non-critical" features.
How does it really work: let's say you have 2 week sprints (quite standard). Then you break the agile/scrum "story points" and velocity: it isn't usefull for your team (but might be for management). After quick estimates, each member take what he estimate to be a week, a week an a day of work on critical bugs, then critical features. Because let's be honest, you alway have more "critical" stuff to do each week, and will never get your backlog small enough to reach non-critical stuff.
Once you're done with your "critical task", depending on the time you have left, you take a non-critical stuff, in preference in backlog order if you're senior and long-time IC (lot of bugfixing), preference to "i'm sure i know how to do that" then "unknow project but seems easy enough" for juniors, and seniors who just arrived.
Having a gigantic backlog isn't an issue as long as each task are assigned to a product and a Major version: that will allows you to discard those tasks easily if the product isn't sold anymore of if the version changed.
How does it really work: let's say you have 2 week sprints (quite standard). Then you break the agile/scrum "story points" and velocity: it isn't usefull for your team (but might be for management). After quick estimates, each member take what he estimate to be a week, a week an a day of work on critical bugs, then critical features. Because let's be honest, you alway have more "critical" stuff to do each week, and will never get your backlog small enough to reach non-critical stuff.
Once you're done with your "critical task", depending on the time you have left, you take a non-critical stuff, in preference in backlog order if you're senior and long-time IC (lot of bugfixing), preference to "i'm sure i know how to do that" then "unknow project but seems easy enough" for juniors, and seniors who just arrived.
Having a gigantic backlog isn't an issue as long as each task are assigned to a product and a Major version: that will allows you to discard those tasks easily if the product isn't sold anymore of if the version changed.
It is also useful to have handful of "easy to fix" bugs on backlog to use as on-boarding for new developers .
Your point on intellectual honesty really resonates.
I personally like the idea that for every bug you either: a) fix it with highest priority, or b) mark it as "won't fix".
I think this would really force you to make a decision on a bug, rather than adding it to some never ending list of lists.
If a bug is worth fixing, it will come up again
I personally like the idea that for every bug you either: a) fix it with highest priority, or b) mark it as "won't fix".
I think this would really force you to make a decision on a bug, rather than adding it to some never ending list of lists.
If a bug is worth fixing, it will come up again
- New features help close sales.
- New features also add tech debt, complexity, cost, bugs, makes you slower, and will lose you some sales that way.
- At 0 features you really need to implement features instead of messing about with the bugs in the devops scripts.
- At e.g. Windows scale, maybe stop messing about with new features nobody asked for and fix some bugs, yes.
- In a Laffer-curve-like effect, there must be a point where it peaks and it's better to fix bugs than to implement a new feature.
- It's a very difficult to identify where you are in the curve.
- One of the measures is simple "do X, get money from this guy I have on the phone"
- The other measure is fuzzy, lags, is subjective, can't be traced to a particular feature.
Good luck!
- New features also add tech debt, complexity, cost, bugs, makes you slower, and will lose you some sales that way.
- At 0 features you really need to implement features instead of messing about with the bugs in the devops scripts.
- At e.g. Windows scale, maybe stop messing about with new features nobody asked for and fix some bugs, yes.
- In a Laffer-curve-like effect, there must be a point where it peaks and it's better to fix bugs than to implement a new feature.
- It's a very difficult to identify where you are in the curve.
- One of the measures is simple "do X, get money from this guy I have on the phone"
- The other measure is fuzzy, lags, is subjective, can't be traced to a particular feature.
Good luck!
I like the curve idea. Makes sense.
When customers aren't signing up because of lacking feature -> build features. When customers are churning because of bugs -> fix bugs. Else -> somewhere in the middle
When customers aren't signing up because of lacking feature -> build features. When customers are churning because of bugs -> fix bugs. Else -> somewhere in the middle
On a sufficiently complex product it is impossible to fix all known bugs, even classified as will fix. Another issue - it will take so much time that your company will be left behind by the competitors. Maybe less so in the web, but with hardware it's a question of multiyear contracts, so getting abandoned by a big customer in favor of a feature rich competitor may mean that you won't get a second chance with them any time soon. Of course a reverse happens too - if your market leading product is more and more buggy over time, they may abandon you or force to fix most bugs, or force to do significant structural changes etc.
As systems get more complex slowing down and getting bugs fixed becomes more and more important. If you don't fix them fairly quickly you end up with code elsewhere (either consumers of your APIs, or within your own application) adapting to your bug, leaving you with new bugs when you get around to fixing it!
> On a sufficiently complex product it is impossible to fix all known bugs
You really think so?
Surely it's just a matter of picking a sufficiently high bar for "will fix" and then focusing some time on it.
You really think so?
Surely it's just a matter of picking a sufficiently high bar for "will fix" and then focusing some time on it.
We had a P0 bug which caused unplanned device reboots in the field with unclear cause. It is still open, 3 years since first detection. And it looks like it will be open until device end of life. We have a mitigation for which affects performance, and that's about it. It was tracked down to deep inside Linux kernel and some firmware interactions and no amount of work produced a fix for it.
And that was a mission critical bug, which was pushed by the biggest and super important customer (and others too).
Ask yourself: given the system in its current state what change will make my customers most happy? Is it fixing this bug, or is it shipping this new feature?
Do the thing that will produce the most customer happiness in the shortest amount of time next.
Do the thing that will produce the most customer happiness in the shortest amount of time next.
This is a nice way to put it. Our platform is already pretty mature, and customers are happy.
Naturally, the wishlist of new features never ceases to shrink ... but the stability of the existing platform is what people really appreciate.
Naturally, the wishlist of new features never ceases to shrink ... but the stability of the existing platform is what people really appreciate.
Morale decline from oscillating priorities.
You are bug-free, start working on new feature, new bug reports come in, and you have to pause and work on them.
Fixing bugs is not fun work because there is usually a quick fix in an ugly way, and then a perfect fix via a large refactor and re-architecture. This results in that "soul-destorying" feeling of: if I had enough time I could fix this properly in the right way with clean code and avoid huge amounts of bugs, but alas I am just piling on tech debt.
You are bug-free, start working on new feature, new bug reports come in, and you have to pause and work on them.
Fixing bugs is not fun work because there is usually a quick fix in an ugly way, and then a perfect fix via a large refactor and re-architecture. This results in that "soul-destorying" feeling of: if I had enough time I could fix this properly in the right way with clean code and avoid huge amounts of bugs, but alas I am just piling on tech debt.
My main concern with the idea is that what counts as a "bug" is sometimes clear, but often subjective. A missing feature can easily masquerade as a bug in some organisations, and the policy you suggest would encourage dressing up missing features as bugs, leading to suboptimal information flows and feedback in the organisation.
Effectively, "I think this is important so I will argue you should work on this before that bug" is much better than "I think this is important so I will argue this is a bug."
Effectively, "I think this is important so I will argue you should work on this before that bug" is much better than "I think this is important so I will argue this is a bug."
I have seen this problem solved with a super simple bug classification that anyone can leverage (even customer success) associated with priority and time to fix expectations.
YMMV, you have to adapt it to your usecase (B2B / B2C? contractual SLA? ...). But it can be something around:
YMMV, you have to adapt it to your usecase (B2B / B2C? contractual SLA? ...). But it can be something around:
- P0 : a bug prevents a significant part of the customers (= paying users) to perform one of the core functionality of the product : at least one dev drops what he is doing right now and investigate, fix it himself or send it to someone responsible for the bug who should drop what he is doing right now and fix it. Target is to have it fixed in under a few hours.
- P1 : a bug prevents a few customers to perform an important yet non core functionality of the product: next dev available have a look at it. Target is to have it fixed in under a few days.
- P2 : a bug customers can live with (concerns few customers / there is a workaround / ... ) -> fixed in best effort, in practice, we fixed them when doing other features near that code. It can take a lot of time to fix them (if we ever fix them, and it's ok. The good thing about tech debt is that it's a debt you don't have to pay, when you remove/replace a feature for instance).This is actually exactly what we had in mind. Except that P2 gets put in a "won't fix" bucket.
- P0 means drop what you're doing and fix it now
- P1 means fix after you've finished what you're doing
- P2 means "won't fix" (but keep a note of it in case we ever get to that perfect situation where we have more time than features to build ;))
- P0 means drop what you're doing and fix it now
- P1 means fix after you've finished what you're doing
- P2 means "won't fix" (but keep a note of it in case we ever get to that perfect situation where we have more time than features to build ;))
If you're driving users away with some major bugs you better try to fix them ASAP. But chasing "100%" is usually a bad business strategy IMO. When you have an absolutist policy like that you risk losing a lot of momentum over something small that isn't driving much value, just to satisfy the arbitrary policy.
Is it worth not making progress on any new features, all because of a smaller bug or issue? Can just one person work on the small bug while the rest of the team starts a new feature?
Sometimes the best bug fix is a new feature that depreciates the bug, so be sure to consider the estimated lifetime of the bug if you keep progressing your platform and maybe don't spend too much time fixing things that will be soon phased out anyways, unless they're really major issues that are rapidly hurting the business or reputation and need immediate fixing.
If you really want to halt all new features, I might try putting it on a calendar. Maybe you can afford 1 month, 1 quarter, or half a year on just bug fixing but eventually you have to keep moving forward in some way. Unless your platform is already pretty feature complete (which it doesn't sound like it is) you might do more harm than good when delaying your next core feature releases.
Is it worth not making progress on any new features, all because of a smaller bug or issue? Can just one person work on the small bug while the rest of the team starts a new feature?
Sometimes the best bug fix is a new feature that depreciates the bug, so be sure to consider the estimated lifetime of the bug if you keep progressing your platform and maybe don't spend too much time fixing things that will be soon phased out anyways, unless they're really major issues that are rapidly hurting the business or reputation and need immediate fixing.
If you really want to halt all new features, I might try putting it on a calendar. Maybe you can afford 1 month, 1 quarter, or half a year on just bug fixing but eventually you have to keep moving forward in some way. Unless your platform is already pretty feature complete (which it doesn't sound like it is) you might do more harm than good when delaying your next core feature releases.
Yes you can do it, yes it can work. However as with all things there is nuance. This is a scheduling problem, and just like with a cpu scheduler there are degenerate cases you may need to avoid/handle.
Two problem cases likely to pop up.
1) Lots of fixing when a bigger refactor is required. A poorly written area of code, or a poor design, may be causing high churn and wasted effort. The solution I’ve found to this problem is to track defects by code area and review once metric exceeds heuristic.
2) A team choked on defects only for a long period of time. This obviously has many negative side effects. It tends to happen in really important component and require most experienced developers. Any new starters run for the hills when a team gets into this state, therefore compounding the issues. The solution to this (though this is just my opinion), is to never allow 100% of time over [fixed interval] to be spent on defects. No more than 50%. The bugs will still get fixed, just take longer, and new development is still happening.
Overall though, I think a “defects first” approach is the right one, just have a plan for these negative cases.
Two problem cases likely to pop up.
1) Lots of fixing when a bigger refactor is required. A poorly written area of code, or a poor design, may be causing high churn and wasted effort. The solution I’ve found to this problem is to track defects by code area and review once metric exceeds heuristic.
2) A team choked on defects only for a long period of time. This obviously has many negative side effects. It tends to happen in really important component and require most experienced developers. Any new starters run for the hills when a team gets into this state, therefore compounding the issues. The solution to this (though this is just my opinion), is to never allow 100% of time over [fixed interval] to be spent on defects. No more than 50%. The bugs will still get fixed, just take longer, and new development is still happening.
Overall though, I think a “defects first” approach is the right one, just have a plan for these negative cases.
Really good point about bug fixing affecting engineer morale. Super important.
One (arguably positive) side-effect I'm wondering might be possible is that: if bugs are always prioritised first .... and engineers are often very creative at solving problems .... will they perhaps come up with creative ways to reduce bugs in the first place?
Or, it might go all wrong -> and we create a dangerous culture of "swallow that exception" :D
One (arguably positive) side-effect I'm wondering might be possible is that: if bugs are always prioritised first .... and engineers are often very creative at solving problems .... will they perhaps come up with creative ways to reduce bugs in the first place?
Or, it might go all wrong -> and we create a dangerous culture of "swallow that exception" :D
Assuming these are bugs which have workarounds and don't cause things like data corruption. Then it probably falls into the adage from advertising "50% of all advertising is a waste, you just don't know which 50%".
Another way to look at it is the delayed effect of doing nothing in either area. Bugs creeping in over the months and years may only become a problem when a competitor starts to be noticeably more stable.
Features that are delayed may have a delayed effect of a competitor getting ahead of you in the market and launching months before you'd be ready.
So I would say, "it depends". If you're in a growth market and are trying to capture market share, features might be best before non-critical bugs.
If you're in a stable market serving a huge amount of people, then fixing bugs has a much larger impact on your users.
You also have to consider the team and their morale over time. Too much churning through low value bugs can be demoralising where individuals might need some type of higher level thinking and creativity.
If it were me, I'd look at bugfix only sprints and adjust the frequency based on the above factors.
Another way to look at it is the delayed effect of doing nothing in either area. Bugs creeping in over the months and years may only become a problem when a competitor starts to be noticeably more stable.
Features that are delayed may have a delayed effect of a competitor getting ahead of you in the market and launching months before you'd be ready.
So I would say, "it depends". If you're in a growth market and are trying to capture market share, features might be best before non-critical bugs.
If you're in a stable market serving a huge amount of people, then fixing bugs has a much larger impact on your users.
You also have to consider the team and their morale over time. Too much churning through low value bugs can be demoralising where individuals might need some type of higher level thinking and creativity.
If it were me, I'd look at bugfix only sprints and adjust the frequency based on the above factors.
It would really depend on the severity of the bugs... assign severity levels as part of triaging.
Also what kind of bugs are coming up is important... I think bug do tell stories; it might help you identify issues with feature assumptions on workdlow or use-cases and that needs to inform your product development as a feedback loop to avoid technical debt and having to rewrite stuff later.
Depending on the size of your team, I would have a 2-4 people focusing on just bugs / QA (so you can catch most bugs going forward) while the rest of the team focuses on new features.
Also what kind of bugs are coming up is important... I think bug do tell stories; it might help you identify issues with feature assumptions on workdlow or use-cases and that needs to inform your product development as a feedback loop to avoid technical debt and having to rewrite stuff later.
Depending on the size of your team, I would have a 2-4 people focusing on just bugs / QA (so you can catch most bugs going forward) while the rest of the team focuses on new features.
Once you start factoring in reputational damage it becomes easier to choose what/if to fix. If you add new features but the resulting product is something that gives the idea that the platform is unstable or broken you'll end up with people switching to some other provider.
If the product has actual customers it makes ALWAYS sense to prioritize fixes along the hot paths. And it you should be easy even for non-technical people to understand. You lose customers.
A feature is not done until all major bugs or regressions are fixed.
If the product has actual customers it makes ALWAYS sense to prioritize fixes along the hot paths. And it you should be easy even for non-technical people to understand. You lose customers.
A feature is not done until all major bugs or regressions are fixed.
Adding new features will introduce new bugs in your already "bug fixed" parts.
Therefore go for release cycles. Lock features, fix all bugs, then release the version. Repeat for every cycle.
You might be forced to release with a few bugs. The conventional procedure is to publicly document the known bugs for each version.
If you are fighting fire with bugs, then you don't have stable product yet. Cut off up to a reasonable feature set and fix all bugs when you want to release.
Therefore go for release cycles. Lock features, fix all bugs, then release the version. Repeat for every cycle.
You might be forced to release with a few bugs. The conventional procedure is to publicly document the known bugs for each version.
If you are fighting fire with bugs, then you don't have stable product yet. Cut off up to a reasonable feature set and fix all bugs when you want to release.
I develop and support DXLab, a suite of interoperating, free (but not open-source) applications for the worldwide amateur radio community; these applications continuously populate databases with realtime information, and interact with many other applications and physical devices (radios, antenna rotators) via serial ports, DDE, UDP, and TCP links. Since the first public release 22 years ago, my policy has been "all reported defects are corrected within 24 hours". Interaction with the user community is direct - via an online group. I typically make public releases bearing new functionality 2-3 times per month.
This policy's results have been excellent: users are focused on learning to better exploit the applications and suggesting new functionality rather than complaining about long-deferred defect repairs; even minor, easily worked-around defects create a negative user community mindset that can snowball. The absence of defects increases user confidence, and reduces user-perceived complexity.
This policy's results have been excellent: users are focused on learning to better exploit the applications and suggesting new functionality rather than complaining about long-deferred defect repairs; even minor, easily worked-around defects create a negative user community mindset that can snowball. The absence of defects increases user confidence, and reduces user-perceived complexity.
Whether this would work depends on the kind of system and the deadlines imposed by external needs of the company.
Does your system have a lot of (intentional or unintentional) emergent behavior, like a sandbox-heavy video game? You could end up never making a feature again.
Do your customers expect the frequent shipping of new features? Unless you can sell them on the idea of unexpected or infrequent addition of features, you could quickly lose your core audience.
However, if it is a product that no one is expecting tight deadlines on the release of new features, or the product's purpose is straightforward enough that there is no intentional emergent behavior in the system, then I could see someone running it in this way. I don't mean this in a cheeky way: there are definitely products that fit this criteria. It just won't be something that every product can reasonably do while also expecting to retain their user base.
Does your system have a lot of (intentional or unintentional) emergent behavior, like a sandbox-heavy video game? You could end up never making a feature again.
Do your customers expect the frequent shipping of new features? Unless you can sell them on the idea of unexpected or infrequent addition of features, you could quickly lose your core audience.
However, if it is a product that no one is expecting tight deadlines on the release of new features, or the product's purpose is straightforward enough that there is no intentional emergent behavior in the system, then I could see someone running it in this way. I don't mean this in a cheeky way: there are definitely products that fit this criteria. It just won't be something that every product can reasonably do while also expecting to retain their user base.
Your role as a company is to maximize revenue. If presence of bugs makes you lose revenue by customer churning or burned-out developers leaving, fix bugs. If not, create new features. And define a bug as "thing that the customer complained about". If customers don't complain, it's either not a bug, or you are solving a wrong problem.
That seems short-sighted. Just because a customer hasn’t complained about it doesn’t mean it isn’t a bug.
I’m not disagreeing that customer complaints are important for prioritisation, but the problem isn’t quite as first-order as that.
In an extreme case, imagine you found a bug that corrupts data if a customer name begins with z. Would you not fix this bug just because you don’t have any customers whose name begins with z?
I’m not disagreeing that customer complaints are important for prioritisation, but the problem isn’t quite as first-order as that.
In an extreme case, imagine you found a bug that corrupts data if a customer name begins with z. Would you not fix this bug just because you don’t have any customers whose name begins with z?
That's a bit oversimplified. Surely a bug that could lead to data loss needs to be fixed ASAP regardless of whether a customer complains about it or not.
The thing about bugs is their subtle influence on overall product quality.
It's like building a wall; if one layer of bricks is laid unevenly, at least a number of layers built on top of it will have to compensate. Usually, this compensation takes form in increased development times or increased complexity / convolution of new features.
Furthermore, it lowers end users' overall trust in the platform.
Both of these two effects will have at least some negative impact on profitability, though it may be lower than the increased profitability gained by adding new features.
I'm not saying all bugs should be fixed immediately at the expense of new features, but I've rarely been in a situation, where it felt "right" to ignore a bug indefinitely.
It's like building a wall; if one layer of bricks is laid unevenly, at least a number of layers built on top of it will have to compensate. Usually, this compensation takes form in increased development times or increased complexity / convolution of new features.
Furthermore, it lowers end users' overall trust in the platform.
Both of these two effects will have at least some negative impact on profitability, though it may be lower than the increased profitability gained by adding new features.
I'm not saying all bugs should be fixed immediately at the expense of new features, but I've rarely been in a situation, where it felt "right" to ignore a bug indefinitely.
That's risk management. If the risk of losing the customers' (or future prospects') revenue is big enough, you should fix the bug. There might also be the risk of fines.
Many bugs don’t cause customer churn short term, but it does long term. And that is difficult to measure. They might not tell you for all kinds of reasons.
It's often faster and cheaper to fix bugs right away. It often takes longer to put the bug in a bug tracker and prioritize it than to fix it.
I have a simple rule that has worked to keep bugs at zero. If you find a bug, drop what you are doing and fix it. If it takes more than 4 hours, only then, put it in your backlog and prioritize it like everything else. Don't keep a separate bug list! Bugs should be in your backlog if they are serious enough (more than 4 hours to fix).
99% of bugs take less than 4 hours to fix.
Bugs slow you down. Keep them at zero! You will be able to finish everything faster!
I have a simple rule that has worked to keep bugs at zero. If you find a bug, drop what you are doing and fix it. If it takes more than 4 hours, only then, put it in your backlog and prioritize it like everything else. Don't keep a separate bug list! Bugs should be in your backlog if they are serious enough (more than 4 hours to fix).
99% of bugs take less than 4 hours to fix.
Bugs slow you down. Keep them at zero! You will be able to finish everything faster!
Microsoft rather famously tried this[1]. And it seemed to really help them.
I've worked places that have tried it, too. It does work; in my experience, it literally always improves software quality. But I think it is like how pretty much all diets work, at first, but not over the long term.
It becomes unsustainable, as the pressure to work on features grows.
Continuing with my diet analogy, I now think of it like bulking and cutting phases. When you are bulking up your product with new muscles (features) you are also adding fat (bugs). At some point the percentage starts growing unhealthy, slowing you down and causing all sorts of ancillary problems.
Might be time for a cut phase then.
My analogy breaks down (because it's not really a good analogy) with a bigger team. Then you can have people assigned to bug cleanup. But this leads to its own set of problems — do you pay bug cleanup engineers a bunch extra? Because they will tend to be reading Who's Hiring top to bottom after a few weeks of that.
I think this does work great, though, when it is for a pre-determined fixed duration of time. (e.g. "3 iterations" or "one quarter").
[1]: I think where I read about this is now behind a paywall, but this seems to be the same initiative: https://sriramk.com/memos/zerodef.pdf
I've worked places that have tried it, too. It does work; in my experience, it literally always improves software quality. But I think it is like how pretty much all diets work, at first, but not over the long term.
It becomes unsustainable, as the pressure to work on features grows.
Continuing with my diet analogy, I now think of it like bulking and cutting phases. When you are bulking up your product with new muscles (features) you are also adding fat (bugs). At some point the percentage starts growing unhealthy, slowing you down and causing all sorts of ancillary problems.
Might be time for a cut phase then.
My analogy breaks down (because it's not really a good analogy) with a bigger team. Then you can have people assigned to bug cleanup. But this leads to its own set of problems — do you pay bug cleanup engineers a bunch extra? Because they will tend to be reading Who's Hiring top to bottom after a few weeks of that.
I think this does work great, though, when it is for a pre-determined fixed duration of time. (e.g. "3 iterations" or "one quarter").
[1]: I think where I read about this is now behind a paywall, but this seems to be the same initiative: https://sriramk.com/memos/zerodef.pdf
I worked at Microsoft later than that and my team (most of the times there is no "Microsoft did X", it is a team, department or product decision) tried a variation on Zero Bugs policy. We either fixed immediately, closed as won't fix (and here there's the question of correlating future related bugs with different symptoms) or turn into a new feature request of the fix requires bigger changes but is still needed. It worked for a while, but as others said it is hard to maintain this policy over time for a complex product.
I like that idea of a cycle, going between a feature-adding phase and a bug-fixing phase. I imagine having experience in both and knowing the other is coming up shortly will also improve people's forward planning in terms of code structure
How much money are the bugs costing the company vs. how much money (time) it costs to fix them? Classify your bugs on scale of impact on end users, profit loss and other important factors. Depending on the amount and scope of the bugs, one idea is to have a bug fix sprint, I use that term loosely, essentially a time box to work on bugs and make things nicer. When the time box is over get back to feature work.
Using libraries and frameworks gets you speed in delivery because you don't have to write that stuff yourself, the downside is there are likely bugs in that software or in how your system interacts with it. No software will be "bug free", striving for that is a fool's errand.
Using libraries and frameworks gets you speed in delivery because you don't have to write that stuff yourself, the downside is there are likely bugs in that software or in how your system interacts with it. No software will be "bug free", striving for that is a fool's errand.
Going to extremes never works like ONLY bug fixes or address tech debt for X time. Or only shipping features even if the foundation is shit.
The best option is to accept software is soft and malleable.
If it prints money but it’s some legacy code that smells like shit and has bugs, just write a new clean interface on top and build to it. Over time you’ll end up replacing the underlying system while shipping new features and fixing bugs as they pop up. Oh and the money keeps printing.
It it doesn’t print money and the code is shit, the project should just be scrapped. Who cares.
If people constantly keep building shit code and never improve, just quit because that company probably sucks.
Be extreme by being practical.
The best option is to accept software is soft and malleable.
If it prints money but it’s some legacy code that smells like shit and has bugs, just write a new clean interface on top and build to it. Over time you’ll end up replacing the underlying system while shipping new features and fixing bugs as they pop up. Oh and the money keeps printing.
It it doesn’t print money and the code is shit, the project should just be scrapped. Who cares.
If people constantly keep building shit code and never improve, just quit because that company probably sucks.
Be extreme by being practical.
Great. Now what's a "bug" and what's actually not a bug but some other category of thing so that we don't have to drop whatever else we're doing to fix it?
Places with zero bugs policies seem to have much branchier defect taxonomies than everyone else.
I've also seen shops just decide to ignore entire classes of problems so that they don't put too many rows in their bug database. So, willful ignorance is another option. I don't recommend it, though.
Places with zero bugs policies seem to have much branchier defect taxonomies than everyone else.
I've also seen shops just decide to ignore entire classes of problems so that they don't put too many rows in their bug database. So, willful ignorance is another option. I don't recommend it, though.
In a mature product that needs to live long and continuously being developed I just do bug first and features later. I promise that users hate features not working more then features missing. The general idea that we always need to release new features are so misplaced, the best version of Spotify was the original release, the product has not really gotten better with time and more features, the actual reason why 90% use it is just working worse and worse...
This is how I've run my 1-3 man internal corporate line of business development teams for ~20 years. I've always handled bugs first. The biggest side effect I think you'll notice is that you are much more conservative about your architecture and what new features you allow. When everything becomes a potential show-stopper as soon a bug is found, you're much less inclined to try dumb stuff.
> This led us to an interesting question: what if we just split all bugs into "will fix" and "won't fix"...
I feel like this already happens in some places, just implicitly rather than explicitly. There's a reason why people sometimes joke about the backlog being the place where lots of low priority things go to be forgotten about, and never be done. The problem with this approach is that even if you could get around to fixing low priority bugs in X months/years, with this approach you would prematurely toss them aside and decide to never fix them.
> ...and then prioritise every "will fix" above all new features....always. In other words: we commit to only ever adding new features when we're bug free.
This will decrease your ability to ship new features in a timely manner and will put you at a disadvantage if you're up against any sort of competition, that ships "good enough" things soon, versus you shipping "really good" software way later. The benefit of this approach is that it should lead to a more maintainable codebase in the long term, though how much this matters depends on your circumstances.
It will mostly depend on how you choose what goes under "will fix" and how much it matches what's necessary to keep the lights on (KTLO).
For example:
I once wrote some JWT code to allow multiple systems to integrate and communicate securely. I decided that I wouldn't ship it until I got something like 95% test coverage, enforced by CI and everything. It was doable and helped me discover a few bugs to fix during development, as well as refactor with confidence - but it took something like 4x more time than regular development would, which I can't see working well for the majority of projects out there, outside of core functionality and financial transaction related code.
I feel like this already happens in some places, just implicitly rather than explicitly. There's a reason why people sometimes joke about the backlog being the place where lots of low priority things go to be forgotten about, and never be done. The problem with this approach is that even if you could get around to fixing low priority bugs in X months/years, with this approach you would prematurely toss them aside and decide to never fix them.
> ...and then prioritise every "will fix" above all new features....always. In other words: we commit to only ever adding new features when we're bug free.
This will decrease your ability to ship new features in a timely manner and will put you at a disadvantage if you're up against any sort of competition, that ships "good enough" things soon, versus you shipping "really good" software way later. The benefit of this approach is that it should lead to a more maintainable codebase in the long term, though how much this matters depends on your circumstances.
It will mostly depend on how you choose what goes under "will fix" and how much it matches what's necessary to keep the lights on (KTLO).
For example:
- Users can't view a page that's needed for legal compliance, in some popular browsers? Needed for KTLO, fix it, no brainer.
- There are errors when trying to use some niche functionality, which affects around 1-5% of your userbase? Probably should fix it, it depends.
- Some button's logo is offset by a few pixels in a settings page or an info message flashes too quickly after some redirect? Probably nobody cares that much.
I once wrote some JWT code to allow multiple systems to integrate and communicate securely. I decided that I wouldn't ship it until I got something like 95% test coverage, enforced by CI and everything. It was doable and helped me discover a few bugs to fix during development, as well as refactor with confidence - but it took something like 4x more time than regular development would, which I can't see working well for the majority of projects out there, outside of core functionality and financial transaction related code.
John Romero spoke about the early principles of ID Software and one of them is to fix bugs once you find it without delay.
If you fix all problems before doing anything new, assuming that you (team, customers) can even agree a threshold of seriousness for 'bug', you will ossify. A reckless opposite of never bothering to fix bugs, and just being 'disruptive' for the sake of it, is also unworkable.
What do you mean by "ossify"? You think in practice there will always be bugs, so we will stop moving forwards?
Yes.
And in the extreme case one customer (or a competitor on the sly posing as one) decides that one minor (mis)feature that they don't like is a bug and doesn't accept any fixes to it as adequate, what do you do?
Overserialisation of life (and other extreme position taking) is unproductive. Some new things will happen while others get fixed or not. Sometimes you won't even know how to fix 'bugs' without trying new things out.
And in the extreme case one customer (or a competitor on the sly posing as one) decides that one minor (mis)feature that they don't like is a bug and doesn't accept any fixes to it as adequate, what do you do?
Overserialisation of life (and other extreme position taking) is unproductive. Some new things will happen while others get fixed or not. Sometimes you won't even know how to fix 'bugs' without trying new things out.
I'd love if the industry could have some sort of bugfix-only-month. Just one month per year where everyone goes off to fixing something, and new features are forbidden.
In fact, I'd love something like this so much, I'd even be willing to sponsor efforts like that with a bit of cash.
In fact, I'd love something like this so much, I'd even be willing to sponsor efforts like that with a bit of cash.
"Does this move the needle most?"
There is only so much dry powder. Every hour spent doing something that has lower bang:buck ratio in terms of "moving the needle" is an hour of powder burned. Make those hours count.
That said, correctly diving what moves the needle most is startup alchemy.
There is only so much dry powder. Every hour spent doing something that has lower bang:buck ratio in terms of "moving the needle" is an hour of powder burned. Make those hours count.
That said, correctly diving what moves the needle most is startup alchemy.
If my experience suggesting something like this to dev teams is any guide: the world will end, the business will spontaneously implode and burn to the ground, and everyone will die. I have not found a receptive audience for this idea.
It's harder to find new customers than it is to keep existing customers, especially if you have captured a lot of the "market". Fixing bugs to keep existing customers could be cheaper and keep you on a stable financial level.
You should not lose sight of why you have a business, you want to make money. I would suggest, do whatever makes you money in the short AND the long term. Don't worship ideals and chase fantasies.
If your company is a feature factory, it is not a good idea. If the business is stable and has a clear scope, you should be OK.
This is a blunt and unworkable approach. Stopping the shipping of any feature because some bug exists doesn't take what is important into account: the user experience.
A more refined approach is to triage the bugs into 4 or 5 levels of severity and then you might reasonably agree that at least all sev1 bugs must be fixed before new functionality is added.
Just using a bug triage categorisation of "will fix" or "won't fix" is not granular enough.
A more refined approach is to triage the bugs into 4 or 5 levels of severity and then you might reasonably agree that at least all sev1 bugs must be fixed before new functionality is added.
Just using a bug triage categorisation of "will fix" or "won't fix" is not granular enough.
I think the opposite is unworkable, namely just shipping new features and thus building on top of a murky foundation. I know a company that did this and basically needed to set up a taskforce and spend 2-3 months of the whole department's effort (around 50 people) on fixing a bug that that was low in the stack but they just kept building on top of it.
So in my experience, a correct approach is indeed to try to eradicate all known issues in every release cycle by running and completing the regression tests. Yes if that means slowing down a feature release by 2-3 days then that's the cost of having a bug-free application. I read somewhere that users don't care about features if they encounter bugs, they abandon the product rather easily.
>A more refined approach is to triage the bugs into 4 or 5 levels of severity and then you might reasonably agree that at least all sev1 bugs must be fixed before new functionality is added.
Of course, in practice this just means 3 or 4 levels of bugs that will never get fixed and languish in a backlog until the team / project gets re-orged and the entire backlog is wiped clean.
Okay, I'm joking. Sometimes those bugs become old enough they refer to features that no longer exist or can't be reproduced and then get marked as "won't fix"
Of course, in practice this just means 3 or 4 levels of bugs that will never get fixed and languish in a backlog until the team / project gets re-orged and the entire backlog is wiped clean.
Okay, I'm joking. Sometimes those bugs become old enough they refer to features that no longer exist or can't be reproduced and then get marked as "won't fix"
> Of course, in practice this just means 3 or 4 levels of bugs that will never get fixed and languish in a backlog until the team / project gets re-orged and the entire backlog is wiped clean.
I've never had much success getting a "won't fix" decision out of a PM or designer, and my time is too limited and valuable to spend debating it. If developers all know "low priority" means "never do" and "stakeholders" believe that their pet quibble will be fixed one day (even though they will never allocate time to do that) then work can continue in a sensible fashion.
I've never had much success getting a "won't fix" decision out of a PM or designer, and my time is too limited and valuable to spend debating it. If developers all know "low priority" means "never do" and "stakeholders" believe that their pet quibble will be fixed one day (even though they will never allocate time to do that) then work can continue in a sensible fashion.
So be it, if that's the case. At least it showed that a proper trade-off between features and bugs was made at the time. Remember that the triage can and should be rerun from time to time. Already open bugs could then be down- or upgraded in their severity as context evolves.
"in practice this just means 3 or 4 levels of bugs that will never get fixed" - this is so spot on! This is exactly what we were thinking: realistically, if we don't fix it now, it will never actually get fixed.
So, if it's important enough to fix, why not fix it now.
So, if it's important enough to fix, why not fix it now.
A tale of two companies (true story)...
I worked for a company a full year that always prioritized new features over bug fixes, or even QA. This was a sales-driven company. For various reasons, it was a toxic environment and I left.
The next company I worked for prioritized good engineering over shipping a product. This was an engineering-oriented company. It's funding was running low ('cause - well, no product!) and I got caught up in a layoff.
Company #1 - for 5 months of that year, I was the only QA person (with 13 devs). Literally no time to automate regression - there was always the "next feature we need to close a sale to customer 'X'". I told my boss that inevitably I (or my occasional comrades) would miss something and the company would be liable. Sure enough, within 18 months of my exit, they were hit with 3 different lawsuits.
Company #2 - absolutely delightful environment, except for the non-technical CEO. Understandably, he wanted to return value to the investors, but he kept making short-term decisions about direction that, while intending to make money, ended up accelerating the cash burn. I was promoted to be a technical lead of about 8 people on a side-project that was supposed to leverage technology from a company whose name is a homophone of 1 × 10¹⁰⁰. No surprise, mega-company changed the rules and our potential product was DOA. The entire team (including me) was let go.
TL;DR - quality is engineered in, and everyone owns quality. If a genuine defect exists, determine its severity and likelihood of emergence with customers to determine priority. Don't ship/push to production any code that will cause customers to lose data or their ability to run their businesses. If you have to make a choice, choose bug fixes over new features.
I worked for a company a full year that always prioritized new features over bug fixes, or even QA. This was a sales-driven company. For various reasons, it was a toxic environment and I left.
The next company I worked for prioritized good engineering over shipping a product. This was an engineering-oriented company. It's funding was running low ('cause - well, no product!) and I got caught up in a layoff.
Company #1 - for 5 months of that year, I was the only QA person (with 13 devs). Literally no time to automate regression - there was always the "next feature we need to close a sale to customer 'X'". I told my boss that inevitably I (or my occasional comrades) would miss something and the company would be liable. Sure enough, within 18 months of my exit, they were hit with 3 different lawsuits.
Company #2 - absolutely delightful environment, except for the non-technical CEO. Understandably, he wanted to return value to the investors, but he kept making short-term decisions about direction that, while intending to make money, ended up accelerating the cash burn. I was promoted to be a technical lead of about 8 people on a side-project that was supposed to leverage technology from a company whose name is a homophone of 1 × 10¹⁰⁰. No surprise, mega-company changed the rules and our potential product was DOA. The entire team (including me) was let go.
TL;DR - quality is engineered in, and everyone owns quality. If a genuine defect exists, determine its severity and likelihood of emergence with customers to determine priority. Don't ship/push to production any code that will cause customers to lose data or their ability to run their businesses. If you have to make a choice, choose bug fixes over new features.
[deleted]
Redefine "lack of feature" as "bug", problem solved!
Sometimes this is indeed the case. Our software talks with official systems, when they upgrade we have to support the new stuff, or our customers can't do what they need to do.
Of course we try to get there before the feature request turns into a bug...
Of course we try to get there before the feature request turns into a bug...
Loss of traction and bankruptcy, that’s what would happen.
zero known defects. it’s the only way to fly.
This led us to an interesting question: what if we just split all bugs into "will fix" and "won't fix", and then prioritise every "will fix" above all new features....always. In other words: we commit to only ever adding new features when we're bug free.
Has anybody tried this? Can it work?