Don't write bugs(teamten.com)
teamten.com
Don't write bugs
https://www.teamten.com/lawrence/programming/dont-write-bugs.html
112 comments
"Impractical to fix" can also mean writing an awful lot of complicated code to work around edge conditions that may never arise in reality. And the cost of writing that complicated code may result in additional bugs you didn't foresee due to the law of unintended consequences.
It becomes a tradeoff between the bug that you think you're smart enough to see but which may never happen, and the bugs that you're not able to see right now.
And I've definitely been on the wrong side of that equation and dealt with overly complicated code considering edge cases that were not useful, which caused complicated but practical bugs which affected many customers -- and I wound up replacing the overly thought out code with simpler code that was understandable and the "bug" I reintroduced never affected anyone AFAIK years later.
I've also been on the right side of that equation and picked simpler approaches and never had the overly complicated edge conditions that I could "see" in the code actually crop up and affect anything.
It isn't just political, sometimes you really can just technically outsmart yourself.
Plus there's the issue of if design decisions that are 10 years old should be refactored at the cost of introducing breaking changes and pain on everyone using your software. Sometimes you wind up writing some less-robust-than-ideal code which is due to those kinds of foundational issues. And the existence of a single difficult to fix bug is generally not sufficient to restort to blowing up your entire world and starting over again.
It becomes a tradeoff between the bug that you think you're smart enough to see but which may never happen, and the bugs that you're not able to see right now.
And I've definitely been on the wrong side of that equation and dealt with overly complicated code considering edge cases that were not useful, which caused complicated but practical bugs which affected many customers -- and I wound up replacing the overly thought out code with simpler code that was understandable and the "bug" I reintroduced never affected anyone AFAIK years later.
I've also been on the right side of that equation and picked simpler approaches and never had the overly complicated edge conditions that I could "see" in the code actually crop up and affect anything.
It isn't just political, sometimes you really can just technically outsmart yourself.
Plus there's the issue of if design decisions that are 10 years old should be refactored at the cost of introducing breaking changes and pain on everyone using your software. Sometimes you wind up writing some less-robust-than-ideal code which is due to those kinds of foundational issues. And the existence of a single difficult to fix bug is generally not sufficient to restort to blowing up your entire world and starting over again.
I worked with a developer who claimed, with a straight face, that he wrote bug-free code. When I pointed out places where the code would not behave as desired, his usual reaction was to assert that the input was invalid or would never happen, or that the problem was that some other service or code did the wrong thing.
So, while I understand your point about not "gold plating" code and being stuck solving problems of your own making, there really is a limit to how much code can pretend like failures don't happen and inputs can be other than what is expected. Failing to validate inputs from users is a well-known cause of many security problems, for example.
So, while I understand your point about not "gold plating" code and being stuck solving problems of your own making, there really is a limit to how much code can pretend like failures don't happen and inputs can be other than what is expected. Failing to validate inputs from users is a well-known cause of many security problems, for example.
You have greatly misunderstood my point. I'm pointing out that sometimes the cure can be worse than the disease.
Failing to validate inputs is fairly trivial and not what I'm talking about.
And I'm not talking about defending my own code as perfect or any kind of egocentric nonsense like that.
I'm talking about reviewing other people's code that I have a difficult time understanding due to a large amount of conditional logic for edge conditions which have never been reported as bugs before. I've accepted that code and then experienced it blowing up really horribly. It also just isn't clear that its better to solve bugs that nobody will ever hit at the cost to clarity and future maintenance and future bugs which people actually hit.
I'd rather people write "buggier" simpler code if the bugs that are mitigated aren't practical. And what I'm talking about here is the "undefined compiler behavior" grey areas of APIs. Those would of course be better never having been created in the first place and everyone should have done excellent design and input validation to start, but that never happens perfectly (because eventually shipping code takes precedence over perfection) and 10 years later you've got a job to do.
And the TL;DR there is that people can actually outsmart themselves trying to think of every edge condition under the sun, and I've watched that happen to other people and had to mop up the fallout.
Sometimes the edge condition will also simply take too long. Mitigating a bug that has been in the codebase for 10 years, has never been reported, and is unlikely to be reported in the next 10 years is not going to be terribly useful if it takes the next 3 months of your time because of how deeply buried in the design the bug is. That is a good one to make a note of and if it starts to surface as a problem you've then had time to think about strategies that might take it down to 1 month or less.
Please do validate all your inputs and write robust code against all the edge conditions you can think of when you're greenfielding though, it will make things easier later. At the same time if you argue that you're shipping flawlessly perfectly designed code I bet that's a lie.
Oh, I've also ripped out 60% of a codebase that was just horribly overdesigned as well, and was solving problems that the author wanted to teach themselves things that weren't actually relevant to the problemspace.
Failing to validate inputs is fairly trivial and not what I'm talking about.
And I'm not talking about defending my own code as perfect or any kind of egocentric nonsense like that.
I'm talking about reviewing other people's code that I have a difficult time understanding due to a large amount of conditional logic for edge conditions which have never been reported as bugs before. I've accepted that code and then experienced it blowing up really horribly. It also just isn't clear that its better to solve bugs that nobody will ever hit at the cost to clarity and future maintenance and future bugs which people actually hit.
I'd rather people write "buggier" simpler code if the bugs that are mitigated aren't practical. And what I'm talking about here is the "undefined compiler behavior" grey areas of APIs. Those would of course be better never having been created in the first place and everyone should have done excellent design and input validation to start, but that never happens perfectly (because eventually shipping code takes precedence over perfection) and 10 years later you've got a job to do.
And the TL;DR there is that people can actually outsmart themselves trying to think of every edge condition under the sun, and I've watched that happen to other people and had to mop up the fallout.
Sometimes the edge condition will also simply take too long. Mitigating a bug that has been in the codebase for 10 years, has never been reported, and is unlikely to be reported in the next 10 years is not going to be terribly useful if it takes the next 3 months of your time because of how deeply buried in the design the bug is. That is a good one to make a note of and if it starts to surface as a problem you've then had time to think about strategies that might take it down to 1 month or less.
Please do validate all your inputs and write robust code against all the edge conditions you can think of when you're greenfielding though, it will make things easier later. At the same time if you argue that you're shipping flawlessly perfectly designed code I bet that's a lie.
Oh, I've also ripped out 60% of a codebase that was just horribly overdesigned as well, and was solving problems that the author wanted to teach themselves things that weren't actually relevant to the problemspace.
+1. "Don't write bugs" is pretty much the 4Head response to a programmer asking "How do I not write buggy code?"
> if you're aware of ways that your code will fail in edge cases, handle those edge cases.
Another key point I want to add is that for every method you write, you should carefully audit the domain of your input and only accept information necessary for your method to work. That way you don't handle edge cases not relevant to your method. The exception probably here is critical edge cases that will SIGSEV, SIGILL, etc and kill your whole process. You usually want to check those - and if you are having to do that frequently that's code smell.
> if you're aware of ways that your code will fail in edge cases, handle those edge cases.
Another key point I want to add is that for every method you write, you should carefully audit the domain of your input and only accept information necessary for your method to work. That way you don't handle edge cases not relevant to your method. The exception probably here is critical edge cases that will SIGSEV, SIGILL, etc and kill your whole process. You usually want to check those - and if you are having to do that frequently that's code smell.
Aftee a decade as a professional developer, the key practice I've found that makes all the difference is this: make it very difficult for someone using my code to use it wrong. Because no matter how well I write my code, I'm just one developer in a much larger system.
An example: imagine I have an API that takes in a users tax id. In the US, let's presume that's an SSN. In Canada, a SIN. In Brazil, a CPF/CNPJ, and so on.
One option is to give a free form field that let's you say country id and tax ID. Simple, works for all countries, extensible later, no problem. Except that okay, in Brazil we also need a tax address. Fine, add an address field as well. But now when a user is using this API to submit Canadian tax numbers, there's a field for tax address. What's that? Do I need it? I guess I'll give the users home address? And a developer trying to submit Brazilian tax IDs sees that the address field is optional, so I guess I don't need that. ("Why didn't they read the docs that said it wasn't in this case?" Because no one does!).
I've made it possible to use this API incorrectly. I did it to save myself effort and blamed the other developer for using it wrong.
On the other hand, if my API had a different input type for each country's tax ID, with required fields saying they are, it becomes impossible to use it wrong. You can't have a bug, a mistake, because the type enforces correct usage. Go even further and have the constructor for the input type for that county explicitly require the fields it must have, and now you're really making it hard to do it wrong.
Types are powerful tools to prevent bugs. A bug caught sooner is less expensive to fix, and how much sooner can you get than "the moment you wrote it"?
An example: imagine I have an API that takes in a users tax id. In the US, let's presume that's an SSN. In Canada, a SIN. In Brazil, a CPF/CNPJ, and so on.
One option is to give a free form field that let's you say country id and tax ID. Simple, works for all countries, extensible later, no problem. Except that okay, in Brazil we also need a tax address. Fine, add an address field as well. But now when a user is using this API to submit Canadian tax numbers, there's a field for tax address. What's that? Do I need it? I guess I'll give the users home address? And a developer trying to submit Brazilian tax IDs sees that the address field is optional, so I guess I don't need that. ("Why didn't they read the docs that said it wasn't in this case?" Because no one does!).
I've made it possible to use this API incorrectly. I did it to save myself effort and blamed the other developer for using it wrong.
On the other hand, if my API had a different input type for each country's tax ID, with required fields saying they are, it becomes impossible to use it wrong. You can't have a bug, a mistake, because the type enforces correct usage. Go even further and have the constructor for the input type for that county explicitly require the fields it must have, and now you're really making it hard to do it wrong.
Types are powerful tools to prevent bugs. A bug caught sooner is less expensive to fix, and how much sooner can you get than "the moment you wrote it"?
> if my API had a different input type for each country's tax ID
Like you just add a new field for every country you support? I can imagine quite a few ways to break that.
For instance, my system has users from USA and CA, and I just send you everything in the SSN field, because I didn't consider the CA users when I wrote it. The simple way would have just worked, but now the API is broken for me.
Like you just add a new field for every country you support? I can imagine quite a few ways to break that.
For instance, my system has users from USA and CA, and I just send you everything in the SSN field, because I didn't consider the CA users when I wrote it. The simple way would have just worked, but now the API is broken for me.
No, add a new type. If your SSN is of type "USSocialSecurity" and the canadian system needs "CanadianTaxID" (sorry, I don't know what it is) then the API will not accept the wrong thing in the wrong place.
"But what if I label it incorrectly and send it along?" Then that's a bug in your code. No amount of my API design can prevent bugs in your code. All my API can do is not help you write bugs in your code. I can't stop you from reading the random number generator and claiming its output is a USSocialSecurity number, but at least I've done my best for a reviewer of your code to be able to look at that and ask you what you're doing calling that a USSocialSecurity.
"But what if I label it incorrectly and send it along?" Then that's a bug in your code. No amount of my API design can prevent bugs in your code. All my API can do is not help you write bugs in your code. I can't stop you from reading the random number generator and claiming its output is a USSocialSecurity number, but at least I've done my best for a reviewer of your code to be able to look at that and ask you what you're doing calling that a USSocialSecurity.
[deleted]
This seems like a good place to link the seminal work Falsehoods Programmers Believe About Time.
https://infiniteundo.com/post/25326999628/falsehoods-program...
https://infiniteundo.com/post/25326999628/falsehoods-program...
> On the other hand, if my API had a different input type for each country's tax ID,
Then you've significantly increased the work (and opportunity for bugs) required of a downstream programmer, and also made it significantly more likely that they won't support all countries, and of course they could also still send the wrong type...
Then you've significantly increased the work (and opportunity for bugs) required of a downstream programmer, and also made it significantly more likely that they won't support all countries, and of course they could also still send the wrong type...
Plus, this stuff likely has to go into a database. Now we have polymorphic OOP stuff needing to be connected to records and fields in a table.
Maybe just have a generic "taxblob" fields which is a serialized object which indicates its type and all the properties by name.
Maybe just have a generic "taxblob" fields which is a serialized object which indicates its type and all the properties by name.
The term "bug" is kind of overloaded at this point. It would be more useful to talk about different classes of software defects:
- Incomplete/ambiguous specification
- Unhandled IO errors
- Unbounded buffers/memory, unhandled OOM errors
- Defects introduced by state mutation
- Defects introduced by concurrency
- Lack of idempotence
- Lack of back-pressure
- Non-determinism
- API changes
- Low performance
Only some of these can "not be written", and only a few can "not be written" in actual code.
Most have to be solved at another layer (the implementation language/tools, the network, the database, the system architecture at large, design documents or team organisation).
- Incomplete/ambiguous specification
- Unhandled IO errors
- Unbounded buffers/memory, unhandled OOM errors
- Defects introduced by state mutation
- Defects introduced by concurrency
- Lack of idempotence
- Lack of back-pressure
- Non-determinism
- API changes
- Low performance
Only some of these can "not be written", and only a few can "not be written" in actual code.
Most have to be solved at another layer (the implementation language/tools, the network, the database, the system architecture at large, design documents or team organisation).
There has been for a long time the spark of a science of software quality that analyzes and categories software defects, how they are created, and how they are found, e.g. work by Boris Beizer in the past and more recent work by Cem Kaner. But, frankly, there isn't enough interest from either academia or industry to go much further than that and most undergraduate programs give software quality and testing at best a passing mention. Kind of ironic, considering how abundant bugs are these days.
Lack of back pressure? What kind of software defect is that? I've never heard of "back pressure" in a software context.
When a system is unable to serve any more requests it should signal back to the client (“backpressure”) instead of accepting requests unconditionally. Example: DDoS your own system by using unbounded queues. Something like blocking queue + timeout can help avoid it.
This is so funny. I can’t even.
It’s true that you can write code that has no bugs that would be relevant in a programming contest. And that you can avoid the kinds of bugs that you’d find if you read your code 2-3 times.
But if you write code that has to live for a long time, on a lot of devices, used by a lot of people, and that gets attacked by adversaries, then you really start to appreciate the inevitability of bugs even in code that seemed flawless.
It’s true that you can write code that has no bugs that would be relevant in a programming contest. And that you can avoid the kinds of bugs that you’d find if you read your code 2-3 times.
But if you write code that has to live for a long time, on a lot of devices, used by a lot of people, and that gets attacked by adversaries, then you really start to appreciate the inevitability of bugs even in code that seemed flawless.
I think that's pretty fatalistic. The way I see it, bugs are not inevitable, but a consequence of (often good) decisions that get made in a software project. Decisions by developers, managers, product owners, marketers lead to them. Decisions to adhere to a deadline, to add another feature, to focus on this and not that, to learn or not learn from previous mistakes. The fact that these decisions tend to be made in a certain way at a lot of companies makes it look like bugs are intrinsically inevitable in software.
Bug-free software can exist, but almost always doesn't because you have to ship, or because you don't have enough staff or time to fix them all, or because the requirements are not perfect, or any number of other perfectly valid reasons.
Bug-free software can exist, but almost always doesn't because you have to ship, or because you don't have enough staff or time to fix them all, or because the requirements are not perfect, or any number of other perfectly valid reasons.
> The way I see it, bugs are not inevitable
There's a lot of argument over just how many "facts" the average person can keep in their mind at any one time, but let's be extraordinarily generous and say you can keep track of twenty "things", simultaneously.
Now, let's say you need to connect to a URL, download an RSS file, process it, turn it into a list of podcasts, determine which ones haven't already been listened to or downloaded, and download them. Metadata will be persisted in SQLite. Binaries will be written to the file system. You need to do this on MacOS, iOS, Android, Windows, and Linux.
It would be trivial to come up with more than twenty "things" you need to keep track of to do this. The subtle differences between network connections on the various host OSes, the differences in the file system. Are you on SQLite 2 or 3, and what does that mean for the concurrency mechanism? What happens if the user asks you to update the list while you're in the middle of downloading an episode? What do you do when an episode appears in your database, but not the RSS feed? What happens when your download fails? What happens if the user shuts off their phone in the middle of a download?
Software is hard because software is massively, massively complex. There are a thousand things happening under the covers. Every single line of code we write is likely to have side effects, and we usually aren't even going to be sure what those side effects are.
That's why bugs are inevitable. It isn't out of laziness, though lazy coders do write buggier code. It's because we have built a teetering Jenga tower of abstractions, and we keep piling on top of it. This makes us much more productive, in the long run, but it also makes "perfect" code essentially impossible.
There's a lot of argument over just how many "facts" the average person can keep in their mind at any one time, but let's be extraordinarily generous and say you can keep track of twenty "things", simultaneously.
Now, let's say you need to connect to a URL, download an RSS file, process it, turn it into a list of podcasts, determine which ones haven't already been listened to or downloaded, and download them. Metadata will be persisted in SQLite. Binaries will be written to the file system. You need to do this on MacOS, iOS, Android, Windows, and Linux.
It would be trivial to come up with more than twenty "things" you need to keep track of to do this. The subtle differences between network connections on the various host OSes, the differences in the file system. Are you on SQLite 2 or 3, and what does that mean for the concurrency mechanism? What happens if the user asks you to update the list while you're in the middle of downloading an episode? What do you do when an episode appears in your database, but not the RSS feed? What happens when your download fails? What happens if the user shuts off their phone in the middle of a download?
Software is hard because software is massively, massively complex. There are a thousand things happening under the covers. Every single line of code we write is likely to have side effects, and we usually aren't even going to be sure what those side effects are.
That's why bugs are inevitable. It isn't out of laziness, though lazy coders do write buggier code. It's because we have built a teetering Jenga tower of abstractions, and we keep piling on top of it. This makes us much more productive, in the long run, but it also makes "perfect" code essentially impossible.
I have no idea what you mean by fatalistic, and I don't think you're in a position to predict my thought process.
Bugs exist because it is not possible for any one human or even organization of humans to know all of the ways in which the software is buggy. Worse, in the limit, there's not even a clear distinction between a bug and the absence of a feature.
If your software is in English, then that's a bug for someone who speaks Polish.
If your software has any linear time algorithm, then that's a bug for anyone whose input is large enough to cause that to algorithm to produce an unacceptable delay. What's unacceptable? Depends on the user.
If your software allocates memory, then eventually that memory allocation will fail because there wasn't enough memory, and that will be a bug. It'll be a bug even if your software reports the nicest possible error message. If you say, "sorry I couldn't execute your image filter because I ran out of memory", then that'll be a bug to the poor fellow who needed the image filter to finish executing.
If your software uses color to convey data, then that's a bug for someone who is color blind.
And so on!
And then there are security bugs. Security bugs exist, in part, because it's not possible for any mortal or organization of mortals to predict the entire decision tree that software of any meaningful complexity is executing. There will be some rotting branches in that tree, not because of ship dates or lack of trying, but because code that has enough control flow in it will have too many paths for anyone to reason about. Finding the one path that is broken is way easier than proving that no such path exists.
Some of my favorite bugs are the ones where everyone thought that the OS or CPU or compiler must do one thing, but they simply did not know that the thing that was expected is in no way guaranteed. Sometimes to discover the bug requires extreme acts of brilliance. Spectre is a fun example, and there will be more things like Spectre in the future (and I don't mean more timing side channels -- I mean there will be an entirely new thing, that isn't due to timing and isn't a side channel, that we will learn was wrong with computers all along).
And say that you try to solve this with formal methods. That will require a specification. I guarantee you will get the specification wrong, for all of the same reasons that I think you would have gotten the software wrong. That's not to say that formal methods aren't useful -- they're incredibly useful because they force you to state what the software is doing twice, in different forms, which hopefully enables you to catch bugs by detecting inconsistencies between the two statements about the software (the statement that is the spec an the statement that is the code). But then there will still be a bottomless well of bugs in those areas where the spec and the code both got it wrong. I guess Spectre was a case where everyone's specification of the CPU was wrong.
It's super unhelpful to frame the presence of bugs as an issue caused by the need to ship. That might be true for some bugs, but it misses the point. Bugs creep in any time code gets written. If you need to write code to fix a bug, you'll just introduce another bug by fixing the other one. So if ship dates were pushed back to infinity, then at infinity, you'd still have bugs.
Bugs exist because it is not possible for any one human or even organization of humans to know all of the ways in which the software is buggy. Worse, in the limit, there's not even a clear distinction between a bug and the absence of a feature.
If your software is in English, then that's a bug for someone who speaks Polish.
If your software has any linear time algorithm, then that's a bug for anyone whose input is large enough to cause that to algorithm to produce an unacceptable delay. What's unacceptable? Depends on the user.
If your software allocates memory, then eventually that memory allocation will fail because there wasn't enough memory, and that will be a bug. It'll be a bug even if your software reports the nicest possible error message. If you say, "sorry I couldn't execute your image filter because I ran out of memory", then that'll be a bug to the poor fellow who needed the image filter to finish executing.
If your software uses color to convey data, then that's a bug for someone who is color blind.
And so on!
And then there are security bugs. Security bugs exist, in part, because it's not possible for any mortal or organization of mortals to predict the entire decision tree that software of any meaningful complexity is executing. There will be some rotting branches in that tree, not because of ship dates or lack of trying, but because code that has enough control flow in it will have too many paths for anyone to reason about. Finding the one path that is broken is way easier than proving that no such path exists.
Some of my favorite bugs are the ones where everyone thought that the OS or CPU or compiler must do one thing, but they simply did not know that the thing that was expected is in no way guaranteed. Sometimes to discover the bug requires extreme acts of brilliance. Spectre is a fun example, and there will be more things like Spectre in the future (and I don't mean more timing side channels -- I mean there will be an entirely new thing, that isn't due to timing and isn't a side channel, that we will learn was wrong with computers all along).
And say that you try to solve this with formal methods. That will require a specification. I guarantee you will get the specification wrong, for all of the same reasons that I think you would have gotten the software wrong. That's not to say that formal methods aren't useful -- they're incredibly useful because they force you to state what the software is doing twice, in different forms, which hopefully enables you to catch bugs by detecting inconsistencies between the two statements about the software (the statement that is the spec an the statement that is the code). But then there will still be a bottomless well of bugs in those areas where the spec and the code both got it wrong. I guess Spectre was a case where everyone's specification of the CPU was wrong.
It's super unhelpful to frame the presence of bugs as an issue caused by the need to ship. That might be true for some bugs, but it misses the point. Bugs creep in any time code gets written. If you need to write code to fix a bug, you'll just introduce another bug by fixing the other one. So if ship dates were pushed back to infinity, then at infinity, you'd still have bugs.
Just saying "Don't write bugs" sounds a little but naive for me. I never saw someone that could achieve that task and nothing makes me believe that the author's statement might be remotely truthful.
My two cents: I can only write average software by writing tests simultaneously or right after writing the functionality. Reading the code thousands times without writing tests would just not work for me.
My two cents: I can only write average software by writing tests simultaneously or right after writing the functionality. Reading the code thousands times without writing tests would just not work for me.
A friend of mine was supervising students while doing his PhD. He gave them a task to write an implementation of binary search that contained no errors. He received twelve submissions.
If I am recalling the story right, he verified each submission in a proof assistant. Eight of the submissions contained errors. These were not first-year, green students writing binary search for the first time.
Empirical studies, few as they are, aren't forgiving either. It turns out that memory management, life times, concurrency, and ordering of computational effects are simply hard for the human mind to comprehend on the scale of non-trivial programs. We need better tools for thinking to help us manage. Otherwise we lean on heuristics and practices and assume some level of tolerance for errors to be made.
Don't write bugs, is not going to be very forgiving when even the most well-trained among us will, eventually, introduce an error into a program.
If I am recalling the story right, he verified each submission in a proof assistant. Eight of the submissions contained errors. These were not first-year, green students writing binary search for the first time.
Empirical studies, few as they are, aren't forgiving either. It turns out that memory management, life times, concurrency, and ordering of computational effects are simply hard for the human mind to comprehend on the scale of non-trivial programs. We need better tools for thinking to help us manage. Otherwise we lean on heuristics and practices and assume some level of tolerance for errors to be made.
Don't write bugs, is not going to be very forgiving when even the most well-trained among us will, eventually, introduce an error into a program.
Author has a lot of terrible advice on his site. Also this code is their recommend way to convert a double to float[1]:
public float doubleToFloat(double d) {
return (float) d;
}
I also think a lot of advice on that website is terrible, but there are some good bits there, too. For example while the general advice to never use bottom-up programming is awfully wrong, there is some value in the example they give later - the one showing it is good to have a working version of the system as early as possible, even if it has many gaps.
> After writing a few lines of code (3 to 6 lines, a short block within a function), re-read them. That habit will save you more time than any other simple change you can make.
Except maybe one: write a test which somehow exercises those 3 to 6 lines, guided by the cases that occur therein.
Except maybe one: write a test which somehow exercises those 3 to 6 lines, guided by the cases that occur therein.
> substantial programs (thousands of lines)
Well, yes, with incredibly tiny programs like that, it's possible to be bug free. With real world programs that often run to 100x-1,000x that size, you aren't going to completely avoid bugs just by thinking carefully about what you write.
Well, yes, with incredibly tiny programs like that, it's possible to be bug free. With real world programs that often run to 100x-1,000x that size, you aren't going to completely avoid bugs just by thinking carefully about what you write.
"Re-read your code frequently."
And who does not ?
"I was able to write substantial programs (thousands of lines) and run them without compile or run-time errors on the first try" LMAO :) I believe that :D
"I was able to write substantial programs (thousands of lines) and run them without compile or run-time errors on the first try" LMAO :) I believe that :D
What if the requirements for the program were wrong and you wrote it correct to specification. Is that a bug if product teams deem it so later? That would require later debugging.
Do lots of devs not re-read the code they write? Darn, I re-read my code probably 10 times.
Would Henry Ford tax programmers for the time they spent debugging?
Programmers can, managers don't always give the time
I thought there'd be something meatier than "don't write bugs" and "reread your code" but I was wrong.
I think the author put specific recommendations into their own sibling posts, accessible from this table of contents: https://www.teamten.com/lawrence/programming/
These are all actually good advice. Even "Avoid fields for communication between methods" although there are enough cases where it's better than the proposed alternative: when you have the method's implementation split into 10 private methods that need to pass 4-5 different pieces of data (for a total of 10-15 pieces) between each other, it's easier to just invent a private Context object, shove the data into its fields, attach those methods to it, and implement the public method as "return = new Context(...).doWork();".
I wish the author had expanded more. It would have been great to hear about the most frequent types of bugs they logged and other mitigation strategies.
Follow the C link: https://www.teamten.com/lawrence/style/
I also failed to find more details about the actual bugs, unless that is implicit in the style advice. Too bad, since actual detailed descriptions of bugs with reasoning about why they occured is usually quite interesting (and I'm always amazed by people who manage to keep notes about such).
The author says in the article that the style site is "unpublished", since the advice is too low-level and outdated. I actually think much of the very basic C advice is still sound and would agree with most of it, but it was kind of jarring when it jumped into "cute tricks", including Duff's device.
I have programmed in C for 25+ years, and have never used Duff's device in anger. While I could have used this guide in 1994 for the basic hints about good C, today I would be very sceptical reviewing code using Duff's device. :)
The author says in the article that the style site is "unpublished", since the advice is too low-level and outdated. I actually think much of the very basic C advice is still sound and would agree with most of it, but it was kind of jarring when it jumped into "cute tricks", including Duff's device.
I have programmed in C for 25+ years, and have never used Duff's device in anger. While I could have used this guide in 1994 for the basic hints about good C, today I would be very sceptical reviewing code using Duff's device. :)
Yeah sure, but what about when you need to add a new feature to an existing buggy code base? A lot of these articles assume you wrote all the code yourself from the ground up.
At first glance this may sound like advice for a baseball player - “don’t strike out” - but that’s exactly what good players do - they work at identifying the things that get them to strike out and try to minimize them. Tony Gwynn spent hours reviewing tapes of his at bats.
Keeping a log of your mistakes and analyzing them is helpful in any number of things, including coding. Can you not write any bugs? Probably not. But you can certainly reduce the number, and it’s worth taking the time to do so.
The “read what you wrote” advice is good - look for and notice bugs early so they’re easy to fix. Basically be your own pair programmer.
Keeping a log of your mistakes and analyzing them is helpful in any number of things, including coding. Can you not write any bugs? Probably not. But you can certainly reduce the number, and it’s worth taking the time to do so.
The “read what you wrote” advice is good - look for and notice bugs early so they’re easy to fix. Basically be your own pair programmer.
This is exactly like the classic fighting game advice "don't get hit"
It really annoys me that whenever I argue for "don't write bugs" as a quality practice, people immediately respond with all sorts of justifications why that's not possible. Of course it's not. But that doesn't mean we can't improve.
We spend time and money on CI, linting systems, sprint retros — so the allergy to "think more carefully about what you wrote" surprises me.
We spend time and money on CI, linting systems, sprint retros — so the allergy to "think more carefully about what you wrote" surprises me.
People are lazy and would rather pay out the ass for a way out of their problems than actually change. Becoming a better programmer is difficult. Spending money on CI is easy.
There's a big difference between "Analyse your mistakes so you're less likely to repeat them." and "Don't make mistakes."
The first sounds like sensible advice. The second sounds like a completely unrealistic thing that would only be said by someone who doesn't understand fallibility, or humans.
The first sounds like sensible advice. The second sounds like a completely unrealistic thing that would only be said by someone who doesn't understand fallibility, or humans.
"Don't write bugs" means opportunity cost and that's generally bad for business.
If bugs are business problems, figure out why and deal with the business problems. Maybe that means avoiding bugs but maybe otherwise changing the business makes more sense.
If bugs are business problems, figure out why and deal with the business problems. Maybe that means avoiding bugs but maybe otherwise changing the business makes more sense.
There is no allergy. The advice given is as basic as it gets. That’s what people already have been doing for years and decades. It’s comically insufficient. what else can be done? How do we deal with the reality of mistakes?
Bug journals, code reviews, prioritizing fixes, tooling support, enough sleep and enough time! If you give me time, then I come up with more and more failure modes, checks, analysis, improved design, better tests. Who pays for that?
Bug journals, code reviews, prioritizing fixes, tooling support, enough sleep and enough time! If you give me time, then I come up with more and more failure modes, checks, analysis, improved design, better tests. Who pays for that?
Maybe a slight rephrasing, "don't write stupid bugs"; there's a lot of small mistakes that people can make that can be solved by enforcing stricter rules, e.g. forcing the use of `===` in languages like Javascript and PHP to enforce the developer to think better about data types used.
Using linters, formatters, and IDE tools have taught me a lot of easily overlooked issues in those languages.
Using linters, formatters, and IDE tools have taught me a lot of easily overlooked issues in those languages.
"Don't write bugs" implies it is possible to write substantial code with no bugs, which isn't true (despite what this guy says).
You should say "try really hard not to write bugs" which is what you actually mean.
You should say "try really hard not to write bugs" which is what you actually mean.
I think it is the opposite. The reason we spend time and money on linting or CI is because we know people write bugs. If "don't write bugs" was an option we wouldn't need to spend that time and money. Good engineering practices should make systems that are resistant to human error.
There are also techniques that allow for less mistakes without more effort.
You can say "try harder to not cut your hand off with the table saw" or you can get a push stick.
You can say "try harder to not cut your hand off with the table saw" or you can get a push stick.
> My Junior year of college I joined the programming team.
School and the software industry are different beasts. Once the pressure of "real life" kicks in, your brain starts to work differently, often trading some of that academic brilliance for other things you need to survive.
Ironically, I have seen some of the best code quality at places with crappy salaries. These companies are aware that they cannot compete with the bigger fish, so they are not as prone to squeeze their developers for that extra cent.
School and the software industry are different beasts. Once the pressure of "real life" kicks in, your brain starts to work differently, often trading some of that academic brilliance for other things you need to survive.
Ironically, I have seen some of the best code quality at places with crappy salaries. These companies are aware that they cannot compete with the bigger fish, so they are not as prone to squeeze their developers for that extra cent.
Same experience here. Software quality outside pure software houses is often better, but the companies pay less. Sure, the scope is completely different in most cases.
A specific suggestion the author makes in a post related to the TFA:
> Everyone knows good programmers should plan
> for the future, should avoid boilerplate, should
> write elegant abstractions. Don’t. Those “good
> programmers” write heaps of garbage that get
> thrown away.I know it's not very much in vogue these days, but this is exactly what the Personal Software Process is all about, but in a very structured and rigorous format backed by data.
https://en.m.wikipedia.org/wiki/Personal_software_process
https://en.m.wikipedia.org/wiki/Personal_software_process
When I do competitive programming, I celebrate the "one and done" moments where my first compile and test against the samples works, and then the submission too yields Answer Correct. Even for problems I'd consider hard I can semi reliably get this. But I find it much rarer for this to happen in my work, probably because I have to work with unfamiliar, often poorly (if that) documented code. But even with well written code, I don't have the same level of internal context juggling when making changes as I do with freshly written contest code, which I think is the key difference.
Same thing here. With LeetCode or AdventOfCode problems its usually simple "write test submit celebrate". With real world problems its usually like: try to reproduce a vague customer crash/fail report, have several meetings on how to handle the new unexpected scenario, rewrite a portion of the code to support this use-case, tweak everything until unit tests pass, send it to QA for detailed testing, maybe repeat...
Real-world problems tend also not to have existing comprehensive test suites that you can just instantly verify your implementation with. Writing the tests is not only something that usually has to be done by the same individual or team writing the implementation, but also can suffer from the same pitfalls as writing the implementation (incorrect assumptions, missed cases, simply doing the wrong thing, etc.)
The test suites in competitive programming is a crutch, you aren't supposed to rely on them. Using them to guide your solution is like using the same data to train and test your machine learning model, if you do that then there is no point in practicing.
Thanks for the advice, I will consider not writing bugs.
And the best part about not writing bugs is that you then don't have to write tests either.
How often do we get to write green fields ground up code? Of course when doing that we try to architect stuff with no bugs. 80% of the job is refactoring old code, updating code that many other files depend on, updating/replacing dependencies and more. It is great advice if you just write C code in a silo, but the complexity of a modern app is beyond just re-reading stuff.
From what I think is the article's punch line, in context of software bugs:
"We put them there, and we can decide to not put them there"
I don't think this is strictly true. Sure from a strict computational theory or mathematical proof perspective we can 'produce no bugs' but from a meat-space reality standpoint programming ecosystems do not work this way.
The game changes when you need to ship some bits from one place to another, or demand contract on the representation of truths over time, or guarantee no interruption of service on vague notions of "indefinitely".
How perfect can any one individual execute the exacting theory of the stack: ASM(s), C-derivatives, OS Kernels, TCP/UDP, DNS, TLS, pythons/javascripts/golangs, HTTP, SQL-derivatives, libraries, frameworks, cluster orchestrators, oh my the list just keeps getting bigger each and every year...
So we write bugs, because we have to. Imperfect knowledge is part of our professional practice. Much like the sciences our bugs persist not because they were "wrong" but because we build on top of them as new facts emerge.
From the articles principle advice:
"If you want a single piece of advice to reduce your bug count, it’s this: Re-read your code frequently. After writing a few lines of code (3 to 6 lines, a short block within a function), re-read them. That habit will save you more time than any other simple change you can make."
We do this. Programmers can tend to be narcissistic, we love reading our code we just wrote: oh the beauty, oh the wonderful shortcut, oh the performance, all while enjoying the quickly fading context in which we wrote it. If our knowledge was wrong in constructing the code, assumptions are still wrong while reviewing it. But in practice we run code reviews, automated compilation/linters/tests/etc to ensure that the quality of our code is not left to the individual programmer who is quick to forget that it was not for them in the first place. Bugs have a harder time surviving multiple perspectives (generally).
Now back to searching for this years bash incantations :)
"We put them there, and we can decide to not put them there"
I don't think this is strictly true. Sure from a strict computational theory or mathematical proof perspective we can 'produce no bugs' but from a meat-space reality standpoint programming ecosystems do not work this way.
The game changes when you need to ship some bits from one place to another, or demand contract on the representation of truths over time, or guarantee no interruption of service on vague notions of "indefinitely".
How perfect can any one individual execute the exacting theory of the stack: ASM(s), C-derivatives, OS Kernels, TCP/UDP, DNS, TLS, pythons/javascripts/golangs, HTTP, SQL-derivatives, libraries, frameworks, cluster orchestrators, oh my the list just keeps getting bigger each and every year...
So we write bugs, because we have to. Imperfect knowledge is part of our professional practice. Much like the sciences our bugs persist not because they were "wrong" but because we build on top of them as new facts emerge.
From the articles principle advice:
"If you want a single piece of advice to reduce your bug count, it’s this: Re-read your code frequently. After writing a few lines of code (3 to 6 lines, a short block within a function), re-read them. That habit will save you more time than any other simple change you can make."
We do this. Programmers can tend to be narcissistic, we love reading our code we just wrote: oh the beauty, oh the wonderful shortcut, oh the performance, all while enjoying the quickly fading context in which we wrote it. If our knowledge was wrong in constructing the code, assumptions are still wrong while reviewing it. But in practice we run code reviews, automated compilation/linters/tests/etc to ensure that the quality of our code is not left to the individual programmer who is quick to forget that it was not for them in the first place. Bugs have a harder time surviving multiple perspectives (generally).
Now back to searching for this years bash incantations :)
See the work of Margaret Hamilton, coiner of the term "software engineering" and developer of software for the Apollo mission.
> In 1976, Hamilton co-founded with Saydean Zeldin a company called Higher Order Software (HOS)[46] to further develop ideas about error prevention and fault tolerance emerging from their experience at MIT working on the Apollo program.
https://en.wikipedia.org/wiki/Margaret_Hamilton_%28scientist...
Crapped on by Dijkstra himself: https://www.cs.utexas.edu/users/EWD/ewd08xx/EWD852.PDF
Written up by James Martin (without proper attribution in my opinion) in "System design from provably correct constructs : the beginnings of true software engineering": https://archive.org/details/systemdesignfrom00mart/mode/2up
See also: https://en.wikipedia.org/wiki/Universal_Systems_Language
> In 1976, Hamilton co-founded with Saydean Zeldin a company called Higher Order Software (HOS)[46] to further develop ideas about error prevention and fault tolerance emerging from their experience at MIT working on the Apollo program.
https://en.wikipedia.org/wiki/Margaret_Hamilton_%28scientist...
Crapped on by Dijkstra himself: https://www.cs.utexas.edu/users/EWD/ewd08xx/EWD852.PDF
Written up by James Martin (without proper attribution in my opinion) in "System design from provably correct constructs : the beginnings of true software engineering": https://archive.org/details/systemdesignfrom00mart/mode/2up
See also: https://en.wikipedia.org/wiki/Universal_Systems_Language
Use tools that make it harder. Immutable datastructures, side effects constrained to a few places, good static typing etc.
Where I worked before we had 100k LoC Elm projects with virtually no bugs.
Now I'm working in a python/django codebase with probably thousands of small hidden bugs.
Where I worked before we had 100k LoC Elm projects with virtually no bugs.
Now I'm working in a python/django codebase with probably thousands of small hidden bugs.
I think that's what the book and guidelines mentioned in the article are about. Best practices and design patterns for software development. For C, I can imagine the author promoting the use of Yoda conditions to prevent one category of bugs.
Overall, strictness and "anal" rules is how you can reduce the amount of common bugs - as long as they don't require too much active thought. Then you can focus on the domain, on the actual problem, and bugs on that level will be from your own limited understanding of said domain and logic, not the programming language, memory management, memory leaks, nil dereferences, etc. Those categories of bugs are solved problems, with either best practices, tooling / linters, analyzers, or newer and better programming languages.
Overall, strictness and "anal" rules is how you can reduce the amount of common bugs - as long as they don't require too much active thought. Then you can focus on the domain, on the actual problem, and bugs on that level will be from your own limited understanding of said domain and logic, not the programming language, memory management, memory leaks, nil dereferences, etc. Those categories of bugs are solved problems, with either best practices, tooling / linters, analyzers, or newer and better programming languages.
Use tools that make it harder. Immutable datastructures, side effects constrained to a few places, good static typing etc.
Those are good things, but they only eliminate the class of bugs where the code is wrong. Some bugs, and most important bugs, are problems where you've written entirely valid and working code that does the wrong thing. That is where going back and reading your code again, and having layers of review from your peers, will help most.
Those are good things, but they only eliminate the class of bugs where the code is wrong. Some bugs, and most important bugs, are problems where you've written entirely valid and working code that does the wrong thing. That is where going back and reading your code again, and having layers of review from your peers, will help most.
Yes, but eliminating those classes allow you to spend more time where it matters.
One thing that helps me a lot is: fail fast.
You program must break when something is not right because shitty lasts forever.
Immutable objects are great but you need to validate them when you construct them and throw an exception when something is not right. And functions should return what you would expect.
For example, this:
You program must break when something is not right because shitty lasts forever.
Immutable objects are great but you need to validate them when you construct them and throw an exception when something is not right. And functions should return what you would expect.
For example, this:
function GetEntitryById(int id):Entity
{
var entity = [action to get an entity];
if(entity == null) {
throw new Exception("Entity not found");
}
return entity;
}
function GetEntitryById(int id):Entity?
{
var entity = [action to get an entity];
return entity;
}Or use Option/Maybe or Either to represent this and let callers explicitly handle this rather than catching an exception. Is an entity not being found truly an exceptional case?
In the case that you're providing an explicit id as an argument, yes. If it were a query, no. It's dangerous to use an empty value for all failed lookups, as you can't differentiate between "not found because it doesn't exist" versus "not found because the database went offline for a minute".
If it's a bug, i.e a error made by the programmer then assert and dump core.
I think that code example depends on the type system. In elm, the compiler forces you to handle both cases if you return a Maybe. In Java everything can be null and blow up which is bad. In Kotlin you have to handle the possible null, but it's a bit brittle when calling into Java code.
But yeah, better to fail fast. That's why I want tools to catch stuff as I write the code, not when I run it, as I cannot run all cases so then it won't hit until prod.
But yeah, better to fail fast. That's why I want tools to catch stuff as I write the code, not when I run it, as I cannot run all cases so then it won't hit until prod.
This doesn't seem to take into account different codebases. At work I may have a decade old piece of code that has been partially rewritten multiple times and gets constant updates - it's also communicating with the outside world in many ways. The skill required to manage code like this without any bugs or even compiler errors is immense. But writing a 1000 line brand new project from scratch in a sane language is probably doable if you invest enough time - remember no compiler errors are permitted, so you have to write this in a basic text editor, then run the compiler and get 0 errors and bugfree software.
I can't stop picking on the 0 compiler errors thing. I could write brainf*ck and get 0 compiler errors, but no guarantees on the buginess of the code. I could also write Rust and the compiler throws errors when an assumption of mine was wrong, or I was trying to do something that the compiler does not yet support, or the codebase was large enough that all of it didn't fit into my mindmap. Not sure how to fully mitigate those causes.
I can't stop picking on the 0 compiler errors thing. I could write brainf*ck and get 0 compiler errors, but no guarantees on the buginess of the code. I could also write Rust and the compiler throws errors when an assumption of mine was wrong, or I was trying to do something that the compiler does not yet support, or the codebase was large enough that all of it didn't fit into my mindmap. Not sure how to fully mitigate those causes.
Compile time errors are cheap and it would be silly to optimize for that. IDE just tells you: hey, this line, you forgot something. It is a few seconds to fix. It doesn't and CAN'T go to production. It doesn't need debugging. And, if 1 data point can tell something, it comes naturally - the more I code, the less (if any) compile time errors I get. But if I do, it takes seconds to fix them.
Of course IDE and intellisense goes a long way to prevent compile time errors - it just tells you on-the-fly how not to make a mistake.
Untyped languages could be another story, as "compile-less time" is actual runtime bug. But I don't know - I work primarily with C#.
Of course IDE and intellisense goes a long way to prevent compile time errors - it just tells you on-the-fly how not to make a mistake.
Untyped languages could be another story, as "compile-less time" is actual runtime bug. But I don't know - I work primarily with C#.
The linked article literally says “write code that compiles successfully from the first time without bugs”.
I do have some issue with that one, because it kinda implies that compiling is a big, expensive and final job. But with current-day languages and tooling, you can effectively run trial compiles constantly while writing the code. The sooner you spot an issue, the sooner it's fixed and the less painful it is.
I take the performance hit and prefer my editor to have syntax highlighting, syntax checking, linters and (at least at some point in my career) relevant unit tests automatically running in the background on save.
I take the performance hit and prefer my editor to have syntax highlighting, syntax checking, linters and (at least at some point in my career) relevant unit tests automatically running in the background on save.
It is about deliberate practice. Being able to write code that compiles successfully without needing to test against the compiler is mostly the same skill as writing code that does what you intend it to without need for debugging. However practicing writing code that compiles the first time is a ton easier than practicing writing code that has no other bugs the first time, so you go practice that. It isn't perfect, but it is low hanging fruit that takes a few days to learn so why not.
Or another perspective, the compiler throwing errors at you for your code implies that your intuitive mental model for the language isn't 100% accurate. If you write code the compiler accepts then you most likely have a near perfect understanding of how the language works.
Or another perspective, the compiler throwing errors at you for your code implies that your intuitive mental model for the language isn't 100% accurate. If you write code the compiler accepts then you most likely have a near perfect understanding of how the language works.
You have control over the code you write yourself, but you rarely have control over what code others write - even if your code review process is incredibly rigorous, there's rarely resources to check every single line of code that people write. And his becomes more infeasible, as size and complexity grows.
Furthermore, rarely can commercial products afford the luxury of such exhaustive protocols. Even slow moving, state-funded projects where safety is paramount, and thus - and the devs. basically have years to develop something, without the imminent pressure of showing a fully working product, you'll see bugs appear all over the place.
But I agree with others - there are definitive ways to minimize the problems from the start. Using right tools, enforce the right practices, etc. But with that said, bug-free coding becomes more unfeasible as MLOC, people involved, and new third party tools introduced increases.
Furthermore, rarely can commercial products afford the luxury of such exhaustive protocols. Even slow moving, state-funded projects where safety is paramount, and thus - and the devs. basically have years to develop something, without the imminent pressure of showing a fully working product, you'll see bugs appear all over the place.
But I agree with others - there are definitive ways to minimize the problems from the start. Using right tools, enforce the right practices, etc. But with that said, bug-free coding becomes more unfeasible as MLOC, people involved, and new third party tools introduced increases.
There's a big category of issues and bugs that can be prevented by tooling and choice of language though, so make sure that those are in place at the start of a project. Shortlist: Typed language, linters, formatters, and set up your CI so that 'fast forward' on your main branch is off and anything is only merged after a rebase on top of main, a code review, and a green CI pipeline.
In theory, the only bugs that make it in production then are logic or requirements bugs. Having an extra step somewhere of a manual test might help mitigate those.
In theory, the only bugs that make it in production then are logic or requirements bugs. Having an extra step somewhere of a manual test might help mitigate those.
Probably it's true for type of bugs which could be catched by some kind of static analysis tooling (using uninitialized variables and etc.). You need just to remember some set of rules and never do "bad things".
But what about bugs produced by unexpected inputs, system misconfigurations and so on
[deleted]
We should offload this responsibility to our tools as much as we possibly can.
https://blog.codinghorror.com/falling-into-the-pit-of-succes...
https://blog.codinghorror.com/falling-into-the-pit-of-succes...
When writing code you have a mental model of the machine, and the language and libraries. Back in the day those mental models could be close enough to reality that writing bug free code could be a choice. It would take time but you can essentially run the program in your head.
This is an old timer, back when machines and abstractions were simpler.
The post makes sense in that context, but today you can't have a complete mental model of the thing you're coding. It's impossible. So bugs.
Maybe we need to simplify.
This is an old timer, back when machines and abstractions were simpler.
The post makes sense in that context, but today you can't have a complete mental model of the thing you're coding. It's impossible. So bugs.
Maybe we need to simplify.
But we HAVE simplified. I can now type 'fetch this data matching these conditions sorted by this' and be confident about not having any bugs in my code, while underneath there's millions of lines and decades of other people's work and solved bugs being run.
"Back in the day" they would spend weeks devising that system from scratch, hand-writing sorting algorithms, etc.
The mental model of software development is about the problem and domain nowadays, not the low level implementation details. Your job is to solve a business problem, instead of figuring out how to manipulate a CPU and its memory banks to do your bidding.
"Back in the day" they would spend weeks devising that system from scratch, hand-writing sorting algorithms, etc.
The mental model of software development is about the problem and domain nowadays, not the low level implementation details. Your job is to solve a business problem, instead of figuring out how to manipulate a CPU and its memory banks to do your bidding.
We have libraries that solve many problems for us.
But those libraries often have parameters, switches, hints, and secret knowledge of the type "do not solve problem X by doing Y because although it seems okay on the surface, the implementational details will increase the complexity from O(N) to O(N^2); use this trick instead".
So we use libraries to solve the problems, but we also need years of experience using those libraries, or we easily create problems we didn't expect. Often there are many alternative libraries for the same task, and the library-specific knowledge becomes obsolete in later version.
But those libraries often have parameters, switches, hints, and secret knowledge of the type "do not solve problem X by doing Y because although it seems okay on the surface, the implementational details will increase the complexity from O(N) to O(N^2); use this trick instead".
So we use libraries to solve the problems, but we also need years of experience using those libraries, or we easily create problems we didn't expect. Often there are many alternative libraries for the same task, and the library-specific knowledge becomes obsolete in later version.
That's not simplifying, it's the opposite. We've made it easier by adding more complexity.
You absolutely can have a complete mental model of the thing you're coding. Especially if you properly break it down into pieces (modules, libraries, functions, whatever you want to call them) which can be understood as a unit.
The biggest problem is that many people use libraries as a crutch, without any knowledge of how they work or what they actually do.
The biggest problem is that many people use libraries as a crutch, without any knowledge of how they work or what they actually do.
define "complete".
Because I bet you fucking money you aren't modeling the actual behavior of any modern x86_64 cpu in your head.
Because I bet you fucking money you aren't modeling the actual behavior of any modern x86_64 cpu in your head.
You're not coding any modern x86_64 cpu. Your program might need to work around hardware bugs but those bugs are not things you wrote.
Of course, I also never said x86_64 was a good structure. A bit of reading comprehension might even reveal the implication that simple is better.
Of course, I also never said x86_64 was a good structure. A bit of reading comprehension might even reveal the implication that simple is better.
You can get that same effect today, just work in the same stack with the same libraries for years and you will learn exactly how everything works. Just need a stack that doesn't do breaking changes that often.
If programming worked like the more manual jobs like plumbing then you'd work 5 years on a stack before they'd call you proficient at it. But nowadays you work 2 years before changing jobs to another stack, and after 5 you are expected to manage people and no longer write code. So the problem is mostly organizational and not technical.
If programming worked like the more manual jobs like plumbing then you'd work 5 years on a stack before they'd call you proficient at it. But nowadays you work 2 years before changing jobs to another stack, and after 5 you are expected to manage people and no longer write code. So the problem is mostly organizational and not technical.
It's not just your tech stack in the narrow sense, your software also can't talk to anything that changes: no browser, no third party service. Systems are rarely that isolated anymore.
Ah, right, I haven't had a regular webdev job. But then that is because your job is to glue together different services rather than implement a complex chunk of code. Most of the jobs I've had were me implementing a lot of low level complex chunks or doing the architecture to make those chunks easy to write, for example when writing the runtime of a ML framework.
So it isn't that the time changed, the old jobs where you code still exists. But we added millions of glue code jobs on top of that, and that changed how people view software development. But it is field specific, even if webdev is the most common you can still work in any of the many other areas where the rules are different.
Edit: And yes, gluing together components is ridiculously hard. It is just hard in a different way, your job then becomes to learn about new things as quickly as possible so you can properly glue them together rather than trying to think about what code to write. That isn't easy at all.
So it isn't that the time changed, the old jobs where you code still exists. But we added millions of glue code jobs on top of that, and that changed how people view software development. But it is field specific, even if webdev is the most common you can still work in any of the many other areas where the rules are different.
Edit: And yes, gluing together components is ridiculously hard. It is just hard in a different way, your job then becomes to learn about new things as quickly as possible so you can properly glue them together rather than trying to think about what code to write. That isn't easy at all.
The kind of bugs I write these days are quite different from the ones I made decades ago. My mistakes these days are from high level misunderstandings, rather than low level ones.
I have a problem with this way of thinking.
Yes, you can actually focus on not writing any bugs, but should you?
We have limited attention and mental resources.
If you're fixing your car, should you focus on not getting any oil on yourself? Yes, you take precautions but that should not be your main focus.
Yes, you can actually focus on not writing any bugs, but should you?
We have limited attention and mental resources.
If you're fixing your car, should you focus on not getting any oil on yourself? Yes, you take precautions but that should not be your main focus.
In pro kitchens, the chefs DO spend time and effort NOT getting spills on themselves, on cleanjng their surfaces often, on tidying up as they go. This keeps the kitchen running smoothly and the food safer and more presentable. The process allows their creativity with flavour to come to the fore.
Your analogy has flaws. If you get oil on yourself, you didn't break anything in the car. If you fix your car but use a wrong part or install it improperly, that's a bug and that's on you for being careless.
I like writing bugs. You learn a new thing every time you fix one of them.
During the early part of my career I decided to just not write bugs. Was the best decision I made, I've saved myself countless hours
Not sure if you are serious or not, but I agree with that statement. Of course you cannot write completely bug free code, but practicing competitive programming until I could write complex 100 line programs bug free in 10 minutes is probably the most well spent time I've ever done in life.
Being able to write big bug free chunks means that if I think of something I want to do, then write the 500 line implementation without even compiling or running the code it tend to work the first time I run it. Of course that code isn't production ready, but it helps immensely in writing prototypes and mapping out problem spaces. So when another person does their thing and implement 1 solution, I've written 10 different solutions testing a lot of different architectures and picked the best one.
Edit: I think the biggest win from doing this is that although writing code is the easy and effortless part, debugging your code afterwards is extremely mentally draining and takes away mental energy you could be using to think about architecture etc. Think like this, how much high quality thinking have you wasted on debugging your code? Now imagine if you with a little bit of deliberate practice could eliminate most of that waste, wouldn't you do it?
Being able to write big bug free chunks means that if I think of something I want to do, then write the 500 line implementation without even compiling or running the code it tend to work the first time I run it. Of course that code isn't production ready, but it helps immensely in writing prototypes and mapping out problem spaces. So when another person does their thing and implement 1 solution, I've written 10 different solutions testing a lot of different architectures and picked the best one.
Edit: I think the biggest win from doing this is that although writing code is the easy and effortless part, debugging your code afterwards is extremely mentally draining and takes away mental energy you could be using to think about architecture etc. Think like this, how much high quality thinking have you wasted on debugging your code? Now imagine if you with a little bit of deliberate practice could eliminate most of that waste, wouldn't you do it?
Most of my bugs are the result of the wrong punctuation. Get a period instead of a comma after that array element, it's head-smacking and squinting time again.
Written human language is much more forgiving. Just periods and commas are enough most of the time. But with computer languages, it's most of the keys on the \*&#|}|"|{:"?><,./#$(! keyboard.
Written human language is much more forgiving. Just periods and commas are enough most of the time. But with computer languages, it's most of the keys on the \*&#|}|"|{:"?><,./#$(! keyboard.
This is like telling a poor person: Just earn a million dollars!
Then you can try with an smoker: Just don't smoke!
Finally you can tell an obese person to loose weight.
They are such high level statements that it is completely useless advice without implementation details.
Neither me or anyone on my team write significant bugs. We spend a very small part of our time dealing with bugs. But for this lots of knowledge, discipline and practice is needed.
You can fill several books just with that knowledge alone.
Rereading your code for me is completely useless. When I was a kid I reread code for days without finding the bug because I just make the same (wrong and most of the time subconscious) assumption over and over again. Then I learn better strategies.
A simple tool that checks for wrong assumptions finds those bugs in seconds.
You can log your bugs so you identify recurring patterns and can correct those patterns.
With just a team of three, I wrote a tool that gave an error for any "one line if" without parenthesis after fixing the nth time the recurring bug: People will write a one liner if without parenthesis and then a year from that will add an additional line to the if without adding a parenthesis because they were just so focused in the new problem they just could not see anything wrong while rereading the indented line.
We even use statistical Bayesian analysis of code that automatically highlights any code that goes against the style of her author. In that code she is not experienced and could make the wrong assumptions so must be careful.
Asserts everywhere in developer code that are automatically eliminated in production code(but remain in developer). We assert everything because bugs just pop up from the asserts again in seconds.
Also incremental tests between revisions is extremely useful. Your test work flawlessly in last revision, you changed 20 lines. It doesn't work, the error is in those lines. The smaller the increment, the easiest to find the problem.
Those things are obvious for experienced programmers, but for other people could be magic and sorcery, because there are many details needed to make it work.
Then you can try with an smoker: Just don't smoke!
Finally you can tell an obese person to loose weight.
They are such high level statements that it is completely useless advice without implementation details.
Neither me or anyone on my team write significant bugs. We spend a very small part of our time dealing with bugs. But for this lots of knowledge, discipline and practice is needed.
You can fill several books just with that knowledge alone.
Rereading your code for me is completely useless. When I was a kid I reread code for days without finding the bug because I just make the same (wrong and most of the time subconscious) assumption over and over again. Then I learn better strategies.
A simple tool that checks for wrong assumptions finds those bugs in seconds.
You can log your bugs so you identify recurring patterns and can correct those patterns.
With just a team of three, I wrote a tool that gave an error for any "one line if" without parenthesis after fixing the nth time the recurring bug: People will write a one liner if without parenthesis and then a year from that will add an additional line to the if without adding a parenthesis because they were just so focused in the new problem they just could not see anything wrong while rereading the indented line.
We even use statistical Bayesian analysis of code that automatically highlights any code that goes against the style of her author. In that code she is not experienced and could make the wrong assumptions so must be careful.
Asserts everywhere in developer code that are automatically eliminated in production code(but remain in developer). We assert everything because bugs just pop up from the asserts again in seconds.
Also incremental tests between revisions is extremely useful. Your test work flawlessly in last revision, you changed 20 lines. It doesn't work, the error is in those lines. The smaller the increment, the easiest to find the problem.
Those things are obvious for experienced programmers, but for other people could be magic and sorcery, because there are many details needed to make it work.
Firstly, I stopped taking this article seriously at "modern language (Java)". Apart from that, "we can decide to not put bugs in the code" is the most non-sensical thing I've ever read. OK, sure, you can eventually flush some or most bugs out before your code goes in for proper testing from formally-trained QA people, but unless all you're writing is "Hello World" applications or walking skeletons then writing bug-free code is not possible, hence the existance of an entire industry that revolves around finding bugs. Just because YOU can't find any more bugs in your own code, that doesn't mean there aren't any bugs in the code, which is why QA people are trained to employ methodical test techniques. This is commonly known as the "Absence-Of-Errors Fallacy".
Secondly, there's this concept called "Independence Of Testing", which essentially states that the more independent the tester is from the code, the more likely they are to find defects. The opposite of that is also true, the less independend the tester is, e.g. if the tester is the developer, then the less likely they are to find defects in their own code. Read mode here (section 5.1.1): https://www.istqb.org/downloads/send/2-foundation-level-docu...
Lastly, this article is essentially saying that proper testing by QA people is useless, which not only I strongly disagree with but also is something that a terrible developer would say.
Secondly, there's this concept called "Independence Of Testing", which essentially states that the more independent the tester is from the code, the more likely they are to find defects. The opposite of that is also true, the less independend the tester is, e.g. if the tester is the developer, then the less likely they are to find defects in their own code. Read mode here (section 5.1.1): https://www.istqb.org/downloads/send/2-foundation-level-docu...
Lastly, this article is essentially saying that proper testing by QA people is useless, which not only I strongly disagree with but also is something that a terrible developer would say.
In the grand scheme of things Java is a modern language although I dislike how generations of langauges are defined.
But the innovation Java brought is larger than what younger languages bring to the table. You can still dislike the language of course.
But the innovation Java brought is larger than what younger languages bring to the table. You can still dislike the language of course.
One of the great ways of avoiding bugs is to make invalid states unrepresentable. An easy win is to use sum types for the various different flavours of a valid input parameter. The verbosity of having to manually create classes, wrap/unwrap rather than defining at-hoc in-place, and pattern matching leaves it in still the not modern camp, among other things.
On making illegal states unrepresentable - this was my favorite post on that topic: https://fsharpforfunandprofit.com/posts/designing-with-types... (I don’t even write F# myself)
>, so we practiced writing bug-free code. After two years of practicing, and after my previous two years of log-keeping, I was able to write substantial programs (thousands of lines) and run them without compile or run-time errors on the first try. [...] the exercise convinced me that bugs don’t creep into our code by themselves. We put them there, and we can decide to not put them there.
One logic flaw in that self-reported assessment is that us readers don't really know if his 1000-line programs were truly proven to be bug free. But we can still set that aside and entertain the idea that just deciding to not write bugs actually prevents bugs. Consider various disciplines:
- programming: Some search/sort code that had a hidden bug for 20+ years. Both computer experts Jon Bentley at Carnegie Mellon Uni and Joshua Bloch (Java expert) created the same bug: https://ai.googleblog.com/2006/06/extra-extra-read-all-about...
- mathematics: Andrew Wiles 1st attempt at proving Fermat's Last Theorem had a flaw that was discovered by his peer reviewers. And the authors for a SAT math exam unwittingly wrote a bug: https://youtu.be/kN3AOMrnEUs?t=2m05s
- laws : legislators write laws with attempts at precise wording but it still has unintended loopholes or perverse incentives they never foresaw
Did all those people "decide to write bugs" therefore they can decide not to? Of course not. You can reduce bugs by following some best practices (e.g. use well-tested encryption library instead of rolling your own, etc) but bugs will still happen no matter how much you "decide" against it.
Put another way, conscientious people have already decided to not write bugs but since we don't know how to achieve that objective, it doesn't solve the issue.
One logic flaw in that self-reported assessment is that us readers don't really know if his 1000-line programs were truly proven to be bug free. But we can still set that aside and entertain the idea that just deciding to not write bugs actually prevents bugs. Consider various disciplines:
- programming: Some search/sort code that had a hidden bug for 20+ years. Both computer experts Jon Bentley at Carnegie Mellon Uni and Joshua Bloch (Java expert) created the same bug: https://ai.googleblog.com/2006/06/extra-extra-read-all-about...
- mathematics: Andrew Wiles 1st attempt at proving Fermat's Last Theorem had a flaw that was discovered by his peer reviewers. And the authors for a SAT math exam unwittingly wrote a bug: https://youtu.be/kN3AOMrnEUs?t=2m05s
- laws : legislators write laws with attempts at precise wording but it still has unintended loopholes or perverse incentives they never foresaw
Did all those people "decide to write bugs" therefore they can decide not to? Of course not. You can reduce bugs by following some best practices (e.g. use well-tested encryption library instead of rolling your own, etc) but bugs will still happen no matter how much you "decide" against it.
Put another way, conscientious people have already decided to not write bugs but since we don't know how to achieve that objective, it doesn't solve the issue.
> After writing a few lines of code (3 to 6 lines, a short block within a function), re-read them.
That's insufficient in a lot of codebases in a lot of languages. Addition of a few new "locally-correct" lines can break something else in a completely different part of the code.
A trivial example: You've had a correctly-working program, you change a value of a field using correct calculation, and your program becomes incorrect — because other function elsewhere assumed this value won't change.
To avoid this you'd need to re-read the whole program after introducing any change, and that gets exponentially harder.
That's insufficient in a lot of codebases in a lot of languages. Addition of a few new "locally-correct" lines can break something else in a completely different part of the code.
A trivial example: You've had a correctly-working program, you change a value of a field using correct calculation, and your program becomes incorrect — because other function elsewhere assumed this value won't change.
To avoid this you'd need to re-read the whole program after introducing any change, and that gets exponentially harder.
If I would write down every bug I produced I would have filled a decently sized book by now.
But debugging has some additional functions. If you interface your standard wep-api, you might want to take a look at the response. Yes, you could do that without a debugger, but I would argue it is more convinient. That expands to anything that is a blackbox to you.
Stopping execution and taking a look at the state of the program is a decent tool I would not want to miss.
In some languages you have an insane amount of dependencies and you need something to evaluate the behavior of those submodules.
For pure algorithmic and determinist programs the statement of Dijekstra is true, but you often don't have it in reality. A server sending you garbled responses might garble you program too.
You also become blind to your own code. You think it does something that isn't reflected in code. You think this regex certainly matches the input, but it just doesn't because you forgot something. Reading again might not help.
But debugging has some additional functions. If you interface your standard wep-api, you might want to take a look at the response. Yes, you could do that without a debugger, but I would argue it is more convinient. That expands to anything that is a blackbox to you.
Stopping execution and taking a look at the state of the program is a decent tool I would not want to miss.
In some languages you have an insane amount of dependencies and you need something to evaluate the behavior of those submodules.
For pure algorithmic and determinist programs the statement of Dijekstra is true, but you often don't have it in reality. A server sending you garbled responses might garble you program too.
You also become blind to your own code. You think it does something that isn't reflected in code. You think this regex certainly matches the input, but it just doesn't because you forgot something. Reading again might not help.
bugs
However, "Don't write bugs" is a joke for a seasoned programmer who's had to work with political constraints. Bugs often arise from major design flaws that can't be fixed in an afternoon. Other bugs arise from management compromises to meet a deadline or price point. Even worse, some bugs arise from problems with external APIs that are completely out of a programmer's control. At this point, "don't write bugs" can really only translate to trying to defend against corner cases that are impractical to fix.
"Impractical to fix," meaning, fixing the bug requires navigating politics. This could be working with a vendor to fix a broken API. It could also be working with management to allocate time to refactor a defective design.
Also: There are poor programmers who just can't "Don't write bugs." They can't think through edge cases; or a fundamentally incapable of understanding edge cases that lead to bugs. (And handling these kind of programmers is a different topic altogether.)