Things nobody told me about being a software engineer(anaulin.org)
anaulin.org
Things nobody told me about being a software engineer
https://anaulin.org/blog/things-nobody-told-me-about-being-a-software-engineer/
116 comments
>I've written entire sites on my own that happily run today.
This is probably the key issue. You're either working by yourself or you are on small enough team projects that it's easy to recognize when you are breaking things with a change.
Tests are about rapidly detecting regressions in a programmatic way. It should not require manually testing every time you want to make a small change to the code base and if you depend on reviewers seeing that a change will break something, you are in for a world of hurt on any large project.
Every large project I've been on that has skipped tests has ended up turning into a shitshow where master is broken for days at a time and you can't even tell the cause without hours of manual testing with bisection.
This is probably the key issue. You're either working by yourself or you are on small enough team projects that it's easy to recognize when you are breaking things with a change.
Tests are about rapidly detecting regressions in a programmatic way. It should not require manually testing every time you want to make a small change to the code base and if you depend on reviewers seeing that a change will break something, you are in for a world of hurt on any large project.
Every large project I've been on that has skipped tests has ended up turning into a shitshow where master is broken for days at a time and you can't even tell the cause without hours of manual testing with bisection.
> if you depend on reviewers seeing that a change will break something, you are in for a world of hurt on any large project.
I don’t agree with this at all. Here’s a counter example: Linux.
I think the real problem here is that people don’t take review seriously and/or just don’t care about the project. We build these complicated, expensive systems that do all the thinking for us, not because it’s better, but because we simply don’t care.
Having tests is better than letting sloppy developers run amok. In my experience though tests aren’t really needed on projects with a maintainer that actually gives a damn.
I don’t agree with this at all. Here’s a counter example: Linux.
I think the real problem here is that people don’t take review seriously and/or just don’t care about the project. We build these complicated, expensive systems that do all the thinking for us, not because it’s better, but because we simply don’t care.
Having tests is better than letting sloppy developers run amok. In my experience though tests aren’t really needed on projects with a maintainer that actually gives a damn.
Bad example because a massive chunk of linux is independent systems that have no overlap at all. The netlink code has jack-shit to do with a graphics driver from intel, etc.
Within subsystems regressions sneak in all of the time that don't directly cause compile-time failures.
http://linux-test-project.github.io/
Within subsystems regressions sneak in all of the time that don't directly cause compile-time failures.
http://linux-test-project.github.io/
> I don’t agree with this at all. Here’s a counter example: Linux.
are you sure about that?
https://stackoverflow.com/questions/3177338/how-is-the-linux...
are you sure about that?
https://stackoverflow.com/questions/3177338/how-is-the-linux...
I'm in the same boat and have thought a lot about it. The biggest correlation I see from automated testing advocates is that they are usually using dynamic languages for non-trivial core business logic. I mean that's pretty much it. If you're using a dynamic language for such things, like anything past 10k lines of code, then I agree, you pretty much need automated tests since they're partially filling in the role of a compiler and type system. It begs the question of why to use dynamic languages for tricky business logic, but that's a whole different flamewar!
If the testing advocates are using something with even a basic compiler/type-system (e.g. Java), then IME it often comes down to not being fluent with the design patterns, tricks, IDE shortcuts/plugins, tools, etc. that compilers and type-systems can provide to work with you to catch bugs at build time. I mention Java because it's a language that allows you to keep writing "dynamic" code and avoid the compiler/type-system to a good degree if you want (and boy do people want to). E.g. passing top-level Object types around and casting, using HashMaps instead of a proper Class, heavy reflection use, "stringly" typing, etc. Unlike say Haskell or Rust which force correctness down your throat to a much higher degree.
Obviously no matter the language choice, their comes a point where automated testing becomes cost-effective (e.g. safety-critical software). At that point however there should be formal methods and such in play also. And I'd say for the majority of us working on CRUD apps, we're not at that cost-effectiveness point.
If the testing advocates are using something with even a basic compiler/type-system (e.g. Java), then IME it often comes down to not being fluent with the design patterns, tricks, IDE shortcuts/plugins, tools, etc. that compilers and type-systems can provide to work with you to catch bugs at build time. I mention Java because it's a language that allows you to keep writing "dynamic" code and avoid the compiler/type-system to a good degree if you want (and boy do people want to). E.g. passing top-level Object types around and casting, using HashMaps instead of a proper Class, heavy reflection use, "stringly" typing, etc. Unlike say Haskell or Rust which force correctness down your throat to a much higher degree.
Obviously no matter the language choice, their comes a point where automated testing becomes cost-effective (e.g. safety-critical software). At that point however there should be formal methods and such in play also. And I'd say for the majority of us working on CRUD apps, we're not at that cost-effectiveness point.
Currently you read a lot about unit testing. Usually used badly (testing solo methods or classes).
What you want is E2E tests: test that your product conforms to its specifications. Automate the test procedure your testers have.
What you want is E2E tests: test that your product conforms to its specifications. Automate the test procedure your testers have.
Testing is for business logic not for seeing if the project will build.
Type testing can't catch business logic regressions.
You are able to go without tests because you're writing entire apps on your own. You know what each file, class, and module is supposed to do, and how to test it for regressions manually.
If you're working on a large team with a large suite of applications, this isn't the case. Very often I'll need to make changes to code that I don't explicitly own, or that I've never seen before today. If this code doesn't have tests, this is a recipe for disaster. I don't really understand what this code is supposed to do. How do I know that my change didn't break any existing functionality? Is it obvious how or even possible to test the functionality in this file manually? Maybe it's a background job that processes a file off of an SFTP server, how long will it take me to set up a manual test for that?
It's also about iteration time. Automated tests allow to you check the correctness of every single change nearly instantaneously. I don't want to need switch to a browser and wait for a form to submit just to check that my code still works after a minor refactor, or to ensure that my typo was fixed successfully. Tests mean that code is much more likely to be refactored often, and will lead to much cleaner and easier to maintain codebases.
If you're working on a large team with a large suite of applications, this isn't the case. Very often I'll need to make changes to code that I don't explicitly own, or that I've never seen before today. If this code doesn't have tests, this is a recipe for disaster. I don't really understand what this code is supposed to do. How do I know that my change didn't break any existing functionality? Is it obvious how or even possible to test the functionality in this file manually? Maybe it's a background job that processes a file off of an SFTP server, how long will it take me to set up a manual test for that?
It's also about iteration time. Automated tests allow to you check the correctness of every single change nearly instantaneously. I don't want to need switch to a browser and wait for a form to submit just to check that my code still works after a minor refactor, or to ensure that my typo was fixed successfully. Tests mean that code is much more likely to be refactored often, and will lead to much cleaner and easier to maintain codebases.
This argument, and most of the others at this same level, have a glaring error. They only justify integration tests. They do not justify unit tests (in nearly all cases).
Unit tests wouldn't catch the examples you gave:
> How do I know that my change didn't break any existing functionality?
You don't. And with many unit tests ... you still don't. A software engineer with a few years experience will know this.
> Maybe it's a background job that processes a file off of an SFTP server, how long will it take me to set up a manual test for that?
This especially, no unit test is ever going to catch. This behavior would depend on so many things: encryption, networking, scheduling, starvation, graceful error handling, retry logic ... Not something any amount of unit tests could reasonably verify.
So this must be an argument against unit tests ? And yet, I don't think so.
In practice, you see the opposite. These arguments are designed to force people to write unit tests, and actually used against integration tests (which are complex tests that require tweaking in intelligent ways every time you make a serious change). Of course integration tests and "instantaneous" are usually opposites.
So I don't understand this argument. It only makes sense on the surface. It does not actually work on real large code projects. So I don't understand that you make this argument.
And I especially do not understand that nearly every large company follows advice like this. I mean, I can come up with cynical reasons: "more pointless work with zero verifiable results, but nice "quantifiable" (if meaningless) numbers ! Of course we're in favor !". But really, I cannot come up with a coherent to explain the behavior seen in developers in a large company.
Most statistics about unit tests make no sense. Coverage for instance, says nothing about how many edge cases you handle. Says nothing about many guarantees (like bounded and reasonable allocation for "weird" inputs, same with cpu usage, that it still works together with the OS and other daemons on the system it'll be installed on, ...). Since tests have no "real" purpose for software (they're code that doesn't run) number of tests is meaningless too. Number of lines of test code ... that's beyond useless.
The bigger a piece of software, the more useless unit tests become and the more critical realistic integration tests become ... and the less likely you are to actually find them.
And frankly, for small pieces of code, I find that specialized ways to "test" them that provide actual guarantees of correctness (like Coq) actually work. Unit tests ... never catch edge cases, not even with the best of developers. Of course, even Coq sort-of only makes sure you KNOW about every edge case and requires you to verify it's correctness before letting you pass, so bugs in your understanding of the problem will still be bugs.
But at least you have a theoretical guarantee that a developer actually thought about every edge case. Also, it's pretty hard to use. The language could be clearer, but that's not really the problem. The problem is that Coq WILL show you that no matter how well you think you understand a problem, your understanding is not complete. Often in surprising, interesting ways.
Unit tests wouldn't catch the examples you gave:
> How do I know that my change didn't break any existing functionality?
You don't. And with many unit tests ... you still don't. A software engineer with a few years experience will know this.
> Maybe it's a background job that processes a file off of an SFTP server, how long will it take me to set up a manual test for that?
This especially, no unit test is ever going to catch. This behavior would depend on so many things: encryption, networking, scheduling, starvation, graceful error handling, retry logic ... Not something any amount of unit tests could reasonably verify.
So this must be an argument against unit tests ? And yet, I don't think so.
In practice, you see the opposite. These arguments are designed to force people to write unit tests, and actually used against integration tests (which are complex tests that require tweaking in intelligent ways every time you make a serious change). Of course integration tests and "instantaneous" are usually opposites.
So I don't understand this argument. It only makes sense on the surface. It does not actually work on real large code projects. So I don't understand that you make this argument.
And I especially do not understand that nearly every large company follows advice like this. I mean, I can come up with cynical reasons: "more pointless work with zero verifiable results, but nice "quantifiable" (if meaningless) numbers ! Of course we're in favor !". But really, I cannot come up with a coherent to explain the behavior seen in developers in a large company.
Most statistics about unit tests make no sense. Coverage for instance, says nothing about how many edge cases you handle. Says nothing about many guarantees (like bounded and reasonable allocation for "weird" inputs, same with cpu usage, that it still works together with the OS and other daemons on the system it'll be installed on, ...). Since tests have no "real" purpose for software (they're code that doesn't run) number of tests is meaningless too. Number of lines of test code ... that's beyond useless.
The bigger a piece of software, the more useless unit tests become and the more critical realistic integration tests become ... and the less likely you are to actually find them.
And frankly, for small pieces of code, I find that specialized ways to "test" them that provide actual guarantees of correctness (like Coq) actually work. Unit tests ... never catch edge cases, not even with the best of developers. Of course, even Coq sort-of only makes sure you KNOW about every edge case and requires you to verify it's correctness before letting you pass, so bugs in your understanding of the problem will still be bugs.
But at least you have a theoretical guarantee that a developer actually thought about every edge case. Also, it's pretty hard to use. The language could be clearer, but that's not really the problem. The problem is that Coq WILL show you that no matter how well you think you understand a problem, your understanding is not complete. Often in surprising, interesting ways.
The SFTP file processor is actually a quintessential example of how unit tests can help new developers edit existing code.
A unit test for the processor would mock out all of the SFTP fetching, or move that to a different class, and focus on only the core business logic of processing the file. The core logic could easily be unit tested, and changes could be made to the file processor without needing to replicate the entire SFTP environment in order to determine if there were regressions in the core business logic.
The alternative is needing to spin up a test SFTP environment, or somehow do that mocking in my manual test, just in order to do something as simple as refactor the class or make a small business logic change. A unit test empowers any developer to make those changes, without needing much knowledge of the environment the code runs in.
A unit test for the processor would mock out all of the SFTP fetching, or move that to a different class, and focus on only the core business logic of processing the file. The core logic could easily be unit tested, and changes could be made to the file processor without needing to replicate the entire SFTP environment in order to determine if there were regressions in the core business logic.
The alternative is needing to spin up a test SFTP environment, or somehow do that mocking in my manual test, just in order to do something as simple as refactor the class or make a small business logic change. A unit test empowers any developer to make those changes, without needing much knowledge of the environment the code runs in.
> without needing to replicate the entire SFTP environment in order to determine if there were regressions
Jep. And then it doesn't actually work. Because it tries to read every file at the same time. Because the strings it passes in to resolve the server don't work. Because it never schedules the thread reading from the socket. Because it uses a 100 byte buffer (plenty for tests, after all). Because ...
And even then, what you're describing is not a unit test. A unit tests tests a single unit of code. Just 1 function, nothing more, and mocks out everything else.
So you can never test a refactor of code with a unit test. Because a refactor changes how different pieces of code work together, and unit tests by definition explicitly DON'T test that (all interactions should be mocked out). So a unit test would be useless.
Jep. And then it doesn't actually work. Because it tries to read every file at the same time. Because the strings it passes in to resolve the server don't work. Because it never schedules the thread reading from the socket. Because it uses a 100 byte buffer (plenty for tests, after all). Because ...
And even then, what you're describing is not a unit test. A unit tests tests a single unit of code. Just 1 function, nothing more, and mocks out everything else.
So you can never test a refactor of code with a unit test. Because a refactor changes how different pieces of code work together, and unit tests by definition explicitly DON'T test that (all interactions should be mocked out). So a unit test would be useless.
The fact that something might go wrong in the integration test doesn't mean unit tests for the core logic aren't helpful. Besides, you're probably going to be using an external library for the SFTP connect, so it's very likely to go just fine.
And you can totally use unit tests for what I'm describing. Two classes
- SFTPHandler. Connects via SFTP, downloads latest files off the server, passes contents as a string to the processor class `FileProcessor.process(downloaded_file)`
- FileProcessor. Has one public function, process, which processes the file - doing whatever it needs to. This function can then very easily be unit tested, just passing strings for test files into the function. You can also refactor the `process` function as much as you like, not needing to worry about the SFTP connection at all. The `process` function probably calls a bunch of private functions within that class, but your unit tests don't need to worry about that.
I've used a setup like this in production, it works just fine, and allowed us to improve the performance of the file processing logic and make changes to it very easily and often - without worrying about regressions to the core business logic.
And you can totally use unit tests for what I'm describing. Two classes
- SFTPHandler. Connects via SFTP, downloads latest files off the server, passes contents as a string to the processor class `FileProcessor.process(downloaded_file)`
- FileProcessor. Has one public function, process, which processes the file - doing whatever it needs to. This function can then very easily be unit tested, just passing strings for test files into the function. You can also refactor the `process` function as much as you like, not needing to worry about the SFTP connection at all. The `process` function probably calls a bunch of private functions within that class, but your unit tests don't need to worry about that.
I've used a setup like this in production, it works just fine, and allowed us to improve the performance of the file processing logic and make changes to it very easily and often - without worrying about regressions to the core business logic.
In my experience if there's one thing absolutely guaranteed it's that unit tests decrease the performance of whatever they're testing (Because it eventually leads to local wins that are big losses for the program as a whole, because this encourages doing idiotic stuff like allocating large buffers and keeping enormous non-shared caches and maps)
Now in the example given, performance does not matter, so I do wonder why you'd mention it at all.
How about you just answer me this question: Did you still see significant bug volumes after implementing the unit tests for the FileProcessor ?
Obviously I believe the answer to be "yes". I feel like your statement that changes were made "very easily and often" sort of implies that yes, there were many bugs.
Note that testing based on past bugs is not called unit testing. That is, as you might guess, regression testing (and has the important distinction that it's a VERY good practice to go back through your regression tests once a year, and throw out the ones that don't make sense anymore, which should be about half of them)
Besides, I've very rarely seen tests actually catch bugs. Bugs come from pieces of code not doing what developers expect them to do in 2 ways :
1) outright lack of understanding what the code does (this can also mean that they understand the code, but not the problem it's trying to solve, and so code and tests ... are simply both wrong)
2) lack of consideration for edge cases
3) lack of consideration for the environment the code runs in (e.g. scaling issues. Optimizing business logic that processes a 10M file then executing it on 50G of data)
None of these has a good chance of getting caught by unit tests in my experience.
But developers seem to mostly hate integration tests. Tests that start up the whole system, or even multiple copies of it, and then rapidly run past input through the whole system. When it fails, it takes a while to find why it fails. It may fail, despite all components, potentially written by different people, being "correctly written" just not taking each other into account. It may fail because of memory, cpu starvation, filesystem setup. It may fail occassionally because the backend database decided to VACUUM, and the app is not backing off. It may fail after a firmware upgrade on equipment it uses.
The problem I have with these "issues" is simple: they represent reality. They will occur in production.
And in some ways I feel like this is a fair description: unit tests are about "proving you're right", even, and perhaps especially, if you're wrong. "You see, it isn't my code ! Not my fault !".
Now in the example given, performance does not matter, so I do wonder why you'd mention it at all.
How about you just answer me this question: Did you still see significant bug volumes after implementing the unit tests for the FileProcessor ?
Obviously I believe the answer to be "yes". I feel like your statement that changes were made "very easily and often" sort of implies that yes, there were many bugs.
Note that testing based on past bugs is not called unit testing. That is, as you might guess, regression testing (and has the important distinction that it's a VERY good practice to go back through your regression tests once a year, and throw out the ones that don't make sense anymore, which should be about half of them)
Besides, I've very rarely seen tests actually catch bugs. Bugs come from pieces of code not doing what developers expect them to do in 2 ways :
1) outright lack of understanding what the code does (this can also mean that they understand the code, but not the problem it's trying to solve, and so code and tests ... are simply both wrong)
2) lack of consideration for edge cases
3) lack of consideration for the environment the code runs in (e.g. scaling issues. Optimizing business logic that processes a 10M file then executing it on 50G of data)
None of these has a good chance of getting caught by unit tests in my experience.
But developers seem to mostly hate integration tests. Tests that start up the whole system, or even multiple copies of it, and then rapidly run past input through the whole system. When it fails, it takes a while to find why it fails. It may fail, despite all components, potentially written by different people, being "correctly written" just not taking each other into account. It may fail because of memory, cpu starvation, filesystem setup. It may fail occassionally because the backend database decided to VACUUM, and the app is not backing off. It may fail after a firmware upgrade on equipment it uses.
The problem I have with these "issues" is simple: they represent reality. They will occur in production.
And in some ways I feel like this is a fair description: unit tests are about "proving you're right", even, and perhaps especially, if you're wrong. "You see, it isn't my code ! Not my fault !".
It still seems like the core of your argument is "Some things are hard to test, so you might as well not test anything at all" - which I really don't buy.
> How about you just answer me this question: Did you still see significant bug volumes after implementing the unit tests for the FileProcessor ?
Kinda tricky to answer this, since there were test for this class from the start. But, overall the answer is "no" - we did not see significant bugs in that class. Occasionally we'd get a bug because the SFTP connection failed, or execution took to long and the connection closed - the type of bug that to you seems to negate the value of uniting testing the FileProcessor. But, without the unit tests for the FileProcessor, I'd have those bugs plus more bugs/bad behavior in the business logic. How is that better, exactly?
The idea that tests reduce performance is ridiculous. Improving performance requires making changes to a system while ensuring it's behavior hasn't changed. This is exactly what testing provides. Without tests, you can't optimize the code at all without fear of introducing regressions.
> How about you just answer me this question: Did you still see significant bug volumes after implementing the unit tests for the FileProcessor ?
Kinda tricky to answer this, since there were test for this class from the start. But, overall the answer is "no" - we did not see significant bugs in that class. Occasionally we'd get a bug because the SFTP connection failed, or execution took to long and the connection closed - the type of bug that to you seems to negate the value of uniting testing the FileProcessor. But, without the unit tests for the FileProcessor, I'd have those bugs plus more bugs/bad behavior in the business logic. How is that better, exactly?
The idea that tests reduce performance is ridiculous. Improving performance requires making changes to a system while ensuring it's behavior hasn't changed. This is exactly what testing provides. Without tests, you can't optimize the code at all without fear of introducing regressions.
> It still seems like the core of your argument is "Some things are hard to test, so you might as well not test anything at all"
Nope. My argument is twofold:
1) unit tests don't actually provide the guarantees that people keep putting forward as reasons to write them
2) this makes them a dangerous, as they provide "reassurance" that isn't actually backed by reality
> But, without the unit tests for the FileProcessor, I'd have those bugs plus more bugs/bad behavior in the business logic. How is that better, exactly?
So the tests failed to catch problems with the program's behavior. Maybe it's just me, but I call that a problem.
Testing the business logic as a whole is not a unit test, except in the marginal cases where all your business logic fits neatly in a single function, or at least a single class. If it's actually a few algorithms, a few files, a bunch of functions and you test everything together, that's an integration test and not a functional test.
If you use actual (or slightly changed) data to test that business logic, as opposed to artificially crafted data, that again makes it not a unit test.
> The idea that tests reduce performance is ridiculous
If you use tests to optimize code you're optimizing a piece of code in isolation, without taking the rest of the system into account. That works for trivially simple initial optimization, but falls completely on it's face when you're actually writing programs that stress the system.
> Improving performance requires making changes to a system while ensuring it's behavior hasn't changed.
The system as a whole, sort of. Individual units ? Absolutely not.
Besides, tests merely provide FALSE assurance behavior hasn't changed. Time and time again I've had to fix "I've optimized it" bugs. TLDR is always the same. Unit tests pass so the code "must be right" (meaning they don't run the code outside of unit tests, and the unit tests only directly test the code, not blackbox). Then in the actual run edge cases were hit.
And "edge cases" makes it sound like you hardly ever hit this. Just yesterday we had a big issue. What happened ? Well we had a method that disables an inbound phone line (e.g. for maintenance, or changes). All unit tests passed. Unfortunately we really should have tested that it does NOT disable anything else (method was essentially "go through list, if it matches, disable". Needless to say, it disabled every line). Regression testing added.
We had someone optimize the dial plan handling. He didn't realize that his "cache" was in fact recreated at the very beginning of every request, when the code evaluated a dial plan, in other words, it was a serious performance regression rather than an improvement. Really looked like an improvement though. Unit tests ... passed. Of course, "the behavior hadn't changed". About 2 calls got through for ~6 minutes (normally thousands). Now we have a test that turns up the whole system, with the actual production dial plan, and then goes through 10000 calls. If it takes more than 1 minute, test fails. Developers hate it, but it's non-negotiable at this point.
I can go on for quite a while enumerating problems like this. Billing files erased (forgot to append). Entire dialplan erased on startup (essentially same problem). Lots of ways to trigger cpu starvation. Memory exhaustion. Memory leaks (the system is mostly Java). Connecting unrelated calls together (that was pretty fun for the first 2-3 minutes). Ignoring manager signals (one server having to do all calls ... which is not happening)
This is a repeating pattern in our failures. Unit tests ... essentially always pass. And time and time again someone tries to make the point that this must mean the code is correct.
And that's bullshit. That's the problem.
Nope. My argument is twofold:
1) unit tests don't actually provide the guarantees that people keep putting forward as reasons to write them
2) this makes them a dangerous, as they provide "reassurance" that isn't actually backed by reality
> But, without the unit tests for the FileProcessor, I'd have those bugs plus more bugs/bad behavior in the business logic. How is that better, exactly?
So the tests failed to catch problems with the program's behavior. Maybe it's just me, but I call that a problem.
Testing the business logic as a whole is not a unit test, except in the marginal cases where all your business logic fits neatly in a single function, or at least a single class. If it's actually a few algorithms, a few files, a bunch of functions and you test everything together, that's an integration test and not a functional test.
If you use actual (or slightly changed) data to test that business logic, as opposed to artificially crafted data, that again makes it not a unit test.
> The idea that tests reduce performance is ridiculous
If you use tests to optimize code you're optimizing a piece of code in isolation, without taking the rest of the system into account. That works for trivially simple initial optimization, but falls completely on it's face when you're actually writing programs that stress the system.
> Improving performance requires making changes to a system while ensuring it's behavior hasn't changed.
The system as a whole, sort of. Individual units ? Absolutely not.
Besides, tests merely provide FALSE assurance behavior hasn't changed. Time and time again I've had to fix "I've optimized it" bugs. TLDR is always the same. Unit tests pass so the code "must be right" (meaning they don't run the code outside of unit tests, and the unit tests only directly test the code, not blackbox). Then in the actual run edge cases were hit.
And "edge cases" makes it sound like you hardly ever hit this. Just yesterday we had a big issue. What happened ? Well we had a method that disables an inbound phone line (e.g. for maintenance, or changes). All unit tests passed. Unfortunately we really should have tested that it does NOT disable anything else (method was essentially "go through list, if it matches, disable". Needless to say, it disabled every line). Regression testing added.
We had someone optimize the dial plan handling. He didn't realize that his "cache" was in fact recreated at the very beginning of every request, when the code evaluated a dial plan, in other words, it was a serious performance regression rather than an improvement. Really looked like an improvement though. Unit tests ... passed. Of course, "the behavior hadn't changed". About 2 calls got through for ~6 minutes (normally thousands). Now we have a test that turns up the whole system, with the actual production dial plan, and then goes through 10000 calls. If it takes more than 1 minute, test fails. Developers hate it, but it's non-negotiable at this point.
I can go on for quite a while enumerating problems like this. Billing files erased (forgot to append). Entire dialplan erased on startup (essentially same problem). Lots of ways to trigger cpu starvation. Memory exhaustion. Memory leaks (the system is mostly Java). Connecting unrelated calls together (that was pretty fun for the first 2-3 minutes). Ignoring manager signals (one server having to do all calls ... which is not happening)
This is a repeating pattern in our failures. Unit tests ... essentially always pass. And time and time again someone tries to make the point that this must mean the code is correct.
And that's bullshit. That's the problem.
OK, I get that you don't like unit tests. You also seem to have a very strict and unhelpful definition of what a unit test is. I don't really care for the academia of it, I just want to know that my code works. So, if you call it a unit test or an integration test or a functional test or a behavioral test - whatever. The important thing that it allows me get some idea of whether or not the code is doing what it's supposed to do.
What do you propose instead of testing? Manually QA every single piece of the system for every change? Not a lot of companies have the headcount for a the fleet of QA people that would require.
What do you propose instead of testing? Manually QA every single piece of the system for every change? Not a lot of companies have the headcount for a the fleet of QA people that would require.
For what it's worth I think you both make great points. Whomever one agrees with perhaps mostly hinges on whether unit tests can get in the way of integration tests.
I'm inclined to believe that some of the exhaustive unit testing culture can provide a false sense of security, and the work involved to write them can get in the way of writing proper integration tests (or 'degrees of' if you see it as a spectrum).
Provided that proper test coverage includes both integration tests and unit tests, it probably won't hurt to do both. I like unit tests as part of my process (the red-green TDD process can be quite addictive and useful), but under pressure I'd prioritize integration tests, knowing that the more time I spend unit testing, the less I'll spend integration testing.
I'm inclined to believe that some of the exhaustive unit testing culture can provide a false sense of security, and the work involved to write them can get in the way of writing proper integration tests (or 'degrees of' if you see it as a spectrum).
Provided that proper test coverage includes both integration tests and unit tests, it probably won't hurt to do both. I like unit tests as part of my process (the red-green TDD process can be quite addictive and useful), but under pressure I'd prioritize integration tests, knowing that the more time I spend unit testing, the less I'll spend integration testing.
I've found the sweet spot for tests are
- complex code that will change with definable inputs/outputs that won't
- large projects
I've found they're not nearly as helpful in
- small projects
- UI intensive code
- code where the requirements are ill defined and change often
- complex code that will change with definable inputs/outputs that won't
- large projects
I've found they're not nearly as helpful in
- small projects
- UI intensive code
- code where the requirements are ill defined and change often
It comes down to the cost of failure. If the cost of failure for a particular module say, payments processing or rocket guidance system, is high then writing tests could be much cheaper than waiting for a failure, especially in production.
Your code is full of bugs, you just haven't noticed yet.
Whether that matters or not (to you or your end users) is your grey ground in the middle.
It's possible you've only ever worked on things that are simple or obvious enough to not have the need for tests, and that code that never changes. Especially if that's UI code, where you're testing it manually as you write it, you may be able to live with this on small code bases.
If you have never worked in an environment where the benefit of at least some level of automated testing is not blindingly obvious, my advice to you is to stop doing the same thing over and over and find a new job.
Whether that matters or not (to you or your end users) is your grey ground in the middle.
It's possible you've only ever worked on things that are simple or obvious enough to not have the need for tests, and that code that never changes. Especially if that's UI code, where you're testing it manually as you write it, you may be able to live with this on small code bases.
If you have never worked in an environment where the benefit of at least some level of automated testing is not blindingly obvious, my advice to you is to stop doing the same thing over and over and find a new job.
Projects with unit tests are also riddled with bugs.
I have absolutely no respect for your attitude or your comment. Sanctimonious zealotry like yours should have no place in our industry.
It's disgusting that you think you can tell someone they're shit or trivialize their work because you disagree with their workflow.
I have absolutely no respect for your attitude or your comment. Sanctimonious zealotry like yours should have no place in our industry.
It's disgusting that you think you can tell someone they're shit or trivialize their work because you disagree with their workflow.
I hardly think that encouraging someone to seek out alternative positions from which to view the world, and a suggestion that there is a grey area for all this constitutes zealotry. Quite the opposite, in fact.
And in my experience projects with unit tests are far _less_ riddled with bugs, which is the whole point, no?
And I don't know understand why on earth you think I'm telling anyone they are "shit". I'd advise someone to broaden their horizons if they said they thought functional programming was worthless, or that assembly language was always pointlessly low level, or any one of a number of possible assertions that wouldn't sit well with me. That is far from "you are shit" - it's much more assuming they are not shit, but haven't had the right context yet.
And in my experience projects with unit tests are far _less_ riddled with bugs, which is the whole point, no?
And I don't know understand why on earth you think I'm telling anyone they are "shit". I'd advise someone to broaden their horizons if they said they thought functional programming was worthless, or that assembly language was always pointlessly low level, or any one of a number of possible assertions that wouldn't sit well with me. That is far from "you are shit" - it's much more assuming they are not shit, but haven't had the right context yet.
There is something in the middle actually. Just write test for the hard tasks or the algos that you implemented your self. (Some stuff that I don't undertand end up much easier to write once I've created some tests to help me iterate faster)
Testing basic class re-use (I'm thinking about serializers, or ViewSets like in django) doesn't make much sense to me, and in fact was 100% automatised.
Those "simple" test end-up being useful when you start hitting your app with the hammer to change a lot of stuff in it. I havn't had any use for these otherwhise.
Testing basic class re-use (I'm thinking about serializers, or ViewSets like in django) doesn't make much sense to me, and in fact was 100% automatised.
Those "simple" test end-up being useful when you start hitting your app with the hammer to change a lot of stuff in it. I havn't had any use for these otherwhise.
Many if not most tests are a waste of time (especially when the same guarantees are provided by, say, a type system).
That said, automated tests catch a problem that would make it to production just about weekly in my experience. If you're trying to do any kind of HA/CI/CD they're really really helpful. Beyond that, isolated "unit tests" are somewhat helpful too in my experience, but only so far as they allow you to easily run your test suite (and rarely, find bugs faster).
That said, automated tests catch a problem that would make it to production just about weekly in my experience. If you're trying to do any kind of HA/CI/CD they're really really helpful. Beyond that, isolated "unit tests" are somewhat helpful too in my experience, but only so far as they allow you to easily run your test suite (and rarely, find bugs faster).
Broadly speaking, there are several steps in the development process where defects are caught:
1. During development: you code something, run it, see something fail and fix it. Most people don't consider this part of QA because it's coding/debugging.
2. Code review: a teammate reads your code and catches a bug or an opportunity for a bug to arise.
3. Manual QA: a teammate tries the new feature, finds a defect and reports it.
4. Production: a user reports the error.
5. Monitoring: tools like Sentry may warn you before a user reports the issue.
With your experience, you probably have learned to avoid patterns in your code that make #1 occur often. Maybe you use static typing, linters, an IDE or some other tools to help you avoid silly mistakes.
If you perform code review (#2), then you help less-experienced teammates avoid mistakes even if they haven't learned to avoid error-prone patterns or don't use tooling like you do. They spend more time in #1 than you do.
If you care (above average for developer standards) about your job, you probably manually test the software you make to see how your users experience your creation (#3) before checking-in code, while code-reviewing, and/or afterwards to help testers. This takes time and if you find mistakes, that means other teammates that did this before you invested time also.
If your company releases progressively (using feature flags) you get info from users (#4) and fix before it's considered an issue. Also, if using monitoring tools (#5) you may avoid users ever noticing there was an issue. These all lead to rework and are risky.
I don't have enough information about what process you follow, so I tried making references to generic practices. In theory, writing tests first reduces the amount of time reacting in situations 1-5. I have felt the benefits in my own team and seen them when coaching other teams.
If none of what I have written applies to you and your team, please let me know. Would love to buy you all a meal to hear about what you do to avoid defects in your software. I'm not being snarky here. Have genuinely been asking myself how do these developers I read about online manage to ship bug-free software really fast without writing tests.
1. During development: you code something, run it, see something fail and fix it. Most people don't consider this part of QA because it's coding/debugging.
2. Code review: a teammate reads your code and catches a bug or an opportunity for a bug to arise.
3. Manual QA: a teammate tries the new feature, finds a defect and reports it.
4. Production: a user reports the error.
5. Monitoring: tools like Sentry may warn you before a user reports the issue.
With your experience, you probably have learned to avoid patterns in your code that make #1 occur often. Maybe you use static typing, linters, an IDE or some other tools to help you avoid silly mistakes.
If you perform code review (#2), then you help less-experienced teammates avoid mistakes even if they haven't learned to avoid error-prone patterns or don't use tooling like you do. They spend more time in #1 than you do.
If you care (above average for developer standards) about your job, you probably manually test the software you make to see how your users experience your creation (#3) before checking-in code, while code-reviewing, and/or afterwards to help testers. This takes time and if you find mistakes, that means other teammates that did this before you invested time also.
If your company releases progressively (using feature flags) you get info from users (#4) and fix before it's considered an issue. Also, if using monitoring tools (#5) you may avoid users ever noticing there was an issue. These all lead to rework and are risky.
I don't have enough information about what process you follow, so I tried making references to generic practices. In theory, writing tests first reduces the amount of time reacting in situations 1-5. I have felt the benefits in my own team and seen them when coaching other teams.
If none of what I have written applies to you and your team, please let me know. Would love to buy you all a meal to hear about what you do to avoid defects in your software. I'm not being snarky here. Have genuinely been asking myself how do these developers I read about online manage to ship bug-free software really fast without writing tests.
It's probably a combination of #3 (I always do that now, use the feature I just made/improved/fixed) and that I always make sure I can reliably repro a bug before I try and fix the bug, so I always know it's fixed.
Also a significant majority of my code has been statically typed as you say and I'm moving even more over to static typing now I'm using TS over JS in all new code.
Also a significant majority of my code has been statically typed as you say and I'm moving even more over to static typing now I'm using TS over JS in all new code.
> Am I (and colleagues) a magic snowflake, or are tests a massive waste of time, or is it somewhere in the middle?
Tests are definitely a massive waste of time if you never need them. Write and forget code exists and can certainly run happily for years. On the other hand, frequently changing something in a large existing codebase, especially without static typing, is almost certainly going to end up more expensive without automated tests.
Tests are definitely a massive waste of time if you never need them. Write and forget code exists and can certainly run happily for years. On the other hand, frequently changing something in a large existing codebase, especially without static typing, is almost certainly going to end up more expensive without automated tests.
[deleted]
I'm not a professional programmer, but I think it comes down to whether you expect to ever do major refactoring. If requirements change substantially, would you write new code, or would you modify the existing code and attempt to maintain some kind of backward compatibility? If the latter, tests can be worth it.
I have a library that never had any tests until I wanted to re-architect it in a fundamental way whilst porting it from Python 2 to Python 3. Since it was a wire-protocol type of thing, the Python bytes–unicode distinction made this hard. So I wrote tests for each piece of functionality as I ported that piece. I keep the tests around nowadays even though the porting is complete, but am kind of lazy writing ones for new functionality.
Of course tests also seem really important when you're accepting 3rd party contributions to your codebase. I've made what I thought were totally innocuous pull requests to projects only to see in the CI tests that my code causes regressions.
I have a library that never had any tests until I wanted to re-architect it in a fundamental way whilst porting it from Python 2 to Python 3. Since it was a wire-protocol type of thing, the Python bytes–unicode distinction made this hard. So I wrote tests for each piece of functionality as I ported that piece. I keep the tests around nowadays even though the porting is complete, but am kind of lazy writing ones for new functionality.
Of course tests also seem really important when you're accepting 3rd party contributions to your codebase. I've made what I thought were totally innocuous pull requests to projects only to see in the CI tests that my code causes regressions.
How much maintenance have those sites seen?
I usually focus on end-to-end tests: when I had a new feature or do some refactoring I don't have to try things manually. I just launch my test suite go prepare some coffee and if the suite passes I'm confident in not having broken anything. Also writing new tests for new features helps thinking about the edge-cases you'll encounter.
When someone reports a bug I reproduce it. Then I write a test reproducing the problem. And at last I correct it. Now I know 6 months or one year from now I won't have a regression and see this bug coming back. People really hate when something they reported months ago is a problem again.
I usually focus on end-to-end tests: when I had a new feature or do some refactoring I don't have to try things manually. I just launch my test suite go prepare some coffee and if the suite passes I'm confident in not having broken anything. Also writing new tests for new features helps thinking about the edge-cases you'll encounter.
When someone reports a bug I reproduce it. Then I write a test reproducing the problem. And at last I correct it. Now I know 6 months or one year from now I won't have a regression and see this bug coming back. People really hate when something they reported months ago is a problem again.
Huge amounts, you don't have a million users for failed projects.
I still maintain various code bases for multiple clients that are over 10 years old, at varying levels of TLOC (10k, 100k, million k).
The truth is bugs almost never come back like that. I know because of the thousands, maybe 10s of thousand, of bugs I've fixed over the years.
I still maintain various code bases for multiple clients that are over 10 years old, at varying levels of TLOC (10k, 100k, million k).
The truth is bugs almost never come back like that. I know because of the thousands, maybe 10s of thousand, of bugs I've fixed over the years.
Tests fail all the time during development. Test fails indicate a bug. You fix the bug before you commit. The bug never makes it to prod. It's impossible that only one test ever failed during development.
When I was a rookie programmer 12 years ago, I was assigned a task to optimise a routine. It took current vehicle speed as input, performed integration and calculated total distance traveled. I made a mistake that was let through code review and was caught by an automated test. My calculations were off by 1% which accumulated to a larger amount in 10 minutes. After that incident we never had another bug for 5 years in that module. Does this make the tests invalid? Our does this improve confidence in our code?
God help the person who has to pick up after you, do refactoring etc etc. Not writing tests is unprofessional as it introduces risk. If code is being used by 1 person or is some non-critical internal application then fine. If it is being used by 1 million people to earn a living the people responsible should lose their jobs. I spend a significant amount of my time tidying up after people who don't write tests and leave a mess and it's a nightmare (but keeps my family fed, housed and clothed so...).
As always, "it depends." Testing is usually about larger teams, cost of failure, and sleeping better at night during refactors.
If the project is small, fits in one skull, and not critical enough to be maintained/redesigned for years on end tests buy you less.
Also, technology can prevent some classes of problems, such as type checkers, linters, etc. One hopes you are at least using these.
#1 gives the impression the author is using a dynamic "scripting" language, which may require a higher ratio of tests per line as a project matures.
If the project is small, fits in one skull, and not critical enough to be maintained/redesigned for years on end tests buy you less.
Also, technology can prevent some classes of problems, such as type checkers, linters, etc. One hopes you are at least using these.
#1 gives the impression the author is using a dynamic "scripting" language, which may require a higher ratio of tests per line as a project matures.
Writing tests makes me write better software. The resulting test suite is just a nice side effect.
>We'd have picked it up anyway when actually testing it.
Why not automate the tests you often run?
Why not automate the tests you often run?
Test interfaces, not implementations.
> 8. That having decent people skills makes my technical skills suspect, in the eyes of some.
I'm questioning how true this is, especially in today's world of whiteboard interviews and open office spaces. My personal experience seems to suggest that better people skills suggest better technical skill, especially to management.
I'm questioning how true this is, especially in today's world of whiteboard interviews and open office spaces. My personal experience seems to suggest that better people skills suggest better technical skill, especially to management.
I have seen that. I also seen people openly talking about social skills being opposite of coding skills and literally making implications about others. I have seen people pretend lower social skill then they actually have.
The most common through is assuming that someone with bad social skills is genius in coding with zero grounds for it. Without seeing code or even talking with those who know his code closely, anything like that. Assumption was that that those people are better in coding because got knows why. I suspect part of it is wish to compensate, but part is stereotype.
Open offices have nothing with it, except making people with poor social skills more actively annoying. Which can cancel some of effect I guess.
The most common through is assuming that someone with bad social skills is genius in coding with zero grounds for it. Without seeing code or even talking with those who know his code closely, anything like that. Assumption was that that those people are better in coding because got knows why. I suspect part of it is wish to compensate, but part is stereotype.
Open offices have nothing with it, except making people with poor social skills more actively annoying. Which can cancel some of effect I guess.
Not long ago, I failed two sql whiteboard problems and passed 2 c# problems. I was informed by HR that they are making me an offer based on my great people skills, even though I failed their SQL quesitons.
I sucked at white boarding(still do because I don't like grinding leetcode before interviews, and I usually make it known). When I first started interviewing I needed like 10 minutes to come up with palindrome solution even though I did it in school so many times. That time I also received an offer.
Now, that I think of it I my success rate is 75% and that includes a fortune 1k company. However, it is a pretty small sample size. Now, that I think of it I need to interview more.
I sucked at white boarding(still do because I don't like grinding leetcode before interviews, and I usually make it known). When I first started interviewing I needed like 10 minutes to come up with palindrome solution even though I did it in school so many times. That time I also received an offer.
Now, that I think of it I my success rate is 75% and that includes a fortune 1k company. However, it is a pretty small sample size. Now, that I think of it I need to interview more.
Very true. Back when I was trying to become a dev I'd show up in a suit and try to be as socially pleasant as possible. When I started dressing more sloppy, not shaving, etc. and being slightly more awkward (mimicing the interviewer) I actually started getting offers..
I think that fits into 4 more than 8. You have good enough people skill to recognize that you're different than the interviewer, and then putting up an facade to convince them you're like them.
People with poor people skills incidentally act similar to other people with poor people skills. People with good people skills can choose to act similar to people with poor people skills.
People with poor people skills incidentally act similar to other people with poor people skills. People with good people skills can choose to act similar to people with poor people skills.
I feel like they might be referring to peers here rather than management
Given the author's name, I think she's female. I fear that, for a female, #8 is more likely to be true than for a male.
> That my gender or my age or my ethnicity or my sexual orientation or my weight or my clothes might (will!) have an impact on the perceived quality of the software I build. (Or, in other words, that this is not really a meritocracy, and doing a good job is not nearly enough.)
Does there exist any body of proof around this one? Would like to see it and add it to my corpus of knowledge
Does there exist any body of proof around this one? Would like to see it and add it to my corpus of knowledge
Yes. One typical test is to have people evaluate identical resumes, with the thing being checked for constituting the only difference (e.g. "ethnic" ames vs. "regular" names).
Other data points are gathered by studying success at finding internships. A recent study found a rather staggering difference - names indicating local ethnicity got a place in 1-2 letters, while some other ethnicities had to write over 20 (if memory serves).
You'll want to find studies for the country you're living in, as the parameters inevitably vary (e.g. which ethnicities to compare). I don't think the outcomes cary much, but i didn't do a comparative study on the literature.
Other data points are gathered by studying success at finding internships. A recent study found a rather staggering difference - names indicating local ethnicity got a place in 1-2 letters, while some other ethnicities had to write over 20 (if memory serves).
You'll want to find studies for the country you're living in, as the parameters inevitably vary (e.g. which ethnicities to compare). I don't think the outcomes cary much, but i didn't do a comparative study on the literature.
I am upvoting you, but you didn't actually answer the question. Your examples have to do with getting a job, not perceived quality of work once one has a job. Both are bad, and your examples are arguably worse, but it doesn't shed any light on how one's clothes, name, etc. can affect the perception of one's actual work.
Maybe names shouldn't be disclosed until the actual hiring is done.
It's not that simple since it's essentially impossible to do a true blind hiring process, due to the fact that interviews happen in person (on video, physically, etc.) and so hiding the name can get a foot in the door but biases take over once the interviews start.
That's not going to work, unless we don't have in person interviews. Since most jobs involve working physically with other people, and, theoretically, you want the interview to simulate the actual working conditions as closely as possible, so it constitutes a valid "work sample test," that doesn't seem workable.
Well I certainly hope I havent been growing out this Unix beard for nothing!
Grey beards no longer allowed at some places, so grab a Just for Men.
I think this is true for most professions. Just look around you at work. You might see similarities/differences among/between groups: managers vs ICs, senior vs junior, execs vs non-execs. It's not homogenous across professions but will be idiosyncratic to your company or even to your department. There's the adage: "Dress for the job you want, not the job you have." You can liberally interpret the meaning of the word "dress" as more than just clothing.
On mobile so haven't got exact links to hand so you might have to do some googling to find the exact things I mention. There's a study which examined pull requests to certain open source projects on GitHub and compared approval rates for accounts which could be identified as female (name or profile picture) with those of men, and also of those known to belong to females but where you could not tell from the profile. They found interesting differences in approvals ratings.
On a more general note the book Lean In by Sheryl Sandberg is a good read if you're interested in learning about biases you (whether man or woman) might have. The book is thoroughly cited and aims to be productive rather than an attack on men and does a great job in my opinion. It points out certain things such as how certain traits are seen as positive in men and negative in women and how this manifests itself in the workplace. It definitely helped me spot things I was guilty of without even realising before
On a more general note the book Lean In by Sheryl Sandberg is a good read if you're interested in learning about biases you (whether man or woman) might have. The book is thoroughly cited and aims to be productive rather than an attack on men and does a great job in my opinion. It points out certain things such as how certain traits are seen as positive in men and negative in women and how this manifests itself in the workplace. It definitely helped me spot things I was guilty of without even realising before
Thank you for the book recommendation. Added to my reading list
What clothes will make my manager and coworkers think I write better software? (Serious question.)
Also regarding presentation improving the perceived quality of work, I noticed that when I started writing my homework assignments in LaTeX in grad school (math), I started getting better grades. I don't think my actual quality of work increased, but I was able to get away with less convincing arguments.
Also regarding presentation improving the perceived quality of work, I noticed that when I started writing my homework assignments in LaTeX in grad school (math), I started getting better grades. I don't think my actual quality of work increased, but I was able to get away with less convincing arguments.
[deleted]
Guilt by association. If you belong to a certain group then there will be certain assumptions made about you. It is human nature.
This will happen every time you meet someone new, even when the person belongs to the same group as you.
The question is: what do you want to do about it? Is it a reasonable ask for society to suppress how it identifies you? Should you acknowledge the reality and work with it?
Different groups and communities have approached this issue in different ways: band together and support each other, organize politically, organize protests, use guilt, stay low and work hard, use another strength to gain leverage, etc. Some of these strategies and tactics will gain the community respect. Others will strengthen the stereotype - they may get what they want but will lose something else.
This will happen every time you meet someone new, even when the person belongs to the same group as you.
The question is: what do you want to do about it? Is it a reasonable ask for society to suppress how it identifies you? Should you acknowledge the reality and work with it?
Different groups and communities have approached this issue in different ways: band together and support each other, organize politically, organize protests, use guilt, stay low and work hard, use another strength to gain leverage, etc. Some of these strategies and tactics will gain the community respect. Others will strengthen the stereotype - they may get what they want but will lose something else.
I suggest going to the Harvard Project Implicit tests and taking a few. After that, study the design of the test. Because humans are the decision makers (not some objective code review acceptance program for example) all you would need to 'prove' is that humans are bad at making objective decisions. We use heuristics consciously and more importantly _unconsciously_ to the detriment of people who don't fit norms.
Also consider alternative perspectives on IAT, for example https://www.psychologytoday.com/us/blog/culture-conscious/20....
Inline excerpts:
> The test-retest reliability (repeatability) of the Race IAT is only .42, which falls well below the psychometric standard of .80. Your score on the IAT can fluctuate significantly from one testing to the next. Hart Blanton, a psychologist at the University of Connecticut, has noted the ease with which people can decrease their racial bias score “by simply exposing people to pictures of African-Americans enjoying a picnic.”
> The validity of IAT scores has also come under scrutiny. In 2013, Frederick Oswald and his research team published a meta-analysis of 46 studies.[1] They found that IAT scores are poor predictors of actual behavior and policy preferences. They also found that IAT scores predicted behaviors and policy preferences no better than scores on simple paper-and-pencil measures of prejudice.
> Perhaps the most interesting feature of the debate concerns how to interpret IAT scores. Some social psychologists believe the IAT measures prejudice, while others believe the IAT measures our knowledge of common stereotypes.
> They then took a version of the Race IAT with Noffians and Fasites standing in for Blacks and Whites, respectively. The students showed a consistent bias against Noffians, not because they perceived Noffians as bad but because they perceived them as bad off.
Inline excerpts:
> The test-retest reliability (repeatability) of the Race IAT is only .42, which falls well below the psychometric standard of .80. Your score on the IAT can fluctuate significantly from one testing to the next. Hart Blanton, a psychologist at the University of Connecticut, has noted the ease with which people can decrease their racial bias score “by simply exposing people to pictures of African-Americans enjoying a picnic.”
> The validity of IAT scores has also come under scrutiny. In 2013, Frederick Oswald and his research team published a meta-analysis of 46 studies.[1] They found that IAT scores are poor predictors of actual behavior and policy preferences. They also found that IAT scores predicted behaviors and policy preferences no better than scores on simple paper-and-pencil measures of prejudice.
> Perhaps the most interesting feature of the debate concerns how to interpret IAT scores. Some social psychologists believe the IAT measures prejudice, while others believe the IAT measures our knowledge of common stereotypes.
> They then took a version of the Race IAT with Noffians and Fasites standing in for Blacks and Whites, respectively. The students showed a consistent bias against Noffians, not because they perceived Noffians as bad but because they perceived them as bad off.
That's good, and here is some more including different ways to measure implicit bias that have emerged. The point stands that people aren't objective:
>White applicants get about 50 percent more call-backs than black applicants with the same resumes; college professors are 26 percent more likely to respond to a student’s email when it is signed by Brad rather than Lamar; and physicians recommend less pain medication for black patients than white patients with the same injury.
https://www.scientificamerican.com/article/how-to-think-abou...
>White applicants get about 50 percent more call-backs than black applicants with the same resumes; college professors are 26 percent more likely to respond to a student’s email when it is signed by Brad rather than Lamar; and physicians recommend less pain medication for black patients than white patients with the same injury.
https://www.scientificamerican.com/article/how-to-think-abou...
Names have a small, but pervasive, influence on economic outcomes. This was/is well known and name anglicization was/is a mass phenomenon. See for example, http://ftp.iza.org/dp7725.pdf or https://en.wikipedia.org/wiki/Anglicisation_of_names.
> Almost a third of naturalizing immigrants abandoned their first names by 1930 and acquired popular American names such as William, John or Charles. [...] Widespread name Americanization prompts the question of whether it had an impact on migrants’ economic success. Figure 1 provides a preliminary answer to this question. Name Americanization into the most popular names - e.g. the top quartile - was associated with an occupation-based earnings increase of above 10%. These gains were larger than those experienced by migrants Americanizing into less popular names - e.g. the first quartile - and even more so than those experienced by migrants who kept their original name or changed to a more distinctive name.
As to speculation on why that is, could be as simple as the fact that the brain capacity is rather limited. Handling unfamiliar letter/phoneme sequences requires more energy. As entropic creatures, we simply take the lowest energy path whenever possible.
> Almost a third of naturalizing immigrants abandoned their first names by 1930 and acquired popular American names such as William, John or Charles. [...] Widespread name Americanization prompts the question of whether it had an impact on migrants’ economic success. Figure 1 provides a preliminary answer to this question. Name Americanization into the most popular names - e.g. the top quartile - was associated with an occupation-based earnings increase of above 10%. These gains were larger than those experienced by migrants Americanizing into less popular names - e.g. the first quartile - and even more so than those experienced by migrants who kept their original name or changed to a more distinctive name.
As to speculation on why that is, could be as simple as the fact that the brain capacity is rather limited. Handling unfamiliar letter/phoneme sequences requires more energy. As entropic creatures, we simply take the lowest energy path whenever possible.
go out in the world and pay attention for a few days, you'll get all the proof you need.
> why is this not a standard feature of testing frameworks? I want some way to re-run tests flipping some of the assertions, to make sure they are testing what I think they are.
That's mutation testing and it the next big thing for tests.
That's mutation testing and it the next big thing for tests.
The author was maybe the best engineer I've had the pleasure to work with directly in my 13 years in the industry. What a treat to read her internal thoughts on the job. Highly recommend her other posts on e.g. measuring performance of engineers https://anaulin.org/blog/why-do-we-have-performance-evaluati...
Hand-wringing and debating about how we should use the bug reporting system, triage, sprint planning, user stories, burn-down charts, etc. will keep the leads and PMs busy for decades on end, even if the tools never change.
Re #18: It exists, it's called mutation testing, and it's awesome (I only know the Java variant called http://pitest.org)
Stryker is a JS version. Google has some interesting whitepapers about their usage of mutation testing as well: https://ai.google/research/pubs/pub46584
Ruby, too: https://github.com/mbj/mutant I think most languages have something like this now.
Don't agree with all the rules, some depend on where you work (e.g, we don't have a lot of tests in our ~30 year old codebase) .
However, Rule #2 is true for me. Accidentally pressing those keybindings in a textfield that does not accept them has become common.
However, Rule #2 is true for me. Accidentally pressing those keybindings in a textfield that does not accept them has become common.
Try pressing them in other situations and you might be surprised how often they work, Youtube player for example to pause, forward/rewind.
> 20. That it is all people, all the way down. They are as cute as turtles, but don’t have hard shells.
A thousand times this. I used to be amazed by people's intelligence. Now I am amazed by their kindness.
A thousand times this. I used to be amazed by people's intelligence. Now I am amazed by their kindness.
This analogy is lost on me. Could you explain?
Pretty much all of these ring true for me (20+ years in the industry).
Frontend programmers are very, very undervalued, I 100% agree. This is because they build the things that your user interacts directly with, where the "rubber meets the road", so to speak. Good FE coders are worth their weight in gold, to any company that has users using their software.
On this point about testing... what’s still surprising to me after twenty years of doing this is how many errors aren’t technical errors, but rather because someone forgot / failed to document / communicate how something was supposed to work. So something fails, fingers get pointed in the direction of the technical team, but the root cause is all too human.
Regarding #2, I have a harder time switching between touch screen vs. touch pad than I do between text editor key bindings. I seamlessly shift between vi and emacs throughout the day depending on what I'm editing, but put my laptop and tablet side by side and I am forever trying to touch the laptop screen.
> PHP is cool again
Come again?
Come again?
> That you can have more than a 100% base salary difference doing the same job, depending on if you work at a small startup or one of the large companies.
Even among and within large companies. Somewhat less so within, usually. But one element of that is the (US?) taboo around sharing compensation info.
Even among and within large companies. Somewhat less so within, usually. But one element of that is the (US?) taboo around sharing compensation info.
> That the new Altavista competitor would become a less bad Microsoft Office. That the new version of My Space would help damage democracies the world over.
And that programmers would get a reputation almost as bad as bankers ...
And that programmers would get a reputation almost as bad as bankers ...
I think it is more enterpreneurs and startup founders who get all the hate, not so much run of mill programmers.
The thing no one told me is the number of requirements folk, like ux, ba's that don't understand their role in agile development and responsibilities in getting solid requirements across to developers
Lost me at #13. CSS is not a programming language, nor is it complex..
funny how people confuse being told something and integrating it. We're "told" millions of things (more exactly we read all day long), some of it sticks some of it doesn't, it has nothing to do with what we believe is important later in life.
I think we have to stop with those things I wish i was told/nobody told me/I would told my younger self. We just decide some points are salient with experience, and I'm pretty sure the list is different after every decade in life.
I think we have to stop with those things I wish i was told/nobody told me/I would told my younger self. We just decide some points are salient with experience, and I'm pretty sure the list is different after every decade in life.
I'm working in Drupal 8, which makes rule 17 = true.
How about: one's abilities count for about 10%, and skill level in high school politics the other 90%.
Article full of hyperboles, without-proof points and self-pity. Doesn't bring anything useful.
That people who write all those cool blogs/talks and people who write good code or manage projects in great way are different in important ways. Meaning what is good tech/code and what is hip on blogs and forums are two distinct groups. Meaning what is good management is completely different then both feel good and look at me I tough bs you find on blogs.
"That my gender or my age or my ethnicity or my sexual orientation or my weight or my clothes might (will!) have an impact on the perceived quality of the software I build."
After nearly three decades writing software, I'd say if that's the case, you're probably at the wrong place. With such circumstances I'd suspect a whole lot of other bad things are probably true, too.
After nearly three decades writing software, I'd say if that's the case, you're probably at the wrong place. With such circumstances I'd suspect a whole lot of other bad things are probably true, too.
I interpret this comment as saying: if you find yourself in that kind of situation, look around and try to find other opportunities. In other words, life is too short.
I also can see the other side: life is often not fair, particularly in the short-run. Life constraints such as geography are real barriers for many people.
I also can see the other side: life is often not fair, particularly in the short-run. Life constraints such as geography are real barriers for many people.
You are not only an excellent interpreter, but an astute reflector.
How about: When searching for a new job, your previous years of experience, github repos, personal projects, consulting gigs and all other relevant information about your ability to produce quality software will not matter at all compared to your ability to do leetcode style programming challenges.
"America lacks top notch engineers" - Says every large tech company. It didn't dawn on me until now that, "top notch engineers" really just means "people that are good at passing algorithm interviews". So many people I work with can code every algorithm on leetcode in a jiffy but can't write quality software that works.
The "tech shortage" in a nutshell.
That depends on the company you are wanting to work at, though. I refuse to work at companies that give those kinds of bullshit puzzle interviews.
Isn't this all of them?
> I refuse to work at companies that give those kinds of bullshit puzzle interviews.
Good luck finding a job then. All companies of all sizes and locations do this.
Good luck finding a job then. All companies of all sizes and locations do this.
I don't apply to many jobs (I've got a tolerable one and am picky about location and other things), but the one startup tech job I applied to didn't do puzzles, they did do a screenshared coding portion, but it wasn't a puzzle kind of thing.
Um, no, they don't. I know of at least one exception - the one I work at. And I'm part of the interviews for our team, so I know how they're done.
Ah ok. I 've been interviewing bunch of places recently. All of them seem to have identical format.
I remember there a site I saw here HN that listed companies without whiteboard interviews. Can't think of the name.
I remember there a site I saw here HN that listed companies without whiteboard interviews. Can't think of the name.
Oh, no, we do whiteboard interviews. But they aren't those kind of stupid puzzles. We just have candidates write a little code, and do a little design. The "write a little code" is not one of those leetcode questions - it's really a simple problem. We just want to watch them code, and to hear what they're thinking as they do it.
This is the way to do it. How an applicant applies process to problem solving and communicates their ideas, is way more important than being able to regurgitate bubble-sort.
> previous years of experience, github repos, personal projects, consulting gigs and all other relevant information about your ability
As far as evidence for a hiring decision goes, all of the things you listed are things that can either be easily faked, are hard to verify, or require implicitly trusting another organization. How am I supposed to spend the time reading all the code on your github account and forming a judgment of your ability based on that? Even if I do that, how can I be sure that you're the one who wrote that code? How do I know that your previous employers and clients weren't all bozos or that you didn't slack off and let your teammates do all the work? Even if I call your references, they'll probably just say, "yes, this guy definitely worked here, and by HR policy I'm not allowed to say more". Even if they do say more, how am I supposed to trust them? There's not nearly enough information there to make a hiring decision.
As far as evidence for a hiring decision goes, all of the things you listed are things that can either be easily faked, are hard to verify, or require implicitly trusting another organization. How am I supposed to spend the time reading all the code on your github account and forming a judgment of your ability based on that? Even if I do that, how can I be sure that you're the one who wrote that code? How do I know that your previous employers and clients weren't all bozos or that you didn't slack off and let your teammates do all the work? Even if I call your references, they'll probably just say, "yes, this guy definitely worked here, and by HR policy I'm not allowed to say more". Even if they do say more, how am I supposed to trust them? There's not nearly enough information there to make a hiring decision.
> How am I supposed to spend the time reading all the code on your github account
Easy - spend the 45 minutes that you had allotted for whiteboard challenges.
> how can I be sure that you're the one who wrote that code?
You can't be, if the Github account has a single repo with a single commit. But impressive OSS has a history of commits, maybe even a community working with the applicant.
As a random example, do you really think it would be possible for someone to fake an account like this? https://github.com/pedronauck
Even if someone went through the trouble to clone someone else's repo, modify all the commit messages to re-attribute them, and modify the code and commit messages enough to make it not obvious plagiarism, do you really think they could bullshit their way through a quick conversation about the code? Spend 20 minutes digging into it, and then ask pertinent questions about decisions made in the architecture. It'll become clear if this person is the author.
It's theoretically possible that someone could put in all the work to pull off a convincing fraud, but do you really think that's likely?
> How do I know that your previous employers and clients weren't all bozos or that you didn't slack off and let your teammates do all the work?
It's true that there is no perfect signal here, but the odds of this being true are not 50/50. It's not 100% accurate, but having references and employment history is better than nothing.
---
To be clear, I don't think OSS inspection is perfect as a hiring tool - it biases towards folks with the privilege to spend their free time working on passion projects - and I'd object to it being the sole determining factor. But I think the signal is far stronger than it is from whiteboarding, where the false negative rate is super high (I doubt most competent programmers would accurately showcase their skill in this environment).
There is no perfect hiring practice, but to immediately disregard anything that has even a negligible potential for fraud feels a bit silly to me.
Easy - spend the 45 minutes that you had allotted for whiteboard challenges.
> how can I be sure that you're the one who wrote that code?
You can't be, if the Github account has a single repo with a single commit. But impressive OSS has a history of commits, maybe even a community working with the applicant.
As a random example, do you really think it would be possible for someone to fake an account like this? https://github.com/pedronauck
Even if someone went through the trouble to clone someone else's repo, modify all the commit messages to re-attribute them, and modify the code and commit messages enough to make it not obvious plagiarism, do you really think they could bullshit their way through a quick conversation about the code? Spend 20 minutes digging into it, and then ask pertinent questions about decisions made in the architecture. It'll become clear if this person is the author.
It's theoretically possible that someone could put in all the work to pull off a convincing fraud, but do you really think that's likely?
> How do I know that your previous employers and clients weren't all bozos or that you didn't slack off and let your teammates do all the work?
It's true that there is no perfect signal here, but the odds of this being true are not 50/50. It's not 100% accurate, but having references and employment history is better than nothing.
---
To be clear, I don't think OSS inspection is perfect as a hiring tool - it biases towards folks with the privilege to spend their free time working on passion projects - and I'd object to it being the sole determining factor. But I think the signal is far stronger than it is from whiteboarding, where the false negative rate is super high (I doubt most competent programmers would accurately showcase their skill in this environment).
There is no perfect hiring practice, but to immediately disregard anything that has even a negligible potential for fraud feels a bit silly to me.
> But impressive OSS has a history of commits, maybe even a community working with the applicant.
That's going to be a single-digit percentage of the entire developer community and an even smaller minority of developers who go looking for jobs.
> As a random example, do you really think it would be possible for someone to fake an account like this? https://github.com/pedronauck
No, but it would also take a lot longer than an hour for me to judge, from that Github account, whether that guy is a bozo.
> It's not 100% accurate, but having references and employment history is better than nothing.
Which is why they merit the dozen or so man-hours involved in an interview process.
> To be clear, I don't think OSS inspection is perfect as a hiring tool - it biases towards folks with the privilege to spend their free time working on passion projects - and I'd object to it being the sole determining factor.
There are even more flaws than that. Unless you only hire people who make a lot of OSS contributions, you need some other measurement. And it's fairer and more accurate to use the same measurement for everyone, so if you need some other measurement anyway, you should just use it.
Another issue is that OSS contributions are a good signal if and only if you are hiring people to do the same type of work. If you're investing in a specific open-source project and you want to hire the primary maintainer to maintain that project, fine. Otherwise, you have to rely on finding someone who has a rich OSS contribution history in the same basic stuff you're hiring them to do, which is really hard when e.g. most OSS is on the platform level and you're developing products, or if you want to hire generalists rather than specialists.
That's going to be a single-digit percentage of the entire developer community and an even smaller minority of developers who go looking for jobs.
> As a random example, do you really think it would be possible for someone to fake an account like this? https://github.com/pedronauck
No, but it would also take a lot longer than an hour for me to judge, from that Github account, whether that guy is a bozo.
> It's not 100% accurate, but having references and employment history is better than nothing.
Which is why they merit the dozen or so man-hours involved in an interview process.
> To be clear, I don't think OSS inspection is perfect as a hiring tool - it biases towards folks with the privilege to spend their free time working on passion projects - and I'd object to it being the sole determining factor.
There are even more flaws than that. Unless you only hire people who make a lot of OSS contributions, you need some other measurement. And it's fairer and more accurate to use the same measurement for everyone, so if you need some other measurement anyway, you should just use it.
Another issue is that OSS contributions are a good signal if and only if you are hiring people to do the same type of work. If you're investing in a specific open-source project and you want to hire the primary maintainer to maintain that project, fine. Otherwise, you have to rely on finding someone who has a rich OSS contribution history in the same basic stuff you're hiring them to do, which is really hard when e.g. most OSS is on the platform level and you're developing products, or if you want to hire generalists rather than specialists.
This strategy usually falls apart for two reasons:
1. Some people are really good talkers and some people are really bad talkers, especially in an interview setting. That said, all the software companies that people usually complain about do have part of the interview process dedicated to past experience, more the more senior you are.
2. It doesn't scale. To ask really pertinent questions that can trip up a good talker you usually need someone with a lot of experience. But you also don't want them to spend all day doing interviews. You need some type of interview that more junior employees can collect signal from.
1. Some people are really good talkers and some people are really bad talkers, especially in an interview setting. That said, all the software companies that people usually complain about do have part of the interview process dedicated to past experience, more the more senior you are.
2. It doesn't scale. To ask really pertinent questions that can trip up a good talker you usually need someone with a lot of experience. But you also don't want them to spend all day doing interviews. You need some type of interview that more junior employees can collect signal from.
> It's theoretically possible that someone could put in all the work to pull off a convincing fraud, but do you really think that's likely?
Someone who could do that might be worth hiring anyway.
Someone who could do that might be worth hiring anyway.
> Someone who could do that might be worth hiring anyway.
If you specifically wanted to hire people who were skilled at dishonesty. Dishonesty seems like a disqualifier most of the time, though.
If you specifically wanted to hire people who were skilled at dishonesty. Dishonesty seems like a disqualifier most of the time, though.
I don't write tests. Projects that already had tests, those tests caught a single bug in about 3.5 years of working on that code base.
Yes, 1 bug. We'd have picked it up anyway when actually testing it.
Why do you feel #1 or #16 are actually true?
Am I (and colleagues) a magic snowflake, or are tests a massive waste of time, or is it somewhere in the middle?