Ask HN: What's the best way to learn about testing/TDD?
48 comments
Just remember that automated tests only prove that the tests pass; they don't prove that your product works. You still need some amount of manual testing.
What automated tests do, though, is make it significantly easier to test. You will find bugs faster, and dramatically shorten the amount of manual testing that you need to do.
IMO, the best way to learn TDD is via experience. You need to work with a team that's willing to support a junior engineer. From a senior engineer's perspective, the amount of handholding that some junior developers need through writing decent unit tests is quite time consuming.
Thus, if I were you, I'd just be selective about who I work for. I'd look for a team that's willing to be patient while I learn, and that has enough senior developers that I can learn by following a good example. A team that wants a lot of experience in this or that is probably not able to allow you the time needed to learn.
What automated tests do, though, is make it significantly easier to test. You will find bugs faster, and dramatically shorten the amount of manual testing that you need to do.
IMO, the best way to learn TDD is via experience. You need to work with a team that's willing to support a junior engineer. From a senior engineer's perspective, the amount of handholding that some junior developers need through writing decent unit tests is quite time consuming.
Thus, if I were you, I'd just be selective about who I work for. I'd look for a team that's willing to be patient while I learn, and that has enough senior developers that I can learn by following a good example. A team that wants a lot of experience in this or that is probably not able to allow you the time needed to learn.
There is nothing more reliable about manual testing than automated testing. If you are finding that your automated tests are often passing when the product is broken then your approach to testing likely needs more work, this is common when people rely too heavily on unit testing.
A good approach to avoiding this situation is, what is that manual test you do to make sure stuff works? automated it.
A good approach to avoiding this situation is, what is that manual test you do to make sure stuff works? automated it.
> Just remember that automated tests only prove that the tests pass;
They also are code and therefore may contain bugs, impose additional maintenance overhead, and have all the other pros and cons that code has.
Also, it's possible to over-test in the same way it's possible to overengineer.
They also are code and therefore may contain bugs, impose additional maintenance overhead, and have all the other pros and cons that code has.
Also, it's possible to over-test in the same way it's possible to overengineer.
> Just remember that automated tests only prove that the tests pass; they don't prove that your product works.
Doesn't this show that either your test case is not product requirements based or the design is not aligned with the product requirements?
Doesn't this show that either your test case is not product requirements based or the design is not aligned with the product requirements?
It's true, but making sure that the tests perfectly cover the product requirements, and making sure that the specified product requirements actually match the expectations of the product are both barely solvable problems for any sufficiently complex product.
Trying to keep pace with most markets, it's hard for software developers to maintain detailed enough requirements to even hope to ever achieve exhaustive automated testing.
Trying to keep pace with most markets, it's hard for software developers to maintain detailed enough requirements to even hope to ever achieve exhaustive automated testing.
Which design? This weeks, or next months?
> You still need some amount of manual testing
What is it that you're testing manually that you can't automate?
What is it that you're testing manually that you can't automate?
That a human agrees that the product works correctly. These can be silly things like observing that buttons overlap, or observing misspellings.
"What is it that you're testing manually that you can't automate?" I can't automate what I can't predict.
Simple example: Would you get in a robotic car that only had automated testing? What about a robotic flying car?
Real example: 20 years ago, I bought a modem that had an "obvious" problem when calling a dial-up BBS. It didn't flush the buffer appropriately, leaving me with an incomplete screen of text.
I'm sure the modem passed all of their automated tests! But, did whoever wrote the automated tests predict that the buffer should flush if no data came within a very small amount of time? Nope!
Another example: Have you recently used Netflix on Android with Chromecast? It breaks in a lot of corner cases when you put your phone in your pocket. Netflix proudly laid off their manual testers and only does automated testing. Can you really automate every aspect of the Chromecast in a test harness? I bet you can come close, but not achieve 100%.
"What is it that you're testing manually that you can't automate?" I can't automate what I can't predict.
Simple example: Would you get in a robotic car that only had automated testing? What about a robotic flying car?
Real example: 20 years ago, I bought a modem that had an "obvious" problem when calling a dial-up BBS. It didn't flush the buffer appropriately, leaving me with an incomplete screen of text.
I'm sure the modem passed all of their automated tests! But, did whoever wrote the automated tests predict that the buffer should flush if no data came within a very small amount of time? Nope!
Another example: Have you recently used Netflix on Android with Chromecast? It breaks in a lot of corner cases when you put your phone in your pocket. Netflix proudly laid off their manual testers and only does automated testing. Can you really automate every aspect of the Chromecast in a test harness? I bet you can come close, but not achieve 100%.
My tests usually involve common expected/actual cases. Manual testing can uncover "unique" scenarios where my code can break.
Manual testing can discover bugs. But the first step toward debugging should be writing an automated test to exercise the manually discovered bug, imho.
That said, bug-driven testing is a separate issue from TDD.
That said, bug-driven testing is a separate issue from TDD.
Any advice on finding such companies?
Treat your interview as a two-way street. You should interview the company as much as they are interviewing you.
Ask questions about their testing approach. Remember, no company is perfect. Often, small startups decide it's in their best interest to rely on manual testing and accept that their products are buggy. This can be an opportunity if you're able to clean things up, though.
If a company is honest, and takes the time to answer your questions, then it's probably a good sign. If the company thinks you should be grateful for a job and grovel for whatever crumbs they drop, RUN!
Remember, though, no company is perfect. It's better to work for a company that has an imperfect TDD then one that treats you like dirt.
Ask questions about their testing approach. Remember, no company is perfect. Often, small startups decide it's in their best interest to rely on manual testing and accept that their products are buggy. This can be an opportunity if you're able to clean things up, though.
If a company is honest, and takes the time to answer your questions, then it's probably a good sign. If the company thinks you should be grateful for a job and grovel for whatever crumbs they drop, RUN!
Remember, though, no company is perfect. It's better to work for a company that has an imperfect TDD then one that treats you like dirt.
It's hard. Because everyone claims to be doing TDD, just asking in the interview "do you do TDD" is pointless.
Example. Last year I consulted at an IC company, building a huge "moon-shot" chip. You might expect TDD to be an absolute certainty in such an operation. Not so, individuals had all sorts of justifications for not doing TDD.
Here is one idea. In the technical interviews, when the inevitable "and do you have any questions for me?" moment comes up, ask them to talk about their best test.
If they have a good answer, it's likely you've found somewhere that genuinely values TDD.
Example. Last year I consulted at an IC company, building a huge "moon-shot" chip. You might expect TDD to be an absolute certainty in such an operation. Not so, individuals had all sorts of justifications for not doing TDD.
Here is one idea. In the technical interviews, when the inevitable "and do you have any questions for me?" moment comes up, ask them to talk about their best test.
If they have a good answer, it's likely you've found somewhere that genuinely values TDD.
I agree with this. I've interviewed a lot of people recently for a senior developer role. The difference in people's ideas of what TDD or testing in general, is quite marked.
I love it when people ask me about how we test. I can tell you the pains we've had with tying our tests too tightly with implementation. :D
I love it when people ask me about how we test. I can tell you the pains we've had with tying our tests too tightly with implementation. :D
Working Effectively with Unit Tests by Jay Fields[1] is a great book on the subject. There's tons of books out there that are also good, but this is one I like to recommend as a starting point.
Another thing I'll do a lot of times when I'm trying to figure out how to test a weird situation is try to think of another API that works similarly and look to see if it has any tests that are instructive. And I habitually go through OSS projects I use to look at their tests and see what techniques I might not know about. This is really useful when you're picking up a new language or paradigm (e.g. when I started using functional languages over the OO languages I was used to.)
The basic thing you need to remember about testing is the AAA approach[2]: Arrange (setup); Act(do the thing you're testing); Assert (measure the outcome to make sure it is what you were expecting). Just about everything you read on testing at a lower level is going to come down to that procedure. There's additional techniques like fuzz testing and property testing, but they're not common. At a junior level I'd just expect you to have a basic understanding of AAA and how to apply it to common situations with the toolkit of your choice (e.g. for a javascript dev, maybe jest).
When you think about it, you already test your software, as you write it—its just that instead of writing down the steps in a way that your test framework can use, you're doing it manually. So really you just need to learn how to write down what you're doing in the browser (or whatever your interface is). Further refinement of that process is to use break your software up into pieces that can be more easily tested on their own.
I really just meant to post the first link, but testing is one of those areas that's near and dear to my heart, so my apologies for the rambling :)
[1] https://www.amazon.com/Working-Effectively-Unit-Tests-Fields...
[2] http://wiki.c2.com/?ArrangeActAssert
Another thing I'll do a lot of times when I'm trying to figure out how to test a weird situation is try to think of another API that works similarly and look to see if it has any tests that are instructive. And I habitually go through OSS projects I use to look at their tests and see what techniques I might not know about. This is really useful when you're picking up a new language or paradigm (e.g. when I started using functional languages over the OO languages I was used to.)
The basic thing you need to remember about testing is the AAA approach[2]: Arrange (setup); Act(do the thing you're testing); Assert (measure the outcome to make sure it is what you were expecting). Just about everything you read on testing at a lower level is going to come down to that procedure. There's additional techniques like fuzz testing and property testing, but they're not common. At a junior level I'd just expect you to have a basic understanding of AAA and how to apply it to common situations with the toolkit of your choice (e.g. for a javascript dev, maybe jest).
When you think about it, you already test your software, as you write it—its just that instead of writing down the steps in a way that your test framework can use, you're doing it manually. So really you just need to learn how to write down what you're doing in the browser (or whatever your interface is). Further refinement of that process is to use break your software up into pieces that can be more easily tested on their own.
I really just meant to post the first link, but testing is one of those areas that's near and dear to my heart, so my apologies for the rambling :)
[1] https://www.amazon.com/Working-Effectively-Unit-Tests-Fields...
[2] http://wiki.c2.com/?ArrangeActAssert
I'm glad you rambled! :)
Your comment was very useful in itself aside from the link. Thanks!
Your comment was very useful in itself aside from the link. Thanks!
Obey the Testing Goat (http://www.obeythetestinggoat.com/) will take you through creating a small web app (python/django) using test driven development. The language choice may not match up with what you're using, but the concepts are useful across many domains. This book made a huge impact in how I think about writing tests, and in turn how I think about writing code.
I do not pretend to be any kind of expert, but on our team we needed to teach people a little bit about TDD as it related to Angular using Typescript with and intro to bundling and a whole slew of other things, so I made a video series that covers synch and async fundamentals inside of that framework using Jasmine: https://www.youtube.com/playlist?list=PLr4KpsL6BPn0dzn84zjmx...
There are tons of really good books out there on the subject from Packt too. React using Jest isn't super widely filmed, but has good docs. If you do become a master, create a video series and help the next guy out. =) Happy coding!
There are tons of really good books out there on the subject from Packt too. React using Jest isn't super widely filmed, but has good docs. If you do become a master, create a video series and help the next guy out. =) Happy coding!
On the flipside of alexdrans, I suggest looking into using Airbnb's Enzyme[0] for React and Facebook's Jest[1] for testing React components.
[0]: https://github.com/airbnb/enzyme
[1]: https://github.com/facebook/jest
[0]: https://github.com/airbnb/enzyme
[1]: https://github.com/facebook/jest
Beware that TDD is not testing. TDD is a development process which involves some form of testing, but you can do development and testing without doing TDD.
TDD is all the hype these days, but there are some who are very skeptical about it. For a good discussion, see this: https://www.infoq.com/interviews/coplien-martin-tdd
Since you are junior all this won't probably matter all that much to you - just wanted to make sure you don't put an = sign between these two terms accidentally.
TDD is all the hype these days, but there are some who are very skeptical about it. For a good discussion, see this: https://www.infoq.com/interviews/coplien-martin-tdd
Since you are junior all this won't probably matter all that much to you - just wanted to make sure you don't put an = sign between these two terms accidentally.
This book covers a ton of testing methods https://cs.gmu.edu/~offutt/softwaretest/
The authors have recorded lectures too https://cs.gmu.edu/~offutt/softwaretest/videos/ plus open course slides and assignments http://cs.gmu.edu/~offutt/classes/737/schedule.html
The authors have recorded lectures too https://cs.gmu.edu/~offutt/softwaretest/videos/ plus open course slides and assignments http://cs.gmu.edu/~offutt/classes/737/schedule.html
I'd recommend checking out James Shore's Javascript TDD site: https://www.letscodejavascript.com/
Shore charges $24.95 a month--but you could probably get the core of what you need within a month or two, and for that it's well worth it. I believe there's also a 7-day trial.
Find someone to pair with, particularly someone who's done a bit of TDD before. It'll be a lot quicker to work through.
At some point, as another poster suggested, read some of the debate on TDD. But learn it first, and from those who have succeeded with it in a professional capacity--so you can be better educated about where its value lies. It's a valuable tool, but no tool solves all problems. (While I currently teach/pair with devs on TDD, I spent at least 5 years doing TDD on professional, "real" systems.)
Find someone to pair with, particularly someone who's done a bit of TDD before. It'll be a lot quicker to work through.
At some point, as another poster suggested, read some of the debate on TDD. But learn it first, and from those who have succeeded with it in a professional capacity--so you can be better educated about where its value lies. It's a valuable tool, but no tool solves all problems. (While I currently teach/pair with devs on TDD, I spent at least 5 years doing TDD on professional, "real" systems.)
Hey soneca,
I'm in the process of writing a course about JavaScript testing so it's good timing. I write a lot of unit and integration tests for our large codebase here at the BBC in the UK. We've got 3 Node + Express applications plus React for the UI for the 2 user facing applications. The other one is an API but it's got solid tests as it has does some complicated logic for redirecting users and updating sessions and tokens.
I'd love to get you on the mailing list for when I launch it and then you can fire me some questions via email. I'd love to help you get that job!
http://www.marclittlemore.com/courses/javascript-testing-mad...
Cheers, Marc
I'm in the process of writing a course about JavaScript testing so it's good timing. I write a lot of unit and integration tests for our large codebase here at the BBC in the UK. We've got 3 Node + Express applications plus React for the UI for the 2 user facing applications. The other one is an API but it's got solid tests as it has does some complicated logic for redirecting users and updating sessions and tokens.
I'd love to get you on the mailing list for when I launch it and then you can fire me some questions via email. I'd love to help you get that job!
http://www.marclittlemore.com/courses/javascript-testing-mad...
Cheers, Marc
done! Good luck with the course!
Thanks! You should have got an email so you can reply to me. Let me know any questions and I'll get back to you today.
I worked in the videogames industry for nearly 20 years and they didn't do much unit/integration testing but I've immersed myself in learning about testing JS for the past 4 years since I switched industries. Hardest thing is knowing what to test I always found.
I'm happy to do some 1-on-1 with you to help.
I worked in the videogames industry for nearly 20 years and they didn't do much unit/integration testing but I've immersed myself in learning about testing JS for the past 4 years since I switched industries. Hardest thing is knowing what to test I always found.
I'm happy to do some 1-on-1 with you to help.
This is also a pretty solid resource which I saw on Medium recently. A good overview of the JS testing landscape at the moment.
https://medium.com/powtoon-engineering/a-complete-guide-to-t...
https://medium.com/powtoon-engineering/a-complete-guide-to-t...
[deleted]
I read Kent Beck's book on the topic a long time ago and concluded, at the time, that it was bullshit. It just seemed ridiculous.
(I got to tell him this anecdote about a year ago. He was amused).
Much later I joined Pivotal Labs, hoping to see what the fuss was about, and got taught TDD by sitting next to people who are good at it. People who will go to enormous lengths to create automated testing from the outside in. It felt uncomfortable and unnatural at first. But now it seems obvious.
My advice is: seek out a mentor. Anyone who can sit with you for a few hours.
(I got to tell him this anecdote about a year ago. He was amused).
Much later I joined Pivotal Labs, hoping to see what the fuss was about, and got taught TDD by sitting next to people who are good at it. People who will go to enormous lengths to create automated testing from the outside in. It felt uncomfortable and unnatural at first. But now it seems obvious.
My advice is: seek out a mentor. Anyone who can sit with you for a few hours.
Sorry I can't contribute more, this needs to be a quick message; but I recommend that you take a look at substack/tape⁰ for testing in JS; browser and server side. It forces you to write 'simple' tests; the likes of Mocha give you too much fluff, and use globals (i.e. magic).
[0] https://github.com/substack/tape
[0] https://github.com/substack/tape
A TDD-specific observation, and not really an "introduction"...
I love TDD because it makes me feel more comfortable as a programmer. The cycle of "write a test, test fails, write code, test passes, write another test" helps keep me centered. I know what I'm supposed to be doing at that moment, and don't get lost.
I don't think I write faster, and I may not write more bug-free. But I sure feel better with TDD.
I love TDD because it makes me feel more comfortable as a programmer. The cycle of "write a test, test fails, write code, test passes, write another test" helps keep me centered. I know what I'm supposed to be doing at that moment, and don't get lost.
I don't think I write faster, and I may not write more bug-free. But I sure feel better with TDD.
>> I don't think I write faster, and I may not write more bug-free. But I sure feel better with TDD.
False sense of achievement maybe ? You write more code, because you also simultaneously write tests. But, have you ever noticed that, while your tests are great, your design sucks ? To me TDD is more a distraction than a tool for helping me think first before coding. I see tests only as a mandatory bureaucratic paperwork that needs to be done at the end of design process. Don't get me wrong, I think tests are a must, but only for ONE reason, to prevent software regressions. TDD for me is very pointless if not a dangerous practice.
False sense of achievement maybe ? You write more code, because you also simultaneously write tests. But, have you ever noticed that, while your tests are great, your design sucks ? To me TDD is more a distraction than a tool for helping me think first before coding. I see tests only as a mandatory bureaucratic paperwork that needs to be done at the end of design process. Don't get me wrong, I think tests are a must, but only for ONE reason, to prevent software regressions. TDD for me is very pointless if not a dangerous practice.
TDD is awful if you expect all of the tests you produced during the process to stay static. With care, you can pare down your test suite, or add additional tests, to be appropriate as a regression suite for the software each time it is iterated to a deliverable state.
I actually find TDD helps me design components with fewer responsibilities, almost always as a result of realizing the complexity via writing permutations of test cases. It helps me think about 'what each thing should do' because I don't have that innate ability like some folks.
It is especially helpful in pointing out cases where adding a layer of abstraction or introducing subclass variants are beneficial.
I actually find TDD helps me design components with fewer responsibilities, almost always as a result of realizing the complexity via writing permutations of test cases. It helps me think about 'what each thing should do' because I don't have that innate ability like some folks.
It is especially helpful in pointing out cases where adding a layer of abstraction or introducing subclass variants are beneficial.
Why is it a false sense of achievement? When I do TDD, I have demonstrably working code. What doesn't give me a sense of achievement is writing a ton of code with no evidence that it works or is bug-free.
Heck, even before coding, I like using Cucumber as a form to think about code behavior and responsibilities. I don't really enjoy fully automated Cucumber in practice, but the format is terrific.
If all you see tests for is "mandatory bureaucratic paperwork", maybe you should be concerned about overconfidence in your ability to do flawless design and implementation. Writing tests is one of the best tools I've found for making sure I actually understand what it is I'm trying to do (and if what I'm trying to do is actually wrong).
Heck, even before coding, I like using Cucumber as a form to think about code behavior and responsibilities. I don't really enjoy fully automated Cucumber in practice, but the format is terrific.
If all you see tests for is "mandatory bureaucratic paperwork", maybe you should be concerned about overconfidence in your ability to do flawless design and implementation. Writing tests is one of the best tools I've found for making sure I actually understand what it is I'm trying to do (and if what I'm trying to do is actually wrong).
Sorry for late reply.
>> If all you see tests for is "mandatory bureaucratic paperwork", maybe you should be concerned about overconfidence in your ability to do flawless design and implementation.
I don't buy into the idea that TDD or any kind of testing methodollogy gives you confidence to do good design or in any way helps you make good design decisions.
>> When I do TDD, I have demonstrably working code. What doesn't give me a sense of achievement is writing a ton of code with no evidence that it works or is bug-free
False sense of program correctness right there ;) Are you telling me that by doing example based testing, you are 100% bug free ? Your tests must be humongous.
>> Writing tests is one of the best tools I've found for making sure I actually understand what it is I'm trying to do (and if what I'm trying to do is actually wrong).
Pen and paper for some and more code for others.
>> If all you see tests for is "mandatory bureaucratic paperwork", maybe you should be concerned about overconfidence in your ability to do flawless design and implementation.
I don't buy into the idea that TDD or any kind of testing methodollogy gives you confidence to do good design or in any way helps you make good design decisions.
>> When I do TDD, I have demonstrably working code. What doesn't give me a sense of achievement is writing a ton of code with no evidence that it works or is bug-free
False sense of program correctness right there ;) Are you telling me that by doing example based testing, you are 100% bug free ? Your tests must be humongous.
>> Writing tests is one of the best tools I've found for making sure I actually understand what it is I'm trying to do (and if what I'm trying to do is actually wrong).
Pen and paper for some and more code for others.
One hard part of learning testing is getting the testing framework set up for your langauge/toolset. One way to approach this is to follow a tutorial that introduces tests. A year ago, I would have suggested https://teropa.info/blog/2015/09/10/full-stack-redux-tutoria..., but react/js moves so fast that perhaps this is out-of-date.
Anyone have online resources for testing C/C++ software? The resources I find online always seem to not cover much.
no online resources that I'd recommend, but I like the book Modern C++ With Test-Driven Development. (I'm a bit biased on this though...)
In the same vein, are there any open source projects that need unit tests? I've searched github for "Unit Test/Testing" but came up with mostly abandoned/old/very small projects.
I'd like to practice writing tests but would prefer if they didn't go to waste, or at least added value to something else.
I'd like to practice writing tests but would prefer if they didn't go to waste, or at least added value to something else.
I am a maintainer of PouchDB - https://github.com/pouchdb/pouchdb/, I probably spend more time working on the tests than any other part of PouchDB. We have a large test suite but still plenty of room to work on improving. If you were looking to work on testing then would be happy to give some guidance.
There's a big difference between writing tests for existing code and TDD. I think it's a great idea to first try and write tests for someone else's existing code so you discover what a pain it is.
https://searchcode.com can be useful in helping find repos based on code criteria.
Were it me, I'd build something small and interesting to me, write some unit tests against it after-the-fact. Then I'd do it again, using TDD the 2nd time to see how the approaches differed.
https://searchcode.com can be useful in helping find repos based on code criteria.
Were it me, I'd build something small and interesting to me, write some unit tests against it after-the-fact. Then I'd do it again, using TDD the 2nd time to see how the approaches differed.
There was an recent article about testing in JavaScript that looked really promising: https://medium.com/powtoon-engineering/a-complete-guide-to-t...
Neal Ford on test driven design... https://www.safaribooksonline.com/library/view/neal-ford-on/...
Sorry not suggesting online resource. But ensure you are using watch like feature so that the execution of tests is automated. This really made TDD smoother for me. Node ex:
something like: `"test": "mocha --watch"` in your package.json.
the Classic seasons of Destroy All Software: https://www.destroyallsoftware.com/screencasts
For testing engineered systems, find someone where extreme reliability is paramount and for whom testing comes naturally so it's their life's work.
Then listen to them.
Then listen to them.
But I noticed that lack of knowledge and practice in testing is hurting my chances. I was assuming that it would be something I would better learn in practice as part of an experienced team in a professional environment and that it was ok for a junior dev not knowing much about it. I changed my mind, so...
What online resources are great as an introduction to testing?