I touched on this a bit up thread, but I just want to note that my intention wasn't to get anyone to "do SMURF correctly". My goal was to create an idea to compete with the "Test Pyramid" which, while a useful guide in an environment with limited or no testing, didn't lead to productive conversations in an organization with a lot of tests.
My hope is that this little mnemonic will help engineers remember and discuss the practical concerns and real world tradeoffs that abstract concepts like unit, integration, and E2E entail. If you and your team are already talking about these tradeoffs when you discuss how to manage a growing test suite, then you're you will likely find this guidance a bit redundant, and that's fine by me :)
After 10+ years working on testing practices inside Google, I have found that even the most obvious practices somehow get ignored or misunderstood. As with a lot of programing practices, for every person that has thought deeply about why the practices exists, there exist many many more who just apply the practice as a matter of course (eg mocking, dependency injection, micro services, etc).
It might be useful to provide a little more context for why I wanted to write this in the first place - Over the last 15 or so years we have been tremendously successful at getting folks to write tests. And like any system, once you remove a bottleneck or resource constraint in one place, you inevitably find one somewhere else. In our case we used to take running our tests for granted, but now the cost of doing so now has actual cost implications that we need to consider. I also observed some in internal discussions that had become a little to strident about the absolutes of one kind of test or another, and often in such a way that treated terms like "unit" or "integration" as a sort of universal categories, completely ignoring the broad, practical implications we have bound together into a few shorthand terms.
My goal when trying to develop this idea was to find a way to succinctly combine the important set of tradeoffs teams should consider when thinking, not about a single test, but their entire test suite. I wanted to create a meme (in the Dawkin's sense) that would sit in the background of an engineer's mind that helped them quickly evaluate their test suite's quality over time.
I wanted to drop in and say we had a version of this discussion internally while I was putting this post together. Your observation about fixing a bunch of tests for a simple one line change is something I have seen as well. What we ultimately landed on is that, especially in our service-heavy environment (though not necessarily micro services), the cost of creating and maintaining integration testing infrastructure that is reliable, reasonably fast, and reflective of something prod shaped turns out to be even more expensive. Specifically, we looked at things like the costs of creating parallel auth infra, realistic test data, and the larger, more complex test harness setups and on balance it actually ends up being more expensive on a per-test basis. In fact, in some cases we see meaningful gaps in integration testing where teams have been scared off by the cost.
This isn't to say that unit tests, especially those with heavy mocking or other maintenance issues don't carry their own costs, they absolutely do! But, and I think importantly, the cost-per breakage is often lower as the fix is much more likely to be localized to the test case or a single class. Whereas problems in integration tests or E2E tests can start to approach debugging the prod system.
As with any "experiential opinion" like this, YMMV. I just set out to try to contribute something to the public discourse that's been reflective of our internal experience.
My hope is that this little mnemonic will help engineers remember and discuss the practical concerns and real world tradeoffs that abstract concepts like unit, integration, and E2E entail. If you and your team are already talking about these tradeoffs when you discuss how to manage a growing test suite, then you're you will likely find this guidance a bit redundant, and that's fine by me :)