I think this depends on the library. There are some libraries that are just used in a handful of services, and in those cases I don't think that wrapping is worth the overhead.
However for some of the core platform functionality depended on by all services, we've seen a lot of value in wrapping because it:
- We can provide a more opinionated interface (a third party library is typically more flexible because it has vastly more variety in its use cases)
- We can hook in our own instrumentation easily (e.g. metrics, logging) or change functionality with config/feature flags
- We can transparently change the implementation in the future
I think we've gone back on forth on this over the years. The rate of new services is decreasing, so I think we have shifted more from lots of very small services (low thousands lines of code) to bigger ones that are more like an entire product (e.g. 100k LoC+). But I wouldn't be surprised if the pendulum shifts back again in the future - there are downsides with the larger services, like greater contention withe other engineers.
This is a potential downside of this architecture. As others have already said, there are mitigations, but fundamentally each edge request is going to be more costly (time or money) to serve than having it served by a single machine/DB.
One of our engineering principals is to avoid premature optimisation, which is possibly one of the reasons our architecture has grown in this way. So far, whenever we've needed to fix a performance issue we've been able to solve it locally rather than change the architecture.
At the business level, we've been optimising for growth rather than costs, but this could change in the future, at which point we may need to reconsider our architecture. But for now it's working for us.
I think the other responses have explained the definition I had in mind for "anti-value".
"Frugality" is a bit less explicit than "move fast _and break things_. I think the reason it could be considered an anti-value is because it implies obvious _costs_. For example it can cost time in "shopping around", or it could mean that you miss out on opportunities - for example missing out on a great employee because you pay below market rate.
Thank you for raising the point about contradictions in values. I tried to work it into the original blog post, but I felt like it detracted from my main point.
I previously felt these contradictions are problematic because they make it harder to use values for prioritisation. It's a good point you make about different values being applicable in different contexts. I wonder whether the context can be incorporated into the value?
For example "Bias for action when the cost of failure is low". In an oncall incident, restarting a stateless service is worth trying even before the problem is understood in depth, because risk of failure is low. There are potential actions in an oncall incident that could quite easily make things much worse - then it's probably worth diving deeper before taking action.
It might not be pithy enough for the value itself, but I think it's at least adding this kind of context to the subtext like Amazon have done in the page you linked to.
I had a go at doing roughly the same thing a couple of years ago [0]. It was mostly straightforward, but the part I really struggled with were rendering the sea. The encoding of OSM coastline is quite quirky [1]. When the edge of the rendering intersects with the coastline it's very tricky to compute which side of a coastline is sea and which is land. As an example - how would you render this [2]?
I wasn't sure how I could solve this, so I wrote up a more abstract formulation of the problem here [3], and asked for help. I think the proposed solution make sense, but I think I would have to implement it by rendering individual pixels and wouldn't be able to lean on a higher level graphics library.
I'm looking forward to seeing how the author solved this problem.
I see this sentiment a lot in the Go community. I think it is reasonable in some cases, but there are many use cases (vanilla CRUD web apps) where a web framework is really helpful.
The standard library is very low level. Want sessions? DIY. Want user auth? DIY. Want CSRF protection? DIY. The list goes on.
It feels like a waste of time implementing these "solved problems" from scratch, but the biggest problem is how easy it is to introduce security vulnerabilities when implementing from scratch, or forgetting to do so.
It’s nice to learn concepts from first principals by using the standard library. But once I know how these things work, I’d rather rely on someone else’s battle tested code and best practices.
Yes, you can add in separate libraries to solve these specific problems, but they are less likely to compose as well as they would in a framework. On top of this, each time you pull in a new library you have to spend time evaluating it. When I use a framework I don't have to think.
However for some of the core platform functionality depended on by all services, we've seen a lot of value in wrapping because it:
- We can provide a more opinionated interface (a third party library is typically more flexible because it has vastly more variety in its use cases) - We can hook in our own instrumentation easily (e.g. metrics, logging) or change functionality with config/feature flags - We can transparently change the implementation in the future