Thank you for explaining how you work, and I would never laugh at a developer for copy/pasting code. We've all done it!
I'm still intrigued to understand exactly how you share code across services. Let's say that you've written a piece of code for logging, which you want to use in both service A and service B. Do you package it up in a library? If so, doesn't that mean you have to place that library in a repository, so both services can access it? Doesn't that mean that if you want to make changes to the logging code that you now have to publish a new version of the library, and remember to update both services to depend on the new version?
That's the friction I'm talking about.
With Polylith, the logging code would live in a component that's directly accessible to all the other components in the system. That's because Polylith lets us work with all our components as if they're a monolith (even if we chose to deploy them as multiple services). This means that when we update the logging component, there's zero friction to update any impacted components in the services.
If the change only affects the logging component's implementation (and not its interface) then no other components need to be updated, and we can just redeploy the system. If it's a breaking change to the interface, then we can immediately fix the impacted components within our monolithic development environment. If the change is a refactor of the logging component's interface, then the other components will be automatically updated by our refactoring tool!
Hopefully that explains how Polylith solves this challenge so elegantly.
Thank you for your comprehensive and detailed feedback!
I'll leave Joakim to respond to your comments about his presentation, and I'll respond directly to your comments about the nutshell video.
> you've actually never demonstrated your product to me once
That's partially true, although I'd argue that by showing the components and bases from an example system, we are showing "the product". Perhaps you're thinking about showing code or filesystem structure from a complete production Polylith workspace? We do that in other videos, but didn't think it would be a good fit in the 10-minute "why?" introduction.
> The toothpick model isn't a mess, it's a masterpiece. I admire the person who made it, but you're telling me that I shouldn't want to build software with that model.
I agree that the toothpick model is a masterpiece! However, the reason we shouldn't want to build software in the same way comes down to one word: change. The real Whitehouse doesn't change shape very often, but all the software I've ever been involved in building does. That's why building software with LEGO pieces is so much better than building it with toothpicks and glue. Which is exactly why it's a such strong metaphor.
> Secondly, the Lego analogy is overdone. As I said in another post, every architecture has made this claim, even if they are actually toothpicks.
You might be right about this, but most of the other recent architectures we'd come across seemed to steer away from talking about LEGO (onion, hexagon, DDD, microservices, etc.), so we were hoping it wasn't overused.
I'm not sure we're understanding each other. When I asked "How does it solve the problem of sharing code across service boundaries?", I meant how does your language or architecture enable you to share code across separately deployed artefacts, for example across two microservices.
There are three approaches that I'm aware of for solving this problem: 1) copy/paste the shared code between the two repositories, 2) freeze the code into a library that both services depend on, 3) keep both services in one repository, extract the shared code into a module or component, both services depend on the shared module, deploy the services as separate artefacts.
1) is bad for obvious reasons, 2) adds unnecessary friction to the development process, and 3) is how Polylith solves it.
I was wondering if you've come across another way to achieve 3), or perhaps a fourth approach?
Technically, a Polylith interface is a collection of "pass-through" functions, each of which delegate their function call to an "implementation" function within the component.
I agree with you that many languages have syntax to support building interfaces. However, I've yet to come across one that offers all the benefits of Polylith's approach with components.
That's right, it's a highly opinionated approach to building software, which comes with a host of benefits.
Including my favourite; a complete untangling of your development and production environments. With Polylith you always develop your system as a monolith (because that's the most effective way to build software), but you're able to deploy it as multiple services (because that's sometimes the most effective way to run software). It turns out that separating deployment complexity from development complexity is a game-changer, and something that I haven't come across from other architectures.
It's true that you don't reap all the benefits of Polylith until your entire codebase uses the same structure, which feels a bit like "all or nothing". However, many of the benefits are unlocked "as you go", so even converting one or two existing microservices to Polylith will feel like a nicer codebase to work with.
> so I shall simply wish you good luck with that part
I was confused because you were replying to a comment which mentioned the "long video", so I assumed you were both talking about the forty-minute one on the homepage of the documentation. My mistake.
Did my response to `cjg` in this thread help to clarify anything for you?
The idea with that part of the video was to couch the new concept (components) within the framework of existing concepts (functions, objects, layers, etc.) to try to help people connect the new concept at the right place in their knowledge graph. Though your feedback makes me think it didn't work as well as I hoped.
Components do have attributes in common with microservices and with stateless objects (e.g. a public interface and encapsulated implementation).
Where components differ from microservices is that a component's interface is simply a collection of functions, rather than network-facing endpoints. This means that multiple components can be deployed into a single artefact, keeping deployment complexity and cost down.
Where components differ from objects is that a component is a higher-level abstraction, closer in scope to a microservice.
However, we think that Polylith's biggest differentiator is the separation it gives between development and production. Let's say you have a Polylith project with 100 components, that you deploy in production across 10 services. You can work with all 100 components in a single development environment, and test them as though they're a monolith, even though they're not deployed that way in production. It's a lot like building systems with LEGO, and we think its just as fun!
Your understanding of Polylith is essentially correct. It is a combination of existing ideas, such as monorepos, convention, components, interfaces, encapsulation, and static code analysis. However, I'd argue that what emerged in Polylith from those ingredients was more than the sum of its parts - due to how those particular concepts resonate with each other.
For example, because of the forced conventions, it's trivial for the Polylith Tool to perform static code analysis on a Polylith codebase. This means that the tool can identify the subset of tests to run, based on the components that have changed since the last run. Which leads to a fast-feedback loop, and encourages both good testing practices and fine-grained component modularity.
Polylith gives you a system-level architectural building block; the component, which encourages a modular design and separation of concerns. However, you're right that it's still possible to create spaghetti code with Polylith. All it would take is poorly designed components, with bad names, multiple reasons to change, and exposing their state everywhere. However, I'd argue that when you give someone a well designed tool (like Polylith), then they're more likely to craft a well designed product with it.
To understand how builds and deployments work with Polylith, I'd recommend reading the "Workflow" section in the Polylith Tool's documentation: https://polylith.gitbook.io/poly/workflow/shell (especially "Build", "Git", "Continuous Integration", and "Testing").
It's tricky to pick the right level of detail when explaining an idea like Polylith. If we could precisely identify the level of understanding of the reader, then we could tailor the content to perfectly match their experience level and vocabulary. Unfortunately, we can't do that, so we have to pick a middle ground, where we probably over explain some concepts, and under explain others.
However, we'd be grateful if you could give us some specific examples of sections or pages that you didn't find useful or enlightening.
Before you completely give up on understanding the concept, I would like to suggest you try the 10-minute "Polylith in a Nutshell" video: https://www.youtube.com/watch?v=Xz8slbpGvnk
If you watch it at 1.5x speed, then it'll take less than 7 minutes of your time.
If you're still not getting the "why" of Polylith after this video, then I'd be very grateful if you could give us some quick feedback on what you're missing. That will help us figure out how to explain the concepts better in the future.
Thanks! It was a bit nerve-racking to give traffic light ratings to existing architectures that people use and enjoy, but I hope we were fair in our assessment.
Thanks for the feedback, and I'm glad to hear that you understood the concept from the text documentation.
We decided to include both videos and text in the documentation because we know that people are different, and have different preferences for learning.
You're right that people are opinionated, and that it's difficult to convince them to try new ideas. However, I've been pleasantly surprised at how open the Clojure community has been to our concept and at the momentum it's starting to build.
It'll be interesting to see if other language communities have the same mindset.
I'm a member of the team that created Polylith, so feel free to ask questions or give feedback.
Polylith has been gaining momentum within the Clojure community. As an example, you can follow along with Sean Corfield's journey migrating the World Singles codebase to Polylith: https://corfield.org/blog/2021/02/23/deps-edn-monorepo/
Polylith is, first and foremost, an opinionated file structure for codebases (similar to how a framework forces you to structure your code in a particular way). Polylith's structure gives a host of benefits for the modularity, composability, testability, and "growability" of the codebase, and for giving a near friction-free development experience when working with your code in an editor/IDE.
One of the key benefits is how it separates development from deployment, giving you the freedom to work with your code as though it's a monolith, but choose to deploy it as any number of services.
To make the development experience even more delightful, we've also built a tool (currently only supporting Clojure, but with nothing stopping the development of tools for other languages) which gives instant creation of new components and bases, incremental tests (only test the code that's impacted by the last change), build support, and project visualisation.
So to answer your question, there's definitely some code rewriting involved to convert to Polylith from an existing codebase, but usually the steps involved are quite clear, and the outcome will be a codebase that we predict you'll be delighted to work with!
We've simplified the architecture, by reducing the number of concepts down to just three: components, bases, and projects. The tool is simpler, gives faster feedback, and a better visualisation of your projects, due to (amount other design changes) switching from Leiningen to tools.deps.
The original version of the tool used:
- Leiningen to compile and build the artefacts
- symlinks to keep the code in one place (but allow for reuse in multiple artefacts)
- "workspace interfaces" (empty component interfaces) to guarantee the decoupling between components
The new version of the tool uses:
- tools.deps to build the artefacts and remove the need for symlinks
- static code analysis to remove the need for "workspace components"
I'm still intrigued to understand exactly how you share code across services. Let's say that you've written a piece of code for logging, which you want to use in both service A and service B. Do you package it up in a library? If so, doesn't that mean you have to place that library in a repository, so both services can access it? Doesn't that mean that if you want to make changes to the logging code that you now have to publish a new version of the library, and remember to update both services to depend on the new version?
That's the friction I'm talking about.
With Polylith, the logging code would live in a component that's directly accessible to all the other components in the system. That's because Polylith lets us work with all our components as if they're a monolith (even if we chose to deploy them as multiple services). This means that when we update the logging component, there's zero friction to update any impacted components in the services.
If the change only affects the logging component's implementation (and not its interface) then no other components need to be updated, and we can just redeploy the system. If it's a breaking change to the interface, then we can immediately fix the impacted components within our monolithic development environment. If the change is a refactor of the logging component's interface, then the other components will be automatically updated by our refactoring tool!
Hopefully that explains how Polylith solves this challenge so elegantly.