Cloudflare definitely isn't with their Durable Object design. As an infra developer/Django user myself, I know it's a pretty big paradigm shift to think in the 1 DB per user realm.
However, I do think it'll unlock "globally available" apps as a first class citizen moving forward. If Uber were made today, I suspect it'd follow the 1 DB per user and be deployed really close to the edge with providers like Fly.io, Koyeb, or Cloudflare.
We had python scripts that generated GitLab CI/CD yaml [1]. Tooting my own horn here, but it was super cool to ship fairly fast for the first year or so. By the end, we had something like 5 MB of yaml, but in order for the GitLab SaaS backend to process it, it took something like 32 gigs of ram on their MergeRequestProcessor SideKiq worker.
They had to open a whole epic in order to reduce the memory usage, but I think all that work just let us continue to use GitLab as the number of services we grew increased. They recommended we use something called parent/child pipelines, but it would have been a fairly large rewrite of our logic.
I built something like this at my previous startup, Pangea [1]. Overall I think looking back on our journey I'd sign up for it again, but it's not a panacea.
Here were the downsides we ran into
- Getting buy in to do everything through the repo. We had our feature flags controlled via a yaml file in the repo as well, and pretty quickly people got mad at the time it took for us to update a feature flag (open MR -> merge MR -> have CI update feature flag in our envs), and optimizing that took quite a while. It then made branch invariants harder to reason about (everything in the production branch is what is in our live environments, but except for feature flags). So, we moved that out of the monorepo into an actual service.
- CI time and complexity. When we started getting to around 20 services that deployed independently, GitLab started choking on the size of our CI configuration and we'd see a spinner for about 5 minutes before our pipeline even launched. Couple that with special snowflakes like the feature flag system I mentioned above, eventually it got to the point that only a few people knew exactly how rollouts edge cases worked. The juice was not worth the squeeze at that point (the juice being - "the repo is the source of truth for everything")
- Test times. We ran some e2e UI tests with Cypress that required a lot of beefy instances, and for safety we'd run them every single time. Couple that with flakiness, and you'd have a lot of red pipelines when the goal was 100% green all the time.
That being said, we got a ton of good stuff out of it too. I distinctly remember one day that I updated all but 2 of our services to run on ARM without involving service authors and our compute spend went down by 70% for that month because nobody was using the m8g spot instances, which had just been released.
Does anyone who's participated in M&A know how they come up with a breakup fee? I believe this one is $5 Billion (per Bloomberg), and Adobe <-> Figma was $1 Billion.
Interested to understand the modeling that goes into it.
This is what we built at my last startup Pangea (except with GitLab CI/CD). It worked pretty well and I always wanted to open-source it, but alas in a startup there's never enough time.