thanks for the post. I always found this site a good place to start[0] and a review of the various approaches[1] (disclaimer: I took part in writing that one)
There's an infrastructure engineer, or engineers that support DevOps operations. But DevOps by definition was about Devs doing more Ops, having an engineer 'do it for you' never made sense. 'DevOps engineer' was an unfortunate rebranding of infra engineers to maintain the status quo.
What would it take to create a runtime that bridges programming language ecosystems? A runtime that lets us import libraries from any language and use them idiomatically (as much as possible) with a straight forward DX? (check out the url for a simple example)
Hey friends! I recently published a blog post discussing common infrastructure reorganization models that companies adopt and why they often fail. After interviewing dozens of engineering leaders and infrastructure teams over the past two years, I've noticed companies tend to evolve their infrastructure orgs into a few canonical forms with distinct tradeoffs.
The post covers:
- 4 primary infrastructure org models
- Reasons companies consolidate into centralized platform teams or decompose them
- Tradeoffs, failure modes and pitfalls when reorg-ing infrastructure
- A new approach porposal for leveraging cloud intelligence to empower developers
Rather than continually rearranging org boundaries and responsibilities, I argue that we need to shift the focus to tools that bring the benefits of platform orgs into infrastructure orgs.
Would love to hear what you think! What infrastructure org structures have you seen succeed or struggle?
That's what we do with Klotho [0], annotations are only used to close behavioral gaps that are not available in application code. (I'm one of the founders)
"The primary reason to introduce a new example is when the LLM is incorrectly identifying technology as ABSTRACT vs not, missing connections, or even missing resources entirely. However, we don’t have unlimited tokens for every example we might need. An optimization we came up with here is Dynamic Examples using pre-filtering. Instead of providing examples that would generalize to everything, we focus the examples on what we can guess is in the queries.
We extract a list of technologies using word lists, which is easier than extracting their intents, and if we don’t find many matches, we assume that more ABSTRACT resources are present. Once extracted, we can then create a custom prompt by selecting specific examples to the technologies mentioned in the query and a set of bedrock examples including baseline rules for the different actions that expand language understanding."
I led a large (50 people+) platform org - Organizations struggle with modern backend systems and cloud, and find themselves in a cyclical loop of hiring, reorganization and reprioritization. I just wrote about it here a few days ago: https://klo.dev/evolution-of-cloud-infrastructure-and-platfo...
thanks for the shoutout @shaiber! We're fans of all those projects at Klotho, as we're all going after raising the abstraction layer in different ways. For those interested in Infra-from-Code I put together a state of IfC review a while back (https://klo.dev/state-of-infrastructure-from-code-2023/ and talk https://www.youtube.com/watch?v=Yx4eM9Sh-fY) that covers alot of the different approaches taken
InfraCopilot is built on Klotho which is designed on top of an adaptive architectures[0] paradigm - meaning every element in the internal representations is interchangeable. However if you think of an architecture as a puzzle, changing one element to another doesn't always just 'click in' - so the Copilot engine propagates the changes into the adjacent puzzle pieces and continues to reshape them until everything clicks again, ensuring that the result is valid. If it doesn't know how, it'll let you know, and why.
Right now the architectural patterns are curated, but algorithmically tested. The next phase is to combine curation with patterns from the community.
We use OpenAI/GPT for NLP only - parsing intent - oversimplifying but normalizing things like "I want a serverless function that connects to RDS" is translated to a JSON schema that says serverless <-> RDS. That's then passed to the InfraCopilot engine which then unpacks what that means into a larger architectural representation (we have an Architecture-as-Code blog post coming soon).
AWS Application Composer[0], Brainboard[1] and Cloudmaker[2] are a 1:1 mapping tool between a visual representation and an infra-as-code representation. They're more similar to Scratch[3]. You still need to know what to place to get an outcome you're interested in - If you want to set up a globally low latency static web site, or connect an RDS to a Lambda, you still need to figure out CloudFront, custom domains, VPCs, RDS Proxy, draw it all in, etc. And if you were to change RDS into DynamoDB, you'll have to manually undo/reshape the diagram/topology/IaC because some elements no longer make sense or are needed
InfraCopilot is more akin to Wolfram Alpha, in the sense that it has an intelligence/understanding of architecture. You can use high level design to describe your intent and requirements/constraints, and it will deterministically implement it (this isn't LLMs or ChatGPT). When you attempt low level changes, it will validate that they maintain correctness, because it has an understanding of impacts.
When you reshape elements, it has the understanding of follow-on effects, and how to propagate them into the rest of the architecture, all while staying valid.
One of the key challenges we observe is that if you're small enough, a Heroku like experience works well - and most of your needs would be covered by virtually any combination of techstacks.
It gets significantly more challenging when you grow, either in feature complexity or scale complexity - and then very few services can offer what AWS/GCP/Azure offer - albeit at the increased engineering/monetary cost of using them.
We're building a different kind of approach[0] that aims to absorb the mechanical cost of using public cloud capabilities (that are proven to scale) without hiding it altogether.
We're building klotho[0] for many of the reasons mentioned here. (happy to answer questions). We transform plain code to cloud native code. The majority of the complexity is moved into the Klotho compiler, and what devs handle is the simplest bundle that's easy to deploy and operate on public clouds using standard tools.
[0] https://infrastructurefromcode.com/ [1] https://klo.dev/state-of-infrastructure-from-code-2023/