My concerns with modern infra abstractions
Hello, it’s my first time posting here. I figured this would be a good place to discuss my current concerns about modern infrastructure management abstractions. I am hoping to have a fruitful discuss about how to best tackle multi scale infra management (i.e. side project -> FAANG) in a way that is easy for devs to manage. I basically have 3 MAJOR problems with most infrastructure solutions. The first is premature abstraction. In this case you end up with things like AWS amplify. For these cases they are great for very early beginners. As in “this is my first full stack project” beginners (I hope that isn’t offensive, just trying to give context). You get basic management behind a UI or CLI, and rarely if ever is it transparent about what is actually going on. What do you all think about services like these? Are they useful? In my experience most have been a headache EXCEPT firebase for auth management. The next I see pretty consistently is mixing code and infrastructure (even iac). This is a problem due to state management, but things like terraform are helpful here (hot take). It’s just that when you have no state management it’s hard to scale beyond a simple app. It’s also difficult to do right without a proper vcs to manage your infrastructure. But you’ll then run into the problem that early founders or solo engineers or whoever don’t want to do the work to prepare to scale efficiently. Which is partially why, in my opinion, those abstraction (read: aws wrapper) services are so lucrative and prevalent. And then as you scale you run into the issue of transient compute more and more (especially in Data science and ML). And even if you have a good pipeline, state management, etc. You still have a bad dev experience because there’s no good way to transition between a pre-devops and post devops team. Maybe I don’t have enough experience, but I am also looking for opinions on what may make that transition better. The last major issue I see commonly is that modularity is still a problem. Bundling, building, and deploying code is still configured in yaml files. Makes it really hard to write reusable infrastructure OR in the UI case virtually impossible (though I’m not sure why you’d use a simple UI to manage infra after a certain point, so maybe someone can shed some light on that).
0 comments
—