Thanks for responding. I love these conversations.
>1) How? If you want me to try something, either big "TRY ME" unless it involves becoming a client which that case, I see you as replacing me so my motivation is zero. :D
To be clear, I don't see this as having the capability to replace engineers. This is a new way to interact with your infrastructure.
But it also feeds into larger AI conversations as well. (probably out of scope for this conversation :) )
> This blog post reads, we have this great new Windows Server thing that will blow your Linux Server stuff away completely with GUI and Siri.
Now you crossed a line :D
Jokes aside, there is a HUGE community in the TF and now OT space, you can't argue that. The ecosystem of third-party tools to help support workflows is gigantic. Putting all of that aside, will that be the best way forever? I'm not saying that SI will replace either one of those, but I would say it's a new and refreshing way to tackle a similar problem space.
> Sure, your system may update itself more frequently than only when I run "tofu plan/apply" but at end of the day, it doesn't matter.
Correct me if I'm wrong here. In my experience you have to "apply" before state is updated. This would mean we weren't quite operating on the source of truth. (aws in this case).
100% it's a solvable problem with a TF centric tool chain. But it's still a problem that needs solving.
In my experience with SI it fades to the background. Now, I'm sure there is an edge case where someone edits something outside of SI while I'm trying to simultaneously update it in SI where things might break. I haven't run into it yet.
> All I'm saying as SRE, you have done poor job selling this to me
Can't argue this, but I would say like any other new tool, it's worth checking out. :)
Thanks for the feedback. I'm new to the platform, and certainly appreciate the interaction.
I think I described SI a bit better in another reply, and you can certainly check their website for a better description than I can give here.
I'll try to high level our particular issues to give you a sense of why this is important to us.
Traditionally, we've managed our customers via TF. I made a big push years back to try and standardize how we delivered infrastructure to our customers. We started pushing module libraries, abstract variables via yaml, and leveraged terra grunt to try and be as dry as possible. We followed along best practices to try and minimize state files for reduced blast radius etc.
What became apparent was that despite how much we tried to standardize there was always something that didn't fit between customers. So quickly each customer became a snowflake. It would have its own special version of some module or some specialized logic to match their workflow. Then over time as the modules evolved, so the questions start to come up:
- Do we go back and update every customer with the new version of the module?
- Does the new module have different provider/submodule/tf version requirements?
- Did the customer make some other changes to infra that aren't captured?
Making minor changes could end up taking way longer than necessary. Making large changes could be a nightmare.
In working with SI the mindset has shifted. Rather than manage the hypothetical (ie what's written in TF), let's manage the actual. Trying to reconcile in code why a container has 2cpus instead of 4, find the issue and fix it. If want to upgrade something, find it and upgrade it.
I can go into greater depth if you care or have questions, but this at a high level explains this post a bit more.
Essentially. I'm not sure you could call it IAC specifically, but the same ideas apply.
Regarding lock in: I don't necessarily think there is anything here that is stopping you from writing TF and importing objects. Conversely, SI is great for importing resources into their model.
So the objects are essentially modeled in type script on the back end so support for other vendors is available. It's just whether or not they are created yet. I'll let the SI folks dive into details there.
We want a representation that is as close as possible to what actually is in AWS. That way any proposed changes have a high probability of success when they are applied. SI's approach keeps an extremely up to date representation of what's in AWS.
Why do we need a representation and not just go directly to the AWS API? Among other items, it removes the capability of reviewing changes before they are applied. It gives us a safety net if you will.
Thanks for the feedback. My plan is to spend a little more time to dive into the details on a follow up post.
I'll try to explain our experience here in a little better detail though.
In a traditional IAC tool (tf for example). The flow would go something like this (YMMV)
Update TF -> Plan -> PR -> Review (auto or peer) -> Merge -> TF Reviews State File -> TF Makes changes -> Updates State.
Some issues we could run into:
- We support multiple customers each with their own teams that may or may not have updated infra so drift is always present.
- We support customers over time so modules and versions age, and we aren't always given the time to go make sure that past tf is updated. So version pins need to be updated among other dependencies.
Each of those could take a bit of time to resolve so that the tf plans clean and our updates are applied. Of course there are tools such as HCP Cloud, Spacelift, Terrateam etc. But, in my experience it shifts a lot of the same problems to different parts of the workflow.
The work flow with SI is closer to the following:
Ask AI for a change -> AI builds a changeset (PR) -> Review -> Apply
The secret sauce is SI's "digital twin". We aren't just using AI to update code, we're actually using it to initiate changes to AWS via SI. While I would never want to have a team make changes directly to AWS without a peer review or something similar, it is sitting closer to what the actual infrastructure is. Even with changes that are happening to the infrastructure naturally.
This has allowed us to move quite a bit faster in updating and maintaining our customers infrastructure. While still sticking as close as possible to best practices.
>1) How? If you want me to try something, either big "TRY ME" unless it involves becoming a client which that case, I see you as replacing me so my motivation is zero. :D
To be clear, I don't see this as having the capability to replace engineers. This is a new way to interact with your infrastructure.
But it also feeds into larger AI conversations as well. (probably out of scope for this conversation :) )
> This blog post reads, we have this great new Windows Server thing that will blow your Linux Server stuff away completely with GUI and Siri.
Now you crossed a line :D
Jokes aside, there is a HUGE community in the TF and now OT space, you can't argue that. The ecosystem of third-party tools to help support workflows is gigantic. Putting all of that aside, will that be the best way forever? I'm not saying that SI will replace either one of those, but I would say it's a new and refreshing way to tackle a similar problem space.
>2) I'm on Azure Sorry ;-P