friend has been familiarizing himself with agentic workflows recently and was initially struggling. he figured out an instructive model that works for him, and generated a blog post about it. thought i'd show him some love, and share for others who might be feeling the same pain as him.
yeah our findings were similar. the issues we saw with iptables rules, especially at scale with ephemeral workloads, was starting to cause us a lot of operational toil. nftables ftw
yeah, like the others have said, the tradeoff isn't really worth it for us as a business. spot instances also generally come with low qos guarantees (since they tend to be interruptible). tbf there are on-demand alternatives with better guarantees though
another thing to note is that we bootstrap the hosts, and tune them a decent amount, to support certain high-performance features which takes time and makes control + fixed-term ownership desirable