The “KISS solution” didn’t scale to the requirements of modern business. I remember running chef - essentially a complicated ruby script - on 100ks of servers, each of which with their own local daemon & a central management plane orchestrating it. The problem was that if a server failed… it failed, alongside everything on it.
Compared to that setup, k8s is a godsend - auto healing, immutable deployments, scaling, etc - and ultimately, you were already running a node agent, API, and state store, so the complexity lift wasn’t noticeable.
The problem came about when companies who need to run 5 containers ended up deploying a k8s cluster :-)
>The high cost arises from needing to provision a bare-bones cluster with redundant management nodes.
?!
How else are you managing your infrastructure besides having redundancy in the control plane? Even if it’s a chef or puppet server, or even sysadmin Dave running scripts on his laptop, you should still have redundancy.
> Moreover, Kubernetes’s slow autoscaling meant I had to over-provision services to ensure availability, paying for unused resources rather than scaling based on demand.
??????
Slow compared to what exactly? Anecdotally, k8s with karpenter is significantly faster to scale than auto scaling groups.
> I noticed that there is only one k3s node that has the control-plane,master role. If that node fails you can no longer manager the cluster.
This feels like a failing of k3s that should be addressed within k3s, rather than adding duct tape elsewhere. Kubernetes has built-in mechanisms (Leases) for handling these kinds of problems.
All cloud providers provide public APIs for managing infrastructure, how is kubernetes any different?
Your viewpoint operates on the assumption that a private network provides additional security in comparison to the public internet, which is a very dangerous assumption to make.
The “KISS solution” didn’t scale to the requirements of modern business. I remember running chef - essentially a complicated ruby script - on 100ks of servers, each of which with their own local daemon & a central management plane orchestrating it. The problem was that if a server failed… it failed, alongside everything on it.
Compared to that setup, k8s is a godsend - auto healing, immutable deployments, scaling, etc - and ultimately, you were already running a node agent, API, and state store, so the complexity lift wasn’t noticeable.
The problem came about when companies who need to run 5 containers ended up deploying a k8s cluster :-)