Because plenty of people share your POV and kinda - a little bit - behave like there was no life before k8s, I will try to address your points.
>What happens if a container in the VM goes down or the app inside of it crashes, how do you recover?
Docker will restart container automatically. You don't have to do anything.
Docker-compose will restart after VM restart. You don't have to do anything.
If a VM goes down - I do have a HA (another VM at another provider) and DNS load balancing.
>Now you need some self-recovery mechanism via systemD or whatever, which will grow in complexity and fickleness over time. Congrats, you are now doing your own version of kubernetes.
While I don't like systemd, it does this automatically, while, it's not really used here.
> What happens when you need to upgrade/restart your VM? Ok, make a standby VM as backup that will mostly sit idle, or require a full-app redeploy any time you need to do anything to the first VM. Now you need to design a blue/green mechanism between them, and probably some networking layer work. Congrats, you are now doing your own version of kubernetes.
This has been pretty much answered already but, upgrades does not affect containers (unless docker engine upgrade).
Restarts - docker will handle these automatically - nothing to do here.
> What happens, if running in cloud, you have a regional outage or degradation? Stand up another VM in another region and manage the networking layer between them. Or, if running locally, your ISP has an outage because of a backhoe or something. Ok, we'll rent rack space in another data center as backup. Own all the mechanisms between cutting between those two now. Congrats, you are now doing your own version of kubernetes.
This actually handles way better w/o managed kubernetes, as it's usually a single region and your cluster and workloads would simply be completely down, while mine would work, because of provider redundancy.
> What happens if your app gets huge volume during peak times, and very little volume during non-peak, and you find yourself overprovisioning to the point your CFO/CTO freaks out about the bill? Well, we'll make our own dynamic scaling mechanism. Congrats, you are now doing your own version of kubernetes.
Kubernetes with autoscaling wins hands down here, but, it's not automatic, nor hassle free.
You are also assuming overprovisiong which is usually not the case for traffic spikes.
> What happens when your app traffic gets so large you start running into OS limitations, like file descriptor limits? Start trying some of the aforementioned solutions. Congrats, you are now doing your own version of kubernetes.
This also affects k8s, exactly the same way.
> What happens if you need service discovery, monitoring, or ensure network isolation between various services? Different VM's + your own hacked together service mesh, or wire something in the VM. Congrats, you are now doing your own version of kubernetes.
I do have service discovery and network isolation built into docker, thanks.
> What happens when you need to guarantee secret isolation between containers? Congrats, you are now doing your own version of kubernetes.
Believe it or not, it's the default with docker.
> Let's say you don't actually need any of this or think you never will. Fine! That's valid. But what you don't want, is to suddenly hit some scale and any of these things (I could list way more but I feel I am belaboring the point), migrating off these setups can become a year+ project, if not way longer. I know because I have had to do this twice now. I cannot possibly understate how painful it is.
All my workloads are containerized and I can just move them to a k8s cluster whenever I want, if needed.
2) the risk of the VM + container spiraling into complexity is perceived. as way more than going more complex at the start.
The risk of your k8s ecosystem spiraling into operators madness and argoapps over helmfiles all while trying to accommodate for ci/cd and costs offing the chart is - IMHO - way higher.
While I agree with you I'm not sure the rest of the world does.
Over the past decade, I'm seeing k8s used everywhere for everything, companies setting up clusters to run literally one simple app with couple of hundred requests per hour.
Not in my projects it seems. Perhaps you can share your best practices?
Moreover, avoiding these should be the default behaviour. Currently the default is to drain your pockets.
P.S
CLAUDE.md is sometimes useful but, it's a yet another token drain. Especially that it can grow exponentially.
LLMs (Claude Code in particular) will explicitly create token intensive steps, plans and responses - "just to be sure" - "need to check" - "verify no leftovers", will do git diff even tho not asked for, create python scripts for simple tasks, etc.
Absolutely no cache (except the memory which is meh) nor indexing whatsoever.
Pro plan for 20 bucks per month is essentially worthless and, because of this and we are entering new era - the era of $100+ monthly single subscription being something normal and natural.
"A plugin/annotation system where users can teach the agent about custom resource types would scale better than hard-coding each one." - this is a fantastic observation and feedback! Many thanks!
"requiring N consecutive failures before marking down" - I do have the code for it, it's just hidden currently. StatusDude supports 2 types of worker/agents - cloud agents - that will re-verify from multiregion the service status and private agents - the ones we're talking about here - that I might just bring this option back as it makes more sense.
Correlating failures is a bit tricky as usually it requires some sort of manual dependency creation but, I guess for k8s ingress and similar I should be able to figure this out and at least send alerts with appropriate priorities and order.
As for the status page auto generation - currently it's based on namespace - I didn't wanted to bloat the user dashboard too much. Each monitor is tagged with cluster id, namespace and labels. Status Pages pickup monitors based on labels. Users are free to modify these and show exactly what they want :)
It's more like the farmer was giving leftovers for free to schools and it was so good that it made him famous. People from all over the country came in, including businessmen who told the farmer he is missing out and should be charging more for his food.
He started a restaurant chain but, the businessmen went further and said that a quality product cannot be given away for free and made him stop supporting schools and shelters which got him rich and famous in the first place. Even tho, he was just handing over leftovers (it cost around USD 100 to host a docker image - yearly)
Think EA, Microsoft and Xbox, Broadcom and bitnami.
It's absolutely stunning that people actually defend this behaviour!
The community is having an outrage - and rightfully so - about a silently discontinued artifact delivery at a very critical time.
Which is their opinion and every human being is entitled to have their own opinion and state it openly.
It is also perfectly fine to expect a standardised behaviour to continue.
However, what is most important is that is perfectly fine to shame an open source product for pulling features and money grabbing people after years of gathering community and locking them in.
OVH sells bare metal servers with 6 cores 32 gb ram and 512 nvme disks in raid 1 (!) with *unmetered* 1gbps line, for 70 bucks. 70 united states dollars.
How much of AWS EC2 you can get out for 70 bucks?
Now, this OVH still makes money on it. They make money, despite these servers require actual human being to put them into datacenter, plug network, power, etc.
You are literally getting the OG RAW POWER and a slice of a datacenter for 70 bucks and they still make money.
How much amazon makes on every single silly vm that they charge for compute, storage, network, ip, network again, oh and credits, cpu credits, startup credits, whatever credits, oh an api calls ;]