If you use a VM on AWS, you also need to know that you need to configure a vm snapshot (very easy, totally agreeing here with you).
But you can also use a managed k8s from AWS which you can also backup as they are all on PV and they have snapshotfeatures.
I don't want to compare a VM + Snapshotting 1:1 with kubernetes though. It wouldn't be fair to k8s and it wouldn't be fair for all usecases which work very very well on one VM.
K8s doesn't solve problems which haven't been solved before. It doesn't do any particular magic in itself. The handful of things kubernetes does, are easy to explain but the impact is big nonetheless.
It is trustworthy because it is FOSS, certified and lots of companies use it because of this.
Lets take Java vs. PHP: PHP is developed by one group of people. Thats it. There was facebook hhvm/php alternative which then became something independent of php. Quite frustrating if you were hoping that Facebook gives back to the community.
Then take java: you have a spec, you have a reference implementation and then you have validated alternatives of it. At least you had this for a long time on the JavaEE area and with the oracle support thing, you also now have independent JVMs. This makes Java, in my opinion, better. This makes it a great platform, easy to migrate out of one ecosystem and it prevents 'vendor lock-in'.
Nomad is from hashicorp. You have mesos which works well as well. But no normal cloud provider provides nomad or mesos as a service. They provide their own thing. App Engine, Heroku etc.
Kubernetes broke through this. Lots of smaller cloud providers provide a managed kubernetes. You can see kubernetes here as the universal appengine if you like. Google provides Autogke. Their managed kubernetes service which abstracts away k8s even further. This interface, k8s provides, allows you to run your k8s based workloads at home, onprem, in private, in any other cloud provider AND on Google.
Instead of having Vendor lock-in it switches the operation model of those companies: They can't lock you in as easily before so they need to make the best offering for it. It switches the mental model and the level of competition to a more consumer/customer focused level.
It is very similar on a mental model switch as what Microsoft did with linux: Instead of hating against linux, they embrase it now and incorporate it. I never considered windows as a good developer OS just because of the missing shell support or the required workarounds or the non native cli feeling it gave you. Now i can use WSL2 and it becomes a real option.
For me, k8s is THE FOSS infrastructure abstraction layer. Protected and aligned through the CNCF and certification process.
I only answered the question why it is different with kubernetes.
I dont have anything against VMs. Feel free to click yourself a VM on any cloud provider, use it however you like.
K8s abstracts VMs away and i have and had real issues with maintaining VMs. Docker filling up the node with logs. Unable to upgrade the BaseOS due to python dependencies. Managing the same VM stack through ansible and everything ansible or chef brings to the table.
There have been plenty of self healing mechanism in place which do solve unfortunate issues. Memory? The service restarted, was offline for 3 minutes and is now working again. Node disk full? Pods get scheduled away, new node comes up, done. Update/upgrade of nodes? Nodepool does it for me.
For me, k8s has 2 real issues like memory (swap support is wip finally!) and stateful workloads like a database. But the concept of an operator shows a bright future.
k8s also does one thing very nice: It enforces certain aspects which are a pain in the ass later. That VM which wasn't updated for years and run just fine? Now there is an issue and it needs to be fixed asap. But now the debian repositories are no longer available. I have to fix apt srce list first, then i need to fix dependencies and then i need to restart it.
I still don't get your motivation on writing your criticism.
What is your endgoal? Getting people not to like k8s? Because you don't like to work with it?
To push people away from k8s?
How do you add value to the current infrastructure/platform ecosystem by 'hating' on it without providing something different?
Of course companies present this k8s story as a successful thing. Why would that documentary be negative?
And while you have 'A LOT of issues with the things you posted above' just to be clear: For me and a lot of other people who like kubernetes, it solves real problems, its a great choice and there are of course things which need to be optimized. But if you only rant about it in the next blog post from you, i'm not seeing any value you really add to the ecosystem.
For me, i never seen anything like kubernetes in the last 12 years. I can get certified k8s from many companies in many different forms (gke, aks, aws, digitalocean, ranger, rke2, k3s, minicube, microk8s). ArgoCD is a dream come true.
Can you do it differently with other tools? Yes sure, did we ever had something like k8s before? no. We never had that holistic view on Infrastructure in such a FOSS project.
Again what do you want to achieve? A real discussion on specific issues or just hating against something? Or doyou have the feeling that the blog posts writing about k8s are to one sided?
This was 3 years ago and it even provides a solution in that talk already. Have you rechecked your assumption or are you only arguing from a talk from 2019?
Índependent of this, the abstraction layer (k8s api, kinds etc.) is independent of the code base btw. You can easily refactor and fix all issues in k8s. The abstraction is already there.
I actually believe that the k8s api could be the best abstraction for services we ever had.
While i'm running k3s at home (which is very nice to be honest) and big instances at work, i would prefer to have more managed k8s offerings but many already exist. They exist from DigitalOcean, Google, Azure, AWS and its probably way easier for smaller service providers to make a managed solution available. You can also use rancher or gardener to create and manage k8s clusters 'raw' by yourself.
I can tell you what it does for me:
Full disk on your VM? Nope. Storage is abstracted away. Your server will not fill up anymore. Only one service might break which might heal itself.
Is your VM/node broken? It will heal itself because you throw it away and the new VM/node is fixed.
It enforces the abstraction of Service and VM. You will not install normal software on that VM just because you can. You don't need to give access to a VM to a developer who then needs root access and has dependencies and doesn't update the VM.
You no longer have dependencies to your VM because you can't have dependencies on your VM OS.
Abstracting it away from your VM also streamlines things like logfiles. You no longer need to collect all logfiles from VMs because you do it for your services and for your services, you only do it once (if even, log to stdout and be done with it)
Its just a very bad and uneducated stand from some person.
I'm running a small k8s instance at home, for a small startup and at my job in a big version.
Abstraction of VMs is a real benefit: Have you ever had to restart a VM because of some security issues? Yes? Were you worried that your server comes up again?
With k8s, you know that 1. its cloud native to a certain extend. It will come up again because it came up before. 2. you have more nodes available. Either to surge or because you have more than just one node running.
Your pod will be scheduled away from your node, thats it.
you have a very stable and smart abstraction layer for sooo many features you get as soon as you configure them ONCE centrally:
- LoadBalancing
- certificate management
- Volume abstraction -> making snapshots from your PV? yes!
- Rollout strategies
- health checks (readiness and liveness probes)
- declaritive style (setup a prometheus, every service can be autoscraped due to convetion over configuration)
- Certified opensource abstraction layer! (get yourself a certified k8s distribution and stop worrying about vendor lock in)
- Unified setup for plenty of apps (monitoring, logging, app store, tracing, storage systems, iam etc. etc. etc.) We had deb before and rpm and whatnot. Now you have a helm chart for a certified k8s platform)
- Already quite small -> there is k3s. ubuntu supports it also with not that much overhead
- IaC as first class citizen. Due to k8s being declarative, IaC is much easier than it was before.
- FOSS
- Central easy policy implementation and management. Write your central policies, allow your teams to manage their own namespace and make sure to allow only certain registries etc.
- ArgoCD / GitOps (a dream come true srsly!)
I cant understate how much i love k8s and how much better it is then everything i have seen before. This is the main reason why i even spend the time writing here because that çritical review' is just utterly bullshit.
Did we had similiar things somehow before? yes. So whats new on k8s? K8s unites across companies and just drives this further. For me k8s is the winner of this race which happened in parallel (mesos, docker, nomad etc. etc.)
Or 'can':
If you use a VM on AWS, you also need to know that you need to configure a vm snapshot (very easy, totally agreeing here with you).
But you can also use a managed k8s from AWS which you can also backup as they are all on PV and they have snapshotfeatures.
I don't want to compare a VM + Snapshotting 1:1 with kubernetes though. It wouldn't be fair to k8s and it wouldn't be fair for all usecases which work very very well on one VM.