Sure you can use an AWS ASG, but I assume you also tie that into an AWS AlB/NLB. Then you use ACM for certs and now you are locked in to AWS times 3.
Instead you can do those 3 and more in k8s and it would be the same manifests regardless which k8s cluster you deploy to, EKS, AKS, GKE, on prem, etc.
Plus you don't get service discovery across VMs, you don't get a CSI so good luck if your app is stateful. How do you handle secrets, configs? How do you deploy everything, Ansible, Chef? The list goes on and on.
If your app is simple sure, I haven't seen simple app in years.
I think what they should have said is "there isn't a tool like Helm for ECS" If you want to deploy a full prometheus, grafana, alertmanager, etc stack on ECS, good luck with that, no one has written the task definition for you to consume and override values.
With k8s you can easily deploy a helm chart that will deploy lots of things that all work together fairly easily.
It doesn't have to be that way though. You can use the AWS ingress controller, or you can use ingress-nginx. You can use external secrets operator and tie it into AWS Secrets manager, or you can tie it into 1pass, or Hashicorp Vault.
Just like picking EKS you have to be aware of the pros and cons of picking the cloud provider tool or not. Luckily the CNCF is doing a lot for reducing vender lock in and I think it will only continue.
I agree with this somewhat. The other day I was driving home and I saw a sprinkler head and broke on the side of the road and was spraying water everywhere. It made me think, why aren't sprinkler systems designed with HA in mind? Why aren't there dual water lines with dual sprinkler heads everywhere with an electronic component that detects a break in a line and automatically switches to the backup water line? It's because the downside of having the water spray everywhere, the grass become unhealthy or die is less than how much it would cost to deploy it HA.
In the software/tech industry it's common place to just accept that your app can't be down for any amount of time no matter what. No one checked to see how much more it would cost (engineering time & infra costs) to deploy the app so it would be HA, so no one checked to see if it would be worth it.
I blame this logic on the low interest rates for a decade. I could be wrong.
k8s is complex, if you don't need the following you probably shouldn't use it:
* Service discovery
* Auto bin packing
* Load Balancing
* Automated rollouts and rollbacks
* Horizonal scaling
* Probably more I forgot about
You also have secret and config management built in. If you use k8s you also have the added benefit of making it easier to move your workloads between clouds and bare metal. As long as you have a k8s cluster you can mostly move your app there.
Problem is most companies I've worked at in the past 10 years needed multiple of the features above, and they decided to roll their own solution with Ansible/Chef, Terraform, ASGs, Packer, custom scripts, custom apps, etc. The solutions have always been worse than what k8s provides, and it's a bespoke tool that you can't hire for.
For what k8s provides, it isn't complex, and it's all documented very well, AND it's extensible so you can build your own apps on top of it.
I think there are more SWE on HN than Infra/Platform/Devops/buzzword engineers. As a result there are a lot of people who don't have a lot of experience managing infra and think that spinning up their docker container on a VM is the same as putting an app in k8s. That's my opinion on why k8s gets so much hate on HN.
Helm Charts and Terraform are different things IMO. Terraform is better used to deploying cloud resources (s3 bucket, EKS cluster, EKS workers, RDS, etc). Sure you can manage your k8s workloads with Terraform, but I wouldn't recommend it. Terraform having state when you already have your start in k8s makes working with Terraform + k8s a pain. Helm is purpose built for k8s, Terraform is not.
I'm not a fan of Helm either though, templat-ed yaml sucks, you still have the "indent 4" insanity too. Kustomize is nice when things are simple, but once your app is complex Kustomize is worse than Helm IMO. Try to deploy an app that has a ING, with a TLS cert and external-DNS with Kustomize for multiple environments; you have to patch the resources 3 times instead of just have 1 variable you and use in 3 places.
Helm is popular, Terraform is popular so they both are talked a lot, but IMO there is a tool that is yet to become popular that will replace both of these tools.
Most people on Nostr are pro Bitcoin, not pro cryptocurrency. Bitcoin and crypto are not the same. There have been tons of scams in crypto, but not in Bitcoin. Sure there were people who created a scam and told people to send them their Bitcoin (Blockfi, Celsius, Netflix doc "Bitconned"), but that isn't the same as the crypto scams. The Bitcoin core devs never created a token and then ran a pump and dump. It's like saying someone scammed someone with USD, therefore USD is a scam.
e.g. if you read anything on your local cities subreddit you will think everyone in the city is left leaning. You will think that anyone who isn't left leaning is the devil; they are pedophiles, conspiracy theorists, etc.
If you go out into the city 99.9% of the people are perfectly pleasant regardless of their political leanings.
Very cool stuff. This type of life is why I originally got into tech. Few thoughts:
- I wish this guy had a personal blog that I could subscribe to. I like that post, but I could care less about the rest of the blog posts on his employers site.
- I'd be interested in why he has stuck with a Model Y, seems to not be a good fit. I'd love to see him switch to something better (probably an ICE even though I know tech people love EVs).
All in all kudos to this guy for all the work he has put in to make this life work, love the iterative approach.
All the cloud providers offer a storage class for k8s. The storage class allows you to tell k8s that you want a persistent volume (PV) and it will make API calls to the cloud provider to get you a block storage device. You can tell k8s you want to use that PV in your pod and k8s will automatically mount the block storage to the worker node that your pod lives on and makes it available to the pod.
OP uses Longhorn which is a whole other thing that I've only read about.
For at home you can use other storage classes like ceph, NFS, etc.
IME yes, yes it is. I usually tell people running a k8s cluster is similar to running your own "cloud". If you want to just deploy an EC2 instance you just tell AWS you want an EC2 instance and you are done (mostly). You don't have to worry about if the hardware under the EC2 VM has enough resources, you do have to worry about that with k8s though. If you want to lock down the EC2 VM to have certain permissions you use AWS IAM, with k8s you have to use cluster roles and cluster role bindings. You can apply these 2 examples to many other things in k8s vs "cloud" provider, Ingress, persistent volumes, etc.
> ...the problem arises immediately when you try to resize a data volume and you can't because the API doesn't support it
What API doesn't support it? k8s has support for resizing PVs and has for a while now. AFAIK all 3 cloud providers (and more) support increasing the PV using their storage class.
> K8s is not really for stateful systems, yet
Is this written somewhere or is it just your opinion?
This seems to be the common advise given, but I don't fully agree. There have been many times in my career where a DB was on a VM with the storage attached via a cloud providers block storage. When asked if we should move it to k8s, people are quick to mention k8s doesn't do well with persistent storage. However all of the big cloud providers offer the ability to easily create a persistent volume in k8s that then just creates a block device, attaches it to the k8s host and makes it available to the pod.
So in both situations you have the same IO limits of block storage. The question is does k8s persistent volume api add enough of an IO bottleneck to cause issues, IME that isn't the case.
Now if you want direct attached NVMe drives for higher IO than a network attached block storage will give, then it might be easier with a VM vs k8s, but I can't speak to that much.
Especially with the migration to k8s. K8s is much more complex than Heroku, some even say it requires an entire infra engineering team to manage.