Acorn: A simple application deployment framework for Kubernetes(acorn.io)
acorn.io
Acorn: A simple application deployment framework for Kubernetes
https://acorn.io/
141 comments
How do you implement healthcheck?
Does the loadbalancer know how the healthceck is implemented?
How do you determine it's time to scale?
How do you implement always-on-process? service unit, initd, cron?
How do you export the logs?
How do you inject configs? /etc/environment, profile.d, systemd config, /etc/bestestapp/config?
What about secrets?
Service discovery? Is unbound/bind9?
And on and on and on and on.
These items are best done in a standard way. That's a major selling point in k8s. You define and implement these in a consistent way that also integrates seamlessly with other tools. Doubly so on cloud. Less reinventing the wheel means less money. The more teams, the faster this becomes exponentially expensive. Take this into consideration before your next interview.
Does the loadbalancer know how the healthceck is implemented?
How do you determine it's time to scale?
How do you implement always-on-process? service unit, initd, cron?
How do you export the logs?
How do you inject configs? /etc/environment, profile.d, systemd config, /etc/bestestapp/config?
What about secrets?
Service discovery? Is unbound/bind9?
And on and on and on and on.
These items are best done in a standard way. That's a major selling point in k8s. You define and implement these in a consistent way that also integrates seamlessly with other tools. Doubly so on cloud. Less reinventing the wheel means less money. The more teams, the faster this becomes exponentially expensive. Take this into consideration before your next interview.
100% this. Pre-K8s I’ve done EC2 auto-scaling with ansible and packer, our own in-house healthcheck framework (https://github.com/hootsuite/health-checks-api), our own rolling deploys, our own in-house service discovery (https://medium.com/hootsuite-engineering/how-our-service-mes...), our own in-house udp logging to an ELK stack (https://github.com/hootsuite/udp-logger), our own init scripts, etc. It was a lot more complexity for us to manage than K8s, far more bugs/issues, far less uniform across teams/services. We switched to mesos, then K8s, and K8s was the simplest by far. My new company also uses K8s, and it’s also overall easier to manage with fewer issues than any alternative I’ve used.
I think if you’re a small org with just a few devs and one or a handful of services, sure, K8s is overkill. But for a decent sized dev org, it’s probably way better and significantly simpler than what you’d rollout without it.
I think if you’re a small org with just a few devs and one or a handful of services, sure, K8s is overkill. But for a decent sized dev org, it’s probably way better and significantly simpler than what you’d rollout without it.
Disclaimer: I work at Hashicorp but I have been saying what I'm about to say for years before joining (check my blog, linked in bio, for reference).
There is a way to get all of those necessary things without having the massive complexity of k8s - use Nomad, a vastly simpler orchestrator that does many things better than k8s - e.g. integrated templating instead of go templates over yaml with Helm or other similar abominations; not using yaml for configuration; self-contained configuration and handling of updates (e.g. if your ConfigMap or Secret in k8s change, you have to manually restart/reload your pods; with Nomad you can configure it to handle that directly). It doesn't have nearly the same ecosystem, but there's enough of it through CSI, CNI, various load balancer and autoscaling integrations that you get very far with a production environment.
There is a way to get all of those necessary things without having the massive complexity of k8s - use Nomad, a vastly simpler orchestrator that does many things better than k8s - e.g. integrated templating instead of go templates over yaml with Helm or other similar abominations; not using yaml for configuration; self-contained configuration and handling of updates (e.g. if your ConfigMap or Secret in k8s change, you have to manually restart/reload your pods; with Nomad you can configure it to handle that directly). It doesn't have nearly the same ecosystem, but there's enough of it through CSI, CNI, various load balancer and autoscaling integrations that you get very far with a production environment.
Why are people using Terraform, a relatively horrible and inconsistent tool to deploy their software as code ? As it is better than the alternatives and everyone is using it. It is the same with K8s. Yes, it is complex, however, as everyone is using it it is super easy to find documentation or example for pretty much anything. Finding the same documentation for Nomad is much harder.
Career wise all major platforms (Azure, GCP, and AWS) offer K8s, and therefore it is much better to learn the principles of K8s instead of focussing on the relatively obscure Nomad platform.
Career wise all major platforms (Azure, GCP, and AWS) offer K8s, and therefore it is much better to learn the principles of K8s instead of focussing on the relatively obscure Nomad platform.
My issue with nomad is I also have to handle a second cluster, Consul, and that's it's own overhead. I'd rather just stay with k8s if the alternative is Nomad. Id totally jump to Nomad if all I had to do was deploy nomad, and consul was embedded in all nodes. I'm not an expert in consul, and really don't want another time sink.
Nomad 1.3 has builtin service discovery and 1.4 is around the corner with features for secrets so you can get started without vault too.
(I work for HashiCorp on Nomad)
(I work for HashiCorp on Nomad)
At the core of K8s is an Etcd cluster, so if you’re spinning your own it’s the same thing. However, part of the value-add of cloud providers in the K8s space is that they hide it from you.
I understand that, but I already invested the time to work with etcd and can handle a few of k8s failure modes. Really do not want to throw that away for something that has a similar dependency, and that I do not now know well. Nomad might be simpler to use, but not to setup or manage due to Consul dependency.
Also, a lot of these products are missing a lot of documentation. I often find the tutorials are inflexible and focus on only 2 use cases out of the 4 most common. For example, focus on cloud when lots of engineers would rather deploy on bare metal. I would love for there so be one simple Nomad ansible role that would help setup nomad on bare metal with wireguard networking. This would probably be easy if consul was not required and it did not need so much babysitting.
Also, a lot of these products are missing a lot of documentation. I often find the tutorials are inflexible and focus on only 2 use cases out of the 4 most common. For example, focus on cloud when lots of engineers would rather deploy on bare metal. I would love for there so be one simple Nomad ansible role that would help setup nomad on bare metal with wireguard networking. This would probably be easy if consul was not required and it did not need so much babysitting.
Just throwing this out there:
Since Nomad 1.3, Consul is no longer required for Service Discovery, and since 1.4 Vault will no longer be needed for KV secrets.
And for what it's worth, i used to run Nomad+Consul in production on bare metal and it was easy to setup (2x 3 servers with a binary to start with some basic conf) and maintain (an update every month or so, which consisted of upgrading the binary on each node and restarting the service after 5 mins of reading the release notes).
Since Nomad 1.3, Consul is no longer required for Service Discovery, and since 1.4 Vault will no longer be needed for KV secrets.
And for what it's worth, i used to run Nomad+Consul in production on bare metal and it was easy to setup (2x 3 servers with a binary to start with some basic conf) and maintain (an update every month or so, which consisted of upgrading the binary on each node and restarting the service after 5 mins of reading the release notes).
Etcd has come a long way and generally doesn't behave strangely or cause big issues these days.
The cloud providers generally aren't actually doing any significant amount of work on your behalf on the etcd front.
The cloud providers generally aren't actually doing any significant amount of work on your behalf on the etcd front.
this happened very recently
Etcd v3.5.[0-2] is not recommended for production => https://groups.google.com/a/kubernetes.io/g/dev/c/B7gJs88XtQ...
https://github.com/etcd-io/etcd/issues/13766#issuecomment-10...
Etcd v3.5.[0-2] is not recommended for production => https://groups.google.com/a/kubernetes.io/g/dev/c/B7gJs88XtQ...
https://github.com/etcd-io/etcd/issues/13766#issuecomment-10...
Last week, when that same discussion popped up, "schmichael" (who works on Nomad I understand) said "Nomad supports CSI like Kubernetes. Unfortunately many CSI storage providers assume kubernetes or adhere to kubernetes behavior even when it deviates from the spec, so there are gotchas."
https://news.ycombinator.com/item?id=32588931
That makes be a bit wary to use it because I would like my cloud storage to be reliable.
https://news.ycombinator.com/item?id=32588931
That makes be a bit wary to use it because I would like my cloud storage to be reliable.
Yep, some CSI providers do that, but as far as i know it's more niche ones (like VMware's), not any of the big cloud providers.
I tried using a CSI driver with Nomad (AWS) and AFAIK you have to pre-create the EBS volumes? I couldn't figure out how to spin them up on demand.
That was probably before the Nomad 1.3 release which added a bunch of new CSI features, including dynamic volume creation IIRC.
I wouldn't call vmware a niche provider, they own a large share of on prem virualization market.
Which is a dying breed, and furthermore running an orchestrator on top of another one (Nomad or Kubernetes on top of vSphere) is a terrible anti-pattern that wastes CPU and makes performance issues hard to debug due to the two layers.
My only problem with nomad is that I have to manage the infrastructure underneath as well. Give me nomad-on-fargate and I'll happily jump ship.
But Only problem of nomad is all examples are running on Vagrant which is designed for VMs . Everyone who is looking these days are containerization. There may be non-standard ways to run Nomad with containers but , putting Vagrant as main example is not a good idea.
I'm not sure i follow. Nomad is an orchestrator which handles containers and other things (firecracker microVMs, LXC, etc.). The Vagrant examples are to get started and test it out, because running a container orchestrator in containers has some non-obvious pitfalls. You can just as well download the binary anywhere to play with it.
That doesn't mean Nomad can't orchestrate containers (in fact that's the main use case for it).
That doesn't mean Nomad can't orchestrate containers (in fact that's the main use case for it).
All the examples I’ve found are Docker…
Most use of Vagrant I saw was in demoing how to set up and deploy a cluster. So deploying 3 Nomad nodes within 3 VMs created in Vagrant.
Most use of Vagrant I saw was in demoing how to set up and deploy a cluster. So deploying 3 Nomad nodes within 3 VMs created in Vagrant.
Vagrant supports containers since v1.6 (released in 2014).
yeah but examples uses vagrant vm, i am just checking nomad and use of vm discourages me / not sure how much compatible if i to placed them with nomad/lxd/dockers.
Any good tutorials with Noman + Containers would be nice for CI/CD/Executor setup.
Any good tutorials with Noman + Containers would be nice for CI/CD/Executor setup.
I was amazed how easy it was to setup Nomad. I have a couple of VMs running on a VPS and Nomad works great for this use case. Super easy to maintain and works really well. Networking is my favourite part.
Couldn't agree more here. The amount of complexity involved in running moderately complex apps at scale on native compute and the cost to maintain over time vs. the ops savings from the standardization of most of that from Kubernetes is incomparable
I only can advise to try running more demanding apps. After all that's where the fun is. You will more-or-less instantly switch camps.
I think you misread previous comment.
> How do you determine it's time to scale?
This part is still really hard for a lot of workloads. We have some applications that have a really long startup time (5-10minutes) and also have on the whole predicable traffic patterns. It'd be great to be able to tell k8s to scale up/down based on time of day in addition to the usual metrics without having to resort to a hacky cronjob to adjust the min replicas.
Sure it won't help with a unexpected traffic spike, and it would be better if the applications could come online faster (vendor app out of our control) but most days this feature would allow us to right size the workload way easier.
There's been a few issues like https://github.com/kubernetes/kubernetes/issues/49931 but it doesn't seem to be going anywhere right now
This part is still really hard for a lot of workloads. We have some applications that have a really long startup time (5-10minutes) and also have on the whole predicable traffic patterns. It'd be great to be able to tell k8s to scale up/down based on time of day in addition to the usual metrics without having to resort to a hacky cronjob to adjust the min replicas.
Sure it won't help with a unexpected traffic spike, and it would be better if the applications could come online faster (vendor app out of our control) but most days this feature would allow us to right size the workload way easier.
There's been a few issues like https://github.com/kubernetes/kubernetes/issues/49931 but it doesn't seem to be going anywhere right now
Maybe consider using KEDA[0]?
[0]: https://keda.sh/
[0]: https://keda.sh/
That actually looks great - I could definitely make use of scaling based on message queue depths as well. Will have to look into it further, thanks!
Set up large baremetal so that you don't have to scale for a long time and by the time you do, you have had your first million users and you can afford spending for inanely complex infra and don't care about aws billing anymore. Set up user services with systemctl, access your logs through journalctl, change your configs with rsync. And op was arguing against the cloud so giving cloud examples is beside the point. No need for all the sass.
And when your one big server fails and your first million customers start looking for more stable alternatives?
One big server ( 6 Physical Core Xeon with 64GB of Ram , 1TB Nvme SSD) only cost 80$ a month .
Can start with 2-3 big servers across 3 different DC. Equlivant specs would cost easily 300$ a month on AWS/GCP/Azure .
Then you have a few hours of downtime. Happens to the best of us.
Because all the cloud providers never have downtime, DNS issues, etc? :)
At least when US east is down, half the internet is down.
At least when US east is down, half the internet is down.
Then, you had disaster recovery plans, you devised a strategy of what is an acceptable data loss, execute the reconstruction plan on the new big server, under some hours, the new big server is up again.
You apologize, offer some financial compensation, etc.
And you learn from the situation, understand what are your new tolerance for data loss, failures, etc., then start new plans accordingly.
And you learn from the situation, understand what are your new tolerance for data loss, failures, etc., then start new plans accordingly.
I think behind most anti-k8s comments is the thought that these things are not needed for "your app". "Your app" is used sporadically by a very small number of people and they don't care if it is down for up to a day.
The "your" part of that is the problem. Other people are not in a position to know what my requirements are, so telling me what my app does or doesn't need comes off as arrogant. The anti-k8s crowd always talks as though the only reason anyone would choose k8s is that they're brainwashed by the hype. It's never possible that a team actually weighed all of their constraints and made a rational choice.
> The anti-k8s crowd always talks as though the only reason anyone would choose k8s is that they're brainwashed by the hype
Maybe, but the pro-k8s camp always assumes everyone is building microservices or web services that need to scale.
Maybe, but the pro-k8s camp always assumes everyone is building microservices or web services that need to scale.
If all you want to do is to scale, you can just use auto scaling with prebaked VM images on the cloud or whatever.
K8s really shines when what you want to do is to optimize usage of your resources (I'd argue especially when running on bare metal)
K8s really shines when what you want to do is to optimize usage of your resources (I'd argue especially when running on bare metal)
I'm tired to read this as if it was impossible.
> How do you implement healthcheck?
Most applications implement a simple endpoint which returns a known answer (whether it's an HTTP 200 code, a banner, etc.) You can just hit this endpoint using your favorite tool.
> Does the loadbalancer know how the healthceck is implemented?
Well, a lot of tooling supports native (periodic) healthchecks, e.g. NGINX or NGINX Plus (if you want to pay something else than managing K8S).
> How do you determine it's time to scale?
It is not that expensive to run performance monitoring such as netdata, sometimes, it is just a call to your distribution package manager, put it in your intranet, then you're done.
> How do you implement always-on-process? service unit, initd, cron?
Depending on what you're using in Linux distributions, why bother with initd and cron if you're running on a systemd-based distribution? Just use unit, timers, service, scope, slice and so on.
> How do you export the logs?
Again, if you're using a systemd-based distribution: `man systemd-journald-remote`.
Otherwise, Loki, rsyslog, many solutions do exist for these scenarios and do not require that much fancy configuration.
For mono-nodes, I would argue that the null logger exporter, called: "rsync your logs somewhere", is quite efficient.
> How do you inject configs? /etc/environment, profile.d, systemd config, /etc/bestestapp/config?
I do not even know why people worry about that. If you're using a systemd-based distribution, your configuration should live more or less in the service unit using Environment, EnvironmentFile and LoadCredential(Encrypted), if your app want some files, sure, put it in /etc/xxx or /var/lib/xxx/config.
As long as you do separate configuration from state (and use StateDirectory, etc.) and you do backup them, it is okay.
> What about secrets?
As said earlier, LoadCredential do the job, if you want to be fancy, you can run a UNIX socket server to distribute the secrets and some secret engine such as HashiCorp Vault.
Uncertain that K8S has a better story than that.
> Service discovery? Is unbound/bind9?
Uncertain you actually need service discovery in early stages. Assuming you need, you could run envoy if you have advanced needs. Or you could run PowerDNS/nsd/unbound/BIND9 as you said with a thin wrapper to register your services.
> And on and on and on and on.
Well, I do not agree, those are items you choose. Not everyone has these requirements, and not everyone needs to solve them the k8s way.
> These items are best done in a standard way. That's a major selling point in k8s. You define and implement these in a consistent way that also integrates seamlessly with other tools. Doubly so on cloud. Less reinventing the wheel means less money. The more teams, the faster this becomes exponentially expensive. Take this into consideration before your next interview.
That is a fallacy, k8s is not a standard way, they push for their standard way and it is in no case what everyone is doing.
Defining and implementing thing in a "consistent way" is something we have been doing for a long time before k8s, through standardization process with IETF and W3C for example. k8s creates its own world, and it is indeed compatible with the external world sometimes, but I do not believe that a pile of complexity on the top of YAML files and YAML manipulation tooling is a "consistent way" to define and implement things, no offense here.
> Doubly so on cloud.
Yeah, because the "cloud ecosystem" is pushing for this, but it has not been a standard.
> Less reinventing the wheel means less money.
Using a "simple feature" is not reinventing the wheel, but if we go this way, I have a challenger: NixOS. Just reuse their expert modules database and do your classical YAML stuff, but this time with a simple functional programming language.
> The more teams, the faster this becomes exponentially expensive. Take this into consideration before your next interview.
I think GP was discussing of an early stage and I do not think that all teams require to push at a very fast rate, there is definitely time for packaging and shipping it. BTW, in these organizations where I have worked, they adopted k8s and these "teams" could not use it because it was too complicated and wondered where their packaging team has gone.
There is no silver bullet and choosing a deployment platform stack is not a figured out science, but there is a lot of value of choosing composable and simple tools before moving to more integrated ecosystems.
> Take this into consideration before your next interview.
That is an amazing level of arrogance, I only hope you may accept there are alternatives. :)
> How do you implement healthcheck?
Most applications implement a simple endpoint which returns a known answer (whether it's an HTTP 200 code, a banner, etc.) You can just hit this endpoint using your favorite tool.
> Does the loadbalancer know how the healthceck is implemented?
Well, a lot of tooling supports native (periodic) healthchecks, e.g. NGINX or NGINX Plus (if you want to pay something else than managing K8S).
> How do you determine it's time to scale?
It is not that expensive to run performance monitoring such as netdata, sometimes, it is just a call to your distribution package manager, put it in your intranet, then you're done.
> How do you implement always-on-process? service unit, initd, cron?
Depending on what you're using in Linux distributions, why bother with initd and cron if you're running on a systemd-based distribution? Just use unit, timers, service, scope, slice and so on.
> How do you export the logs?
Again, if you're using a systemd-based distribution: `man systemd-journald-remote`.
Otherwise, Loki, rsyslog, many solutions do exist for these scenarios and do not require that much fancy configuration.
For mono-nodes, I would argue that the null logger exporter, called: "rsync your logs somewhere", is quite efficient.
> How do you inject configs? /etc/environment, profile.d, systemd config, /etc/bestestapp/config?
I do not even know why people worry about that. If you're using a systemd-based distribution, your configuration should live more or less in the service unit using Environment, EnvironmentFile and LoadCredential(Encrypted), if your app want some files, sure, put it in /etc/xxx or /var/lib/xxx/config.
As long as you do separate configuration from state (and use StateDirectory, etc.) and you do backup them, it is okay.
> What about secrets?
As said earlier, LoadCredential do the job, if you want to be fancy, you can run a UNIX socket server to distribute the secrets and some secret engine such as HashiCorp Vault.
Uncertain that K8S has a better story than that.
> Service discovery? Is unbound/bind9?
Uncertain you actually need service discovery in early stages. Assuming you need, you could run envoy if you have advanced needs. Or you could run PowerDNS/nsd/unbound/BIND9 as you said with a thin wrapper to register your services.
> And on and on and on and on.
Well, I do not agree, those are items you choose. Not everyone has these requirements, and not everyone needs to solve them the k8s way.
> These items are best done in a standard way. That's a major selling point in k8s. You define and implement these in a consistent way that also integrates seamlessly with other tools. Doubly so on cloud. Less reinventing the wheel means less money. The more teams, the faster this becomes exponentially expensive. Take this into consideration before your next interview.
That is a fallacy, k8s is not a standard way, they push for their standard way and it is in no case what everyone is doing.
Defining and implementing thing in a "consistent way" is something we have been doing for a long time before k8s, through standardization process with IETF and W3C for example. k8s creates its own world, and it is indeed compatible with the external world sometimes, but I do not believe that a pile of complexity on the top of YAML files and YAML manipulation tooling is a "consistent way" to define and implement things, no offense here.
> Doubly so on cloud.
Yeah, because the "cloud ecosystem" is pushing for this, but it has not been a standard.
> Less reinventing the wheel means less money.
Using a "simple feature" is not reinventing the wheel, but if we go this way, I have a challenger: NixOS. Just reuse their expert modules database and do your classical YAML stuff, but this time with a simple functional programming language.
> The more teams, the faster this becomes exponentially expensive. Take this into consideration before your next interview.
I think GP was discussing of an early stage and I do not think that all teams require to push at a very fast rate, there is definitely time for packaging and shipping it. BTW, in these organizations where I have worked, they adopted k8s and these "teams" could not use it because it was too complicated and wondered where their packaging team has gone.
There is no silver bullet and choosing a deployment platform stack is not a figured out science, but there is a lot of value of choosing composable and simple tools before moving to more integrated ecosystems.
> Take this into consideration before your next interview.
That is an amazing level of arrogance, I only hope you may accept there are alternatives. :)
Nomad, Docker Swarm and other solutions support most of these out of the box, Kubernetes is just the most popular and flexible (with which comes a lot of complexity) solution, it seems.
For example, even something as basic as Docker Swarm will see you a lot of the way through.
> How do you implement healthcheck?
Supported by Docker: https://docs.docker.com/engine/reference/builder/#healthchec...
> Does the loadbalancer know how the healthceck is implemented?
When the health checks pass in accordance with the above config, the container state will change from "starting" to "healthy" and traffic will be able to be routed to it. Until then you can have a web server or whatever show a different page/implement circuit breaking or whatever.
> How do you determine it's time to scale?
Docker Swarm doesn't have an abstraction for autoscaling, though there are a few community projects. One can feasibly even write something like that themselves in an evening: https://github.com/jcwimer/docker-swarm-autoscaler
That said, I mostly ignore this concern because I'm yet to see a workload that needs to dynamically scale in any number of private or government projects that I've worked on. Most of the time people want predictable infrastructure and being able to deal with backpressure (e.g. message queue), though that's different with startups.
> How do you implement always-on-process? service unit, initd, cron?
The service abstraction comes out of the box: https://docs.docker.com/engine/swarm/how-swarm-mode-works/se...
You might also want to decide how to best schedule it: wherever available, on a particular node (hostname/tag/...) or on all nodes, which is actually what Portainer agent does! Example: https://docs.portainer.io/start/install/server/swarm/linux
> How do you export the logs?
Docker supports multiple logging drivers: https://docs.docker.com/config/containers/logging/configure/
> How do you inject configs? /etc/environment, profile.d, systemd config, /etc/bestestapp/config?
Docker and Compose/Swarm support environment variables: https://docs.docker.com/compose/compose-file/#environment
If you need config files, you can also use bind mounts: https://docs.docker.com/storage/bind-mounts/
> What about secrets?
Docker supports secrets out of the box: https://docs.docker.com/engine/swarm/secrets/
> Service discovery? Is unbound/bind9?
Docker Swarm supports built in DNS, even allows for multiple separate networks: https://docs.docker.com/engine/swarm/networking/
> These items are best done in a standard way.
Agreed! Though I'd say that the only two options being "running everything on *nix directly" and "running everything in Kubernetes" is a false narrative! The former can work but can also lead to non-standard and error-prone environments with a horrible waste of human resources, whereas the latter can work but can also lead to overcomplicated and hard to debug environments with a horrible waste of human resources.
The best path for many folk probably lies somewhere in the middle, with Nomad/Swarm/Compose/Docker, regardless of what others might claim. The best path for folks interested in a DevOps career is probably running on cloud managed Kubernetes clusters and just using their APIs to lots of great results, not caring about how expensive that is or how easy it would be to self-host on-prem.
For example, even something as basic as Docker Swarm will see you a lot of the way through.
> How do you implement healthcheck?
Supported by Docker: https://docs.docker.com/engine/reference/builder/#healthchec...
> Does the loadbalancer know how the healthceck is implemented?
When the health checks pass in accordance with the above config, the container state will change from "starting" to "healthy" and traffic will be able to be routed to it. Until then you can have a web server or whatever show a different page/implement circuit breaking or whatever.
> How do you determine it's time to scale?
Docker Swarm doesn't have an abstraction for autoscaling, though there are a few community projects. One can feasibly even write something like that themselves in an evening: https://github.com/jcwimer/docker-swarm-autoscaler
That said, I mostly ignore this concern because I'm yet to see a workload that needs to dynamically scale in any number of private or government projects that I've worked on. Most of the time people want predictable infrastructure and being able to deal with backpressure (e.g. message queue), though that's different with startups.
> How do you implement always-on-process? service unit, initd, cron?
The service abstraction comes out of the box: https://docs.docker.com/engine/swarm/how-swarm-mode-works/se...
You might also want to decide how to best schedule it: wherever available, on a particular node (hostname/tag/...) or on all nodes, which is actually what Portainer agent does! Example: https://docs.portainer.io/start/install/server/swarm/linux
> How do you export the logs?
Docker supports multiple logging drivers: https://docs.docker.com/config/containers/logging/configure/
> How do you inject configs? /etc/environment, profile.d, systemd config, /etc/bestestapp/config?
Docker and Compose/Swarm support environment variables: https://docs.docker.com/compose/compose-file/#environment
If you need config files, you can also use bind mounts: https://docs.docker.com/storage/bind-mounts/
> What about secrets?
Docker supports secrets out of the box: https://docs.docker.com/engine/swarm/secrets/
> Service discovery? Is unbound/bind9?
Docker Swarm supports built in DNS, even allows for multiple separate networks: https://docs.docker.com/engine/swarm/networking/
> These items are best done in a standard way.
Agreed! Though I'd say that the only two options being "running everything on *nix directly" and "running everything in Kubernetes" is a false narrative! The former can work but can also lead to non-standard and error-prone environments with a horrible waste of human resources, whereas the latter can work but can also lead to overcomplicated and hard to debug environments with a horrible waste of human resources.
The best path for many folk probably lies somewhere in the middle, with Nomad/Swarm/Compose/Docker, regardless of what others might claim. The best path for folks interested in a DevOps career is probably running on cloud managed Kubernetes clusters and just using their APIs to lots of great results, not caring about how expensive that is or how easy it would be to self-host on-prem.
Swarm is deprecated. I bet Nomad has no future and will surrender in the coming years. Using anything but k8s in 2022 is not wise.
> Swarm is deprecated.
I am yet to see this warning when running any workload on Swarm. Feature complete is not necessarily deprecated, Swarm simply won't get any fancy cloud platform integrations due to hype lying elsewhere instead, yet there has been no official word of an EOL, like: https://www.centos.org/centos-linux-eol/
Of course, one can also argue that with the Mirantis acquisition the writing is on the wall and if you care about maximum hireability, then you should grok Kubernetes as well.
None of that prevents me from also running Swarm in setups where it makes sense (e.g. a step up from Docker Compose, limited resources, not lots of time for DevOps related stuff etc.).
> I bet Nomad has no future and will surrender in the coming years.
I'm not so sure about this, Nomad integrates nicely in the rest of the HashiStack and it seems that there are plenty of large orgs investing in it, even when they don't need something like Consul or other offerings, due to its simplicity and more active development.
Currently this claim could use more of a factual basis, otherwise it ends up sounding like ".NET will kill Java" or "Linux on the desktop will kill Windows" or something of that variety.
> Using anything but k8s in 2022 is not wise.
If you live exclusively in the cloud and use only managed offerings, that may as well be true.
Otherwise it seems like a clear clash between the arguments of "pick the right tool for the job" and "pick whatever is popular and widely supported". In the deployments I've worked with, Docker Swarm is better for the first claim (limited complexity, low resource usage, easy to use) which matters more than the latter (on-prem). Someone else who runs stuff primarily in the cloud might have the opposite takeaway, due to how widely Kubernetes is supported, even despite the technical complexity or resource usage.
That said, personally I think that the K3s Kubernetes distribution is one of the better options for many of the deployments out there: https://k3s.io/
In addition, tools like Kompose can make moving from Swarm to Kubernetes something that would take less than a day: https://kompose.io/ (or you can just use Helm), but until the day where Swarm will get a clear deprecation notice or I'll have a good reason to put a particular project on Kubernetes (as some already are), I'll just stick with Swarm/Nomad.
Of course, if a team/employer tells you to just use technology X, do that. Or better yet, make running your OCI compatible containers someone else's problem.
I am yet to see this warning when running any workload on Swarm. Feature complete is not necessarily deprecated, Swarm simply won't get any fancy cloud platform integrations due to hype lying elsewhere instead, yet there has been no official word of an EOL, like: https://www.centos.org/centos-linux-eol/
Of course, one can also argue that with the Mirantis acquisition the writing is on the wall and if you care about maximum hireability, then you should grok Kubernetes as well.
None of that prevents me from also running Swarm in setups where it makes sense (e.g. a step up from Docker Compose, limited resources, not lots of time for DevOps related stuff etc.).
> I bet Nomad has no future and will surrender in the coming years.
I'm not so sure about this, Nomad integrates nicely in the rest of the HashiStack and it seems that there are plenty of large orgs investing in it, even when they don't need something like Consul or other offerings, due to its simplicity and more active development.
Currently this claim could use more of a factual basis, otherwise it ends up sounding like ".NET will kill Java" or "Linux on the desktop will kill Windows" or something of that variety.
> Using anything but k8s in 2022 is not wise.
If you live exclusively in the cloud and use only managed offerings, that may as well be true.
Otherwise it seems like a clear clash between the arguments of "pick the right tool for the job" and "pick whatever is popular and widely supported". In the deployments I've worked with, Docker Swarm is better for the first claim (limited complexity, low resource usage, easy to use) which matters more than the latter (on-prem). Someone else who runs stuff primarily in the cloud might have the opposite takeaway, due to how widely Kubernetes is supported, even despite the technical complexity or resource usage.
That said, personally I think that the K3s Kubernetes distribution is one of the better options for many of the deployments out there: https://k3s.io/
In addition, tools like Kompose can make moving from Swarm to Kubernetes something that would take less than a day: https://kompose.io/ (or you can just use Helm), but until the day where Swarm will get a clear deprecation notice or I'll have a good reason to put a particular project on Kubernetes (as some already are), I'll just stick with Swarm/Nomad.
Of course, if a team/employer tells you to just use technology X, do that. Or better yet, make running your OCI compatible containers someone else's problem.
I think that the most important thing Kubernetes does is vocabulary. You can replace docker build with podman or kaniko, but Dockerfile is a standard. You’ll not rewrite them. You can use k3s, gke. May be there’ll be independent implementations. But your infra code will be the same. It liberates you from dependency on particular cloud or even technology.
And it enables easier distribution of software. Layer one is source code. You need to build it. Layer two is OCI image. Smart people, often original developers did the build and Linux dependency management, and you just need some runtime to run it. But you still need to manage external service dependencies, which ports to expose, which HTTP paths to expose. K8s yamls with some generator like Helm or Kustomize complete that picture, so you can deploy some service along its dependencies with one command.
Docker-compose could be a better alternative. I like it more. But it’s not feature-complete (e.g. no ingresses, security is a complete afterthought) and at this point will not be a viable competitor.
And it enables easier distribution of software. Layer one is source code. You need to build it. Layer two is OCI image. Smart people, often original developers did the build and Linux dependency management, and you just need some runtime to run it. But you still need to manage external service dependencies, which ports to expose, which HTTP paths to expose. K8s yamls with some generator like Helm or Kustomize complete that picture, so you can deploy some service along its dependencies with one command.
Docker-compose could be a better alternative. I like it more. But it’s not feature-complete (e.g. no ingresses, security is a complete afterthought) and at this point will not be a viable competitor.
Finally a sane response on a HN thread about k8s.
> How do you implement healthcheck?
> Does the loadbalancer know how the healthceck is implemented?
Write a ping/sping endpoint like people did pre-Kubernetes? LBs provided by Cloud provider all have a trivial way of setting a healthcheck endpoint. If you are not in the cloud, nginx and HAProxy make it equally simple to configure a healthcheck. There is no k8s secret sauce behind healtchecks.
> How do you determine it's time to scale?
Monitor relevant metrics (CPU/Memory/DB), have alerts and add new vms or whatever when they hit a specified threshold. Or use your cloud provider's "elastic scaling".
> How do you implement always-on-process? service unit, initd, cron?
Yes. And the problem with that is? Processmanagers have been around forever, are simple to use and a well understood technology.
> How do you export the logs?
Cloud providers generally have a built in mechanism for pumping logs out, if you are not a cloud provider log export daemons exist that much simpler to work than running k8s deployment.
> How do you inject configs? /etc/environment, profile.d, systemd config, /etc/bestestapp/config?
Do whatever your organizations feels comfortable with... config files, json, .properties files, env vars whatever, it doesn't really matter, since the code needs to be aware of the configuration mechanism anyways it doesn't make a difference. And again not worth the tradeoff of running full blown k8s deployment with its significant complexities
> What about secrets?
Cloud providers have convenient ways of solving this problem without involving k8s. If you are not on the cloud one can use cloud secret solutions (AWS Secret Manager, Google Secret Manager, Cloud Vault). All of which are much simpler to use than k8s secrets
> Service discovery? Is unbound/bind9?
DNS works well enough for most projects. Static configuration which lists domain names of whatever services you depend on (which are behind a load balancer) likely works for 90% of all applications.
I am not trying to be argumentative just making a point that "standardizing" all these things you mentioned "the Kubernetes way" are not worth the significant complexity of running and managing k8s for most organizations. And certainly, non-k8s ways of doing these things are much simpler to reason about for developers. A team of 2-3 developers can handle of these things and be able to keep the entire state of their system in there head, whereas with k8s it is suicide to have a k8s cluster without having a dedicated devops team.
And I disagree that there needs to be an industry wide standard of doing these things in the first place (and certainly not a standard like k8s that adds mountains of complexity). The standardization if needed, needs to happen at a team or organization level. Individual organizations should be free to pick and choose how they solve these problems rather than be pressured into the complexity of k8s.
> Does the loadbalancer know how the healthceck is implemented?
Write a ping/sping endpoint like people did pre-Kubernetes? LBs provided by Cloud provider all have a trivial way of setting a healthcheck endpoint. If you are not in the cloud, nginx and HAProxy make it equally simple to configure a healthcheck. There is no k8s secret sauce behind healtchecks.
> How do you determine it's time to scale?
Monitor relevant metrics (CPU/Memory/DB), have alerts and add new vms or whatever when they hit a specified threshold. Or use your cloud provider's "elastic scaling".
> How do you implement always-on-process? service unit, initd, cron?
Yes. And the problem with that is? Processmanagers have been around forever, are simple to use and a well understood technology.
> How do you export the logs?
Cloud providers generally have a built in mechanism for pumping logs out, if you are not a cloud provider log export daemons exist that much simpler to work than running k8s deployment.
> How do you inject configs? /etc/environment, profile.d, systemd config, /etc/bestestapp/config?
Do whatever your organizations feels comfortable with... config files, json, .properties files, env vars whatever, it doesn't really matter, since the code needs to be aware of the configuration mechanism anyways it doesn't make a difference. And again not worth the tradeoff of running full blown k8s deployment with its significant complexities
> What about secrets?
Cloud providers have convenient ways of solving this problem without involving k8s. If you are not on the cloud one can use cloud secret solutions (AWS Secret Manager, Google Secret Manager, Cloud Vault). All of which are much simpler to use than k8s secrets
> Service discovery? Is unbound/bind9?
DNS works well enough for most projects. Static configuration which lists domain names of whatever services you depend on (which are behind a load balancer) likely works for 90% of all applications.
I am not trying to be argumentative just making a point that "standardizing" all these things you mentioned "the Kubernetes way" are not worth the significant complexity of running and managing k8s for most organizations. And certainly, non-k8s ways of doing these things are much simpler to reason about for developers. A team of 2-3 developers can handle of these things and be able to keep the entire state of their system in there head, whereas with k8s it is suicide to have a k8s cluster without having a dedicated devops team.
And I disagree that there needs to be an industry wide standard of doing these things in the first place (and certainly not a standard like k8s that adds mountains of complexity). The standardization if needed, needs to happen at a team or organization level. Individual organizations should be free to pick and choose how they solve these problems rather than be pressured into the complexity of k8s.
> These items are best done in a standard way.
k8s is not a "standard" in any way. It is clunky, prototype-level code that has never been engineered for elegance or ease of use. It's pretty much the same problem systemd has, except to an even greater extent (because clustering is inherently a harder problem than tweaking a single system to enable the use cases systemd devs care about).
k8s is not a "standard" in any way. It is clunky, prototype-level code that has never been engineered for elegance or ease of use. It's pretty much the same problem systemd has, except to an even greater extent (because clustering is inherently a harder problem than tweaking a single system to enable the use cases systemd devs care about).
Anything people agree on is a standard. It may be a horrible standard, but it's still a standard.
> It is clunky, prototype-level code that has never been engineered for elegance or ease of use
How did you come to this conclusion? And what did you compare to? I'm not a devops so I don't have experience with Kubernetes.
How did you come to this conclusion? And what did you compare to? I'm not a devops so I don't have experience with Kubernetes.
You're missing the point of Kubernetes. Kubernetes is so organizations can have a standard way to deploy infrastructure that makes talent and knowledge portable between organizations. Everyone building their infrastructure from a kluge of their own scripts is not the way forward for the industry imo.
You're fair to argue then we should have a less complicated tool that's being standardized on but Kubernetes is where the industry is congregating now and that less complicated tool, that satisfies the most use cases and a high level of standardization, has yet to emerge, there have been multiple that have tried but never gained traction or outright proved to actually be more complex to manage once you go beyond small businesses.
Personally as a maturing profession we should stop reinventing the wheel for much of our tooling, adopt common tooling across as much of the industry as we can and then draft and produce standards engineers can work by so we stop having so much slop across the industry.
You're fair to argue then we should have a less complicated tool that's being standardized on but Kubernetes is where the industry is congregating now and that less complicated tool, that satisfies the most use cases and a high level of standardization, has yet to emerge, there have been multiple that have tried but never gained traction or outright proved to actually be more complex to manage once you go beyond small businesses.
Personally as a maturing profession we should stop reinventing the wheel for much of our tooling, adopt common tooling across as much of the industry as we can and then draft and produce standards engineers can work by so we stop having so much slop across the industry.
I haven’t met two organizations that do Kubernetes the same way lol
I haven't met two organisations that use Linux in the same way either but what it gives you is a common terminology, understanding and way of thinking.
But you have met two organizations that do kubernetes. Kubernetes may be overly flexible, but having everyone on k8s is a heck of a lot more standardized than having everyone writing their own bash files.
Kubernetes creates more technical debt and bullshit jobs than anything I've witnessed in the past two decades.
We've been using K8s to scale incredibly high user and revenue generating services across banks, telcos, insurers for the last half decade. Works pretty well in our opinion ;)
This sounds like you worked for people who wanted to put kubernetes on their CV instead of using it to solve an actual problem.
So, most ‘experts’ then? I try to know as little about k8s and aws as I can but because of ‘people who embellished their resume’ (I am trying to be nice here), I had to because most that had it on their resume did ‘something’ with it but not anything usable. Now I can do it myself (but hate it very much) and ask questions during interviews 99% of so called ‘k8s experts’ (…) cannot answer. And I see this now happening more and more in security as well; people ran a vuns detection script once and put security expert on their resume.
[deleted]
It seems to me perhaps you didn't spend much time with OpenStack.
Have you heard about our lord and saviour React.js?
Be curious how Kubernetes actually creates technical debt.
If Kubernetes is solving a problem of employees, making it easier to find jobs elsewhere, and making it easier for companies to find standard people, why are Kubernetes profiles more expensive?
Dev ops skills are expensive in general for one, but the second reason is Kubernetes is only starting to get to popularity and mass adoption. Look at the 2021 and 2022 stack overflow surveys under tooling to see the difference in adoption rates. It’s ramping up. I’d say it’ll take about 5 years or more until it’s at ubiquity and then that won’t be a hard to find skill and then not more pricey.
very much agree.
moving between cloud providers is rare. being able to transfer knowledge easily is a bigger benefit of k8s.
moving between cloud providers is rare. being able to transfer knowledge easily is a bigger benefit of k8s.
[deleted]
"But the real problem is Kubernetes."
Yeah, that's true. Setting up full k8s with scaling, logging, prometheus, etc. for Wordpress(real example from Acorn) - that's definitely a sign you are doing it wrong. 90% of internet websites don't need k8s at all, docker compose covers all needs at very efficient manner.
Any website need k8s. Reliability, scalability, observability is needed by everyone.
K8s might be too hard to use right now. Too expensive for simple websites. That’s a problem we should solve.
K8s might be too hard to use right now. Too expensive for simple websites. That’s a problem we should solve.
Yes, that’s too expensive, so not for any website. And you can’t solve this.
Why do you think you can’t solve this?
Not me. I’m sane enough. You can’t sell Mercedes with price of Kia.
I don't see any particular difference between selling apache/php/mysql hosting and selling k8s namespace with managed monitoring/logging/gitops/etc. It's not a mercedes, it's the same shared CPU/RAM/Storage, just with different software. May be k8s is not ready yet to host completely different applications on the same cluster, but it just means that there's still work to be done for better isolation.
I can't speak to production deployment of long lived containers, but absolutely for short-lived CI worker nodes and the like? Being able to point your Jenkins or GitLab CI config at a microk8s (or whatever) installation is a huge step forward from configuring non isolated workers or dealing with docker in that role.
(Acorn cofounder) We largely agree ourselves (so much we wrote Rancher 1.x with its own simpler runtime, before k8s took over the world).
But that's the obvious runtime the market is interested in right now. So what we are focusing on is let the infrastructure/ops team do their k8s thing, but insulate the devs/people that just want to run apps from it.
If real demand for it appears, with that abstraction we could write an alternate backend that talks to say Nomad or a-fancy-shell-script-that-runs-docker-cli instead of k8s. And then your app's Acornfile could now run on a totally different runtime, with no (or maybe minimal) changes.
But that's the obvious runtime the market is interested in right now. So what we are focusing on is let the infrastructure/ops team do their k8s thing, but insulate the devs/people that just want to run apps from it.
If real demand for it appears, with that abstraction we could write an alternate backend that talks to say Nomad or a-fancy-shell-script-that-runs-docker-cli instead of k8s. And then your app's Acornfile could now run on a totally different runtime, with no (or maybe minimal) changes.
I’m migrating from few giant servers to k8s. So far I like it.
[deleted]
The New Stack had a writeup that actually covered it's usage nicely. It looks like it's a pretty fast/convenient way to package apps & declare some dependencies (DBs) qyickly, & get a nice managed namespace of everything one would expect.
https://thenewstack.io/acorn-a-lightweight-portable-paas-for...
https://thenewstack.io/acorn-a-lightweight-portable-paas-for...
I don't really see the big value proposition here.
Sure, deploying a single app config is much more awkward than it should be. (Build config: Helm, Jsonnet, Cue, kustomize,...? How to deploy: Gitops/CI, manually , Octopus, Tekton, Argo, ... ?)
But the real difficulty is in connecting different parts of your architecture.
The Getting Started example is extremely flawed: for local development you want a local Postgres and Redis, but in production those will either be provided by the environment (RDS etc) or self-managed,but as a separate deployment because the authorization/lifetime/scaling/upgrade requirements are very different.
Local, staging and production are so diverse that you will always need extensive customization. And there is no way around that, because the needs are fundamentally different.
There definitely is an opportunity to make this easier. It would have to either be done with tooling that has extensive integrations with cloud providers, or with a distinct end to end platform.
But I don't see how Acorn helps with that all.
Sure, deploying a single app config is much more awkward than it should be. (Build config: Helm, Jsonnet, Cue, kustomize,...? How to deploy: Gitops/CI, manually , Octopus, Tekton, Argo, ... ?)
But the real difficulty is in connecting different parts of your architecture.
The Getting Started example is extremely flawed: for local development you want a local Postgres and Redis, but in production those will either be provided by the environment (RDS etc) or self-managed,but as a separate deployment because the authorization/lifetime/scaling/upgrade requirements are very different.
Local, staging and production are so diverse that you will always need extensive customization. And there is no way around that, because the needs are fundamentally different.
There definitely is an opportunity to make this easier. It would have to either be done with tooling that has extensive integrations with cloud providers, or with a distinct end to end platform.
But I don't see how Acorn helps with that all.
At $dayjob someone complained that all of the deployments I had done were inconsistent, and that I wasn’t using repeatable patterns.
I said: until you give me two consistent applications I can’t give you consistent patterns!
Literally everything is unique and special and has distinct requirements for each environment.
The orgs that can get the economies of scale are doing some combination of:
- developing everything in house for the same platform in the same langue with the same tooling.
- deploying thousands of apps, which means even rare configurations will have dozens of identical examples.
- never deploying weird proprietary apps with quirky licensing or crazy pricing models for non-production.
- don’t care about the cost of non-production and are happy to overpay for hardware to ensure consistency with production.
Even at large government departments none of the above is true.
This results in many unique deployments and no cost savings from templating or automation.
I said: until you give me two consistent applications I can’t give you consistent patterns!
Literally everything is unique and special and has distinct requirements for each environment.
The orgs that can get the economies of scale are doing some combination of:
- developing everything in house for the same platform in the same langue with the same tooling.
- deploying thousands of apps, which means even rare configurations will have dozens of identical examples.
- never deploying weird proprietary apps with quirky licensing or crazy pricing models for non-production.
- don’t care about the cost of non-production and are happy to overpay for hardware to ensure consistency with production.
Even at large government departments none of the above is true.
This results in many unique deployments and no cost savings from templating or automation.
We always wasted tons of money in cloud providers to keep alive dev and staging in all my jobs.
In one company they were spending 400k in aws for non production envs (of a 1.2M budget we committed to spend to get better rates).
Tons of microservices, all in node (with three exceptions), a mix of managed databases.
Things were consistent across envs (only the number of replicas were different, same containers) but we could have used less envs.
In one company they were spending 400k in aws for non production envs (of a 1.2M budget we committed to spend to get better rates).
Tons of microservices, all in node (with three exceptions), a mix of managed databases.
Things were consistent across envs (only the number of replicas were different, same containers) but we could have used less envs.
funny if you think that you can hire 2-3 FTE engineers for that hardware easily anywhere non-SV. And then (still) can afford to buy a bit of colocation, fibre and compute (200k for 2000 cores + 8 TB memory we got quoted now) - and that still leaves enough that you just buy NetApp for storage...
To be fair: if you have hundreds (or even tens) of developers that extra infra expenditure is relatively meaningless.
I think that Heroku was really nice, post-acquisition mismanagement aside. My deployments weren't special but I got a lot done.
This exact use case is built into acorn. In the getting starting guide redis/postgres are included in the acorn, but in production those services can easily be overwritten with any k8s service and the embedded containers will not be ran. This is referred to as linking: https://docs.acorn.io/running/linking-acorns The docs only refer to linking acorns but the exact same thing works with linking a k8s service to an acorn.
Disclosure: acorn creator.
Disclosure: acorn creator.
I have good evidence to believe one of the PaaS we use at work uses K8s for deployments under the hood but it’s not like that provides any portability to us; and they certainly don’t advertise that. How is this any different? I guess my point is that at this level of abstraction k8s seems almost irrelevant.
[deleted]
A proprietary wrapper around a few k8s objects? Where's the big win?
It's not clear what problem Acorn is solving.
It's not clear what problem Acorn is solving.
> A proprietary wrapper
Forgive me if this is obvious, but isn't Acorn open source[1]? How is it proprietary?
[1] https://github.com/acorn-io/acorn
Forgive me if this is obvious, but isn't Acorn open source[1]? How is it proprietary?
[1] https://github.com/acorn-io/acorn
They were probably reaching for the opposite of “is defined by a standard with an open-membership standards body”, found that there isn’t really a word for that, and so used the next-closest thing in concept-space.
(Though also, they’re technically correct — “proprietary” just means “controlled by one party.” Open-source things can still be proprietary: namely, if you’re free to fork, but the original upstream can only be merged to by employees of a corporation. I believe e.g. most of Apple’s “open source” is proprietary in this sense.)
(Though also, they’re technically correct — “proprietary” just means “controlled by one party.” Open-source things can still be proprietary: namely, if you’re free to fork, but the original upstream can only be merged to by employees of a corporation. I believe e.g. most of Apple’s “open source” is proprietary in this sense.)
> Though also, they’re technically correct — “proprietary” just means “controlled by one party.” Open-source things can still be proprietary
Awesome, thanks for this explanation -- this is why I asked. I had always equated proprietary to closed source. Good to have my definition expanded. Thank you!
Awesome, thanks for this explanation -- this is why I asked. I had always equated proprietary to closed source. Good to have my definition expanded. Thank you!
I don't understand why everyone is using their own OCI compatible image formats for PaaS, it seems anti user, and likely decreases portability. Adding your own image format over complicates this when you can rely on docker images or nixpacks or whatever. Having 10 competing image formats is anything but useful.
Most things are storing more than just container images in OCI registries. This stores images plus metadata like kubernetes manifests, configuration, etc. all in the registry (I assume, I only skimmed the docs). People do this because the registry is the source of truth for what their production can talk to, for example if they use a managed registry or deploy their own like Harbor it simplifies access control and locking down your service.
What does the following refer to in their GitHub readme?
> Caution: Consuming this project can expose you to chemicals, which are known to the State of California to cause cancer and birth defects or other reproductive harm.
https://github.com/acorn-io/acorn/blob/8a474b9e593956c2ad56b...
> Caution: Consuming this project can expose you to chemicals, which are known to the State of California to cause cancer and birth defects or other reproductive harm.
https://github.com/acorn-io/acorn/blob/8a474b9e593956c2ad56b...
A warning[1] that is present in so many items in California that including it in spurious labels is it's own in-joke.
[1] https://www.nytimes.com/wirecutter/blog/what-is-prop-65/amp/
[1] https://www.nytimes.com/wirecutter/blog/what-is-prop-65/amp/
Yes Kubernetes YAMLs are a PITA. But you can just wrap them in a scripting language like Ruby that makes it easy to create a DSL.
[0]: https://www.brightbox.com/blog/2020/02/12/rubynetes-kubernet...
Use a language rather than a tool if you need the flexibility.
[0]: https://www.brightbox.com/blog/2020/02/12/rubynetes-kubernet...
Use a language rather than a tool if you need the flexibility.
> Use a language rather than a tool if you need the flexibility
The problem with a fully fledged language is the flexibility. There is enough of it to make a mess unless you and your teammates are highly skilled and disciplined. Messy code sucks to deal with, debug, modify; let alone when it's infrastructure code where a wrong logical loop can delete your database or persistent volume.
Infrastructure as code, not with code. DSLs, with their limitations, ensure there's a lot less you can shoot yourself in the foot with, and that everyone vaguely familiar with that DSL can read your code and be absolutely sure what it does, worst case scenario with a Google search or two. No such thing is possible with code in a language as complex and as flexible as Ruby, Python or JavaScript.
Disclaimer: i work at Hashicorp that happens to have one of the most popular IaC DSLs out there, but i have believed that tools like Pulimi are little more than a disaster waiting to happen long before joining.
The problem with a fully fledged language is the flexibility. There is enough of it to make a mess unless you and your teammates are highly skilled and disciplined. Messy code sucks to deal with, debug, modify; let alone when it's infrastructure code where a wrong logical loop can delete your database or persistent volume.
Infrastructure as code, not with code. DSLs, with their limitations, ensure there's a lot less you can shoot yourself in the foot with, and that everyone vaguely familiar with that DSL can read your code and be absolutely sure what it does, worst case scenario with a Google search or two. No such thing is possible with code in a language as complex and as flexible as Ruby, Python or JavaScript.
Disclaimer: i work at Hashicorp that happens to have one of the most popular IaC DSLs out there, but i have believed that tools like Pulimi are little more than a disaster waiting to happen long before joining.
What about when you just want things to work, rather than running into some weird abstraction related issues along the way, that you cannot solve due to any such solution not being popular enough for your use case to be covered?
That's kind of why personally I'd just go for Helm or something of that variety - popular enough to have plenty of tutorials and answers about it, simple enough to just be viewed as a bunch of templating, going from values.yaml to your filled out template files that can be applied against a Kubernetes cluster.
It even generates a lot of what you might need (deployment, service etc.) as a part of the base set of templates, thus you don't have to waste your time trying to write manifests from scratch and have plenty of workable examples for setting env values etc.
That's kind of why personally I'd just go for Helm or something of that variety - popular enough to have plenty of tutorials and answers about it, simple enough to just be viewed as a bunch of templating, going from values.yaml to your filled out template files that can be applied against a Kubernetes cluster.
It even generates a lot of what you might need (deployment, service etc.) as a part of the base set of templates, thus you don't have to waste your time trying to write manifests from scratch and have plenty of workable examples for setting env values etc.
Is it a copy of https://github.com/theketchio/ketch ?
How does this compare against https://porter.run/ ?
[deleted]
Oh dear, also the name of a JS parser https://github.com/acornjs/acorn
How does Rancher have any money left?
Rancher the company was acquired by Suse: https://www.suse.com/c/suse-acquires-rancher/
It looks like Acorn is not related to Rancher at all beyond the creators being former Rancher founders.
It looks like Acorn is not related to Rancher at all beyond the creators being former Rancher founders.
Acornfiles? I feel like when you introduce a new config format, a common expectation will be having a prominent "Why Acornfiles?" page where you explain why this new format is necessary. Without it, I personally groan and look away. With it, you are still wasting time explaining your config format, time that should be spent explaining the benefits of the larger system you're offering.
I'm not sure what existing off-the-shelf thing you would use instead. There is some overlap with things like docker-compose, serverless.yml, helm charts, etc. But each of those things is targeting a different use case and/or level of abstraction.
So if acorn is also a different use case or level of abstraction, it can't really reuse those formats. Their set of 8 top level structures[1] seems reasonable at first glance to me.
[1] args, profiles, containers, volumes, jobs, acorns, secrets, localData
So if acorn is also a different use case or level of abstraction, it can't really reuse those formats. Their set of 8 top level structures[1] seems reasonable at first glance to me.
[1] args, profiles, containers, volumes, jobs, acorns, secrets, localData
I think the problem here is that it introduces a new syntax, not the configuration options itself.
Why not just use yaml as a syntax? At the top of the document they already say it’s very similar anyway, I don’t see what this does that can’t be done with yaml?
http://docs.acorn.io/authoring/overview
Why not just use yaml as a syntax? At the top of the document they already say it’s very similar anyway, I don’t see what this does that can’t be done with yaml?
http://docs.acorn.io/authoring/overview
Ah yep. I didn't look carefully, and just assumed it was json, but looking again it's not.
What do you think is a suitable alternative to custom yaml formats?
Inventing new configuration languages to get away from yaml is a blisteringly terrible plan.
Yaml doesnt spark joy for me, but it's ok & there's lots of good tools & libraries to support it. Json & toml are basically the only other modestly well supported options out there.
This looks like a Yet Another Next Generation (YANG) variant. At least just use YANG. Even though it's vastly less supported.
For real, the amount of griping about YAML is unbelievable. It's not great but so what. It does the job. What people really are griping about usually is how much configuration/state is required. Where the spaces go, whether there's parenthesis or not is so far down the list of important factors. Metric miles behind "are there good tools & libraries?," for one.
Yaml doesnt spark joy for me, but it's ok & there's lots of good tools & libraries to support it. Json & toml are basically the only other modestly well supported options out there.
This looks like a Yet Another Next Generation (YANG) variant. At least just use YANG. Even though it's vastly less supported.
For real, the amount of griping about YAML is unbelievable. It's not great but so what. It does the job. What people really are griping about usually is how much configuration/state is required. Where the spaces go, whether there's parenthesis or not is so far down the list of important factors. Metric miles behind "are there good tools & libraries?," for one.
You can use parenthesis with YAML though. It’s just not a common idiom.
Why is a custom format needed? Why not just use JSON or YAML?
That's totally valid! I don't have an opinion either way. I've heard people complain about yaml before and now seeing a strong reaction to another custom format I was curious to see what people would propose.
It would seem however from the responses I've been getting, that though YAML can be difficult or messy it's still better than another custom format.
It would seem however from the responses I've been getting, that though YAML can be difficult or messy it's still better than another custom format.
I think HCL is very nice. Easy to read and write, has clear types, has basic functions (arithmetic, string manipulation, etc) with the ability to extend with custom functions as well. It can also be easily converted to JSON, which is a pretty cool way of adapting for machine manipulation.
Toml
JSON if you hate comments
JSON if you hate comments
for me, comment is a must for configuration format.
It's JSON with some syntactical sugar. Is that really so bad?
Yes. There are enough formats out there.
In software (unlike many (all?) other fields) there is enough of a lot of things out there; web frameworks, programming languages, crms, content management systems; the list goes on. People feel a flaw of sorts and roll their own. Most won’t get traction and will end up dead on GitHub (hopefully forever though; hopefully archive.org does/will clone all public repos).
Exactly this. I groan every time there’s a new yaml config format to learn, develop for and support. It feels like:
https://xkcd.com/927/
https://xkcd.com/927/
What would you suggest as an alternative? Legitimately curious what people think here.
Interface Script
http://songseed.org/post/20200607.aa.interface_script.html (n-dimension csv)
YAML or JSON, mainly just any format that already exist. I don’t understand why a new syntax is required, especially when the ecosystem they operate in (k8s) is already dominated by YAML.
Just use the Deployment YAMLs we already have.
[deleted]
Overall it looks pretty hacked together. For instance, instead of using an object model or tuples to represent ports, acorn uses a list of strings where each string holds multiple delimited values.
Ex. ports: [ "80/http", "8080/http", ]
While a stylistic choice, it also makes it a PITA for others to write parsers for their DSL, and generally isn’t as readable as say the vanilla Kubernetes port model.
Next let’s take a look at the level of abstraction provided. Typically with a PaaS you are at a higher level of abstraction than an IaaS platform.
args: { // defines arguments the consumer can provide } profiles: { // defines a set of default arguments for different deployment types } containers: { // defines the containers to run the application } volumes: { // defines persistent storage volumes for the containers to consume } jobs: { // defines tasks to run on changes or via cron } acorns: { // other Acorn applications that need to be deployed with your app (databases, etc.) } secrets: { // defines secret bits of data that are automatically generated or passed by the user } localData: { // default data and configuration variables }
Looking at the list above of what we need (or can) provide for a deployment, it doesn’t seem we are at any higher a level of abstraction than we are when just using Kubernetes.
It’s incredibly easy to have the same artifact deployed in multiple environments using Kubernetes with no additional tooling.
So all this leaves me with the question, why?
Ex. ports: [ "80/http", "8080/http", ]
While a stylistic choice, it also makes it a PITA for others to write parsers for their DSL, and generally isn’t as readable as say the vanilla Kubernetes port model.
Next let’s take a look at the level of abstraction provided. Typically with a PaaS you are at a higher level of abstraction than an IaaS platform.
args: { // defines arguments the consumer can provide } profiles: { // defines a set of default arguments for different deployment types } containers: { // defines the containers to run the application } volumes: { // defines persistent storage volumes for the containers to consume } jobs: { // defines tasks to run on changes or via cron } acorns: { // other Acorn applications that need to be deployed with your app (databases, etc.) } secrets: { // defines secret bits of data that are automatically generated or passed by the user } localData: { // default data and configuration variables }
Looking at the list above of what we need (or can) provide for a deployment, it doesn’t seem we are at any higher a level of abstraction than we are when just using Kubernetes.
It’s incredibly easy to have the same artifact deployed in multiple environments using Kubernetes with no additional tooling.
So all this leaves me with the question, why?
> Ex. ports: [ "80/http", "8080/http", ]
> While a stylistic choice, it also makes it a PITA for others to write parsers for their DSL, and generally isn’t as readable as say the vanilla Kubernetes port model.
These are CLI shorthands. There is a canonical expanded-out object that you would use if you were scripting something or talking to the API.
> Looking at the list above of what we need (or can) provide for a deployment, it doesn’t seem we are at any higher a level of abstraction than we are when just using Kubernetes.
Of course the same basic concepts of what to run, where to store output and how to take input are needed to describe computing. But there's two ways to run things ("containers" stay running, "jobs" finish) instead of 8 to choose from (bare pod, deployment, replicationcontroller, replicaset, daemonset, statefulset, cronjob, job) with all slightly varying syntaxes.
The args/volumes/secrets aren't just things that will be created, but also documentation and first-class objects that can be configured/overridden by the consumer of the Acorn. Without you adding dozens of lines of values.yaml and golang template soup to a helm chart to allow every possible configuration a user might want (and never quite the same way in two different helm charts from 2 different people).
You can create an image with reasonable defaults to work for dev/toying around on a laptop; run a local database, generate the secret for it automatically, etc. With the tools needed to ship and run that same artifact in production but pointed at a real DB in RDS. Packaged up as one OCI image so there's one artifact to manage from one source, instead of a helm chart + n images from n containers from n registries.
(Acorn cofounder)
These are CLI shorthands. There is a canonical expanded-out object that you would use if you were scripting something or talking to the API.
> Looking at the list above of what we need (or can) provide for a deployment, it doesn’t seem we are at any higher a level of abstraction than we are when just using Kubernetes.
Of course the same basic concepts of what to run, where to store output and how to take input are needed to describe computing. But there's two ways to run things ("containers" stay running, "jobs" finish) instead of 8 to choose from (bare pod, deployment, replicationcontroller, replicaset, daemonset, statefulset, cronjob, job) with all slightly varying syntaxes.
The args/volumes/secrets aren't just things that will be created, but also documentation and first-class objects that can be configured/overridden by the consumer of the Acorn. Without you adding dozens of lines of values.yaml and golang template soup to a helm chart to allow every possible configuration a user might want (and never quite the same way in two different helm charts from 2 different people).
You can create an image with reasonable defaults to work for dev/toying around on a laptop; run a local database, generate the secret for it automatically, etc. With the tools needed to ship and run that same artifact in production but pointed at a real DB in RDS. Packaged up as one OCI image so there's one artifact to manage from one source, instead of a helm chart + n images from n containers from n registries.
(Acorn cofounder)
[deleted]
Good luck to them and all that but given the significant legacy of the Acorn name in computing history, they could maybe have dug deeper for a title.
How do you run Flink on it anyway?
[deleted]
That has been posted three times in the last 30 days. One time with comments: https://news.ycombinator.com/item?id=32331892
[deleted]
But the real problem is Kubernetes. What do I mean by that? I mean the setup, care and feeding of K8s.
It's nice to have a tool that you can quickly push stuff to K8s. There are other tools out there that do that too.
The thing is this, the majority of people who use K8s, don't need to use K8s. Those people could easily get by with something like Dokku or even just start containers by hand to deploy.
Acorn is just perpetuation the cycle of recommending the over-engineering of K8s for small workloads and small companies that really and truly don't need it.
I'll say it again people. Spin up a few giant servers and call it a day. Save thousands of dollars in hosting costs and tens of thousands if not hundreds of thousands of dollars in engineer hiring costs. Bonus side effect? 10x performance than a passel of wimpy VPS's.
P.S. I'm coming from using K8s in both Google Cloud and on Bare Metal.