Kubernetes YAML Generator(k8syaml.com)
k8syaml.com
Kubernetes YAML Generator
https://k8syaml.com/
126 comments
I've always felt that Kubernetes isn't appropriate for most businesses to use directly, but rather it's a platform for simpler platforms--someone like Heroku would build on top of Kubernetes and expose a much simpler interface for their users so they don't have to think about SSL, DNS, logging, load balancing, auto-scaling, etc. Alternatively, maybe the problem is solved with "distributions" of Kubernetes analogous to Linux distros--preconfigured Kubernetes installations so you don't have to figure out how to configure your own Kubernetes service (arguably this is also what cloud providers give us with their managed k8s offerings). I'm curious to hear what others think.
Lots of PaaS offerings have been doing this for a long time, RedHat OpenShift being the original one; they contributed a lot of 1.0 k8s code and provided a lot of key early architectural feedback by implementing a PaaS using k8s. Another early one was Deis, who were at the first KubeCon.
All this is to say that you're at very least certainly half-correct, in that k8s is a very flexible tool that can be used to build a very simple, elegant, and ergonomic PaaS.
I'm not sure I agree that it's inappropriate for most businesses though, unless you think that only a PaaS like Heroku is appropriate for most businesses; the analogy I'd suggest is "Heroku vs. running your own VMs" circa 2010. Heroku is great for getting started, and lets you move fast by abstracting away a bunch of infra. But it's also restrictive; you can't pick and choose your components freely. As you grow past a certain point you'll almost certainly need the flexibility (or just cost-effectiveness) that you get from running your own infrastructure.
K8s is an improvement here because you can run a managed cluster on something like GKE, which takes away most of the operational toil, while still giving you a lot of flexibility on what components / pieces to include. The k8s domain API does a great job of abstracting away true infra concerns like volumes, compute, scheduling, load-balancing, etc, while making it really easy to package, use, and iterate on the stuff that sits on top of that infrastructure.
I'd probably not encourage a seed-stage startup to use k8s unless you're very familiar with the tool; a PaaS like Heroku would likely be more appropriate. However at the point that you'd usually graduate to running your own VMs (wherever that is on your own company's developmental path), I'd say that using k8s is now a better choice.
All this is to say that you're at very least certainly half-correct, in that k8s is a very flexible tool that can be used to build a very simple, elegant, and ergonomic PaaS.
I'm not sure I agree that it's inappropriate for most businesses though, unless you think that only a PaaS like Heroku is appropriate for most businesses; the analogy I'd suggest is "Heroku vs. running your own VMs" circa 2010. Heroku is great for getting started, and lets you move fast by abstracting away a bunch of infra. But it's also restrictive; you can't pick and choose your components freely. As you grow past a certain point you'll almost certainly need the flexibility (or just cost-effectiveness) that you get from running your own infrastructure.
K8s is an improvement here because you can run a managed cluster on something like GKE, which takes away most of the operational toil, while still giving you a lot of flexibility on what components / pieces to include. The k8s domain API does a great job of abstracting away true infra concerns like volumes, compute, scheduling, load-balancing, etc, while making it really easy to package, use, and iterate on the stuff that sits on top of that infrastructure.
I'd probably not encourage a seed-stage startup to use k8s unless you're very familiar with the tool; a PaaS like Heroku would likely be more appropriate. However at the point that you'd usually graduate to running your own VMs (wherever that is on your own company's developmental path), I'd say that using k8s is now a better choice.
I think I generally agree. Maybe saying "most businesses" is a poor word choice; I don't really have the breadth of experience to speak for the majority of the industry; however, I do strongly suspect that a lot of companies are employing an ops team to wrestle with Kubernetes when their needs could probably be met by a PaaS, and I think this will only get more true as the first generation of PaaS-on-k8s arrive on the market in mature form. I generally agree with this:
> However at the point that you'd usually graduate to running your own VMs (wherever that is on your own company's developmental path), I'd say that using k8s is now a better choice.
However, even then there are intermediate options between VMs and full-on k8s, such as AWS ECS/Fargate, a Kubernetes distro, or managed Kubernetes offering (e.g., GKE) which give me the flexibility to interact with k8s, but they come with sane, pre-configured (or easily configured) solutions for logging, monitoring, ingress, load balancing, upgrades, etc.
> However at the point that you'd usually graduate to running your own VMs (wherever that is on your own company's developmental path), I'd say that using k8s is now a better choice.
However, even then there are intermediate options between VMs and full-on k8s, such as AWS ECS/Fargate, a Kubernetes distro, or managed Kubernetes offering (e.g., GKE) which give me the flexibility to interact with k8s, but they come with sane, pre-configured (or easily configured) solutions for logging, monitoring, ingress, load balancing, upgrades, etc.
That's exactly what I am doing with https://primcloud.com :)
This seems really cool; I was poking around, but half of the links in your footer are broken (presumably placeholders for future pages?).
Yes sorry, I had a simple splash landing page up but people were asking for more info, so I put this up while I continue building the infra/platform.
I will over time expand on the content, but I'm focusing on the platform itself over marketing stuff.
I will over time expand on the content, but I'm focusing on the platform itself over marketing stuff.
Do you support on-premises kubernetes?
I'm still building so I currently don't. Kubernetes is just a tool I use to deploy my customers sites on, it's not the selling point.
I do however have plans of supporting enterprise by packaging a version of the app into an on-prem deployment system so you can just deploy your own kubernetes cluster and install the app and have your own enterprise version of Primcloud.
I do however have plans of supporting enterprise by packaging a version of the app into an on-prem deployment system so you can just deploy your own kubernetes cluster and install the app and have your own enterprise version of Primcloud.
https://kubesail.com offers the exact service you describe!
Thanks for the shout-out! We’re trying to be like an “open-Heroku” - back when I worked on OpenStack it was called “the open cloud” and no one knew what we were on about - open stack was enterprise as enterprise gets!
Our goal is to make a Heroku-ish platform for getting an app online - but one that doesn’t hold you over a barrel later on. You can even host from a spare home server :)
(Disclosure: I’m the CTO)
Our goal is to make a Heroku-ish platform for getting an app online - but one that doesn’t hold you over a barrel later on. You can even host from a spare home server :)
(Disclosure: I’m the CTO)
Do you still expose the k8s bits?
I'm interested in/have previously given up on a product where the control plane is managed for me, I can join the cluster with bare metal nodes, and then it's just Kubernetes.
I'm interested in/have previously given up on a product where the control plane is managed for me, I can join the cluster with bare metal nodes, and then it's just Kubernetes.
Yep - we expose the Kubernetes API (and add mutating controllers to help automate some of its actions).
Right now we don't do "hosted control plane", rather - you can either use our hosted clusters where you have namespace-level access (so you still get to use Kubernetes, just not all of the resources, for example, no `Nodes`), or you can attach your own cluster to our UI (with the advantage that we can setup ingress and forward traffic to you, which is perfect for a home-hosting setup).
Hosted control plane is something im watching closely though - I'd really love to offer that as a service, but since we're small, we're trying to focus hard on a core offering. Will be considered in the future though!
Right now we don't do "hosted control plane", rather - you can either use our hosted clusters where you have namespace-level access (so you still get to use Kubernetes, just not all of the resources, for example, no `Nodes`), or you can attach your own cluster to our UI (with the advantage that we can setup ingress and forward traffic to you, which is perfect for a home-hosting setup).
Hosted control plane is something im watching closely though - I'd really love to offer that as a service, but since we're small, we're trying to focus hard on a core offering. Will be considered in the future though!
kubesail has really nice UI for an hosted option. Another alternative I found is https://kalm.dev/, which is open sourced and geared towards configuring a heroku on top of a k8s cluster.
100% agree. Kubernetes alone is just a """framework""" to describe your infrastructure, definitely not an "end-user product".
That is why in many businesses there is an OPS team managing the Kubernetes and providing tools like Cert-Manager, Istio, ... and the rest of the company who just use what the OPS team made.
Right now, everyone is building its own distro, proving IMHO the need for it.
That is why in many businesses there is an OPS team managing the Kubernetes and providing tools like Cert-Manager, Istio, ... and the rest of the company who just use what the OPS team made.
Right now, everyone is building its own distro, proving IMHO the need for it.
Well said!
Most of my colleagues find the yaml interface to be a lost cause. I tend to prefer to use the gRPC API as it's effectively the same declarative operations but you get to control it in software, which just feels right. YAML is just a crutch for specifying the API calls the client needs to make.
You mean the Kubernetes HTTP API? That one isn’t gRPC based, right?
Ah right, guess I was thinking of the protobuf over HTTP API. In any case, I find using the protobuf types in your language of choice pretty ergonomic compared to YAML.
This is one thing that mesos/aurora got right (the "turing-complete-by-design" config files, written in Python). Templating yaml is like templating html. It works but absolutely sucks at scale if your job is to maintain it.
It's good to know that such an interface could conceivably be built for Kubernetes, using the HTTP API. Even better that it supports any programming language (since it's a network interface); I'd see that as a step-up from aurora configs, in fact.
At some point, I imagine this will become the default way that Smart Organizations build around k8s, once k8s's core API is super-stable and people develop sound frameworks in $POPULAR_LANGUAGE for declaring your configuration.
Then, we can then banish yaml to the same fate as json and xml. One can dream, at least. :)
It's good to know that such an interface could conceivably be built for Kubernetes, using the HTTP API. Even better that it supports any programming language (since it's a network interface); I'd see that as a step-up from aurora configs, in fact.
At some point, I imagine this will become the default way that Smart Organizations build around k8s, once k8s's core API is super-stable and people develop sound frameworks in $POPULAR_LANGUAGE for declaring your configuration.
Then, we can then banish yaml to the same fate as json and xml. One can dream, at least. :)
> the "turing-complete-by-design" config files, written in Python
Funny you mention this, check out what is effectively this but for Kubernetes:
https://github.com/stripe/skycfg
Funny you mention this, check out what is effectively this but for Kubernetes:
https://github.com/stripe/skycfg
Wow, this is great! Thank you for sharing.
SMART_ORGANIZATIONS=$SMART_ORGANIZATIONS:stripe
I wonder if they were inspired by aurora, it's very similar.
SMART_ORGANIZATIONS=$SMART_ORGANIZATIONS:stripe
I wonder if they were inspired by aurora, it's very similar.
Would you happen to have any links to Github repos that would be an example of this?
From light googling I only see examples of building gRPC APIs and microservices.
From light googling I only see examples of building gRPC APIs and microservices.
I don't have an example offhand, but mentioned skycfg in the neighbor comment, which may be of some interest to you. I've used
https://github.com/ericchiang/k8s in the past with great success.
[deleted]
I think of Kubernetes as "the new Operating System", and these complex resources as fiddling with initscripts, fstab, /etc/interfaces, and so on. Writing an operator is like writing your own initscript.
I wouldn't be surprised if we eventually see new abstractions for "you just want a plain 'ol deployment with CI/CD du jour, a persistent volume claim, and a service ingress, just like 90% of all other CRUD webapps? Sure, here's a simple resource for that."
I think we'll start seeing a move towards more "opinionated" tools, just to outsource some of the decision making. No sense learning how to write your own pipelines if you can find a tool that says "we're gonna deploy every time you make a git tag and run `mvn package`, you figure the rest out".
I wouldn't be surprised if we eventually see new abstractions for "you just want a plain 'ol deployment with CI/CD du jour, a persistent volume claim, and a service ingress, just like 90% of all other CRUD webapps? Sure, here's a simple resource for that."
I think we'll start seeing a move towards more "opinionated" tools, just to outsource some of the decision making. No sense learning how to write your own pipelines if you can find a tool that says "we're gonna deploy every time you make a git tag and run `mvn package`, you figure the rest out".
And the tower of ever more obscure abstractions just keeps on growing. Until one of these fragile layers starts to malfunction and the whole thing comes tumbling down and nobody has a clue how to fix it all up.
Collectively, as an industry we have gone insane.
Collectively, as an industry we have gone insane.
People have attempted simpler abstractions and it has always failed. CloudFoundry, Mesos+Marathon, Heroku were all simple at first and none of them revolutionized the industry.
Agreed. Just like how I don't want to roll my own desktop environment on Linux, I will happily accept a good config rollef for me, even if opinionated.
Well prepare to be unsurprised! The CI/CD abstraction already exists in Tekton - https://cloud.google.com/tekton
The rest are soon to follow I’m sure.
The rest are soon to follow I’m sure.
If people think deployment yamls are complicated, what til they see tekton.
Don’t get me wrong, I actually love tekton because I hate everything that is Jenkins and most other CICD tool integration with k8s. So tekton is just amazing in that regard.
But there is a huge learning curve for people who are used to old school tools like Jenkins and its pipelines .
Don’t get me wrong, I actually love tekton because I hate everything that is Jenkins and most other CICD tool integration with k8s. So tekton is just amazing in that regard.
But there is a huge learning curve for people who are used to old school tools like Jenkins and its pipelines .
I think of writing Operators more as writing your own Device Driver but I can see the correlation you're trying to make.
Helm has been doing this since the early days. 99% of our charts are created using:
* `helm create` to get the default scaffold
* modify a handful of entries in the generated values file
* done!
Only thing is the default helm chart starter does not allow for autoconfiguring of volumes, and since we're porting a lot of stateful apps to kubernetes we just modified the default starter to include that capability.
Of course it would be nice to not have to maintain a bunch of different virtually identical templates.
Pulumi looks interesting but unfortunately seems to insist on vendor lock-in (see the jerk-around on https://github.com/pulumi/pulumi/pull/2697). So I'm looking forward to the AWS CDK (https://aws.amazon.com/cdk/) maturing a bit.
* `helm create` to get the default scaffold
* modify a handful of entries in the generated values file
* done!
Only thing is the default helm chart starter does not allow for autoconfiguring of volumes, and since we're porting a lot of stateful apps to kubernetes we just modified the default starter to include that capability.
Of course it would be nice to not have to maintain a bunch of different virtually identical templates.
Pulumi looks interesting but unfortunately seems to insist on vendor lock-in (see the jerk-around on https://github.com/pulumi/pulumi/pull/2697). So I'm looking forward to the AWS CDK (https://aws.amazon.com/cdk/) maturing a bit.
As someone attempting to port a docker-compose application to a Helm chart I'd love to hear more about the resources you used. As someone working with a very simple application I've found the processes to be more difficult than I anticipated.
Kubernetes will be far more complex than docker compose. It'll let you do much, much more also.
Some potentially useful tips:
* One docker container usually maps to one pod, which is created by a deployment. You can put multiple containers in a single pod, but this couples them together tightly.
* Use services to assign hostnames to pods
* Have pods communicate to each other using the service names. This works the same as putting them in the same docker network.
* Volumes depend on your cloud provider or if you're running on bare metal. In the cloud it's easy, you just request one and it gets created on demand and backed by a cloud disk.
* If you're using volumes, you probably want to use the Recreate updatePolicy for your deployment. This will ensure the old pod is shutdown before creating a new one. Which is necessary to work with block volumes.
When using helm start with a `helm create CHARTNAME` and take a look at what it generated for you. You'll get some heavily templated yaml that if you're lucky you will barely have to touch.
But it's best to go through these tutorials and learn how to use the basic building blocks of kubernetes directly: https://kubernetes.io/docs/tutorials/kubernetes-basics/
Once you're familiar with what a Deployment, Service, Ingress and PersistentVolumeClaim are you can use helm to template this for you where necessary.
Some potentially useful tips:
* One docker container usually maps to one pod, which is created by a deployment. You can put multiple containers in a single pod, but this couples them together tightly.
* Use services to assign hostnames to pods
* Have pods communicate to each other using the service names. This works the same as putting them in the same docker network.
* Volumes depend on your cloud provider or if you're running on bare metal. In the cloud it's easy, you just request one and it gets created on demand and backed by a cloud disk.
* If you're using volumes, you probably want to use the Recreate updatePolicy for your deployment. This will ensure the old pod is shutdown before creating a new one. Which is necessary to work with block volumes.
When using helm start with a `helm create CHARTNAME` and take a look at what it generated for you. You'll get some heavily templated yaml that if you're lucky you will barely have to touch.
But it's best to go through these tutorials and learn how to use the basic building blocks of kubernetes directly: https://kubernetes.io/docs/tutorials/kubernetes-basics/
Once you're familiar with what a Deployment, Service, Ingress and PersistentVolumeClaim are you can use helm to template this for you where necessary.
You can give a shot at kompose (https://github.com/kubernetes/kompose) It will convert your docker-compose.yml to kubernetes automatically
Helm is an example of what I mean by "pusing against the boundaries."
We tried using helm, but we kept running into limitations, and the complexity of trying to customize charts that didn't work right ended up being worse than the original problem.
Currently, we're using kustomize, plus a thought out manifest hierarchy, which is working out better.
Agreed. Where I work we have a core team that created a framework that manages the Kubernetes configuration for us. As a backend developer I basically just have to throw a config file into my service with some basic options like which port to listen on and where the docker container is, and then it gets deployed with everything setup for me (using helm I believe).
I really hope we move more towards these opinionated tools that can handle 90% of the use cases. Most people just want to host an app on a port, and it's a pain to have to develop that pipeline at every company that wants to adopt Kubernetes.
I really hope we move more towards these opinionated tools that can handle 90% of the use cases. Most people just want to host an app on a port, and it's a pain to have to develop that pipeline at every company that wants to adopt Kubernetes.
Is it anymore complex than all the old ways?
Was Apache, Asterisk, or loading and hardening a Linux host on bare metal easier?
I seem to remember a lot of wrangling custom kernels to get Asterisk sounding just right, bizarre Apache, & network configs.
It’s just text? It’s always going to turn into a nebulous mess without literal edges and boundaries.
That’s Google’s play with it, IMO. Train tracks. Which is what I hate about it.
Google hasn’t built a less Byzantine text mess. It’s built hype though, with a boring tool
Was Apache, Asterisk, or loading and hardening a Linux host on bare metal easier?
I seem to remember a lot of wrangling custom kernels to get Asterisk sounding just right, bizarre Apache, & network configs.
It’s just text? It’s always going to turn into a nebulous mess without literal edges and boundaries.
That’s Google’s play with it, IMO. Train tracks. Which is what I hate about it.
Google hasn’t built a less Byzantine text mess. It’s built hype though, with a boring tool
There are plenty of online tools to generate Nginx or Vagrant config files.
I don't really get the love for hating on K8 complexity on here - nobody says it's the perfect tool for every use case... but when you do need it, it's amazing, and has many advantages over the current trend for serverless IMO.
I don't really get the love for hating on K8 complexity on here - nobody says it's the perfect tool for every use case... but when you do need it, it's amazing, and has many advantages over the current trend for serverless IMO.
> Is it anymore complex than all the old ways?
> Was Apache, Asterisk, or loading and hardening a Linux host on bare metal easier?
Yes, and by far. Adding a layer on top of all the traditional Linux daemons, tools and libraries does not decrease the total complexity - quite the contrary.
When you have a bug in an application that is related to something in on another layer you have to walk through the whole stack.
Examples: A bug in a network card impacting only large UDP packets. A race condition of file access triggered by NFS or a storage device driver. A vulnerability based on a timing attack due to CPU caches.
The deeper the stack, the worse.
> Was Apache, Asterisk, or loading and hardening a Linux host on bare metal easier?
Yes, and by far. Adding a layer on top of all the traditional Linux daemons, tools and libraries does not decrease the total complexity - quite the contrary.
When you have a bug in an application that is related to something in on another layer you have to walk through the whole stack.
Examples: A bug in a network card impacting only large UDP packets. A race condition of file access triggered by NFS or a storage device driver. A vulnerability based on a timing attack due to CPU caches.
The deeper the stack, the worse.
But wouldn’t the point be that you don’t care about hardware level problems anymore? When I find a node with issues, I can just delete it from the pool and get a fresh one back. The bad network card? That’s for Google/Amazon/DigitalOcean to deal with.
I find the bog-standard Prometheus chart provides me a pretty incredible level of monitoring out of the box, usually it’s pretty easy to pick the bad one out of a graph.
Running your own VMs without something like k8s? Yeah this setup I can deploy and have working in an hour is gonna take you a week to set up properly. Standardization is valuable. Abstraction is valuable.
I find the bog-standard Prometheus chart provides me a pretty incredible level of monitoring out of the box, usually it’s pretty easy to pick the bad one out of a graph.
Running your own VMs without something like k8s? Yeah this setup I can deploy and have working in an hour is gonna take you a week to set up properly. Standardization is valuable. Abstraction is valuable.
> But wouldn’t the point be that you don’t care about hardware level problems anymore?
No. Read my post again: I did not wrote about hardware issues.
Most work around optimization, reliability or security require digging through the whole stack sometimes down to the kernel.
> When I find a node with issues, I can just delete it from the pool and get a fresh one back.
However, a lot of k8s deployments are on-premise, where you have to debug your own hardware.
> The bad network card? That’s for Google/Amazon/DigitalOcean to deal with.
First you have to pinpoint the root cause of that glitch affecting all the containers running on VMs using the same bad drivers. Often it could be the same in 50% of your fleet.
> is gonna take you a week to set up properly.
Most certainly not. I've been deploying large production fleets in minutes since 2005.
No. Read my post again: I did not wrote about hardware issues.
Most work around optimization, reliability or security require digging through the whole stack sometimes down to the kernel.
> When I find a node with issues, I can just delete it from the pool and get a fresh one back.
However, a lot of k8s deployments are on-premise, where you have to debug your own hardware.
> The bad network card? That’s for Google/Amazon/DigitalOcean to deal with.
First you have to pinpoint the root cause of that glitch affecting all the containers running on VMs using the same bad drivers. Often it could be the same in 50% of your fleet.
> is gonna take you a week to set up properly.
Most certainly not. I've been deploying large production fleets in minutes since 2005.
A lot of other "bleeding edge" technologies that blow up and become difficult to use end up with new abstractions, and then abstractions for the abstractions, and the original problem the tool was meant to solve is now not the focus any longer.
I find K8s to be the perfect tool for enterprises.
It provides the promise of portability, so the business feels less locked in to a vendor, while still being dense and impenetrable enough to them that IT won’t be done out of a job.
Simpler, more efficient alternatives are seen as more expensive, because IT hides the true cost of just how much of their salaries is spent of non-productive pottering around with yaml.
These systems and products also threaten to automate IT out of a job and don’t necessarily count much for their resumes. If you’re not a programmer what interest do you have in dumping all of your intricate, fragile environment for a PaaS? Not much
It provides the promise of portability, so the business feels less locked in to a vendor, while still being dense and impenetrable enough to them that IT won’t be done out of a job.
Simpler, more efficient alternatives are seen as more expensive, because IT hides the true cost of just how much of their salaries is spent of non-productive pottering around with yaml.
These systems and products also threaten to automate IT out of a job and don’t necessarily count much for their resumes. If you’re not a programmer what interest do you have in dumping all of your intricate, fragile environment for a PaaS? Not much
Isn't that the case with any tool or service of some complexity? I'd probably have to look at docs or examples for most config files i write.
There's trade-off between a tool being too specific to a usecase vs being too generic with the associated "boilerplate". But I don't know how much simpler Kubernetes could be made while still covering the intended scope?
There's trade-off between a tool being too specific to a usecase vs being too generic with the associated "boilerplate". But I don't know how much simpler Kubernetes could be made while still covering the intended scope?
I would say kubernetes is very simple, in the same way lambda calculus or composing programs out of pipes is simple.
It has very simple basic model, which allows you to recursively built more and more complex abstractions on top of it.
It has very simple basic model, which allows you to recursively built more and more complex abstractions on top of it.
I like reading declarative docs because everything is spelled out but I hate writing them for the same reasons.
More tooling to write good k8s yaml sounds good to me.
More tooling to write good k8s yaml sounds good to me.
It's a never-ending cycle. See: Front-end development. Until yet another abstraction emerges, anyone using Kubernetes for mission-critical applications should be using automated testing to prevent awful outcomes from a single botched line of YAML. There are a bunch of solutions emerging in this space, like Datree[0], made for the sole purpose of preventing Kubernetes misconfigurations.
[0] https://datree.io (Disclaimer: I work with them.)
[0] https://datree.io (Disclaimer: I work with them.)
> I can't write a deployment yaml without googling or copy/pasting
What about Nginx or Apache configuration files? Could you write them without googling?
What about Nginx or Apache configuration files? Could you write them without googling?
What about Nginx or Apache configuration files? Could you write them without googling?
Easily, because noone writes them from scratch, they just modify the default one.
For most of my Kubernetes work I "kubectl describe" something existing into YAML, modify it, then "kubectl apply" it back again.
Easily, because noone writes them from scratch, they just modify the default one.
For most of my Kubernetes work I "kubectl describe" something existing into YAML, modify it, then "kubectl apply" it back again.
You can do this without a cluster also:
kubectl create deployment my-deployment --image nginx --dry-run -oyamlIMO it needs two things:
- a dedicated editor with intelligent autocomplete
- stop using YAML, it soon becomes unreadable. JSON is easier to grok.
- a dedicated editor with intelligent autocomplete
- stop using YAML, it soon becomes unreadable. JSON is easier to grok.
JSON is easier to grok.
For anything non-trivial you will want inline comments.
Also while any JSON can be expressed in YAML, the reverse is not true.
For anything non-trivial you will want inline comments.
Also while any JSON can be expressed in YAML, the reverse is not true.
> For anything non-trivial you will want inline comments.
Yes, JSON needs comment support back.
> Also while any JSON can be expressed in YAML, the reverse is not true.
This is a feature not a bug.
Yes, JSON needs comment support back.
> Also while any JSON can be expressed in YAML, the reverse is not true.
This is a feature not a bug.
> a dedicated editor with intelligent autocomplete
A.K.A. a schema file that a general purpose editor can consume. Please don't make me use some single-purpose editor just for autocomplete.
A.K.A. a schema file that a general purpose editor can consume. Please don't make me use some single-purpose editor just for autocomplete.
Kubernetes schema files and a way to run them can be found at https://www.kubeval.com/
Isn't YAML supposed to be a superset of JSON? So technically, you should be able to use JSON files right now (barring a few exceptions).
Unfortunately, JSON is much more annoying to edit.
Best way is to stop using both, and generate the objects from higher level language, at least something like Jsonnet (which is really just a step up, so better not stop there but I will take what I can)
Best way is to stop using both, and generate the objects from higher level language, at least something like Jsonnet (which is really just a step up, so better not stop there but I will take what I can)
> generate the objects from higher level language
I'm on the fence about this, something like Dahl seems good, but I am not a fan of using fully-fledged higher level languages (the Pulumi approach) as I've been burnt before with devs writing bad code and generating configs in the most confusing, convoluted way possible.
I'm on the fence about this, something like Dahl seems good, but I am not a fan of using fully-fledged higher level languages (the Pulumi approach) as I've been burnt before with devs writing bad code and generating configs in the most confusing, convoluted way possible.
Complexity always exists somewhere. You can't remove it, only try it make it easier to deal with.
What Kubernetes allows you to do is very complex so you need a capable system to express it all. YAML isn't always the best but it works fine for most and tools like these are very helpful.
Do you use an IDE? Does that mean the language and framework is too complex? No, it's just a tool to help you get things done. More tools aren't a bad thing.
What Kubernetes allows you to do is very complex so you need a capable system to express it all. YAML isn't always the best but it works fine for most and tools like these are very helpful.
Do you use an IDE? Does that mean the language and framework is too complex? No, it's just a tool to help you get things done. More tools aren't a bad thing.
>Complexity always exists somewhere. You can't remove it, only try it make it easier to deal with.
This is simply not true. Most complexity in today's systems is completely avoidable. Most developers are just mentally stuck and don't even try. "Managing" complexity is a great way to achieve job security without becoming good at anything specific. Instead of learning how to design system people are learning how to write config files.
This is simply not true. Most complexity in today's systems is completely avoidable. Most developers are just mentally stuck and don't even try. "Managing" complexity is a great way to achieve job security without becoming good at anything specific. Instead of learning how to design system people are learning how to write config files.
That's a different argument. Not needing it doesn't mean it doesn’t exist.
If you don't need Kubernetes then don't use it. But if you do then you can't make it any simpler than the complexity needed to deliver the functionality.
Built in deployments, process health, logging, load balancing, security, high availability, config and secret management, volumes and persistence, and much more in exchange for some YAML files is a pretty good encapsulation of complexity though.
If you don't need Kubernetes then don't use it. But if you do then you can't make it any simpler than the complexity needed to deliver the functionality.
Built in deployments, process health, logging, load balancing, security, high availability, config and secret management, volumes and persistence, and much more in exchange for some YAML files is a pretty good encapsulation of complexity though.
Agree with this, and would say that often a good way to deal with complexity is encapsulation. I imagine there are some developers that say "why wouldn't you just write it in assembly" Because if you are used to that and write small low level binaries it might work well. But trying to write a larger app it becomes more difficult to reason about.
Not all complexity is intrinsic, but all complexity incurs a cost.
Needing an IDE to have a reasonable time is a very common criticism against the expressive power of certain languages.
I don't think better abstraction is going to help the underlying problem: there are a lot of questions to answer, and you don't know the answer.
Without going into the complexity of Deployments, consider the lowly Pod. What configuration does your app need? What is the name of the container that contains it? How much memory does it use? How much CPU does it need? What ports does it listen on? What HTTP endpoint handles the health check? Does that endpoint test liveness or readiness? What filesystems does it need? What setup needs to be done before the main container runs? Does it need any special resources like GPUs? The list goes on.
The problem here is that when you're writing a Pod spec, you're building a single-purpose computer from scratch. In the traditional UNIX world, people answered most of these questions for you. How much RAM can my app use? However much I plugged in. How much CPU can my app use? All of them. What ports does it listen on? Any of them from 1024-65535. What filesystems does it need? Whichever ones I setup in /etc/fstab.
I don't think it's a stretch to call UNIX's "yolo" approach problematic. It is great when you have one server running one app, but servers have gotten gigantic (with pricing to match) while applications have largely stayed the same size. This means you have to pack multiple apps onto one physical server, and to do that, there have to be rules. When you write a Kubernetes manifest, you are just answering every possible question upfront so that the entire system runs smoothly even if your individual component doesn't. It's the cost of having small apps on big computers.
The problem comes from applications that you didn't write, or don't fully understand. Before you can understand how the application behaves, you have to write a manifest. But you don't know the answers to the questions like how much CPU you're going to use, or what the worst case memory usage is, etc. This causes a lot of cognitive dissonance, because the entire file is you admitting to the computer that you have no idea how to configure it. No abstraction layer is going to fix that problem, except by hiding those uncomfortable details from you. (And you will always regret using the "yolo" defaults -- who hasn't tried to SSH into a broken server only to have Linux helpfully OOMKill sshd or your bash instance when you're just trying to kill your malfunctioning app.)
This is largely the fault of application developers. They aren't willing to commit to reasonable resource limits because they don't want to handle support requests that are related to underprovisioning. My experience is that applications that set limits pick them wrong. For example, GCP and DigitalOcean's managed Kubernetes offerings both install monitoring agents to support their dashboards; these apps ship with limits that are too low and any reasonable Prometheus installation will notice that they are being CPU throttled and warn you about it. Now you have to waste your day asking "is this a real problem?"
Many open-source apps go the other way and pick resource limits that truly encapsulate the worst case and require individual nodes that are many times larger than the entire cluster. Yes, it would be nice if I gave each pod 32 CPUs and 128GiB of RAM... but I don't want to pay $2000/month/replica thankyouverymuch. (I've been on the other side of that where resources didn't cost me real money and happily used terabytes of RAM as cache.)
Application-level configuration is also not in a great state. Everyone tries to sell you their curated defaults so they don't have to write any documentation beyond a "quick start". (I'm as guilty of that as anyone in fact!) The application will have some built-in defaults (so the developers writing the app can just "go run main.go" and get the config they need). Then someone comes along to make a Helm chart for you, and they change the defaults so that their local installation doesn't need any customization. This only causes problems because instead of an undocumented underlying application, now you have that AND an undocumented abstraction layer. You may find the answer to your question "how do I configure FooApp to bar?" but have no way of communicating that config through the Helm abstraction layer because the author of the Helm chart never thought anyone would do that.
This rant has gotten quite long so I'll wrap it up. No abstraction layer is ever going to make it so you don't need to answer difficult questions. The actual list of questions to answer is available through "kubectl explain pod.spec" and friends, however.
Without going into the complexity of Deployments, consider the lowly Pod. What configuration does your app need? What is the name of the container that contains it? How much memory does it use? How much CPU does it need? What ports does it listen on? What HTTP endpoint handles the health check? Does that endpoint test liveness or readiness? What filesystems does it need? What setup needs to be done before the main container runs? Does it need any special resources like GPUs? The list goes on.
The problem here is that when you're writing a Pod spec, you're building a single-purpose computer from scratch. In the traditional UNIX world, people answered most of these questions for you. How much RAM can my app use? However much I plugged in. How much CPU can my app use? All of them. What ports does it listen on? Any of them from 1024-65535. What filesystems does it need? Whichever ones I setup in /etc/fstab.
I don't think it's a stretch to call UNIX's "yolo" approach problematic. It is great when you have one server running one app, but servers have gotten gigantic (with pricing to match) while applications have largely stayed the same size. This means you have to pack multiple apps onto one physical server, and to do that, there have to be rules. When you write a Kubernetes manifest, you are just answering every possible question upfront so that the entire system runs smoothly even if your individual component doesn't. It's the cost of having small apps on big computers.
The problem comes from applications that you didn't write, or don't fully understand. Before you can understand how the application behaves, you have to write a manifest. But you don't know the answers to the questions like how much CPU you're going to use, or what the worst case memory usage is, etc. This causes a lot of cognitive dissonance, because the entire file is you admitting to the computer that you have no idea how to configure it. No abstraction layer is going to fix that problem, except by hiding those uncomfortable details from you. (And you will always regret using the "yolo" defaults -- who hasn't tried to SSH into a broken server only to have Linux helpfully OOMKill sshd or your bash instance when you're just trying to kill your malfunctioning app.)
This is largely the fault of application developers. They aren't willing to commit to reasonable resource limits because they don't want to handle support requests that are related to underprovisioning. My experience is that applications that set limits pick them wrong. For example, GCP and DigitalOcean's managed Kubernetes offerings both install monitoring agents to support their dashboards; these apps ship with limits that are too low and any reasonable Prometheus installation will notice that they are being CPU throttled and warn you about it. Now you have to waste your day asking "is this a real problem?"
Many open-source apps go the other way and pick resource limits that truly encapsulate the worst case and require individual nodes that are many times larger than the entire cluster. Yes, it would be nice if I gave each pod 32 CPUs and 128GiB of RAM... but I don't want to pay $2000/month/replica thankyouverymuch. (I've been on the other side of that where resources didn't cost me real money and happily used terabytes of RAM as cache.)
Application-level configuration is also not in a great state. Everyone tries to sell you their curated defaults so they don't have to write any documentation beyond a "quick start". (I'm as guilty of that as anyone in fact!) The application will have some built-in defaults (so the developers writing the app can just "go run main.go" and get the config they need). Then someone comes along to make a Helm chart for you, and they change the defaults so that their local installation doesn't need any customization. This only causes problems because instead of an undocumented underlying application, now you have that AND an undocumented abstraction layer. You may find the answer to your question "how do I configure FooApp to bar?" but have no way of communicating that config through the Helm abstraction layer because the author of the Helm chart never thought anyone would do that.
This rant has gotten quite long so I'll wrap it up. No abstraction layer is ever going to make it so you don't need to answer difficult questions. The actual list of questions to answer is available through "kubectl explain pod.spec" and friends, however.
In my opinion, Kube's yaml blobs are comparable to binary programs. Complaining that an executable has its settings, paths, ports, memory limits, etc, hard coded inside of it is an obvious code smell.
Tools like Helm solve this problem.
Tools like Helm solve this problem.
Abstractions have a cost and a value. Programming languages are more expressive than writing out binary opcodes by hand, but don't really remove any features from the underlying machine. If your machine can do something, your C program can probably do something. So we use C. (It even ADDS the value that you can describe a procedure without knowledge of the underlying hardware. It may not be optimal on every machine, but at least you have a starting point for optimization!)
Helm, like any abstraction, has costs and values. For example, it's very valuable to be able to encode something like "my app consists of a frontend and a backend that run in the same Pod and must each use the same version of the code":
The problem with this abstraction is that there's no escape hatch. If you wanted to run different versions of foo-frontend and foo-backend, there is no way to say "but no, really, this time I'm violating the rules for a good reason". You've reduced the features available... the only way forward is to start over with nothing.
The result of this is that every individual Helm chart has to account for every possibility that the manifests could possibly encode, and invent their own programming language that is identical in expressiveness to the underlying manifests. And they do! Differently every time! For example, if I wanted to allow people to override the container images, I'd have to make my template look like:
I think we can all agree that didn't save anyone any time or effort.
This example conveniently flows into my other complaint with Helm. The "yaml files" that declare templates aren't actually valid YAML. You can't use something like `prettier` to autoformat them. You can't use the YAML language server to provide code advice as you type. You can't use `kubeval` to validate them. You are throwing all of that away to Build Your Own Thing. It is actually very insidious and for that reason I consider Helm to be more harmful than helpful. It isn't an abstraction, it's just a macro that might be good for one person the instant they happened to type something in.
The other problem is that Helm charts have no upgrade path. They are only designed for "please explode this project into my cluster, I promise to clean it up In The Future". It never gets cleaned up and brings a little piece of un-updated Windows 95 nostalgia right into your cluster.
Helm is actively harmful. And people love it, because it saves them a tiny bit of time one day at the cost of a lot of time in the future.
Helm, like any abstraction, has costs and values. For example, it's very valuable to be able to encode something like "my app consists of a frontend and a backend that run in the same Pod and must each use the same version of the code":
apiVersion: v1
type: Pod
metadata:
name: foo
spec:
containers:
- name: frontend
image: my.registry.io/foo-frontend:{{ .tag }}
- name: backend
image: my.registry.io/foo-backend:{{ .tag }}
The problem with this abstraction is that there's no escape hatch. If you wanted to run different versions of foo-frontend and foo-backend, there is no way to say "but no, really, this time I'm violating the rules for a good reason". You've reduced the features available... the only way forward is to start over with nothing.
The result of this is that every individual Helm chart has to account for every possibility that the manifests could possibly encode, and invent their own programming language that is identical in expressiveness to the underlying manifests. And they do! Differently every time! For example, if I wanted to allow people to override the container images, I'd have to make my template look like:
containers:
- name: frontend
{{ if .tag }}
image: my.registry.io/foo-frontned:{{ .tag }}
{{ else }}
image: {{ .frontend_image }}
{{ end }}
- ...
I think we can all agree that didn't save anyone any time or effort.
This example conveniently flows into my other complaint with Helm. The "yaml files" that declare templates aren't actually valid YAML. You can't use something like `prettier` to autoformat them. You can't use the YAML language server to provide code advice as you type. You can't use `kubeval` to validate them. You are throwing all of that away to Build Your Own Thing. It is actually very insidious and for that reason I consider Helm to be more harmful than helpful. It isn't an abstraction, it's just a macro that might be good for one person the instant they happened to type something in.
The other problem is that Helm charts have no upgrade path. They are only designed for "please explode this project into my cluster, I promise to clean it up In The Future". It never gets cleaned up and brings a little piece of un-updated Windows 95 nostalgia right into your cluster.
Helm is actively harmful. And people love it, because it saves them a tiny bit of time one day at the cost of a lot of time in the future.
Templating within yaml is an absolute mess, I harped on ansible in the past for the same reason. However, dealing with that once is far, far better than dealing with thousands of hand-maintained kube yamls multiplied by your number of environments. Helm of course has its own pain points, but that doesn't subtract from the fact that it solves problems.
I see putting your example of two, unrelated, containers in the same pod as the same binary problem I mentioned earlier. I get your point, but it's a scenario one must wedge themselves into by making other poor choices. Why must the frontend and backend be the exact same version? The most obvious possible reason could be that the API used between them isn't versioned. Or maybe there's not even an API!
> You will have to read the code or hope I wrote documentation for my ad-hoc Kubernetes extension.
No, helm's approach to inserting variables into templates means this isn't the case. Every option and default appears in values.yaml and it's a one stop shop to see everything you can customize. The code example you wrote could be better written as:
I'm not sure what you mean about helm charts having no upgrade path. IME you can un-deploy, upgrade, and rollback helm deployments and it takes care of adding/removing kube resources that where also added or removed in the yaml for you. [1]
[1] https://helm.sh/docs/helm/helm_rollback/
I see putting your example of two, unrelated, containers in the same pod as the same binary problem I mentioned earlier. I get your point, but it's a scenario one must wedge themselves into by making other poor choices. Why must the frontend and backend be the exact same version? The most obvious possible reason could be that the API used between them isn't versioned. Or maybe there's not even an API!
> You will have to read the code or hope I wrote documentation for my ad-hoc Kubernetes extension.
No, helm's approach to inserting variables into templates means this isn't the case. Every option and default appears in values.yaml and it's a one stop shop to see everything you can customize. The code example you wrote could be better written as:
containers:
- name: frontend
image: "{{ .Values.image.repository }}:{{ .Values.image.tag }}"
image:
repository: my_default_image
tag: my_default_tag
I'm not sure what you mean about helm charts having no upgrade path. IME you can un-deploy, upgrade, and rollback helm deployments and it takes care of adding/removing kube resources that where also added or removed in the yaml for you. [1]
[1] https://helm.sh/docs/helm/helm_rollback/
what's wrong with googling or copypasting ? toolset (helm template set) with basic blocks is added rather quickly, after that it could be as simple as running helm oneliner with needed variable values in the command line
Open Stack by Google.
"Simple" solutions to "complex" problems, by necessity, only work for certain subsets of the problems.
Modern distributed systems have tons of largely essential complexity.
The only way to reduce it, is to reduce the expressive power of the solution, which might be great for the cases where you fit into the mold, but then fails spectacularly (either with excess resource consumption, or by not working at all) when you push against the boundaries.
I always commit an example k8s YAML to any repo with k8s in it that describes how to get 2048 running.
Honestly, I get the pieces and I know where to look to get what so I'm not too bothered. The problem with wrapper tools is that sometimes I can't get at the insides and then I have to learn the wrapper tool. So I'm going to just stick to raw Kube until one of the wrappers wins out.
Honestly, I get the pieces and I know where to look to get what so I'm not too bothered. The problem with wrapper tools is that sometimes I can't get at the insides and then I have to learn the wrapper tool. So I'm going to just stick to raw Kube until one of the wrappers wins out.
I think abstracting YAML generation to a tool is a good idea. Let the tool handle the underlying abstraction of converting the parameters to YAML.
If k8s changes notation having a layer of abstraction (hopefully) allows me to run the same command to generate new YAML (very helpful in automation)
If k8s changes notation having a layer of abstraction (hopefully) allows me to run the same command to generate new YAML (very helpful in automation)
Super cool to see these sorts of tools. They're great for learning the kube API.
For production-y things however, some meta-config language that allows deterministic templateing would be a huge improvement. It allows you to make sweeping/uniform infrastructure changes from a single library or tool change.
Kubecfg is a good example of the basics one could implement [0] although it's examples aren't as fully fledged and organized as they could be.
[0] - https://github.com/bitnami/kubecfg/blob/master/examples/gues...
For production-y things however, some meta-config language that allows deterministic templateing would be a huge improvement. It allows you to make sweeping/uniform infrastructure changes from a single library or tool change.
Kubecfg is a good example of the basics one could implement [0] although it's examples aren't as fully fledged and organized as they could be.
[0] - https://github.com/bitnami/kubecfg/blob/master/examples/gues...
https://cuelang.org/ is another option.
I can second using jsonnet and kubecfg for this. Not only you start out with some very useful abstraction from kube.libsonnet, but are also able to build your own abstractions easily, ones that are specific to you organization/codebase/release cycle. All from a declarative, lazily evaluated, pure language.
heard good things about https://dhall-lang.org/#
I've used Dhall in production, pushed it fairly hard and can say with utmost certainty it's be an absolute pleasure. We use it as our application configuration format and derive fairly intricate Kubernetes resources from our app config.
Although it is still in it's early days, it still is excellent to use and will only get better with additional tooling.
Although it is still in it's early days, it still is excellent to use and will only get better with additional tooling.
How does it differ from the CDK/CDK8s/terraform-cdk?
CDK uses existing fully featured programming languages that can preform side effecting actions thus lacking reproducibility and that are may not terminate. Dhall is a total language, meaning it will always terminate and a pure language, meaning that a function given an input will always yield the same output. That makes the output extremely predictable.
We're using Pulumi https://www.pulumi.com/ to do our K8 configuration.
We can use TypeScript interfaces (which give us nice ide code completion) to define our yaml.
we can then create functions where we would normally duplicate Yaml. Really nice. https://www.pulumi.com/kubernetes/
We can use TypeScript interfaces (which give us nice ide code completion) to define our yaml.
we can then create functions where we would normally duplicate Yaml. Really nice. https://www.pulumi.com/kubernetes/
+1 to Pulumi. I can't imagine operating k8s without tools like Pulumi or Terraform now. And Pulumi addresses some of the most important shortcomings of Terraform so I really hope they gain more traction.
> we can then create functions where we would normally duplicate Yaml.
This is the way to go for sure. I've done similar by generating CloudFormation from Python (I wrote my own library because I felt Troposphere was not very friendly nor a significant improvement over YAML).
Typing turns out to be pretty useful when you're generating YAML. While my library was fully typed, Python's type checking left a lot to be desired--many stupidly common things still can't be expressed (JSON, kwarg callbacks, etc), getting mypy to find/accept type annotations for libraries is stupidly hard, and the IDE integrations are pretty awful at this point. TypeScript users would enjoy a real leg-up here since its type system isn't half baked.
This is the way to go for sure. I've done similar by generating CloudFormation from Python (I wrote my own library because I felt Troposphere was not very friendly nor a significant improvement over YAML).
Typing turns out to be pretty useful when you're generating YAML. While my library was fully typed, Python's type checking left a lot to be desired--many stupidly common things still can't be expressed (JSON, kwarg callbacks, etc), getting mypy to find/accept type annotations for libraries is stupidly hard, and the IDE integrations are pretty awful at this point. TypeScript users would enjoy a real leg-up here since its type system isn't half baked.
> This is the way to go for sure. I've done similar by generating CloudFormation from Python (I wrote my own library because I felt Troposphere was not very friendly nor a significant improvement over YAML).
Yes and no.
Typing is a must, but a full-blown programming language is too powerful and all abstraction layers start to leak sooner rather than later. I always ended up with a "deployment" function that exposed almost all underlying functionality.
We're big fans of the Cue (https://cuelang.org) approach instead: https://cuelang.org/docs/about
Yes and no.
Typing is a must, but a full-blown programming language is too powerful and all abstraction layers start to leak sooner rather than later. I always ended up with a "deployment" function that exposed almost all underlying functionality.
We're big fans of the Cue (https://cuelang.org) approach instead: https://cuelang.org/docs/about
It depends. If you can reasonably trust your team to not to do I/O in the scripts, then you're fine. If you can't, then you should use something like Starlark (a subset of Python running in a constrained environment that precludes I/O, unbounded loops, etc); however, Starlark doesn't support types yet.
I've looked at Cue a few times (principally out of frustration with the lack of types in Starlark), but I don't really "get it". What I want is a typed embeddable scripting language--specifically the ability to expressively generate YAML, and I'm pretty sure that's not what Cue is. I'm open to the argument that Cue is better than what I want (a la "if Henry Ford asked people what they really want, they'd've said faster horses"), but the value proposition isn't clear to me from perusing the docs. Maybe you can correct me?
I've looked at Cue a few times (principally out of frustration with the lack of types in Starlark), but I don't really "get it". What I want is a typed embeddable scripting language--specifically the ability to expressively generate YAML, and I'm pretty sure that's not what Cue is. I'm open to the argument that Cue is better than what I want (a la "if Henry Ford asked people what they really want, they'd've said faster horses"), but the value proposition isn't clear to me from perusing the docs. Maybe you can correct me?
I have used a bunch of jsonnet and was in your position. I picked up https://jkcfg.github.io and have been very happy. You get TypeScript with (some) hermeticity guarantees.
Are you not worried about people writing arbitrary code to do stuff? I've been burnt before where devs used Turing complete languages (python in my case) to generate configs in probably the most convoluted and complicated manner possible. It was impossible to debug and understand, there were side-effects literally everywhere. It was everything you'd imagine from a normal bit of bad code, but it also happened to spin up hardware.
As it's infrastructure as code it would go through code review. So in that case I wouldn't be worried.
As long as the code is generating something like configs, you can write guard rail sanity check tests against the output, or apply linters, etc.
But now you’re writing code to generate config and code to verify your configs.
Pulumi looks good, but it confuses me. I thought it was like Ansible/Terraform, but I see a "pricing" page and it looks like there's a SaaS that goes with it, can someone shed some light?
We built this internally to test that the UI we built for Kubernetes deployments was producing the expected YAML. Thought it would be useful to share.
If you print out the API docs for all the resources you might deal with for running apps at A4 (8.5 x 11) it comes in at over 1,000 pages. There is just so much and few really know it.
A UI like this is useful to so many. It makes the experience of creating these YAML files easier. Thanks for sharing it.
A UI like this is useful to so many. It makes the experience of creating these YAML files easier. Thanks for sharing it.
Thanks for sharing this useful resource!
Paul, as an aside, we absolutely love how feature packed Octopus is nowadays. We have been using it since 2.0, and I don't think we will ever give it up. Thanks for making one of the best tools we use a daily basis!
Paul, as an aside, we absolutely love how feature packed Octopus is nowadays. We have been using it since 2.0, and I don't think we will ever give it up. Thanks for making one of the best tools we use a daily basis!
Thanks for the kind words!
Any plans on open sourcing this or making the code available?
I don’t know how to feel about Kubernetes configuration apparently being so complicated that you need a generator for it, instead of just having the docs and your IDE open in split screen like with Docker Compose.
That said, this still looks cool. I just hope we won’t need a Kubernetes configuration generator generator anytime soon.
That said, this still looks cool. I just hope we won’t need a Kubernetes configuration generator generator anytime soon.
Its not complicated, which makes it easy to generate. No reason not to make things easier to work with if you can.
Kubernetes manifests are no different from any code one would write. The first few times you need to accomplish something, you consult the docs. After a while, you know what you need. If you find yourself doing the same thing over and over again, automate it. I don't understand the hand-wringing over k8s YAML.
It's actually conceptually simpler than docker compose given what it does.
The bigger issue is that there's a lot of things you can do with it, and editing text YAML/JSON serialization of the objects is probably one of the least efficient ways of dealing with it, but it's also the "default" way people know.
It's much easier when your editor actually understands the object you're editing instead of at best helping you write correct YAML.
The bigger issue is that there's a lot of things you can do with it, and editing text YAML/JSON serialization of the objects is probably one of the least efficient ways of dealing with it, but it's also the "default" way people know.
It's much easier when your editor actually understands the object you're editing instead of at best helping you write correct YAML.
[deleted]
Would be nice to have a linter or something that you know is best practice. Eg if you check your yaml and its more complex than it could be.
There are a number of such tools out there. Here's a short list. I'd be interested in any experiences people have had with them in largish production environments.
https://www.kubeval.com/
https://github.com/zegl/kube-score
https://stelligent.github.io/config-lint/#/
https://github.com/cloud66-oss/copper
https://www.conftest.dev/
https://github.com/FairwindsOps/polaris#cli
https://www.kubeval.com/
https://github.com/zegl/kube-score
https://stelligent.github.io/config-lint/#/
https://github.com/cloud66-oss/copper
https://www.conftest.dev/
https://github.com/FairwindsOps/polaris#cli
Hey, I'm the author of kube-score, and originally built the tool to support an organization using Kubernetes at a fairly large scale as measured in number of engineers, services, and Kubernetes clusters.
I'm obviously biased, but it's been hugely successful! kube-score is working very well out of the box, and there's only a handful of cases where the "ignore" annotation has been used to disable a check that's too strict for the particular use case.
Feel free to reach out if you have any questions or comments.
I'm obviously biased, but it's been hugely successful! kube-score is working very well out of the box, and there's only a handful of cases where the "ignore" annotation has been used to disable a check that's too strict for the particular use case.
Feel free to reach out if you have any questions or comments.
That looks super handy! It seems like it'd be possible to "port" to VSCode. Some other comments mention autocomplete in VSCode, but it'd be nifty to run something like this directly in the VSCode UI.
We built a similar thing and used in our graphical k8s designer : https://k8od.io/.
Ooooh nice! When I saw the title I was hoping this is what it would be. I'd dreamed of building something like this, but never had the time or the buy in.
Aside from making it easy to generate k8s manifests, this could also be a great learning tool. If you allowed this to generate multiple resources that are linked, it could be a great illustration of how different resources fit together.
Aside from making it easy to generate k8s manifests, this could also be a great learning tool. If you allowed this to generate multiple resources that are linked, it could be a great illustration of how different resources fit together.
This is awesome. It is so easy to make mistakes when configuring your k8s services
Awesome work for the deployment resource! Now do this for all the other native k8s types and popular custom resources. :)
Awesome! Was waiting for someone to build it :-)
This is great, thank you!
Looks very useful. In my opinion the configuration format should have been built like this from the get go.
Clear schema (like TypeScript interfaces or something similar) which allows generating an UI.
Clear schema (like TypeScript interfaces or something similar) which allows generating an UI.
All of the standard Kubernetes resource definitions have a schema [1] [2] which is already used to, for instance, generate Go code [3] and documentation [4].
[1] - https://kubernetes.io/docs/concepts/overview/kubernetes-api/...
[2] - https://github.com/kubernetes/kubernetes/blob/release-1.19/a...
[3] - https://godoc.org/k8s.io/client-go/kubernetes/typed/core/v1
[4] - https://kubernetes.io/docs/reference/generated/kubernetes-ap...
[1] - https://kubernetes.io/docs/concepts/overview/kubernetes-api/...
[2] - https://github.com/kubernetes/kubernetes/blob/release-1.19/a...
[3] - https://godoc.org/k8s.io/client-go/kubernetes/typed/core/v1
[4] - https://kubernetes.io/docs/reference/generated/kubernetes-ap...
Of course there is schema at some level, but it feels like it's not utilized. This UI seems like very good use of schema. Another would be if I open up an VSCode, there should be a way to autocomplete / or give error squiggles if I write something wrong, in realtime.
Right, but that's not as much of a Kubernetes issue as a VSCode issue. Everything's there for you to write something like this for your text editor, open source code doesn't grow on trees :).
My point being if k8s configuration was developed UI centrically, this problem would not exist in the first place. Because there would be official K8s GUI for configuration, one would not hunt and beg bits and pieces of docs every time editing a config file.
As a side note, usually the schema languages fail at some point, thus I referred to TypeScript interfaces, which are very flexible way to write validation.
There is already a schema support for JSON validation in VSCode, one can use it using `{ "$schema" : "https://example.com/url/toschema" }` but it uses JSON Schema format, which I think is not accurate enough in edge cases for UI generation.
As a side note, usually the schema languages fail at some point, thus I referred to TypeScript interfaces, which are very flexible way to write validation.
There is already a schema support for JSON validation in VSCode, one can use it using `{ "$schema" : "https://example.com/url/toschema" }` but it uses JSON Schema format, which I think is not accurate enough in edge cases for UI generation.
If it was developed UI-centric, it would have failed much earlier to do anything useful.
The real power of Kubernetes is that it has very simple basic model that allows building complex constructs out of simple pieces, and in fact basic help (which is even used for UI in kubectl explain.
All the bits for good UI support are there.
The real power of Kubernetes is that it has very simple basic model that allows building complex constructs out of simple pieces, and in fact basic help (which is even used for UI in kubectl explain.
All the bits for good UI support are there.
VSCode actually has such a plugin, though it has the hard to discover name “Kubernetes” and is maintained by the little-known “Microsoft”
Do we need a better layer of abstraction, i.e. better adoption and tighter integration for something like kustomize? Have we fucked up completely with Kubernetes due to it being outrageously complicated for simple tasks? How we redesign this to be simpler? Is the complexity even a problem for the target audience?
I've no idea. I just know I'm a kubernetes admin and I can't write a deployment yaml without googling or copy/pasting.