Ask HN: How can we make the current CI/CD ecosystem better?
22 comments
You can try https://ci.sparrowhub.io as well, it’s yaml based but more flexible then standard yaml based CI systems, reporters https://github.com/melezhik/SparrowCI/blob/main/reporters.md allow you to integrate with any others third party systems ( for example with bugzilla , GitHub/GitLab , jira or IRC )
Absolutely, I feel like most CI systems are much harder to configure than necessary.
If I were to build my own CI system, then it would have these features.
- Simple config format, ideally flat and not nested, so maybe toml.
- Ability to run any step, task, job, pipeline locally. It should be possible to do something like `ci run -t build` and have it execute the build task locally. This would greatly help with building CI pipelines and make it possible to just use the CI tool as the local build tool as well instead of having to essentially setup the same thing twice.
- Easy to self host, so not a million micro-services, but just one (or max 3) executable.
- Modular, it should be possible to extend it with plugins using webassembly or containers. Look at how Concourse CI does this for ideas.
Edit: I would pay for a license to something like that, maybe not a lot but I could justify up to $200 as a once off for a self-hosted solution like that if it's well made.
If I were to build my own CI system, then it would have these features.
- Simple config format, ideally flat and not nested, so maybe toml.
- Ability to run any step, task, job, pipeline locally. It should be possible to do something like `ci run -t build` and have it execute the build task locally. This would greatly help with building CI pipelines and make it possible to just use the CI tool as the local build tool as well instead of having to essentially setup the same thing twice.
- Easy to self host, so not a million micro-services, but just one (or max 3) executable.
- Modular, it should be possible to extend it with plugins using webassembly or containers. Look at how Concourse CI does this for ideas.
Edit: I would pay for a license to something like that, maybe not a lot but I could justify up to $200 as a once off for a self-hosted solution like that if it's well made.
This doesn't have everything you mention, but I recently learned about https://dagger.io/ which allows you to write your pipelines as code (Go, Python, etc) and then run them anywhere in containers. I always hated that in order to test my pipelines, I actually have to trigger it on some special machine (the CI server).
Dagger looks interesting, I will check it out.
I think these are all great points! I have came across Concourse CI, but have not looked too deeply into it yet.
I hadn't thought to use TOML for configuring CI pipelines, I'm curious what that might look like. TOML is indeed very flat, so it would be interesting to see what the equivalent TOML pipeline looks like compared to a YAML one.
I hadn't thought to use TOML for configuring CI pipelines, I'm curious what that might look like. TOML is indeed very flat, so it would be interesting to see what the equivalent TOML pipeline looks like compared to a YAML one.
I run Concourse CI for my personal stuff with the code in Fossil. It works decently enough, I like that it's somewhat extensible, but I have hit use cases that it can't really do like building an Alpine Linux cloud image.
As for YAML vs TOML, I am mostly sick of YAML, and I really like using TOML to configure Caddy, so I thought it might be good enough for CI config. I feel like it would be worth exploring, but maybe something slightly more complex or custom would be required.
As for YAML vs TOML, I am mostly sick of YAML, and I really like using TOML to configure Caddy, so I thought it might be good enough for CI config. I feel like it would be worth exploring, but maybe something slightly more complex or custom would be required.
Sounds a bit like TeamCity, except for the local execution. It has:
- Either GUI based configuration (serialized to XML), or Kotlin. So you can express configuration with a real programming language tracked with version control, or just use the GUI to define it.
- Can self host. There's one program to install for the master, and one for the agents.
- Can be extended with plugins. Not webassembly based, but those are theoretical anyway currently due to lack of better-than-C interfaces. You can write JVM plugins instead and get full GCd OOP.
The main thing it's lacking is the ability to run things locally. But then again, usually a TC step is just invoking a build system or script anyway so it's not that big of a deal.
- Either GUI based configuration (serialized to XML), or Kotlin. So you can express configuration with a real programming language tracked with version control, or just use the GUI to define it.
- Can self host. There's one program to install for the master, and one for the agents.
- Can be extended with plugins. Not webassembly based, but those are theoretical anyway currently due to lack of better-than-C interfaces. You can write JVM plugins instead and get full GCd OOP.
The main thing it's lacking is the ability to run things locally. But then again, usually a TC step is just invoking a build system or script anyway so it's not that big of a deal.
In addition, it would need to try and solve for the 80% witch hatches for those that need the last 20%.
Part of the problem with current CI/CD solutions is trying to solve for everything.
And I think the local thing would be a killer if for no other reason than the ability to debug it locally.
Part of the problem with current CI/CD solutions is trying to solve for everything.
And I think the local thing would be a killer if for no other reason than the ability to debug it locally.
That's kinda my idea with plugins, it should be possible to develop a plugin doing your custom thing if you need something in that last 20%.
Concourse CI has a design documented for a better resource/plugin system than what they currently have implemented that's worth checking out. But as far as I am aware they are really far from actually implementing it.
Concourse CI has a design documented for a better resource/plugin system than what they currently have implemented that's worth checking out. But as far as I am aware they are really far from actually implementing it.
I've been thinking lately that how I could do some integration testing FOR my CI-pipelines. My client has very complex pipelines which has business logic inside them and every week we run into new bugs and regressions. I can't think any easy way to write tests for them to make it harder to break when developing new features.
Btw. Gitlab allows splitting your pipelines to multiple files using include[1]. Also we have used gitlab-ci-local[2] to run our pipelines locally when developing them.
1: https://docs.gitlab.com/ee/ci/yaml/includes.html 2: https://github.com/firecow/gitlab-ci-local
Btw. Gitlab allows splitting your pipelines to multiple files using include[1]. Also we have used gitlab-ci-local[2] to run our pipelines locally when developing them.
1: https://docs.gitlab.com/ee/ci/yaml/includes.html 2: https://github.com/firecow/gitlab-ci-local
I havn't heard of any products or services that test the actual CI pipelines themselves, which seems like an interesting target market. I can't say how many times I have wanted to test out some CI workflow on some throw-away clone repo without mucking up the live repo.
It's good to know that you can use includes with Gitlab, And, from my quick glance at that second link, the setup process seems a bit involved. I don't like having to set up new user accounts just so that I can run a CI workflow locally.
It's good to know that you can use includes with Gitlab, And, from my quick glance at that second link, the setup process seems a bit involved. I don't like having to set up new user accounts just so that I can run a CI workflow locally.
I've been able to use the tool without setting up any accounts or other settings. Just run `gitlab-ci-local` in a folder where your `.gitlab-ci.yaml` lies.
The real meat of what CI/CD does is closely tied to the platform, I don't think there's a great way to improve that. Running on different configurations, uploading artifacts, deployment, etc.
Generally, I do as much as possible in the build system proper. The CI should mostly be running simple commands. Avoid GitHub Actions which can be trivially replaced with one or two shell commands (which is a lot of them).
Generally, I do as much as possible in the build system proper. The CI should mostly be running simple commands. Avoid GitHub Actions which can be trivially replaced with one or two shell commands (which is a lot of them).
It should be more achievable than it has ever been before with container tech being in a great place, qemu being a thing and the webassembly ecosystem getting better every day.
Make a bash file. Have the platform specific yaml call your bash file. call it a day
I’m not sure the niggles here are platform things - they’re hard lessons that come from experience in using CI for the most part.
Generally you want to aim for writing things in scripts that your CI system calls, so that they can be run locally trivially and so that migrating away from one CI provider (for whatever reason - cost, employer decides to shift, etc.) is not a huge piece of work.
Generally you want to aim for writing things in scripts that your CI system calls, so that they can be run locally trivially and so that migrating away from one CI provider (for whatever reason - cost, employer decides to shift, etc.) is not a huge piece of work.
So if we have to use a script anyway, then why do we have to write like 20+ lines of YAML to run the script?
Edit: I mean, why is this not the config if we are just doing a script?
Edit: I mean, why is this not the config if we are just doing a script?
src: github.com/...
run: test_and_build.shIt's not that bad:
jobs:
unit-tests:
runs-on: python:3.8.15-slim-buster
- name: "Run tests"
working-directory: ./bin
run: unit-tests.shI'm pretty happy with GitHub actions
* There is little to no inter-op between CI providers. Sure there are 3rd party providers, but they only seem to focus on the basics, like running a job when a commit is pushed. But, if you choose something like GitHub Actions, you are now locked into their platform, making switching Git hosting providers very painful.
* YAML. Everything is YAML. Sometimes it is just one big file (Gitlab, Travis, CircleCI, etc), and it becomes very hard to maintain very quickly. Workflows for GitHub Actions can be split up into different files, but at the end of the day, you are still writing YAML, which means no loops, conditionals, functions, and so forth. In addition, you must follow a strict file structure, meaning very little room for customization. In many cases, you need to write a Makefile/Python/shell script for more complex situations.
Does anyone else share these annoyances? Are there any similar sorts of issues that you deal with on a day to day basis that you wish could be improved?