We use Tekton to manage our CI pipeline and I agree that the way it enforces very little structure is a strength. On the other hand it's new enough that if you need to stretch its capabilities you are going to have to get creative. The primitives it has are nice, but they have their limits.
For instance, if running a bunch of parallel tasks, collating results on a PV is out the window unless your cluster supports multiple writer volume types, which GKE does not. You have to bring in NFS volume types or something like that for it. In the early days of tekton they had a results primitive which synchronized an output dir to GCS, but they decommissioned that. So you are left pushing that logic into your task command. Running gsutil is easy enough, but it means you are pushing logic into your scripts and not declaring steps in the pipeline definition. You could make that command a step but I see little benefit in that.
Additionally there is no way to loop in the configuration to generate tasks, much less loop with an ordinal value. We end up just programmatically generating the resource definitions with ruby erb templates. All of our pipeline specs (including task runs, etc) creates a 2MB yaml file. We push dozens and dozens of these into k8s daily. It works but at the same time our usage of Tekton is more or less as a glorified alternative to batch jobs which works because batch jobs _still_ don't have a proper sidecar capability and also because we rely on the DAG to order dependent taskruns.
If your pipeline is simple, look at Tekton. But if your pipeline is complex... still look at Tekton but expect to do some work. Once you get a good workflow though you can you can scale your pipelines as easily as you can a deployment in k8s. We use node autoscaling and preemptibles (Tekton can retry if a task disappears due to node reclamation) to manage our CI costs quite effectively.
This is funny, the first couple of times I heard it I also wondered if I was going a bit mad. When you are out all alone and you hear a sound that you can't explain I think it's pretty natural to think that.
I have no opinion on anything, FWIW because I actually spent most of my time in Woods Hole. My recollection of all this is from the local paper.
EDIT:
Cliche as it as, the fish bites at Landfall are pretty good! And it was always fun to swing by the aquarium after hours and listen to the seals bumping around in their pool. It's a beautiful place to visit if you ever get a chance.
I wonder what it's like to try to pull a signal up there.
Your comment brings up memories of Lighthouse Beach in Chatham, which has a ton of warnings about Great White Sharks due to the prevalence of seals in the waters. Beautiful place. If the sharks don't get you the radio frequencies will I guess.
Yeah, I thought it was a generator or mower at first. It's just quiet enough that I mistook it for a much louder sound that had grown faint due to my distance from the source.
1. A few years ago I spent some time in Falmouth, MA and there was an ongoing battle over newly installed wind turbines. The humming sound was getting to people, including to the point where they were getting headaches. Some people heard it and others didn't. The locals were going to war with the initiative. This article reminded me of that and I checked in on the ordeal. Looks like they shuttered the project and have started dismantling the turbines. It seems like a total failure: https://www.wsj.com/articles/a-green-new-deal-in-profile-115...
2. I was on a solo canoe trip in a remote part of Maine a couple of weekends ago and I kept hearing the sound of a motor start up and then die down. I heard it multiple times along a 25 or so mile stretch of water with the only other sounds being birds, moose rumbling in the woods, and beavers slapping the water with their tails. I chalked the sound up to some wind turbines I had seen on my drive up there and felt justifiably annoyed at the encroachment of the industrial world into my backwoods trip. I did some research when I got home and it turned out that I was hearing ruffed grouse "drumming" to mark their territory. In retrospect, it's an amazing sound: https://youtu.be/q0obByQW23k?t=21
For certain things like layer 4 and layer 7 routing or firewall policies, health checking and failover, network-attached volumes, etc you have to choose software and configure it on top of getting that configuration in that tooling. So you are doing kernel or iptables or nginx or monit/supervisord configurations and so on.
But basic versions of these things are provided by Kubernetes natively and can be declared in a way that is divorced from configuring the underlying software. So you just learn how to configure these broader concepts as services or ingresses or network policies, etc, and don't worry about the underlying implementations. It's pretty nice actually.
Does anyone know how to initiate a web compare between to branches using this thing? If I could have that + Alfred (or whatever launch bar / cli one uses) that would be amazing.
Great point. I don't speak for the guys that made the decision to append the timestamp to the query, but I assume our concern is in intermediate network caches that don't honor low TTLs. Though I don't know how founded that is, we won't ever have to deal with the issue if we take control of it with the url string.
It'd be interesting to see how wide the key space is due to clock skew. I suppose we could specify some number and consider it a global counter that is incremented every second, then when someone comes in for the first time they can by synced in with the global incrementing counter. That counter is used to ensure a fresh cloudfront hit.
I think at end of the day, these issues haven't been a huge concern for a one month emergency project, but they are good points.
For instance, if running a bunch of parallel tasks, collating results on a PV is out the window unless your cluster supports multiple writer volume types, which GKE does not. You have to bring in NFS volume types or something like that for it. In the early days of tekton they had a results primitive which synchronized an output dir to GCS, but they decommissioned that. So you are left pushing that logic into your task command. Running gsutil is easy enough, but it means you are pushing logic into your scripts and not declaring steps in the pipeline definition. You could make that command a step but I see little benefit in that.
Additionally there is no way to loop in the configuration to generate tasks, much less loop with an ordinal value. We end up just programmatically generating the resource definitions with ruby erb templates. All of our pipeline specs (including task runs, etc) creates a 2MB yaml file. We push dozens and dozens of these into k8s daily. It works but at the same time our usage of Tekton is more or less as a glorified alternative to batch jobs which works because batch jobs _still_ don't have a proper sidecar capability and also because we rely on the DAG to order dependent taskruns.
If your pipeline is simple, look at Tekton. But if your pipeline is complex... still look at Tekton but expect to do some work. Once you get a good workflow though you can you can scale your pipelines as easily as you can a deployment in k8s. We use node autoscaling and preemptibles (Tekton can retry if a task disappears due to node reclamation) to manage our CI costs quite effectively.