Show HN: GitHub as Code – Enable self-service by managing GitHub with Terraform(mineiros.io)
mineiros.io
Show HN: GitHub as Code – Enable self-service by managing GitHub with Terraform
https://mineiros.io/github-as-code
21 comments
I've used the GitHub Terraform provider for similar purposes. Building out something more comprehensive on top of it seems like a cool idea.
Will companies really pay a subscription for this though?
Will companies really pay a subscription for this though?
I've been doing this in the past 3 years, but the GitHub provider is one of the worst in the Terraform ecosystem, unfortunately.
Could you share some of the issues you're experiencing with the provider? We've also hit some problems with the provider but worked around most of the issues in our open-source modules (eg, https://github.com/mineiros-io/terraform-github-repository) and by using Terramate (eg, renaming repositories causes re-creation in the provider, with Terramate we're dynamically creating move statements to prevent this. Check out Terramate if you haven't done it yet: https://github.com/mineiros-io/terramate)
Well, there are a lot of open issues [0] with no outlook to get them ever closed.
We've had issues with branch protection rules (the same code works with old actors in the state, but new actors erroring out without any reason), organization secrets, etc.
[0]: https://github.com/integrations/terraform-provider-github/is...
We've had issues with branch protection rules (the same code works with old actors in the state, but new actors erroring out without any reason), organization secrets, etc.
[0]: https://github.com/integrations/terraform-provider-github/is...
Regarding Terramate - I've reviewed it, it definitely is a better reincarnation of Terragrunt not using templating but HCL, yet, it does not solve all of our issues, so, we chose CDKTF for the glue code.
The fundamental difference is that Terragrunt is a wrapper for Terraform, compared to Terramate, which generates native Terraform code that runs in all kinds of Terraform tooling such as CIs, etc. Also, Terramate adds new concepts such as eg change detection, which is helpful when building pipelines. Terramate may also be used together with Terragrunt.
For details please see https://blog.mineiros.io/introducing-terramate-an-orchestrat... and https://blog.mineiros.io/terramate-and-terragrunt-f27f2ec403...
For details please see https://blog.mineiros.io/introducing-terramate-an-orchestrat... and https://blog.mineiros.io/terramate-and-terragrunt-f27f2ec403...
I didn't mean it behaves the same, but it accomplishes a similar goal. Definitely Terramate is the better tool and integrates with Terraform Cloud unlike Terragrunt.
Thank you!
Thanks for sharing those insights. I'd love to hear what kind of issues you managed to resolve with CDKTF compared to using HCL other than being more developer centric?
I personally like CDKTF, but for me, it's still in a too early state of development to build a stable product on top.
I personally like CDKTF, but for me, it's still in a too early state of development to build a stable product on top.
One example is the handling of dynamic providers. Imagine, instead of having the inventory of environments hardcoded in directory structure, you have them in a configuration setting. Or, another example, having a dynamic list of Kubernetes clusters, which you need to install Helm charts into, for which you need dynamic aliased Kubernetes providers.
Interesting - dynamic providers and configurations are exactly what we're doing with Terramate also. Seems like we're having a very similar approach while using different tooling. Thanks for sharing the insights with me!
I know you can do it - but not as dynamically as I want, too. Maybe I'm not getting something, but I don't see the `for_each` or `count` equivalent in `generate_hcl` [0].
Also, stacks are predefined, not dynamic. With CDKTF, I can generate any number of stacks with one piece of code. In addition, I can define stack dependencies - let's say, my dev database stack depends on dev IAM stack - something that Terraform does not do and developers need to remember the order of `terraform apply` across stacks.
[0]: https://github.com/mineiros-io/terramate/blob/main/docs/code...
Also, stacks are predefined, not dynamic. With CDKTF, I can generate any number of stacks with one piece of code. In addition, I can define stack dependencies - let's say, my dev database stack depends on dev IAM stack - something that Terraform does not do and developers need to remember the order of `terraform apply` across stacks.
[0]: https://github.com/mineiros-io/terramate/blob/main/docs/code...
We do not try to replace existing solutions or technologies. They all have their well known pros and cons.
While working with customers we identified that even Terraforms HCL configuration requests too much knowledge from users. So we tried to come up with a tool where a team can take ownership of the Terraform part while the actually users can just write configurations (also in HCL). You can for sure achieve the same with CDKTF, there is no doubt.
Terramate gives us the same orchestration features that you mentioned as you can define an order of execution between stacks. You can even enforce stacks to always run if another stack is triggered. Change Detection based on our Git Integration allows us to just run stacks that actually have changed and thus reducing wait times for very big set ups to a minimum, which makes our solution very scalable.
We also currently work on easier stack management features to be able to create stacks with a single call (already available) or clone stacks for either promoting them through the environment e.g. from staging to production or to just duplicate resources easily. Those features will be used to create one-click solutions to request a new respository with a default config (as defined by the company) in less than 30 seconds + time to review (if desired) + very fast deployment.
The goal here was to create a great user experience (UX) for teams that normally do not use Terraform or programming languages as GitHub is also used by other departments than engineering like marketing, hr or sales - but also allowing the team managing this solution to easily extend it to own needs.
Our solution is currently based on Terraform but might even evolve to something else as we also see the bottleneck in the provider where both Terraform or CDKTF will face the same issues in the end.
While working with customers we identified that even Terraforms HCL configuration requests too much knowledge from users. So we tried to come up with a tool where a team can take ownership of the Terraform part while the actually users can just write configurations (also in HCL). You can for sure achieve the same with CDKTF, there is no doubt.
Terramate gives us the same orchestration features that you mentioned as you can define an order of execution between stacks. You can even enforce stacks to always run if another stack is triggered. Change Detection based on our Git Integration allows us to just run stacks that actually have changed and thus reducing wait times for very big set ups to a minimum, which makes our solution very scalable.
We also currently work on easier stack management features to be able to create stacks with a single call (already available) or clone stacks for either promoting them through the environment e.g. from staging to production or to just duplicate resources easily. Those features will be used to create one-click solutions to request a new respository with a default config (as defined by the company) in less than 30 seconds + time to review (if desired) + very fast deployment.
The goal here was to create a great user experience (UX) for teams that normally do not use Terraform or programming languages as GitHub is also used by other departments than engineering like marketing, hr or sales - but also allowing the team managing this solution to easily extend it to own needs.
Our solution is currently based on Terraform but might even evolve to something else as we also see the bottleneck in the provider where both Terraform or CDKTF will face the same issues in the end.
On the dynamic side of things we are working right now on adding support for dynamic block generation on generate_hcl, which should help.
Regarding dependencies, did you took a look on the orchestration features on Terramate ?
https://github.com/mineiros-io/terramate/blob/main/docs/orch...
They should be enough to let you easily define ordering and not depend on developers doing the ordering manually.
Regarding dependencies, did you took a look on the orchestration features on Terramate ?
https://github.com/mineiros-io/terramate/blob/main/docs/orch...
They should be enough to let you easily define ordering and not depend on developers doing the ordering manually.
Great news!
Looks interesting but a personal pet peeve of mine is not showing any pricing on site.
Hi!
Thanks for your feedback. As quoted in the pricing section at the bottom of the page, we're calculating the pricing based on the size of your organization. At a later stage, we will definitely make the pricing more transparent.
Thanks for your feedback. As quoted in the pricing section at the bottom of the page, we're calculating the pricing based on the size of your organization. At a later stage, we will definitely make the pricing more transparent.
Why not pick a certain size and put it as an example?
We just made the pricing for the StartUp and SMB tiers more transparent now. On the yearly subscription: We've seen many companies trying to implement something similar with Terraform, which quite often takes them a lot of time for the initial implementation and ongoing maintenance (eg provider updates, new features, bug fixes). Instead of doing it yourself, we're going to maintain it for you.
Let me elaborate with the team on that and come back to you with details tomorrow at latest
I am Soren, one of the founders of mineiros.io - the company that invented GitHub as Code.
GitHub as Code is a comprehensive Terraform blueprint with fully automated and pre-configured GitHub Actions pipelines to enable everyone in your organization to manage on- and off-boarding of users, repositories, branch protections, secrets, and more through code.
We invented GitHub as Code because we've seen organizations struggling when handling GitHub on scale.
I'd love to hear some feedback on the idea and whether or not you think GitHub as Code could be helpful for you also.
Also, please consider our product introduction blog post on medium for details https://blog.mineiros.io/product-introduction-github-as-code...