Patterns for Managing Source Code Branches(martinfowler.com)
martinfowler.com
Patterns for Managing Source Code Branches
https://martinfowler.com/articles/branching-patterns.html
11 comments
For historical context here is the classical paper on source control branching patterns I would normally recommend:
"Streamed Lines: Branching Patterns for Parallel Software Development" by Brad Appleton, Stephen P. Berczuk, Ralph Cabrera, Robert Orenstein. [1]
It was presented at Pattern Languages of Programs '98 (PLoP '98).
Incidentally, looking at the proceedings [2], Martin Fowler as co-author on another paper at the same conference called "Temporal Patterns".
[1] https://hillside.net/plop/plop98/final_submissions/P37.pdf
[2] https://hillside.net/plop/plop98/final_submissions/
"Streamed Lines: Branching Patterns for Parallel Software Development" by Brad Appleton, Stephen P. Berczuk, Ralph Cabrera, Robert Orenstein. [1]
It was presented at Pattern Languages of Programs '98 (PLoP '98).
Incidentally, looking at the proceedings [2], Martin Fowler as co-author on another paper at the same conference called "Temporal Patterns".
[1] https://hillside.net/plop/plop98/final_submissions/P37.pdf
[2] https://hillside.net/plop/plop98/final_submissions/
Came here just to leave the following link : https://www.goodreads.com/book/show/367720.Software_Configur... which is the Berczuk book about such patterns. I read it years ago, it's good.
This is fantastic, having looked through Martin's blog before and found some gems that really leveled up my process on managing code.
As an aside - beyond the coming installments of this series, can anyone recommend sources for setting up CI locally, or for small teams? DDG is giving me pages on GitHub Actions, Atlassian, and DigitalOcean blogs, but these are all process or product specific.
As an aside - beyond the coming installments of this series, can anyone recommend sources for setting up CI locally, or for small teams? DDG is giving me pages on GitHub Actions, Atlassian, and DigitalOcean blogs, but these are all process or product specific.
Mostly the driver doesn't matter; for the CI stuff you basically want to carry out "actions" on "events". As a concrete example you might wish to run your test cases when a new pull-request is opened, or an existing one is updated.
The way that glue/integration works is going to be somewhat specific to the setup you're already using. That's why you'll find Github Actions being documented for Github, Jenkins for self-hosted repositories, etc.
Broadly speaking it doesn't matter which "thing" you use to drive your CI, the key is that it works reliably and that the stuff which is executed is inside your repository. That way:
* Developers can run it manually if they wish (e.g. "make test")
* The CI system can be swapped out in the future, if you need to.
As a concrete example once I migrated a company's repositories from bitbucket, using their pipelines, to (self-hosted) Github Enterprise. Because the pipeline just said "Run .git/tests.sh" porting the glue was trivial.
I even published a github action for those using github, so that you can launch your project-specific tests in a portable and simple fashion:
https://github.com/skx/github-action-tester
The way that glue/integration works is going to be somewhat specific to the setup you're already using. That's why you'll find Github Actions being documented for Github, Jenkins for self-hosted repositories, etc.
Broadly speaking it doesn't matter which "thing" you use to drive your CI, the key is that it works reliably and that the stuff which is executed is inside your repository. That way:
* Developers can run it manually if they wish (e.g. "make test")
* The CI system can be swapped out in the future, if you need to.
As a concrete example once I migrated a company's repositories from bitbucket, using their pipelines, to (self-hosted) Github Enterprise. Because the pipeline just said "Run .git/tests.sh" porting the glue was trivial.
I even published a github action for those using github, so that you can launch your project-specific tests in a portable and simple fashion:
https://github.com/skx/github-action-tester
CI will always be product specific unless you're going to create your own scripts for your pipeline. Even then, your script for executing tests, deploying, etc. will still be dependent on your stack.
Some general guidelines:
When a developer's branch is pushed publicly, compile it and run tests. It must pass before being allowed to merge into develop/master/${specific-branch}. Require that merges to those branches are only fast-forward merges.
The types of tests needed will be dependent on your application. Make sure they're all green before proceeding. If not, log the failing tests. Depending on your application, you may want to have certain testing metrics also fail the build. For example, require that at least 50% of new lines of code have test coverage and 60% of your overall source code have coverage.
Run any static analysis tools and linters. Configure which rules should trigger failures or warnings.
Run any other scripts that aren't related to code quality like minification or building binaries.
Deploy your application to the proper environment.
This is a bare-bones list. Your application may have many more steps, but this should get you started.
Some general guidelines:
When a developer's branch is pushed publicly, compile it and run tests. It must pass before being allowed to merge into develop/master/${specific-branch}. Require that merges to those branches are only fast-forward merges.
The types of tests needed will be dependent on your application. Make sure they're all green before proceeding. If not, log the failing tests. Depending on your application, you may want to have certain testing metrics also fail the build. For example, require that at least 50% of new lines of code have test coverage and 60% of your overall source code have coverage.
Run any static analysis tools and linters. Configure which rules should trigger failures or warnings.
Run any other scripts that aren't related to code quality like minification or building binaries.
Deploy your application to the proper environment.
This is a bare-bones list. Your application may have many more steps, but this should get you started.
Are you looking for advice for tools or for how to configure a pipeline?
The other answer mentions some pipeline steps (lint, test, compile etc), so I'll focus on tools.
The heavyweight champion for self hosted CI is still Jenkins I think. Pipeline config files are written with a scripting language (Groovy) instead of yaml, which gives you a lot of options for customization: https://jenkins.io/
But for a first CI setup for a small team it might be too much. I think Drone is lighter and simpler, while still being popular enough for having a community that creates many plugins: https://drone.io/
The other answer mentions some pipeline steps (lint, test, compile etc), so I'll focus on tools.
The heavyweight champion for self hosted CI is still Jenkins I think. Pipeline config files are written with a scripting language (Groovy) instead of yaml, which gives you a lot of options for customization: https://jenkins.io/
But for a first CI setup for a small team it might be too much. I think Drone is lighter and simpler, while still being popular enough for having a community that creates many plugins: https://drone.io/
Nice. I liked Ken Beck's quote. Insightful as always.
As far as I remember, Continuous Integration (CI) was about merging your code with the rest of the team as frequently as possible. Jenkins, and other tools were not even in the horizon. Because in the waterfall prehistory, some teams used to divide the system in parts and integrate the source code at the end. And it was hell. So now we have these powerful tools like git, and in a way we use them to cheat and not do CI.
As far as I remember, Continuous Integration (CI) was about merging your code with the rest of the team as frequently as possible. Jenkins, and other tools were not even in the horizon. Because in the waterfall prehistory, some teams used to divide the system in parts and integrate the source code at the end. And it was hell. So now we have these powerful tools like git, and in a way we use them to cheat and not do CI.
I don’t think it occurred to me that one day I would have to choose between CI and code reviews, and yet that’s where I’ve been living for years.
I think I’d rather have CI.
I think I’d rather have CI.
Nothing to stop code reviews either. Just don't make them a gateway. Allow code to be merged then reviewed if necessary. If you have good common coding practices and testing then likely more than safe.
Great advice. This is why it is good to go back to the basics. Adding gateways goes against any lean principles, introduces delays (=waist) in the process.
This approach has the advantages of both of the other approaches: