Take the pain out of Git conflict resolution: use diff3 (2017)(blog.nilbus.com)
blog.nilbus.com
Take the pain out of Git conflict resolution: use diff3 (2017)
https://blog.nilbus.com/take-the-pain-out-of-git-conflict-resolution-use-diff3/
68 comments
My experience is that the conflict files (with <<<< ===== >>>>) that git generates do have all the trivial conflicts resolved. In fact, before I learned what the the magic-wand icon did, the IntelliJ merge conflict tool really annoyed me since it showed all sorts of conflicts that git didn't.
There's a (relatively) new feature in Git that makes the conflicts even better.
This write-up is deep and super comprehensive:
https://www.eseth.org/2020/mergetools.html
`git config mergetool.hideResolved true` enables the modern git mergetool behavior.
This write-up is deep and super comprehensive:
https://www.eseth.org/2020/mergetools.html
`git config mergetool.hideResolved true` enables the modern git mergetool behavior.
JetBrains resolves conflicts that git can’t IME. For example multiple branches modifying the same import lines in a Python file (most common merge conflict I see), JetBrains can resolve this.
I’d be interested to know how they solve it, I suspect they run some language-aware logic to do the merge on a given hunk.
I’d be interested to know how they solve it, I suspect they run some language-aware logic to do the merge on a given hunk.
Beyond Compare also offers 3-way merge conflict resolution. Very helpful.
+1 Beyond Compare is straight-up amazing! Its 3-way merge conflict resolution does 90% of the work for you. Best in its class at what it does!
[deleted]
While IntelliJ's 3-way merge is great, there are a few things that could use improvement. If both sides change a line, accepting them both will lead to duplicating that line. Also, sometimes the change includes a line that didn't change, which again can lead to duplication.
It's not hard to fix that stuff manually, but I think there's just a bit more that IntelliJ could do to make this sort of merge smoother.
It's not hard to fix that stuff manually, but I think there's just a bit more that IntelliJ could do to make this sort of merge smoother.
For any neovim users, https://vimawesome.com/plugin/fugitive-vim absolutely nails three way diffing. I honestly enjoy running a :Gvdiff these days.
The diff view with all the features of the normal editor is actually amazing. I've always derided people using a GUI for version control but here I am.
EDIT: derided is a strong word :) just good to learn the command line version before moving to shortcuts
EDIT: derided is a strong word :) just good to learn the command line version before moving to shortcuts
I do agree knowing the commands isn't something to be sidestepped, but frankly I find it difficult to create a mental image of several related branches without a visual aid. I envy people to whom it comes easy, but I'd rather save whatever brainpower I have for solving other problems...
[deleted]
Visual studio and Vs code has this aswell i always used that never found IT hard to do conflicts
VS Code doesn't have 3 way merge view, yet. Looks like it's coming soon, though https://github.com/microsoft/vscode/issues/37350
The article suggests a good first step, but I recommend using a 3-way capable merge tool for an even better experience. 3-way merging with a reasonable UI should be the default, I don't really understand why it still isn't. It makes large and complex merges easy. My merge tool of choice was kdiff3 for the longest time, though these days I'm using Beyond Compare which is a commercial product.
I've been using kdiff3 for over a decades, it's just so good for a free tool.
Meld here, for years. It is capable of editing in place and three way compare. It's a very good tool, and quite lightweight too.
I'd rather use KDE software though, since my desktop environment of choice is Plasma. I haven't found the edition-in-place feature in KDiff3. Is it there? Kate also being my code editor of choice, that would be amazing. If it's not there, how do you solve merge conflicts with KDiff3 if you don't have editing in place?
I'd rather use KDE software though, since my desktop environment of choice is Plasma. I haven't found the edition-in-place feature in KDiff3. Is it there? Kate also being my code editor of choice, that would be amazing. If it's not there, how do you solve merge conflicts with KDiff3 if you don't have editing in place?
Meld is a great, free tool available on pretty much all platforms. Been using it for a long old time now and can't recommend it enough.
2-way, 3-way and folder diff, in-place editing, it has all I need.
2-way, 3-way and folder diff, in-place editing, it has all I need.
Kdiff3 can absolutely do edit in place: just type directly in the merge output window and save the result as usual.
If you're getting a lot of conflicts then you have folks working on the same code and diverging it significantly. I honestly don't think that's a problem tooling will improve because it's really an organizational/communication problem. Engineers have to know when to rebase (or merge in changes from the target) if they're working on code with other engineers. If you merge in/rebase sooner, then things tend to be painless. If it's been weeks or months then let the Circus O' Pain begin. This should be fairly easy to recognize in stand up, eg: "Oh, you're working on that module? I need to track your changes, so am I."
It's much more complicated at scale. Many conflicts can come from things like backporting or working on a feature in a codebase that deals with a lot of churn (like some drivers or the Linux kernel). Some open source projects have internal versions of the code within the company that eventually gets upstreamed after internal validation takes place, but that version of the code may not rebase for a while for some very good reasons (unresolved instability on the main branch, for instance). In that case, the code base is now both significantly ahead (with internal changes) and significantly behind (due to the other changes that have come from other massive organizations working on the same project). I have published on the topic of merge conflicts a couple times, and even I didn't understand how inevitable massive merge conflicts can be until I joined a kernel integration team at a large company.
Open source is a helluva drug :)
Open source is a helluva drug :)
Yeah. Something that I've told myself never to do but have continued to do for the last decade is to take some code that's in review ('cause reviewers be slow) and then branch off of it and write something else. There is always "one little thing" in the code that's out for review, and merging that into the future work that you branched off is always a colossal pain. Every time I do it I regret it.
(Is it just me, or is it not a social norm to treat code reviews like interrupts? I do that; I have Github connected to Slack and treat "I need my code reviewed" at the same level of priority as something like a production outage. It takes 5 minutes and unblocks an entire developer. But that favor seems rarely returned to me. I can usually get a fast review if I whine and nag, but it should just be automatic. And this isn't an artifact of any particular job, it's been like that at every job. I guess we could just stop doing code reviews, but they are pretty valuable for fixing dumb mistakes and sharing knowledge. But I digress...)
(Is it just me, or is it not a social norm to treat code reviews like interrupts? I do that; I have Github connected to Slack and treat "I need my code reviewed" at the same level of priority as something like a production outage. It takes 5 minutes and unblocks an entire developer. But that favor seems rarely returned to me. I can usually get a fast review if I whine and nag, but it should just be automatic. And this isn't an artifact of any particular job, it's been like that at every job. I guess we could just stop doing code reviews, but they are pretty valuable for fixing dumb mistakes and sharing knowledge. But I digress...)
Needing a piece of code reviewed should almost never block a developer. They should have several other tasks they can do in the meantime. The review may take 5 minutes, but it could cost you 30 minutes of context switching if you’re in the middle of something complicated.
Frequent and complex merges are inevitable if one maintains a few patches to an actively developed upstream and the upstream either does not want them or the patches are too specific to the product to be a part of the upstream.
We never use merge except fast forward merges. So we never have merge conflicts.
Of course that means that we might need to rebase and that could hit conflicts. With diff3 style rebase conflicts are generally [1] easy to understand. When a commit does not apply cleanly the diff3 style output gives you 3 sections:
1. What the code looks like after the previous commit that has already been rebased to the new branch under work
2. What the code looked like in the original branch
3. What it looked like 1 commit later in the original branch.
Now the question the human is: You (or some one else) changed it from 2 to 3 in the past. Now your code is not 2 but 1. How do you introduce an equivalent change from 1 to something that works like 3?
Once I have learned that principle I could no longer understand why diff3 is not the default or how anyone can survive without it.
[1] Of course there can always be tricky changes. If an automatic diff tool choses unlucky alignments the result is not ideal for humans. But diff3 does not make things worse and generally it does not happen that often.
Edit: A couple of iterations needed. Don't try to type somewhat complex comments on your phone...
Of course that means that we might need to rebase and that could hit conflicts. With diff3 style rebase conflicts are generally [1] easy to understand. When a commit does not apply cleanly the diff3 style output gives you 3 sections:
1. What the code looks like after the previous commit that has already been rebased to the new branch under work
2. What the code looked like in the original branch
3. What it looked like 1 commit later in the original branch.
Now the question the human is: You (or some one else) changed it from 2 to 3 in the past. Now your code is not 2 but 1. How do you introduce an equivalent change from 1 to something that works like 3?
Once I have learned that principle I could no longer understand why diff3 is not the default or how anyone can survive without it.
[1] Of course there can always be tricky changes. If an automatic diff tool choses unlucky alignments the result is not ideal for humans. But diff3 does not make things worse and generally it does not happen that often.
Edit: A couple of iterations needed. Don't try to type somewhat complex comments on your phone...
> We never use merge except fast forward merges.
Hmm. I've never really liked this pattern. We never use fast-forward merges for anything other than bringing a release branch forward to track the "current release" state[1] (the old HEAD gets renamed to a stable branch name with the version number in its name).
Benefits as far as I'm concerned:
Hmm. I've never really liked this pattern. We never use fast-forward merges for anything other than bringing a release branch forward to track the "current release" state[1] (the old HEAD gets renamed to a stable branch name with the version number in its name).
Benefits as far as I'm concerned:
- the merge commit can record information about the topic as a whole (e.g., the MR number, `Acked-by` and related reviews, etc.)
- it records the merge as a distinct event
- reverting a topic is just a single revert command of the merge itself (unless there's some magic to divine what sequence of commits is a "topic" in linear history these days?)
- backporting branches is just "branch off of the old release, merge into all relevant branches" instead of individual rebases and tandem merge requests
Personally I would probably prefer merge commits, but allowing them only if a fast forward merge were possible, too. So you can see from the history later what commits were merged at the same time (same merge request). Gitlab supports that option, but I have not convinced my team to enable it, so I have no experience. We often have merge requests with more than 1 commit.
This pattern is great. Especially with git log —first-parent. I highly recommend it. We use it to great effect.
Encouraged by your reply I looked at it again. Unfortunately it is not easy. Our QMS requires us to have related issue number(s) in every commit subject. There seems to be no way how gitlab would get that/those number(s) into the merge request automatically. It would just be a simple script that can extract them from the commits being merged, but I don't see a way to run such hook.
So we would need to update our process documentation and our CI to allow merge requests without an issue number. And git log —first-parent would become less useful.
So we would need to update our process documentation and our CI to allow merge requests without an issue number. And git log —first-parent would become less useful.
This is solved on our end by having the merge not done by gitlab itself (FWIW, github is bad at making these things too) but instead by our robot.
I'm a process wonk, but these checkbox items are still the bane of everyone. It'd be far better if such information were more structured. I'd recommend migrating to git trailers if possible (see `git-interpret-trailers(1)`).
While I'm being shameless anyways, features we have in our robot (for merging):
- renaming the source branch in the message (MRs from `master` is not uncommon) - treating the target branch as having a different name (so that we can use `refs/heads/release` for "latest release" while future-proofing its merge commit mentions to be `release-X.Y`) - pulling a block of text from the MR description to add to the merge commit - putting `+1` comments, reactions, and other review information in the merge commit as `Acked-by: …`, `Reviewed-by: …`, etc. - merging to multiple branches at once and synchronization (see my prior comment in this thread)
The primary deployment tool (there's a library for much of the mechanisms): https://gitlab.kitware.com/utils/ghostflow-director
I'm a process wonk, but these checkbox items are still the bane of everyone. It'd be far better if such information were more structured. I'd recommend migrating to git trailers if possible (see `git-interpret-trailers(1)`).
While I'm being shameless anyways, features we have in our robot (for merging):
- renaming the source branch in the message (MRs from `master` is not uncommon) - treating the target branch as having a different name (so that we can use `refs/heads/release` for "latest release" while future-proofing its merge commit mentions to be `release-X.Y`) - pulling a block of text from the MR description to add to the merge commit - putting `+1` comments, reactions, and other review information in the merge commit as `Acked-by: …`, `Reviewed-by: …`, etc. - merging to multiple branches at once and synchronization (see my prior comment in this thread)
The primary deployment tool (there's a library for much of the mechanisms): https://gitlab.kitware.com/utils/ghostflow-director
Interesting stuff, thank you. Wasn't aware if this.
Personally I am mostly skeptical of more tools. Some developers have challenges to understand how the basic stuff works. While automation can reduce manual work and mistakes it adds also complexity and adds new points of failure.
Of course there are always exceptions of truly useful tools you don't want to miss. It just depends where to draw the line.
Of course it also depends on the size of the organization and the teams. We have nobody even remotely dedicated to tools, "ordinary" developers maintain them as "side jobs".
Personally I am mostly skeptical of more tools. Some developers have challenges to understand how the basic stuff works. While automation can reduce manual work and mistakes it adds also complexity and adds new points of failure.
Of course there are always exceptions of truly useful tools you don't want to miss. It just depends where to draw the line.
Of course it also depends on the size of the organization and the teams. We have nobody even remotely dedicated to tools, "ordinary" developers maintain them as "side jobs".
It's a tool just like the forge is. I think of these things more as building blocks to put your desired workflow together. I doubt forges will ever handle reformatting, merging, or back porting how we want, so we'll always use something else additionally I think. We could also drop those patterns, but I foresee that meaning we don't have older supported releases instead.
Even this tool is a "side job", but it did have focused development to get it off the ground. We were migrating from a combo of patched Gerrit and gitosis with custom server side commands to implement these things before. In 2015, no forge was even close to having these things, we had to implement it ourselves. Neither Gitlab not GitHub had a good enough API then either, but we could patch and contribute to Gitlab at least. There are other workflow decisions made by Gitlab as well that I prefer too anyways.
Even this tool is a "side job", but it did have focused development to get it off the ground. We were migrating from a combo of patched Gerrit and gitosis with custom server side commands to implement these things before. In 2015, no forge was even close to having these things, we had to implement it ourselves. Neither Gitlab not GitHub had a good enough API then either, but we could patch and contribute to Gitlab at least. There are other workflow decisions made by Gitlab as well that I prefer too anyways.
Does this not result in potentially having to resolve a conflict several times? If you have 3 commits working in a block of code and then you rebase over master that has one commit changing the same area, do you not end up resolving the conflict for each of your commits?
On the other hand, if master has 3 commits changing the area and your branch has 1... It evens out.
In any case, I'd prefer to solve several smaller merge/rebase conflicts rather than one large one.
In any case, I'd prefer to solve several smaller merge/rebase conflicts rather than one large one.
I think it can happen, especially because I often prefer smaller commits over bigger ones. But if the commit to be rebased was small understanding the conflict and resolving it is usually not hard. For me resolving conflicts has become mostely routine work, not the kind of horror it was when git reported the first conflicts to me in the beginning many years ago.
There is also a command git rerere that records how you resolved a conflict and lets you do it repeatedly the same way. A colleague has tried it and said it worked nicely (not sure whether it was exactly the case yiu describe). I suspect he spent many hours on learning it and I am not convinced it has paid off for him yet. I haven't had the feeling that I urgently want more automation. I just run my git diff --ours (and occasionally also --theirs) to make sure I don't make stupid mistakes during rebasing. And in the end of course git diff @{u}.
There is also a command git rerere that records how you resolved a conflict and lets you do it repeatedly the same way. A colleague has tried it and said it worked nicely (not sure whether it was exactly the case yiu describe). I suspect he spent many hours on learning it and I am not convinced it has paid off for him yet. I haven't had the feeling that I urgently want more automation. I just run my git diff --ours (and occasionally also --theirs) to make sure I don't make stupid mistakes during rebasing. And in the end of course git diff @{u}.
Sure, diff3 is better than nothing. zdiff3 is even better. But there's so much scope for better diffs still. For starters we could have semantically aware diffs that take into account where blocks are (I think there are some efforts to do this).
But even diff3 is missing information. Lets say the code started as A, you changed it to B. Then you rebase and `master` is C. You'll get basically "It's C now! But you changed it from A to B."
That's not enough information. Why was it changed to C? I don't know how to update my diff without that information. Really you need to see the change that changed it to C too.
There's so much scope for an advanced conflict resolution tool. Git gets it wrong in so many situations where you could do better.
But even diff3 is missing information. Lets say the code started as A, you changed it to B. Then you rebase and `master` is C. You'll get basically "It's C now! But you changed it from A to B."
That's not enough information. Why was it changed to C? I don't know how to update my diff without that information. Really you need to see the change that changed it to C too.
There's so much scope for an advanced conflict resolution tool. Git gets it wrong in so many situations where you could do better.
Difflens (https://github.com/marketplace/difflens) is a step in this direction. It produces syntax aware diffs for TS/JS and CSS at the moment.
difftastic (https://difftastic.wilfred.me.uk) is an awesome tool to display diffs in syntax-aware way.
Oooo looks nice. I'm going to try it.
Not sure if I like this better.
It seems even more complicated to track 3 context points in time rather than 2.
I'm open to changing my mind, but would need reasons. Botched rebases aren't a common issue for me (though merge conflict resolution can be unpleasant).
It seems even more complicated to track 3 context points in time rather than 2.
I'm open to changing my mind, but would need reasons. Botched rebases aren't a common issue for me (though merge conflict resolution can be unpleasant).
Oh, I love diff3. Having the common base (usually) makes it easier to see "what _change_ happened in the other branch", and "what _change_ happened in my branch". Then it's (usually) straight forward to apply one change to the other.
Of course, if seeing my code and their code works for you, then by all means keep doing what works.
Of course, if seeing my code and their code works for you, then by all means keep doing what works.
I don't really feel comfortable without it. It seems imprudent not to know the common base.
You don't know the entire common base; only small windows of it where merge conflicts occurred. Elsewhere, a machine automatically merged it. That machine prudently had the common base, but didn't understand any of it.
The reason is that you have two chunks of code (possibly arbitrarily delimited).
You know from the regular two-part conflict marker that developer A wants the code one way, and B wants it some other way. Often that is quite easy to understand and resolve.
But sometimes it's useful to know the original: what is it that A and B started with, to produce those divergent sections of code?
That may be helpful.
It might not. For one thing, suppose the rest of the code merged. That means that in the rest of the code you don't see anything original.
You could end up with:
A and B both deleted the function, and that didn't conflict. Nice to know, but the definition of it isn't present to know what it does.
In other kinds of situations, it may be useful.
You know from the regular two-part conflict marker that developer A wants the code one way, and B wants it some other way. Often that is quite easy to understand and resolve.
But sometimes it's useful to know the original: what is it that A and B started with, to produce those divergent sections of code?
That may be helpful.
It might not. For one thing, suppose the rest of the code merged. That means that in the rest of the code you don't see anything original.
You could end up with:
<<<<<<<
new_function_x();
========
new_function_y();
||||||||
function_that_no_longer_exists();
>>>>>>>>
A and B both deleted the function, and that didn't conflict. Nice to know, but the definition of it isn't present to know what it does.
In other kinds of situations, it may be useful.
I don't see how there is any room for opinion. If you read the article, it opens with an example where it is impossible to determine the correct merge from the given information without diff3.
Bold of you to assume people read articles before commenting.
diff3 has another advantage in the general case as well since it allows you to better understand if something outside of the hunk you're applying also needs to be changed when backporting something.
diff3 has another advantage in the general case as well since it allows you to better understand if something outside of the hunk you're applying also needs to be changed when backporting something.
I like the idea but it is clearly incomplete. It gets the fact that there are 3 versions (and that it wants you to generate a 4th) but it misses the important thing which is that you want to see the differences. This option basically leaves you to do the diff yourself. Now the sinple option of just showing the two diffs probably isn't optimal either as the context will be duplicated noise, but I think that is already better. In fact I think the merge diff representation would work well here except for the fact that you want to be able to compile and test the code which you can do with the diff markers.
I've found configuring merge.conflictstyle diff3 without a tool to be superior than tools like kdiff3 and such. This was without any fancy merge conflict rendering. Nowadays some editors will also highlight the changes in a 3-way diff.
The reason no-tool is better is that you don't have to read files in whatever order kdiff3 shows them to you -- they're just files, and you open them.
The reason no-tool is better is that you don't have to read files in whatever order kdiff3 shows them to you -- they're just files, and you open them.
[deleted]
The best is a 4-way diff
It you merge a file in two commits A and B, it would show version A, version B, the common ancestor, and the resulting merged file in four different views.
xxdiff could do that.
It you merge a file in two commits A and B, it would show version A, version B, the common ancestor, and the resulting merged file in four different views.
xxdiff could do that.
kdiff3 is ok-ish as a GUI option.
Or meld.
But I fall back to GUI only in nasty cases. Over 90% I feel comfortable with diff3 formatted output by git.
But I fall back to GUI only in nasty cases. Over 90% I feel comfortable with diff3 formatted output by git.
I haven't used meld for a long time; the website says its three-way merge is still in development, while kdiff3 has had that for more than twenty years and it's (for me) the main reason I use it. The three-pane display with the merge result in a fourth pane at the bottom is just so easy to use. (The second reason I use it is that it allows you to mark lines that should align with each other.)
As I wrote I rarely use the GUI tool for conflict resolution because I am most of the time just fine with diff3 textual format in my editor. So I have no experience which tool would be best. I used kdiff3 15 years ago (mostly pre-git) and have drifted over to meld over the years. The reason for that has been lost in history, not sure whether it would stay that way if I started to systematically compare them today.
Meld certainly does allow to manually align lines. But sometimes I have struggled when doing several alignments in a single diff. Not sure whether kdiff3 would do better or it's just user error.
Where meld certainly fails is huge files. Its performance can be horrible. But that affects generated files like logs, not source you would have in git.
Meld certainly does allow to manually align lines. But sometimes I have struggled when doing several alignments in a single diff. Not sure whether kdiff3 would do better or it's just user error.
Where meld certainly fails is huge files. Its performance can be horrible. But that affects generated files like logs, not source you would have in git.
I personally never had to leave IntelliJ Idea for merge conflict resolution. So I am super used to it and think it is very convenient.
[as far as I understand this is similar in philosophy to a diff3. True?]
And while searching a bit, it seems that a similar UI is planned for VScode later in May 2022. Cf https://github.com/microsoft/vscode/issues/37350
[Btw, a comment mentions that Sublime Merge already has a UI similar to the one in IntelliJ]
[as far as I understand this is similar in philosophy to a diff3. True?]
And while searching a bit, it seems that a similar UI is planned for VScode later in May 2022. Cf https://github.com/microsoft/vscode/issues/37350
[Btw, a comment mentions that Sublime Merge already has a UI similar to the one in IntelliJ]
I have not used IntelliJ for over 15 years and never with git. Cannot comment.
But git diff3 style being textual format in one column it sounds surprising that a GUI would use the same approach.
But git diff3 style being textual format in one column it sounds surprising that a GUI would use the same approach.
I hope someone solves the merge conflict problem once and for all. Each time there’s a merge conflict the project stops for days as no one knows how to resolve them. In the end the solution always is to delete the git repository and clone it again.
I'm confused by your comment. What kind of merge conflicts stop a project for days? And why would the solution to a merge conflict be to delete the repository and clone it again? Or are you being sarcastic?
All merge conflicts. You can never resolve them as everything is screwed. Takes days to figure it out. You usually just delete the repo and start all over again from a working commit.
You can set a flag in git so it won't start a merge if it can't finish it with a simple fast forward. This avoids this situation.
git pull --rebase can be used to get local changes on top of new remote work.
But just deleting (or renaming the branch) and checking out again is all that is necessary if you end up in a messed up merge.
git pull --rebase can be used to get local changes on top of new remote work.
But just deleting (or renaming the branch) and checking out again is all that is necessary if you end up in a messed up merge.
Thanks!
Embrace the pain. If you try to let a tool help you too much with conflicts you may end up regretting it.
I fail to see how "embracing the pain" could be preferable to making more informed decisions (like knowing what the "merged common ancestor" looked like). We are not talking about some magical button that uses "AI" to resolve the conflicts for you. diff3 just gives you more context about the conflict so you can make a more informed decision....
I'm not clear how this solution removes pain, it could make things more confusing for many.
Diff3 is good but diff4 is life-changing.
[deleted]
Their diff tool also has a magic-wand icon, which resolves trivial conflicts automatically. I still wonder why git doesn't do that by default... It's such a pain to resolve conflicts without that feature.
Also, you get syntax highlighting, structural editing, help, definition quick-lookup and similar IDE features available, while working on the resolved version of the code, which is syntactically correct usually, because there are no explicit, textual conflict markers in the resolved text.