Highlights from Git 2.34(github.blog)
github.blog
Highlights from Git 2.34
https://github.blog/2021-11-15-highlights-from-git-2-34/
99 comments
Related: it's been a year since Pijul (https://pijul.org) began working towards 1.0 (https://news.ycombinator.com/item?id=25032956). Seems to be coming along nicely, they're in alpha now and it's showing.
I keep checking in on that. In my case what I need to start seriously using it is two-way git interop. I need to be able to work in pijul locally and "publish" somehow to git. So far that still doesn't seem to exist.
If the Pijul feature you care most about is "first-class conflicts", you may want to check out my VCS: https://github.com/martinvonz/jj. It has git interop.
That looks very cool, and very wise to use git as the actual storage system. Means you have a chance of people actually using it.
I have one question: how do changes to files in the working copy automatically update the current commit hash? Do you just calculate it on demand, and then actually make the commit when some command would move to different commit (`jj switch someotherbranch` or whatever)?
I have one question: how do changes to files in the working copy automatically update the current commit hash? Do you just calculate it on demand, and then actually make the commit when some command would move to different commit (`jj switch someotherbranch` or whatever)?
Any command that depends on the working copy commit will first check if it needs to create a new commit from the working copy contents. So when you run e.g. `jj status`, it will amend the current working copy commit if there are changes.
In case you're thinking that that seems slow, it's not really significant at least up to git.git scale. For larger repos, I plan to rely on something like watchman.
In case you're thinking that that seems slow, it's not really significant at least up to git.git scale. For larger repos, I plan to rely on something like watchman.
Thanks, I'll check it out. Yeah, that's the main draw for me.
First, you can easily import Git repos into Pijul.
One issue with two-way interop is that while importing a commit into a patch is easy, doing the opposite is more complicated. Indeed, since Pijul is patch-based, there is no way to map patches onto commits in a unique way, since a single patch is quite independent from the context (other patches around it), which is what makes Pijul easy to use.
Also, if Alice and Bob work on a common Pijul server in parallel with their Git server, they'll get artificial conflicts in Pijul, even though they have the same contents.
One issue with two-way interop is that while importing a commit into a patch is easy, doing the opposite is more complicated. Indeed, since Pijul is patch-based, there is no way to map patches onto commits in a unique way, since a single patch is quite independent from the context (other patches around it), which is what makes Pijul easy to use.
Also, if Alice and Bob work on a common Pijul server in parallel with their Git server, they'll get artificial conflicts in Pijul, even though they have the same contents.
I just want to be able to use pijul myself, locally. So I'd want it to just arbitrarily choose a mapping of patches to commits, like by timestamp I guess?
That could work. If your local patches are complex enough you could even rearrange them using patch commutation.
At the moment, Pijul is in a state where there is just one remaining feature I want to implement before beta, with a massive expected impact on disk space (and a limited impact on speed as well).
For the kind of "smaller (but essential) features" like what you're suggestion, there are tons of those, and we can totally implement them after beta. Git export is a common request we get, though. I'm just scared of how people will react when they export and then reimport their patches elsewhere, and get tons of fake conflicts.
At the moment, Pijul is in a state where there is just one remaining feature I want to implement before beta, with a massive expected impact on disk space (and a limited impact on speed as well).
For the kind of "smaller (but essential) features" like what you're suggestion, there are tons of those, and we can totally implement them after beta. Git export is a common request we get, though. I'm just scared of how people will react when they export and then reimport their patches elsewhere, and get tons of fake conflicts.
And yet given that git is the current "default" VCS, it's very important that this is/will be "as smooth as possible"..
I wonder if it wouldn't be possible to memorize "git order" of patches when importing them in Pijul to be able to re-export them in the same order as they were imported before.
I wonder if it wouldn't be possible to memorize "git order" of patches when importing them in Pijul to be able to re-export them in the same order as they were imported before.
I think you will get a lot of complaints, which is unfortunate, but many many will understand that it's not going to be perfect. I was there for the period of switching from svn to git, and similar things happened. Really the only way I even learned git itself is because I could use git locally and the rest of my team couldn't tell.
Regardless though, pijul is really cool. I just badly want to be able to use it at my day job :)
Regardless though, pijul is really cool. I just badly want to be able to use it at my day job :)
I've got some patches in this release. If anyone's got question that someone who contributes to git might be able to answer I'd love to help.
Are there any specific things that git does that you wish were done better? I probably won't be able to help with very generic things like "it's complex", but if there's specific drawbacks in some particular commands I might be able to fix them.
Or anything else, I'll monitor this thread while there's interest.
Are there any specific things that git does that you wish were done better? I probably won't be able to help with very generic things like "it's complex", but if there's specific drawbacks in some particular commands I might be able to fix them.
Or anything else, I'll monitor this thread while there's interest.
Do you know what github does differently when performing a merge? I’ve noticed sometimes github says there’s a merge conflict yet when i do the merge locally using the default recursive strategy it often just merges without any conflicts. I know there are cases it can’t handle when there are conflicts, but it seems odd that it cannot merge when there are no merge conflicts.
GitHub turns off rename detection and turns off recursive base-building when creating merge commits in pull requests. Both of these selections would cause differences in when a merge produces conflicts. This is for backward compatibility with historical mechanisms for merging pull requests, but it seems like something that we might want to revisit.
GitHub (which I'm not affiliated with) does some (all?) of their server-side merging with libgit2, not git itself, although I know they're considering moving to git with the "ort" work. So perhaps it's one of the cases where libgit2's behavior differs?
That's certainly possible, but libgit2 carries an _exhaustive_ test corpus around merge as we aim to be bug-for-bug compatible with git-merge-recursive. This was also proved out by A/B testing the two at scale: https://github.blog/2015-12-15-move-fast/
Aha, that’s useful to know. Thanks!
This might help https://docs.github.com/en/repositories/configuring-branches...
That page details squashing and rebasing strategies. It doesn’t talk about different merge points being used in a merge when done on GH compared to when the same merge is performed locally on the command line.
I'm not always very happy with the conflicts git rebase gives me.
For example when I try to bring an old feature branch forward, or when I have two private branches with a common history that I've touched on one side and want rebase to apply my history change onto the other branch, I'm often made to resolve a lot more conflict that seems necessary.
There are scripts like git-imerge that give some much nicer conflicts. imerge may not be perfect, but I think it shows there's room for a more clever algorithm.
I'd happily have a git rebase --full-resolver --wuauctl option that consumes a mysteriously large amounts of CPU time, if it saves me some manual conflict resolution =)
For example when I try to bring an old feature branch forward, or when I have two private branches with a common history that I've touched on one side and want rebase to apply my history change onto the other branch, I'm often made to resolve a lot more conflict that seems necessary.
There are scripts like git-imerge that give some much nicer conflicts. imerge may not be perfect, but I think it shows there's room for a more clever algorithm.
I'd happily have a git rebase --full-resolver --wuauctl option that consumes a mysteriously large amounts of CPU time, if it saves me some manual conflict resolution =)
They have shipped a new default merging strategy, 'ort', that is supposedly better than the previous default strategy at resolving this kind of issue. You should probably try and see if it makes any difference.
My understanding was that `merge -sort` is generally faster and has less buggy edge cases, but does it resolve conflict any differently than -s recursive ?
That's correct, there may be some very obscure edge cases where it has different behavior (known merge-recursive bugs), but in the general case if you've got UX problems with rebase or merge it won't help.
Unless that UX problem is helped by it being much, much faster, but that's usually only going to be matter for larger repositories.
Unless that UX problem is helped by it being much, much faster, but that's usually only going to be matter for larger repositories.
Something that has been nagging me for my workflow is how Git handles --autosquash.
I really like the idea, and wrote my personal little `git autosquash` command that tries to figure out the oldest commit it has to run `git rebase -ir --autosquash` on in order to merge all current "fixup! …" commits into their counterparts.
However, doing this in a simple shell script is rather dodgy if you run into more generic commit messages. Personally, I'd love it if git allowed me to create "fixup! <commit hash>" commits instead. I mean -- it's a fixup. It's not supposed to be readable anyway.
I really like the idea, and wrote my personal little `git autosquash` command that tries to figure out the oldest commit it has to run `git rebase -ir --autosquash` on in order to merge all current "fixup! …" commits into their counterparts.
However, doing this in a simple shell script is rather dodgy if you run into more generic commit messages. Personally, I'd love it if git allowed me to create "fixup! <commit hash>" commits instead. I mean -- it's a fixup. It's not supposed to be readable anyway.
You can "git commit --fixup <commit>", but that creates a commit whose subject is the subject line of the commit to be fixed up, so that if you run one rebase and the hashes change you can still run --autosquash and the like (and of course, manually move them around).
Do you mean that you've got commit subject that are ambiguous within the series you're rebasing, so --autosquash won't correctly attribute them?
Do you mean that you've got commit subject that are ambiguous within the series you're rebasing, so --autosquash won't correctly attribute them?
Not exactly. My goal is mainly that I'd like to avoid running git log and manually deciding the starting commit (and copying its hash around) before rewriting local branch histories.
What I want to achieve is that I can run my script (i.e. `git autosquash`), and it will automatically start the appropriate interactive rebase for me. That means I still get the rebase description I can check by eye in Vim (and have the option of aborting by emptying the buffer before quitting it).
I usually use this to clean up the history of a private local branch, so I won't trigger anything like `git rebase master`. This means that I have to specify the commit where Git is supposed to start the interactive rebase (i.e. the oldest commit that won't be included). And finding that by looking for the subject line after "fixup!" is rather finnicky (especially if the subject line also matches those of other commits), whereas the simple commit hash would be nice and explicit (and finding the starting point would be a simple `git merge-base --octopus` of all fixup! hashes).
So, my problem isn't the autosquash resolution built into Git, but that I have to reimplement it in a script in order to find the proper rebase starting point. That, and all backward searches for commit messages I know of are either really fuzzy or needlessly convoluted.
The best exact matching search I could come up with is something akin to
What I want to achieve is that I can run my script (i.e. `git autosquash`), and it will automatically start the appropriate interactive rebase for me. That means I still get the rebase description I can check by eye in Vim (and have the option of aborting by emptying the buffer before quitting it).
I usually use this to clean up the history of a private local branch, so I won't trigger anything like `git rebase master`. This means that I have to specify the commit where Git is supposed to start the interactive rebase (i.e. the oldest commit that won't be included). And finding that by looking for the subject line after "fixup!" is rather finnicky (especially if the subject line also matches those of other commits), whereas the simple commit hash would be nice and explicit (and finding the starting point would be a simple `git merge-base --octopus` of all fixup! hashes).
So, my problem isn't the autosquash resolution built into Git, but that I have to reimplement it in a script in order to find the proper rebase starting point. That, and all backward searches for commit messages I know of are either really fuzzy or needlessly convoluted.
The best exact matching search I could come up with is something akin to
git log --pretty=$'%H\x1F%s' "$fixup_commit_hash" | awk -F $'\x1F' -v "subject=$query" '$2 == subject { print $1; exit }'I understand.
Is there any reason you don't (or perhaps you're not aware that you can) use the branch upstream info for this?
Whenever I work on a branch called "topic" I do the equivalent of "git branch --set-upstream-to origin/master" on it. That means that all of:
Is there any reason you don't (or perhaps you're not aware that you can) use the branch upstream info for this?
Whenever I work on a branch called "topic" I do the equivalent of "git branch --set-upstream-to origin/master" on it. That means that all of:
git log @{u}..
git diff @{u}
git diff @{u}...
git rebase -i
git rebase -i @{u}
A foo
B bar
C !fixup bar
D !fixup bar
You're right, I didn't know about @{u}, thanks for the hint!
> You'd open an editor with A..D with that, whereas you perhaps only want B..D?
That wouldn't be much of an issue for me, but I still tried to implement an option that doesn't rely on my repository having a defined remote.
I quite often start a new local git repository whenever I'm about to start a new experiment (or just scripting some one-off task), and most of these will never get pushed onto a git server. However, even in those cases, my scatterbrain often triggers the "gah, that should have been in a previous commit!" reflex.
By the way: Thank you very much for taking your time and reading all this! :)
Edit: I just re-read the upstream-point thing. Wouldn't that cause a `git push` on `thing` to push directly to `origin/master`?
That also wouldn't be compatible with the workflow we have at work (we use GitLab merge requests)…
> You'd open an editor with A..D with that, whereas you perhaps only want B..D?
That wouldn't be much of an issue for me, but I still tried to implement an option that doesn't rely on my repository having a defined remote.
I quite often start a new local git repository whenever I'm about to start a new experiment (or just scripting some one-off task), and most of these will never get pushed onto a git server. However, even in those cases, my scatterbrain often triggers the "gah, that should have been in a previous commit!" reflex.
By the way: Thank you very much for taking your time and reading all this! :)
Edit: I just re-read the upstream-point thing. Wouldn't that cause a `git push` on `thing` to push directly to `origin/master`?
That also wouldn't be compatible with the workflow we have at work (we use GitLab merge requests)…
You can set "upstream" to another local branch, no remote is required, it's just how it's typically used. Under the hood it's just a way of naming another reference as a divergence point.
I use this extensively with Git(Hub|Lab) (PR|MR)'s. I set the upstream to "master" but push to "topic". The "push.default" docs cover how to have your cake and eat it too there.
I use this extensively with Git(Hub|Lab) (PR|MR)'s. I set the upstream to "master" but push to "topic". The "push.default" docs cover how to have your cake and eat it too there.
The reference to the `push.default` config key clarified a lot of things for me. Thank you very much! :)
I have two wishes for git:
1. A consistent cross-language API for reading and writing from a bare/remote git repo in ways that would make sense to an average programmer. Something like `git.stat("/file.txt", commit=...)`, `git.open("/file.txt", branch=...)` `git.list_commits(since=...)`, `git.create_commit(parrent_commits=[], branch="", ...)`
2. A good story, better than LFS, for storing large binaries over time.
For #2 I want to reproducibly build a debian image from vendored packages or recompile gcc from source so I need to check in a pre-built compiler. etc
1. A consistent cross-language API for reading and writing from a bare/remote git repo in ways that would make sense to an average programmer. Something like `git.stat("/file.txt", commit=...)`, `git.open("/file.txt", branch=...)` `git.list_commits(since=...)`, `git.create_commit(parrent_commits=[], branch="", ...)`
2. A good story, better than LFS, for storing large binaries over time.
For #2 I want to reproducibly build a debian image from vendored packages or recompile gcc from source so I need to check in a pre-built compiler. etc
OK: I have a large number of large git submodules that I never edit, but they make every other command that just incidentally creates a diff output slow. I have found command line options for things like git status/diff (--ignore-submodules=dirty) and git commit (--no-status) to avoid this penalty, but I use git rebase -i constantly and wany to use git stash a lot, but the cost of these commands is somewhat brutal when they sometimes cause incidental status reports. What I really want is a setting that like "avoid incidental status" that disables status reports from any command except git status (as I do want to be able to see this status), as I simply don't ever need the noise (such as in the git commit default message) and it really is a pretty annoying delay when rapidly throwing around commands to edit history to have one which suddenly decides to take forever. If nothing else, though, a --no-status for git stash/rebase (I use all these commands through aliases anyway) would be very helpful. (I would be happy to be more specific if this sounds interesting but isn't enough to know what I mean, but I figure it might either be something you wouldn't get behind anyway and I also really want to take a nap and so want to leave a message earlier ;P. I also happen to be [email protected] and @saurik on Twitter.)
I really enjoy git way more than others but working in unreal engine and with binary assets. I've always felt like git filr locks were an area that still needs more attention.
I know file locks are almost an antipattern for git. But there are definitely good use cases and arguments for better git support and related tooling support around them, especially for binary heavier areas.
The other thing I saw other day was sparse index or something. At work we have a monorepo and I wish that was more control or granularity over checking out a subset of a repo somehow. Our clones even with shallow fetch and lfs can be 5+ minutes for an area that should only take 3 seconds. But no good subset checkout.
I know file locks are almost an antipattern for git. But there are definitely good use cases and arguments for better git support and related tooling support around them, especially for binary heavier areas.
The other thing I saw other day was sparse index or something. At work we have a monorepo and I wish that was more control or granularity over checking out a subset of a repo somehow. Our clones even with shallow fetch and lfs can be 5+ minutes for an area that should only take 3 seconds. But no good subset checkout.
> Are there any specific things that git does that you wish were done better?
I wish `git-log`'s `--follow` wasn't optional so GitHub et al would show the complete history for files I `git mv`. Mercurial got this one right :)
I wish `git-log`'s `--follow` wasn't optional so GitHub et al would show the complete history for files I `git mv`. Mercurial got this one right :)
Yeah, me to. I think that was always the plan, but nobody's had the time to go back, re-visit it, and make it so.
I'd say that most of the work of driving features forward in git is hunting down those various edge cases, taking the time to understand them, and explaining them to others. So if anyone's interested in flipping that particular switch it's very open to first-time contributions.
I'd say that most of the work of driving features forward in git is hunting down those various edge cases, taking the time to understand them, and explaining them to others. So if anyone's interested in flipping that particular switch it's very open to first-time contributions.
Well, since you ask. `git add -i` has different "escape"/"exit" semantics for different actions.
If I want to quit/escape adding a patch in the interactive mode I type `q` and then enter.
If I want to quit/escape the interactive add subcommand I type `q` and then enter.
But if I want to quit the `update`, `untracked` etc. I have to enter nothing and then press enter. It would be such a wonderful thing if it also accepted `q` so my muscle "memory" is correct.
I looked into changing this once, but it is written in shell and Perl, which I don't know enough about to figure out.
If I want to quit/escape adding a patch in the interactive mode I type `q` and then enter.
If I want to quit/escape the interactive add subcommand I type `q` and then enter.
But if I want to quit the `update`, `untracked` etc. I have to enter nothing and then press enter. It would be such a wonderful thing if it also accepted `q` so my muscle "memory" is correct.
I looked into changing this once, but it is written in shell and Perl, which I don't know enough about to figure out.
I actually just ran into this yesterday and completely agree! I ended up just C-c-ing out and used -p.
The other thing that was frustrating with that was that the help text for what commands do is printed before everything, so if your window is small and there's a lot of output, it's not obvious that the ? you typed did anything other than re-print the hunk. It seems odd to not print the help information at the end so you will always see it (unless your window is less than one row tall, but.. you're probably here for dragons anyway in that situation).
The other thing that was frustrating with that was that the help text for what commands do is printed before everything, so if your window is small and there's a lot of output, it's not obvious that the ? you typed did anything other than re-print the hunk. It seems odd to not print the help information at the end so you will always see it (unless your window is less than one row tall, but.. you're probably here for dragons anyway in that situation).
Aside from the UX question you've got, which I haven't had time to look at (sorry). We've shipped a pure C version of "add -i" that's been on if you've enabled feature.experimental=true since v2.29.0.
It looks like the ball may have been dropped on opting users in to it by default, but in any case there's numerous bugs in "git add -p" and "git add -i" that got fixed along the way, and fixing existing ones should be a lot easier now.
Note: I've also worked on the perl compiler/interpreter in a past life, so it's not that I'm afraid of hacking on Perl, but doing so in the context of the Git project is and was always particularly painful. If something's written in C you could use all the APIs git had internally, but anything in .sh or .perl was usually stuck parsing text, or reinventing the wheel.
It looks like the ball may have been dropped on opting users in to it by default, but in any case there's numerous bugs in "git add -p" and "git add -i" that got fixed along the way, and fixing existing ones should be a lot easier now.
Note: I've also worked on the perl compiler/interpreter in a past life, so it's not that I'm afraid of hacking on Perl, but doing so in the context of the Git project is and was always particularly painful. If something's written in C you could use all the APIs git had internally, but anything in .sh or .perl was usually stuck parsing text, or reinventing the wheel.
Sometimes I almost lose my mind over these situations where git on windows reports a permissions change (git status claims a diff) but there is no way to commit nor revert. This happens maybe once per year and I’ve no idea what a proper fix is. Typically blocks rebasing etc but revert and commit can’t fix it. (Apologies for the poor and ranty bug report.)
I ended up putting this in my .gitconfig (global)
[alias]
permission-reset = "!git diff -p | grep -E \"^(diff|old mode|new mode)\" | sed -e \"s/^old/NEW/;s/^new/old/;s/^NEW/new/\" | git apply"
It's really neat that ssh keys can be used for signing commits, but I'm not a big fan of this.
When auditing a source repository you want to see how commits are signed over time. In other words you have a key X which is valid at time Y. As Y changes, key X will change as well. To audit a repo, you need to know the history of what key X was valid at time Y.
With GPG you can set up a hierarchy of commit signing keys which have defined lifetimes and are all subordinate to a master key which you do not use for signing (and hopefully keep somewhere secure). One irritant is that the major git-as-a-service products, i.e. github + gitlab, do not respect the use of GPG keys as something varying over time or subordinate to a master key. Instead they let you set a current active key and mark on their own if it was verified at the time. If you want to validate the repository's security yourself, you don't have the proper information to do so.
The use of ssh keys as signing keys furthers this incorrect usage of a single key as something that does not vary over time. You can use ssh certificates as a way to have a hierarchy, but that still doesn't acknowledge that there is historical data of note.
Now, a logical question is: If each git commit is the summation of everything before, why does historical data matter? Why can't I just trust the git{hub,lab} when it says it was "Verified" at the time? A few reasons:
- Github allows someone logged in to make edits via their web editor which they helpfully sign with a universal github key that is always verified. So if your AuthN is bad, that is a problem.
- You are trusting that your git-as-a-service provider will never have their repositories tampered with and your org will just download the newest version of the repo.
- It relies on git-as-a-service providers to track this. If you aren't using a managed service provider you need to roll your own auditing tools.
When auditing a source repository you want to see how commits are signed over time. In other words you have a key X which is valid at time Y. As Y changes, key X will change as well. To audit a repo, you need to know the history of what key X was valid at time Y.
With GPG you can set up a hierarchy of commit signing keys which have defined lifetimes and are all subordinate to a master key which you do not use for signing (and hopefully keep somewhere secure). One irritant is that the major git-as-a-service products, i.e. github + gitlab, do not respect the use of GPG keys as something varying over time or subordinate to a master key. Instead they let you set a current active key and mark on their own if it was verified at the time. If you want to validate the repository's security yourself, you don't have the proper information to do so.
The use of ssh keys as signing keys furthers this incorrect usage of a single key as something that does not vary over time. You can use ssh certificates as a way to have a hierarchy, but that still doesn't acknowledge that there is historical data of note.
Now, a logical question is: If each git commit is the summation of everything before, why does historical data matter? Why can't I just trust the git{hub,lab} when it says it was "Verified" at the time? A few reasons:
- Github allows someone logged in to make edits via their web editor which they helpfully sign with a universal github key that is always verified. So if your AuthN is bad, that is a problem.
- You are trusting that your git-as-a-service provider will never have their repositories tampered with and your org will just download the newest version of the repo.
- It relies on git-as-a-service providers to track this. If you aren't using a managed service provider you need to roll your own auditing tools.
All of this signing is only useful if someone is actually checking or enforcing those signatures, otherwise it's just LARPing.
Being able to sign commits with your SSH keys makes signing actually useful, because it enables a new workflow that developers will use:
- You give every dev on your team a Yubikey
- They generate an ed25519-sk key that only resides on the Yubikey, no software required as it works out of the box with both openssh and GitHub
- They upload the public ID of the key to GitHub, same as before
- You enforce commit signature verification for your GitHub org. You're done, no need to install any software, everything Just Works.
You now have:
- No private keys on developers machines, rendering all types of supply chain attacks like NPM stealing your .ssh files ineffective
- Enforced 2FA for everyone without any hassle
- Every commit signed by developers, enforced and with no developer overhead. Checks a lot of boxes for those SOCs and ISOs.
Being able to sign commits with your SSH keys makes signing actually useful, because it enables a new workflow that developers will use:
- You give every dev on your team a Yubikey
- They generate an ed25519-sk key that only resides on the Yubikey, no software required as it works out of the box with both openssh and GitHub
- They upload the public ID of the key to GitHub, same as before
- You enforce commit signature verification for your GitHub org. You're done, no need to install any software, everything Just Works.
You now have:
- No private keys on developers machines, rendering all types of supply chain attacks like NPM stealing your .ssh files ineffective
- Enforced 2FA for everyone without any hassle
- Every commit signed by developers, enforced and with no developer overhead. Checks a lot of boxes for those SOCs and ISOs.
- No private keys on developers machines, rendering all types of supply chain attacks like NPM stealing your .ssh files ineffective
Would it be able to get the key from the usb device? Or does the hashing happen on the device itself with the private key never exposed? If that's how it works, that would be awesome, but would a malicious program be able to sign something as you, instead of stealing the keys?
I'm going to look into this more and maybe recommend it at work.
Would it be able to get the key from the usb device? Or does the hashing happen on the device itself with the private key never exposed? If that's how it works, that would be awesome, but would a malicious program be able to sign something as you, instead of stealing the keys?
I'm going to look into this more and maybe recommend it at work.
>Would it be able to get the key from the usb device? Or does the hashing happen on the device itself with the private key never exposed?
If you could extract the key, you probably have access to the intelligence resources of a very large nation. With hardware security dongles, the key remains on the dongle; you pass your data in and it passes the signature (or encrypted data) back out.
If you could extract the key, you probably have access to the intelligence resources of a very large nation. With hardware security dongles, the key remains on the dongle; you pass your data in and it passes the signature (or encrypted data) back out.
> They generate an ed25519-sk key that only resides on the Yubikey, no software required as it works out of the box with both openssh and GitHub
How does this work? Can you like me a ressource?
How does this work? Can you like me a ressource?
You run `ssh-keygen -t ed25519-sk -O resident`, and that's it, no additional software required as of OpenSSH 8.2. If you have an older model Yubikey you can try ecdsa-sk instead.
For more details, see:
https://github.blog/2021-05-10-security-keys-supported-ssh-g...
https://www.yubico.com/blog/github-now-supports-ssh-security...
For more details, see:
https://github.blog/2021-05-10-security-keys-supported-ssh-g...
https://www.yubico.com/blog/github-now-supports-ssh-security...
It's possible to set "valid-before" and "valid-after" timestamps on SSH keys in the allowed signers file:
https://man.openbsd.org/ssh-keygen#ALLOWED_SIGNERS
When validating the SSH signature, Git can set the verify-time option to the commit timestamp and the signature will be rejected if the key is not valid at that time. Git doesn't yet do this, but will in a future version.
This is considerably simpler than the rigamarole of GPG subkeys.
https://man.openbsd.org/ssh-keygen#ALLOWED_SIGNERS
When validating the SSH signature, Git can set the verify-time option to the commit timestamp and the signature will be rejected if the key is not valid at that time. Git doesn't yet do this, but will in a future version.
This is considerably simpler than the rigamarole of GPG subkeys.
This is a totally valid criticism IMO, and I think platforms should allow for disallowing signatures based on origin, but there's something that, for the majority of users, really holds true:
> With GPG you can set up a hierarchy of commit signing keys which have defined lifetimes and are all subordinate to a master key which you do not use for signing (and hopefully keep somewhere secure).
Within epsilon of nobody is going to do this, but they all have an SSH key. Heck, I don't do this, and I even actually know how to make GPG do it. It's a huge hassle for what, at this point, is really only useful to me for...signing Git commits? I don't need to sign anything else with it, and GPG mail is nonexistent.
(Also, fwiw, if you aren't using a managed service provider, I hope you have a directory service where you can have those keys. If you aren't using a managed service provider and you don't have a directory service, I'm gonna go out on a limb and say that I'm guessing security isn't that important to you.)
> With GPG you can set up a hierarchy of commit signing keys which have defined lifetimes and are all subordinate to a master key which you do not use for signing (and hopefully keep somewhere secure).
Within epsilon of nobody is going to do this, but they all have an SSH key. Heck, I don't do this, and I even actually know how to make GPG do it. It's a huge hassle for what, at this point, is really only useful to me for...signing Git commits? I don't need to sign anything else with it, and GPG mail is nonexistent.
(Also, fwiw, if you aren't using a managed service provider, I hope you have a directory service where you can have those keys. If you aren't using a managed service provider and you don't have a directory service, I'm gonna go out on a limb and say that I'm guessing security isn't that important to you.)
> When auditing a source repository you want to see how commits are signed over time.
I understand your reasoning and the idea behind GPG and everything, but here's my challenge to all of this: Has this ever happened?
Like, the scenario we're talking about is that at some point some bad code, maybe a backdoor, got introduced into a code repo, and it later is discovered and someone launches an investigation. And they find out "this was committed by XYZ but XYZ claims he didn't do it".
I don't remember any such incident ever. Given that I feel a slightly weaker form with SSH keys may still be justified. (And it is still easily conceivable to have some out-of-band-metadata, e.g. a project that feels this is important could just have a list of developer SSH key fingerprints within the repo and thus have the backlog of who's keys were relevant at what point in time.)
I understand your reasoning and the idea behind GPG and everything, but here's my challenge to all of this: Has this ever happened?
Like, the scenario we're talking about is that at some point some bad code, maybe a backdoor, got introduced into a code repo, and it later is discovered and someone launches an investigation. And they find out "this was committed by XYZ but XYZ claims he didn't do it".
I don't remember any such incident ever. Given that I feel a slightly weaker form with SSH keys may still be justified. (And it is still easily conceivable to have some out-of-band-metadata, e.g. a project that feels this is important could just have a list of developer SSH key fingerprints within the repo and thus have the backlog of who's keys were relevant at what point in time.)
> Has this ever happened?
Unfortunately this has: https://www.wired.com/2015/12/researchers-solve-the-juniper-...
While the full results were not released, it was pretty clear that a rogue commit happened to the VCS. This is the highest profile example that has occurred in my corner of the tech world.
The broader category of this attack is "insider threat" and it is a real thing. Even googling for "malicious code commit" turns up this other example for me as one of the first results: https://www.theregister.com/2021/03/29/php_repository_infect...
Unfortunately this has: https://www.wired.com/2015/12/researchers-solve-the-juniper-...
While the full results were not released, it was pretty clear that a rogue commit happened to the VCS. This is the highest profile example that has occurred in my corner of the tech world.
The broader category of this attack is "insider threat" and it is a real thing. Even googling for "malicious code commit" turns up this other example for me as one of the first results: https://www.theregister.com/2021/03/29/php_repository_infect...
[deleted]
Given Juniper released basically nothing about their investigation, I'm not convinced this is a valid example.
Notably, I'm not just asking if there ever was a bad commit that was investigated - but that the commit came from someone who later claimed she/he didn't do it.
Notably, I'm not just asking if there ever was a bad commit that was investigated - but that the commit came from someone who later claimed she/he didn't do it.
I interviewed one of the creators of the Git sparse index a while back: https://www.softwareatscale.dev/p/software-at-scale-32-derri... which had some interesting tidbits.
Major: A new default merge strategy
> In Git 2.34, ort is now the default merge strategy, so you should notice faster merges with fewer bugs just by upgrading
> a rewrite allowed Git to implement a merge strategy that doesn’t operate on the index
> In Git 2.34, ort is now the default merge strategy, so you should notice faster merges with fewer bugs just by upgrading
> a rewrite allowed Git to implement a merge strategy that doesn’t operate on the index
Git is awesome but it would really be great if they sunsetted merge instead of putting lipstick on it.
Merge of two good commits will silently produce bad commit.
Merges are not repeatable and can't be reasoned about.
Merges get in a way of git bisect.
Merge makes simple "git log" not contain all the log.
Good luck cherry-picking a merge commit.
Merge makes a commit which has important changes not apply to any specific revision of git log.
It's an enormous foot gun and a single contributor may inadvertenly taint your repository for life.
Merge is a make-believe working set operation, like if you could pretend to add apple to mango and get a pomelo.
Merge is VCS analog of lead acetate. Something you have no reason to ever put in your mouth.
Merge of two good commits will silently produce bad commit.
Merges are not repeatable and can't be reasoned about.
Merges get in a way of git bisect.
Merge makes simple "git log" not contain all the log.
Good luck cherry-picking a merge commit.
Merge makes a commit which has important changes not apply to any specific revision of git log.
It's an enormous foot gun and a single contributor may inadvertenly taint your repository for life.
Merge is a make-believe working set operation, like if you could pretend to add apple to mango and get a pomelo.
Merge is VCS analog of lead acetate. Something you have no reason to ever put in your mouth.
How else could we take two diverging branches and mix them back into one?
"... the speedup is over 9000x" :)
[deleted]
It doesn't look like you can actually get that release yet.
It's there now: https://mirrors.edge.kernel.org/pub/software/scm/git/
git-2.34.0.tar.gz 15-Nov-2021 16:48 10M
git-2.34.0.tar.sign 15-Nov-2021 16:48 566
git-2.34.0.tar.xz 15-Nov-2021 16:48 6MIt would be fantastic if someone maintained a bugfix only fork of git 1.7.x.
? What's special about this version?
Works fine; I have no issues with old installations that still use it.
Is it the other side then? Do you have issues with 2.x releases?
btw rhel maintains a 1.8.x release.
btw rhel maintains a 1.8.x release.
> Do you have issues with 2.x releases?
Possibly; I've been hearing disturbing things about developments like different hash algorithms, and repositories being generated that older versions of git cannot read.
I'm nervous about creating a new central repository with newer git.
Possibly; I've been hearing disturbing things about developments like different hash algorithms, and repositories being generated that older versions of git cannot read.
I'm nervous about creating a new central repository with newer git.
If it works fine, why would you need a “bugfix fork”?
There has to be a fork so that the software being used is identified by a formal tag pointing at a specific commit, and this is backed by some sort of project commitment to put out a commit, if necessary.
It works fine in that I'm not running into issues and not missing features, not in the sense that I proved that it's free of defects; I don't require a fork that has any new commits. Surely, bugs have been found between that version and now, some of which can be meaningfully backported.
Programs can break simply because of all the other moving parts in the environments that are out there; it's a fact of life nowadays that software needs to have a maintenance plan to be viable. A compiler, library or kernel upgrade can expose some latent problem in a program.
It works fine in that I'm not running into issues and not missing features, not in the sense that I proved that it's free of defects; I don't require a fork that has any new commits. Surely, bugs have been found between that version and now, some of which can be meaningfully backported.
Programs can break simply because of all the other moving parts in the environments that are out there; it's a fact of life nowadays that software needs to have a maintenance plan to be viable. A compiler, library or kernel upgrade can expose some latent problem in a program.
> ort (short for “ostensibly recursive‘s twin”)
Continuing Git's famously clear naming I see!
Continuing Git's famously clear naming I see!
Can't mention git command naming without linking to the parody Git Man Page Generator[0].
[0]: https://git-man-page-generator.lokaltog.net/
[0]: https://git-man-page-generator.lokaltog.net/
> git-chase-submodule chases remote submodules using downstream shells, while remoting fitting patches to impale the given submodules.
> When COMMIT_ID_DEFINE_LOCAL_CACHE is not patched, the most recent counted grafts chased by submodules in a subtree, but that are not in other-bundle, may be chased by an active head ref. Using sub-recursed or pre-massaged flags only cautiously grips the tag, except when otherwise noted.
> When COMMIT_ID_DEFINE_LOCAL_CACHE is not patched, the most recent counted grafts chased by submodules in a subtree, but that are not in other-bundle, may be chased by an active head ref. Using sub-recursed or pre-massaged flags only cautiously grips the tag, except when otherwise noted.
They did it again. GitHub “stealing” the Git release announcement, contributing to the Git/GitHub confusion.
Comments from the last time: https://news.ycombinator.com/item?id=28207168
For those who don’t know: GitHub is owned by Microsoft, a historical (and current) adversary of all things not Microsoft, particularly Linux¹. Git, the tool, is not made by GitHub; GitHub is a web service which is using Git. Git, the tool, was originally made by the person who also originally made (and currently manages) Linux, and the purpose of Git was to manage the development of Linux itself.
1. https://en.wikipedia.org/wiki/Halloween_documents
Comments from the last time: https://news.ycombinator.com/item?id=28207168
For those who don’t know: GitHub is owned by Microsoft, a historical (and current) adversary of all things not Microsoft, particularly Linux¹. Git, the tool, is not made by GitHub; GitHub is a web service which is using Git. Git, the tool, was originally made by the person who also originally made (and currently manages) Linux, and the purpose of Git was to manage the development of Linux itself.
1. https://en.wikipedia.org/wiki/Halloween_documents
IMO GitHub posting these is fine since they're headlined "Highlights from Git 2.3x", but the last discussion incorrectly just said "Git 2.33", which is likely where the confusion came from. Hopefully this post will keep the distinction front-and-center.
If your name is GitHub and you announce a new release of Git, which your product GitHub uses extentively, and you do not explicitly and clearly mention that you do not have any role in making Git itself, then you are lying by omission; deliberately misinforming people who will naturally assume, given no information to the contrary, that GitHub are the makers of Git.
I think that most people are already under the misapprehension that Git and GitHub are the same thing. This style of deliberate confusion from GitHub must be seen to be by design. They are Microsoft; they do not get the benefit of the doubt.
I think that most people are already under the misapprehension that Git and GitHub are the same thing. This style of deliberate confusion from GitHub must be seen to be by design. They are Microsoft; they do not get the benefit of the doubt.
> that you do not have any role in making Git itself
GitHub employs at least three full time git contributors, including the author of this blog post.
(Disclaimer: I am a GitHub employee who works on git-adjacent things)
GitHub employs at least three full time git contributors, including the author of this blog post.
(Disclaimer: I am a GitHub employee who works on git-adjacent things)
> The open source Git project just released Git 2.34 with features and bug fixes from over 109 contributors, 29 of them new.
It's pretty clear they're talking about a separate project.
It's pretty clear they're talking about a separate project.
"you do not explicitly and clearly mention that you do not have any role in making Git itself, then you are lying by omission;"
Um - I thought Microsoft does have a big role - doesn't Johannes do the Git for Windows stuff and work at Microsoft, and Jeff work for github etc etc.
Um - I thought Microsoft does have a big role - doesn't Johannes do the Git for Windows stuff and work at Microsoft, and Jeff work for github etc etc.
Do you really think that most people that use git and GitHub are under the misapprehension that Git and GitHub are the same thing?
Really? This seems just very much an anti Microsoft view, especially with that final sentence.
Really? This seems just very much an anti Microsoft view, especially with that final sentence.
> Do you really think that most people that use git and GitHub are under the misapprehension that Git and GitHub are the same thing?
Yeah actually, a lot are. There are also a lot of GitHub users who don't know what git is at all.
I don't think this blog post is going to further misinform them though, they also don't read technical blog posts.
Yeah actually, a lot are. There are also a lot of GitHub users who don't know what git is at all.
I don't think this blog post is going to further misinform them though, they also don't read technical blog posts.
Actually my impression is that Microsoft is probably the largest corporate sponsor currently for git devs. They’re primarily focused on large mono repo support and ecosystem improvements that fit into their GitHub strategy. Overall, I’d characterize their Git involvement as positive.
Oh look, the obligatory "Micro$oft is evil" rhetoric based on decades old grievances and very little new evidence. So useful.
It's perfectly reasonable for a company, that uses the tool in its name, to have a blog post going over new features. I'm never amazed the distances people will go to distort a boring blog post into some issue or another based on splitting hairs.
It's perfectly reasonable for a company, that uses the tool in its name, to have a blog post going over new features. I'm never amazed the distances people will go to distort a boring blog post into some issue or another based on splitting hairs.
> […] based on decades old grievances and very little new evidence.
https://news.ycombinator.com/item?id=29288052
https://news.ycombinator.com/item?id=29288052
> […] based on decades old grievances and very little new evidence.
https://news.ycombinator.com/item?id=29251210
https://news.ycombinator.com/item?id=29251210
[deleted]
> very little new evidence
Off topic, but there was that dotnetfederation overreach shenanigans
Off topic, but there was that dotnetfederation overreach shenanigans
Or all the telemetry enabled in Windows and then making local accounts more difficult, although once you sign in to a Microsoft account to download some broken app from their app store then they start uploading all your app launches and activity anyhow.
And repeatedly trying to trick or force us to use Edge and probably a few others.
Are these decisions we don't agree with? Sure. Evil? No. At least not 1999 Microsoft evil. I can find just as many bad decisions from every other major tech company.
For the record, I actually did give Edge Chromium a spin and I was a bit impressed. The Deals feature creeps me out, but I can browse the reddit homepage without an ad blocker and it's actually a smooth experience.
For the record, I actually did give Edge Chromium a spin and I was a bit impressed. The Deals feature creeps me out, but I can browse the reddit homepage without an ad blocker and it's actually a smooth experience.
Not being cartoonishly evil should be a low bar to pass?
Edge is probably nice, forcing users to use it is not.
One should think they'd learned after being forced to advertise other browsers for a couple of years?
Edge is probably nice, forcing users to use it is not.
One should think they'd learned after being forced to advertise other browsers for a couple of years?
I don't like Microsoft, but you can't blame microsoft for the fact that apparently the git project never made a proper webpage. Obviously that means people will try to find information about git elsewhere.
Isn’t [1] the official git home page? I don’t follow your comment.
[1] https://git-scm.com/
[1] https://git-scm.com/
Said website was originally created for Git by GitHub employees, so presumably that site is also incredibly offensive to those who don't like the blog post.
[deleted]
Has git published their own release announcement like this for people to read?
It's in the repo: https://raw.githubusercontent.com/git/git/master/Documentati...
[edit: in fact it's linked from GitHub's blog post, at the bottom, and it's included in the announcement email: https://lore.kernel.org/git/[email protected]/]
[edit: in fact it's linked from GitHub's blog post, at the bottom, and it's included in the announcement email: https://lore.kernel.org/git/[email protected]/]
The official release announcement from the git project is also in the very first sentence of the very first paragraph of the github post.