Git from the Bottom Up (2022)(jwiegley.github.io)
jwiegley.github.io
Git from the Bottom Up (2022)
https://jwiegley.github.io/git-from-the-bottom-up/
30 comments
Git seems to have it all. It's hard to imagine what anyone else can compete with, how they can differentiate. There's such a variety of ways to work & rework git repos, so much sheer capability available to use as you will that it's hard to imagine what more one could want.
Indeed it feels like most competitors are competing by offering less. This is our way! This is easier! But you're not going to win the alpha geeks over with that path. Better needs to be bigger, needs to enable more, and we're just not seeing those offerings materialize; it's unclear how you would be better or bigger.
Having the core abstractions that work & are flexible enables bountiful innovation & improvements. Git not only started with sufficiently malleable core ideas, but it's had endless features, optimization, and tuning baked in over the years. It's almost too much to comprehend. Shout out to Taylor Blau writing on the Github Blog: the man has been doing an amazing job covering what's happening in with the Highlights From Git series, since September 2018. It's really given me a sense of how much work keeps being poured into git, has given me a view of how incredibly expansive the git ecosystem is. https://github.blog/2018-09-10-highlights-from-git-2-19/ https://github.blog/tag/git/
Indeed it feels like most competitors are competing by offering less. This is our way! This is easier! But you're not going to win the alpha geeks over with that path. Better needs to be bigger, needs to enable more, and we're just not seeing those offerings materialize; it's unclear how you would be better or bigger.
Having the core abstractions that work & are flexible enables bountiful innovation & improvements. Git not only started with sufficiently malleable core ideas, but it's had endless features, optimization, and tuning baked in over the years. It's almost too much to comprehend. Shout out to Taylor Blau writing on the Github Blog: the man has been doing an amazing job covering what's happening in with the Highlights From Git series, since September 2018. It's really given me a sense of how much work keeps being poured into git, has given me a view of how incredibly expansive the git ecosystem is. https://github.blog/2018-09-10-highlights-from-git-2-19/ https://github.blog/tag/git/
Because of GitHub. It won, it uses only git, and then network effects.
This is the biggest factor in my opinion.
I suspect most GitHub users didn't realise that it supported SVN until earlier this year though.
https://github.blog/2023-01-20-sunsetting-subversion-support...
I suspect most GitHub users didn't realise that it supported SVN until earlier this year though.
https://github.blog/2023-01-20-sunsetting-subversion-support...
Big names using it and GitHub. There are alternatives like mercurial, fossil, darcs, etc. but GitHub basically took over the social interaction side and majority of integrations. Nothing is better enough to displace it now. (Even if you think some specific feature in another VCS is cool, it's not cool enough to replace it where git's already deployed)
You'll still find different solutions in companies that really need it though, like Google.
You'll still find different solutions in companies that really need it though, like Google.
Probably a combination of all of the above. When git won the VCS war, the local repo aspect was very important compared to prior options. It's fast enough, generally, except when using large files like in game dev where people prefer perforce.
Git is very much "good enough" for many people. The number of foot guns and lack of a well trodden path for many centralized corporate devs is quite high, but many people are content with it.
Personally, I think if the only advantage is git formalizes a merkle tree and sha based text comparisons with a poor cli ui, it's not very good. But many people think this is sufficient.
Git is very much "good enough" for many people. The number of foot guns and lack of a well trodden path for many centralized corporate devs is quite high, but many people are content with it.
Personally, I think if the only advantage is git formalizes a merkle tree and sha based text comparisons with a poor cli ui, it's not very good. But many people think this is sufficient.
I think it's the latter. most people don't care about decentralization. Staging is more of a nuisance, there're some projects which try to eradicate it (https://github.com/martinvonz/jj). But I think git won the last decade because it is conceptually simple and elegant. You can reason about it in the case of any trouble. You can reimplement it if needed and be absolutely sure that reimplementation is fully compatible, because it is THAT simple.
I think that GitHub is a massive factor here. The emergence of distributed version control created the battle ground, but GitHub meant that the entire world of open source and sharing code online would be dominated by git. It was their execution (marketing as well as good tech) that was better than bitbucket and attracted more and more of the open source world to GitHub and therefore git.
There's a powerful network effect with choice of version control tool, because others need to collaborate with you (even if just to pull or browse your repository) so the tool that starts to dominate will likely monopolize. This isn't the same with browsers.
There's a powerful network effect with choice of version control tool, because others need to collaborate with you (even if just to pull or browse your repository) so the tool that starts to dominate will likely monopolize. This isn't the same with browsers.
The current git-saturated market certainly makes it quite challenging for other options to take off. That said jj[0] is super nifty, and it's git-compatible backing store means you can transparently use jj while the rest of your team uses git. This kind of model might give other VCS tools a fighting chance at iteration.
[0]:https://github.com/martinvonz/jj
[0]:https://github.com/martinvonz/jj
The thread discusses the main points. I also think that the performance is a big deal. It's fast enough to open up new workflows which didn't exist before this.
Techs often maintain their position because of their spread which causes a self-sustaining environment . In my region of the world C# was very popular in the early 00’s and most companies who used it then still use it today. As a result the local job-market is pretty C# heavy, with around 30-40% of developer jobs involving working with dotnet. I’m not sure any newer organisations have gone that way, Python/Go/TypeScript has seen a huge increase, but it’s not like C# won’t be around for another few decades (especially if it continues being good). I think Git maintains its monopoly in a similar fashion, but on a much larger scale where basically the entire “western” world uses it. With places like GitHub that monopoly is only strengthen as other mention.
How it got there is a different story. I’m old enough to have been around the Danish CS scene back when computer clusters were just an experimental thing at universities and when CVS arrived around the millennium. While it was mind blowing for its time, it was also terrible to work with. Similarly subversion wasn’t very nice (for many people) and it’s likely because the centralised model just isn’t very work friendly for a lot of organisations. Then came Git, which wasn’t the first decentralised version control, I believe Microsoft came first with their hellish team foundation which proved that distributed systems aren’t good by default. Anyway, at the time Git was such a breath of fresh air in how great it was to use. It was both easy and productive and it held such an increase in quality of life that it was a no brainier to transition to git for most organisations. At least the ones which didn’t have expensive team foundation or subversion contracts.
Which birthed the circle or dominance that might never get broken. I doubt any of the old players will in any case. If they were capable of building great version control they would’ve done so from the start, and since Gits only major problem is its multitude of options it’s hard to imagine that Subcersion would ever cut 90% of their features. Team Foundation mostly trundles along because some organisations still haven’t given up on their licenses (which is the reason a lot of Microsoft things exists), but with Microsoft owning GitHub it’s not likely that they want to do anything other than to keep it good enough to keep that revenue.
How it got there is a different story. I’m old enough to have been around the Danish CS scene back when computer clusters were just an experimental thing at universities and when CVS arrived around the millennium. While it was mind blowing for its time, it was also terrible to work with. Similarly subversion wasn’t very nice (for many people) and it’s likely because the centralised model just isn’t very work friendly for a lot of organisations. Then came Git, which wasn’t the first decentralised version control, I believe Microsoft came first with their hellish team foundation which proved that distributed systems aren’t good by default. Anyway, at the time Git was such a breath of fresh air in how great it was to use. It was both easy and productive and it held such an increase in quality of life that it was a no brainier to transition to git for most organisations. At least the ones which didn’t have expensive team foundation or subversion contracts.
Which birthed the circle or dominance that might never get broken. I doubt any of the old players will in any case. If they were capable of building great version control they would’ve done so from the start, and since Gits only major problem is its multitude of options it’s hard to imagine that Subcersion would ever cut 90% of their features. Team Foundation mostly trundles along because some organisations still haven’t given up on their licenses (which is the reason a lot of Microsoft things exists), but with Microsoft owning GitHub it’s not likely that they want to do anything other than to keep it good enough to keep that revenue.
In my opinion Git won because it was fast. It is fast because it is decentralized. It is fast because it was done by serious C hackers (there I said it). It may have a problematic UX/UI for many, but it is good enough. But it is fast.
How fast is Git? Do you remember or have you tried other VCS-es older than Git? Try getting the first working copy, try looking at changes for a file through all its history, try making a new branch, try tagging. Git is fast.
Because it is fast it was selected by many and the rest is network effects. Now you can do something just as fast or maybe even faster, but everyone else is on git. You may do something with better UX/UI, but this rarely is enough. You have to do something much better and you will not be orders of magnitude faster than git, but that is now a basic requirement.
How fast is Git? Do you remember or have you tried other VCS-es older than Git? Try getting the first working copy, try looking at changes for a file through all its history, try making a new branch, try tagging. Git is fast.
Because it is fast it was selected by many and the rest is network effects. Now you can do something just as fast or maybe even faster, but everyone else is on git. You may do something with better UX/UI, but this rarely is enough. You have to do something much better and you will not be orders of magnitude faster than git, but that is now a basic requirement.
>Does git win because of the decentralized, everyone-has-a-local-repo aspect? Or the staging area, which I understand other VCSes don't have? Or just speed or reliability? Or just being in the right place at the right time?
Your questions are actually covering 2 different categories:
(1) Why did (past tense) git win?
(2) Why does (current tense) git have a monopoly of usage now?
For (1), the various theories for git winning mindshare to create an insurmountable lead include technical and social differences:
- Git being faster than Mercurial. It had "cheaper" branches than Mercurial. And some blamed it on Mercurial being built with python instead of Git's C.
- index/staging area
- the Github free tier being more generous than Bitbucket(Mercurial)
- intangibles such as being created by Linus and high-profile usage by the Linux kernel contributors
Google Trends shows that Git+Github almost immediately outpaced Mercurial+BitBucket from 2008:
https://trends.google.com/trends/explore?date=all&geo=US&q=g...
https://trends.google.com/trends/explore?date=all&geo=US&q=g...
For (2) today, you mostly have inertia. Most developers are not going to bother to research and re-evaluate the VCS landscape and make a deliberate choice on any technical merits. Git is already too massively popular so just go with what everybody else is already using and just move on to something else. Even if another DVCS has some technical superior aspects (e.g. Fossil?), it doesn't matter because git is already too entrenched in the ecosystem.
An example of the network effects of (2) is Python's creator Guido van Rossum suggesting the move from Mercurial to git/Github:
2009 choose Mercurial/hg.python.org: https://mail.python.org/pipermail/python-dev/2009-March/0879...
2014 migrate to git/Github: https://mail.python.org/pipermail/python-dev/2014-November/1...
His 2014 suggestion to switch to git isn't based on technical features. It's about using what's the most popular to reduce friction.
Your questions are actually covering 2 different categories:
(1) Why did (past tense) git win?
(2) Why does (current tense) git have a monopoly of usage now?
For (1), the various theories for git winning mindshare to create an insurmountable lead include technical and social differences:
- Git being faster than Mercurial. It had "cheaper" branches than Mercurial. And some blamed it on Mercurial being built with python instead of Git's C.
- index/staging area
- the Github free tier being more generous than Bitbucket(Mercurial)
- intangibles such as being created by Linus and high-profile usage by the Linux kernel contributors
Google Trends shows that Git+Github almost immediately outpaced Mercurial+BitBucket from 2008:
https://trends.google.com/trends/explore?date=all&geo=US&q=g...
https://trends.google.com/trends/explore?date=all&geo=US&q=g...
For (2) today, you mostly have inertia. Most developers are not going to bother to research and re-evaluate the VCS landscape and make a deliberate choice on any technical merits. Git is already too massively popular so just go with what everybody else is already using and just move on to something else. Even if another DVCS has some technical superior aspects (e.g. Fossil?), it doesn't matter because git is already too entrenched in the ecosystem.
An example of the network effects of (2) is Python's creator Guido van Rossum suggesting the move from Mercurial to git/Github:
2009 choose Mercurial/hg.python.org: https://mail.python.org/pipermail/python-dev/2009-March/0879...
2014 migrate to git/Github: https://mail.python.org/pipermail/python-dev/2014-November/1...
His 2014 suggestion to switch to git isn't based on technical features. It's about using what's the most popular to reduce friction.
One more point I'd add to (2): given its massive inertia / network effect, the tooling and the resources are leagues ahead of everything else. I'm using Darcs for a few personal projects and while the core ideas are great, the tooling is just worse. From the ways to customize the diff utilities to use, to integrations with text editors (the vc-darcs module for Emacs is pretty barebones, especially compared to Magit, but even compared to the basic vc-git).
Darcs is a name that always makes me a little nostalgic, it was such an obvious idea of approaching a VC system. Unfortunately when I used it 10+ years ago we did hit the "merge of doom" problem too often.
I was sorry to see it go away, but mercurial took over for a while, and then later I switched to git because everybody else had done so.
I was sorry to see it go away, but mercurial took over for a while, and then later I switched to git because everybody else had done so.
> His 2014 suggestion to switch to git isn't based on technical features.
Likewise for the 2009 mail.
Likewise for the 2009 mail.
For the same reason HTTP has a near-monopoly on websites. There is a network effect with protocols and file formats. The success of github cemented these formats of git as the winners. Even so, why aren't other VCS at least somewhat popular? Because they refused to invest into supporting the git protocol and file formats.
I don’t know.
But being a monopoly doesn’t mean others can’t use other clients. I was using a Subversion repository in university but really I was just using Git and git-svn (probably). One can do the same today.
So, besides the obvious drawback of having to know two VCS, why does it seem that not many people are doing this? Because I don’t hear that much about it.
But being a monopoly doesn’t mean others can’t use other clients. I was using a Subversion repository in university but really I was just using Git and git-svn (probably). One can do the same today.
So, besides the obvious drawback of having to know two VCS, why does it seem that not many people are doing this? Because I don’t hear that much about it.
right place at the right time and performance level, then GitHub drove network effects / faddish popularity.
> Does git win because of the decentralized, everyone-has-a-local-repo aspect?
of course not, hg and bzr (and arch and monotone) had that, both before git existed.
> Does git win because of the decentralized, everyone-has-a-local-repo aspect?
of course not, hg and bzr (and arch and monotone) had that, both before git existed.
I do love this and it's great for understanding how git works because it's a really cool design. But I'm not convinced this is necessary or sufficient for learning how to use or be effective with git.
The way git stores commits---as blobs and hashes etc---is a really good abstraction. That means you just don't need to know this. You can think of a commit as being a complete copy of your working directory that is identifiable by its hash. That's it.
You can then think of branches and tags as labels for those commits, mutable and immutable, respectively. This gets you pretty far in effective use of git.
But the details are cool. Just don't go teaching any of this in a beginner developer course on git.
The way git stores commits---as blobs and hashes etc---is a really good abstraction. That means you just don't need to know this. You can think of a commit as being a complete copy of your working directory that is identifiable by its hash. That's it.
You can then think of branches and tags as labels for those commits, mutable and immutable, respectively. This gets you pretty far in effective use of git.
But the details are cool. Just don't go teaching any of this in a beginner developer course on git.
But the details are cool. Just don't go teaching any of this in a beginner developer course on git.
I would think that mentioning the big-picture concepts at the outset, and revealing the details as the need arises is an effective pedagogical model.
On the other hand, the minestrone of git terminology which the user is forced to learn and get comfortable with a priori (and moreover which has been used ambiguously in blog posts and whatnot) [1] make the learning curve much steeper than it should be.
Everyone learns differently, that's for sure. I remember my experience learning git from scratch was particularly scrappy, until I sat down for a few hours and read the "Pro Git" book.
[1] staging and index
I would think that mentioning the big-picture concepts at the outset, and revealing the details as the need arises is an effective pedagogical model.
On the other hand, the minestrone of git terminology which the user is forced to learn and get comfortable with a priori (and moreover which has been used ambiguously in blog posts and whatnot) [1] make the learning curve much steeper than it should be.
Everyone learns differently, that's for sure. I remember my experience learning git from scratch was particularly scrappy, until I sat down for a few hours and read the "Pro Git" book.
[1] staging and index
Are there guides in this "From the Bottom Up"-style for other technologies?
Would be very helpful for Docker or Kubernetes, for example. The only one that comes to mind is From Nand to Tetris. https://www.nand2tetris.org/
Would be very helpful for Docker or Kubernetes, for example. The only one that comes to mind is From Nand to Tetris. https://www.nand2tetris.org/
not a guide, but I have a build your own KV store project. It is in TDD fashion with the tests. So, you start with simple functions, pass the tests, and the difficulty level goes up. When all the tests pass, you will have written a persistent key-value store.
https://github.com/avinassh/py-caskdb
https://github.com/avinassh/py-caskdb
These are more practical tutorials, but very much a "bottom up" style of learning: https://github.com/practical-tutorials/project-based-learnin...
Not listed there but I also found Ben Eater's 8 bit computer a fantastic learning tool.
Not listed there but I also found Ben Eater's 8 bit computer a fantastic learning tool.
I have not read the book yet, but one book i marked to read was this:
https://www.manning.com/books/build-an-orchestrator-in-go-fr...
Where you build a simple orchestrator from scratch.
Looks very promising. I was planning on writing something similar in the past, it's astonishing to me how little people who use git daily actually understand it. Interactive rebasing, for example, is a foreign concept to most, while in my opinion it's a central tool for using git. The fundamentals are also important, again I don't know how people use git daily without actually knowing for example what a commit is. (People have fuzzy concepts like it's a diff (bad), some sort of compressed snapshot (slightly better), but very few have the correct mental model of "a commit is a reference to N parents, a tree, plus some metadata, and the commit hash is simply the hash of all these combined".)
Looking forward to sharing this with colleagues. On one hand I am excited because I think this bottom-up approach is the correct way of understanding git, on the other hand I am afraid that they will think it's difficult to understand and just not bother. (Again I can't fathom how programmers, who you know, learned programming, which is quite a steep learning curve in itself for first time, shy away from learning git properly, a tool that's just as important to their daily work as whatever programming language they are using.)
Looking forward to sharing this with colleagues. On one hand I am excited because I think this bottom-up approach is the correct way of understanding git, on the other hand I am afraid that they will think it's difficult to understand and just not bother. (Again I can't fathom how programmers, who you know, learned programming, which is quite a steep learning curve in itself for first time, shy away from learning git properly, a tool that's just as important to their daily work as whatever programming language they are using.)
Interesting to see this pop up, especially after watching Jon Gjengset's stream yesterday on
Implementing (parts of) git from scratch in Rust <https://www.youtube.com/watch?v=u0VotuGzD_w>
I'd heard of John Wiegley from Emacs, and now here he is doing more great stuff. Love to see it!
[deleted]
[deleted]
Usually there are at least a few choices with different compromises. Browsers, OSes, languages, editors, all have multiple actually popular choices.
Does git win because of the decentralized, everyone-has-a-local-repo aspect? Or the staging area, which I understand other VCSes don't have? Or just speed or reliability? Or just being in the right place at the right time?