Why Facebook doesn't use Git(graphite.dev)
graphite.dev
Why Facebook doesn't use Git
https://graphite.dev/blog/why-facebook-doesnt-use-git
21 comments
I'm not sure that this continues to be true, now that Git supports partial clones[1]. It's now a fully supported feature in Git to work with only a partial copy of the code and metadata. In my experience, this plus an fsmonitor[2] makes Git work fine on very, very large repos. This is how Microsoft scaled Git to work with their massive Windows monorepo[3]. (That particular article talks about the earlier work using Git VFS, but it eventually turned into these extensions that got upstreamed into Git, as documented in the Scalar repo[4].)
[1] https://github.blog/2020-12-21-get-up-to-speed-with-partial-...
[2] https://github.blog/2022-06-29-improve-git-monorepo-performa...
[3] https://arstechnica.com/information-technology/2017/02/micro...
[4] https://github.com/microsoft/scalar
[1] https://github.blog/2020-12-21-get-up-to-speed-with-partial-...
[2] https://github.blog/2022-06-29-improve-git-monorepo-performa...
[3] https://arstechnica.com/information-technology/2017/02/micro...
[4] https://github.com/microsoft/scalar
Yeah, these all seem like parallel or inspired developments of the same thing in git. But they're relatively recent and when you use this stuff git basically ceases to be a DVCS system (though the modern git + GitHub workflow mostly works like centralized version control already)
> GitHub workflow mostly works like centralized version
I mean the actual full DVCS capabilities are rarely used anywhere and aren't appropriate for many people at all. Yes, the Linux kernel uses it. But with the kernel there is no real authorative branch. Yes, there is Linus's and his team's branches but each distro has their own where they pull in patches etc. For a Red Hat user you could say their branches are authorative.
Where in enterprise would you want that? Usually, there is a a team that owns each module, and they are the definitive team and you use their artifacts.
The only case I can imagine a true DVCS workflow is a company I worked at had a core library that we customised and build apps around for clients. There were updates that we would choose to accept that fixed bugs that effected our client, but if they didn't we might not. There were fixes that we couldn't because they clashed with our modifications. This is a situation where true DVCS is applicable. (We didn't use it though, it was copy & paste files from sent emails).
I mean the actual full DVCS capabilities are rarely used anywhere and aren't appropriate for many people at all. Yes, the Linux kernel uses it. But with the kernel there is no real authorative branch. Yes, there is Linus's and his team's branches but each distro has their own where they pull in patches etc. For a Red Hat user you could say their branches are authorative.
Where in enterprise would you want that? Usually, there is a a team that owns each module, and they are the definitive team and you use their artifacts.
The only case I can imagine a true DVCS workflow is a company I worked at had a core library that we customised and build apps around for clients. There were updates that we would choose to accept that fixed bugs that effected our client, but if they didn't we might not. There were fixes that we couldn't because they clashed with our modifications. This is a situation where true DVCS is applicable. (We didn't use it though, it was copy & paste files from sent emails).
I prefer the term "derived" from Mercurial. It was Mercurial and slowly but steadily morphed into what's a custom VCS now.
Recently open sourced too https://github.com/facebook/sapling
[deleted]
Microsoft had issues git’s ability to scale. According to this[1] they found and submitted solutions to allow git to scale to their needs and worked with the git team. It sounds like Facebook wasn’t partnering with them to solve the problem? The article mentions Microsoft had repos with 6m files and says things now take seconds.
I’m not personally dealing with repos of this size, but it sounds like this is a problem of the past, thanks to Microsoft. I have a hard time believing Facebook would be a bigger beast than Windows.
[1] https://devblogs.microsoft.com/bharry/scaling-git-and-some-b...
I’m not personally dealing with repos of this size, but it sounds like this is a problem of the past, thanks to Microsoft. I have a hard time believing Facebook would be a bigger beast than Windows.
[1] https://devblogs.microsoft.com/bharry/scaling-git-and-some-b...
I suspect you might be surprised.
They keep almost everything in the same monorepo: every service, Facebook itself, the backend tools, many of the 'spinoff' experiences, the user data processing pipeline, the list goes on. All in one spot. Now, imagine 60-70k engineers, all wanting to have that sweet, sweet impact needed for good ratings and promotions. Add to that the fact that LOC and reviews are used as a metric for productivity.
Windows has to eventually compile into a collection of binaries delivered (at least in part) to their various platforms. Facebook/Meta does not have that kind of limitation; as long as each piece can be extracted for its own purposes and deployed, everyone is 'happy'.
Windows is estimated to be about 50M lines of code. Meta, in one post, simply suggests that it is "many times larger" than the Linux Kernel at 17M. [1]
Is it all needed? Probably not. But it certainly exists.
[1] https://engineering.fb.com/2014/01/07/core-infra/scaling-mer...
They keep almost everything in the same monorepo: every service, Facebook itself, the backend tools, many of the 'spinoff' experiences, the user data processing pipeline, the list goes on. All in one spot. Now, imagine 60-70k engineers, all wanting to have that sweet, sweet impact needed for good ratings and promotions. Add to that the fact that LOC and reviews are used as a metric for productivity.
Windows has to eventually compile into a collection of binaries delivered (at least in part) to their various platforms. Facebook/Meta does not have that kind of limitation; as long as each piece can be extracted for its own purposes and deployed, everyone is 'happy'.
Windows is estimated to be about 50M lines of code. Meta, in one post, simply suggests that it is "many times larger" than the Linux Kernel at 17M. [1]
Is it all needed? Probably not. But it certainly exists.
[1] https://engineering.fb.com/2014/01/07/core-infra/scaling-mer...
If you have infinite engineering resources to throw at a problem, seems best to consider multiple avenues. Sure, you could make patches to git, hope they get upstreamed, probably have to maintain an internal fork for years. Or design a system designed around the idea of huge repos.
I thought most of the reasons - not being welcomed to modifying git - were known already for someone who went looking, but seeing the quotes from ex facebookers is interesting.
I’d be more interested in why facebook ditched mercurial for their own. Once they left more and more notable projects left, though for more than that as reason. The mercurial project is still under development but it feels like a ghost town. It’s a shame as I much prefer mercurial to git but it’s difficult to choose as an option when the integrations and tooling is lacking, and new tool development is less likely to integrate with mercurial when it has so little resources behind it.
I’d be more interested in why facebook ditched mercurial for their own. Once they left more and more notable projects left, though for more than that as reason. The mercurial project is still under development but it feels like a ghost town. It’s a shame as I much prefer mercurial to git but it’s difficult to choose as an option when the integrations and tooling is lacking, and new tool development is less likely to integrate with mercurial when it has so little resources behind it.
Very cool, thanks for sharing. Despite never actually hearing about 'stacked prs' before, I've often encountered the issue and it makes me wonder why a solution like this isn't actually part of Git itself.
Some recent related discussions on this tip:
Ask HN: Can we do better than Git for version control? (December)
https://news.ycombinator.com/item?id=38590080
I kind of killed Mercurial at Mozilla (November)
https://news.ycombinator.com/item?id=38369433
Ask HN: Can we do better than Git for version control? (December)
https://news.ycombinator.com/item?id=38590080
I kind of killed Mercurial at Mozilla (November)
https://news.ycombinator.com/item?id=38369433
For some reason I thought part of the reason was ex-googlers at Facebook.
But it actually seems that it was somewhat the other way around. Mercurial is used at Google partially because Facebook was already using it.
Maybe someone has a more concrete timeline of the cross pollination of VCS between the two.
But it actually seems that it was somewhat the other way around. Mercurial is used at Google partially because Facebook was already using it.
Maybe someone has a more concrete timeline of the cross pollination of VCS between the two.
How much better was performance? Did people prefer or not prefer the new workflows?
The overall performance was massively better. Operations that would take minutes in Git in a large repo would take seconds (and eventually sub-seconds) in Facebook's Mercurial. Latency was a common complaint though. With Git in a small repo, commands can finish in milliseconds. With Facebook's Mercurial most commands took at least a few hundreds milliseconds (Python startup and module initialization alone could take that), which makes the CLI feel more sluggish than most people preferred.
Beyond performance, a lot of design effort was put into the workflows as well. The resulting user experience is significantly different from the open source Mercurial experience and in the long run the vast majority of people preferred the new workflows. Even some of the die-hard Git fans who complained about the transition. Part of the motivation of open sourcing Sapling was because ex-employees wanted to use it outside the company.
(Source: I was on the Facebook source control team for ten years.)
Beyond performance, a lot of design effort was put into the workflows as well. The resulting user experience is significantly different from the open source Mercurial experience and in the long run the vast majority of people preferred the new workflows. Even some of the die-hard Git fans who complained about the transition. Part of the motivation of open sourcing Sapling was because ex-employees wanted to use it outside the company.
(Source: I was on the Facebook source control team for ten years.)
Git and Mercurial came out the same year. Both were created due to changing licenses of BitKeeper and how this would affect Linux kernel development.
Ctrl+F 'Eden'
Yeah the author of this doesn't really know what's going on with regards to source control at Facebook/Meta.
Meta doesn't actually use Mercurial and hasn't for several years now. They basically wrote their own custom centralized version control that uses Mercurial-inspired client commands in order to adapt to the scale of their massive repos. Actual Mercurial scales just as bad as ever if the repo is too large because that breaks a fundamental assumption of distributed version control.
Massive repos simply don't work with the traditional model of "no master server" distributed version control. In DVCS, a client checkout is supposed to contain the entire repo metadata. That's why the git people told Facebook that they were doing it wrong.
Anyways, stacked diffs are great but they're orthogonal to why Facebook doesn't use git. They use their own thing for the same reason that Google uses piper and why some places still use Perforce. They use it because they want to have massive monorepos and DVCS just doesn't work well for that.