Pulling GitHub into the Linux kernel process(lwn.net)
lwn.net
Pulling GitHub into the Linux kernel process
https://lwn.net/SubscriberLink/860607/ef7cdf174ccbe54d/
54 comments
It is insane to me that kernel maintainers do not have build pipelines that automatically run for every PR. The computers should be checking formatting, basic compilation, testing, etc etc.
At the very least, before I opened the floodgates to github-style contributions, I would want to already have damn good CI in place.
While I firmly believe in "trickle up" economics", I still believe in trickle down productivity investment. It is important to increase the productivity of patch review (CI, etc., done by the experts manually today) before increasing the productivity of patch submission (done by everyone else).
While I firmly believe in "trickle up" economics", I still believe in trickle down productivity investment. It is important to increase the productivity of patch review (CI, etc., done by the experts manually today) before increasing the productivity of patch submission (done by everyone else).
I'd love to hear from Linus why they don't have that.
I know it's probably super hard to "unit test" some of this stuff but... does the Linux kernel have effectively 0 testing?
I know it's probably super hard to "unit test" some of this stuff but... does the Linux kernel have effectively 0 testing?
No, they have various testing platforms and harnesses, but they're very component oriented. For example people doing filesystem development will use https://github.com/kdave/xfstests for validation. Others will use different tools.
At Linux kernel level you can't effectively have "a CI". Unless you have massive amounts of funds and time to invest of course. I'm sure MS has some complex pre-release automation.
At Linux kernel level you can't effectively have "a CI". Unless you have massive amounts of funds and time to invest of course. I'm sure MS has some complex pre-release automation.
They actually do, it's just somewhat distributed.
https://01.org/lkp/documentation/0-day-test-service
https://linux.kernelci.org/job/
"kernel test robot" example email: https://lore.kernel.org/netdev/202102070253.xooT3PWR-lkp@int...
https://01.org/lkp/documentation/0-day-test-service
https://linux.kernelci.org/job/
"kernel test robot" example email: https://lore.kernel.org/netdev/202102070253.xooT3PWR-lkp@int...
I submitted some changes a couple years ago and something named "Hulk Robot" at Huawei found my change didn't build with a certain kernel config, and sent a patch to fix. I think there's a lot of distributed checking going on even if kernel.org doesn't have an official pipeline or unit testing.
All this research and not a single mention of Source Hut?
Source Hut is designed to be a web-based interface to Git, but which uses Git's built-in email-based patch workflow instead of some proprietary interface like Github.
It seems close to what they're looking for, and relatively easily hackable if missing anything they need.
https://sourcehut.org/
Good writeup on project philosophy:
https://drewdevault.com/2018/11/15/sr.ht-general-availabilit...
Source Hut is designed to be a web-based interface to Git, but which uses Git's built-in email-based patch workflow instead of some proprietary interface like Github.
It seems close to what they're looking for, and relatively easily hackable if missing anything they need.
https://sourcehut.org/
Good writeup on project philosophy:
https://drewdevault.com/2018/11/15/sr.ht-general-availabilit...
I think part of the appeal of git{hub,lab} is that the ones who won't want to figure out the email workflow are mostly there; there's no friction of having to sign up for yet another forge. sourcehut users are probably more likely to be ok following the kernel's normal workflow.
True, though there are two parts to that - 1) easy web-based patch submission workflow, and 2) everyone is already on github (and to a lesser extent gitlab).
#1 is easily solvable in SourceHut by hacking up mod that adds web-based workflow. It's designed for that modifying.
#2 is less easily solved, but may not need to be, since it also serves as a something of a quality filter. Apparently Linus gets all manner of inane of pull requests to his github repo, so moving it to Source Hut might cut down on that a little bit.
#1 is easily solvable in SourceHut by hacking up mod that adds web-based workflow. It's designed for that modifying.
#2 is less easily solved, but may not need to be, since it also serves as a something of a quality filter. Apparently Linus gets all manner of inane of pull requests to his github repo, so moving it to Source Hut might cut down on that a little bit.
Indeed, SourceHut would be an excellent choice for the kernel workflow.
Wait, why do I keep pronouncing it "Sir Hat"?
Drew originally called it Sir Hat, before changing it to Source Hut:
https://drewdevault.com/2018/11/15/sr.ht-general-availabilit...
https://news.ycombinator.com/item?id=18930554
https://news.ycombinator.com/item?id=19100986
https://drewdevault.com/2018/11/15/sr.ht-general-availabilit...
https://news.ycombinator.com/item?id=18930554
https://news.ycombinator.com/item?id=19100986
It was mentioned, but slightly misunderstood I think:
> Drew DeVault has been working on the reverse direction, from a mailing list to a project forge, as well.
> Drew DeVault has been working on the reverse direction, from a mailing list to a project forge, as well.
Ah thx, I was just looking for Source Hut, glad they're aware of it then.
I, for one, would be quite sad to see Linux on Github. An open source alternative, and preferably one without the political baggage and MS-backing of Github, would be a better fit.
Sadness trigger warning, brace yourself:
https://github.com/torvalds/linux
https://github.com/torvalds/linux
... and look at all the PRs that are not being merged. He should disable them.
All this tool is doing is making Github a fancy web frontend for sending a patch to the LKML. From that perspective, all this does is make it more convenient for those with a GitHub account to not have to learn a totally new workflow. I would think this would be easily portable to any other service (e.g. create a similar bot for Gitlab, Bitbucket, etc).
There's a separate discussion about whether the workflow more broadly should move out of the LKML. This might look like having patch sets posted to the LKML automatically ingested into Gerrit.
All of this seems pretty sane to modernize the workflow & make it a more familiar one for those working on the Linux kernel professionally. E-mailing patch sets around is decidedly a very old way to do code review that's not generally found in modern professional shops (cries of "this is for the worse" notwithstanding because that feels like a VIM vs emacs complaint).
There's a separate discussion about whether the workflow more broadly should move out of the LKML. This might look like having patch sets posted to the LKML automatically ingested into Gerrit.
All of this seems pretty sane to modernize the workflow & make it a more familiar one for those working on the Linux kernel professionally. E-mailing patch sets around is decidedly a very old way to do code review that's not generally found in modern professional shops (cries of "this is for the worse" notwithstanding because that feels like a VIM vs emacs complaint).
It’s not transitioning entirely to GitHub, but is “ a bot that can turn a GitHub pull request (PR) into a well-formed patch series to send to the proper reviewers and mailing lists”.
Also, Linux is already mirrored to GH https://GitHub.com/torvalds/Linux
Also, Linux is already mirrored to GH https://GitHub.com/torvalds/Linux
Not the parent, but I'd still much rather see such a bot for Gitlab or Gitea which are (mostly) open source and provide similar developer conveniences.
I suppose they made this change because Microsoft still hosts the Linux kernel for free on their platform and that's probably the main mirror people clone it from. Still, making proprietary patchwork for such an opinionated open source project seems a little silly to me.
I suppose they made this change because Microsoft still hosts the Linux kernel for free on their platform and that's probably the main mirror people clone it from. Still, making proprietary patchwork for such an opinionated open source project seems a little silly to me.
Ideally, wouldn't we want a bot for every such platform? One for github... but also for gitlab, and gitea, and bitbucket, and sourcehut, etc.
> that's probably the main mirror people clone it from.
I very much doubt this is true.
I very much doubt this is true.
[deleted]
Bad idea. Distributed development ensures the kernel is difficult to corrupt or destroy.
git was designed for this purpose.
If a dev cannot be bothered to learn the distributed process, they should not be submitting code to the kernel.
No corporation such as github, can be trusted with so much power.
+1 respect to Linus as usual.
git was designed for this purpose.
If a dev cannot be bothered to learn the distributed process, they should not be submitting code to the kernel.
No corporation such as github, can be trusted with so much power.
+1 respect to Linus as usual.
No one is talking about moving Linux to GitHub.
Is this what Radicle is designed for?
> Radicle is a decentralized code collaboration network built on open protocols . It enables developers to collaborate on code without relying on trusted intermediaries. Radicle was designed to provide similar functionality to centralized code collaboration platforms — or "forges" — while retaining Git’s peer-to-peer nature, building on what made distributed version control so powerful in the first place.
https://docs.radicle.xyz/docs/what-is-radicle.html
> Radicle is a decentralized code collaboration network built on open protocols . It enables developers to collaborate on code without relying on trusted intermediaries. Radicle was designed to provide similar functionality to centralized code collaboration platforms — or "forges" — while retaining Git’s peer-to-peer nature, building on what made distributed version control so powerful in the first place.
https://docs.radicle.xyz/docs/what-is-radicle.html
No, that looks like some kind of cryptocurrency nonsense. It's not clear that they even have a working product, let alone one which would be appropriate for the Linux kernel. (The discussion of a "decentralized" development model seems particularly strange, and completely at odds with how kernel development works.)
Hold your crypto bashing for a moment. Basic Radicle features do not rely on the blockchain at all. The ethereum extras are only for things like for name reservation and organisation management. (And for project development funding)
The product itself exists and is usable in a very basic form for a while now. (You can download it from https://radicle.xyz) It's not complete though and definitely not ready for the kernel's use case at the moment.
The product itself exists and is usable in a very basic form for a while now. (You can download it from https://radicle.xyz) It's not complete though and definitely not ready for the kernel's use case at the moment.
Kind of - it's closer to the mailing list workflow than GitHub. However the automation integration is not in place yet in any form, which this article is mostly about. Maybe in a couple of years.
The Linux Kernel development process is old-fashioned and there are lot of particular processes and conventions which make submitting a patch difficult. Also, if you use gmail with mutt to submit a patch, you have to change your email preferences to make gmail less secure[1]. It would be nice to have a friendlier way to contribute to the kernel, and a GitHub bridge seems like a good idea.
[1]: https://medium.com/@crhenr/submitting-your-first-patch-to-th...
[1]: https://medium.com/@crhenr/submitting-your-first-patch-to-th...
This is google imposed FUD to exercise control over users. They didn't need to design like that, but they want to make it hard to use things that are not google controlled.
IMAP/SMTP access with only your password is extremely insecure. Especially as most non-techie humans don’t use a password manager or even a strong password.
Yes, but google could easily support per application passwords. They just don't want to... in fact, they used to and removed it.
They could also invest in other ways to make existing workflows work securely instead of imposing their own ones.
They could also invest in other ways to make existing workflows work securely instead of imposing their own ones.
Did they remove app passwords? This page is still there https://support.google.com/accounts/answer/185833?hl=en
For me it originally worked flawlessly.
Then I tried to use it again and lots of the menus the support pages referenced were no longer present. It was an endless cycle of broken links and features being hidden deeper and deeper in menus.
If that haven't removed it (I thought they had), it kind of made the intention clear to me.
Then I tried to use it again and lots of the menus the support pages referenced were no longer present. It was an endless cycle of broken links and features being hidden deeper and deeper in menus.
If that haven't removed it (I thought they had), it kind of made the intention clear to me.
That's just Google doing their usual best at designing UIs and documenting things. It's like watching your 5-year old do their best "helping" doing housework. Cute, but not actually very helpful.
Sounds like oauth, which is an open standard.
OAuth is far from standard in email. Google is the only company I know of that uses it for email.
Per-application passwords are static passwords you generate for special applications. This way, the standard IMAP/SMTP logins will work, but a leaked password does not expose your entire Google account, only the email part. It also allows revocation of one application without re-entering your password on every device.
It works flawlessly for many services, and Google supports/supported it as well. Using it just flags/flagged your account as "insecure" for some reason. Probably because they want you to use their propietary OAuth flow where they have more control over your software.
Per-application passwords are static passwords you generate for special applications. This way, the standard IMAP/SMTP logins will work, but a leaked password does not expose your entire Google account, only the email part. It also allows revocation of one application without re-entering your password on every device.
It works flawlessly for many services, and Google supports/supported it as well. Using it just flags/flagged your account as "insecure" for some reason. Probably because they want you to use their propietary OAuth flow where they have more control over your software.
Microsoft, Yahoo (i.e. OATH) and others also use it, but it has limited adoption beyond the "big giants" of email.
https://docs.microsoft.com/en-us/exchange/client-developer/l...
In a way, server providers prefer it as (in theory) users' long term credentials don't end up stored locally in plain text. A revocable per application token is used instead, and they can trace that back to a specific application or developer registration.
That it trains users it's ok to enter passwords in bespoke web views using abnormal or no clear URL view is a big issue though in my opinion - I've never liked flows that encourage entering passwords into web browsers you didn't open yourself, let alone ones provided by third party software, which can capture anything happening within the app itself.
https://docs.microsoft.com/en-us/exchange/client-developer/l...
In a way, server providers prefer it as (in theory) users' long term credentials don't end up stored locally in plain text. A revocable per application token is used instead, and they can trace that back to a specific application or developer registration.
That it trains users it's ok to enter passwords in bespoke web views using abnormal or no clear URL view is a big issue though in my opinion - I've never liked flows that encourage entering passwords into web browsers you didn't open yourself, let alone ones provided by third party software, which can capture anything happening within the app itself.
Why would you use mutt instead of git send-email?
> Also, if you use gmail with mutt to submit a patch, you have to change your email preferences to make gmail less secure[1].
According to the article, this is because mutt uses IMAP, and needs to use password authentication (which might not even be accurate anymore). This has nothing to do with kernel development, beyond that some kernel developers like to use mutt as an email client.
According to the article, this is because mutt uses IMAP, and needs to use password authentication (which might not even be accurate anymore). This has nothing to do with kernel development, beyond that some kernel developers like to use mutt as an email client.
> less secure[1]
I see instructions around to make mutt use oauth, do they not work right? Like https://luxing.im/mutt-integration-with-gmail-using-oauth/
I see instructions around to make mutt use oauth, do they not work right? Like https://luxing.im/mutt-integration-with-gmail-using-oauth/
You could imagine the constant flow of drive-by spam PR's you would be getting.