Rust in the NetBSD Kernel, and other odd decisions(bentsukun.ch)
bentsukun.ch
Rust in the NetBSD Kernel, and other odd decisions
https://bentsukun.ch/posts/netbsd-rust-kernel/
33 コメント
And this is good because?
Seriously, modern devs have lost their minds if they think that this state of affairs is okay.
For a good while this process was just straight up not possible for C and C++, until someone put in a lot of effort to recreate the bootstrap process for reasons of reproducible builds (and it's still a long and complicated process I think only a few people have done). For decades people were just building from source with compilers that had extremely long and undocumented bootstrap chains (they may have documentation for how to bootstrap from a different C or C++ compiler, but there wasn't one that would start from scratch).
Go need a smaller chainloading process from C/C++ too. But the good thing it's there are prebuilt self-contained binary ports from every OS to almost any OS so you can bootstrap it with very little.
O really? Please.. can I bootstrap it here?
CYGWIN_NT-5.2 1.3.22(0.78/3/2)
CYGWIN_NT-5.2 1.3.22(0.78/3/2)
There are some Go backports to XP.
https://github.com/BieHDC/go-backports
https://github.com/BieHDC/go-backports
I laughed reading the comment above yours. I also laughed reading yours, you are so right on. Next, I am expecting someone telling that there is a javascript on github to automate that build process...
This is not what I was expecting computer science to become, 30 years ago...
This is not what I was expecting computer science to become, 30 years ago...
And that JS project needs only 1Gb of node.js modules to run.
Aren't computers fun? /s
> There are many architectures that NetBSD supports where Rust is not available
Are there many architectures that NetBSD supports where GCC is not available?
The rustc_codegen_gcc project allows using the rustc compiler to target any architecture that GCC supports. I think it is pretty far along.
https://github.com/rust-lang/rustc_codegen_gcc
> getting a Rust compiler running in the first place is hard
> bootstrap relies on a binary package
> the compiler would also have to be part of the base system
The gccrs project aims to make Rust a fully supported language in GCC. It is fully self-hosting and can be built from source. Once accepted, it will be one of the languages you get when you build GCC.
https://github.com/Rust-GCC/gccrs
When gccrs is ready, it will address all the above objections.
Linux faces the same issues. Today, Rust is optional in Linux and can be skipped on any platform not supported in LLVM and rustc. For Rust to become mandatory in Linux, gccrs is going to be required (for some platforms).
> the release cycles of Rust are not compatible with the NetBSD ones
I do not fully grasp the objection here. Obviously NetBSD would not be forced to use any Rust features they cannot compile. And using gccrs for the NetBSD kernel would not prevent them from bundling LLVM and rustc as well if they want.
Anyway, I am not advocating the use of Rust in NetBSD. I am merely pointing out that solutions to the issues raised are being worked on.
Are there many architectures that NetBSD supports where GCC is not available?
The rustc_codegen_gcc project allows using the rustc compiler to target any architecture that GCC supports. I think it is pretty far along.
https://github.com/rust-lang/rustc_codegen_gcc
> getting a Rust compiler running in the first place is hard
> bootstrap relies on a binary package
> the compiler would also have to be part of the base system
The gccrs project aims to make Rust a fully supported language in GCC. It is fully self-hosting and can be built from source. Once accepted, it will be one of the languages you get when you build GCC.
https://github.com/Rust-GCC/gccrs
When gccrs is ready, it will address all the above objections.
Linux faces the same issues. Today, Rust is optional in Linux and can be skipped on any platform not supported in LLVM and rustc. For Rust to become mandatory in Linux, gccrs is going to be required (for some platforms).
> the release cycles of Rust are not compatible with the NetBSD ones
I do not fully grasp the objection here. Obviously NetBSD would not be forced to use any Rust features they cannot compile. And using gccrs for the NetBSD kernel would not prevent them from bundling LLVM and rustc as well if they want.
Anyway, I am not advocating the use of Rust in NetBSD. I am merely pointing out that solutions to the issues raised are being worked on.
I just noticed that rustc_codegen_gcc has been accepted into the mainline Rust compiler, or at least the nightly builds.
https://github.com/rust-lang/rust/pull/151156
This means that you will be able to use either GCC or LLVM to build your Rust code with rustc (the main Rust compiler). That means you can compile your Rust for any platform that GCC supports.
This only addresses the first point in the blog post (architecture support) but it is an interesting development.
https://github.com/rust-lang/rust/pull/151156
This means that you will be able to use either GCC or LLVM to build your Rust code with rustc (the main Rust compiler). That means you can compile your Rust for any platform that GCC supports.
This only addresses the first point in the blog post (architecture support) but it is an interesting development.
We need memory safety but Rust is not the answer.
It has no formal spec, changes too fast, depends on third party libraries that change faster than I can breath, and is controlled by a foundation that is controlled by big tech corps.
What could go wrong?
It has no formal spec, changes too fast, depends on third party libraries that change faster than I can breath, and is controlled by a foundation that is controlled by big tech corps.
What could go wrong?
It has a partial spec.
https://ferrous-systems.com/blog/ferrocene-25-11-0/
Lets not forget not having a formal spec apparently wasn't an issue for C, which only got standardized in 1989, and even K&R C only specified a subset of its behaviours, which is a reason why there is so much UB, and implementation specific behaviours with YOLO C, as the Fil-C author likes to call it.
https://ferrous-systems.com/blog/ferrocene-25-11-0/
Lets not forget not having a formal spec apparently wasn't an issue for C, which only got standardized in 1989, and even K&R C only specified a subset of its behaviours, which is a reason why there is so much UB, and implementation specific behaviours with YOLO C, as the Fil-C author likes to call it.
> Lets not forget not having a formal spec apparently wasn't an issue for C
C emerged in the 1970s when there weren't many other options, and it was rapidly propagated because US antitrust laws forced AT&T to inexpensively license Unix (and by extension C) to universities and research institutions.
That was half a century ago.
I am a supporter of Rust adoption, but the line of reasoning that "C didn't need X, so Rust shouldn't either" is fundamentally flawed.
The time difference between today and C's public emergence is the same as the time between C's emergence and the 1920s. The field of computing is radically different today and any language that wishes to see wider adoption now must play by a completely different set of rules than those of 50 years ago.
C emerged in the 1970s when there weren't many other options, and it was rapidly propagated because US antitrust laws forced AT&T to inexpensively license Unix (and by extension C) to universities and research institutions.
That was half a century ago.
I am a supporter of Rust adoption, but the line of reasoning that "C didn't need X, so Rust shouldn't either" is fundamentally flawed.
The time difference between today and C's public emergence is the same as the time between C's emergence and the 1920s. The field of computing is radically different today and any language that wishes to see wider adoption now must play by a completely different set of rules than those of 50 years ago.
There have been options since JOVIAL in 1958, as there have been OS written in high level language a decade before C came to be invented.
C has to thank its adoption by being freely available with UNIX, that AT&T was forbidden to take commercial advantage of, for many years, the only reference implementation was whatever cc does.
C has to thank its adoption by being freely available with UNIX, that AT&T was forbidden to take commercial advantage of, for many years, the only reference implementation was whatever cc does.
> There have been options since JOVIAL in 1958
Yeah, pedantically speaking there were other options but not for the average person. Languages like PL/I and JOVIAL were locked away in corporate institutions behind exorbitant licensing fees. Discoverability was limited as there was no WWW and there was no concept of open source. If you were an established company, then sure you could pay IBM $20000 for PL/I and be done with it.
But if you were a founder building something new, then you were stuck with what was available and familiar, and if you went to a university in the USA after the mid 1970s and learned anything about computers then Unix and C is what you knew. It just so happens that such companies were the ones who created much of the modern computing landscape, so C came along for the ride.
But anyway,
> C has to thank its adoption by being freely available with UNIX, that AT&T was forbidden to take commercial advantage of, for many years, the only reference implementation was whatever cc does.
This is exactly my point. The path that C took to wide adoption cannot be replicated by a modern language, so any drawing parallels along the lines of "Well C didn't need a formal standard" are not useful.
It's like thinking that you can be an internationally successful superstar by greasing up your hair and playing rockabilly music just because Elvis did it in the 1950s.
The world has changed and Rust will need to play a different game than C did.
Yeah, pedantically speaking there were other options but not for the average person. Languages like PL/I and JOVIAL were locked away in corporate institutions behind exorbitant licensing fees. Discoverability was limited as there was no WWW and there was no concept of open source. If you were an established company, then sure you could pay IBM $20000 for PL/I and be done with it.
But if you were a founder building something new, then you were stuck with what was available and familiar, and if you went to a university in the USA after the mid 1970s and learned anything about computers then Unix and C is what you knew. It just so happens that such companies were the ones who created much of the modern computing landscape, so C came along for the ride.
But anyway,
> C has to thank its adoption by being freely available with UNIX, that AT&T was forbidden to take commercial advantage of, for many years, the only reference implementation was whatever cc does.
This is exactly my point. The path that C took to wide adoption cannot be replicated by a modern language, so any drawing parallels along the lines of "Well C didn't need a formal standard" are not useful.
It's like thinking that you can be an internationally successful superstar by greasing up your hair and playing rockabilly music just because Elvis did it in the 1950s.
The world has changed and Rust will need to play a different game than C did.
There were no average persons writing operating systems in those decades.
Rust already did, it takes safety seriously, guess why Multics got an higher security score than UNIX?
Rust already did, it takes safety seriously, guess why Multics got an higher security score than UNIX?
> changes too fast
The core language has been static for ages, and breaking changes are handled by the edition system so you can use a modern compiler to build code on old syntax. Since the 1.0 release ten years ago there have been four editions.
It's absolutely not changing too fast
> depends on third party libraries that change faster than I can breath
No it doesn't. The standard library is already sufficient for a lot of work; and there is an unhosted version with a "core" version of that standard library which has zero dependencies.
Modern Rust, Java, Python, TypeScript etc. developers choose to use a lot of third party libraries; but that's only because the tooling and ecosystem are both good enough to facilitate that. Nothing about the language forces it.
The core language has been static for ages, and breaking changes are handled by the edition system so you can use a modern compiler to build code on old syntax. Since the 1.0 release ten years ago there have been four editions.
It's absolutely not changing too fast
> depends on third party libraries that change faster than I can breath
No it doesn't. The standard library is already sufficient for a lot of work; and there is an unhosted version with a "core" version of that standard library which has zero dependencies.
Modern Rust, Java, Python, TypeScript etc. developers choose to use a lot of third party libraries; but that's only because the tooling and ecosystem are both good enough to facilitate that. Nothing about the language forces it.
the proof is right there in all the discussion about rust in the linux kernel too.
Not sure if you mean proof of changing too fast …or not fast enough?
Linux has a wishlist of features they want for kernel development, and Rust has been working towards adding them.
Here's the paradox: Rust is very careful about compatibility and stability, the stable releases are changing slowly. But the Rust for Linux project wants to use the newly prototyped features right away, so they depend on not-yet-released features from unstable nightly versions of Rust.
Linux has a wishlist of features they want for kernel development, and Rust has been working towards adding them.
Here's the paradox: Rust is very careful about compatibility and stability, the stable releases are changing slowly. But the Rust for Linux project wants to use the newly prototyped features right away, so they depend on not-yet-released features from unstable nightly versions of Rust.
> the Rust for Linux project wants to use the newly prototyped features right away
> they depend on not-yet-released features from unstable nightly versions of Rust
This is not true. Since kernel 6.11 they have specified a minimum version that is already stable. The strategy for the Rust kernel is to use the version of Rust that ships with Debian Stable. That is very far from using "the newly prototyped features right away".
https://rust-for-linux.com/rust-version-policy
Of course, the kernel continues to inform Rust evolution. But you do not need an unstable version of Rust to compile Linux.
This is not true. Since kernel 6.11 they have specified a minimum version that is already stable. The strategy for the Rust kernel is to use the version of Rust that ships with Debian Stable. That is very far from using "the newly prototyped features right away".
https://rust-for-linux.com/rust-version-policy
Of course, the kernel continues to inform Rust evolution. But you do not need an unstable version of Rust to compile Linux.
That not true is not true.
Rust For Linux has a hard dependency on unfinished Rust features that were not released as stable in the Rust version that Debian has.
However, every build of rustc also needs to be able to build its own standard library and a future version of itself. To do this, rustc recognizes a special private env var that exposes compiler-internal language extensions and not-yet-stabilized features meant for the compiler itself, and not for end users.
Rust for Linux relies on using this private env var to bypass feature stability checks and trick the "stable" compiler from Debian into allowing use of compiler-private and experimental features that were suppressed to stay disabled.
RfL uses an old Rust binary from Debian, but still depends on experimental features that haven't officially shipped yet in any Rust version.
Rust For Linux has a hard dependency on unfinished Rust features that were not released as stable in the Rust version that Debian has.
However, every build of rustc also needs to be able to build its own standard library and a future version of itself. To do this, rustc recognizes a special private env var that exposes compiler-internal language extensions and not-yet-stabilized features meant for the compiler itself, and not for end users.
Rust for Linux relies on using this private env var to bypass feature stability checks and trick the "stable" compiler from Debian into allowing use of compiler-private and experimental features that were suppressed to stay disabled.
RfL uses an old Rust binary from Debian, but still depends on experimental features that haven't officially shipped yet in any Rust version.
this is the right way to do it. just explain why you think rust is not a good match for your project right now. no politics and no hate for the language.
if it can't be build with Ansi C or perhaps old style C++ it is not self contained i think. depending a Language on some fast moving binarys is in my eye a bad decission. Excuse my english. im not a native speaker.
Why are C and C++ special in this regard? I could just as well claim it's not self-contained unless you can start from source and a snippet of inspectable machine code, like Guix can (which still has fairly significant gaps in what you could call 'completely self contained', if you wanted to poke holes in it).
C is special because it's possible to relatively easily implement a good-enough C compiler for bootstrapping if needed. It would be under 15,000 lines of code in most languages.
C++ certainly isn't that special. It's a pain to implement, but so is Rust.
C++ certainly isn't that special. It's a pain to implement, but so is Rust.
And how did you got the binary for that language?
This argument can be pushed all the way down to build your own CPUs from sand, get to build the first hexdump monitor with electric switches, followed by writting the first Assembler in hexdump, and only then the first compiler for one of thoses languages in Assembly, and finally you get C.
This argument can be pushed all the way down to build your own CPUs from sand, get to build the first hexdump monitor with electric switches, followed by writting the first Assembler in hexdump, and only then the first compiler for one of thoses languages in Assembly, and finally you get C.
Yes, that's how bootstrapping works. C is still easier to bootstrap than Rust, because it's simpler to implement and requires fewer steps. Of course, you trade off other features for that simplicity.
Bootstrapping Rust would involve the same steps, only at some point you've got to write a Rust compiler in assembly, or write a C compiler in assembly that can compile mrustc, which probably means writing another C compiler in C so you can easily add all the needed features.
Bootstrapping Rust would involve the same steps, only at some point you've got to write a Rust compiler in assembly, or write a C compiler in assembly that can compile mrustc, which probably means writing another C compiler in C so you can easily add all the needed features.
Well, if we are going hardcore nothing prevents to write a Ferrocene compliant Rust compiler in Assembly.
Or even better, lets follow the hype, use bootstrap strategies from the 1960's, and use WebAssembly instead, like many bytecode based compilers on those days, having written a macro Assembler instead, thus mapping the bytecodes into macros for the specific home made CPU.
Or even better, lets follow the hype, use bootstrap strategies from the 1960's, and use WebAssembly instead, like many bytecode based compilers on those days, having written a macro Assembler instead, thus mapping the bytecodes into macros for the specific home made CPU.
There are self-hosted Forths
Subleq and Eforth can self-compile, yes.
Being base.dec a subleq EForth image and eforth.fth the source code, to create a new one:
Being base.dec a subleq EForth image and eforth.fth the source code, to create a new one:
./subleq base.dec < tuned_eforth.fth > new.dec
And Uxn partially. You can run the compiler as a UXN rom (Drimflim.rom).
This does not paint the full picture. Rust can be bootstrapped with mrustc, which is written in C++
https://github.com/thepowersgang/mrustc
Now, mrustc supports only Rust 1.74. To build Rust 1.92, you need almost 20 builds. But this can be done from source
Guix has written about bootstrapping Rust from source (they care a lot about this). Here is how it looked like in 2018
https://guix.gnu.org/nb-NO/blog/2018/bootstrapping-rust/