Rust is the future, C is the new Assembly: Josh Triplett (Intel)(hub.packtpub.com)
hub.packtpub.com
Rust is the future, C is the new Assembly: Josh Triplett (Intel)
https://hub.packtpub.com/rust-is-the-future-of-systems-programming-c-is-the-new-assembly-intel-principal-engineer-josh-triplett/
241 comments
https://news.ycombinator.com/item?id=20790681
If you invent the next image format, compression scheme, encryption scheme, etc., and you write a Go library for it, then it can be used by Go programmers. If you write a C/C++ library, it can be used by any programmer (perhaps with a wrapper) on billions of devices.
Now, we add Rust to the list. You can write that library in Rust and it can be used by anyone, too. That's because there is no GC or other extraneous code, and because you can make structures with specifically-defined layout in memory.
Making a list of options go from two to three is a big deal. Especially when, a lot of the time, C++ ends up looking like C when you need to interface it with something else. So Rust is kind of like going from one to two optons.
I'm ignoring other good languages, I know, but those are often just not chosen for a variety of reasons. Rust seems to be breaking out of that trap because of a good community, a few nice features, the fact that it's new, and a bit of luck.
Now, we add Rust to the list. You can write that library in Rust and it can be used by anyone, too. That's because there is no GC or other extraneous code, and because you can make structures with specifically-defined layout in memory.
Making a list of options go from two to three is a big deal. Especially when, a lot of the time, C++ ends up looking like C when you need to interface it with something else. So Rust is kind of like going from one to two optons.
I'm ignoring other good languages, I know, but those are often just not chosen for a variety of reasons. Rust seems to be breaking out of that trap because of a good community, a few nice features, the fact that it's new, and a bit of luck.
I have written a proof of concept for making a lib in Rust and then use JNR (a wrapper of the abominable JNI) to call and be called from that lib. I was surprised to see how much easy it is to write an ABI in rust to be used in the JVM. I am sure this lib will even have an easier time integrating with Python/Ruby comparing to Java.
With that in mind, I think for my future native ABI development, I would be seriously considering building in Rust instead of C/C++.
Fun fact: you can expose a Go library in the same way using the c-shared build mode.
But how many want to link the Go runtime into their program? Maybe they just don't want it there, or maybe they already have other runtimes linked in their program and don't want one more, or maybe they have runtime version conflicts, etc.
Also, such interfaces generally require more copying around of structures to match the format the language is expecting. With rust, it can use plain C structs directly without translation.
Also, such interfaces generally require more copying around of structures to match the format the language is expecting. With rust, it can use plain C structs directly without translation.
In Windows I just implement in COM and it can also be used by any language on the platform, without resorting to plain old C.
Yes. But then you have to use COM :P
All major Windows APIs are based on COM since Vista, as the .NET Longhorn ideas got rebooted as COM libraries.
With Windows 8, it got a new facelift (WinRT, UAP, UWP) and the large majority of Win32 APIs are as they were in Windows 7.
Anyone doing modern Windows development, has to use COM in some form.
With Windows 8, it got a new facelift (WinRT, UAP, UWP) and the large majority of Win32 APIs are as they were in Windows 7.
Anyone doing modern Windows development, has to use COM in some form.
Incidentally I recently added better support for classic COM interop to the Rust WinRT projection, this allows it to work with Windows.UI.Composition for example (albeit still awkwardly since we still don't support WinRT inheritance yet): https://github.com/contextfree/rust-winui-experiments
Also COM ABI is really small, easily portable to other OSes. I’ve recently ported to Linux and implemented .NET core interop. The C++ runtime is just a couple pages of code.
https://github.com/Const-me/ComLightInterop
https://github.com/Const-me/ComLightInterop
I knew this one guy who still did a little VBA when he thought no one was looking. . .
> Now, we add Rust to the list. You can write that library in Rust and it can be used by anyone, too. That's because there is no GC or other extraneous code, and because you can make structures with specifically-defined layout in memory.
Genuine question: Can you link a Rust binary with a C/C++ binary?
Genuine question: Can you link a Rust binary with a C/C++ binary?
notably firefox has a few modules implemented in rust.
You can link a Rust library to a C/C++ binary and you can link a C/C++ library to a Rust binary. https://doc.rust-lang.org/nomicon/ffi.html
Notably you need to use the C ABI to call things from C++ as Rust FFI doesn't support C++ directly.
Notably you need to use the C ABI to call things from C++ as Rust FFI doesn't support C++ directly.
Absolutely.
I use C a lot on my personal projects, and I have enjoyed learning Rust and playing with it to potentially replace my use of C. For the most part Rust has everything I want, but interfacing with C libs and headers can be so unergonomic. I wish there was a good alternative to bindgen that didn't shoot your build times up. Offline binding generation works, but then you have to manually deal with versioning. I don't really know what the solution is, but it's currently one of the pain points I have with Rust
That's one of the items on the list to get better C parity. Binding to C libraries should work as easily as C's #include.
Wouldn't the simplest solution be to just make bindgen emit "from <file name>@<hash>" into the /* automatically generated by rust-bindgen */ comment?
Then when it loads a .h, hash it, and if the hash is unchanged don't write anything. This way, the cargo/rust incremental compilation does the rest of the work. Minimal change to how it works now, yet the compile times go way down.
If you are willing to live dangerously, you could always use modification times instead of hashes to avoid having to even read the .h files on compile, but frankly that sounds like it'd just make enough heisenbugs that people would start doing clean compiles just in case.
Then when it loads a .h, hash it, and if the hash is unchanged don't write anything. This way, the cargo/rust incremental compilation does the rest of the work. Minimal change to how it works now, yet the compile times go way down.
If you are willing to live dangerously, you could always use modification times instead of hashes to avoid having to even read the .h files on compile, but frankly that sounds like it'd just make enough heisenbugs that people would start doing clean compiles just in case.
Caching would certainly help improve the performance of bindgen. But I'd also like to improve the quality of the bindings, avoid the need for a build script to invoke bindgen, and reduce the amount of boilerplate needed to make safe wrappers.
Hard to predict what the "Future" is.
Rust fills a niche. It's not the answer to everything.
The learning curve is steep, and for many applications, the benefits are nonexistent.
Although you can use Rust (or anything, even Bash if that's your thing) to write web applications, Ruby, Python, PHP and Javascript are far more productive and this is not going to change tomorrow.
Rust's borrow checker doesn't add anything besides complexity here.
Go is also a nice balance between being simple to use and powerful as a system language (unless "has no GC" is how you define a system language).
Swift has most of the best parts from the language above.
Crystal is a pleasure to write and runs fast.
Zig may eventually get more traction.
Something like Rust, but accessible, readable, that compiles quickly, is stable and has a natural way to do async operations could quickly make Rust obsolete.
Rust fills a niche. It's not the answer to everything.
The learning curve is steep, and for many applications, the benefits are nonexistent.
Although you can use Rust (or anything, even Bash if that's your thing) to write web applications, Ruby, Python, PHP and Javascript are far more productive and this is not going to change tomorrow.
Rust's borrow checker doesn't add anything besides complexity here.
Go is also a nice balance between being simple to use and powerful as a system language (unless "has no GC" is how you define a system language).
Swift has most of the best parts from the language above.
Crystal is a pleasure to write and runs fast.
Zig may eventually get more traction.
Something like Rust, but accessible, readable, that compiles quickly, is stable and has a natural way to do async operations could quickly make Rust obsolete.
I often see "steep learning curve" cited as a negative about Rust. But I think it is far less of a learning curve than it is to be a reliable C and C++ programmer.
I am not sure how much Rust you have done yet, but the borrow checker is a tremendous confidence booster. Especially refactoring code. I have never had anything where I can feel at ease in making big changes to an existing code base. As all things, I don't think you notice it once you learn how to write safe Rust code.
Go is certainly a good balance. One of its best things is the rich standard library. I wish Rust took more of that approach.
One thing about Rust that is maybe overlooked is that it is, like C/C++, truly "full-stack". By that I mean you can go from low level embedded (microcontroller with no MMU) all the way up to web server, etc. I don't think it is really a "niche" language--at least it has a very horizontal use case.
Having been at this for a super long time, I have seen the popularity languages come and go from ASM, Perl, Java, Python, .Net, etc. I can assure you nothing is gonna "quickly" make another language obsolete. New languages are incredibly hard to get off the ground and be relevant for a long time.
I am not sure how much Rust you have done yet, but the borrow checker is a tremendous confidence booster. Especially refactoring code. I have never had anything where I can feel at ease in making big changes to an existing code base. As all things, I don't think you notice it once you learn how to write safe Rust code.
Go is certainly a good balance. One of its best things is the rich standard library. I wish Rust took more of that approach.
One thing about Rust that is maybe overlooked is that it is, like C/C++, truly "full-stack". By that I mean you can go from low level embedded (microcontroller with no MMU) all the way up to web server, etc. I don't think it is really a "niche" language--at least it has a very horizontal use case.
Having been at this for a super long time, I have seen the popularity languages come and go from ASM, Perl, Java, Python, .Net, etc. I can assure you nothing is gonna "quickly" make another language obsolete. New languages are incredibly hard to get off the ground and be relevant for a long time.
This reflects the feelings I had when I tried Rust for a few side projects. I read everything I could and did all the tutorials, but I still felt there was some sort of invisible glas wall between me and enlightenment. I tried very hard to program in structures I thought I had to use, mainly out of experience from other languages (OOP...)
At some point the friction just grew too hard and I said to myself: "Ok you stupid Rust, I will do it your way, I just want this to work" and then it made click and everything fell into place – builder patterns, traits, composition over inheritence – I realized that clinging to all the patterns I had learned was idiotic, because Rust has given me maybe even more powerful ways of expressing myself.
If it wasn't for the friendly Compiler yelling at me, I doubt I would've gotten so far.
At some point the friction just grew too hard and I said to myself: "Ok you stupid Rust, I will do it your way, I just want this to work" and then it made click and everything fell into place – builder patterns, traits, composition over inheritence – I realized that clinging to all the patterns I had learned was idiotic, because Rust has given me maybe even more powerful ways of expressing myself.
If it wasn't for the friendly Compiler yelling at me, I doubt I would've gotten so far.
>I often see "steep learning curve" cited as a negative about Rust. But I think it is far less of a learning curve than it is to be a reliable C and C++ programmer.
Man of us are already competent in C and/or C++, so to make a move there has to be a big value prop if it's a large investment. I'm not saying there's not, but most of us aren't deciding to pick up C, C++ or Rust from scratch.
Man of us are already competent in C and/or C++, so to make a move there has to be a big value prop if it's a large investment. I'm not saying there's not, but most of us aren't deciding to pick up C, C++ or Rust from scratch.
I think Rust adoption has pointed out that the idea of safe C/C++ developers are a myth. It's an appeal to authority to be sure, but if Intel and Microsoft (and others, but those two have talked about it) are finding that their security-conscious C/C++ developers are still putting memory safety bugs into production - there's an argument that the idea of "competent" use of C/C++ for safety critical is possible (in general) needs to be brought into question.
On a personal note, learning Rust has made me a much better C++ developer. I'm more conscious of various degrees of aliasing and multiple ownership, to the point where much of the C++ I've written of late is inspired by the borrow checker's conventions. And since what I do is usually executed concurrently, I've seen a concrete decrease in bug count and severity without cost in performance. Which means the time investment in learning Rust has paid off, in my opinion.
On a personal note, learning Rust has made me a much better C++ developer. I'm more conscious of various degrees of aliasing and multiple ownership, to the point where much of the C++ I've written of late is inspired by the borrow checker's conventions. And since what I do is usually executed concurrently, I've seen a concrete decrease in bug count and severity without cost in performance. Which means the time investment in learning Rust has paid off, in my opinion.
While I agree, and I do like Rust, C++ still has a winning hand for those of us on Java/.NET.
Just to give a possible example, given everything that Microsoft has been doing about Rust.
On what concerns Windows systems programming, Rust must be able to integrate with .NET, allow mixed mode debugging, authoring of COM/UWP, use of the GUI frameworks, just as easy as VC++ allows us to.
So I am eager to get to know about whatever Microsoft might announce in this regard.
Just to give a possible example, given everything that Microsoft has been doing about Rust.
On what concerns Windows systems programming, Rust must be able to integrate with .NET, allow mixed mode debugging, authoring of COM/UWP, use of the GUI frameworks, just as easy as VC++ allows us to.
So I am eager to get to know about whatever Microsoft might announce in this regard.
You can integrate with COM already through winapi [1] (and author your own COM objects too with com-impl[2]). GUI stuff is still a long way from being useful, but a huge chunk of code has nothing to do with UIs. And Rust can integrate with anything that can call C.
w.r.t debugging, I haven't had much trouble mixing with C++ but I haven't tried to do anything through C#.
[1] https://github.com/retep998/winapi-rs
[2] https://github.com/Connicpu/com-impl
w.r.t debugging, I haven't had much trouble mixing with C++ but I haven't tried to do anything through C#.
[1] https://github.com/retep998/winapi-rs
[2] https://github.com/Connicpu/com-impl
You can also make fire with two stones.
That is the difference between using C++/CX, C++/WinRT or the existing Rust libraries.
So how do you integrate Rust with VS debugging tools?
That is the difference between using C++/CX, C++/WinRT or the existing Rust libraries.
So how do you integrate Rust with VS debugging tools?
>You can also make fire with two stones.
I'd love to see that trick.
I'd love to see that trick.
Here you go,
https://youtu.be/W_HJ1czydiI
https://youtu.be/W_HJ1czydiI
Yeah totally agree for the experienced C/C++ developer, it is like unlearning all the things that make you good. Totally agree. Good developer is like building a series of great mental models. It is hard to start over. Like trying to do linked lists in Rust...
But that being said, when you see that the majority of vulnerabilities are essentially memory lifetime issues (https://msrc-blog.microsoft.com/2019/07/22/why-rust-for-safe...), you realize that we are all human. So something like Rust could help--especially new developers.
But that being said, when you see that the majority of vulnerabilities are essentially memory lifetime issues (https://msrc-blog.microsoft.com/2019/07/22/why-rust-for-safe...), you realize that we are all human. So something like Rust could help--especially new developers.
> I am not sure how much Rust you have done yet, but the borrow checker is a tremendous confidence booster
I recently spent an entire week diving into Rust for a personal project. While it certainly had a learning curve that slowed me down (particularly regarding the borrow checker), I realized that I only had 1(!) runtime error all week...in a language I was unfamiliar with. That gave me a huge amount of confidence in memory safety.
I recently spent an entire week diving into Rust for a personal project. While it certainly had a learning curve that slowed me down (particularly regarding the borrow checker), I realized that I only had 1(!) runtime error all week...in a language I was unfamiliar with. That gave me a huge amount of confidence in memory safety.
> Triplett broadly defines systems programming as “anything that isn’t an app.” It includes things like BIOS, firmware, boot loaders, operating systems kernels, embedded and similar types of low-level code, virtual machine implementations. Triplett also counts a web browser as a system software as it is more than “just an app,” they are actually “platforms for websites and web apps,” he says.
Even though I love golang and use it daily for my job, I'd suggest that golang is not a good fit for Triplett's definition for systems programming.
Even though I love golang and use it daily for my job, I'd suggest that golang is not a good fit for Triplett's definition for systems programming.
Ah that quote actually clarified for me why browsers feel different than "normal" applications: in their role as a platform for other applications, they have a lot in common with straight-up VMs.
But then Triplett's definition for systems programming is not some ISO standard. In lot of companies I have seen they call their large internal applications development as systems programing.
Also compilers, runtimes, GCs, databases, cloud schedulers (Kubernetes etc) are also not 'apps' could fit in systems programing definition.
Also compilers, runtimes, GCs, databases, cloud schedulers (Kubernetes etc) are also not 'apps' could fit in systems programing definition.
[deleted]
Go and Crystal are slower than Java which has a much better ecosystem.
> Crystal is a pleasure to write and runs fast.
Crystal has no support for parallel execution, which is an important feature nowadays, at least, when one compares it to Go and so on.
Crystal has no support for parallel execution, which is an important feature nowadays, at least, when one compares it to Go and so on.
Kaveren(2)
Rust really does seem to be taking off in the realm of native languages. Personally I would've preferred something simpler, such as nim or Kotlin Native.
However I'm excited for the future of system-level programming: we were stuck with C/C++ for a long, long time and maybe that's finally changing thanks to Rust!
However I'm excited for the future of system-level programming: we were stuck with C/C++ for a long, long time and maybe that's finally changing thanks to Rust!
Both Nim and Kotlin Native have a runtime in the form of a GC/automatic memory management, so you'd still need another language "beneath" them to get the job done.
In the system programming world, "simple" means having the program do what you tell it and no more, so that there are no surprises when you come to run it.
Rust has a very complicated compiler, and a moderately complicated syntax, but the semantics of the language are amongst the simplest out there - far simpler than C (purely because of the UB), Nim or Kotlin Native IMO, and that really helps when writing low-level code, or even just when contributing to an unfamiliar code-base.
In the system programming world, "simple" means having the program do what you tell it and no more, so that there are no surprises when you come to run it.
Rust has a very complicated compiler, and a moderately complicated syntax, but the semantics of the language are amongst the simplest out there - far simpler than C (purely because of the UB), Nim or Kotlin Native IMO, and that really helps when writing low-level code, or even just when contributing to an unfamiliar code-base.
That's definitely the most critical property of Rust. Not long before 1.0, they decided to drop the runtime and green-threads support entirely. If they'd kept them, I think Rust would have been just one more runtime-based language in a sea of such languages. But by ditching them, Rust became a language with no manual memory management and no garbage collection, exactly what systems programming needs.
I don't know how to put this politely.
There's nothing simple about Rust.
The compiler is complex and slow.
The borrower semantics is anything but simple.
It isn't uncommon for the syntax to look arcane.
The standard library is shallow often making developers have to resort to third party libraries for trivial things.
Rust improved upon languages like Cyclone but it's a far cry from being the next C/C++ when it comes to potential market adoption.
There's nothing simple about Rust.
The compiler is complex and slow.
The borrower semantics is anything but simple.
It isn't uncommon for the syntax to look arcane.
The standard library is shallow often making developers have to resort to third party libraries for trivial things.
Rust improved upon languages like Cyclone but it's a far cry from being the next C/C++ when it comes to potential market adoption.
Rust definitely has a lot to improve on. I'm a daily Rust user, and the compile times are still absolutely terrible, and I have to agree it isn't a simple language for any sane definition of simple.
That said: almost all your points apply to C++.
There's nothing simple about C++.
it can be incredibly complex and slow (though faster than Rust, but that's not much of an achievement).
The rules around what constitutes valid (non-UB) code is anything but simple.
It isn't uncommon for the syntax to look arcane (IMO much more so than Rust, although obviously YMMV).
The standard library is extremely shallow, making developers have to resort to third party libraries for trivial things, except there's no package manager or standard build system, so good luck with that.
And yet, here we are, with C++ dominating the market. So I don't think any of those reasons are dealbreakers at all.
I believe Rust has the potential for market adoption as a serious C++ competitor. It still has a long road to get there, but C++ didn't reach this point in a day either. We see more and more adoption in big companies of Rust. Microsoft recently expressed interest in using rust in critical components to reduce security bugs for instance. More and more companies are experimenting with Rust, and some are even using it in production.
Besides, there's another victory to Rust: more and more languages are looking into integrating borrow semantics in their own language, like swift[0]. This is proof that Rust's core ideas work. If those trickle down to other languages, the "complexity" of borrows will probably become just one of the other things you just have to get used to to learn programming.
[0]: https://github.com/apple/swift/blob/master/docs/OwnershipMan...
That said: almost all your points apply to C++.
There's nothing simple about C++.
it can be incredibly complex and slow (though faster than Rust, but that's not much of an achievement).
The rules around what constitutes valid (non-UB) code is anything but simple.
It isn't uncommon for the syntax to look arcane (IMO much more so than Rust, although obviously YMMV).
The standard library is extremely shallow, making developers have to resort to third party libraries for trivial things, except there's no package manager or standard build system, so good luck with that.
And yet, here we are, with C++ dominating the market. So I don't think any of those reasons are dealbreakers at all.
I believe Rust has the potential for market adoption as a serious C++ competitor. It still has a long road to get there, but C++ didn't reach this point in a day either. We see more and more adoption in big companies of Rust. Microsoft recently expressed interest in using rust in critical components to reduce security bugs for instance. More and more companies are experimenting with Rust, and some are even using it in production.
Besides, there's another victory to Rust: more and more languages are looking into integrating borrow semantics in their own language, like swift[0]. This is proof that Rust's core ideas work. If those trickle down to other languages, the "complexity" of borrows will probably become just one of the other things you just have to get used to to learn programming.
[0]: https://github.com/apple/swift/blob/master/docs/OwnershipMan...
> That said: almost all your points apply to C++.
And nobody here said C++ was a good language. Actually, is it even considered a systems programming language?
And nobody here said C++ was a good language. Actually, is it even considered a systems programming language?
C++ is absolutely a systems language and widely used for that purpose. Indeed, I would argue it is the primary use case for which it is well-suited these days. In 2019 there are few if any reasonable alternatives for writing software like high-performance database engines, so C++ essentially wins by default.
> Actually, is it even considered a systems programming language?
Yes, absolutely. It's really not even under any sort of doubt at all..?
Yes, absolutely. It's really not even under any sort of doubt at all..?
The borrower semantics is anything but simple.
I keep seeing people saying this when discussing Rust in relation to C++. It's not untrue, but I would counter by saying that people who were writing C++ without having a full understanding of ownership and lifetime semantics were writing buggy code. Rust just makes understanding those semantics required to get your code to compile.
I keep seeing people saying this when discussing Rust in relation to C++. It's not untrue, but I would counter by saying that people who were writing C++ without having a full understanding of ownership and lifetime semantics were writing buggy code. Rust just makes understanding those semantics required to get your code to compile.
I have a somewhat related question -
I took CS 101 ~ 301 in college, and have worked briefly in C++ here and there during my career. I like to think I have a working understanding of pointers and references.
Despite that, whenever I go into a C++ codebase (recently, the Chromium codebase) I get _completely lost_ trying to figure out what the owner and lifetime of an object is, whether it's appropriate to pass by value, reference, or clone. I have absolutely no clue the impact of these decisions aside from a vague sense that pass-by-reference is more memory efficient and cloning is safest (When do we care? Is it something you can just pick one and forget about until performance matters?).
How do you start to pick up an intuition for manually working with memory this way? Is there a set of references that can take you past the beginner level? I find the C++ reference documentation completely opaque, and most search hits tend to be too domain specific to be useful.
Is this something that Rust would help with? Is their model easier to understand when each case is correct? Is it something that comes with language exposure? Or tooling?
I took CS 101 ~ 301 in college, and have worked briefly in C++ here and there during my career. I like to think I have a working understanding of pointers and references.
Despite that, whenever I go into a C++ codebase (recently, the Chromium codebase) I get _completely lost_ trying to figure out what the owner and lifetime of an object is, whether it's appropriate to pass by value, reference, or clone. I have absolutely no clue the impact of these decisions aside from a vague sense that pass-by-reference is more memory efficient and cloning is safest (When do we care? Is it something you can just pick one and forget about until performance matters?).
How do you start to pick up an intuition for manually working with memory this way? Is there a set of references that can take you past the beginner level? I find the C++ reference documentation completely opaque, and most search hits tend to be too domain specific to be useful.
Is this something that Rust would help with? Is their model easier to understand when each case is correct? Is it something that comes with language exposure? Or tooling?
Rust is a huge help with exactly this problem. The compiler will clearly tell you exactly where you've made a mistake if you give it code that violates the ownership rules, like trying to keep a reference past its valid lifetime.
It took me a little bit to get used to, but learning this definitely helped me learn to think about ownership and lifetimes better. I highly recommend you try learning some Rust.
It took me a little bit to get used to, but learning this definitely helped me learn to think about ownership and lifetimes better. I highly recommend you try learning some Rust.
I know C++ since Turbo C++ , have written C++ code at CERN, make use of static analysis tooling in C++, and yet it took me a couple of days to deal with Gtk-rs, trying to minimise the use of Rc<RefCell<>>.
While I did preserve to make it work, not everyone is willing to keep trying.
While I did preserve to make it work, not everyone is willing to keep trying.
so how did you manage to not keep track of object lifetimes before? serious question, rust just forces you to do what c++ expects you to do, threatening undefined behavior at runtime if you don't. anecdotally i can say about myself that learning rust made me a better python programmer, of all things.
Modern c++ makes lifetimes pretty implicit.
Default assumption: all data lives in a value type, or an STL container. The destructor/copy constructors will automatically deal with lifetimes for you.
Fallback #1: the remainder lives in a std::unique_ptr, or perhaps std::shared_ptr. The destructor/copy constructors will deal with lifetimes for you.
Fallback #2: Take a moment to reflect upon the mistakes that you've made that have led you here. This is your fault. Write a class with correct constructor, destructor, copy constructor, move constructor, and move/copy assignment operators. This is roughly the equivalent of resorting to unsafe in rust in a place that makes calls to other rust code. (as opposed to a syscall or a c function call or somewhere else that it's obviously required) If you're here, it's because you fucked up.
Since c++11 had become available, I've eliminated all use of new/delete from new code that I write. I've slowly refactored old code (by myself and others) to do the same. The only significant attention I've paid to lifetimes is when I'm using a c library or am writing c# where half my objects are IDisposable. (yes, really, I do more manual memory management in a garbage collected language)
The problem with c++ isn't that you have to worry about lifetimes, it's that you used to have to worry about object lifetimes and many codebases were written during that time. Sure, we could rewrite the world in rust and all of that would go away, or we could rewrite the world in modern c++ and it would also go away.
Default assumption: all data lives in a value type, or an STL container. The destructor/copy constructors will automatically deal with lifetimes for you.
Fallback #1: the remainder lives in a std::unique_ptr, or perhaps std::shared_ptr. The destructor/copy constructors will deal with lifetimes for you.
Fallback #2: Take a moment to reflect upon the mistakes that you've made that have led you here. This is your fault. Write a class with correct constructor, destructor, copy constructor, move constructor, and move/copy assignment operators. This is roughly the equivalent of resorting to unsafe in rust in a place that makes calls to other rust code. (as opposed to a syscall or a c function call or somewhere else that it's obviously required) If you're here, it's because you fucked up.
Since c++11 had become available, I've eliminated all use of new/delete from new code that I write. I've slowly refactored old code (by myself and others) to do the same. The only significant attention I've paid to lifetimes is when I'm using a c library or am writing c# where half my objects are IDisposable. (yes, really, I do more manual memory management in a garbage collected language)
The problem with c++ isn't that you have to worry about lifetimes, it's that you used to have to worry about object lifetimes and many codebases were written during that time. Sure, we could rewrite the world in rust and all of that would go away, or we could rewrite the world in modern c++ and it would also go away.
I think you missed the point of Rust. It's not about replacing new/delete with RAII (even though that's a good idea for sure).
How does modern C++ help you with tracking whether a pointer/reference outlives the lifetime of the pointee? Not at all.
By using the lifetime static analysis recently introduced in clang tidy and Visual C++ 2019.
It is still in the early stages and work is being done to make it a common feature across major compiler, while improving their capabilities.
It is still in the early stages and work is being done to make it a common feature across major compiler, while improving their capabilities.
RAII, STL data structures, smart pointers, warnings as errors, static analysis as errors, and refusing to write C in C++.
There are many other instances where ownership/lifetimes kick in in ways that would not be a problem in a different language.
Anywhere you have any kind of containment hiearchy, Rust starts becoming more invasive than other languages. In languages like Go, you can have complex graphs of objects referring to each other, and you can — often arguably safely — work with these structures without thinking about who owns what.
As an example of something that got me stuck, I recently had some code that populated a map of mutable buffers:
This is all probably entirely obvious to Rust experts, but not so to new developers. Swapping out the entire map with std::mem::replace() would never have occurred to me. Even if you understand borrowing in principle, you have think about a container means in terms of borrowing, and how stuff will move in and out of a container. And you have to design the way you interact with them accordingly.
In principle, I think this awareness is a good thing, and that the resultant code will be smarter and more optimal as a result, but at the same time, it doesn't make for such an ergonomic developer experience; so far it's been a drain on my productivity more than a gain. I like to say that Rust "scales down" towards the low levels, but does less well "scaling up"; you can't write high-level code without also thinking about the low levels, when even things like the size of your data type is always in your face.
My solution above may not even be the most optimal, idiomatic way of doing things. I'm sure someone will come along and point out that there's a trick involving using something other than RefCell or whatever that simplifies everything.
Not necessarily a criticism of Rust, just a data point in terms of complexity and learning curve.
Anywhere you have any kind of containment hiearchy, Rust starts becoming more invasive than other languages. In languages like Go, you can have complex graphs of objects referring to each other, and you can — often arguably safely — work with these structures without thinking about who owns what.
As an example of something that got me stuck, I recently had some code that populated a map of mutable buffers:
struct Builder {
buffers: BTreeMap<Term, RefCell<PostingBuffer>>,
}
for (term, buf) in &self.buffers {
// into_inner consumes buf, so it moves; fails with "Cannot move out of borrowed context"
let data = buf.into_inner().get_data();
w.write(term, data)?;
}
self.buffers = BTreeMap::new();
for (term, v) in std::mem::replace(&mut self.buffers, BTreeMap::new()) {
let data = v.into_inner().get_data();
w.write(term, data)?;
}
This is all probably entirely obvious to Rust experts, but not so to new developers. Swapping out the entire map with std::mem::replace() would never have occurred to me. Even if you understand borrowing in principle, you have think about a container means in terms of borrowing, and how stuff will move in and out of a container. And you have to design the way you interact with them accordingly.
In principle, I think this awareness is a good thing, and that the resultant code will be smarter and more optimal as a result, but at the same time, it doesn't make for such an ergonomic developer experience; so far it's been a drain on my productivity more than a gain. I like to say that Rust "scales down" towards the low levels, but does less well "scaling up"; you can't write high-level code without also thinking about the low levels, when even things like the size of your data type is always in your face.
My solution above may not even be the most optimal, idiomatic way of doing things. I'm sure someone will come along and point out that there's a trick involving using something other than RefCell or whatever that simplifies everything.
Not necessarily a criticism of Rust, just a data point in terms of complexity and learning curve.
Fully acknowledge your final paragraphs here, for sure. But in case you're curious...
I think that if you wrote
I think that if you wrote
for (term, buff) in self.buffers {
No, I tried that! That actually moves the "Cannot move" error to the "self.buffers" use itself.
The reason I can't just iterate by value is that I'd be consuming data that is already held by the map. The buffer is defined thus:
The reason I can't just iterate by value is that I'd be consuming data that is already held by the map. The buffer is defined thus:
use bitstream_io::{BigEndian, BitWriter};
struct PostingBuffer<W: io::Write> {
bw: BitWriter<W, BigEndian>,
// ...
}
impl PostingBuffer {
pub fn get_data(self) -> W {
self.bw.into_writer()
}
}
pub fn into_writer(self) -> W {
self.writer
}
Ah, ah, I see now. Yes, right.
This is exactly why I couched my last paragraph the way I did, heh.
This is exactly why I couched my last paragraph the way I did, heh.
I upvoted you for the first six lines, while feeling compelled to chime in that cargo is great, an example of Rust doing things right imho.
I'd rather have a thriving and single (looking at you Python) ecosystem of third-party packages, than a standards-driven pile-on of standard libraries, with all the footguns and deprecation that tends to come with it.
I'd rather have a thriving and single (looking at you Python) ecosystem of third-party packages, than a standards-driven pile-on of standard libraries, with all the footguns and deprecation that tends to come with it.
> The compiler is complex and slow.
It doesn't mean that the language is complex.
> The borrower semantics is anything but simple.
The borrow checker doesn't change the meaning of the program in any way. You can perfectly understand Rust code without any knowledge of the borrow checker (there is a working Rust compiler that doesn't even have a borrow checker), so it cannot make the semantics of the language more complicated.
> It isn't uncommon for the syntax to look arcane.
Again, I did call out the syntax has being moderately complicated. The great thing about syntax is that it's superficial. Syntax has to be pretty bad to continue being an issue beyond the initial stages of learning a language, and I don't think Rust's syntax is close to being that bad, it just has some unfamiliar elements to a lot of people.
> The standard library is shallow often making developers have to resort to third party libraries for trivial things.
This sits on the "Rust is simple" side.
> Rust improved upon languages like Cyclone but it's a far cry from being the next C/C++ when it comes to potential market adoption.
In terms of actual market adoption I'd agree. But you said potential market adoption, so I don't.
It doesn't mean that the language is complex.
> The borrower semantics is anything but simple.
The borrow checker doesn't change the meaning of the program in any way. You can perfectly understand Rust code without any knowledge of the borrow checker (there is a working Rust compiler that doesn't even have a borrow checker), so it cannot make the semantics of the language more complicated.
> It isn't uncommon for the syntax to look arcane.
Again, I did call out the syntax has being moderately complicated. The great thing about syntax is that it's superficial. Syntax has to be pretty bad to continue being an issue beyond the initial stages of learning a language, and I don't think Rust's syntax is close to being that bad, it just has some unfamiliar elements to a lot of people.
> The standard library is shallow often making developers have to resort to third party libraries for trivial things.
This sits on the "Rust is simple" side.
> Rust improved upon languages like Cyclone but it's a far cry from being the next C/C++ when it comes to potential market adoption.
In terms of actual market adoption I'd agree. But you said potential market adoption, so I don't.
You don't think the type system is part of language semantics?
The borrow/lifetime system never changes the behavior of any program, the only thing it can do is to disallow some programs from compiling. This is not how the rest of the type system works.
This means that it's possible to write a compiler that correctly compiles all correct rust programs without having a lifetime system at all. It also means that having the lifetime system in makes writing programs more complicated, but it makes reading them, and understanding what a program does, easier.
This means that it's possible to write a compiler that correctly compiles all correct rust programs without having a lifetime system at all. It also means that having the lifetime system in makes writing programs more complicated, but it makes reading them, and understanding what a program does, easier.
The type system is yes, just not the borrow checker or the type checker.
The type system is still relatively simple compared the type systems in Java/C#/C++/etc. Let's take Java's as an example because I think people regard it as being simple:
Building blocks of the type system: Rust: Types, Traits Java: Primitive types, classes, interfaces, (value types yet?)
Relationships between those building blocks: Rust: Implements (Type <=> Trait) Java: Extends (Class => Class | Interface => Interface), Implements (Class => Interface), Auto-boxes (Primitive/value type => Class)
Generics: Works mostly the same in both languages. In Java you have to deal with the side-effects of type-erasure from time to time, and there's also the `? extends Y` syntax, for which Rust has no counterpart. In Rust you can have a generic parameter which is a lifetime, for which Java has no counterpart. Again though, these lifetimes are only used by the borrow checker, unlike normal type parameters, they do not affect the generated code.
Java appears simple because it is built on OOP concepts that are very familiar to most people. However, those OOP concepts contain a huge amount of complexity (inheritance trees, constructors, abstract classes, virtual/final methods, monitors, etc) and Rust simply gets rid of all of those concepts entirely. That's why I think it's simple.
The type system is still relatively simple compared the type systems in Java/C#/C++/etc. Let's take Java's as an example because I think people regard it as being simple:
Building blocks of the type system: Rust: Types, Traits Java: Primitive types, classes, interfaces, (value types yet?)
Relationships between those building blocks: Rust: Implements (Type <=> Trait) Java: Extends (Class => Class | Interface => Interface), Implements (Class => Interface), Auto-boxes (Primitive/value type => Class)
Generics: Works mostly the same in both languages. In Java you have to deal with the side-effects of type-erasure from time to time, and there's also the `? extends Y` syntax, for which Rust has no counterpart. In Rust you can have a generic parameter which is a lifetime, for which Java has no counterpart. Again though, these lifetimes are only used by the borrow checker, unlike normal type parameters, they do not affect the generated code.
Java appears simple because it is built on OOP concepts that are very familiar to most people. However, those OOP concepts contain a huge amount of complexity (inheritance trees, constructors, abstract classes, virtual/final methods, monitors, etc) and Rust simply gets rid of all of those concepts entirely. That's why I think it's simple.
>Rust improved upon languages like Cyclone but it's a far cry from being the next C/C++
I was with you till this point. Rust is ugly, complex, but far batter that C++ in this regards.
* Functions instead of 666 constructors with obscure rules of which you should implement in pairs
* Hygienic macros instead of that ugly unsafe mess in C and C++ called macro and templates
* Move by default and Copy by calling a copy() function
* Everything is so explicit
I was with you till this point. Rust is ugly, complex, but far batter that C++ in this regards.
* Functions instead of 666 constructors with obscure rules of which you should implement in pairs
* Hygienic macros instead of that ugly unsafe mess in C and C++ called macro and templates
* Move by default and Copy by calling a copy() function
* Everything is so explicit
For a C programmer, Rust's standard library is gigantic. And the cargo story is one of Rust's strongest points, how easy it is to handle dependencies.
But the best parts of Rust are its move semantics, enum datatypes and traits, everything that makes programs a joy to express.
But the best parts of Rust are its move semantics, enum datatypes and traits, everything that makes programs a joy to express.
I agreed with everything you said except:
>but it's a far cry from being the next C/C++ when it comes to potential market adoption.
Because C++ is very similarly complex/arcane etc.
>but it's a far cry from being the next C/C++ when it comes to potential market adoption.
Because C++ is very similarly complex/arcane etc.
The semantics of Rust aren't simple at all. I've used a huge number of languages, both professionally and for personal projects, and Rust is the first one I'm too stupid to understand.
("Modern" JavaScript comes close, but that's not because of semantics, it's more because everything is incredibly overengineered...
("Modern" JavaScript comes close, but that's not because of semantics, it's more because everything is incredibly overengineered...
Rust seems intimidating at first, but under the hood the language semantics are (or used to be, see below) fairly simple.
The big borrow checker hurdle comes from developers not being used to thinking in/about lifetimes. It takes some getting used to because it enforces constraints that should be there for a non-GC language, but are not enforced anywhere else. But it is a hurdle you can get over after a few weeks of feeling like an idiot. At one point it just clicks. You start to design your types and code with lifetimes in mind, and the borrow checker is not a problem anymore and feels natural - and a helpful safeguard even.
The other complex part is the lack of inheritance and the trait system, which is also foreign for many. I feel like if you had contact with ML languages, and Haskell in particular, you will have a much easier time. It takes a different approach for structuring code.
Sadly Rust is in fact getting mroe more complex all the time though, with things like async/await added and other complex features in the pipeline (like specialization, higher kinded types, generators, ...).
All of those make sense and make the language more powerful, but at the expense of simplicity.
The big borrow checker hurdle comes from developers not being used to thinking in/about lifetimes. It takes some getting used to because it enforces constraints that should be there for a non-GC language, but are not enforced anywhere else. But it is a hurdle you can get over after a few weeks of feeling like an idiot. At one point it just clicks. You start to design your types and code with lifetimes in mind, and the borrow checker is not a problem anymore and feels natural - and a helpful safeguard even.
The other complex part is the lack of inheritance and the trait system, which is also foreign for many. I feel like if you had contact with ML languages, and Haskell in particular, you will have a much easier time. It takes a different approach for structuring code.
Sadly Rust is in fact getting mroe more complex all the time though, with things like async/await added and other complex features in the pipeline (like specialization, higher kinded types, generators, ...).
All of those make sense and make the language more powerful, but at the expense of simplicity.
The borrow checker is more problematic for systems software than you suggest. It is unable to express lifetime models that are essential for some types of systems software, forcing you to either litter your code with "unsafe" or to sacrifice significant amounts of performance. If performance is critical (it is for my use cases), and achieving that requires disabling memory safety, what other unique value does Rust bring to the table? That is a question to which I have been unable to find an obvious answer.
Complexity doesn't bother me (obviously, since I use C++ regularly) but lack of expressiveness is a big issue.
Complexity doesn't bother me (obviously, since I use C++ regularly) but lack of expressiveness is a big issue.
Sometimes, safety is faster than unsafety, because it lets you achieve algorithmic speedups in a more maintainable way. For example, Stylo, the new parallel rendering engine in Firefox, was attempted in C++ a few times. It failed each time. However, the project succeeded in Rust instead, and is much faster.
Additionally, the goal of unsafe is not to throw away all safety; it's to prove to the compiler something that you know, but it does not know. This means that you can contain the unsafety, and still gain the benefits as a user. The goal is to isolate and contain it, making it easier to write correctly.
Additionally, the goal of unsafe is not to throw away all safety; it's to prove to the compiler something that you know, but it does not know. This means that you can contain the unsafety, and still gain the benefits as a user. The goal is to isolate and contain it, making it easier to write correctly.
I haven't done a lot of C programming myself, but I've heard very mixed opinions from C devs. Some say they hate the borrow checker because it introduces artificial constraints.
Others say that they love it because it enforces how they write their code anyway, but now with compiler checks that prevent them from messing up.
I think it's a fair criticism that the borrow checker is sometimes not powerful enough to understand a perfectly valid access pattern, but unsafe is always there if you know better.
More often than not (in my experience), the access pattern is not actually valid though, or risky and easy to mess up. And the compiler rubs your nose in a suboptimal design.
I've found that when I needed to reach for `unsafe`, I often could restructure the code instead to achieve a safe result with minimal performance loss.
Others say that they love it because it enforces how they write their code anyway, but now with compiler checks that prevent them from messing up.
I think it's a fair criticism that the borrow checker is sometimes not powerful enough to understand a perfectly valid access pattern, but unsafe is always there if you know better.
More often than not (in my experience), the access pattern is not actually valid though, or risky and easy to mess up. And the compiler rubs your nose in a suboptimal design.
I've found that when I needed to reach for `unsafe`, I often could restructure the code instead to achieve a safe result with minimal performance loss.
> Sadly Rust is in fact getting mroe more complex all the time though, with things like async/await added and other complex features in the pipeline (like specialization, higher kinded types, generators, ...).
Some of those have fairly simple semantics though.
Generators are really just very a very big enum, with a function moving through the different states of the enum. Async/await is just a fancy name for a generator that returns a Poll.
Specialization is very constrained - it only applies to trait methods qualified with "default"), but I agree the current rules around it are rather complex, and they aren't even sound so they'll probably get even more complex.
HKT has no concrete plan so it's hard to know how much mental overhead it will add. But it's likely to be very non-trivial too.
Some of those have fairly simple semantics though.
Generators are really just very a very big enum, with a function moving through the different states of the enum. Async/await is just a fancy name for a generator that returns a Poll.
Specialization is very constrained - it only applies to trait methods qualified with "default"), but I agree the current rules around it are rather complex, and they aren't even sound so they'll probably get even more complex.
HKT has no concrete plan so it's hard to know how much mental overhead it will add. But it's likely to be very non-trivial too.
> Some of those have fairly simple semantics though.
I'd disagree with that. Yes technically, a generator is just a enum representing a state machine that can be stepped through. And async/await is just a generator with some extra compiler utility.
But they completely change how code is executed, to the point that you don't really think about it as a state machine and have to work hard (mentally) to reconstruct what is happening under the hood. Debugging gets much more complex. And it also adds another part that you need to understand and work with.
Async/await is particularly bad because it's not a standalone feature: it comes with a large ecosystem of crates that provide streams, executors, reactors, timers, ....
It's a huge strain on the complexity budget.
I'd disagree with that. Yes technically, a generator is just a enum representing a state machine that can be stepped through. And async/await is just a generator with some extra compiler utility.
But they completely change how code is executed, to the point that you don't really think about it as a state machine and have to work hard (mentally) to reconstruct what is happening under the hood. Debugging gets much more complex. And it also adds another part that you need to understand and work with.
Async/await is particularly bad because it's not a standalone feature: it comes with a large ecosystem of crates that provide streams, executors, reactors, timers, ....
It's a huge strain on the complexity budget.
> The other complex part is the lack of inheritance and the trait system, which is also foreign for many
I agree that the trait system is less familiar to most programmers than inheritance, but I'm not convinced it's more complex; inheritance itself isn't really something I'd describe as simple, just something that people are more familiar with.
I agree that the trait system is less familiar to most programmers than inheritance, but I'm not convinced it's more complex; inheritance itself isn't really something I'd describe as simple, just something that people are more familiar with.
> The big borrow checker hurdle comes from developers not being used to thinking in/about lifetimes. It takes some getting used to because it enforces constraints that should be there for a non-GC language, but are not enforced anywhere else.
This!
This!
It's not that hard, it just brings very unfamiliar ideas to the table. If you've used a bunch of different languages, most other languages can be picked up pretty quickly by combining different aspects you're familiar with from the ones you know.
That's not the case for Rust. It takes a few solid weeks of feeling stupid until you really start to grasp borrowing, and that experience may feel offensive to people who are used to knowing what they're doing, but a few weeks really isn't that much time in the grand scheme of things.
That's not the case for Rust. It takes a few solid weeks of feeling stupid until you really start to grasp borrowing, and that experience may feel offensive to people who are used to knowing what they're doing, but a few weeks really isn't that much time in the grand scheme of things.
Hmm... I'm not so sure the ideas themselves are unfamiliar. Any one who has used C or C++ has to deal with lifetimes (or at least should) and things like iterators are a very common pattern nowadays.
What Rust does differently is making ownership and unique/shared references a central feature of the language in a way that is amenable to static analysis. Obviously this is often a struggle for programmers to get used to, there's no denying that, but I don't think it's the concepts themselves that are unfamiliar. Unless of course the programmer is new to low level languages.
What Rust does differently is making ownership and unique/shared references a central feature of the language in a way that is amenable to static analysis. Obviously this is often a struggle for programmers to get used to, there's no denying that, but I don't think it's the concepts themselves that are unfamiliar. Unless of course the programmer is new to low level languages.
I used C++ for 4 years before trying Rust and I still felt like 50% of the latter was completely new. There's a huge difference between understanding how to manage memory and understanding how to satisfy the borrow-checker. Certain things - sometimes legitimate things! - that you could do normally are simply not allowed, because they can't be proven to be correct.
> Rust is the first one I'm too stupid to understand.
That's a typical experience, but it's not because Rust is that complicated, but because it's different.
People approach Rust with their intuition from other languages, such as C's or Java's where you can have a web of objects referencing each other willy-nilly, and are stumped by Rust's insistence on clear, single ownership, and simpler tree/DAG-like structure of program's data.
Single ownership itself is simple, you just need to unlearn multiple-shared-ownership habits.
That's a typical experience, but it's not because Rust is that complicated, but because it's different.
People approach Rust with their intuition from other languages, such as C's or Java's where you can have a web of objects referencing each other willy-nilly, and are stumped by Rust's insistence on clear, single ownership, and simpler tree/DAG-like structure of program's data.
Single ownership itself is simple, you just need to unlearn multiple-shared-ownership habits.
Single ownership isn't a preference, it's a constraint and a limitation. A constraint (in a good way) because that's the entire point of the language--preventing you from losing track of memory ownership. It's a limitation because there are some extremely important and common relationships (e.g. putting an object in multiple containers simultaneously) that cannot be expressed in Rust's type system without using unsafe, RC boxes, etc, which create tremendous friction. Yet a low-level systems language is precisely the environment where you most want and need to express these relationships and data structures cleanly and efficiently.
A data structure isn't something you import from a module. A good program is composed of layers of bespoke data structures that culminate in a giant data structure tailored for a particular task. So the fact that you can just import a hash table or tree implementation (friction-free) is irrelevant; useful, but irrelevant because it's combining such data structures that is the primary task at hand when writing a non-trivial piece of software, and combining them necessarily results in complicated ownership dependencies.
And this is all especially true for applications that do more than transform some input into some output in one shot.
I'm not trying to criticize Rust. They took a good idea and ran with it, and expressing more complex ownership semantics is still an open problem (perhaps with no satisfying answer, ever). But minimizing or dismissing the sore points isn't constructive.
A data structure isn't something you import from a module. A good program is composed of layers of bespoke data structures that culminate in a giant data structure tailored for a particular task. So the fact that you can just import a hash table or tree implementation (friction-free) is irrelevant; useful, but irrelevant because it's combining such data structures that is the primary task at hand when writing a non-trivial piece of software, and combining them necessarily results in complicated ownership dependencies.
And this is all especially true for applications that do more than transform some input into some output in one shot.
I'm not trying to criticize Rust. They took a good idea and ran with it, and expressing more complex ownership semantics is still an open problem (perhaps with no satisfying answer, ever). But minimizing or dismissing the sore points isn't constructive.
> It's a limitation because there are some extremely important and common relationships (e.g. putting an object in multiple containers simultaneously) that cannot be expressed in Rust's type system without using unsafe, RC boxes, etc, which create tremendous friction.
That is not a difficult relationship to capture, IMHO. The problem is that it's too easy to conflate "address" and "name" in most languages. An address is the physical location of a thing. A name is a location-independent way to look up a thing. Only one thing can live at an address, but you can use a name to look up multiple addresses depending on what question you want answered. (There is a parallel here with domain names and IP addresses.)
Rust associates ownership with every address. This causes serious problems when you've been using addresses as names. The solution is to use different names.
For instance, let's represent a graph as a vector of nodes. Every node has an index within that vector. Given this index, we can look up any node -- but critically, having the index does not confer ownership of the node. It's a completely orthogonal concern. So a node can quite happily possess a vector of the indices of its neighbors.
You might complain that this is a poor naming scheme for a graph structure, since it complicates deletion of nodes from the vector. Great! We have these exact same problems when using pointers (observe that the index plus the vector base address is the node address), but now that naming is a first-class concept, we can consider it on its own terms. There are alternative index spaces we can apply.
I've used this kind of design in a compiler hobby project. The intermediate representation is a set of tables representing different information about the program. With each pass, I can easily add new tables or transform existing ones, so long as I keep the names consistent. I can easily represent relationships between entities by using their indices. And I can easily attach more information to certain entities in compiler passes by adding a new table keyed on the same index.
That is not a difficult relationship to capture, IMHO. The problem is that it's too easy to conflate "address" and "name" in most languages. An address is the physical location of a thing. A name is a location-independent way to look up a thing. Only one thing can live at an address, but you can use a name to look up multiple addresses depending on what question you want answered. (There is a parallel here with domain names and IP addresses.)
Rust associates ownership with every address. This causes serious problems when you've been using addresses as names. The solution is to use different names.
For instance, let's represent a graph as a vector of nodes. Every node has an index within that vector. Given this index, we can look up any node -- but critically, having the index does not confer ownership of the node. It's a completely orthogonal concern. So a node can quite happily possess a vector of the indices of its neighbors.
You might complain that this is a poor naming scheme for a graph structure, since it complicates deletion of nodes from the vector. Great! We have these exact same problems when using pointers (observe that the index plus the vector base address is the node address), but now that naming is a first-class concept, we can consider it on its own terms. There are alternative index spaces we can apply.
I've used this kind of design in a compiler hobby project. The intermediate representation is a set of tables representing different information about the program. With each pass, I can easily add new tables or transform existing ones, so long as I keep the names consistent. I can easily represent relationships between entities by using their indices. And I can easily attach more information to certain entities in compiler passes by adding a new table keyed on the same index.
I've always found this a very odd rationalization. These games are unnecessary to get the same sort of type safety and double-free protection.[1] They're sufficient to subvert the Rust compiler, but that's it. The "benefit" only exists in the context of Rust because it's the only way to accomplish this.
And of course the extra indirection is costly, especially for a systems language. And you're also doing much more memory management. Hacks like these are why people say handling OOM is too difficult, because such hacks turn a simple logical operation into a series of multiple memory allocations, each of which could individually fail, which explodes the number of possible failure points, creating significantly more tension than necessary between memory management and functional clarity.
[1] The analog in the land of pointers is simply to NULL node member pointers and fail if a member is non-NULL at destruction or insertion, which is effectively the same as with the index tables hack. And for type safety you can of course use container-specific member types, which is exactly how the canonical linked-list and tree implementations work in BSD. (See https://man.openbsd.org/queue#LIST_EXAMPLE and https://man.openbsd.org/tree#EXAMPLES.) And like with many things in Rust, but-for the single ownership limitation Rust would otherwise make doing all of this much easier because it makes it easier to maintain the NULL invariants.
And of course the extra indirection is costly, especially for a systems language. And you're also doing much more memory management. Hacks like these are why people say handling OOM is too difficult, because such hacks turn a simple logical operation into a series of multiple memory allocations, each of which could individually fail, which explodes the number of possible failure points, creating significantly more tension than necessary between memory management and functional clarity.
[1] The analog in the land of pointers is simply to NULL node member pointers and fail if a member is non-NULL at destruction or insertion, which is effectively the same as with the index tables hack. And for type safety you can of course use container-specific member types, which is exactly how the canonical linked-list and tree implementations work in BSD. (See https://man.openbsd.org/queue#LIST_EXAMPLE and https://man.openbsd.org/tree#EXAMPLES.) And like with many things in Rust, but-for the single ownership limitation Rust would otherwise make doing all of this much easier because it makes it easier to maintain the NULL invariants.
I think that we are subtly talking past each other, because what you're saying doesn't line up against what I thought I was saying. I'm not looking for additional type safety or security protections from this model. See my sibling comment to your earlier comment for a little more context [1].
As someone who works with Rust often, your assertions don't match my experience. I want to engage honestly and understand where Rust itself falters, versus where we need to invest in educating newcomers about what kinds of models are better represented in Rust than others. I have found the model I'm describing to be valuable in many ways -- more than a "hack" to get around Rust.
> And of course the extra indirection is costly, especially for a systems language.
To my knowledge, base-offset addressing is natively present on at least x86. With a vector-based index space like I was describing, a[b] == * (a + b) is no more costly than * b. If you would need a more complex index space, you were probably doing more than bare pointers already.
> And you're also doing much more memory management.
> because such hacks turn a simple logical operation into a series of multiple memory allocations
No, no more than you would have needed with pointer-based naming. The top-level arena (a vector in the graph example) owns everything and can be deallocated all at once when you're done with it. Arenas were already a well-known pattern before Rust, and they can reduce the amount of manual memory management over tracking a bunch of small, separate heap allocations.
Relatedly, Catherine West gave an excellent talk at RustConf 2018 on how the Entity-Component-System architecture maps nicely into Rust [2]. The address/name distinction comes from ECS, but it becomes even more useful in Rust.
> The analog in the land of pointers is simply to NULL node member pointers and fail if a member is non-NULL at destruction or insertion
I don't see how this is hindered by Rust. We use an Option type -- with Some(T) and None variants -- to model something that might not be present, so the logic can be exactly the same. Rust can even optimize away the "extra" space you would need for the Some/None variant flag in common situations [3].
[1] https://news.ycombinator.com/item?id=20822003
[2] https://kyren.github.io/2018/09/14/rustconf-talk.html
[3] https://github.com/rust-lang/rust/pull/45225
As someone who works with Rust often, your assertions don't match my experience. I want to engage honestly and understand where Rust itself falters, versus where we need to invest in educating newcomers about what kinds of models are better represented in Rust than others. I have found the model I'm describing to be valuable in many ways -- more than a "hack" to get around Rust.
> And of course the extra indirection is costly, especially for a systems language.
To my knowledge, base-offset addressing is natively present on at least x86. With a vector-based index space like I was describing, a[b] == * (a + b) is no more costly than * b. If you would need a more complex index space, you were probably doing more than bare pointers already.
> And you're also doing much more memory management.
> because such hacks turn a simple logical operation into a series of multiple memory allocations
No, no more than you would have needed with pointer-based naming. The top-level arena (a vector in the graph example) owns everything and can be deallocated all at once when you're done with it. Arenas were already a well-known pattern before Rust, and they can reduce the amount of manual memory management over tracking a bunch of small, separate heap allocations.
Relatedly, Catherine West gave an excellent talk at RustConf 2018 on how the Entity-Component-System architecture maps nicely into Rust [2]. The address/name distinction comes from ECS, but it becomes even more useful in Rust.
> The analog in the land of pointers is simply to NULL node member pointers and fail if a member is non-NULL at destruction or insertion
I don't see how this is hindered by Rust. We use an Option type -- with Some(T) and None variants -- to model something that might not be present, so the logic can be exactly the same. Rust can even optimize away the "extra" space you would need for the Some/None variant flag in common situations [3].
[1] https://news.ycombinator.com/item?id=20822003
[2] https://kyren.github.io/2018/09/14/rustconf-talk.html
[3] https://github.com/rust-lang/rust/pull/45225
My original point is that real-world programs are complex data structures where any particular object may need to be referenced from multiple containers. At which point you're right back to square one--which container is the true owner responsible for maintaining all invariants and releasing the object when it's dead? Rust memory management does not and cannot automatically "free" resources with such references; you're still doing this manually. Indeed, this is irreducible complexity; it's why programming is hard, even in GC'd languages.
The indexing scheme is a hack to subvert Rust's typing system. It's even admitted in the talk you mentioned,
> One more criticism of this style is that you might say that using indexes instead of pointers is “safe” in the strict sense, but possibly only technically, you’re trading UB and potential crashes with pointers for “random but unspecified” behavior if you access the wrong or outdated index, and potentially panics. You’d be right, btw!
The generational indexing discussed therein is just a runtime invariant assertion, just as-if you asserted on internal pointer members being NULL at destruction (which implies but doesn't guarantee that the node has been properly unlinked).
This is all just more memory games.[1] In some respects they may be marginally safer, but they're also marginally more complicated. And that's friction. And even if on the whole it makes programs safer, the cost exists. ATS is an even safer language than Rust. If we're gonna pretend that the friction doesn't matter why wouldn't we just pretend we should all use ATS?
I'm not arguing that C or C++ is better than Rust. It just bugs me the way people rationalize some of the difficulty with arguments about why it makes things easier. Yeah, single-ownership does make things easier. Good C programs and good programmers generally learn to strive for single ownership semantics. Without an argument, Rust's enforcement of this invariant makes programs safer. But arguing that the enforcement of the invariant makes it easier to program? Come on! I know single-ownership is safer and simpler, which means when I choose to have a more complex ownership pattern it's for a darned good reason. Having to jump through hoops to accomplish it, even if it's "good" for me, is still jumping through hoops.
[1] And game makers love memory hacks; it's why their apps crash all the time, and why the thought of handling allocation failure would never cross their minds. To understand why people stopped using arenas and ad hoc invariant assertions for systems programming, see Heartbleed.
The indexing scheme is a hack to subvert Rust's typing system. It's even admitted in the talk you mentioned,
> One more criticism of this style is that you might say that using indexes instead of pointers is “safe” in the strict sense, but possibly only technically, you’re trading UB and potential crashes with pointers for “random but unspecified” behavior if you access the wrong or outdated index, and potentially panics. You’d be right, btw!
The generational indexing discussed therein is just a runtime invariant assertion, just as-if you asserted on internal pointer members being NULL at destruction (which implies but doesn't guarantee that the node has been properly unlinked).
This is all just more memory games.[1] In some respects they may be marginally safer, but they're also marginally more complicated. And that's friction. And even if on the whole it makes programs safer, the cost exists. ATS is an even safer language than Rust. If we're gonna pretend that the friction doesn't matter why wouldn't we just pretend we should all use ATS?
I'm not arguing that C or C++ is better than Rust. It just bugs me the way people rationalize some of the difficulty with arguments about why it makes things easier. Yeah, single-ownership does make things easier. Good C programs and good programmers generally learn to strive for single ownership semantics. Without an argument, Rust's enforcement of this invariant makes programs safer. But arguing that the enforcement of the invariant makes it easier to program? Come on! I know single-ownership is safer and simpler, which means when I choose to have a more complex ownership pattern it's for a darned good reason. Having to jump through hoops to accomplish it, even if it's "good" for me, is still jumping through hoops.
[1] And game makers love memory hacks; it's why their apps crash all the time, and why the thought of handling allocation failure would never cross their minds. To understand why people stopped using arenas and ad hoc invariant assertions for systems programming, see Heartbleed.
> With a vector-based index space like I was describing, a[b] == * (a + b) is no more costly than * b. If you would need a more complex index space, you were probably doing more than bare pointers already.
Of course you're doing more than bare pointers. Which is why your example isn't applicable. (a + b) is fast when you're accessing record fields, because you already have a in a register and b is usually a constant. In the vector scheme you're doing (v + a + b), where v needs to be fetched (often through another level of indirection), and unless you're doing a long series of operation to the same structure, you're constantly reloading v and recalculating a.[1]
It doesn't follow that technique A is equivalent to B simply because when you reduce B to a microbenchmark the difference to A is de minimis. Microbenchmarks obscure and often elide the very reasons B would be slower than A. That said, I'm usually dubious of such performance arguments. This level of performance rarely matters, though when you do it ubiquitously it's more likely to matter, which is why it crossed my mind.
[1] It's often pointed out that loops like `for (size_t i = 0; i < n; i++) {}` are as fast as `for (; p < pe; p++) { ... }`, and that the former is preferable for various dubious reasons related to the ease of implementing boundary checks. But the performance is often the same because (and to the extent that) the compiler can reduce the former to the latter. Even today I regularly encounter situations where using pointers is cleaner and faster than indexing. Though that fact that it's faster is more a novelty; what matters most to me is clarity and correctness, which is highly context dependent.
Of course you're doing more than bare pointers. Which is why your example isn't applicable. (a + b) is fast when you're accessing record fields, because you already have a in a register and b is usually a constant. In the vector scheme you're doing (v + a + b), where v needs to be fetched (often through another level of indirection), and unless you're doing a long series of operation to the same structure, you're constantly reloading v and recalculating a.[1]
It doesn't follow that technique A is equivalent to B simply because when you reduce B to a microbenchmark the difference to A is de minimis. Microbenchmarks obscure and often elide the very reasons B would be slower than A. That said, I'm usually dubious of such performance arguments. This level of performance rarely matters, though when you do it ubiquitously it's more likely to matter, which is why it crossed my mind.
[1] It's often pointed out that loops like `for (size_t i = 0; i < n; i++) {}` are as fast as `for (; p < pe; p++) { ... }`, and that the former is preferable for various dubious reasons related to the ease of implementing boundary checks. But the performance is often the same because (and to the extent that) the compiler can reduce the former to the latter. Even today I regularly encounter situations where using pointers is cleaner and faster than indexing. Though that fact that it's faster is more a novelty; what matters most to me is clarity and correctness, which is highly context dependent.
> Though that fact that it's faster is more a novelty; what matters most to me is clarity and correctness, which is highly context dependent.
We agree.
We agree.
This jibes very well with my own experience. When working in languages like Java, I feel myself overwhelmed with keeping a grip on the runtime topology of the object graph. Who can talk to whom? Who can _affect_ whom? My usual tools for covering my bases here -- immutability and/or value types -- are just not ergonomic in Java. If I have an "immutable" object with a getter that returns a reference to some mutable object, then it's not really immutable. Quoting Joe Armstrong:
> You wanted a banana but what you got was a gorilla holding the banana and the entire jungle.
The clarity and simplicity of a single ownership model trumps just about everything else for me.
> You wanted a banana but what you got was a gorilla holding the banana and the entire jungle.
The clarity and simplicity of a single ownership model trumps just about everything else for me.
There are very important and valid use cases for logical ownership both where N > 1 and N == 0. Ironically, the latter case doesn't have a use case outside of systems programming, so it is deficient for a systems programming language to not support it. (N == 0 does not imply literally "no owner", just that the owner is intrinsically non-visible to the compiler.)
Yes, it is possible to awkwardly and inefficiently shoehorn these cases into an N == 1 model but it comes at a substantial cost.
Yes, it is possible to awkwardly and inefficiently shoehorn these cases into an N == 1 model but it comes at a substantial cost.
The main point about Rust is that you're never stuck with a dead-end substantial cost. In the worst case you slap `unsafe` and you're no worse than in C (and better everywhere else where zero-cost abstractions worked and you didn't need to use `unsafe`).
There are awkward cases (self-referential structs, ugh), but let's not forget that in many many cases there are straightforward, zero-cost solutions. E.g. parent incorrectly stated you can't put one object in multiple containers. You can: one will own it, the rest will borrow it. And if you can't explain in your program which one is the owner, chances are you have the same problem in other languages:
• Many C programs with very complex and performance-sensitive ownership just use arenas. This pattern works in Rust, too (at zero cost).
• If you're certain you can get the custom ownership right with C pointers, Rust has C pointers, too (at zero runtime cost).
• When ownership is truly dynamic, C/C++ programs also have to use some kind of refcount or a flag to track it (e.g. setting freed pointer to null == Option.take() in Rust, bool free_me = borrow::Cow in Rust). Rust's refcounting is very efficient (intrusive, atomics not required), equal or better than `shared_ptr`.
With practice the frequency of "awkwardness" when writing code drops. You may still need to be more precise/restrained (especially around mutability and global state), but that pays off as soon as you want to make your code multi-threaded.
There are awkward cases (self-referential structs, ugh), but let's not forget that in many many cases there are straightforward, zero-cost solutions. E.g. parent incorrectly stated you can't put one object in multiple containers. You can: one will own it, the rest will borrow it. And if you can't explain in your program which one is the owner, chances are you have the same problem in other languages:
• Many C programs with very complex and performance-sensitive ownership just use arenas. This pattern works in Rust, too (at zero cost).
• If you're certain you can get the custom ownership right with C pointers, Rust has C pointers, too (at zero runtime cost).
• When ownership is truly dynamic, C/C++ programs also have to use some kind of refcount or a flag to track it (e.g. setting freed pointer to null == Option.take() in Rust, bool free_me = borrow::Cow in Rust). Rust's refcounting is very efficient (intrusive, atomics not required), equal or better than `shared_ptr`.
With practice the frequency of "awkwardness" when writing code drops. You may still need to be more precise/restrained (especially around mutability and global state), but that pays off as soon as you want to make your code multi-threaded.
For my use cases we can ignore the case of self-referential structs. Slapping "unsafe" on everything isn't equivalent to C++ because Rust is less expressive in some important ways.
I mostly work on high-end database engines, which appear to violate assumptions of Rust about what a sensible memory model can look like. In a modern database engine, almost all of your working memory is unavoidably paged, opaque, and directly DMA-ed. The concept of a lifetime cannot be attached to a memory address and virtually all of your data structures are "unsafe" in a Rust sense. You have to consider implications like destructors being effectively non-deterministic.
Fortunately, schedule-based safety models are transparent to developers, do not require "borrowing" so it works well with references not visible at compile-time, and all references can be mutable. To be clear, these models don't generalize to all software either, they are just well-suited to the requirements of high-throughput server engines. High-scale database engines are effectively single-threaded, so the only cost is a relatively trivial number of non-atomic ref counts. The underlying C++ to make this be handled automagically from a developer perspective is not trivial though.
Rust's memory safety model looks like it was designed for more traditional applications, like web browsers. Modern server software that is I/O intensive is going to look pretty similar to the above if performance matters, and that is a rough fit for Rust's memory model.
I mostly work on high-end database engines, which appear to violate assumptions of Rust about what a sensible memory model can look like. In a modern database engine, almost all of your working memory is unavoidably paged, opaque, and directly DMA-ed. The concept of a lifetime cannot be attached to a memory address and virtually all of your data structures are "unsafe" in a Rust sense. You have to consider implications like destructors being effectively non-deterministic.
Fortunately, schedule-based safety models are transparent to developers, do not require "borrowing" so it works well with references not visible at compile-time, and all references can be mutable. To be clear, these models don't generalize to all software either, they are just well-suited to the requirements of high-throughput server engines. High-scale database engines are effectively single-threaded, so the only cost is a relatively trivial number of non-atomic ref counts. The underlying C++ to make this be handled automagically from a developer perspective is not trivial though.
Rust's memory safety model looks like it was designed for more traditional applications, like web browsers. Modern server software that is I/O intensive is going to look pretty similar to the above if performance matters, and that is a rough fit for Rust's memory model.
Have you looked at TiKV/TiDB? I’m curious what you think of them.
TiDB is built as a set of abstractions on top of RocksDB, inheriting weaknesses intrinsic to that style of architecture. Two things I would call out about the design:
First, loose coupling through the layers of the stack probably gives up an order of magnitude in throughput on the same hardware versus a state-of-the-art tightly coupled architecture for many workloads. Many optimizations are enabled when high level orchestration and operation semantics (like a SQL join) are understood at every level of the stack down to DMA scheduling. Most open source databases use loose coupling because (1) it is much simpler to design and (2) developers can reuse third-party code for parts they lack the expertise or resources to implement themselves.
Second, RocksDB is a modern implementation of an obsolete design. As a practical matter, this has a high cost in terms of storage scalability and throughput. Many open source projects use it because they don't have a clear idea of how to design something better. Applications where storage engine performance and scalability are critical, e.g. real-time sensor data models, is an area where open source is not remotely competitive at the moment.
Unfortunately, these criticisms could be leveled at most open source database engine designs, which tend to optimize for expediency and ease of implementation rather than performance, and with little awareness of how much performance is being sacrificed. The divergence between academic literature and state-of-the-art in databases is now almost unrecognizable, and the handful of people with expertise in high-end implementation are not working on open source systems.
First, loose coupling through the layers of the stack probably gives up an order of magnitude in throughput on the same hardware versus a state-of-the-art tightly coupled architecture for many workloads. Many optimizations are enabled when high level orchestration and operation semantics (like a SQL join) are understood at every level of the stack down to DMA scheduling. Most open source databases use loose coupling because (1) it is much simpler to design and (2) developers can reuse third-party code for parts they lack the expertise or resources to implement themselves.
Second, RocksDB is a modern implementation of an obsolete design. As a practical matter, this has a high cost in terms of storage scalability and throughput. Many open source projects use it because they don't have a clear idea of how to design something better. Applications where storage engine performance and scalability are critical, e.g. real-time sensor data models, is an area where open source is not remotely competitive at the moment.
Unfortunately, these criticisms could be leveled at most open source database engine designs, which tend to optimize for expediency and ease of implementation rather than performance, and with little awareness of how much performance is being sacrificed. The divergence between academic literature and state-of-the-art in databases is now almost unrecognizable, and the handful of people with expertise in high-end implementation are not working on open source systems.
How are schedule-based safety models different than arena-based models?
The borrow checker is only the first problem. Some objects gave an implicit member called await, except unlike every other member accessed with . this one does some crazy thing where the function returns early. Why does this look like member access? Any other weird members I need to know about?
`await` isn't a member its a post-fix operator. Its also a keyword so it will be highlighted in most environments and you won't be able to write actual members with the name unless you use and access it with the raw syntax `r#await`. Its also only available on types that impl the future trait (either manually or async functions).
The only other post-fix operator in rust is `?` which will return early and do error conversion like the old `try!` macro.
Both of these operators are post-fix so that you can easily intermix them with method calls and member accesses without temporaries. Going from a sync function to an async function is often little more than putting `async` in the function prototype and sprinkling some `.await`s in. If `.await` wasn't post-fix you would need to rewrite a large number of lines to introduce new temporaries.
Ya it looks a little weird at first but this is a pretty lazy criticism that doesn't hold water for anyone who's spent more than 5 minutes looking at async Rust code.
The only other post-fix operator in rust is `?` which will return early and do error conversion like the old `try!` macro.
Both of these operators are post-fix so that you can easily intermix them with method calls and member accesses without temporaries. Going from a sync function to an async function is often little more than putting `async` in the function prototype and sprinkling some `.await`s in. If `.await` wasn't post-fix you would need to rewrite a large number of lines to introduce new temporaries.
Ya it looks a little weird at first but this is a pretty lazy criticism that doesn't hold water for anyone who's spent more than 5 minutes looking at async Rust code.
> Ya it looks a little weird at first but this is a pretty lazy criticism that doesn't hold water for anyone who's spent more than 5 minutes looking at async Rust code.
To be fair to your parent, this exact critique was brought up by many experienced Rustaceans during the development of async/await. In the end, it was decided that this drawback is worth it, but this is a drawback.
To be fair to your parent, this exact critique was brought up by many experienced Rustaceans during the development of async/await. In the end, it was decided that this drawback is worth it, but this is a drawback.
> this exact critique was brought up by many experienced Rustaceans
Just because they're experienced doesn't make it a great argument. Additionally they generally articulated more substantial concerns than "it looks weird."
To be exact my argument is that strange syntax does not increase the complexity of the feature. The complexity of async/await, especially when compared to features like the borrow checker as CJefferson complained, is in its semantics. You need to learn general async/await semantics like the fact that awaits always have yields regardless of which side of the expression you put it on among other things. And a number of rust-specific complexities that are very different from other languages (cold by default futures, executors vs reactors, pinning, etc.)
Just because they're experienced doesn't make it a great argument. Additionally they generally articulated more substantial concerns than "it looks weird."
To be exact my argument is that strange syntax does not increase the complexity of the feature. The complexity of async/await, especially when compared to features like the borrow checker as CJefferson complained, is in its semantics. You need to learn general async/await semantics like the fact that awaits always have yields regardless of which side of the expression you put it on among other things. And a number of rust-specific complexities that are very different from other languages (cold by default futures, executors vs reactors, pinning, etc.)
> Just because they're experienced doesn't make it a great argument.
Neither does saying "this is a pretty lazy criticism that doesn't hold water for anyone who's spent more than 5 minutes looking at async Rust code." It clearly does and did.
Neither does saying "this is a pretty lazy criticism that doesn't hold water for anyone who's spent more than 5 minutes looking at async Rust code." It clearly does and did.
The problem is, someone has to tell you it's a keyword. I have written a bunch of rust. I started seeing await looking like a member, I assumed it was a member.
Now I wonder what other bits of rust examples I've been misreading. What other postfix operators are there where I might assume it's a member?
Now I wonder what other bits of rust examples I've been misreading. What other postfix operators are there where I might assume it's a member?
The main advantage of Rust is that once the app compiles, it's likely to run quite smoothly.
The syntax is strict, but this avoids some bugs that would have taken time to spot at runtime.
For some use cases, this is a time saver. For everything else, you may spend more time fighting the compiler than you would have done debugging these issues afterwards.
But yeah, it's a language that constantly makes you feel stupid. The compiler is barfing errors that are hard to understand, for code that looks trivial.
The syntax is strict, but this avoids some bugs that would have taken time to spot at runtime.
For some use cases, this is a time saver. For everything else, you may spend more time fighting the compiler than you would have done debugging these issues afterwards.
But yeah, it's a language that constantly makes you feel stupid. The compiler is barfing errors that are hard to understand, for code that looks trivial.
I think GP is using 'semantics' to refer to the emitted code, which is "simple" in the sense that it e.g. doesn't have a runtime or a garbage collector. What isn't simple is all the features of rust that are unheard of in other languages, like the borrow checker, but that is a feature of Rust's syntax, and (I am not a Rust expert, but my understanding is that) many of them "vanish" when the Rust code is compiled.
> Rust is the first one I'm too stupid to understand.
More complicated than Haskell, etc.?
More complicated than Haskell, etc.?
I'm probably too stupid for Haskell, too, but in my experience the Haskell community actually acknowledges that the language is hard!
The patronizing attitude of rust defenders here on HN, on the other hand, man... it just pushes me further away.
The patronizing attitude of rust defenders here on HN, on the other hand, man... it just pushes me further away.
Does your list of language include Erlang? I'd be surprised if a language has more esoteric semantics than Erlang.
Erlang is super cool! Not really my first choice when starting something new, but that's more because Erlang's strengths don't really match the kind of projects I'm interested in!
I would be wary of calling 'simple' the semantics of a language that requires intimate knowledge of particular optimisations in its code generation framework to implement dynamicaly allocated arrays.
https://doc.rust-lang.org/nomicon/vec-alloc.html
https://doc.rust-lang.org/nomicon/vec-alloc.html
> Both Nim and Kotlin Native have a runtime in the form of a GC/automatic memory management, so you'd still need another language "beneath" them to get the job done.
That is not necessarily true. Not sure in the case of Nim or Kotlin Native, but the GC itself can be written in Nim/Kotlin Native in principle as well.
That is not necessarily true. Not sure in the case of Nim or Kotlin Native, but the GC itself can be written in Nim/Kotlin Native in principle as well.
In that case, the "language below" Nim or Kotlin would itself be Nim or Kotlin, but you'd still need that additional layer.
It's not a crazy idea. Go actually works that way already; the runtime of Go is implemented in Go. For example, here's the sweeping component of the gc: https://github.com/golang/go/blob/master/src/runtime/mgcswee... But that doesn't mean you can get a Go without a runtime; the final executable's GC may have come from compiled Go rather than something else, but it still irreducibly has GC in it. The whole thing is bootstrapped, so at any given time when some bit of Go code is running, it is supported by GC and the rest of the non-optional runtime, even when compiling code that itself is going to be the runtime of some other executable in the future.
It's not a crazy idea. Go actually works that way already; the runtime of Go is implemented in Go. For example, here's the sweeping component of the gc: https://github.com/golang/go/blob/master/src/runtime/mgcswee... But that doesn't mean you can get a Go without a runtime; the final executable's GC may have come from compiled Go rather than something else, but it still irreducibly has GC in it. The whole thing is bootstrapped, so at any given time when some bit of Go code is running, it is supported by GC and the rest of the non-optional runtime, even when compiling code that itself is going to be the runtime of some other executable in the future.
For reference: https://nim-lang.org/docs/gc.html
IIRC at least in Nim the GC is optional. Never tried it myself though, so I have no idea how it feels to manage memory manually with Nim.
Agreed overall, this might be why Rust is winning. (By the way neither Nim nor Kotlin Native have a heavy runtime in the same way that Java / JavaScript / Python do. Garbage collection in Nim occurs only at specified times.)
Just like Oberon has the whole OS written in it.
http://www.projectoberon.com/
http://www.projectoberon.com/
I'm hoping for Zig [1] to reach maturity and offer that kind of simplicity, because I just can't get a handle on Rust (not that I've spent a massive amount of time trying).
[1]: https://ziglang.org
[1]: https://ziglang.org
I'm hoping for it to get more exposure.
The fact that Mozilla was behind Rust played a lot in Rust becoming popular.
But a lot of nice languages have stayed under the radar, and have been eventually abandoned.
To succeed, Zig needs maturity, but also marketing. And that is hard without some big name advocating it.
The fact that Mozilla was behind Rust played a lot in Rust becoming popular.
But a lot of nice languages have stayed under the radar, and have been eventually abandoned.
To succeed, Zig needs maturity, but also marketing. And that is hard without some big name advocating it.
Both V (https://vlang.io/) and Zig seem really interesting, I also hope they will reach maturity
+1 for Zig, tho it has a long way to go.
I've seen ppl say that something seems amiss with the V project.
I've seen ppl say that something seems amiss with the V project.
There is also:
https://github.com/odin-lang/Odin
and
https://www.kitlang.org/
That are trying to get into the C replacement business.
That are trying to get into the C replacement business.
Ah ! Nim... as a Pythonista, I'm loving its aesthetics. However in the end it's all about community support, available libs, tooling...
It's taking off, sure, but every single piece of Rust software I've tried has been, uh, I'll be nice and say not to my taste. Almost like the safety guarantees Rust provides means that developers stop caring about portability, UNIX conventions or just... general... comfiness.
As a Windows user, the Rust ecosystem feels far more portable than any other ecosystem I've used; I run into issues extremely rarely, even though most people are on Mac OS or Linux. (Our last survey did show ~30% of people using Windows, though.)
Yeah this is a big benefit that is undersold. If I were in the C or C++ ecosystem, I probably wouldn't be able to support Windows as well as I do. It would be too much work. The foundations on which Rust's standard library are built means I can regularly go through entire release cycles without opening my Windows laptop to test anything. Which saves a lot of context switching overhead for me personally.
Do you have some examples of what you've experienced that you didn't like? I keep reading about people replacing tools written in C with Rust versions but I haven't really pursued it further and seen them in action.
The last one I tried was ripgrep, and something about the default behavior just drove me insane... I'll freely admit that it's all about aesthetics and there's no right answer, but that doesn't mean my preferences go away ;)
alias rg="rg -uuu"
Did you not also mention problems with portability and UNIX conventions?
Yes, and sorry, that's not specifically related to ripgrep. About a year ago I tried to compile a few rust tools and a bunch of them just straight up didn't work at all because they made assumptions that are only valid for debian/ubuntu based distros.
I'm glad people are enjoying rust, and sure, memory safety is nice. Fun and comfort are even better, though, so I'll stick to C, Lua and Go for my personal projects.
I'm glad people are enjoying rust, and sure, memory safety is nice. Fun and comfort are even better, though, so I'll stick to C, Lua and Go for my personal projects.
Yeah, I was just surprised since I specifically found Rust tools to generally both fun and comfortable due to having (what I consider) excellent UX.
Here is for instance a Rust tool that I just stumbled upon that seems to have an exemplary UX and adherence to Unix principles while packing a lot of features which I would personally call "fun".
https://github.com/sharkdp/pastel
https://github.com/sharkdp/pastel
Wow, that's a tool that I didn't know I needed until now. Being a back-end engineer I don't often deal with color information, but every once in a while I have to build a web page or tweak some CSS and it always a pain for me to remember color hex values. This tool should make this much easier. As a bonus, it gives you similar colors and can generate random or distinct color pallets.
It's not meant to work exactly like `grep`, if that's what you're on about. If anything, I'd say reimplementing working tools in another language is pointless.
It's like when bands cover songs. Who cares if another band plays the song if they don't put their own spin on it?
So it's more that you don't care for the tool than the language in this case. Although, I invite you to try ripgrep again and/or lay out some of your gripes with it. I find it a very good replacement for grep.
It's like when bands cover songs. Who cares if another band plays the song if they don't put their own spin on it?
So it's more that you don't care for the tool than the language in this case. Although, I invite you to try ripgrep again and/or lay out some of your gripes with it. I find it a very good replacement for grep.
When it comes to portability, Go is really awesome.
Anything that wants to unthrone C not only has to be better and no worse than C but also has to beat "C plus domain-specific libraries". A lot of things C itself lacks can be added with relatively thin libraries. The problem with that, of course, is that we can always circumvent libraries, conventions, type checkers, etc. and do dangerous things.
In that sense Rust does seem very interesting.
In that sense Rust does seem very interesting.
The Rust community definitely believes in the idea that if you repeat something enough times it becomes reality.
At least the Rust subreddit figured out that there's this other language which fulfills the same requirements of being a higher level C, except unlike Rust it has a massive ecosystem, mind share and vendor support - including from Intel. It does have worse PR, but this is mostly because Rust is unbeatable here.
It would be interesting to also hear from Josh's colleagues which work on a large number of C and C++ projects. Do they agree that Rust is the future? And does that mean that all of Intel's money-making software is essentially obsolete? Is Intel the company aware of that?
At least the Rust subreddit figured out that there's this other language which fulfills the same requirements of being a higher level C, except unlike Rust it has a massive ecosystem, mind share and vendor support - including from Intel. It does have worse PR, but this is mostly because Rust is unbeatable here.
It would be interesting to also hear from Josh's colleagues which work on a large number of C and C++ projects. Do they agree that Rust is the future? And does that mean that all of Intel's money-making software is essentially obsolete? Is Intel the company aware of that?
I like Rust because C++ is simply beyond my cognitive abilities.
>At least the Rust subreddit figured out that there's this other language which fulfills the same requirements of being a higher level C, except unlike Rust it has a massive ecosystem, mind share and vendor support - including from Intel. It does have worse PR, but this is mostly because Rust is unbeatable here.
Which language are you talking about? C++?
Which language are you talking about? C++?
Yes, C++. Although I'm not convinced that Intel's ready to move away from C to be honest.
I think Rust is a cool language, but I've never seen any job postings for it.
I'm still not sure what type of applications to use it for to be honest- not having good support for embedded systems and no CUDA/OpenCL support makes it not appealing.
I'm still not sure what type of applications to use it for to be honest- not having good support for embedded systems and no CUDA/OpenCL support makes it not appealing.
> What sets it apart from other modern languages is that it does not have a garbage collector or runtime system of any kind.
This is not strictly true. Most Rust programs, like most c programs, do have a runtime. See https://doc.rust-lang.org/0.12.0/guide-runtime.html
However, this runtime can be replaced which is what makes Rust suitable for certain systems programming scenarios (like building operating system kernels). This is the realm of [no_std]
This is not strictly true. Most Rust programs, like most c programs, do have a runtime. See https://doc.rust-lang.org/0.12.0/guide-runtime.html
However, this runtime can be replaced which is what makes Rust suitable for certain systems programming scenarios (like building operating system kernels). This is the realm of [no_std]
>Most Rust programs, like most c programs, do have a runtime. See https://doc.rust-lang.org/0.12.0/guide-runtime.html
You've linked to the docs of a release from 2014, back when Rust did have a runtime.
(Unfortunately it's very easy to get results for ancient Rust releases from search engines. Similarly there are three different versions of the Rust book and many of the search results are for the oldest one.)
>However, this runtime can be replaced which is what makes Rust suitable for certain systems programming scenarios (like building operating system kernels). This is the realm of [no_std]
What `no_std` allows you to do is not use libstd, ie the standard library. libstd is certainly a "runtime" in the sense that libc is the C runtime, but it doesn't mean it enforces green threading or garbage collection like the word "runtime" evokes for other languages and is used in the page you linked. This latter meaning is also what the sentence in the TFA was using.
You've linked to the docs of a release from 2014, back when Rust did have a runtime.
(Unfortunately it's very easy to get results for ancient Rust releases from search engines. Similarly there are three different versions of the Rust book and many of the search results are for the oldest one.)
>However, this runtime can be replaced which is what makes Rust suitable for certain systems programming scenarios (like building operating system kernels). This is the realm of [no_std]
What `no_std` allows you to do is not use libstd, ie the standard library. libstd is certainly a "runtime" in the sense that libc is the C runtime, but it doesn't mean it enforces green threading or garbage collection like the word "runtime" evokes for other languages and is used in the page you linked. This latter meaning is also what the sentence in the TFA was using.
I think you're being downvoted a bit unfairly.
1. Your link is incorrect, as the sibling comments say, but you're also correct that Rust has a runtime. It's about the same amount of runtime as C, but like C (as you say), it does exist.
2. no_std does not remove the runtime. I think the closest you can get is no_core, but to be honest, I'm not actually 100% sure.
1. Your link is incorrect, as the sibling comments say, but you're also correct that Rust has a runtime. It's about the same amount of runtime as C, but like C (as you say), it does exist.
2. no_std does not remove the runtime. I think the closest you can get is no_core, but to be honest, I'm not actually 100% sure.
[deleted]
You're linking to an old version of the rust documentation from before it was even 1.0.0. Rust used to have a runtime before 1.0.0 to implement green threading. But this is not the case anymore. Rust can be entirely statically linked into a self-contained library that has no runtime requirements whatsoever.
Maybe it's the new Assembly. With what little hacking and programming I am willing to put effort into, I will always prefer to work at the direct hardware access level rather than the massive abstraction that is required to run C & the relevant interpreters. Then again, I prefer working on 6xxx architecture (6502,6507,6809,etc) and cap out around 286 IBM compatible with MS-DOS for any real tasks.
Rust and Python. I see Rython in the future.
edit/sorry?
edit/sorry?
How do you see that working? Rust and Python come from very different philosophies. I like them both, but they are a strange mix. Python extensions in Rust? That could be a good thing. Much easier to write a reliable Python extension. I imagine tooling for that already exists and I am simply unaware.
Or, maybe Rust semantics with Python’s readability? I would like that. Rust is the wave of the future, but ultimately the language will not be a long-term survivor, entirely because of the ASCII-salad syntax. I predict that we will learn huge amounts about language semantics from Rust, thank the team, and then replace it with something pleasant to read.
Rust is the Algol of a new generation. I mean that as both high praise and as a sad prophecy.
Or, maybe Rust semantics with Python’s readability? I would like that. Rust is the wave of the future, but ultimately the language will not be a long-term survivor, entirely because of the ASCII-salad syntax. I predict that we will learn huge amounts about language semantics from Rust, thank the team, and then replace it with something pleasant to read.
Rust is the Algol of a new generation. I mean that as both high praise and as a sad prophecy.
I too am having problems 'getting into' Rust because of all that salad-y syntax. It looks like Perl at some times ;__;
My thoughts exactly, it reminds me a lot of Perls (lack of) readability. I would love a Rust variant with a mix of Python/Nim/F#/OCaml syntax.
let dest: Vec<String> = dest.split(',').map(|name| name.trim().to_string()).collect();
impl< $($kind: TryFrom< Value >,)* F > allMut< ($($kind,)*) > + 'static, F::Output:
impl< $($kind: TryFrom< Value >,)* F > allMut< ($($kind,)*) > + 'static, F::Output:
The first line is formatted by rustfmt to be
That second line, while intense, is the syntax for writing macros, not normal Rust code. As always, it depends on what you're doing, but you almost never need to write macros yourself.
let dest: Vec<String> = dest
.split(',')
.map(|name| name.trim().to_string())
.collect();
That second line, while intense, is the syntax for writing macros, not normal Rust code. As always, it depends on what you're doing, but you almost never need to write macros yourself.
Yes, the formatted version seems better. My complaint with Rust is the heavy use of symbols. I do not know what is going on most of the time because of it. How would you format this example?
fn deserialize< D: de::Deserializer< 'de > >( deserializer: D ) -> Result< Self, D::Error > {
In my opinion it is not obvious what most of the code is doing when you have hundreds of lines of code and each line contains lots of symbols. Again, this is just my opinion and perhaps my limitation.
I looked at many codebases and found that they were not easy on chaining and using symbols. I do not deny that they could have written it in a more readable way, but generally it was not the case in my experience, based on these projects at least.
fn deserialize< D: de::Deserializer< 'de > >( deserializer: D ) -> Result< Self, D::Error > {
In my opinion it is not obvious what most of the code is doing when you have hundreds of lines of code and each line contains lots of symbols. Again, this is just my opinion and perhaps my limitation.
I looked at many codebases and found that they were not easy on chaining and using symbols. I do not deny that they could have written it in a more readable way, but generally it was not the case in my experience, based on these projects at least.
I think it really depends on what you're used to.
You could format this alternately as:
The symbols in this example:
* <> are used for generics, which is common in other languages.
* : is for two things: trait bounds; that is, restricting a type parameter by a given trait. I'm not sure where this came from, but it feels fairly intuitive after a bit, at least to me. Second, for indicating the type of arguments. Both are about defining the types of things, so even though they're different, they feel consistent, at least to me.
* ' is used for lifetimes; this comes from OCaml, which many programmers aren't familiar with. Nobody likes this syntax, but nobody could come up with a better syntax.
* () is used for the parameter list to a function, this is extremely common in languages.
* -> indicates the return type
* :: is for navigating namespaces; the first instance is for a module, the latter is for an associated type. Similar to :, these are different but similar things, so the similar syntax makes it easier to me.
It's all just practice. Being familiar with a wide variety of languages helps, because Rust draws from a lot of other places.
You could format this alternately as:
fn deserialize<D>(deserializer: D) -> Result<Self, D::Error>
where
D: de::Deserializer<'de>,
{
The symbols in this example:
* <> are used for generics, which is common in other languages.
* : is for two things: trait bounds; that is, restricting a type parameter by a given trait. I'm not sure where this came from, but it feels fairly intuitive after a bit, at least to me. Second, for indicating the type of arguments. Both are about defining the types of things, so even though they're different, they feel consistent, at least to me.
* ' is used for lifetimes; this comes from OCaml, which many programmers aren't familiar with. Nobody likes this syntax, but nobody could come up with a better syntax.
* () is used for the parameter list to a function, this is extremely common in languages.
* -> indicates the return type
* :: is for navigating namespaces; the first instance is for a module, the latter is for an associated type. Similar to :, these are different but similar things, so the similar syntax makes it easier to me.
It's all just practice. Being familiar with a wide variety of languages helps, because Rust draws from a lot of other places.
Thank you for the brief explanation (or the meaning) of the symbols! Is there a place where it is laid out in a similar fashion? It would help tremendously.
Any time!
https://doc.rust-lang.org/stable/book/appendix-02-operators....
(I also just realized HN formatting messed up my re-formatting, I've fixed it now)
https://doc.rust-lang.org/stable/book/appendix-02-operators....
(I also just realized HN formatting messed up my re-formatting, I've fixed it now)
> Python extensions in Rust? That could be a good thing. Much easier to write a reliable Python extension. I imagine tooling for that already exists and I am simply unaware.
There is https://pyo3.rs/v0.7.0/
There is https://pyo3.rs/v0.7.0/
Well I was thinking similar to Cython which is just python that gets compiled(to over simplify I'm sure)
>Rust is the wave of the future, but ultimately the language will not be a long-term survivor, entirely because of the ASCII-salad syntax.
I can agree with that. Hence Rython.
Rust is certainly the future when compared to C or C++.
>Rust is the wave of the future, but ultimately the language will not be a long-term survivor, entirely because of the ASCII-salad syntax.
I can agree with that. Hence Rython.
Rust is certainly the future when compared to C or C++.
Ah, I get it.
From what little I have done with Cython, you done really see the C compiler much. Just the data types. Rython would still have the Python garbage collector ultimately “owning” all the variables, though, I think.
From what little I have done with Cython, you done really see the C compiler much. Just the data types. Rython would still have the Python garbage collector ultimately “owning” all the variables, though, I think.
There are Rust crates for implementing Python modules in Rust, and for embedding a Python interpreter.
Thanks! Now I need an excuse to write a Python extension. Seen any shaggy yaks around here?
BGP python library.
C is not the new assembly. This line has been reused for years, even decades. And it has always been a lie.
Memory safety and automatic memory management come at a cost. Even in Rust. C will always have a use for critical performance in the embedded world.
Memory safety and automatic memory management come at a cost. Even in Rust. C will always have a use for critical performance in the embedded world.
> C will always have a use for critical performance in the embedded world.
Based off of what reasoning? It's not like C is any faster or more efficient than Rust or C++. Heck you can often go faster in C++/Rust than you can in C thanks to the ease of using abstractions that just make things go-faster for you automatically (such as SSO in things like strings, or constexpr for compile-time resolution).
The embedded world isn't using C because it's a good fit, they are using it because of legacy inertia and lack of motivation to move to anything else. IoT is starting to change that, as all those C memory vulnerability issues turn into product support costs.
And also realistically embedded isn't even restricted to C already. Very popular microcontrollers like the esp8266 or esp32 also have javascript runtimes.
Based off of what reasoning? It's not like C is any faster or more efficient than Rust or C++. Heck you can often go faster in C++/Rust than you can in C thanks to the ease of using abstractions that just make things go-faster for you automatically (such as SSO in things like strings, or constexpr for compile-time resolution).
The embedded world isn't using C because it's a good fit, they are using it because of legacy inertia and lack of motivation to move to anything else. IoT is starting to change that, as all those C memory vulnerability issues turn into product support costs.
And also realistically embedded isn't even restricted to C already. Very popular microcontrollers like the esp8266 or esp32 also have javascript runtimes.
It's not like C is any faster or more efficient than Rust or C++
Huh? C is certainly faster than C++.
go-faster for you automatically (such as SSO in things like strings, or constexpr for compile-time resolution).
That is a terrible example. You shouldn't be using the STL (i.e. std::string copying) for performance critical sections to begin with.
Very popular microcontrollers like the esp8266 or esp32 also have javascript runtimes.
First, esps are not low end microcontrollers at all. Second, no one uses JS runtimes for performance critical code on microcontrollers (e.g. JS will not give you direct memory access). What an absurd claim.
Huh? C is certainly faster than C++.
go-faster for you automatically (such as SSO in things like strings, or constexpr for compile-time resolution).
That is a terrible example. You shouldn't be using the STL (i.e. std::string copying) for performance critical sections to begin with.
Very popular microcontrollers like the esp8266 or esp32 also have javascript runtimes.
First, esps are not low end microcontrollers at all. Second, no one uses JS runtimes for performance critical code on microcontrollers (e.g. JS will not give you direct memory access). What an absurd claim.
> Huh? C is certainly faster than C++.
It literally isn't. Quite often the reverse is true, as C++ gives you better tools for compile-time optimizations.
> That is a terrible example. You shouldn't be using the STL (i.e. std::string copying) for performance critical sections to begin with.
I think you're confusing the example with a recommendation to use the STL?
std::string is just an example of SSO. The concept is generally applicable and used in more places than just the STL.
Although "You shouldn't be using the STL for performance critical sections to begin with." is also complete fucking bullshit anyway. There's nothing wrong with the performance of eg std::array. Or unique_ptr. Or vector. Or a variety of other things in the STL. You're not outperforming any of those with raw C.
It's not 2003 anymore so stop acting like it.
It literally isn't. Quite often the reverse is true, as C++ gives you better tools for compile-time optimizations.
> That is a terrible example. You shouldn't be using the STL (i.e. std::string copying) for performance critical sections to begin with.
I think you're confusing the example with a recommendation to use the STL?
std::string is just an example of SSO. The concept is generally applicable and used in more places than just the STL.
Although "You shouldn't be using the STL for performance critical sections to begin with." is also complete fucking bullshit anyway. There's nothing wrong with the performance of eg std::array. Or unique_ptr. Or vector. Or a variety of other things in the STL. You're not outperforming any of those with raw C.
It's not 2003 anymore so stop acting like it.
It is 2019 and even game programmers still avoid STL for performance reasons.
And optimization is taken to another level from that on embedded systems.
I hate to remind you again, but it is 2019 and C is still very relevant on embedded systems. And Javascript, not so much. Snicker
And optimization is taken to another level from that on embedded systems.
I hate to remind you again, but it is 2019 and C is still very relevant on embedded systems. And Javascript, not so much. Snicker
> It is 2019 and even game programmers still avoid STL for performance reasons.
No they don't. They avoid parts of the STL. GDC round-table as far back as 2001 had 40% of shipping games using the STL: http://www.tantalon.com/pete/gdc01_round_table_report.htm
Many of the containers in the STL have performance problems. Yes avoid those (and avoid the god-awful iostream). But large chunks of the STL have no performance issues at all, and re-inventing it is just a waste of your time.
Games don't avoid things like std::vector for performance issues, they use their own thing because they want even more features.
No they don't. They avoid parts of the STL. GDC round-table as far back as 2001 had 40% of shipping games using the STL: http://www.tantalon.com/pete/gdc01_round_table_report.htm
Many of the containers in the STL have performance problems. Yes avoid those (and avoid the god-awful iostream). But large chunks of the STL have no performance issues at all, and re-inventing it is just a waste of your time.
Games don't avoid things like std::vector for performance issues, they use their own thing because they want even more features.
Surely not the same game programmers that use Unreal with its C++ GC.
What does that have to do with anything?
Yes, those same game programmers avoid STL.
No one is advocating replacing C++ with C on capable hardware, troll.
Yes, those same game programmers avoid STL.
No one is advocating replacing C++ with C on capable hardware, troll.
In what benchmarks?!?
Virtual functions, runtime type information, STL, larger executable code size, etc...
Memory footprint will be smaller on C without using C++ abstractions. Which is important for embedded systems.
Memory footprint will be smaller on C without using C++ abstractions. Which is important for embedded systems.
Probably the most important C++ ideal is that you don’t pay for what you don’t use, and you _still_ get a more expressive language than C if you don’t use those features.
Your cons:
- Virtual functions. Don’t use them if you don’t want to incur an extra indirection.
- STL. Don’t use it if you don’t want to.
- Large binary size. Don’t spin up hundreds of template instances, problem solved.
- No RTTI? Literally a compiler switch away.
Some pros:
- constexpr
- true type safe generic programming (at the cost of code size if you get crazy with it and like TMP)
- type safe no-overhead containers (std::array, std::tuple)
- RAII ownership semantics (I.e. std::unique_ptr)
- Can still emit a C ABI
- Multi-paradigm at a language level, rather than procedural with other paradigms bolted on ad-hoc
The reason C++ (IMHO) is always a better choice than C is that if you don’t want to incur the cost of feature X, drop down to how you would do it in C and it works exactly the same and you can still use the zero-cost C++ features that will never land in C.
Your cons:
- Virtual functions. Don’t use them if you don’t want to incur an extra indirection.
- STL. Don’t use it if you don’t want to.
- Large binary size. Don’t spin up hundreds of template instances, problem solved.
- No RTTI? Literally a compiler switch away.
Some pros:
- constexpr
- true type safe generic programming (at the cost of code size if you get crazy with it and like TMP)
- type safe no-overhead containers (std::array, std::tuple)
- RAII ownership semantics (I.e. std::unique_ptr)
- Can still emit a C ABI
- Multi-paradigm at a language level, rather than procedural with other paradigms bolted on ad-hoc
The reason C++ (IMHO) is always a better choice than C is that if you don’t want to incur the cost of feature X, drop down to how you would do it in C and it works exactly the same and you can still use the zero-cost C++ features that will never land in C.
If C++ is just as fast as C except when it's not, then it's not as fast as C.
More importantly, it's not easy to tell when you're going to hit performance issues with C++. You need an obscene amount of knowledge of both the language as well as hardware to get a sense of what to use to match C performance.
More importantly, it's not easy to tell when you're going to hit performance issues with C++. You need an obscene amount of knowledge of both the language as well as hardware to get a sense of what to use to match C performance.
I would agree with this assessment in most cases. The problem is it's hard to determine which language features perform worse than C. But usually the cost of being wrong is low or inconsequential for modern hardware.
But it is not true for embedded systems. And the industry disagrees too, and continues to use C because of this.
But it is not true for embedded systems. And the industry disagrees too, and continues to use C because of this.
> But it is not true for embedded systems. And the industry disagrees too, and continues to use C because of this.
What resource constrained embedded systems even exist? Fucking lightbulbs run JavaScript these days. SmartCards have been running Java for 20 years. What is the actual market you're so up in arms about that you think is choosing C on technical merits instead of purely inertia? What is this "performance constrained embedded" market to you?
What resource constrained embedded systems even exist? Fucking lightbulbs run JavaScript these days. SmartCards have been running Java for 20 years. What is the actual market you're so up in arms about that you think is choosing C on technical merits instead of purely inertia? What is this "performance constrained embedded" market to you?
What is this "performance constrained embedded" market to you?
The very definition of embedded systems?
I would say it is more much likely the many folks who work on embedded systems in C are right and do so with technical merit. And it is much more likely that it is you who is wrong. Which is quite clear as someone who seems to advocate using JS on embedded hardware for serious tasks. A detail that is seriously amusing in its own right.
The very definition of embedded systems?
I would say it is more much likely the many folks who work on embedded systems in C are right and do so with technical merit. And it is much more likely that it is you who is wrong. Which is quite clear as someone who seems to advocate using JS on embedded hardware for serious tasks. A detail that is seriously amusing in its own right.
Where is that definition written on?
Are we talking about PICs with 4KB, or ESP32 that would run circles around the Amstrad PC 1512 that I used to play Defender of the Crown on?
A system that already had quite a few high level programming languages available for us to learn programming on.
Ah maybe we are speaking about my Visa card running a set of tiny Java Card applications on 256KB, to handle my authentication processes at POS.
Are we talking about PICs with 4KB, or ESP32 that would run circles around the Amstrad PC 1512 that I used to play Defender of the Crown on?
A system that already had quite a few high level programming languages available for us to learn programming on.
Ah maybe we are speaking about my Visa card running a set of tiny Java Card applications on 256KB, to handle my authentication processes at POS.
I worked on embedded systems for trains and we used C++. Many people use C just because the chips they target only have a C compiler.
> The very definition of embedded systems?
Microcontrollers got a lot faster and more powerful over the years. What problem in that space kept up with that speed advancement? Who is paying for hyper-optimized C/asm code instead of just spending an extra $.50 for a dual-core 240mhz ESP32? And who is doing that decision and has decided C is somehow faster than C++?
> Which is quite clear as someone who seems to advocate using JS on embedded hardware for serious tasks.
I never advocated for it. I said it happened.
Microcontrollers got a lot faster and more powerful over the years. What problem in that space kept up with that speed advancement? Who is paying for hyper-optimized C/asm code instead of just spending an extra $.50 for a dual-core 240mhz ESP32? And who is doing that decision and has decided C is somehow faster than C++?
> Which is quite clear as someone who seems to advocate using JS on embedded hardware for serious tasks.
I never advocated for it. I said it happened.
Can you do high speed sampling, motor control, signal analysis,... in javascript in realtime? Usually microcontrollers have several memory region with different speeds and access characteristics. Can you fine tune in which parts of the memory part of your javascript program will go into? Can you write a driver for a high speed bus or low speed for that matter in javascript on a mcu?
No, but you can use JavaScript on ARM mbed OS, running on $5 ARM-based microcontrollers.
https://os.mbed.com/javascript-on-mbed/
On Samsung's SmartThings embedded devices
https://smartthings.developer.samsung.com/docs/index.html
SigFox customized IoT solutions
https://build.sigfox.com/
Or on the other side from ARM mbed's offering, a beefy ARM 580 MHz with 96 MB (RAM + Flash), using JavaScript alongside Rust.
https://tessel.io/
Now if your only definition of embedded are constrained micro-controllers that aren't even able to cope with standard ISO C89 without having additional proprietary compiler extensions, then naturally everything else are just miniature desktop computers.
Meanwhile some forward looking companies keep on delivering innovative products.
https://os.mbed.com/javascript-on-mbed/
On Samsung's SmartThings embedded devices
https://smartthings.developer.samsung.com/docs/index.html
SigFox customized IoT solutions
https://build.sigfox.com/
Or on the other side from ARM mbed's offering, a beefy ARM 580 MHz with 96 MB (RAM + Flash), using JavaScript alongside Rust.
https://tessel.io/
Now if your only definition of embedded are constrained micro-controllers that aren't even able to cope with standard ISO C89 without having additional proprietary compiler extensions, then naturally everything else are just miniature desktop computers.
Meanwhile some forward looking companies keep on delivering innovative products.
Parent poster wondered if there were such embedded applications that can't be done with javascript. And I listed some. That doesn't mean i'm implying that javascript can't be use on a MCU! Where did you get that idea from?
I don't think I ever gave a definition of an embedded system, but you can quote me to prove me wrong. A PC embedded in a slot machine for instance can also be called an embedded system. A more important distinction is whether the system must run in real time or not. And you can have a beast of a microprocessor and you will not be able to do low level real time processing with it in javascript.
I don't think I ever gave a definition of an embedded system, but you can quote me to prove me wrong. A PC embedded in a slot machine for instance can also be called an embedded system. A more important distinction is whether the system must run in real time or not. And you can have a beast of a microprocessor and you will not be able to do low level real time processing with it in javascript.
> RAII ownership semantics (I.e. std::unique_ptr)
RAII is baad for performance. Really bad. It's an OOP programming style that can be used to glew a few high-level constructs together. But if you use it on the lower levels, you will miss out on a lot of (systemic) optimization possibilities, because RAII gives you piecemeal construction/destruction pairs.
constexpr/generic programming might have a few nice applications, but in most places they're used without a real need (from what I've seen), while leading to extremely slow compiles. (Those in turn lead to longer turnaround times, worse program quality, worse efficiency...).
std::array, like std::vector, is some nice sugar compared to the C primitives, but if you use them at API boundaries you will get bad coupling effects.
The C++ programmers that I look up to all write basically C with maybe a few select C++ features. Many never write "class" and never include C++ headers. Some just write plain C.
RAII is baad for performance. Really bad. It's an OOP programming style that can be used to glew a few high-level constructs together. But if you use it on the lower levels, you will miss out on a lot of (systemic) optimization possibilities, because RAII gives you piecemeal construction/destruction pairs.
constexpr/generic programming might have a few nice applications, but in most places they're used without a real need (from what I've seen), while leading to extremely slow compiles. (Those in turn lead to longer turnaround times, worse program quality, worse efficiency...).
std::array, like std::vector, is some nice sugar compared to the C primitives, but if you use them at API boundaries you will get bad coupling effects.
The C++ programmers that I look up to all write basically C with maybe a few select C++ features. Many never write "class" and never include C++ headers. Some just write plain C.
Curious, do you look up to Mike Acton?
I've never seen him programming, but he seems like a very knowledgeable guy. I find a lot to agree with in his talks.
Yeah, because now he is one of the devs leading the High Performance C# effort at Unity.
People do change their mind.
People do change their mind.
I know, and I can't follow your conclusions. (Btw. come to Handmade Seattle, he will give a talk there ;->)
For someone that used to advocate C, did not like C++ features, I find peculiar that he now stands behind C#.
Yes, I do know that HPC# is somehow constrained, yet even as C# subset, it still supports quite a few features that Mike wasn't so found of having to deal with in C++, as per his CppCon talk.
Yes, I do know that HPC# is somehow constrained, yet even as C# subset, it still supports quite a few features that Mike wasn't so found of having to deal with in C++, as per his CppCon talk.
You mean like a Commodore 64?
https://youtu.be/zBkNBP00wJE
https://youtu.be/zBkNBP00wJE
Yawn.
Yes. But even C was considered too slow for something like a Motorola 68000 back in the day. Of course it's possible.
Besides, C++ can be just as performant if you ignore a huge proportion of its language features and abstractions, but what's the point then? Just use C.
And this is precisely why embedded engineers do so.
Yes. But even C was considered too slow for something like a Motorola 68000 back in the day. Of course it's possible.
Besides, C++ can be just as performant if you ignore a huge proportion of its language features and abstractions, but what's the point then? Just use C.
And this is precisely why embedded engineers do so.
The point is that C will never have proper arrays, strings, bounds checking, real constants instead of magic defines, compile time programming, modules.
Yet somehow it has stayed relevant for more than 45 years.
Go figure.
Go figure.
One cannot teach new tricks to old dogs.
What has made relevant was the rise of FOSS UNIX clones and the GNU guidelines for using C.
All major desktop OSes were already on the path of adopting C++ frameworks. BeOS, Windows, OS/2, Mac OS.
Thankfully C has been shown the door on Apple, Google, Microsoft, Sony, Nintendo and ARM platforms.
Others will follow.
FOSS UNIX clones and old time embedded developers are the only ones keeping C relevant.
What has made relevant was the rise of FOSS UNIX clones and the GNU guidelines for using C.
All major desktop OSes were already on the path of adopting C++ frameworks. BeOS, Windows, OS/2, Mac OS.
Thankfully C has been shown the door on Apple, Google, Microsoft, Sony, Nintendo and ARM platforms.
Others will follow.
FOSS UNIX clones and old time embedded developers are the only ones keeping C relevant.
codesushi42(1)
I am pretty sure the article didn't claim that C won't always have it's place – alone for all the existing codebases it makes sense to use it. If people still learn latin many centuries after the fall of Rome, why would they ever stop to deal with C?
I am somebody who uses C on embedded but really likes the way Rust does things too. Sometimes my naive Rust implementations outperformed my optimized C code very easilly (maybe I am not good enough, who knows). What I especially liked are the zero cost abstractions, which allows you to keep that cost you were talking about very low.
And if I like to do stuff on my own I just do it inside an unsafe block and I am very close to what I could/would do in C.
Even if Rust would fade away it teached me a few hard lessons that definitly improved my C programming.
I am somebody who uses C on embedded but really likes the way Rust does things too. Sometimes my naive Rust implementations outperformed my optimized C code very easilly (maybe I am not good enough, who knows). What I especially liked are the zero cost abstractions, which allows you to keep that cost you were talking about very low.
And if I like to do stuff on my own I just do it inside an unsafe block and I am very close to what I could/would do in C.
Even if Rust would fade away it teached me a few hard lessons that definitly improved my C programming.
That is fair. I am responding to the click baity title of the article.
C has been declared dead and replaced so many times over the years that I have lost count. It is not going to be replaced enitrely by Rust, and definitely not in the embedded world. C will not be the new assembly, it will continue to be C.
C++ could be an entirely different story.
C has been declared dead and replaced so many times over the years that I have lost count. It is not going to be replaced enitrely by Rust, and definitely not in the embedded world. C will not be the new assembly, it will continue to be C.
C++ could be an entirely different story.
C will be replaced, as soon at the last dev that keeps religiously using C89 with language extensions retires.
What's the cost? If you're not afraid of `unsafe`, you can do in Rust exactly the same stuff you do in C, while still having strong guarantees for all your safe code. You might argue that fighting the borrow checker has development costs, but the memory issues that it prevents also have a development cost.
Can you show some sample code in Rust with this memory management cost that C doesn't have?
People who say "C is the new assembly" don't understand why people moved from assembly to C.
Assembly is bad because it is typeless. Because it is typeless, there is no way to check for program correctness. Rust is okay because it doesn't require garbage collection and types, but the best language would be C with a highly-elaborate strong type system and no closures or other unsuitable concepts for systems programming.
Assembly is bad because it is typeless. Because it is typeless, there is no way to check for program correctness. Rust is okay because it doesn't require garbage collection and types, but the best language would be C with a highly-elaborate strong type system and no closures or other unsuitable concepts for systems programming.
Why are closures unsuitable for systems programming? Rust does them really well, not even allocating in the most common cases.
Closure often have really weird lifetimes, so typically they've ended up implemented either with GC of some sort (possibly refcounting), or required such exceedingly careful handling that the cost/benefit analysis gets a big fat lump of "cost" that isn't a problem in most other environments because trying to track the transfer of ownerships across scopes, up and down stacks, and perhaps even between threads is just insane. Typically the entire purpose of the closure is precisely to inject some code's logic into a quite distant other bit of code, so the problem is not amenable to normal techniques around rearranging things to be on the stack and so on. The very nature of a closure-type abstraction in C is go to stomping right over the generally-fragile informal, unsupported organization of what code is responsible for what memory.
I once was working with libpurple in a glib environment that made heavy use of segfaults in C. I mean closures. Closures in C that made it really easy to segfault when calling to a closure that had been freed, or really easy to leak if you didn't free them. I encountered situations where the common paths used by the "standard UI" for libpurple worked, but even when I was quite sure I was handling one of my closures correctly, the system would crash because it still free'd my closure, even though it should have been my responsibility to do so, and so on.
Prior to Rust, I'd agree with the common wisdom that closures and systems programming had gone together poorly. It's a demonstration of the power of Rust's approach to the matter that closures become practical in system programming with it; that had not previously been the case.
I once was working with libpurple in a glib environment that made heavy use of segfaults in C. I mean closures. Closures in C that made it really easy to segfault when calling to a closure that had been freed, or really easy to leak if you didn't free them. I encountered situations where the common paths used by the "standard UI" for libpurple worked, but even when I was quite sure I was handling one of my closures correctly, the system would crash because it still free'd my closure, even though it should have been my responsibility to do so, and so on.
Prior to Rust, I'd agree with the common wisdom that closures and systems programming had gone together poorly. It's a demonstration of the power of Rust's approach to the matter that closures become practical in system programming with it; that had not previously been the case.
Right, there's definitely all sorts of issues with closures (esp async ones) if you don't have lifetime analysis. But Rust is the only systems programming language that I take seriously.
(0..1000).map(|x| x*2).sum()
mov eax, 999000
Interesting, because clang does support closures in C, thanks to Apple.
That's not why people moved from assembly to C, C is portable, assembly is CPU specific, the move to C was about abstraction to a higher level language in the name of portability to multiple processors, not about verifying program correctness.
At Bell Labs, on the outside people where already using plenty of Algol and PL/I variants.
Rust has a npm package problem.
For reference: https://twitter.com/bascule/status/1166583003021283333
For reference: https://twitter.com/bascule/status/1166583003021283333
This is FUD. Almost all of those dependencies exist only to support Redox, an OS you likely don't use. If you actually compiled it for Linux or Windows, it would have 3 dependencies. One of the people replying to that tweet actually attached a screenshot of this.
Since you commented "I don't actually care about Rust, I'm only farming karma to downvote posts on this trash news aggregator", I hope your bad faith attempt to spread FUD fails.
Since you commented "I don't actually care about Rust, I'm only farming karma to downvote posts on this trash news aggregator", I hope your bad faith attempt to spread FUD fails.
Those dependencies still get downloaded and shipped by tools like cargo vendor, because cargo vendor has no target specific vendoring support [1]. The dependencies still bloat up the Cargo.lock files and thus probably have a negative impact on Cargo resolution graph creation (which is an NP complete problem). And to top this, the code isn't even used on Redox [2].
[1]: https://github.com/rust-lang/cargo/issues/7058
[2]: https://gitlab.redox-os.org/redox-os/users/issues/26
[1]: https://github.com/rust-lang/cargo/issues/7058
[2]: https://gitlab.redox-os.org/redox-os/users/issues/26
If you read into it a bit more, those are optional dependencies and if you drop support for redox OS while on windows you lose all but five of the dependencies and those are a config dependency, libc, and winapi and winapi architecture packages (32bit support + 64 bit support). On Unix/Linux, it only depends on the config dep and libc.
But also has a solution: https://github.com/crev-dev/cargo-crev
As great as that tool is (I do think it's good) I wouldn't call that a solution, I'd call that a mitigation. If the ecosystem is terrible (dependency-bloat wise), then a tool like that doesn't make a big impact.
I think the biggest problem crev has is adoption. But if it solves that, then I disagree that it doesn't help with the dependency bloat problem. It would serve as pressure against adding dependencies, because with crev, adding dependencies becomes a lot more work because of the code review system. That in and of itself may be enough pressure to reduce the dependency bloat problem.
It's a big if---especially adoption---but it makes sense to me.
It's a big if---especially adoption---but it makes sense to me.
Something that requires a lot of work sounds like something people will be hesitant to use.
Hence my acknowledgment that adoption is the biggest problem to overcome for crev. :-)
I hope this takes off. In this really good talk https://www.hillelwayne.com/talks/what-we-know-we-dont-know/ Hillel mentions that (I'm paraphrasing) one metric that positively affects code quality is code review.
This is a solution to a problem that shouldn't have happened in the first place.
It's a naturally occurring phenomena when publishing packages is easy. There are various ways to address it:
* Make it harder to publish
* Make it easier to find quality packages
I'd prefer the latter.
* Make it harder to publish
* Make it easier to find quality packages
I'd prefer the latter.
[deleted]
The alternatives are:
* Have every single common thing live in a standard library which makes them stagnate, slows down language development and so on.
* Have people roll their own half-baked, half-broken code every time.
I stand firmly that big organic ecosystem of packages and thin library is the right way, and all we need is just better tools to manage trust and quality.
https://twitter.com/dpc_pw/status/1166754923381280768
* Have every single common thing live in a standard library which makes them stagnate, slows down language development and so on.
* Have people roll their own half-baked, half-broken code every time.
I stand firmly that big organic ecosystem of packages and thin library is the right way, and all we need is just better tools to manage trust and quality.
https://twitter.com/dpc_pw/status/1166754923381280768
1. If it is really common, then yes. Why not? It does not necessarily stagnate or slow down anything. People working on current official crates can do that were it in the standard library.
2. I am not so sure about that. There are many crates that are just a few lines of code. They are not that difficult to get right, but the mentality dictates that you import a crate instead of writing those "easy" one to ten lines.
I am not sure it is the right way of doing it because it suffers from the same problems npm does. Making (and maintaining) more tools to patch up these issues is not exactly a great solution to me. What if these tools have issues, too? Do we create tools to patch up the issues of these other tools, ad infinitum?
Few lines of code you write yourself, or: https://pbs.twimg.com/media/EDCMkVMXkAIwdI1.png
2. I am not so sure about that. There are many crates that are just a few lines of code. They are not that difficult to get right, but the mentality dictates that you import a crate instead of writing those "easy" one to ten lines.
I am not sure it is the right way of doing it because it suffers from the same problems npm does. Making (and maintaining) more tools to patch up these issues is not exactly a great solution to me. What if these tools have issues, too? Do we create tools to patch up the issues of these other tools, ad infinitum?
Few lines of code you write yourself, or: https://pbs.twimg.com/media/EDCMkVMXkAIwdI1.png
The Rust standard library endeavours to do as little as possible and even then, there have been cases where something was added and later folks realised it was a mistake. For example std::sync::mpsc. No one is recommended to use it (crossbeam crate is the preferred alternative). Getopt used to be in the standard library (clap crate is better).
Obviously mpsc shouldn't be in the standard library but it can't be removed without breaking backwards compat - essentially stagnating. So a suboptimal solution remains.
Rust's solution is to have an official "cookbook" that shows how to use these crates to accomplish common tasks. For example, crossbeam is featured here - https://rust-lang-nursery.github.io/rust-cookbook/concurrenc.... In case a better library comes along, it's simple to update the cookbook. In this scenario's no one's programs are broken and the standard library remains slim.
Do you disagree with the cookbook approach?
Obviously mpsc shouldn't be in the standard library but it can't be removed without breaking backwards compat - essentially stagnating. So a suboptimal solution remains.
Rust's solution is to have an official "cookbook" that shows how to use these crates to accomplish common tasks. For example, crossbeam is featured here - https://rust-lang-nursery.github.io/rust-cookbook/concurrenc.... In case a better library comes along, it's simple to update the cookbook. In this scenario's no one's programs are broken and the standard library remains slim.
Do you disagree with the cookbook approach?
> Why not? It does not necessarily stagnate or slow down anything. People working on current official crates can do that were it in the standard library.
It is much, much, much harder to work on the standard library than an external package. If you think Rust compile times are poor, you should try building the compiler (which is needed to work on the standard library.) This alone slows down development a bunch. You also have to deal with the PR queue, RFCs for new functionality, etc. It's more heavy weight for good reason, but it's also heavy weight.
It also means that the API must be set in stone forever.
It is much, much, much harder to work on the standard library than an external package. If you think Rust compile times are poor, you should try building the compiler (which is needed to work on the standard library.) This alone slows down development a bunch. You also have to deal with the PR queue, RFCs for new functionality, etc. It's more heavy weight for good reason, but it's also heavy weight.
It also means that the API must be set in stone forever.
No, it means that a process must be put in place to deprecate them, and remove them after a couple of major releases.
Even Java has started to actually remove deprecated APIs.
Even Java has started to actually remove deprecated APIs.
Sure, If the rules were different, things would be different. We’re not doing that, nor do we foresee doing that any time soon, if ever.
Is there any language / distro that doesn't?
In the end you need gatekeeping. That's a security-critical function, and it's a people function, not a function of language.
In the end you need gatekeeping. That's a security-critical function, and it's a people function, not a function of language.
Interestingly the situation is somewhat similar to Javascript: a very lean standard library, paired with a great package manager that makes sharing code simple.
I agree that the dependency problem is getting continuously worse though. An important factor is people splitting up crates into multiple sub-crates, proc macro crates (which have to live in a separate crate), and generally having a much too cavalier attitude about adding dependencies.
The last year has been particularly bad, but on the flip side there is also increased awareness and motivation to counter this.
Rust will never have a stdlib like Go, with http server/client, compression and crypto algorithms, ...
But I do think that it needs to be conservatively extended with some core primitives that are missing. A problem is the still heavily evolving language. A few upcoming features could change idiomatic API design, so putting anything in std now could be a big mistake in the long term.
I agree that the dependency problem is getting continuously worse though. An important factor is people splitting up crates into multiple sub-crates, proc macro crates (which have to live in a separate crate), and generally having a much too cavalier attitude about adding dependencies.
The last year has been particularly bad, but on the flip side there is also increased awareness and motivation to counter this.
Rust will never have a stdlib like Go, with http server/client, compression and crypto algorithms, ...
But I do think that it needs to be conservatively extended with some core primitives that are missing. A problem is the still heavily evolving language. A few upcoming features could change idiomatic API design, so putting anything in std now could be a big mistake in the long term.
So, a single example enough to state there's a problem?
I admit - Rust is on the easier side of packaging, which tends towards proliferation of dependencies, but the general feeling I get right now from rust communities is a deliberate restraint when it comes to dependencies.
I admit - Rust is on the easier side of packaging, which tends towards proliferation of dependencies, but the general feeling I get right now from rust communities is a deliberate restraint when it comes to dependencies.