Removing garbage collection from the Rust language (2013)(pcwalton.github.io)
pcwalton.github.io
Removing garbage collection from the Rust language (2013)
http://pcwalton.github.io/_posts/2013-06-02-removing-garbage-collection-from-the-rust-language.html
212 comments
> even if I do get frustrated with mind-numbing Async issues from time to time.
A noticeable portion of async issues come from the fact that a lot of people use Tokio's multithreaded async runtime. Tokio allows you to mix async and native theads, which is both a virtuoso technical accomplishment and also a bit ridiculous.
If you use Tokio's single-threaded runtime, things get simpler.
The remaining async challenges are mostly the usual "Rust tax", turned up to 11. Rust wants you to be painfully aware that memory allocations are expensive and that sharing memory in a complex concurrent system is risky.
In sync Rust, the usual advice is "Don't get too tricky with lifetimes. Use `clone` when you need to."
In async Rust without native threads, the rules are something like:
1. Boxing your futures only costs you a heap allocation, and it vastly simplifies many things.
2. If you want a future to remain around while you do other stuff, have it take ownership of all its parameters.
Where people get in the most trouble is when they say, "I want to mix green threads and OS threads willy-nilly, and I want to go to heroic lengths to never call `malloc`." Rust makes that approach look far too tempting. And worse, it requires you to understand and decide whether you're taking that approach.
But if you remember "Box more futures, own more parameters, and consider using a single-threaded runtime" then async Rust offers some pretty unique features in exchange for a pretty manageable amount of pain.
Also, seriously, more people should consider Kotlin. It has many Rust-like features, but it has a GC. And you don't need to be constantly aware of the tradeoffs between allocation and sharing, if that's not a thing you actually care about.
A noticeable portion of async issues come from the fact that a lot of people use Tokio's multithreaded async runtime. Tokio allows you to mix async and native theads, which is both a virtuoso technical accomplishment and also a bit ridiculous.
If you use Tokio's single-threaded runtime, things get simpler.
The remaining async challenges are mostly the usual "Rust tax", turned up to 11. Rust wants you to be painfully aware that memory allocations are expensive and that sharing memory in a complex concurrent system is risky.
In sync Rust, the usual advice is "Don't get too tricky with lifetimes. Use `clone` when you need to."
In async Rust without native threads, the rules are something like:
1. Boxing your futures only costs you a heap allocation, and it vastly simplifies many things.
2. If you want a future to remain around while you do other stuff, have it take ownership of all its parameters.
Where people get in the most trouble is when they say, "I want to mix green threads and OS threads willy-nilly, and I want to go to heroic lengths to never call `malloc`." Rust makes that approach look far too tempting. And worse, it requires you to understand and decide whether you're taking that approach.
But if you remember "Box more futures, own more parameters, and consider using a single-threaded runtime" then async Rust offers some pretty unique features in exchange for a pretty manageable amount of pain.
Also, seriously, more people should consider Kotlin. It has many Rust-like features, but it has a GC. And you don't need to be constantly aware of the tradeoffs between allocation and sharing, if that's not a thing you actually care about.
Maybe I should try smol instead of Tokio. I'm guessing a lot of libraries are made to fit with Tokio, but I imagine for a typical desktop app, a multi-threaded runtime is total overkill. Plus I could just run multiple runtimes if I want.
Like I suppose if I needed a blocking task that was also async, I could send that to a thread pool and then internally spawn a worker async runtime. It's a thinker.
Like I suppose if I needed a blocking task that was also async, I could send that to a thread pool and then internally spawn a worker async runtime. It's a thinker.
A single threaded Tokio runtime still has a separate thread pool for heavy tasks. Maybe block_on? I can't remember where I read that tho.
Edit: not block_on, but spawn_blocking. See https://www.reddit.com/r/rust/comments/16ebdi1/comment/jzy7f...
Edit: not block_on, but spawn_blocking. See https://www.reddit.com/r/rust/comments/16ebdi1/comment/jzy7f...
Or, you know, that single threaded runtime that OP just mentioned. ;P
> If you use Tokio's single-threaded runtime, things get simpler.
A major issue when dealing with the tokio runtime is Send bounds requirements, I don’t think the single-threaded runtime changes anything because these are API-level issues. You can use spawn_local to avoid migrations but you can do that on the multithreaded runtime just as well.
And then a lot of tools and libraries which get layered over tokio (or assume a tokio environment) will require send futures anyway.
A major issue when dealing with the tokio runtime is Send bounds requirements, I don’t think the single-threaded runtime changes anything because these are API-level issues. You can use spawn_local to avoid migrations but you can do that on the multithreaded runtime just as well.
And then a lot of tools and libraries which get layered over tokio (or assume a tokio environment) will require send futures anyway.
LocalSet does not require Send. https://docs.rs/tokio/latest/tokio/task/struct.LocalSet.html
EDIT: Sorry to join in the multiple responses. Clarity for others: "current_thread" by itself does not relax Send, and many libraries aren't configurable to make everything use LocalSet.
EDIT: Sorry to join in the multiple responses. Clarity for others: "current_thread" by itself does not relax Send, and many libraries aren't configurable to make everything use LocalSet.
I mentioned spawn_local, which is basically the same feature as localset (you can only spawn_local within the scope of a localset).
[deleted]
That's incorrect, the Send bounds are only there in the multithreaded runtime (because it might need to Send a task across threads). The single threaded runtime will never do that, so its futures don't need to be Send.
Somewhat relatedly, a major (early) stumbling block for me with the multithreaded runtime was trying to keep references across await points. While boxing them is always an option, things also got much easier when I realized that while &T is only Send if T is sync, _&mut T is send if T is send_.
Somewhat relatedly, a major (early) stumbling block for me with the multithreaded runtime was trying to keep references across await points. While boxing them is always an option, things also got much easier when I realized that while &T is only Send if T is sync, _&mut T is send if T is send_.
> That's incorrect
Is it?
> the Send bounds are only there in the multithreaded runtime (because it might need to Send a task across threads).
Tokio is interacted with using free functions which dynamically look up the current runtime, they could not have a different signature even if tokio had different runtime types, which it does not.
> The single threaded runtime will never do that, so its futures don't need to be Send.
Please do point to the ?Send spawn (not spawn_local) which supposedly exists for the current_thread runtime. You can't even spawn_local at the toplevel of the current_thread runtime.
Is it?
> the Send bounds are only there in the multithreaded runtime (because it might need to Send a task across threads).
Tokio is interacted with using free functions which dynamically look up the current runtime, they could not have a different signature even if tokio had different runtime types, which it does not.
> The single threaded runtime will never do that, so its futures don't need to be Send.
Please do point to the ?Send spawn (not spawn_local) which supposedly exists for the current_thread runtime. You can't even spawn_local at the toplevel of the current_thread runtime.
Unless I'm mistaken, switching to a single threaded tokio runtime made Send requirements go away for me.
Changing the type of runtime doesn't change the API at all.
Even in a single threaded runtime spawn requires Send and you need to use LocalSet + spawn_local for !Send futures.
Even in a single threaded runtime spawn requires Send and you need to use LocalSet + spawn_local for !Send futures.
I wasn't using spawn. It's an I/O bound program with a big select! loop at it's core. Rust spit out errors about Send and I made them go away by switching to flavor = "current_thread".
Would be nice if you could find it again, because from what I know of tokio I'd assume you made other unrelated changes which fixed the issue: tokio only has one runtime type, the flavour's effects are internal, the top-level future (run by Runtime::block_on) is always !Send, but from that you can only run Send futures (via spawn and spawn_blocking), unless you create a LocalSet.
current_thread doesn't even create an implicit LocalSet, if you try to `spawn_local` from the top-level of a current_thread runtime you get a panic, exactly like a multi_thread runtime.
current_thread doesn't even create an implicit LocalSet, if you try to `spawn_local` from the top-level of a current_thread runtime you get a panic, exactly like a multi_thread runtime.
What's the problem with Send bounds? So you use Arc instead of Rc.
The main problem is that for a future to be send, everything held across an await point has to be Send. Which is quite constraining and annoying, especially because this often does not play well with trait objects or `impl Trait` as people will commonly not think to add trait extra trait bounds.
And of course it's an accumulation of trait bounds on everything, which makes for a downgrade in readability.
And of course it's an accumulation of trait bounds on everything, which makes for a downgrade in readability.
If you're counting on people to drop down into a single-threaded runtime when using async constructs, you've really lost the mark. Parallelism is the user's goal most of the time, right? Perhaps in web context to avoid runaway thread spawning and slow loris attacks it's not (the main selling point of node.js), but otherwise people want to go fast.
>Also, seriously, more people should consider Kotlin.
I like Kotlin, but I find its coroutine machinations to be far more confusing that just plain threads (over which Java already had/has some nice quality of life abstractions). And debugging broken Kotlin coroutine code is hell. You will not get a normal-looking stack trace when things go wrong.
>Also, seriously, more people should consider Kotlin.
I like Kotlin, but I find its coroutine machinations to be far more confusing that just plain threads (over which Java already had/has some nice quality of life abstractions). And debugging broken Kotlin coroutine code is hell. You will not get a normal-looking stack trace when things go wrong.
The main goal of using async for me is to not have to handle all the IO wait state machines myself. It does a pretty good job of that, and as long as my program consists of a bunch of tasks concurrently waiting for IO to finish, a single thread is perfectly fine.
Take the browser. It gets an incredible amount of performance out people’s devices even though 99% of javascript is bound to a single thread. The way they accomplish that is via their asynchronous architecture. So, not always.
> Parallelism is the user's goal most of the time, right?
I'm not sure you're right about that.
I'm not sure you're right about that.
In the general sense? Probably not. In the context of Rust users? It certainly begs the question: why use Rust if you aren't going for speed? Just write it in Node/Go/JVM-lang if performance doesn't matter and you just want an event loop that looks like threads.
> If you use Tokio's single-threaded runtime, things get simpler.
I don't do a lot of low level work, but coming out of Elixir / Erlang I would expect a greenlet system to manage the # of threads based on the hardware its running on. I.e. not single threaded or "you manage the threads too" but "a standard piece of code spreads your greenlet processes between N managed hardware threads where N is determined by hardware + settings." Is that not a thing that the Rust async libraries support?
I don't do a lot of low level work, but coming out of Elixir / Erlang I would expect a greenlet system to manage the # of threads based on the hardware its running on. I.e. not single threaded or "you manage the threads too" but "a standard piece of code spreads your greenlet processes between N managed hardware threads where N is determined by hardware + settings." Is that not a thing that the Rust async libraries support?
Elixir/Erlang is actually a really interesting case, because:
1. It has a garbage collector, and
2. It relies even more heavily on immutable data than Rust.
This means that the Beam VM can seamlessly move green threads around CPUs and preempt them at arbitrary points, all without breaking code. You never need to know who "owns" something, and you never need to worry about another process mutating it while you're looking at it. The Beam VM is amazing.
Tokio operates under different constraints: There's no garbage collector, ownership can be "lent" to other code, and mutable state exists in a carefully controlled fashion. Despite this, Tokio absolutely supports spreading green threads across all your CPUs and moving them as needed using a work-stealing scheduler (IIRC). But this only works if all your closures are `Send` (safe to move between CPUs). And any closure that outlives the creating code must generally be `'static` (it does not refer to references borrowed from its creator's scope).
Oh, and Rust keeps trying create and manage your async & multithreaded processes without trying to allocate heap memory at all. Unless you explicitly ask it to allocate memory. Which you often should.
It is totally possible to make your closures `Send + 'static`. I maintain several production Rust programs which do that, no problem. But doing so requires understanding a moderate amount of Rust and paying a "cognitive tax" by making a bunch of extra decisions about how to represent things. And I think that cognitive tax is a unwise tradeoff for many problems and teams. But I'm still very happy with my async Rust projects, because they get a lot of value out of fast, memory-efficient async code.
1. It has a garbage collector, and
2. It relies even more heavily on immutable data than Rust.
This means that the Beam VM can seamlessly move green threads around CPUs and preempt them at arbitrary points, all without breaking code. You never need to know who "owns" something, and you never need to worry about another process mutating it while you're looking at it. The Beam VM is amazing.
Tokio operates under different constraints: There's no garbage collector, ownership can be "lent" to other code, and mutable state exists in a carefully controlled fashion. Despite this, Tokio absolutely supports spreading green threads across all your CPUs and moving them as needed using a work-stealing scheduler (IIRC). But this only works if all your closures are `Send` (safe to move between CPUs). And any closure that outlives the creating code must generally be `'static` (it does not refer to references borrowed from its creator's scope).
Oh, and Rust keeps trying create and manage your async & multithreaded processes without trying to allocate heap memory at all. Unless you explicitly ask it to allocate memory. Which you often should.
It is totally possible to make your closures `Send + 'static`. I maintain several production Rust programs which do that, no problem. But doing so requires understanding a moderate amount of Rust and paying a "cognitive tax" by making a bunch of extra decisions about how to represent things. And I think that cognitive tax is a unwise tradeoff for many problems and teams. But I'm still very happy with my async Rust projects, because they get a lot of value out of fast, memory-efficient async code.
That makes a lot of sense! I guess I was imagining that a greenlet system would have a kind of 'light vm' by default, but you're absolutely right that the constraints of Erlang make that a LOT easier in their VM. I wasn't really thinking about the the locality difficulties of managing process resources (or I guess I was imagining that the meta-info collected by Rust would help but I guess it's mostly not runtime).
By default, Tokio uses multi-threaded runtime, but you can force it to use the single-threaded one: https://docs.rs/tokio/latest/tokio/attr.main.html#current-th...
> consider Kotlin
Or C#. C# has async/await (it originated there) and does have a multithreaded event loop, but due to having GC is just a lot easier to work with than Rust+Tokio.
If you're in the "don't colour my function" camp, GoLang is also worth throwing in the mix.
Or C#. C# has async/await (it originated there) and does have a multithreaded event loop, but due to having GC is just a lot easier to work with than Rust+Tokio.
If you're in the "don't colour my function" camp, GoLang is also worth throwing in the mix.
C# is probably what most teams want but don't know it.
- Language is very, very similar to TypeScript. If you're already doing TS on the backend with Node (or even JS), it's a very small lift to C#
- Very rich standard libraries and first party libraries; reduces the need to import a bunch of third party code
- .NET minimal web APIs are very similar to Express now and perhaps even easier since you don't need to import anything to get a microservice up and running
- .NET AOT with .NET 8 will dramatically improve the cold-start for use cases like serverless functions (I find the cold start already pretty good with .NET 7 on Google Cloud Run with the CPU Boost feature turned on).
- C# has a lot of functional features as a result of F#
- Compiles fast and has hot reload via `dotnet watch`
- Provides access to low level primitives where extra performance is needed100%.
> NET AOT with .NET 8 will dramatically improve the cold-start
Don't get your hopes up on AOT. It still has lots of limitations, reflection being a big one. ASP.NET initialization relies heavily on reflection so it will be a while before we can have AOT compiled, HTTP microservices.
> I find the cold start already pretty good with .NET 7 on Google Cloud Run
Thanks for the tip! We're mostly using GKE but have a few services on Cloud Run that might benefit. Do you use the "always allocate CPU" option? We're seeing some memory creep we suspect would be solved by giving the GC cycles when a request isn't in flight.
> NET AOT with .NET 8 will dramatically improve the cold-start
Don't get your hopes up on AOT. It still has lots of limitations, reflection being a big one. ASP.NET initialization relies heavily on reflection so it will be a while before we can have AOT compiled, HTTP microservices.
> I find the cold start already pretty good with .NET 7 on Google Cloud Run
Thanks for the tip! We're mostly using GKE but have a few services on Cloud Run that might benefit. Do you use the "always allocate CPU" option? We're seeing some memory creep we suspect would be solved by giving the GC cycles when a request isn't in flight.
> ASP.NET initialization relies heavily on reflection
.NET 8 AOT is specifically focused on ASP.NET; they've switched over many of the reflection paths to use source generation instead. I have hopes that some time during the .NET 8 lifecycle or .NET 9 horizon, we'll see full support: https://learn.microsoft.com/en-us/aspnet/core/fundamentals/n...
> Do you use the "always allocate CPU" option?
The magic of Google Cloud and Cloud Run is really that you don't need to be always on.
Don't forget:
- unlike Java, C# supports value types so you can avoid gratuitous overhead or ugly code when all you need is a simple wrapper
And Haskell. The only 2 languages I know that got this right.
> Tokio allows you to mix async and native theads, which is both a virtuoso technical accomplishment and also a bit ridiculous.
Can you describe what issues you’re referring to? I use the multi-threaded Tokio runtime, I’ve not noticed any overhead with that in terms of development vs. single threaded. Also, multi-threaded async runtimes are generally what you want to make sure you don’t have any single tasks blocking others that could make progress in parallel.
Can you describe what issues you’re referring to? I use the multi-threaded Tokio runtime, I’ve not noticed any overhead with that in terms of development vs. single threaded. Also, multi-threaded async runtimes are generally what you want to make sure you don’t have any single tasks blocking others that could make progress in parallel.
"Also, multi-threaded async runtimes are generally what you want to make sure you don’t have any single tasks blocking others that could make progress in parallel."
Is there an advantage to multi-threaded async if you're IO-bound? If you want a bunch of concurrent system calls or network requests it seems like single-threaded async can handle that pretty nicely ala Node.js.
Is there an advantage to multi-threaded async if you're IO-bound? If you want a bunch of concurrent system calls or network requests it seems like single-threaded async can handle that pretty nicely ala Node.js.
Depends on your load, but many I/O bound loads are bigger than what you can do in a single thread.
> moving from a GC’d language
Rust was never actually a GC’d language. It had a smart pointer called Gc (and before that @), but that was only ever implemented as refcounting (according to the changelogs some preliminary work had been done towards a precise GC but AFAIK there was no followup).
This is in large part why it was dropped: it was technically redundant with Arc, and could give users the wrong impression, and could always be added back later if that made sense.
Rust was never actually a GC’d language. It had a smart pointer called Gc (and before that @), but that was only ever implemented as refcounting (according to the changelogs some preliminary work had been done towards a precise GC but AFAIK there was no followup).
This is in large part why it was dropped: it was technically redundant with Arc, and could give users the wrong impression, and could always be added back later if that made sense.
The original design by Graydon envisioned a real GC, but one per task/actor instead of global, with limited message passing between threads.
http://venge.net/graydon/talks/intro-talk-2.pdf
http://venge.net/graydon/talks/intro-talk-2.pdf
Maybe but that was never actually a thing. It was not even a thing which got moved away from like type states or internat iterators, it never was.
Technically from CS point of view, refcounting is a GC implementation algorithm.
Yes, but does that make circa-2013 Rust a "garbage collected language?" It seems to me that when we talk about GC'ed languages, we're talking about languages where the heap is managed implicitly.
If I understand it right, @ is explicitly invoked by the user, but the implementation is embedded within the language. With Arc/Rc, there are Deref/Drop implementations somewhere in the stdlib that do the reference counting.
If I understand it right, @ is explicitly invoked by the user, but the implementation is embedded within the language. With Arc/Rc, there are Deref/Drop implementations somewhere in the stdlib that do the reference counting.
I fail to see what that has to do with anything, unless you’re asserting that current rust is GC’d on the back of arc existing.
Rust as a language is not GC'd, but has a refcounted (that is, GC'd) pointer type if you want/need it.
It has to do everything with the wording on your comment that kind of makes the usual lay man distinction between GC and refcounting, nothing to do with Arc, as using it is manual work anyway.
The point of a garbage connector is to simulate infinite memory[0], which is the point of Arc. So current Rust has GC :)
[0] https://octodon.social/@cwebber/110815313802832972
[0] https://octodon.social/@cwebber/110815313802832972
Refcounting is also just (dumb and slow) garbage collection though.
The main point is that rust was never a ref counted langage, it had a purportedly GC’d opt-in pointer type.
And while refcounting has lower throughput than more advanced forms of garbage collection, it has a much higher reactivity / lower memory overhead, and it integrates much better with other methods of resource management.
And while refcounting has lower throughput than more advanced forms of garbage collection, it has a much higher reactivity / lower memory overhead, and it integrates much better with other methods of resource management.
For anyone interested there was a big discussion on async in rust a few days ago. Rust is on todo list to learn but I found reading some of these comments quite interesting:
https://news.ycombinator.com/item?id=37435515
https://news.ycombinator.com/item?id=37435515
Note also this rebuttal: https://news.ycombinator.com/item?id=37448460
As someone who works with async rust professionally, I wouldn’t let takes like the one you posted dissuade you. My personal opinion is that async rust is easier to work with than in most other languages because of the extra concurrency guarantees that rust gives you. There are some rough edges, but not nearly as many as the link you posted suggests.
As someone who works with async rust professionally, I wouldn’t let takes like the one you posted dissuade you. My personal opinion is that async rust is easier to work with than in most other languages because of the extra concurrency guarantees that rust gives you. There are some rough edges, but not nearly as many as the link you posted suggests.
I wasn't, I actually didn't even read the attached article only the HN comments who seemed to mainly disagree with the article and mentioned quite a lot about the different approaches to async in Rust. I really want to learn it but prioritising up-skilling my DevOps capabilities at the minute so it will have to be next year.
Glad to hear it! I am ~2.5 years into my professional Rust career, ~4.5 years into working with the language, and it's still my favorite. It's certainly not the only great language out there, but it's definitely one of them.
This exactly.
Sure Async Rust requires you to learn a few new things, but the blog post is mostly a rant from someone not very familiar with the topic (like what you'd expect from something called “<X> is a bad language” TBH).
Sure Async Rust requires you to learn a few new things, but the blog post is mostly a rant from someone not very familiar with the topic (like what you'd expect from something called “<X> is a bad language” TBH).
Is there any other languages that let you use async on embedded/bare metal? I think Rust could learn from them. Especially on the ergonomics side. Otherwise, what Rust might currently do is blazing a trail through a forest.
From a net-effect standpoint, Rust's implementation of async/await is fairly similar to protothreads, which were used even on smaller AVR and MSP430 targets where rtos threads weren't practical.
That being said, protothreads were implemented using wildly cursed C macros, and offered none of the safety guardrails that Rust has, nor the ergonomics, and had some insane constraints, like "you can't use local variables at all, only statics" because like rust, they were stackless coroutines that meant you would have no control of the stack across await points.
If you were VERY CAREFUL, you could get the same lightweight concurrency with almost the same fundamental model. Woe be on you if you ever had to debug though.
edit - here's a look at how protothreads were expanded: https://dunkels.com/adam/pt/expansion.html
That being said, protothreads were implemented using wildly cursed C macros, and offered none of the safety guardrails that Rust has, nor the ergonomics, and had some insane constraints, like "you can't use local variables at all, only statics" because like rust, they were stackless coroutines that meant you would have no control of the stack across await points.
If you were VERY CAREFUL, you could get the same lightweight concurrency with almost the same fundamental model. Woe be on you if you ever had to debug though.
edit - here's a look at how protothreads were expanded: https://dunkels.com/adam/pt/expansion.html
What should it learn? If you combine async with Rust’s nested lifetimes, you get quite a bit of complexity from the borrowchecker, period. There is not much else to it.
Vala offers async/yield and compiles down to native code (after being transpiled to C): https://wiki.gnome.org/Projects/Vala/Tutorial#Asynchronous_M...
More info: https://vala.dev/
More info: https://vala.dev/
There is no other language that does what Rust is trying to do. But that is not "using async on embedded/bare metal", this one is easy, jut use a garbage collector.
Hm, I think Pony did try doing what Rust is doing, with an even more advanced system for "borrows" (there's several types of borrow, not just two): https://www.ponylang.io/
Also, Vale is on the way: https://vale.dev/
And D is also working on it: https://dlang.org/blog/2019/07/15/ownership-and-borrowing-in...
Also, Vale is on the way: https://vale.dev/
And D is also working on it: https://dlang.org/blog/2019/07/15/ownership-and-borrowing-in...
>"Is there any other languages that let you use async on embedded/bare metal?"
So you do not know if there are and what kind and other important details.
>"I think Rust could learn from them."
Yet you think
So you do not know if there are and what kind and other important details.
>"I think Rust could learn from them."
Yet you think
It’s worth noting, given the aspirational conclusion, that @T managed pointers were actually equivalent to Rc<T>, since they were only ever implemented with reference counting without cycle collection. (This would have changed had they stayed in the language, but winds changed instead.) The standard library doesn’t ship tracing GC, and it’s still not altogether clear how it would be best to do it. https://manishearth.github.io/blog/2021/04/05/a-tour-of-safe... is useful further reading. I think it’s fair to say that the “ship tracing GC as part of the standard library” ship has sailed, but power and convenience could still warrant language/standard library features to support it, though I doubt anything will ever happen, or at least in the next decade. The focus of the language shifted and was clarified quite a lot between the time of this article and even Rust 1.0 just under two years later.
And only 2 years later Rust 1.0 was released with a borrow checker (which is not mentioned in the blog post). I think one of the reasons why Rust turned out to be so well-designed is that Rust has a very open development approach where people with many different experiences can share their wisdom. Many other languages follow the design of a few people, but Rust is really a language of the internet community.
I don't agree. Before Rust 1.0 the community was really, really tiny. I don't have numbers but it was really a few people discussing the design issues... it was not like now where you literally have thousands of people with different incentives trying to force their views on things, which makes development go much slower than in the pre-1.0 days.
I think the lesson is that, as with any software, having an open discussion between people who are on the same page about things and have similar incentives (with room for disagreements, as the initial author of Rust had plenty of) is greatly beneficial, without a doubt... but once you start getting people with polar opposite views on things, each backed by a substantial faction behind them, things start getting messy. I'm not saying this has happened in Rust, just that it's not a case the more people, the merrier.
I think the lesson is that, as with any software, having an open discussion between people who are on the same page about things and have similar incentives (with room for disagreements, as the initial author of Rust had plenty of) is greatly beneficial, without a doubt... but once you start getting people with polar opposite views on things, each backed by a substantial faction behind them, things start getting messy. I'm not saying this has happened in Rust, just that it's not a case the more people, the merrier.
I'm sure it'd be different for everyone reading this, but for me this story reinforces my hunch that the BDFL model moves faster and satisfies more people, in the sense that its hard to design by committee and harder to satisfy everyone.
I don't feel that Rust is actually well designed. The borrow checker and lack of GC makes it well suited to the tasks that traditionally one would use C or C++ for, and it is definitely safer than those language.
But in terms of overall design it doesn't feel very cohesive, and priority is put in odd places. For example, error handling is an area where people almost always resort to 3rd party libraries. That, to me, having to rely on third party libraries for such basic features is a sign of a serious design flaw. Meanwhile, "clever" things like zero cost map and filter are prioritized and in the language for a long time.
Overall it feels like a language that prioritizes "clever" features at the expense of boring but basic needs. Also any criticism is met with hostility rather than accommodation.
It's a fine language, but I can't say that I love it.
But in terms of overall design it doesn't feel very cohesive, and priority is put in odd places. For example, error handling is an area where people almost always resort to 3rd party libraries. That, to me, having to rely on third party libraries for such basic features is a sign of a serious design flaw. Meanwhile, "clever" things like zero cost map and filter are prioritized and in the language for a long time.
Overall it feels like a language that prioritizes "clever" features at the expense of boring but basic needs. Also any criticism is met with hostility rather than accommodation.
It's a fine language, but I can't say that I love it.
> For example, error handling is an area where people almost always resort to 3rd party libraries.
IME people mostly resort to those libraries just to avoid verbosity, since implementing error types is frankly boring and macros/etc make it a non-issue.
Some have neat context-attachment functionality that can be useful as well, though I've personally never needed it.
IME people mostly resort to those libraries just to avoid verbosity, since implementing error types is frankly boring and macros/etc make it a non-issue.
Some have neat context-attachment functionality that can be useful as well, though I've personally never needed it.
I would consider error handling in rust my favorite of all the lanuages i use by a long margin, im not sure how it would be improved. Python I have to try catch, JS i have to try catch, C++ i have to try catch, Go I need more boilerplate than rust. Swift is close to rust, but without the Map Error logic its less ergonomic.
An incredibly typical scenario is that a function makes several calls that each have their own error. E.g. maybe initialize the database and a web server.
If you want this function to return an error that encompasses any of the sub-error types you are forced to write the most incredible amount of boilerplate that I've ever seen in a language. You need to define a composite error type and then implement all the required traits for it.
I'm not sure how some people avoid this situation, but it's an incredibly common scenario and the only reasonable solution is to use a library like anyhow. Defining your own errors is also heavily boilerplate prone giving rise to things like thiserror.
Why these third party libraries are not made into first class language features is beyond me, but it is an example of very poor design and is typical of what I mean. Rust implemented a solid error handling core, but then just didn't bother with the things that would make the error handling actually good. Fancy over pragmatic.
If you want this function to return an error that encompasses any of the sub-error types you are forced to write the most incredible amount of boilerplate that I've ever seen in a language. You need to define a composite error type and then implement all the required traits for it.
I'm not sure how some people avoid this situation, but it's an incredibly common scenario and the only reasonable solution is to use a library like anyhow. Defining your own errors is also heavily boilerplate prone giving rise to things like thiserror.
Why these third party libraries are not made into first class language features is beyond me, but it is an example of very poor design and is typical of what I mean. Rust implemented a solid error handling core, but then just didn't bother with the things that would make the error handling actually good. Fancy over pragmatic.
I don't understand what you would include in the std lib exactly. Defining error types is just defining regular types with domain-specific error information. Macros to make that less verbose seems like the exact sort of thing you would want to be implemented in third-party libraries.
Except as a newcomer to the language, I didn't know about anyhow or thiserror and spent an immense amount of time figuring out how to solve this problem, and writing all the boilerplate. This is not pragmatic and not ok.
I would say a language like Go is well designed in that it has an overall design philosophy and you can see that throughout the language, including in the limitations which are often intentionally chosen. One part of their philosophy that I really appreciate is their intention to make the language pragmatic in industry settings, and making typical things obvious to newcomers is a huge part of that.
Even if your solution is macros (which may be ok, but can also make the underlying generated code more opaque), it should be part of the standard library and part of the language manual. Making things the "officially approved way of handling a problem" has benefits beyond newcomers too, it gives the entire language more consistency resulting in a far more cohesive design.
I would say a language like Go is well designed in that it has an overall design philosophy and you can see that throughout the language, including in the limitations which are often intentionally chosen. One part of their philosophy that I really appreciate is their intention to make the language pragmatic in industry settings, and making typical things obvious to newcomers is a huge part of that.
Even if your solution is macros (which may be ok, but can also make the underlying generated code more opaque), it should be part of the standard library and part of the language manual. Making things the "officially approved way of handling a problem" has benefits beyond newcomers too, it gives the entire language more consistency resulting in a far more cohesive design.
"The" solution is not macros. The solution is what is provided by the language. If you are calling a bunch of different functions that return different error types then you either have to map them manually (using map_err) or you can create your own error type and implement From<XYZError> to automatically lift into your own error type. I agree that it is a lot of boilerplate which is why there are good libraries for generating the boilerplate with macros. But the underlying mechanism is not really obscure. It is covered quite well in the rust book (https://doc.rust-lang.org/book/ch09-02-recoverable-errors-wi...).
> Meanwhile, "clever" things like zero cost map and filter are prioritized and in the language for a long time.
This is because of what the language is trying to do. Providing map and filter with zero is a priority 0. I think people think Rust had more development manpower than it actually had. Java/C# (and I would bet Go) have had significantly more money poured into them than Rust.
This is because of what the language is trying to do. Providing map and filter with zero is a priority 0. I think people think Rust had more development manpower than it actually had. Java/C# (and I would bet Go) have had significantly more money poured into them than Rust.
I would like to link Graydon's post on how the language turned out in his view, as it seems relevant: https://graydon2.dreamwidth.org/307291.html
> I think one of the reasons why Rust turned out to be so well-designed is that Rust has a very open development approach where people with many different experiences can share their wisdom.
Probably also why Rust has such highly publicized drama, as well. The drama is out in the open, too.
Probably also why Rust has such highly publicized drama, as well. The drama is out in the open, too.
> Rust has a very open development approach where people with many different experiences can share their wisdom
Interesting take, but as someone that drifted away from Rust with the publication of the linked blog post, and wrote their last meaningful code in the language a year later, it does not describe my experience. YMMV, but compared to other languages, I'd say as a first approximation that they had no interest in comments from outside the inner circle.
Interesting take, but as someone that drifted away from Rust with the publication of the linked blog post, and wrote their last meaningful code in the language a year later, it does not describe my experience. YMMV, but compared to other languages, I'd say as a first approximation that they had no interest in comments from outside the inner circle.
The fact they've managed to bring safety to the masses without a GC is arguably the #1 reason for Rust's success. This has been a good move.
This claim is a bit too broad based on my limited understanding. Please correct me if I'm wrong:
First, Rust is not actually a memory safe language. It makes writing memory safe code easy and nudges you towards writing "unsafe" code in specific places. This is a good thing and many get away with not requiring programmer asserted safety. But it's also important to note that ultimately it relies on it.
Second, The borrow checker seems to primarily check that you're doing RAII properly. In a sense it answers the question of:
"What if we get the predictable nature of stack allocated memory for the heap?"
Clever, but that leads to a proliferation of lifetime annotations (which are part of your types) and makes code very brittle and rigid if spelled out as is. Because every lifetime that is encoded that way, has to fully cover the scope that encloses all of its usage. And if that weren't enough, it also infects almost every data structure that is some way composed of references.
There seems to be multiple ways of dealing with this issue that I've come across when learning the language:
1. Avoiding pointer references whenever possible and using self-managed vector/slice/hashmap references.
2. Introducing GC via reference counting.
3. Cloning.
4. Macro or trait abstraction.
When we're doing 1, we don't gain much utility from the borrow checker.
When doing 2/3 we would be better served if we used a language with a battle hardened and optimized GC runtime.
I've seen 4 in some cases, but never looked like it's "bringing safety to the masses". The whole API around traits and macros is very rich, very sophisticated and very subtle.
I would rather phrase it as: "Rust brings ML/functional concepts to C++ programmers and it explores a new space of compiler optimizations based on that."
However the Rust _community_ does bring these concepts to the masses. They have written excellent books and recorded multi hour long videos explaining and exploring the language and making this all accessible.
First, Rust is not actually a memory safe language. It makes writing memory safe code easy and nudges you towards writing "unsafe" code in specific places. This is a good thing and many get away with not requiring programmer asserted safety. But it's also important to note that ultimately it relies on it.
Second, The borrow checker seems to primarily check that you're doing RAII properly. In a sense it answers the question of:
"What if we get the predictable nature of stack allocated memory for the heap?"
Clever, but that leads to a proliferation of lifetime annotations (which are part of your types) and makes code very brittle and rigid if spelled out as is. Because every lifetime that is encoded that way, has to fully cover the scope that encloses all of its usage. And if that weren't enough, it also infects almost every data structure that is some way composed of references.
There seems to be multiple ways of dealing with this issue that I've come across when learning the language:
1. Avoiding pointer references whenever possible and using self-managed vector/slice/hashmap references.
2. Introducing GC via reference counting.
3. Cloning.
4. Macro or trait abstraction.
When we're doing 1, we don't gain much utility from the borrow checker.
When doing 2/3 we would be better served if we used a language with a battle hardened and optimized GC runtime.
I've seen 4 in some cases, but never looked like it's "bringing safety to the masses". The whole API around traits and macros is very rich, very sophisticated and very subtle.
I would rather phrase it as: "Rust brings ML/functional concepts to C++ programmers and it explores a new space of compiler optimizations based on that."
However the Rust _community_ does bring these concepts to the masses. They have written excellent books and recorded multi hour long videos explaining and exploring the language and making this all accessible.
By the standard of your “first,” no language is a memory safe language. They all must rely on unsafety in the implementation of their runtimes in the same sense that Rust builds safe abstractions on top of unsafe code, and many even offer FFI, which is conceptually similar to calling an unsafe function.
Most memory safe languages lack enough power to implement their own runtime systems (except in an extremely inefficient way), because so many languages ultimately rely on an implementation in another unsafe language that has access to machine capabilities. Implementing the runtime system in the language itself is a black art that requires an unsafe dialect or extension. E.g. all the Java-in-Java VMs sneak dialect features in through intrinsic classes like "Pointer" and such that are recognized by their own extended compilers.
> The fact they've managed to bring safety to the masses without a GC is arguably the #1 reason for Rust's success. This has been a good move.
That's an understatement. Rust doesn't require a framework / runtime. Unlike NodeJS, Python, Java, C#, ect; when you ship a Rust program, it has no external requirements.
More important: Because Rust can create libraries that adhere to the C calling convention, you can create libraries that you can call from NodeJS, Python, Java, C#, ect. The fact that those systems have a heavyweight runtime (with GC) makes it very hard to create a library in one environment and call it from the other.
I do think there's room for an "R2" that is semantically identical, but designed to be easier to understand. Probably the biggest room for improvement is using generics for things like RC, ARC, Mutex. I'd rather use something like keywords; and leave generics for user-defined types. This way, when trying to understand what something "is," the "how the memory is tracked" is semantically different from "what the struct is." (Even though under the hood they are the same thing.)
That's an understatement. Rust doesn't require a framework / runtime. Unlike NodeJS, Python, Java, C#, ect; when you ship a Rust program, it has no external requirements.
More important: Because Rust can create libraries that adhere to the C calling convention, you can create libraries that you can call from NodeJS, Python, Java, C#, ect. The fact that those systems have a heavyweight runtime (with GC) makes it very hard to create a library in one environment and call it from the other.
I do think there's room for an "R2" that is semantically identical, but designed to be easier to understand. Probably the biggest room for improvement is using generics for things like RC, ARC, Mutex. I'd rather use something like keywords; and leave generics for user-defined types. This way, when trying to understand what something "is," the "how the memory is tracked" is semantically different from "what the struct is." (Even though under the hood they are the same thing.)
> using generics for things like RC, ARC
There is an unstable feature for "box syntax" which both allows you to construct a `Box<T>` as let foo = box a (instead of let foo = Box::new(a)) but also to use box as a keyword in pattern matches to deref the box automatically. So struct Foo { value: Box<Bar> };
let Foo { value: box Bar } (in which case value is bound to a &Bar) which was really nice.
It got subsumed into a more generic "deref patterns" project which doesn't seem to be going anywhere but I hope it does because it makes the language much more ergonomic IMO.
There is an unstable feature for "box syntax" which both allows you to construct a `Box<T>` as let foo = box a (instead of let foo = Box::new(a)) but also to use box as a keyword in pattern matches to deref the box automatically. So struct Foo { value: Box<Bar> };
let Foo { value: box Bar } (in which case value is bound to a &Bar) which was really nice.
It got subsumed into a more generic "deref patterns" project which doesn't seem to be going anywhere but I hope it does because it makes the language much more ergonomic IMO.
The `box` syntax to create `Box`es has been removed because it was broken. The initial idea was to have it initialize the value directly in the heap, but that never worked for function calls. The `box` pattern instead remained as it was still useful to have (really unfortunate for the perma-unstable status though).
Ah didn't realized it had been removed. I gave up on it after it became clear it was never going to make it into stable rust. I do hope deref patterns gets some traction though as it could really be a nice feature.
What I increasingly want to see is a language where a garbage collector is optional in the main executable, but not (directly) available to library code.
Because I want to have a language that I can use to write lean libraries that are available to any language with a C FFI, and that can be linked directly by AOT-compiled languages, without getting into a horrible quagmire of dueling heaps and copying. But I also want to have proper functional and asynchronous programming, and generally just to not have to manually fuss with memory in the higher-level code.
Python and C/C++/Rust/Cython/etc extensions kindasorta achieves this, and it's a huge factor in the language's ascendance in scientific computing applications. Game development has a long history of achieving a similar effect by embedding Lua or Lisp. But I think that it might be more pleasant to have it formalized and baked into a single language.
Because I want to have a language that I can use to write lean libraries that are available to any language with a C FFI, and that can be linked directly by AOT-compiled languages, without getting into a horrible quagmire of dueling heaps and copying. But I also want to have proper functional and asynchronous programming, and generally just to not have to manually fuss with memory in the higher-level code.
Python and C/C++/Rust/Cython/etc extensions kindasorta achieves this, and it's a huge factor in the language's ascendance in scientific computing applications. Game development has a long history of achieving a similar effect by embedding Lua or Lisp. But I think that it might be more pleasant to have it formalized and baked into a single language.
I'm reminded of wuffs, which only allows writing libraries and explicitly doesn't support allocateing memory, requiring the calling program to provide memory. It seemed like a nice separation of concerns in my non-expert view. (And then wuffs compiles to C, which is nice for interoperability)
I hadn’t seen wuffs before. Interesting approach to writing safe libraries:
Wuffs is not a general purpose programming language. It is for writing libraries, not programs. Wuffs code is hermetic and can only compute (e.g. convert "compressed bytes" to "decompressed bytes"). It cannot make any syscalls (e.g. it has no ambient authority to read your files), implying that it cannot allocate or free memory (and is therefore trivially safe against things like memory leaks, use-after-frees and double-frees).
https://github.com/google/wuffs
Wuffs is not a general purpose programming language. It is for writing libraries, not programs. Wuffs code is hermetic and can only compute (e.g. convert "compressed bytes" to "decompressed bytes"). It cannot make any syscalls (e.g. it has no ambient authority to read your files), implying that it cannot allocate or free memory (and is therefore trivially safe against things like memory leaks, use-after-frees and double-frees).
https://github.com/google/wuffs
Possibly Rust needs a GC<> wrapper type, which makes everything it contains garbage collectable. And then you could be bare metal, except when you want to not be.
There are a number of efforts along these lines, the most interesting is probably Samsara https://github.com/chc4/samsara https://redvice.org/2023/samsara-garbage-collector/ which implements a concurrent, thread-safe GC with no global "stop the world" phase.
[deleted]
I think it's quite unfortunate that there isn't a language with
1) Rust's type system or something as expressive 2) C#-level performance 3) GC by default 4) Ecosystem that is as big as Rust
The closest thing seems to be TypeScript (weirdly).
1) Rust's type system or something as expressive 2) C#-level performance 3) GC by default 4) Ecosystem that is as big as Rust
The closest thing seems to be TypeScript (weirdly).
Excuse my ignorance (I have only just started learning Rust), but why is GC by default desirable? If a programming language can tell when a variable goes out of scope, or its lifetime ends and use that information to automatically release allocated memory, then why is having a garbage collector important? Does Rust (as a result of not having GC) put restrictions on the kind of code you can write, or force one to write code in such a way that it's difficult to reason about?
Yes, it has a borrow checker which restricts some valid code and occasionally makes you jump through hoops and write in a different style.
Also, a garbage collector that is state of the art (bump allocation, generational, and compacting) is faster overall typically (throughput-wise, but with less predictable latency) than naïve Rc/Arc all over the place (due to usage of "free list" allocator and counter bumps). That isn't to say blisteringly fast Rust isn't possible to write, and in fact it is pretty easy to write by avoiding Rc/Arc except where necessary and using the stack with the borrow checker, but this entails a writing style that is more "low level" and thus takes a bit more thinking.
Also, a garbage collector that is state of the art (bump allocation, generational, and compacting) is faster overall typically (throughput-wise, but with less predictable latency) than naïve Rc/Arc all over the place (due to usage of "free list" allocator and counter bumps). That isn't to say blisteringly fast Rust isn't possible to write, and in fact it is pretty easy to write by avoiding Rc/Arc except where necessary and using the stack with the borrow checker, but this entails a writing style that is more "low level" and thus takes a bit more thinking.
IANAGCE (I am not a GC expert.) I thought there were potential costs to using GS.
1. Interrupting the program to sweep, resulting in unpredictable performance.
2. Possibility that a circularly linked group of variables could be impossible to sweep, resulting in memory leaks.
3. Need to check reference counts (along with bounds checks) that degrade performance.
Lack of GC (and bounds checking) are factors that make C/C++ performant and then lead to the kind of bugs that result in programs that don't do what they're supposed to do (and at worst, result in security vulnerabilities.)
I thought a key goal of Rust was to fix these problems without sacrificing performance.
1. Interrupting the program to sweep, resulting in unpredictable performance.
2. Possibility that a circularly linked group of variables could be impossible to sweep, resulting in memory leaks.
3. Need to check reference counts (along with bounds checks) that degrade performance.
Lack of GC (and bounds checking) are factors that make C/C++ performant and then lead to the kind of bugs that result in programs that don't do what they're supposed to do (and at worst, result in security vulnerabilities.)
I thought a key goal of Rust was to fix these problems without sacrificing performance.
> 1. Interrupting the program to sweep, resulting in unpredictable performance.
See my comment regarding "less predictable latency" (although new advances are making for much more predictable latency - see some of the work done in Java GCs for example)
> 2. Possibility that a circularly linked group of variables could be impossible to sweep, resulting in memory leaks.
That is not possible in a precise tracing collector, only in reference counting (Rc/Arc)
> 3. Need to check reference counts (along with bounds checks) that degrade performance.
I think you are referring to reference counting again. Both Rust and C++ have reference counting types as an add on.
State of the art tracing collectors don't collect/check dead objects, they compact live ones, and often don't use reference counting directly. Most objects die young.
> Lack of GC (and bounds checking) are factors that make C/C++ performant and then lead to the kind of bugs that result in programs that don't do what they're supposed to do (and at worst, result in security vulnerabilities.)
Replace "performant" with "predictable performance"
> I thought a key goal of Rust was to fix these problems without sacrificing performance.
Rust, like C/C++, wants you to be able to predict performance and latency and without the baggage of a runtime. Nothing more than trade offs - neither is better or worse. GCs often do perform better with the trade off of latency predictability, but as always it depends on use case as to what is appropriate. Rust likely made the right choice for its domain.
See my comment regarding "less predictable latency" (although new advances are making for much more predictable latency - see some of the work done in Java GCs for example)
> 2. Possibility that a circularly linked group of variables could be impossible to sweep, resulting in memory leaks.
That is not possible in a precise tracing collector, only in reference counting (Rc/Arc)
> 3. Need to check reference counts (along with bounds checks) that degrade performance.
I think you are referring to reference counting again. Both Rust and C++ have reference counting types as an add on.
State of the art tracing collectors don't collect/check dead objects, they compact live ones, and often don't use reference counting directly. Most objects die young.
> Lack of GC (and bounds checking) are factors that make C/C++ performant and then lead to the kind of bugs that result in programs that don't do what they're supposed to do (and at worst, result in security vulnerabilities.)
Replace "performant" with "predictable performance"
> I thought a key goal of Rust was to fix these problems without sacrificing performance.
Rust, like C/C++, wants you to be able to predict performance and latency and without the baggage of a runtime. Nothing more than trade offs - neither is better or worse. GCs often do perform better with the trade off of latency predictability, but as always it depends on use case as to what is appropriate. Rust likely made the right choice for its domain.
What kind of GC do you talk about here? (RC is also a GC algorithm, but you seem to mix some of its shortcomings with that of trading GCs).
1) this is generally true, but many part of this can be done concurrently, and if we want to improve latency at the cost of some throughput than there are low-lat GCs that maximize the pause times (it is basically independent from the heap size), so in practice you have similar interrupts as you would from the OS alone.
2) this is not a problem with tracing GCs, only with refcounting (most well known is Python, ObjC and Swift for this perhaps). You can still leak memory everywhere by e.g. storing them in a huge list forever, though, but that is a much rarer and easy to debug bug.
3) this is again only true for RC, and it is the reason why it is slower than tracing GCs, especially when it is multithreaded and the increment/decrement has to be an atomic operation.
1) this is generally true, but many part of this can be done concurrently, and if we want to improve latency at the cost of some throughput than there are low-lat GCs that maximize the pause times (it is basically independent from the heap size), so in practice you have similar interrupts as you would from the OS alone.
2) this is not a problem with tracing GCs, only with refcounting (most well known is Python, ObjC and Swift for this perhaps). You can still leak memory everywhere by e.g. storing them in a huge list forever, though, but that is a much rarer and easy to debug bug.
3) this is again only true for RC, and it is the reason why it is slower than tracing GCs, especially when it is multithreaded and the increment/decrement has to be an atomic operation.
Thanks for that explanation! I'm going to have to dive into Rc/Arc [0]. Reference counting built into the standard library is really interesting. Several years ago I wrote a Windows kernel minifilter and reference counting is used a lot by anything that interacts with the filter manager. RC was very helpful to ensure the minifilter didn't leak memory when it was unloaded.
[0]: https://doc.rust-lang.org/std/sync/struct.Arc.html
[0]: https://doc.rust-lang.org/std/sync/struct.Arc.html
> If a programming language can tell when a variable goes out of scope, or its lifetime ends
Whether a variable goes out of scope is trivial in many languages. The problem is with determining whether the lifetime of a value ends.
For example:
Whether a variable goes out of scope is trivial in many languages. The problem is with determining whether the lifetime of a value ends.
For example:
func foo(items, moreItems) {
b = new bar()
if coinflip() {
items.append(b)
}
if coinflip() {
moreItems.append(b)
}
// ‘b’ goes out of scope here, but its value
// may live on inside ‘items’ and/or moreItems
// and will have to be destroyed when it’s no
// longer stored in either.
}
Yes, I agree that determining the lifetime of a value is the hard part. I wrote "... and use that information to automatically release allocated memory, ..." to imply the language got lifetime detection right.
With a GC you can basically write whatever you want with reckless abandon and because it's cleaned up at runtime, it's mostly kosher.
Without a GC (i.e. rust), in order to be able to make guarantees about things that it cannot determine at compile time (this is a mathematical impossibility) it restricts the set of programs you can write.
Without a GC (i.e. rust), in order to be able to make guarantees about things that it cannot determine at compile time (this is a mathematical impossibility) it restricts the set of programs you can write.
> Does Rust (as a result of not having GC) put restrictions on the kind of code you can write
Depending on what you mean by kind: yes. That is, Rust does limit your code’s architecture to a subset that is fine for most things, but not everything.
Depending on what you mean by kind: yes. That is, Rust does limit your code’s architecture to a subset that is fine for most things, but not everything.
Java?
* Type system is even accidentally Turing complete
* Very good perf, but language doesn't help by being indirection-friendly. Value types will help a lot.
* SOTA GCs
* Ecosystem big
* Cheap threads now. Don't do async! Just block.
* Structured concurrency soonish
* Type system is even accidentally Turing complete
* Very good perf, but language doesn't help by being indirection-friendly. Value types will help a lot.
* SOTA GCs
* Ecosystem big
* Cheap threads now. Don't do async! Just block.
* Structured concurrency soonish
> Type system is even accidentally Turing complete
A turing complete type system is easier to stumble into than to avoid.
That doesn’t mean the type system is expressive. Turing tarpits are turing complete by definition, and nobody would call Thue, Iota, or the average OISC expressive.
A turing complete type system is easier to stumble into than to avoid.
That doesn’t mean the type system is expressive. Turing tarpits are turing complete by definition, and nobody would call Thue, Iota, or the average OISC expressive.
Or if someone really want to play around with types: Scala, especially Scala 3. People can’t even say that it is not nullsafe, as it is.
I haven't used Scala since 2. Did something change related to null with 3? Can you no longer use null? I know it has the Option type which can be used to safely represent nullable values, but that is (or at least was) in addition to, rather than in replacement of null.
It has a compiler flag which will remove the null value from most type’s sets, that is a String will never contain null, if you want it to be nullable you have to write `String | None` (or Null? Something like that).
Sounds like you want OCaml (or StandardML but OCaml is more "active")
Since, as I feel it, Rust was/is started as an attempt to bring roughly the ML (Hindley-Milner) type system to the area of `systems` (non-garbage collected) development.
In the early days of Rust I thought of it as type inference & algebraic data types meets C++ (now kiss!). But then the borrow checker stuff went in there and it took a different turn.
And I think you'd be surprised by how large the OCaml package community is.
People who work in Rust would likely find OCaml very familiar after a week or two of hacking.
The other option is Swift, but it's pretty ghetto-ized into the Apple ecosystem.
Since, as I feel it, Rust was/is started as an attempt to bring roughly the ML (Hindley-Milner) type system to the area of `systems` (non-garbage collected) development.
In the early days of Rust I thought of it as type inference & algebraic data types meets C++ (now kiss!). But then the borrow checker stuff went in there and it took a different turn.
And I think you'd be surprised by how large the OCaml package community is.
People who work in Rust would likely find OCaml very familiar after a week or two of hacking.
The other option is Swift, but it's pretty ghetto-ized into the Apple ecosystem.
Not trying to be cheeky, but why not c#? Has pts 2, 3 and 4. For pt 1, what in Rusts type system are you looking for?
> Not trying to be cheeky, but why not c#?
C# is my main language. I consider it a very good all-round language.
Rust's type system has some advantages over C# tho, for example Sum Type, Option (C# has ? but it was added later so you need to be careful when interacting with old code, kinda like TypeScript <-> JavaScript to a lesser extent), exhaustive enum, etc.
Another thing I don't like about C# is the runtime startup time which prevents me from using it for command line tools (Yes I prefer static typed languages even for "scripting"). I think Go has proven that you can have both GC and extremely fast startup time.
C# is my main language. I consider it a very good all-round language.
Rust's type system has some advantages over C# tho, for example Sum Type, Option (C# has ? but it was added later so you need to be careful when interacting with old code, kinda like TypeScript <-> JavaScript to a lesser extent), exhaustive enum, etc.
Another thing I don't like about C# is the runtime startup time which prevents me from using it for command line tools (Yes I prefer static typed languages even for "scripting"). I think Go has proven that you can have both GC and extremely fast startup time.
Could you try a few sample scenarios for a CLI tool written in C# with the following publish options?
JIT:
p.s.: AOT on macOS requires .NET 8 preview (will be released in November)
JIT:
dotnet publish -c release -o publish -p:PublishSingleFile=true -p:PublishTrimmed=true
dotnet publish -c release -o publish -p:PublishAot=true
p.s.: AOT on macOS requires .NET 8 preview (will be released in November)
Interesting, I remembered I've tried something similar to your JIT example but maybe my memory is playing tricks on me. I'll try these again later
20 milliseconds? On my 7 year old Linux box, this little Nim program https://github.com/c-blake/bu/blob/main/wsz.nim runs to completion in 109.62 +- 0.17 microseconds when fully statically linked with musl libc on Linux. That's with a stripped environment (with `env -i`). It takes more like 118.1 +- 1.1 microseconds with my usual 54 environment variables. The program only does about 17 system calls total, though.
Additionally, https://github.com/c-blake/cligen makes decent CLI tools a real breeze. If you like some of Go's qualities but the language seems too limited, you might like Nim: https://nim-lang.org. I generally find getting good performance much less of a challenge with Nim, but Nim is undeniably less well known with a smaller ecosystem and less corporate backing.
EDIT: I make only observations here, not demands. Another observation on the same machine is `python-3.11.5 </dev/null` with an empty environment taking 7.85 +- 0.02 ms.
Additionally, https://github.com/c-blake/cligen makes decent CLI tools a real breeze. If you like some of Go's qualities but the language seems too limited, you might like Nim: https://nim-lang.org. I generally find getting good performance much less of a challenge with Nim, but Nim is undeniably less well known with a smaller ecosystem and less corporate backing.
EDIT: I make only observations here, not demands. Another observation on the same machine is `python-3.11.5 </dev/null` with an empty environment taking 7.85 +- 0.02 ms.
20ms as measured with `time` on Fish shell on macOS. Let's be real, comparing versus Nim here is the same as comparing with C - both are a much lower level languages and don't do as much work on startup (threadpool, GC, etc.) and, for the reference, 60fps is 16.6ms per frame. The difference is unlikely to be noticeable. And this isn't measuring program execution time but back to back console receiving a command to launch a binary, OS doing so, binary starting, doing useful work (displaying hint that the command was not in the correct format) and only then exiting.
I agree this particular C# being >180x slower may not matter much for one-off commands keyed-in by and watched by humans, but that may not be all that matters. E.g., some people might `find . -print | xargs -n1 cmd`. Almost everything almost always "all depends". (On a whole lot. E.g., only @raincole can elaborate on his use cases and what might be missing from the Nim ecosystem.)
EDIT: Also, it's misleading to bundle Nim with C and C's many & storied footguns. While "low-level" is somewhat subjective and you can opt-in to go as low as C (if you so desire), most Nim code is as high-level as Python or C# with various choices in automatic memory management, and the language has very high-level capabilities. E.g., Nim has user-defined operators like Julia. Want to add `=~` or `~` for regex pattern matching? No problemo. In that aspect, Nim is arguably higher-level than C#.
EDIT: Also, it's misleading to bundle Nim with C and C's many & storied footguns. While "low-level" is somewhat subjective and you can opt-in to go as low as C (if you so desire), most Nim code is as high-level as Python or C# with various choices in automatic memory management, and the language has very high-level capabilities. E.g., Nim has user-defined operators like Julia. Want to add `=~` or `~` for regex pattern matching? No problemo. In that aspect, Nim is arguably higher-level than C#.
> C# is my main language.
Then the answer is F#. OCaml has everything except the big ecosystem. Haskell has a way bigger Ecosystem than OCaml, but is still not comparable to Rust.
Then the answer is F#. OCaml has everything except the big ecosystem. Haskell has a way bigger Ecosystem than OCaml, but is still not comparable to Rust.
Runtime startup might be a fixed issue soon for smaller CLI programs. Correct me if I'm wrong, but I believe they are working on (and it is already available with a compiler option) to compile your C# program ahead of time (C# AOT).
Tried Kotlin native?
- sum types via sealed classes (not great, admittedly)
- enum "when" expressions are exhaustive
- option type via built-in nullable types (honestly the superior solution)
- sum types via sealed classes (not great, admittedly)
- enum "when" expressions are exhaustive
- option type via built-in nullable types (honestly the superior solution)
Rust's type system has some advantages over C#
Pick up some F# then. Both can interop, I've built many things in a combo of the two languages, plus it'll make you a better C# developer and your type system power greatly increases. Be careful though you might not want to go back to C#.
Pick up some F# then. Both can interop, I've built many things in a combo of the two languages, plus it'll make you a better C# developer and your type system power greatly increases. Be careful though you might not want to go back to C#.
[deleted]
C#'s type system is nowhere near as expressive. Lack of sum types/discriminated unions and pattern matching/type refinement.
Sum types, which can only be approximated in C# via records, and no exceptions, which is even a problem for F#.
I don't know C#, so maybe it has these, but it's unlikely:
- absence of null pointers
- proper sum types which can be statically checked for exhaustiveness
- statically controlled mutability (yes, borrow-checking is still useful if you have a GC!)
Also no exceptions, but that's not a type system feature.
- absence of null pointers
- proper sum types which can be statically checked for exhaustiveness
- statically controlled mutability (yes, borrow-checking is still useful if you have a GC!)
Also no exceptions, but that's not a type system feature.
C# calls the first one "nullable reference types." When that build option is enabled, all types are non-nullable by default, and you can make them optionally nullable by declaring them like "MyClass? cls = null;"
The compiler will ensure you check for null before de-referencing a nullable type.
The compiler will ensure you check for null before de-referencing a nullable type.
How does this interact with third-party dependencies you use? In my experience, most gradual typing things break down at that boundary.
This isn't gradual typing? If libraries are not compiled with the compiler option, then all their reference types will be deemed nullable and I need to check for null before dereferencing. I can't do something like "GetItem().Name" if "GetItem()" returns a reference type like I could in Java, GoLang, JS, etc. If that library did use the flag and was provably non-nullable, then I could.
Then what about F#?
I expected someone to write a rust-based scripting language which tightly integrated with rust itself.
In reality, it seems like the python developers and toolchain are embracing rust enough to reduce the benefits to a new alternative.
https://github.com/PyO3/pyo3
In reality, it seems like the python developers and toolchain are embracing rust enough to reduce the benefits to a new alternative.
https://github.com/PyO3/pyo3
One issue with GC’d language and the associated promiscuity is it’s quite hard to mix with affine or even linear types. Yet those turn out to be quite handy.
Nim-lang fits 1, 2 and 3! And in a couple of years the ecosystem is up to level ;)
Nim doesn't have sum types and pattern matching, which are an essential part of an expressive type system. It also appears to have `nil` be a valid value for most types by default?
This comment is misleading &| misinformed.
Sum types are built-in [1] for formal parameters. `nil` is only for `ref|ptr` types. In much code you can just use stack allocated value types and there is neither GC concern nor nil concern, but there is also a mode to help: https://nim-lang.github.io/Nim/manual_experimental_strictnot...
Nim has an easy-ish to use Lisp-like syntax macro system where you just receive & process an AST. So, to do the rest you can make libraries adding the feature without relying upon upstream compiler: such as https://github.com/beef331/sumtypes for variables with sum types or pattern matching libs like https://andreaferretti.github.io/patty/ | https://github.com/alehander92/gara.
Sum types are built-in [1] for formal parameters. `nil` is only for `ref|ptr` types. In much code you can just use stack allocated value types and there is neither GC concern nor nil concern, but there is also a mode to help: https://nim-lang.github.io/Nim/manual_experimental_strictnot...
Nim has an easy-ish to use Lisp-like syntax macro system where you just receive & process an AST. So, to do the rest you can make libraries adding the feature without relying upon upstream compiler: such as https://github.com/beef331/sumtypes for variables with sum types or pattern matching libs like https://andreaferretti.github.io/patty/ | https://github.com/alehander92/gara.
Mojo could potentially fill this gap in a year. They still have a long way to go, but they're working on traits and sum types right now
Isn't Rust's type system mostly about not having GC? What specific type system features are you looking for outside of borrow checking? Macros?
Probably the closest would be Kotlin. It has the huge JVM ecosystem, it uses GC, programs run fast, they can be compiled to native binaries using at least two different native compilers (kotlin/native and graalvm). The type system is fairly expressive, albeit not quite as much as TypeScript. It makes up for it with just being a much cleaner and more logical language in general, as it didn't inherit much historical baggage.
There's also a compiler plugin API which is maybe the closest equivalent of macros. It's not really documented or stable in Kotlin 1.x, they're fixing that for Kotlin 2, but there are already a bunch of useful plugins that add various features via compile-time generation and reflection.
Probably the closest would be Kotlin. It has the huge JVM ecosystem, it uses GC, programs run fast, they can be compiled to native binaries using at least two different native compilers (kotlin/native and graalvm). The type system is fairly expressive, albeit not quite as much as TypeScript. It makes up for it with just being a much cleaner and more logical language in general, as it didn't inherit much historical baggage.
There's also a compiler plugin API which is maybe the closest equivalent of macros. It's not really documented or stable in Kotlin 1.x, they're fixing that for Kotlin 2, but there are already a bunch of useful plugins that add various features via compile-time generation and reflection.
> Isn't Rust's type system mostly about not having GC? What specific type system features are you looking for outside of borrow checking? Macros?
1. ADTs with exhaustive pattern matching
2. No exceptions
3. Zero cost generics
4. Traits
5. No nulls
6. No inheritance
1. ADTs with exhaustive pattern matching
2. No exceptions
3. Zero cost generics
4. Traits
5. No nulls
6. No inheritance
Rust has exceptions. They are used throughout the standard libraries to report some errors, and as far as I can tell, they are deeply ingrained into the default testing framework.
As a rule of thumb, languages that loudly claim not to have exceptions actually use them in some way (see POSIX C, Perl, Go).
As a rule of thumb, languages that loudly claim not to have exceptions actually use them in some way (see POSIX C, Perl, Go).
Saying that Rust "has exceptions" is dishonest. What matters isn't whether they technically exist, but if and how they're used.
In idiomatically written Rust you will never obtain an exception/panic (unless there is a bug), and exceptions are not used for control flow. This is not the case in Java or C++ or many other languages.
In idiomatically written Rust you will never obtain an exception/panic (unless there is a bug), and exceptions are not used for control flow. This is not the case in Java or C++ or many other languages.
I don't know. It seems fairly common to use std::panic::catch_unwind as a top-level error handler. compiler/rustc_driver_impl/src/lib.rs has this:
There's another example in src/tools/rustfmt/src/lib.rs, in format_snippet, where panics apparently are just suppressed.
I expect application servers for Rust to handle panics in a similar way: abort a specific request, but not terminate the whole process.
/// The compiler currently unwinds with a special sentinel value to abort
/// compilation on fatal errors. This function catches that sentinel and turns
/// the panic into a `Result` instead.
There's another example in src/tools/rustfmt/src/lib.rs, in format_snippet, where panics apparently are just suppressed.
I expect application servers for Rust to handle panics in a similar way: abort a specific request, but not terminate the whole process.
In JRE terms, Rust doesn't have Exceptions, but it has Errors.
> Isn't Rust's type system mostly about not having GC? What specific type system features are you looking for outside of borrow checking?
Sun types, affine types, traits.
Sun types, affine types, traits.
F# probably?
Yeah GP’s points 1 through 4 seem like they could be on an F# pitch deck. It’s an ML with full access to the .net ecosystem and the .net VM’s speed and stability.
we have a massive mixed c# and f# codebase at work.
F# is not magical. Yes it's an ML, but frankly, C# is better in every way, to the point we're slowly moving away from F# entirely.
The main reason is perf, it's really easy to shoot yourself in the foot with performance in c# (e.g. huge allocations, accidentally evaluating seqs twice, etc). Also IDE support for F# sucks when you get to the hundreds of project solutions like we have.
For side projects, sure use F#. For everything else, stick with C#.
F# is not magical. Yes it's an ML, but frankly, C# is better in every way, to the point we're slowly moving away from F# entirely.
The main reason is perf, it's really easy to shoot yourself in the foot with performance in c# (e.g. huge allocations, accidentally evaluating seqs twice, etc). Also IDE support for F# sucks when you get to the hundreds of project solutions like we have.
For side projects, sure use F#. For everything else, stick with C#.
Yes wanted to comment that as well.
I think #1 needs to be expanded to imply what it DOESN'T have as well:
1. No null pointers 2. No exceptions
1. No null pointers 2. No exceptions
Scala's type system is about as expressive as it gets in mainstream languages. By virtue of running on the JVM it is GC'd and has Java/C#-like performance. And the ecosystem of Scala implemented libraries is huge.
Seems like we are looking for the same thing so I invent a new language to fit my need: https://github.com/ultimaweapon/nitro
What do you consider Rust's relevant properties in this context? Traits? Inner object references? Associated types and constants? Compilation tending towards monomorphization?
C++/CLI covers some of these aspects, but I haven't used it and don't know how large the vcpkg ecosystem is.
C++/CLI covers some of these aspects, but I haven't used it and don't know how large the vcpkg ecosystem is.
D (Dlang) checks 3 of those marks. If you like cursed things, Vala isn't that bad...
Ha, I feel exactly the same. I asked HN about it a year ago[0], and there was some interesting discussion.
[0] https://news.ycombinator.com/item?id=32984776
[0] https://news.ycombinator.com/item?id=32984776
Haskell has a better type system. In fact rusts type system is derived from ML languages ...
Tons of languages have better type systems.
Tons of languages have better type systems.
OCaml probably has everything but 4.
Also just remembered that F# exists, which is probably exactly all of that and runs on the CLR like C#.
Kotlin ticks all of your boxes, plus you benefit from the vast ecosystem of the JVM.
It's unfortunate that Kotlin lacks pattern matching, especially now that Java has it. I can only hope that the release of the new compiler will spur further language features.
What kinds of pattern matching that Java now has is Kotlin missing?
At the moment Kotlin only has smart casts and exhaustive type checking (e.g., making sure you didn't forget a switch case). It doesn't let you destructure records, add guards to cases, etc.
https://www.infoq.com/articles/pattern-matching-for-switch/
https://www.infoq.com/articles/pattern-matching-for-switch/
OCaml or F#?
F#
OCaml?
yes something like kotlin + go would be excellent
I'm pretty sure that Rust is a great language for low level stuff, the domain of C/C++. Try to shoehorn it to do more higher level stuff (i.e: vast majority of software developed today), And you'll be fighting battles with the language at each step. For those use cases, Java, JavaScript, Python et al will reign supreme for many years to come. Most of the time, GC stalls are the least of your problems.
Rust just took C++ std::unique and std::shared ptr and made those integrated directly in the language, and the only option for allocation. Which is awesome.
It would be nice to see if we can have a sub set of C++ that forces us to only use std::make_unique or std::make_shared calls.
It would be nice to see if we can have a sub set of C++ that forces us to only use std::make_unique or std::make_shared calls.
> Rust just took C++ std::unique and std::shared ptr and made those integrated directly in the language, and the only option for allocation
Not really. Both Box and Rc/Arc are first and foremost library features implemented using the equivalent of malloc() and free(). Box is a bit special due to its deref semantics, but other than that, there's nothing stopping you from implementing them or something else yourself.
Not really. Both Box and Rc/Arc are first and foremost library features implemented using the equivalent of malloc() and free(). Box is a bit special due to its deref semantics, but other than that, there's nothing stopping you from implementing them or something else yourself.
Thanks, never wrote Rust so I'm just guessing. What else there is, besides static type checks? Is there a runtime side too?
There is no runtime (other than init/exit handling). The main thing that provides memory safety without a GC is the borrow checker, which is a language feature and independent of the smart pointer types in the standard library.
You can already write a very very very simple linter that bans use of the "new" keyword. Rust did quite a bit more to make this ergonomic than just force you to use smart pointers.
Discussed at the time:
Removing garbage collection from the Rust language - https://news.ycombinator.com/item?id=5811854 - June 2013 (130 comments)
Removing garbage collection from the Rust language - https://news.ycombinator.com/item?id=5811854 - June 2013 (130 comments)
Early Rust was really incomprehensible to me, and it's unquestionably better than it once was from an ergonomics and ecosystem perspective. The ~ and @ sigils everywhere were verging on Perl!
I actually think removing ~ was a mistake.
a) A lot of code ends up littered with Box anyways, which frankly isn't any more readable since "Box" doesn't tell you anything until you already know what it is and once you do it's just verbosity.
b) As I understand it, Box gets "special treatment" by the compiler/type system, so pretending it's just a pure standard library component is a bit obfuscatory
c) Heap allocation and heap pointers are first class citizens in other languages, why wouldn't they be so in Rust?
a) A lot of code ends up littered with Box anyways, which frankly isn't any more readable since "Box" doesn't tell you anything until you already know what it is and once you do it's just verbosity.
b) As I understand it, Box gets "special treatment" by the compiler/type system, so pretending it's just a pure standard library component is a bit obfuscatory
c) Heap allocation and heap pointers are first class citizens in other languages, why wouldn't they be so in Rust?
> b) As I understand it, Box gets "special treatment" by the compiler/type system, so pretending it's just a pure standard library component is a bit obfuscatory
There is a strong desire to stop that and have Box be a normal std type, the main thing that is blocking this at the moment is that Box has special deref magic that is not possible to implement with surface level rust syntax (even with nightly features).
There is a strong desire to stop that and have Box be a normal std type, the main thing that is blocking this at the moment is that Box has special deref magic that is not possible to implement with surface level rust syntax (even with nightly features).
>"The sigils make the code unfamiliar before the concepts are learned. Unlike the rest of the punctuation in Rust, ~ and @ are not part of the standard repertoire of punctuation in C-like languages, and as a result the language can seem intimidating."
It's still intimidating with lifetimes and Arc<Box<Rc>> like idioms. Still, it's blazingly fast (tm). I wonder what Rust would be like if the GC was kept in like in Go.
It's still intimidating with lifetimes and Arc<Box<Rc>> like idioms. Still, it's blazingly fast (tm). I wonder what Rust would be like if the GC was kept in like in Go.
> I wonder what Rust would be like if the GC was kept in like in Go.
It would be one of the litany of managed languages that doesn’t significantly differ in anything from each other, and we would have no reason to be hyped about.
It would be one of the litany of managed languages that doesn’t significantly differ in anything from each other, and we would have no reason to be hyped about.
Hype is subjective but I do welcome a safer C++.
Here are some notes on the later history of GC in Rust:
RFC 256, 2014-09. https://rust-lang.github.io/rfcs/0256-remove-refcounting-gc-...
Includes « I (and I think the majority of the Rust core team) still believe that there are use cases that would be well handled by a proper tracing garbage collector. »
https://news.ycombinator.com/item?id=8312327 A core developer says
« I wouldn't be so quick to give up on GC support yet! "We have a plan!" But I don't think we'll have it in for Rust 1.0. And it's true that, even if we never do get it to work in a satisfactory way, the language works just fine without it. »
By 2015-04 ("Fearless Concurrency"), "Memory safety without garbage collection." is a "key value proposition" (this isn't quite the same as saying "we never want Garbage Collection", of course).
2016-08 https://manishearth.github.io/blog/2016/08/18/gc-support-in-... « Recently we (Felix, Niko, and I) have been working on getting compiler-level GC support for Rust. »
2018-10 withoutboats has a research garbage collector as a library: https://boats.gitlab.io/blog/post/shifgrethor-i/ The intro post includes « I do not expect we will ever “add GC to Rust” ».
2021 summary of options: https://manishearth.github.io/blog/2021/04/05/a-tour-of-safe...
RFC 256, 2014-09. https://rust-lang.github.io/rfcs/0256-remove-refcounting-gc-...
Includes « I (and I think the majority of the Rust core team) still believe that there are use cases that would be well handled by a proper tracing garbage collector. »
https://news.ycombinator.com/item?id=8312327 A core developer says
« I wouldn't be so quick to give up on GC support yet! "We have a plan!" But I don't think we'll have it in for Rust 1.0. And it's true that, even if we never do get it to work in a satisfactory way, the language works just fine without it. »
By 2015-04 ("Fearless Concurrency"), "Memory safety without garbage collection." is a "key value proposition" (this isn't quite the same as saying "we never want Garbage Collection", of course).
2016-08 https://manishearth.github.io/blog/2016/08/18/gc-support-in-... « Recently we (Felix, Niko, and I) have been working on getting compiler-level GC support for Rust. »
2018-10 withoutboats has a research garbage collector as a library: https://boats.gitlab.io/blog/post/shifgrethor-i/ The intro post includes « I do not expect we will ever “add GC to Rust” ».
2021 summary of options: https://manishearth.github.io/blog/2021/04/05/a-tour-of-safe...
> remove garbage collection from the core language and relegate it to the standard library, with a minimal set of language hooks in place to allow for flexible, pluggable automatic memory management.
They did part one, but not part two...
They did part one, but not part two...
Would adding the GC back make the syntax easier to read and remove all those special characters like the way lifetime is handled?
Even if in theory it would, Rust's commitment to backwards compatibility would mean that those things could not be removed.
You're asking for a different language.
You're asking for a different language.
One of the biggest mistake, in my opinion.
Especially with async which complicates lifetimes a lot.
Especially with async which complicates lifetimes a lot.
I (also like sibling comments) respectfully disagree.
Rust is aimed and focused as a safe C or a sane C++ substitute, and is meant to intermingle with both. It is not an application "high" level language. You can use it as such, which is great. For anything you would use C or C++ you can use Rust instead. As a cryptographer I find that great.
Regardless of all the lack of latency or other control -- beyond fine tuning -- a garbage collected language makes critical memory choices we want to make instead. There are times we want to swap or use our own memory allocator, never mind having to add a garbage collector in the mix. (There is a good number of languages that scratch that itch, and you can likely link and use your C/Rust code with them.)
As far as async: also respectfully disagree. Async is sugar for here is a Future<..>. If you want to poll it locally you can. You can scope it also. If you want to use a cross thread work stealing algorithm you can. But you need like memory management to consciously make these design decisions. This is similarly why a lot of things are not built in in C.
Rust is aimed and focused as a safe C or a sane C++ substitute, and is meant to intermingle with both. It is not an application "high" level language. You can use it as such, which is great. For anything you would use C or C++ you can use Rust instead. As a cryptographer I find that great.
Regardless of all the lack of latency or other control -- beyond fine tuning -- a garbage collected language makes critical memory choices we want to make instead. There are times we want to swap or use our own memory allocator, never mind having to add a garbage collector in the mix. (There is a good number of languages that scratch that itch, and you can likely link and use your C/Rust code with them.)
As far as async: also respectfully disagree. Async is sugar for here is a Future<..>. If you want to poll it locally you can. You can scope it also. If you want to use a cross thread work stealing algorithm you can. But you need like memory management to consciously make these design decisions. This is similarly why a lot of things are not built in in C.
They wanted a systems language. A GC would be very appropriate for much of the code I write, but would not have been for an OS kernel. Green threads removed as well. I admit the GC version with green threads I would have preferred, but again, not appropriate for the domain they wanted. I still want my halfway between Go and Rust language.
A GC-free systems language isn't just for OS kernels.
Rust's main use case was to replace C++ components in firefox. This wouldn't have been possible with a language that brings along its own GC, as Firefox already has a JS GC. Multiple garbage collectors in the same process are a quick way to madness, especially if you have reference cycles stretching across multiple GC heaps.
This also comes up when writing Python extension modules, base libraries that are meant to be usable across languages, etc...
This is part of why Rust is so successful -- it's the first real alternative for this space since C++ came along. For most application development, it's better to use a garbage collected language. But in the application space there is a much bigger choice of languages already available, Rust wouldn't have been a big deal over there.
This is part of why Rust is so successful -- it's the first real alternative for this space since C++ came along. For most application development, it's better to use a garbage collected language. But in the application space there is a much bigger choice of languages already available, Rust wouldn't have been a big deal over there.
I do wonder why there hasn't been more exploration in a series of languages that have a very similar style and toolchain but have different audiences. It would be great to have Rust, Rust with a GC, and then some sort of interpreted language that has a similar syntax. When jumping between languages I feel like half the battle is overcoming muscle memory.
This is my dream. An interpreted language, implemented in Rust which essentially boxes all data and can do message passing to the host language. This would then give a somewhat plausible upgrade path to port the interpreted code into Rust bit by bit as required.
The interpreted language even be slower than Python, so long as the escape hatch to Rust was simple and safe enough to implement the interfaces.
The interpreted language even be slower than Python, so long as the escape hatch to Rust was simple and safe enough to implement the interfaces.
I heard this brilliant summary in a video from ThePrimeagen:
In Rust, lifetimes color your types, like async colors your functions.
It is a great condensed summary of what makes lifetimes a great difficulty of async Rust. It's a language that has the function coloring that is typically introduced by async (likewise in JavaScript), and on top of that the typesystem itself gets colored by lifetime annotations. You can have a well written and working program... then due to some new need or refactoring, wanting to add a 'static somewhere will cause it all to break down.
It's part of the language, nothing bad with that. But it is an extra layer of difficulty that needs to be mastered. To me it shows that Rust might only be the initial step towards future programming languages where this kind of issue doesn't lean so much on developer knowledge.
In Rust, lifetimes color your types, like async colors your functions.
It is a great condensed summary of what makes lifetimes a great difficulty of async Rust. It's a language that has the function coloring that is typically introduced by async (likewise in JavaScript), and on top of that the typesystem itself gets colored by lifetime annotations. You can have a well written and working program... then due to some new need or refactoring, wanting to add a 'static somewhere will cause it all to break down.
It's part of the language, nothing bad with that. But it is an extra layer of difficulty that needs to be mastered. To me it shows that Rust might only be the initial step towards future programming languages where this kind of issue doesn't lean so much on developer knowledge.
Can you post a link to this video? Sounds interesting
Wasn't sure, but found it!
https://www.youtube.com/watch?v=p-tb1ZfkwgQ&t=340s
(in case it doesn't work: on the 5:40 mark)
https://www.youtube.com/watch?v=p-tb1ZfkwgQ&t=340s
(in case it doesn't work: on the 5:40 mark)
Thanks!
A GC would have made Rust yet another Dlang, which is pointless because Dlang has existed for a way longer time.
So why Dlang has not gotten as popular as rust or go?
a. it never had a real corporate backer, until it was too late (and Rust already stole all of its mindshare)
b. It had a garbage collector, which meant that realistically it could not replace C++. Or rather, it could but you basically had two different Dlangs - one with GC and classes, and one without. It was arguably not necessary, because people already had GC languages that were fast enough and worked for their needs, and those who needed speed were better served by C++, which arguably isn't that bad of a language if you know how to use it.
Rust is arguably the first real contender to C++'s reign because it really brings to the table features you'd be a fool to pass on. D was nice, but it was not worth the switch. Eliminating whole classes of bugs instead is.
Rust is arguably the first real contender to C++'s reign because it really brings to the table features you'd be a fool to pass on. D was nice, but it was not worth the switch. Eliminating whole classes of bugs instead is.
Lack of a big corporate backer is my best guess.
Or more realistically, another JVM language.
I actually wouldn't mind a subset of Rust that targets the JVM.
I actually wouldn't mind a subset of Rust that targets the JVM.
With GC rust would have been just another slightly different language.
Then you can use D lang.
Rust noob here. Can you please explain how removing '~' and '@' by moving the GC to a library makes async harder ?
I suspect GP only read the title and not the article. A Java-style GC might make async code easier, but that never existed in Rust.
Lifetimes complicated. Having a proof of them at compile time is difficult. You can't prove everything for starters. A lot of patterns are just a no-go. Why not defer that to the runtime, and just observe which variables stick around (that's GC)
IO complicated. The cycle of doing code then waiting for IO is wasteful (sequentially you waste millions of CPU cycles waiting for the network to answer back). To max out usage of your hardware resources you could just aggregate IO requests with your compiler. Switching back and forth between code that uses IO, as IO request come back (that's async)
Problem is: switching back and forth between code that uses IO is recklessly hard wrt coming up with a proof of the lifetimes of the variables.
And a language/runtime _needs_ to have the trifecta of compiler/GC-or-memory-management/memory-model coherent and within the runtime.
compiler/GC-or-memory-management: the GC-or-memory-management needs to know who writes, who owns, who reads
GC-or-memory-management/memory-model: you need to know when and how you can read your writes, what are the rules
memory-model/compiler: you'll be managing memory barriers so that you can cram together sequences of writes that are compatible, for maximal performance
This trifecta dependence is foundation to a language/runtime, and a change to one affects the others quite deeply. Changing the compiler (bringing async here) affects the other ends and you can't do that when GC-or-memory-management is all over the place (as a lib, or god forbid in user hands)
I'm afraid async is just something unaffordable for a language that wants to be that close to the metal. And even then, async is just a bandaid for a costly threading IO model.
----
Come over to the dark side of Erlang, Go, and Java. We have small threads now. You can just block like there is no tomorrow, and the runtime will have a cheap back and forth. You can just forget about the lifetimes, as the GC will sweep after you (and concurrently, outside of the critical allocation path). Forget it all my friend. Java is love, Java is life.
IO complicated. The cycle of doing code then waiting for IO is wasteful (sequentially you waste millions of CPU cycles waiting for the network to answer back). To max out usage of your hardware resources you could just aggregate IO requests with your compiler. Switching back and forth between code that uses IO, as IO request come back (that's async)
Problem is: switching back and forth between code that uses IO is recklessly hard wrt coming up with a proof of the lifetimes of the variables.
And a language/runtime _needs_ to have the trifecta of compiler/GC-or-memory-management/memory-model coherent and within the runtime.
compiler/GC-or-memory-management: the GC-or-memory-management needs to know who writes, who owns, who reads
GC-or-memory-management/memory-model: you need to know when and how you can read your writes, what are the rules
memory-model/compiler: you'll be managing memory barriers so that you can cram together sequences of writes that are compatible, for maximal performance
This trifecta dependence is foundation to a language/runtime, and a change to one affects the others quite deeply. Changing the compiler (bringing async here) affects the other ends and you can't do that when GC-or-memory-management is all over the place (as a lib, or god forbid in user hands)
I'm afraid async is just something unaffordable for a language that wants to be that close to the metal. And even then, async is just a bandaid for a costly threading IO model.
----
Come over to the dark side of Erlang, Go, and Java. We have small threads now. You can just block like there is no tomorrow, and the runtime will have a cheap back and forth. You can just forget about the lifetimes, as the GC will sweep after you (and concurrently, outside of the critical allocation path). Forget it all my friend. Java is love, Java is life.
Rather than those operators and how GC is done, I think the key aspect is how much easier async is in other languages that have easy automatic memory manage without ownership through garbage collection. Go and JavaScript/Dart are good examples.
But such models would also take away everything that makes Rust... Rust.
But such models would also take away everything that makes Rust... Rust.
it's not the things that were removed, it's the things you have to add to the source now to make it work within the limits of the borrow checker.
in the simplest case, you'd add Arc<..> everywhere @ used to be.
in the simplest case, you'd add Arc<..> everywhere @ used to be.
Rust is a language for writing the GC, runtime, OS kernel, etc.
And?
Plenty of GC enabled system programming languages have achieved similar feats, with bigger outcome than Rust has managed to on the desktop space, e.g. Xerox Workstations, across Smalltalk, Intelisp-D and Mesa/Cedar.
Redox is still not as feature rich as Mesa/Cedar was on the Dorado in 1981.
https://www.youtube.com/watch?v=z_dt7NG38V4
By the way Go, D, Eiffel, Nim, Common Lisp, Scheme, some JVM implementations are bootstrapped, meta-circular, with their own GC implemented on them.
Plenty of GC enabled system programming languages have achieved similar feats, with bigger outcome than Rust has managed to on the desktop space, e.g. Xerox Workstations, across Smalltalk, Intelisp-D and Mesa/Cedar.
Redox is still not as feature rich as Mesa/Cedar was on the Dorado in 1981.
https://www.youtube.com/watch?v=z_dt7NG38V4
By the way Go, D, Eiffel, Nim, Common Lisp, Scheme, some JVM implementations are bootstrapped, meta-circular, with their own GC implemented on them.
call me crazy, but I found old-style Rust, runtime and all, much more appealing.
Then choose from one of the countless garbage-collected languages with sum types and a runtime; Rust is successful because it's offering us systems programmers something unique.
Rust has other nice features but I started with Rust not in spite of borrow checker but because of it. Without borrow checker why would one choose Rust? I would choose from large pool of GC-languages.
You're not alone.
They are treading a line that is uniquely difficult to tread, and I think it’s mostly working. Async is still a bit of a mess but it seems that’s because it’s inherent to the constraints they had to impose on themselves.
It’s kind of cool that I can use Async stuff in embedded devices[1] and a web app, even if I do get frustrated with mind-numbing Async issues from time to time.
1. https://github.com/rust-lang/rfcs/pull/230
2. https://embassy.dev/