Six ways to make async Rust easier(carllerche.com)
carllerche.com
Six ways to make async Rust easier
https://carllerche.com/2021/06/17/six-ways-to-make-async-rust-easier/
56 comments
I believe it would be possible to implement using an edition in a way that any *old* code is compatible with new code.
Regarding implicitness, a lot in Rust already is implicit (e.g. type inference). Good implicitness only feels weird at first, before we get used to it. I linked to Aaron Turon's article on reasoning footprint, but as a follow up, there is withoutboat's article on "not explicit" that goes into the topic more: https://boats.gitlab.io/blog/post/2017-12-27-things-explicit...
Regarding implicitness, a lot in Rust already is implicit (e.g. type inference). Good implicitness only feels weird at first, before we get used to it. I linked to Aaron Turon's article on reasoning footprint, but as a follow up, there is withoutboat's article on "not explicit" that goes into the topic more: https://boats.gitlab.io/blog/post/2017-12-27-things-explicit...
It feels like the fundamental issue here is shared mutable state, and much of the solution is to just not have that.
I don't believe this requires a change to async-await though. It just means you want to wrap up your async-await in its own synchronization primitive, or just avoid the need for it.
This is easy to do with a channel system, or an actor system, where the queue is the synchronizing primitive. An actor's message handler can internally be asynchronous, so long as it processes messages one at a time (but it can yield during processing to another actor).
We can get actors in rust pretty trivially imo. It's just a task with state and a message interface.
I don't believe this requires a change to async-await though. It just means you want to wrap up your async-await in its own synchronization primitive, or just avoid the need for it.
This is easy to do with a channel system, or an actor system, where the queue is the synchronizing primitive. An actor's message handler can internally be asynchronous, so long as it processes messages one at a time (but it can yield during processing to another actor).
We can get actors in rust pretty trivially imo. It's just a task with state and a message interface.
Interesting, could you elaborate on what you mean by "shared mutable state"? In the parse_line example (first one in the post), what state is shared and with whom is it shared?
Not GP, but perhaps he's thinking of the TcpStream, which is a sort of "shared state" between executions of parse_line?
Canceling an async task feels a lot like aborting a partially executed DB transaction without rolling back any changes.
Canceling an async task feels a lot like aborting a partially executed DB transaction without rolling back any changes.
&TcpStream is a shared reference with interior mutability, and its state can change when you await and let other futures take a turn at accessing the same TcpStream. I prefer how CondVar encodes this property into its types: it gives you a MutexGuard (providing exclusive access), but requires you explicitly release it when you let other threads have a turn at accessing the shared data: https://doc.rust-lang.org/std/sync/struct.Condvar.html#metho...
Is there a reason the function in the article couldn't be written to accept a &mut TcpStream, which doesn't let others interact with the same TcpStream while the function is running or awaiting?
Is there a reason the function in the article couldn't be written to accept a &mut TcpStream, which doesn't let others interact with the same TcpStream while the function is running or awaiting?
I imagine it's &TcpStream because the TcpStream is used by both the reading and writing halves. There are helpers to split this [1] but maybe carlleche avoided them in this example for simplicity. [edit: oh, I guess the real TcpStream is actually like this. [2] The syscalls don't require &mut of course because that state's in the kernel, and I guess they didn't choose to add a &mut to enforce reasoning about the state. But in real code you'd wrap it in a buffer, and that would require &mut anyway.]
I don't think making it &mut TcpStream would solve the problem. It's not that something else is reading between a parse_line() call's read_u32() and read_exact() but that the parse_line() future is going away entirely between those and the read_exact() never happens (or never completes, as it's probably not atomic either). When that happens, it leaves the TcpStream in an unexpected state (halfway through a line) and the next parse_line() is misaligned.
But I do think having one task which owns reading from the stream to completion would solve the problem. If it dies, the whole TcpStream does too. It could talk with other tasks via channels with atomic messages. (Although I've been trying to minimize my use of channels right now because neither the tokio crates nor the futures crates offer an unbuffered/rendezvous channel, and I don't like sticking extra buffers in the middle of everything.)
[1] https://docs.rs/tokio/1.7.0/tokio/io/fn.split.html
[2] https://docs.rs/tokio/1.7.0/tokio/net/struct.TcpStream.html
I don't think making it &mut TcpStream would solve the problem. It's not that something else is reading between a parse_line() call's read_u32() and read_exact() but that the parse_line() future is going away entirely between those and the read_exact() never happens (or never completes, as it's probably not atomic either). When that happens, it leaves the TcpStream in an unexpected state (halfway through a line) and the next parse_line() is misaligned.
But I do think having one task which owns reading from the stream to completion would solve the problem. If it dies, the whole TcpStream does too. It could talk with other tasks via channels with atomic messages. (Although I've been trying to minimize my use of channels right now because neither the tokio crates nor the futures crates offer an unbuffered/rendezvous channel, and I don't like sticking extra buffers in the middle of everything.)
[1] https://docs.rs/tokio/1.7.0/tokio/io/fn.split.html
[2] https://docs.rs/tokio/1.7.0/tokio/net/struct.TcpStream.html
> I imagine it's &TcpStream because the TcpStream is used by both the reading and writing halves.
Yes, and in fact in older versions of tokio reading and writing required &mut, even for things like UdpSocket, which was a bit of a pain when you wanted to write to a socket concurrently as you couldn't just use an Arc.
Yes, and in fact in older versions of tokio reading and writing required &mut, even for things like UdpSocket, which was a bit of a pain when you wanted to write to a socket concurrently as you couldn't just use an Arc.
> It's not that something else is reading between a parse_line() call's read_u32() and read_exact() but that the parse_line() future is going away entirely between those and the read_exact() never happens (or never completes, as it's probably not atomic either). When that happens, it leaves the TcpStream in an unexpected state (halfway through a line) and the next parse_line() is misaligned.
Oops. I stand corrected.
Oops. I stand corrected.
Yep, you nailed it.
The mutable state isn't explicit here (it's internal mutability), but it's the TcpStream.
In an actor system the TcpStream would be local to an actor. You'd say "parse_line" by sending it a message. That message would be processed to-completion before another message could be processed - even if there is yielding internally.
You could also wrap TcpStream with some sort of wrapper that doesn't expose its internal mutability - that would let the type system ensure there isn't shared mutability across tasks I think...
In an actor system the TcpStream would be local to an actor. You'd say "parse_line" by sending it a message. That message would be processed to-completion before another message could be processed - even if there is yielding internally.
You could also wrap TcpStream with some sort of wrapper that doesn't expose its internal mutability - that would let the type system ensure there isn't shared mutability across tasks I think...
> I believe that the better reason for asynchronous is it enables modeling complex flow control efficiently. For example, patterns like pausing or canceling an in-flight operation are challenging without asynchronous programming.
This is exactly why I think async/await is a must have for any language aimed at back-end. Nodejs has a lot of pitfalls (promise cancellation being a major one), but moving from it to Go felt like huge step back to me: having to use channels to synchronize your tasks feels really cumbersome when you're used to promises/Future with combinators and async/await.
This is exactly why I think async/await is a must have for any language aimed at back-end. Nodejs has a lot of pitfalls (promise cancellation being a major one), but moving from it to Go felt like huge step back to me: having to use channels to synchronize your tasks feels really cumbersome when you're used to promises/Future with combinators and async/await.
I obviously can't see your code, but in my experience, working with Go daily for years, explicit channels are an extremely rare occurrence.
If you want to run multiple tasks and wait for them to finish, you use a WaitGroup. If you want to stop processing on first error encountered, you use an ErrGroup. And cancellation with contexts also works very well overall.
I actually think Go does it way better than all the "visible async/await" languages. Because usually when you write code, you just want to await the result, so in Go, awaiting is the default. In order to run a task without awaiting immediately, you use the 'go' keyword (at least that's the semantics of it all). It also omits the function coloring problem (this is a very big advantage), because all functions are async.
For my use cases channels are usually only useful in two situations:
1. Asychronous producers and consumers.
2. Creating an "actor" which receives messages on a channel/s and processes them in a serialized manner. Where the messages come from multiple event sources.
As an aside, I do understand why performance oriented languages like Rust choose the async/await path, as async by default carries a performance penalty / requires a runtime. I don't however accept it in non-performance oriented languages.
If you want to run multiple tasks and wait for them to finish, you use a WaitGroup. If you want to stop processing on first error encountered, you use an ErrGroup. And cancellation with contexts also works very well overall.
I actually think Go does it way better than all the "visible async/await" languages. Because usually when you write code, you just want to await the result, so in Go, awaiting is the default. In order to run a task without awaiting immediately, you use the 'go' keyword (at least that's the semantics of it all). It also omits the function coloring problem (this is a very big advantage), because all functions are async.
For my use cases channels are usually only useful in two situations:
1. Asychronous producers and consumers.
2. Creating an "actor" which receives messages on a channel/s and processes them in a serialized manner. Where the messages come from multiple event sources.
As an aside, I do understand why performance oriented languages like Rust choose the async/await path, as async by default carries a performance penalty / requires a runtime. I don't however accept it in non-performance oriented languages.
> As an aside, I do understand why performance oriented languages like Rust choose the async/await path, as async by default carries a performance penalty / requires a runtime. I don't however accept it in non-performance oriented languages.
Ironically, I think Rust futures could be expanded into Go-like uncolored functions relatively easily without even much of a runtime. Since they're stack allocated by default, driving a future to completion is a matter of calling Future::poll in a while loop and handling the waker callback. I feel like the runtimes are there mostly to provide a unified interface to kernel APIs like io_uring/epoll/etc and ease ownership/task allocation.
Ironically, I think Rust futures could be expanded into Go-like uncolored functions relatively easily without even much of a runtime. Since they're stack allocated by default, driving a future to completion is a matter of calling Future::poll in a while loop and handling the waker callback. I feel like the runtimes are there mostly to provide a unified interface to kernel APIs like io_uring/epoll/etc and ease ownership/task allocation.
Go really does do this better. Go is a green thread system.
In Rust, if you're actually doing any compute work, you're stalling out the async system. In Go, if you compute for a while, the scheduler will let someone else run. You can get all the CPUs working. This is well matched to writing web back ends.
Rust's "Async" seems to be designed for a very specific use case - a program running a very large number of network connections, most of which are waiting. If you're doing something where you need all available CPUs to get the work done, it's a bad fit.
In Rust, if you're actually doing any compute work, you're stalling out the async system. In Go, if you compute for a while, the scheduler will let someone else run. You can get all the CPUs working. This is well matched to writing web back ends.
Rust's "Async" seems to be designed for a very specific use case - a program running a very large number of network connections, most of which are waiting. If you're doing something where you need all available CPUs to get the work done, it's a bad fit.
> In Rust, if you're actually doing any compute work, you're stalling out the async system. In Go, if you compute for a while, the scheduler will let someone else run. You can get all the CPUs working. This is well matched to writing web back ends.
For the first ten years of Go at least, it was pretty easy to stall the system as well, since the scheduler wasn't able to preempt in the middle of a hot loop[1]. This has only been fixed in 1.14 last year!
> Rust's "Async" seems to be designed for a very specific use case - a program running a very large number of network connections, most of which are waiting. If you're doing something where you need all available CPUs to get the work done, it's a bad fit.
In Rust, if you have a CPU intensive job to do, you can use threads (and as it's completely orthogonal to async, it combines well with it).
[1]: https://github.com/golang/go/issues/10958
For the first ten years of Go at least, it was pretty easy to stall the system as well, since the scheduler wasn't able to preempt in the middle of a hot loop[1]. This has only been fixed in 1.14 last year!
> Rust's "Async" seems to be designed for a very specific use case - a program running a very large number of network connections, most of which are waiting. If you're doing something where you need all available CPUs to get the work done, it's a bad fit.
In Rust, if you have a CPU intensive job to do, you can use threads (and as it's completely orthogonal to async, it combines well with it).
[1]: https://github.com/golang/go/issues/10958
The trouble with a green thread system is that you can't opt out. Like the article's example of everything being implicitly cancellable by a higher-level select, in a green thread system everything is implicitly threadshiftable by a lower-level function.
If your scheduler works right, then green threads are great, but when your scheduler breaks (and eventually it will) they're impossible to fix. Lightweight-but-not-invisible yields keep things as simple as possible, but no simpler.
If your scheduler works right, then green threads are great, but when your scheduler breaks (and eventually it will) they're impossible to fix. Lightweight-but-not-invisible yields keep things as simple as possible, but no simpler.
Sure you can and that is one of the design ideas behind Project Loom or .NET Tasks.
Have a scheduler API available that is able to take those decisions, while providing default schedulers for the most common patterns.
Just like in many design decisions, Go design team just decided not to expose the same level of power to their users.
Have a scheduler API available that is able to take those decisions, while providing default schedulers for the most common patterns.
Just like in many design decisions, Go design team just decided not to expose the same level of power to their users.
> In Rust, if you're actually doing any compute work, you're stalling out the async system. In Go, if you compute for a while, the scheduler will let someone else run. You can get all the CPUs working. This is well matched to writing web back ends.
This is not a choice in Rust but instead a choice that is handled differently in the async runtimes, which are libraries. If I'm not mistaken, async-std (an async runtime in Rust) adds automatic yields similarly to how Go does.
This is not a choice in Rust but instead a choice that is handled differently in the async runtimes, which are libraries. If I'm not mistaken, async-std (an async runtime in Rust) adds automatic yields similarly to how Go does.
> In Rust, if you're actually doing any compute work, you're stalling out the async system.
Just use threads? Or add explicit yields.
Just use threads? Or add explicit yields.
explicit yields.
That's so 1984 Macintosh. "Cooperative multitasking".
The usual effect of that style of programming is stuck window managers.
That's so 1984 Macintosh. "Cooperative multitasking".
The usual effect of that style of programming is stuck window managers.
So use threads! That's what they're there for.
Go's solution is threads, just with a particularly idiosyncratic M:N implementation. Rust threads are 1:1. But they're the same concept. You can even get M:N if you really want it (nobody does, which is why mioco gets no use).
Go's solution is threads, just with a particularly idiosyncratic M:N implementation. Rust threads are 1:1. But they're the same concept. You can even get M:N if you really want it (nobody does, which is why mioco gets no use).
There's nothing wrong with explicit yields within a single process. It's only when the kernel scheduler can't preempt you that you end up with stuck window managers.
Exactly. It's painful that the same problems get revisited as new things.
Computation needs to be scheduled to a CPU or core for parallel progress to occur. That means kernel threads.
Computation needs to be scheduled to a CPU or core for parallel progress to occur. That means kernel threads.
You still get preemption at the thread level.
If you call a library function, not written by you, where you don't know whether it's going to return in a few nanoseconds or might take a long time, you the caller can't do the right thing yourself.
A thread is too inefficient for the fast calls, but no thread is disastrous for the rest of the program's concurrency for the slow calls.
So you're reliant on the library, not written by you, to always choose an efficient behaviour for each call. In other words, to decide dynamically when to use a thread, and when not to use a thread. And that library is reliant on all of its transitive dependencies in the same way.
It's a nice ideal, but it's not realistic.
And it still doesn't work all that efficiently anyway. If you call a library function many times, or just many functions, and any of those calls (outside your knowledge) has some long-running part where a thread ought to be used, to do it efficiently a single thread would span your code's multiple calls, and not just last for one library call.
Essentially, to know whether to make a library call in a thread or not, and at what level to use a thread, you need to know what the behaviour of that call is going to be in advance. In practice, even when carefully optimising programs, we almost always assume one behaviour or another. Maybe measure, and assume the measured behaviour will remain similar. This doesn't have good adaptive behaviour when the assumption is wrong or out of date, so it doesn't work well for calls to things whose duration varies a lot, and especially not for large systems of calls between different modules, each of which has unpredictable timing.
I first encounted this issue in a video games engine, long before Go or Rust existed (and before async-await). NPCs usually did a tiny amount of calculation, and there where tens of thousands, called every display frame, so this had to be fast. But some of them, known only to themselves, occasionally decided to make network calls, read cached data (which might need network calls or not), or call the filesystem, or do intensive "AI" planning using a burst of CPU. To maintain the frame rate while efficiently processing all NPCs, it was necessary to detect when some deep library call from an NPC had used the NPC's "short call" time budget, and switch it over to a thread.
In that video games environment, the Go model worked. No matter how each NPC was coded and by whomever did it, an NPC might slow down if it called something slow but the game would stay smooth, and all other NPCs would continue to update at the full frame rate. The Rust model would result in the whole game frame rate stalling, which is much less acceptable. The threads-when-anything-could-be-slow model was too slow, and the use-threads-when-you-will-be-slow model gave NPC code too much power to affect the system outside the NPC.
A thread is too inefficient for the fast calls, but no thread is disastrous for the rest of the program's concurrency for the slow calls.
So you're reliant on the library, not written by you, to always choose an efficient behaviour for each call. In other words, to decide dynamically when to use a thread, and when not to use a thread. And that library is reliant on all of its transitive dependencies in the same way.
It's a nice ideal, but it's not realistic.
And it still doesn't work all that efficiently anyway. If you call a library function many times, or just many functions, and any of those calls (outside your knowledge) has some long-running part where a thread ought to be used, to do it efficiently a single thread would span your code's multiple calls, and not just last for one library call.
Essentially, to know whether to make a library call in a thread or not, and at what level to use a thread, you need to know what the behaviour of that call is going to be in advance. In practice, even when carefully optimising programs, we almost always assume one behaviour or another. Maybe measure, and assume the measured behaviour will remain similar. This doesn't have good adaptive behaviour when the assumption is wrong or out of date, so it doesn't work well for calls to things whose duration varies a lot, and especially not for large systems of calls between different modules, each of which has unpredictable timing.
I first encounted this issue in a video games engine, long before Go or Rust existed (and before async-await). NPCs usually did a tiny amount of calculation, and there where tens of thousands, called every display frame, so this had to be fast. But some of them, known only to themselves, occasionally decided to make network calls, read cached data (which might need network calls or not), or call the filesystem, or do intensive "AI" planning using a burst of CPU. To maintain the frame rate while efficiently processing all NPCs, it was necessary to detect when some deep library call from an NPC had used the NPC's "short call" time budget, and switch it over to a thread.
In that video games environment, the Go model worked. No matter how each NPC was coded and by whomever did it, an NPC might slow down if it called something slow but the game would stay smooth, and all other NPCs would continue to update at the full frame rate. The Rust model would result in the whole game frame rate stalling, which is much less acceptable. The threads-when-anything-could-be-slow model was too slow, and the use-threads-when-you-will-be-slow model gave NPC code too much power to affect the system outside the NPC.
I bet if you implemented this system with every NPC getting a separate thread, it would be quite comparable in performance to the Go implementation on a modern OS. Goroutines are heavier, and OS threads lighter, than many people think.
(This speaks more to the fact that goroutine-per-entity wouldn't be feasible than to the idea that thread-per-entity would be.)
(This speaks more to the fact that goroutine-per-entity wouldn't be feasible than to the idea that thread-per-entity would be.)
I don't know about goroutine performance, and maybe you're right about their weight relative to modern OS threads.
I'm talking about the scheduling model rather than the specific implementations in Go and Rust. The game engine I worked predates them both, and at the time, there is no way 10,000+ OS threads could advance in every rendering frame. Just entering and exiting the kernel for each thread would take longer than the frame budget. It had to be a userspace queue.
I'm talking about the scheduling model rather than the specific implementations in Go and Rust. The game engine I worked predates them both, and at the time, there is no way 10,000+ OS threads could advance in every rendering frame. Just entering and exiting the kernel for each thread would take longer than the frame budget. It had to be a userspace queue.
What you're arguing for is task preemption, and it has a global cost. If you're willing to pay global costs, Rust is probably not the right language for you.
Technically it's not pre-emption. It's activating parallelism on another CPU core when a time condition is triggered, in order to reduce tail latencies of unrelated tasks and prevent starvation. Nothing is directly pre-empted by that event.
But let's run with pre-emption, because that does apply on single cores.
No, it doesn't work out to be a global cost, assuming you mean performance (as opposed to developer cognitive cost). I have experience with this - the game engine. We don't half-ass performance in games like that, it's a central factor. We optimise for the highest performance on middle-of-the-road hardware, and measure constantly.
The Rust model is same or worse on each global performance metric that matters in that scenario. The global metrics are things like graphics throughput, tail latency and variance, interactive response, task starvation. So the "cost" of "pre-emption" ranges from zero to negative.
I will grant you that pre-emption adds shared resource locking overheads if you're comparing with an async-await model that uses only a single thread for all tasks globally, so locking is not required. (I.e. JavaScript). But that's not the Rust model.
Pre-emption has a small, positive local cost, in that you need to check when to pre-empt. But that can be made small. You don't need a kernel to do it, and you don't need to be constantly reading clocks either. The cost is a few nanoseconds per microtask, i.e. book-keeping like with any execution loop, and some periodic interrupts or signals, which you might have anyway. Alternatively you can use performance counters on some CPUs. The actual act of pre-empting is also cheap - it's just a coroutine jump with some book-keeping. But it's also comparatively rare compared with async task transitions, so the cost doesn't really matter anyway.
But let's run with pre-emption, because that does apply on single cores.
No, it doesn't work out to be a global cost, assuming you mean performance (as opposed to developer cognitive cost). I have experience with this - the game engine. We don't half-ass performance in games like that, it's a central factor. We optimise for the highest performance on middle-of-the-road hardware, and measure constantly.
The Rust model is same or worse on each global performance metric that matters in that scenario. The global metrics are things like graphics throughput, tail latency and variance, interactive response, task starvation. So the "cost" of "pre-emption" ranges from zero to negative.
I will grant you that pre-emption adds shared resource locking overheads if you're comparing with an async-await model that uses only a single thread for all tasks globally, so locking is not required. (I.e. JavaScript). But that's not the Rust model.
Pre-emption has a small, positive local cost, in that you need to check when to pre-empt. But that can be made small. You don't need a kernel to do it, and you don't need to be constantly reading clocks either. The cost is a few nanoseconds per microtask, i.e. book-keeping like with any execution loop, and some periodic interrupts or signals, which you might have anyway. Alternatively you can use performance counters on some CPUs. The actual act of pre-empting is also cheap - it's just a coroutine jump with some book-keeping. But it's also comparatively rare compared with async task transitions, so the cost doesn't really matter anyway.
Preemption is costly, because you need to save the entire state of the execution you're preempting, in order to restore it later (which is also, as you should know, what makes context-switches slow). Go and async Rust are both cooperatively scheduled, switching back and forth only in specific yield points, which reduces the amount of state that needs to be saved. Adding such a yield point is costly too though, that's why Go didn't add them inside tight loops until 1.14.[1]
Regarding scheduling and how task-switching is done, async Rust and Go are fundamentally the same model (and BTW, the authors of Tokio never hide that they took a lot of inspiration from the Go runtime). The three big differences are the following:
- Go's tasks are stackful, they have their own stack, which is grow-able (the stack is copied into a bigger stack when it's full)
- Go's yield points are automatically inserted by the compiler based on heuristics which vary depending on the version of the compiler. Whereas Rust yield point are manually inserted by the developer. This resonate strongly with the philosophy of both languages (best-effort automatic vs explicit manual control leading to the best performance) which can also be seen on the topic of memory allocation (automatic boxing based on escape analysis vs manual Box pointer).
- Go only has green thread, which means that if the runtime is failing to keep its promises[1][2], you don't have much alternative. Rust has both Async tasks and OS threads, meaning you can choose what works best for your workload.
[1] https://github.com/golang/go/issues/10958 [2] https://github.com/golang/go/issues/36365
Regarding scheduling and how task-switching is done, async Rust and Go are fundamentally the same model (and BTW, the authors of Tokio never hide that they took a lot of inspiration from the Go runtime). The three big differences are the following:
- Go's tasks are stackful, they have their own stack, which is grow-able (the stack is copied into a bigger stack when it's full)
- Go's yield points are automatically inserted by the compiler based on heuristics which vary depending on the version of the compiler. Whereas Rust yield point are manually inserted by the developer. This resonate strongly with the philosophy of both languages (best-effort automatic vs explicit manual control leading to the best performance) which can also be seen on the topic of memory allocation (automatic boxing based on escape analysis vs manual Box pointer).
- Go only has green thread, which means that if the runtime is failing to keep its promises[1][2], you don't have much alternative. Rust has both Async tasks and OS threads, meaning you can choose what works best for your workload.
[1] https://github.com/golang/go/issues/10958 [2] https://github.com/golang/go/issues/36365
> Preemption is costly, because you need to save the entire state of the execution you're preempting, in order to restore it later (which is also, as you should know, what makes context-switches slow).
In-userspace stackful context switches are not at all slow in a sane userspace-switching runtime. They are approximately a combination of setjmp() and longjmp(), but without saving all registers. Typically a few tens of nanoseconds or less on a modern CPU. Pre-emptive context switches are nearly always stackful, even if non-pre-emptive (async) context switches are stackless in the same runtime.
A pre-emptive context switch in a userspace co-operative scheduling system is slower than non-pre-emptive context switch if it is caused by a dedicated interrupt of some kind. In the case of userspace pre-emption, generally a signal and return, typically single digit microseconds.
However, the cost there is mainly the signal. If the pre-emptive context switch is caused by a signal that triggered anyway for another purpose, the actual context switch is, again, cheap.
In the model I described (the game environment), pre-emptive context switches are rare. It wouldn't matter if they took longer, because >= 99.99% of context switches are co-operative in that model. The important factor is that no task blows the frame budget or prevents other tasks from running at the full frame rate. In web services a similar target is tail latency, in the presence of diverse tasks you cannot predict in advance.
> meaning you can choose what works best for your workload.
Exactly. And as I've tried to explain, for some varying, complex workloads whatever you choose statically has poor metrics; it must be adaptive to maintain good timing metrics.
> Rust yield point are manually inserted by the developer
Indeed, and in the case of NPCs in a game engine, or a large program composed of many libraries written by hundreds of different authors (e.g. some web servers), that causes complex, interdependent performance characteristics, where timing behaviour in one of them ruins performance for everything, unless they are isolated using threads, in which case it's too slow. There is no "the developer" who can ensure this doesn't happen.
(Aside, if Rust's type system helped with this, that would be great, the same way types help with other "programming in the large" safety issues, but it doesn't address timing characteristics as far as I know.)
> leading to the best performance [..] Rust has both Async tasks and OS threads, meaning you can choose what works best for your workload.
You could summarise my point as: "For some dynamic workloads, especially in timing-sensitive large programs with many components working independently and unpredictably, neither async tasks or OS threads perform best for your workload (or even adequately sometimes). The optimal (or required) combination requires some dynamic responses, and cannot be achieved solely by static placement of yield points and thread initiations."
In-userspace stackful context switches are not at all slow in a sane userspace-switching runtime. They are approximately a combination of setjmp() and longjmp(), but without saving all registers. Typically a few tens of nanoseconds or less on a modern CPU. Pre-emptive context switches are nearly always stackful, even if non-pre-emptive (async) context switches are stackless in the same runtime.
A pre-emptive context switch in a userspace co-operative scheduling system is slower than non-pre-emptive context switch if it is caused by a dedicated interrupt of some kind. In the case of userspace pre-emption, generally a signal and return, typically single digit microseconds.
However, the cost there is mainly the signal. If the pre-emptive context switch is caused by a signal that triggered anyway for another purpose, the actual context switch is, again, cheap.
In the model I described (the game environment), pre-emptive context switches are rare. It wouldn't matter if they took longer, because >= 99.99% of context switches are co-operative in that model. The important factor is that no task blows the frame budget or prevents other tasks from running at the full frame rate. In web services a similar target is tail latency, in the presence of diverse tasks you cannot predict in advance.
> meaning you can choose what works best for your workload.
Exactly. And as I've tried to explain, for some varying, complex workloads whatever you choose statically has poor metrics; it must be adaptive to maintain good timing metrics.
> Rust yield point are manually inserted by the developer
Indeed, and in the case of NPCs in a game engine, or a large program composed of many libraries written by hundreds of different authors (e.g. some web servers), that causes complex, interdependent performance characteristics, where timing behaviour in one of them ruins performance for everything, unless they are isolated using threads, in which case it's too slow. There is no "the developer" who can ensure this doesn't happen.
(Aside, if Rust's type system helped with this, that would be great, the same way types help with other "programming in the large" safety issues, but it doesn't address timing characteristics as far as I know.)
> leading to the best performance [..] Rust has both Async tasks and OS threads, meaning you can choose what works best for your workload.
You could summarise my point as: "For some dynamic workloads, especially in timing-sensitive large programs with many components working independently and unpredictably, neither async tasks or OS threads perform best for your workload (or even adequately sometimes). The optimal (or required) combination requires some dynamic responses, and cannot be achieved solely by static placement of yield points and thread initiations."
Goroutines are asynchronously preemptible since Go 1.14, released February 25th, 2020.
https://blog.golang.org/go1.14
https://blog.golang.org/go1.14
Concurrency and parallelism are different things (and the existence of an implementation for one does not preclude the other).
You make it sound like Rust doesn’t support background CPU computation. Rust has always had threads (https://doc.rust-lang.org/book/ch16-01-threads.html), along with incredibly powerful libraries like Rayon (https://github.com/rayon-rs/rayon) to take advantage of them.
async/await wasn’t specifically targeted for that use case because it didn’t need to be. But you can use threads with async, if you want, using a multi-threaded scheduler like Tokio with the rt-multi-thread feature flag (https://docs.rs/tokio/1.7.0/tokio/runtime/index.html#multi-t...).
async/await wasn’t specifically targeted for that use case because it didn’t need to be. But you can use threads with async, if you want, using a multi-threaded scheduler like Tokio with the rt-multi-thread feature flag (https://docs.rs/tokio/1.7.0/tokio/runtime/index.html#multi-t...).
What you're describing is not actually a problem with async/await. Rather, async/await places a burden of knowledge on the developer to explicitly avoid this problem. Go makes a tradeoff on overhead in exchange for removing that burden of knowledge.
Explicit cooperative multitasking absolutely can be used effectively for CPU-bound work, but it requires the developer to know how to do that, rather than relying on preemption to cover for them. It's similar to the different pros and cons of garbage collection vs. explicit memory management.
Explicit cooperative multitasking absolutely can be used effectively for CPU-bound work, but it requires the developer to know how to do that, rather than relying on preemption to cover for them. It's similar to the different pros and cons of garbage collection vs. explicit memory management.
> async/await places a burden of knowledge on the developer to explicitly avoid this problem
How many large projects do you know where the developers know every CPU run-length histogram of every library dependencies and their transitive dependencies? Even if they can measure them, they can't realistically alter many of them.
And how many library developers do you know who know the CPU run-length histogram expectations of everything that depends on them, as well as their own dependencies?
In the Go model, the system balances competing modules dynamically in response to unpredictable load patterns, and ensures some amount of fairness. Something which is called, no matter how deep in the call stack and no matter how many steps removed from another module, can do some computation and it doesn't severely affect the rest of the program. In particular, it doesn't cause a giant spike in latency of processing unrelated things.
In the Rust model, timings of totally unrelated modules have a stronger effect on each other. Unrelated modules are not as decoupled. To be conservative, especially in a library, it's better to avoid any lengthy computations in your async functions, breaking them up in to smaller parts just in case something unrelated needs to be able to make progress. In such cases, preemption is more efficient.
The Rust model also leads to an interesting metastability in library design motivation among independent developers. A library that provides an async API and breaks its work up into many small, non-sequentially-dependent tasks will tend to get a higher share of CPU execution than one which uses fewer large tasks for the same job - because the scheduler is not trying to be fair. So there's an incentive for every library developer to break things up into many small async tasks, to make their own library perform better, even though that is less efficient overall.
Overall, I think the Rust async scheduling model is better suited to smaller programs than the Go scheduling model.
How many large projects do you know where the developers know every CPU run-length histogram of every library dependencies and their transitive dependencies? Even if they can measure them, they can't realistically alter many of them.
And how many library developers do you know who know the CPU run-length histogram expectations of everything that depends on them, as well as their own dependencies?
In the Go model, the system balances competing modules dynamically in response to unpredictable load patterns, and ensures some amount of fairness. Something which is called, no matter how deep in the call stack and no matter how many steps removed from another module, can do some computation and it doesn't severely affect the rest of the program. In particular, it doesn't cause a giant spike in latency of processing unrelated things.
In the Rust model, timings of totally unrelated modules have a stronger effect on each other. Unrelated modules are not as decoupled. To be conservative, especially in a library, it's better to avoid any lengthy computations in your async functions, breaking them up in to smaller parts just in case something unrelated needs to be able to make progress. In such cases, preemption is more efficient.
The Rust model also leads to an interesting metastability in library design motivation among independent developers. A library that provides an async API and breaks its work up into many small, non-sequentially-dependent tasks will tend to get a higher share of CPU execution than one which uses fewer large tasks for the same job - because the scheduler is not trying to be fair. So there's an incentive for every library developer to break things up into many small async tasks, to make their own library perform better, even though that is less efficient overall.
Overall, I think the Rust async scheduling model is better suited to smaller programs than the Go scheduling model.
> In the Go model, the system balances competing modules dynamically in response to unpredictable load patterns, and ensures some amount of fairness. Something which is called, no matter how deep in the call stack and no matter how many steps removed from another module, can do some computation and it doesn't severely affect the rest of the program. In particular, it doesn't cause a giant spike in latency of processing unrelated things.
There's some kind of magical thinking here. The Go runtime attempts to hide as much complexity as it can, and while it works OK most of the time, there are a lot of edge cases that the runtime doesn't handle well [1]. And implementing a runtime that handle these things automagically is a hard task, and nasty bugs ensue[2].
Also, scheduler fairness isn't related to the language itself, since Rust doesn't ship a scheduler the runtime being a third-party library.
[1]: https://github.com/golang/go/issues/36365 [2]: https://github.com/golang/go/issues/40722
There's some kind of magical thinking here. The Go runtime attempts to hide as much complexity as it can, and while it works OK most of the time, there are a lot of edge cases that the runtime doesn't handle well [1]. And implementing a runtime that handle these things automagically is a hard task, and nasty bugs ensue[2].
Also, scheduler fairness isn't related to the language itself, since Rust doesn't ship a scheduler the runtime being a third-party library.
[1]: https://github.com/golang/go/issues/36365 [2]: https://github.com/golang/go/issues/40722
I think the problem is that many people seem to think of rust as a back end language to begin with. And while there are crates that aim at that, the big problem there is that rust itself is not meant for that and it is simply bad at it. Rust is a systems language and this is where it shines. I'm currently using it for micro controllers and sensors and it is a perfect fit in this space. It genuinely feels like someone has given me a set of precision tools after doing everything with sticks and stones in the form of C. The reason why I think it's a bad choice for back end is not because of the limited ecosystem either. Take actix for example. In order to handle resources, your codebase inevitably becomes a huge pile of Arc<Mutex<_>> and Arc<RwLock<_>>s and mpsc's and sooner rather then later this end up biting back, most commonly in the form of an endless rabbit hole of read/write locks. And when I first saw sync/await in the rfcs, that's exactly what I feared would happen. Tokio is another elephant in the room that needs to be addressed: a ton of crates depend on different and completely incompatible versions of it. Most people get this wrong when they get overexcited about rust. And then there are those who get it right. Full disclosure, I haven't tried it or gone through the source code but InfluxDB IOx seems like a perfect choice for rust and a good use case for async/awaits.
I find continuations alot easier to reason about as a programmer. I think async/await was chosen here because in simple cases it pisses off the borrower less
Interesting, I find continuations utterly impossible to reason about, to the point where even if something is implemented with continuations I need some restricted model to reason about it in (e.g. iteratees).
It's not clear to me how an implicit await could be implemented; a counterexample that immediately comes to mind is any of the FutureExt combinators in the futures crate. The await keyword indicates the point at which something is done, but there could be multiple points in an expression that change the resulting type in the rest of the expression!
Ex: my_fut().map(|x| x.y).then(other_async_thing)
Awaiting after my_fut or .map() produces different types for resolving the rest of the expression.
Also, I've often done things like binding a future in a scope and later awaiting it at different times (but always completing it). Making await implicit would make this impossible, unless a backdoor was kept in to indicate that you want an explicit await (which just comes down to the call site and shouldn't affect the Future implementation).
Maybe these are moot points if futures must always complete, I haven't thought through the cases enough yet. Perhaps if futures can't be cancelled then FutureExt would not be necessary, and instead of passing `Future` types (into functions and elsewhere) we'd pass around `FnOnce() -> impl Future` types, or better yet `AsyncFnOnce()` since currently `FnOnce` suffers from not being able to specify a lifetime of an argument in the returned Future (which has hindered me many times).
Ex: my_fut().map(|x| x.y).then(other_async_thing)
Awaiting after my_fut or .map() produces different types for resolving the rest of the expression.
Also, I've often done things like binding a future in a scope and later awaiting it at different times (but always completing it). Making await implicit would make this impossible, unless a backdoor was kept in to indicate that you want an explicit await (which just comes down to the call site and shouldn't affect the Future implementation).
Maybe these are moot points if futures must always complete, I haven't thought through the cases enough yet. Perhaps if futures can't be cancelled then FutureExt would not be necessary, and instead of passing `Future` types (into functions and elsewhere) we'd pass around `FnOnce() -> impl Future` types, or better yet `AsyncFnOnce()` since currently `FnOnce` suffers from not being able to specify a lifetime of an argument in the returned Future (which has hindered me many times).
This is very interesting: I have a bit of an aversion to implicit await, but guaranteed completion does make that slightly less bad in my mind.
There are a few things that I felt were a bit glossed over:
1) How exactly does an executor fullfill the contract of the new unsafe "poll" method? eg. what happens if there is a panic in one of the futures, or in the executor itself? Even if the executor catches the panic, what about the future that itself panicked - clearly that will not be polled to completion.
2) How does cancellation work exactly? You mentioned that somehow the underlying async operations return an "interrupted" error. Who controls cancellation of a future? (the executor?). How does that agent know which "leaf futures" need to be interrupted? How does it signal to those "leaf futures" that they should return immediately? Does this mean that all cancellable futures must return a `Result<T, impl From<InterruptedError>>`?
3) Do you think this change would help with problems like https://github.com/http-rs/async-h1/pull/179 ?
There are a few things that I felt were a bit glossed over:
1) How exactly does an executor fullfill the contract of the new unsafe "poll" method? eg. what happens if there is a panic in one of the futures, or in the executor itself? Even if the executor catches the panic, what about the future that itself panicked - clearly that will not be polled to completion.
2) How does cancellation work exactly? You mentioned that somehow the underlying async operations return an "interrupted" error. Who controls cancellation of a future? (the executor?). How does that agent know which "leaf futures" need to be interrupted? How does it signal to those "leaf futures" that they should return immediately? Does this mean that all cancellable futures must return a `Result<T, impl From<InterruptedError>>`?
3) Do you think this change would help with problems like https://github.com/http-rs/async-h1/pull/179 ?
In the implicit await example, how does the compiler know which sections can be run in parallel and which require the results of previous computation.
async fn my_fn_one() -> uint32 {
println!("two");
5
}
async fn my_fn_two() {
println!("two");
}
async fn my_fn_three(x: uint32) {…}
async fn mixup() {
let
my_fn_two();
my_fn_three(one)
}
As far as I can tell there is no parallelism in this code (fyi rust doesn't have a uint32 type), if you want things to run in parallel you need to spawn a task just as you do with threads. Async gives you concurrency, each formerly blocking thing turns into a yield where a scheduler is free to run some other task, but it doesn't automatically give you parallelism.
> Does this mean then that one, two, three all have their execution scheduled synchronously with no way to express parallelism?
> Or is the compiler figuring out the dependencies and generating a magic state machine where non-dependencies are spawned in parallel and joined before returning?
No for both.
Firstly, when you do:
To your second question, the compiler won't create a dependency graph of futures and concurrently execute futures that are independent to each other. The good news is that futures can be executed concurrently, but you need to explicitly build a dependency graph, by spawning a task [1] and/or using `futures::join` macro [2].
For example, you can concurrently execute my_fn_two and my_fn_three by:
[2] https://docs.rs/futures/0.3.15/futures/macro.join.html
* or more precisely "of an anonymous type that implements"
> Or is the compiler figuring out the dependencies and generating a magic state machine where non-dependencies are spawned in parallel and joined before returning?
No for both.
Firstly, when you do:
let
let
let one_value: u32 = one_future.await;
async fn foo() {
let x = bar().await;
let y = 42; // This line won't be executed until awaiting bar is finished
let z = baz().await; // Neither will this line
}
To your second question, the compiler won't create a dependency graph of futures and concurrently execute futures that are independent to each other. The good news is that futures can be executed concurrently, but you need to explicitly build a dependency graph, by spawning a task [1] and/or using `futures::join` macro [2].
For example, you can concurrently execute my_fn_two and my_fn_three by:
async fn mixup() {
let
// Execute my_fn_two and my_fn_three concurrently.
// Notice that there is no await because we pass futures to futures::join.
futures::join!(my_fn_two(), my_fn_three(one_value));
}
async fn mixup() {
let {
let
my_fn_three(one_value).await;
};
futures::join!(my_fn_two(), one_three_future);
}
[2] https://docs.rs/futures/0.3.15/futures/macro.join.html
* or more precisely "of an anonymous type that implements"
This is exactly what I'm raising. You describe the system as today. I'm talking about the article's proposal of implicit await. Today, async functions return a Future<T>. With implicit await described in the article, they return a T. So, in an implicit await world, how do I express the mixup function?
I'm unfamiliar with the use of in as a keyword in the following snippet from the OP. I can't find anything on Google about it, either. Can anyone tell me more about it?
while let Some(line_in) in parse_line(&reader).await? {
broadcast_line(line_in)?;
}It's a mistake, it's supposed to be a "=".
Thanks, I thought something new got snuck in after taking my thumb off the pulse of Rust development a while back.
I don't think you're allowed to do this, but it's in a hypothetical, so the code doesn't actually work anyway.
If you were allowed to use in this way, it feels reasonable right? parse_line eventually gave us an Iterator of line_in values, we iterate it until we get a None instead of a Some with a line_in value, and that exits the loop.
If you were allowed to use in this way, it feels reasonable right? parse_line eventually gave us an Iterator of line_in values, we iterate it until we get a None instead of a Some with a line_in value, and that exits the loop.
> If you were allowed to use in this way, it feels reasonable right?
Yeah, that's why I was confused. I can imagine seeing something like this implemented with macros.
Yeah, that's why I was confused. I can imagine seeing something like this implemented with macros.
"With the 2026 edition, adding #[abort_safe] to an asynchronous statement would generate an AbortSafeFuture implementation instead of Future. Any async function written with pre-2026 editions implements the AbortSafeFuture trait, making all existing async code compatible with the new edition (recall that an abort-safe statement is callable from any context)."
I can't help but read this as a spoof on the spelling reform of the English language? [0]
0: http://www.i18nguy.com/twain.html
I can't help but read this as a spoof on the spelling reform of the English language? [0]
0: http://www.i18nguy.com/twain.html
These personas (i.e. “Alan”) are annoying enough when used by business people, but in the context of a programming language it’s beyond cringe.
I found it helpful that it was making the use cases concrete. So much of the debates around programming languages suffer from people not being concrete enough.
Prefer Alice and Bob?
I'm not fully sold on Rust's existing async/await implementation but how would this suggestion even be implemented? It's a massive breaking change on multiple fronts and I don't know how a new edition would paper over the impedance mismatch between the new system and async libraries written in the older edition. Feels like I need another year or two of using async rust in production to even begin to form an opinion on this.
How would this work on embedded? I haven't done embedded since long before I could use async/await or Rust but it sounds like implicit anything (except maybe autoref/deref) is a deal breaker.