Tokio 1.0 – async runtime for Rust(tokio.rs)
tokio.rs
Tokio 1.0 – async runtime for Rust
https://tokio.rs/blog/2020-12-tokio-1-0
418 comments
I had the same thing initially.
The upside of async over a simple event loop is, in my experience, when things become less simple, and you end up with hard-to-read little state machines all over the place.
With async, you can have your event loop, but the state machines are handled by the compiler. Code is like threaded code. That can be very convenient.
Threads, obviously, accomplish the same thing, and arguably more easily. But threads have a performance problem when they must interact heavily. Cross-thread communication is expensive. Single-threaded async task interaction is very cheap, comparatively (I use tokio only in single-threaded mode, as an event loop replacement; its multi-threaded scheduler performs terribly on serious I/O). I think the interaction problem is often more important than just the number of tasks.
As for the function coloring argument, I've started to see the async keyword as documentation. A non-async function is "regular logic", it must complete without blocking. An async function is a state machine; as such it must be part of a larger state machine (the event loop), and it can go down into smaller sub-state machines (i.e. call other async functions). If you make that distinction explicit, it all makes a lot of sense.
The upside of async over a simple event loop is, in my experience, when things become less simple, and you end up with hard-to-read little state machines all over the place.
With async, you can have your event loop, but the state machines are handled by the compiler. Code is like threaded code. That can be very convenient.
Threads, obviously, accomplish the same thing, and arguably more easily. But threads have a performance problem when they must interact heavily. Cross-thread communication is expensive. Single-threaded async task interaction is very cheap, comparatively (I use tokio only in single-threaded mode, as an event loop replacement; its multi-threaded scheduler performs terribly on serious I/O). I think the interaction problem is often more important than just the number of tasks.
As for the function coloring argument, I've started to see the async keyword as documentation. A non-async function is "regular logic", it must complete without blocking. An async function is a state machine; as such it must be part of a larger state machine (the event loop), and it can go down into smaller sub-state machines (i.e. call other async functions). If you make that distinction explicit, it all makes a lot of sense.
> Cross-thread communication is expensive. Single-threaded async task interaction is very cheap, comparatively
This all depends on how threads are implemented. If they're scheduled preemptively then communication can be expensive, relatively speaking, because of the need for locking and atomic operations. But you can also schedule cooperatively in user space, just as Tokio does when serially resuming async tasks; or as Java's Project Loom does for its new "lightweight" threads.
Note that unlike JavaScript, Tokio and Project Loom can also run different tasks on different, preemptively scheduled threads. And while I don't know that much Rust, I imagine you're going to need to use either unsafe or Rc or maybe even Arc if you intend to share data between different Tokio tasks--i.e. data that doesn't fit the normal caller/callee borrow semantics.
The other part of the problem is space requirements. Usually where you have preemptively scheduled threads the stack space for a thread is allocated lazily as a function is called and faults in pages via the OS' virtual memory system, much like single-thread, single-stack processes in a preemptive process OS. This means the minimum space allocation for a thread is at least 2x the page size (e.g. 4096 * 2). But many times a thread of execution only goes a couple of function calls deep, with minimal amounts of function-local (i.e. stack-allocated) data. If you have 1 thread per network connection, with hundreds of thousands or millions of connections that overhead could be significant.
But this, too, is a function of the implementation. Goroutines in Go use normal heap memory for stacks, and the compiler emits code to grow and move threads automatically. Rust proponents will tell you that async functions don't require any runtime cost because the stack requirements can be calculated statically. But to calculate this statically you can't support recursive functions. And if you can statically calculate your space requirements for the hidden async state object, you could also statically calculate the stack size for a thread just the same.
So really what it all comes down to isn't whether "async" is better or worse than "threads" along any of these dimensions. Abstractly, all threading implementations are async, and all async implementations effectively implement threads (i.e. a data structure that encapsulates a program counter, local automatic storage, etc). The real reason you choose one over the other is external factors. For Rust that dominate factor is interoperability with native C ABIs, particularly native stack disciplines. Because Rust can't implement much magic in the lower layers of the runtime environment while maintaining the degree of interoperability with C, C++, and other language libraries (via the C ABI) that they're committed to, they have no choice but to put most of the instrumentation into the language itself. And this necessitates the async contortions, independent of any other preferences. Contrast that with Go, where calling into C is slightly more costly because they preferred to push more of the async/thread abstraction beneath the language syntax.
But perhaps what this tells us is that we should think about revisiting native stack disciplines and thread scheduling semantics. IIRC, Linux will soon get scheduler activations (i.e. ability for userland to efficiently switch execution to another specified kernel-visible thread). That's a small step in the right direction, and if it catches on more operating systems will adopt this--after having ditched them 20 years ago, ironically, before async network I/O became popular and when 1:1 thread scheduling became the preferred kernel model).
This all depends on how threads are implemented. If they're scheduled preemptively then communication can be expensive, relatively speaking, because of the need for locking and atomic operations. But you can also schedule cooperatively in user space, just as Tokio does when serially resuming async tasks; or as Java's Project Loom does for its new "lightweight" threads.
Note that unlike JavaScript, Tokio and Project Loom can also run different tasks on different, preemptively scheduled threads. And while I don't know that much Rust, I imagine you're going to need to use either unsafe or Rc or maybe even Arc if you intend to share data between different Tokio tasks--i.e. data that doesn't fit the normal caller/callee borrow semantics.
The other part of the problem is space requirements. Usually where you have preemptively scheduled threads the stack space for a thread is allocated lazily as a function is called and faults in pages via the OS' virtual memory system, much like single-thread, single-stack processes in a preemptive process OS. This means the minimum space allocation for a thread is at least 2x the page size (e.g. 4096 * 2). But many times a thread of execution only goes a couple of function calls deep, with minimal amounts of function-local (i.e. stack-allocated) data. If you have 1 thread per network connection, with hundreds of thousands or millions of connections that overhead could be significant.
But this, too, is a function of the implementation. Goroutines in Go use normal heap memory for stacks, and the compiler emits code to grow and move threads automatically. Rust proponents will tell you that async functions don't require any runtime cost because the stack requirements can be calculated statically. But to calculate this statically you can't support recursive functions. And if you can statically calculate your space requirements for the hidden async state object, you could also statically calculate the stack size for a thread just the same.
So really what it all comes down to isn't whether "async" is better or worse than "threads" along any of these dimensions. Abstractly, all threading implementations are async, and all async implementations effectively implement threads (i.e. a data structure that encapsulates a program counter, local automatic storage, etc). The real reason you choose one over the other is external factors. For Rust that dominate factor is interoperability with native C ABIs, particularly native stack disciplines. Because Rust can't implement much magic in the lower layers of the runtime environment while maintaining the degree of interoperability with C, C++, and other language libraries (via the C ABI) that they're committed to, they have no choice but to put most of the instrumentation into the language itself. And this necessitates the async contortions, independent of any other preferences. Contrast that with Go, where calling into C is slightly more costly because they preferred to push more of the async/thread abstraction beneath the language syntax.
But perhaps what this tells us is that we should think about revisiting native stack disciplines and thread scheduling semantics. IIRC, Linux will soon get scheduler activations (i.e. ability for userland to efficiently switch execution to another specified kernel-visible thread). That's a small step in the right direction, and if it catches on more operating systems will adopt this--after having ditched them 20 years ago, ironically, before async network I/O became popular and when 1:1 thread scheduling became the preferred kernel model).
Bryan Cantrill's undergrad thesis would be cool to replicate today. 1:1 won because of performance pathologies in M:N.
I agree, it would be great if we could just write threads and not worry about performance. In the end, at least for me, async is a poor compromise between ergonomics and performance.
Unfortunately we're not there yet. Golang with GOMAXPROCS set to 1 comes close, but now I lose the ability to spawn real threads for expensive computation.
Unfortunately we're not there yet. Golang with GOMAXPROCS set to 1 comes close, but now I lose the ability to spawn real threads for expensive computation.
I've been amused to watch how Rust now does a simple blocking HTTP request. A few years ago, you used the "hyper" crate, which was a convenient wrapper around the "http" crate. Now, you're supposed to use "reqwest", which is a convenient wrapper around the "hyper" crate.
"Reqwest" uses the Tokio machinery, even for a blocking request. If you turn on "Trace" level logging, you can watch it start up a thread pool and go through a 35-step process, using all the async and futures machinery, to do one synchronous request. Log messages include "handshake complete, spawning background dispatcher task" and "signaled close for runtime thread (ThreadId(2))"
This seems excessive.
"Reqwest" uses the Tokio machinery, even for a blocking request. If you turn on "Trace" level logging, you can watch it start up a thread pool and go through a 35-step process, using all the async and futures machinery, to do one synchronous request. Log messages include "handshake complete, spawning background dispatcher task" and "signaled close for runtime thread (ThreadId(2))"
This seems excessive.
> A few years ago, you used the "hyper" crate, which was a convenient wrapper around the "http" crate. Now, you're supposed to use "reqwest", which is a convenient wrapper around the "hyper" crate.
That's not right. http is supposed to be a common library of types for HTTP servers and frameworks (although developers of some competing frameworks have rejected it). It was never an HTTP client like make it sound like, and it's actually newer than hyper.
As for reqwest vs. hyper, the former offers synchronous wrappers over the async ones, easier TLS support and other niceties (compression, proxy support, cookies, WASM). It's high-level and easier to use, somewhat like requests over urllib3 in the Python world.
That's not right. http is supposed to be a common library of types for HTTP servers and frameworks (although developers of some competing frameworks have rejected it). It was never an HTTP client like make it sound like, and it's actually newer than hyper.
As for reqwest vs. hyper, the former offers synchronous wrappers over the async ones, easier TLS support and other niceties (compression, proxy support, cookies, WASM). It's high-level and easier to use, somewhat like requests over urllib3 in the Python world.
But then that is nothing inherent to Rust, it's a choice made by a particular external library. That is probably the worst downside of async, it splits the ecosystem. Reqwest probably tries to offer both choices by hiding one behind the other (although what you describe sounds excessive - running a single task with tokio's single-threaded executor is actually quite lightweight).
That split between "sync" and "async" was always there, for networking libraries. E.g. on C/C++ you find libs that run on libevent, or Boost.Asio, or other varieties, and they don't mix, so you end up spawning a seperate thread - exactly the same thing.
And this is, IMHO, how we should see Tokio + async: as a more ergonomic libevent.
That split between "sync" and "async" was always there, for networking libraries. E.g. on C/C++ you find libs that run on libevent, or Boost.Asio, or other varieties, and they don't mix, so you end up spawning a seperate thread - exactly the same thing.
And this is, IMHO, how we should see Tokio + async: as a more ergonomic libevent.
> it's a choice made by a particular external library
Yeah I think it points to a culture problem. In some ways because dependency management is so easy with Cargo, I think it creates the temptation to just throw in some dependencies to make something work without truly understanding the overall complexity of what you're creating. Something very similar happens in the NodeJS world.
> it splits the ecosystem
This is something I've really noticed with Rust: it almost seems like there are really two things: Rust, and Rust+Tokio. I'm a bit ambivalent about Tokio being baked into so many libraries: I think it's great to have as an option, but once I decide to use one library built around Tokio, it imposes a lot of constraints about how the flow of control is going to work in my program.
Yeah I think it points to a culture problem. In some ways because dependency management is so easy with Cargo, I think it creates the temptation to just throw in some dependencies to make something work without truly understanding the overall complexity of what you're creating. Something very similar happens in the NodeJS world.
> it splits the ecosystem
This is something I've really noticed with Rust: it almost seems like there are really two things: Rust, and Rust+Tokio. I'm a bit ambivalent about Tokio being baked into so many libraries: I think it's great to have as an option, but once I decide to use one library built around Tokio, it imposes a lot of constraints about how the flow of control is going to work in my program.
I think rust is kind of a perfect language for being profligate with dependencies, because the safety guarantees, typing, etc make it very hard to misuse a library, and relatively easy to design a library that is hard to misuse.
A lot of what is not enjoyable about rust as a user is really nice when it's being imposed on people who are not you, whose work you're interfacing with.
A lot of what is not enjoyable about rust as a user is really nice when it's being imposed on people who are not you, whose work you're interfacing with.
Just because a library is safe does not make it good. To the point of the previous poster, you might for instance have an http library which does a lot of unnecessary async work behind the scenes to do a simple synchronous request.
If we all have the attitude "it's good/fast because it's rust" this is not going to lead to a lot of cruft making its way into the ecosystem.
If we all have the attitude "it's good/fast because it's rust" this is not going to lead to a lot of cruft making its way into the ecosystem.
I think if a dependency is a perfectly sealed abstraction, where a complex function is reduced to a simple one with no leakage, then there's no reason not to use it.
Obviously, in the real world, this basically never happens. Performance is one thing that's basically always going to 'leak', so you still get people rewriting stuff in assembly, or making custom asics, because the abstractions that higher level languages offer are not perfect.
In a strongly typed language, with strong safety guarantees, I think there are less ways an abstraction can leak (for instance, by corrupting memory or whatever), so there's a correspondingly lower cost to pulling in a dependency than there would be if you were working in an unsafe language, or a dynamic one.
I also think if performance is the only way in which your dependency leaks implementation details, then it still makes sense to pull in a dependency first, profile, then swap out if necessary.
Obviously, in the real world, this basically never happens. Performance is one thing that's basically always going to 'leak', so you still get people rewriting stuff in assembly, or making custom asics, because the abstractions that higher level languages offer are not perfect.
In a strongly typed language, with strong safety guarantees, I think there are less ways an abstraction can leak (for instance, by corrupting memory or whatever), so there's a correspondingly lower cost to pulling in a dependency than there would be if you were working in an unsafe language, or a dynamic one.
I also think if performance is the only way in which your dependency leaks implementation details, then it still makes sense to pull in a dependency first, profile, then swap out if necessary.
Agreed with your point about Cargo. It's a double edged sword.
We absolutely have an NPM/leftpad culture in Rust.
Is that better or worse than C and C++ where dependencies are so painful that you end up reinventing the wheel most of the time? I honestly don't know.
We absolutely have an NPM/leftpad culture in Rust.
Is that better or worse than C and C++ where dependencies are so painful that you end up reinventing the wheel most of the time? I honestly don't know.
Yes I think it's a really difficult problem to be honest. I am definitely grateful for how easy it is to make rust projects reproducible, but it's not without disadvantages.
Dependencies are relatively easy, actually. Just most don't bother learning how to do it, and do it PHP style with header includes.
On UNIX systems just using pkg-config and similar tools, or just adopt either conan or vcpkg, which contrary to cargo also support binary libraries out of the box.
Plus vendoring C and C++ libraries is not a dark science, only known by old druids.
On UNIX systems just using pkg-config and similar tools, or just adopt either conan or vcpkg, which contrary to cargo also support binary libraries out of the box.
Plus vendoring C and C++ libraries is not a dark science, only known by old druids.
Maybe I've just missed it, but I have found pkg-config difficult to use and poorly documented. It's fine if you are installing things with a package manager, but I found it took some trail-and-error to figure out how to do this for my own lbraries, or for things built manually from source.
Also with c/c++ style system dependencies, I feel like there are a lot of issues with things like version conflicts which are solved much more simply by a package manager like cargo.
I agree that it's functional, but to say relatively easy I think is a bit of a stretch.
Also with c/c++ style system dependencies, I feel like there are a lot of issues with things like version conflicts which are solved much more simply by a package manager like cargo.
I agree that it's functional, but to say relatively easy I think is a bit of a stretch.
Cargo also has issues when two crates have incompatible dependencies, or at very least you end with the same crate being compiled a couple of times, as the hashes don't match up.
Usually when compiling from source many libraries provide pkg-config configuration files on "make install".
Usually when compiling from source many libraries provide pkg-config configuration files on "make install".
Yes. Rust encourages version pinning. You go to "crates.io", and it gives you a specific version number to put in your "cargo.toml" file. Now you're nailed to that version for your program or crate. Crates have their own "cargo.toml" file, with their own versions, and it's quite possible to pull in multiple versions.
Right now, I'm using the latest version of "reqwest" known to "crates.io." It's pulling in Tokio v0.2.23, not the new tokio v1.0.0. No surprise there, the new version only came out yesterday. So we'll see how the new version works at some time in the future.
It's good to get to version 1. The semantic versioning rules allow breaking changes without changing the first digit when the first digit is 0. Typical complaint on forums: "bignum happens to use rand internally, and it happens to only use version 0.5.0, with restrictions against using a higher version due to breaking changes." Rust still has many low-level crates at version 0.x.x, from "bytes" to "uuid". Reaching 1 indicates greater stability.
Right now, I'm using the latest version of "reqwest" known to "crates.io." It's pulling in Tokio v0.2.23, not the new tokio v1.0.0. No surprise there, the new version only came out yesterday. So we'll see how the new version works at some time in the future.
It's good to get to version 1. The semantic versioning rules allow breaking changes without changing the first digit when the first digit is 0. Typical complaint on forums: "bignum happens to use rand internally, and it happens to only use version 0.5.0, with restrictions against using a higher version due to breaking changes." Rust still has many low-level crates at version 0.x.x, from "bytes" to "uuid". Reaching 1 indicates greater stability.
> Yes. Rust encourages version pinning.
Rust makes version pinning feasible, e.g. by allowing multiple versions of the same package in a build (not all module systems have this feature!) but doesn't encourage it. You've identified a problem with using 0.x.y-versioned packages as dependencies (which means de-facto opting out of semver), but that's not a problem with Rust specifically; it could occur in any language.
Rust makes version pinning feasible, e.g. by allowing multiple versions of the same package in a build (not all module systems have this feature!) but doesn't encourage it. You've identified a problem with using 0.x.y-versioned packages as dependencies (which means de-facto opting out of semver), but that's not a problem with Rust specifically; it could occur in any language.
To be clear, the default semantics are ^, not =. So even if you put 1.2.3 in your Cargo.toml, you may get 1.3.0, you just won’t get 2.0.0.
But with Cargo it's scoped to the crate you're compiling right? So it only matters if there's a collision in the dependencies of a given project.
Isn't it the case with pkg-config that everything is stored in a central location?
In any case, I think you can't seriously argue that the c/c++ dependency management solution is anywhere close to running `cargo build`/`cargo run` in terms of simplicity.
Isn't it the case with pkg-config that everything is stored in a central location?
In any case, I think you can't seriously argue that the c/c++ dependency management solution is anywhere close to running `cargo build`/`cargo run` in terms of simplicity.
Yes I surely can, because Conan and vcpkg do exist, and contain all major well known libraries in the C and C++ communities.
Specially the C++ community has seen lack of something like cargo as weaknesses and moved to sort it out.
Specially the C++ community has seen lack of something like cargo as weaknesses and moved to sort it out.
Yes I had the same reaction - it seems unreasonably complicated to do a simple, blocking HTTP request in Rust.
Give ureq a try: https://github.com/algesten/ureq
It is blocking only and has a pretty simple API.
> the state machines are handled by the compiler
I think this is the aspect of modern approaches async which I am most ambivalent about. One of the things I have learned about programming over the past 10 years is that I, as the programmer, really want to own the flow of control of my program. Once I hand that over to some other system, usually in the name of convenience, I will start to have issues which are difficult to understand and solve.
For instance, a while ago, I was working on a project which was using making heavy use of RXJava. One of my colleagues pushed a commit, and suddenly CI was failing on a unit test which passed when run locally. It turns out the problem was because the CI server was running tests with a different scheduler, so GC was happening at a different time, creating an NPE which didn't happen locally. Imo when you start to see unit tests behaving inconsistently based on a factor which is completely outside the actual code you yourself have written, this is a sign you are going down the wrong path.
I also wonder how much a lot of the buzz around async actually has to do with the fact that it's a bit brain-bending to wrap your head around at first, as compared to its actual utility. I think for a lot of us as programmers, we enjoy that feeling of understanding something difficult - like when you really get recursion for the first time - and we're attracted to the idea of really fundamentally new concepts being introduced to programming.
But it seems to me that async is one of those concepts which brings us farther away from actually programming the hardware, and puts a kind of middle-man between us and the CPU, and I'm not sure that has ever been a good thing.
I think this is the aspect of modern approaches async which I am most ambivalent about. One of the things I have learned about programming over the past 10 years is that I, as the programmer, really want to own the flow of control of my program. Once I hand that over to some other system, usually in the name of convenience, I will start to have issues which are difficult to understand and solve.
For instance, a while ago, I was working on a project which was using making heavy use of RXJava. One of my colleagues pushed a commit, and suddenly CI was failing on a unit test which passed when run locally. It turns out the problem was because the CI server was running tests with a different scheduler, so GC was happening at a different time, creating an NPE which didn't happen locally. Imo when you start to see unit tests behaving inconsistently based on a factor which is completely outside the actual code you yourself have written, this is a sign you are going down the wrong path.
I also wonder how much a lot of the buzz around async actually has to do with the fact that it's a bit brain-bending to wrap your head around at first, as compared to its actual utility. I think for a lot of us as programmers, we enjoy that feeling of understanding something difficult - like when you really get recursion for the first time - and we're attracted to the idea of really fundamentally new concepts being introduced to programming.
But it seems to me that async is one of those concepts which brings us farther away from actually programming the hardware, and puts a kind of middle-man between us and the CPU, and I'm not sure that has ever been a good thing.
I think the pain you went through with Java is not quite compareable, as the described havoc (and I do really feel your pain here) would not happen like that in Rust, for the reason that you would have to model these things more explicitly. In detail, it sounds like a cascade of implicit null-ness (aka the billion dollar mistake) and weak references. In Rust null (called None) is explicit through the Option type and the Weak type returns exactly that.
More to your point, as I think you were using this as an analogy for the perils of giving up control: Rust's explicitness should entail all the semantics of your program, and hence async Rust makes you model out all the potentially-racy async interactions with Arc, Mutex, etc. The same middle-man (the borrow checker) who watches over your regular ol' sync code's memory-correctness, now expects extra constraints to be upheld for values passing through async-boundaries. And for me the whole point of Rust is that this correctness proof will do a better job than any person could, for any moderately sized program. So this is middle man you'd want between you and the CPU.
That said, your async runtime could definitely do shenanigans that screw up your nicely modeled program, but that would be a bug in that specific runtime. I haven't deeply read Tokio's source and even if I did, making a qualified judgment about it is beyond me.
More to your point, as I think you were using this as an analogy for the perils of giving up control: Rust's explicitness should entail all the semantics of your program, and hence async Rust makes you model out all the potentially-racy async interactions with Arc, Mutex, etc. The same middle-man (the borrow checker) who watches over your regular ol' sync code's memory-correctness, now expects extra constraints to be upheld for values passing through async-boundaries. And for me the whole point of Rust is that this correctness proof will do a better job than any person could, for any moderately sized program. So this is middle man you'd want between you and the CPU.
That said, your async runtime could definitely do shenanigans that screw up your nicely modeled program, but that would be a bug in that specific runtime. I haven't deeply read Tokio's source and even if I did, making a qualified judgment about it is beyond me.
So I would not say that the borrow checker is a middle-man. The borrow checker does impose constraints on programming, but at the end of the day it's only relevant at compile time, and you still end up with code which maps in a predictable way to the hardware. If you hand me a synchronous rust code, I can imagine, at least in some approximate way, which set of assembly code would be equivalent.
An async runtime is a totally different animal. If you hand me a block of async rust code and ask me how it will execute, the answer I have to give is "it depends on the runtime". This is the disconnect I am talking about.
An async runtime is a totally different animal. If you hand me a block of async rust code and ask me how it will execute, the answer I have to give is "it depends on the runtime". This is the disconnect I am talking about.
Fair distinction, thanks for making it! My async use cases so far have been in a realm where everything modeled was everything I cared about, and those specifics of the runtime didn't become relevant. I wanted to refer to your example because I do see how that is a thing that needs to be explicitly modeled.
I'd be curious to hear about examples where the runtime did or would surprise you!
I'd be curious to hear about examples where the runtime did or would surprise you!
I don't have a ton of experience writing async rust programs, but I can imagine some types of problems which might come up:
- So what if I am implementing a high-throughput, performance critical system which makes heavy use of async, and under certain circumstances the runtime I'm using falls off a performance cliff. It's going to be difficult to diagnose and solve this problem, because the critical path of my program actually winds through a library which is essentially a black box to me.
- What if I have two dependencies, and each one internally depends on a separate async runtime. And what if each of these runtimes is designed with the assumption that it is the main owner of system resources, like threads. There may be conflicts which are very difficult to understand but have effects on the performance of my program.
I think fundamentally, an issue with this type of "middleware" is that by its nature, an async runtime, like Tokio for example, has to be implemented with a lot of assumptions about how "the generic program" should optimally handle async. It may work great for the vast majority of use-cases, but fundamentally whenever you design a super general, abstract system like this you have to make tradeoffs.
In some ways Rust has taken probably the best possible approach to this, by making it modular and allowing you to bring your own runtime, but I think in practice, if the use of async continues to become pervasive in Rust and certain libraries get locked into certain ecosystems, it will not be so easy in practice to take advantage of that modularity.
- So what if I am implementing a high-throughput, performance critical system which makes heavy use of async, and under certain circumstances the runtime I'm using falls off a performance cliff. It's going to be difficult to diagnose and solve this problem, because the critical path of my program actually winds through a library which is essentially a black box to me.
- What if I have two dependencies, and each one internally depends on a separate async runtime. And what if each of these runtimes is designed with the assumption that it is the main owner of system resources, like threads. There may be conflicts which are very difficult to understand but have effects on the performance of my program.
I think fundamentally, an issue with this type of "middleware" is that by its nature, an async runtime, like Tokio for example, has to be implemented with a lot of assumptions about how "the generic program" should optimally handle async. It may work great for the vast majority of use-cases, but fundamentally whenever you design a super general, abstract system like this you have to make tradeoffs.
In some ways Rust has taken probably the best possible approach to this, by making it modular and allowing you to bring your own runtime, but I think in practice, if the use of async continues to become pervasive in Rust and certain libraries get locked into certain ecosystems, it will not be so easy in practice to take advantage of that modularity.
>Threads, obviously, accomplish the same thing, and arguably more easily. But threads have a performance problem when they must interact heavily. Cross-thread communication is expensive.
I didn't know that cross "async" communication was cheaper, that does seem like a good selling point, but what exactly makes it cheaper? After all threads share the same address space, so you can just pass pointers around the same way you would within the same thread. I expected the overhead to be roughly similar.
Things can get cache-expensive if the code is running on different cores, but then again using all the hardware resources available is generally something you want to do if you care about performance.
I didn't know that cross "async" communication was cheaper, that does seem like a good selling point, but what exactly makes it cheaper? After all threads share the same address space, so you can just pass pointers around the same way you would within the same thread. I expected the overhead to be roughly similar.
Things can get cache-expensive if the code is running on different cores, but then again using all the hardware resources available is generally something you want to do if you care about performance.
I think the point is that you can omit atomics in async case, if you are always using async in single thread mode.
Not just atomics, you'd probably need mutexes or rwlocks in a lot of scenarios, and these can become a bottleneck quickly if you don't think it through. Async has the benefit of context switches (handing off execution) being explicit, so you're fine as long as you don't leave any half-updated state before you do an async function call.
Rust also lets you avoid atomics by using structs that implement Send. Having an async function return such a struct is a lot easier to map mentally (for me, at least)
It's faster than multiple threads even on a single core. There are syscalls involved to wait, and to wake up. That doesn't matter for I/O, since syscalls are involved anyway, but it does for mutexes and condition variables. With async, handing off control to one or more other tasks is cheap (tokio around 100 ns), for threads it's more expensive (2-3 us).
And of course with threads it's harder to actually run single-core, you need to dedicated a specific core which brings operational complexity.
And of course with threads it's harder to actually run single-core, you need to dedicated a specific core which brings operational complexity.
> A non-async function is "regular logic", it must complete without blocking.
Maybe one can enforce this convention in the particular project, but there's no ecosystem-wide consensus on this, and in fact I don't want this to be consensus. I write blocking non-async functions every day. Why am I wrong to do so?
Maybe one can enforce this convention in the particular project, but there's no ecosystem-wide consensus on this, and in fact I don't want this to be consensus. I write blocking non-async functions every day. Why am I wrong to do so?
There is consensus in some ecosystems. Javascript absolutely maintains that invariant. There are (almost) no blocking functions in the javascript / node standard libraries and we work hard to keep it that way. Go maintains similar discipline at the OS syscall level.
I feel like the "what color is your function" thing is incomplete. There are arguably 3 types of functions:
- Functions which do all their work synchronously and return without blocking
- Async functions which contain an internal state machine
- Functions which block on expensive IO or long computations
Mixing blocking functions and async functions in the same kernel thread leads to various performance disasters. Javascript is so meticulous about not having blocking IO in part because its basically impossible to tell from a function's signature whether it will block the thread. Lua has this problem - callback oriented lua feels like a natural fit for the language, but lots of 3rd party libraries are packed with blocking calls. Writing asyncronous lua feels like fighting a river. You have to constantly guard against calling blocking code, and most API docs won't tell you where they block.
I feel like the "what color is your function" thing is incomplete. There are arguably 3 types of functions:
- Functions which do all their work synchronously and return without blocking
- Async functions which contain an internal state machine
- Functions which block on expensive IO or long computations
Mixing blocking functions and async functions in the same kernel thread leads to various performance disasters. Javascript is so meticulous about not having blocking IO in part because its basically impossible to tell from a function's signature whether it will block the thread. Lua has this problem - callback oriented lua feels like a natural fit for the language, but lots of 3rd party libraries are packed with blocking calls. Writing asyncronous lua feels like fighting a river. You have to constantly guard against calling blocking code, and most API docs won't tell you where they block.
I have seen fs.*Sync functions being called deep in async-land more than once or twice in Node.
Those methods were added super early (node 0.2 or something) and can’t be removed because of backwards compatibility. Many of the core node team think they should never have been added - for that exact reason.
We are talking about Rust. There is no consensus in Rust.
I think the OP was using examples from a different ecosystems to demonstrate how such a consensus could be reached, if people wished for it.
> I write blocking non-async functions every day. Why am I wrong to do so?
This (in my opinion) not "wrong". At least not in general. There are instances where it might be more or less probematic though.
It's probably problematic if you already have a bunch of async code in the codebase, because other readers of the code are likley to expect blocking functions to be async.
It's maybe problematic for high performance or high scale code. Synchronous blocking functions are more likely to hit OS limits (file handles, network sockets, etc) than async code. If the code is obviously written from the ground up for high scale/performance, this is less likely to be a problem, but if it's proof of concept code that's likely to get pushed into production by over eager PMs as soon as it passes tests, it'll be worse.
It's possibly problematic if done in a language/frameworks where async is idiomatic - it'd be wrong to write using blocking functions in a nodejs codebase, because you'd be breaking other people expectations when reading/understanding the code.
Maybe a useful rule of thumb might be "if more than some number (perhaps 30 or 50%) of other people working on the code might think 'hang on, I'm gonna refactor this to use async', then maybe using a non-asyn blocking function was the wrong choice. That means it's _never_ the wrong choice for one person codebases. It means it's probably almost always a wrong choice in a javascript codebase. For everything else? "It depends". I'd always choose to go with "the principle of least astonishment" - do whatever other people who might be affected would expect you to wherever possible.
This (in my opinion) not "wrong". At least not in general. There are instances where it might be more or less probematic though.
It's probably problematic if you already have a bunch of async code in the codebase, because other readers of the code are likley to expect blocking functions to be async.
It's maybe problematic for high performance or high scale code. Synchronous blocking functions are more likely to hit OS limits (file handles, network sockets, etc) than async code. If the code is obviously written from the ground up for high scale/performance, this is less likely to be a problem, but if it's proof of concept code that's likely to get pushed into production by over eager PMs as soon as it passes tests, it'll be worse.
It's possibly problematic if done in a language/frameworks where async is idiomatic - it'd be wrong to write using blocking functions in a nodejs codebase, because you'd be breaking other people expectations when reading/understanding the code.
Maybe a useful rule of thumb might be "if more than some number (perhaps 30 or 50%) of other people working on the code might think 'hang on, I'm gonna refactor this to use async', then maybe using a non-asyn blocking function was the wrong choice. That means it's _never_ the wrong choice for one person codebases. It means it's probably almost always a wrong choice in a javascript codebase. For everything else? "It depends". I'd always choose to go with "the principle of least astonishment" - do whatever other people who might be affected would expect you to wherever possible.
I believe the idea is that within a project that uses async functionality, you should only use non-async functions when the logic does not call blocking functionality. If you are mixing async functionality and synchronous functions with/blocking I would consider the latter a defect unless it is handled properly within an asynchronous context.
I really don't understand the logic of this on a multi-threaded system. The vast majority of functions I write are best executed synchronously, the remainder is usually composed of logic wrapping heavy computations which can be executed in parallel or logic surrounding I/O which can be executed concurrently.
An async system which poisons the rest of my code to force async usage doesn't seem like it will scale to code leveraging multiple libraries and will likely fail at the first lib where the author decided not to bother. The beauty of coroutines in go and Java(soon) is that the async functionality remains local to the code that can make use of it - everyone else just sees a thread-like API.
An async system which poisons the rest of my code to force async usage doesn't seem like it will scale to code leveraging multiple libraries and will likely fail at the first lib where the author decided not to bother. The beauty of coroutines in go and Java(soon) is that the async functionality remains local to the code that can make use of it - everyone else just sees a thread-like API.
> An async system which poisons the rest of my code to force async usage doesn't seem like it will scale to code leveraging multiple libraries and will likely fail at the first lib where the author decided not to bother.
You are quite correct. This happens.
You are quite correct. This happens.
> logic surrounding I/O which can be executed concurrently.
That's the bit you'd use an async runtime for, in a (mostly) dedicated thread.
The heavy computations would be in other threads that aren't doing so.
That's the bit you'd use an async runtime for, in a (mostly) dedicated thread.
The heavy computations would be in other threads that aren't doing so.
I think you're right on one level: if your codebase is pervasively, implicitly multithreaded, then there's little value in explicitly marking yield points. But if your codebase is pervasively, implicitly multithreaded, then it's impossible to maintain without locking everywhere (and difficult even then), and combining async with (blocking) locking does not work well.
In a codebase where concurrency is carefully controlled and constrained, an async system that gives you visibility into where the yield points are is very valuable: https://glyph.twistedmatrix.com/2014/02/unyielding.html .
In a codebase where concurrency is carefully controlled and constrained, an async system that gives you visibility into where the yield points are is very valuable: https://glyph.twistedmatrix.com/2014/02/unyielding.html .
> A non-async function is "regular logic", it must complete without blocking.
What does 'blocking' mean? I would expect the definition of synchronous to be the exact opposite; i.e., a synchronous function must block the caller until the function has finished executing. For that matter, what is "regular logic"? The name implies there is some sort of "irregular logic" to contrast it with.
I get the feeling that the writing may be unclear because the concepts are themselves not well-defined.
What does 'blocking' mean? I would expect the definition of synchronous to be the exact opposite; i.e., a synchronous function must block the caller until the function has finished executing. For that matter, what is "regular logic"? The name implies there is some sort of "irregular logic" to contrast it with.
I get the feeling that the writing may be unclear because the concepts are themselves not well-defined.
Sorry for the wording, the term blocking is common in network programming.
With blocking I mean waiting. For I/O to complete, for time to pass, or for another task to complete something. In event-based programming, functions must not block. Async functions may seem to block, but they don't rely because a state machine is involved.
With blocking I mean waiting. For I/O to complete, for time to pass, or for another task to complete something. In event-based programming, functions must not block. Async functions may seem to block, but they don't rely because a state machine is involved.
That does answer my question, but I don't really understand why the distinction is made. To the caller, a function that spends time waiting and a function that spends the same time calculating both look the same, don't they?
The difference is that the runtime can schedule something else if the blocking is async. It looks the same as sync blocking to the caller, but not the scheduler. The point of async is you can write code that looks synchronous but is actually participating in cooperative multitasking.
I was expecting there to be more to it than that, but I understand their point now.
My reading of that was 'you must write your non-async "regular logic" functions so that they cannot block.'
[deleted]
Async/await matters in Rust specifically because the borrow checker makes writing code without it difficult, inefficient, and unergonomic: http://aturon.github.io/tech/2018/04/24/async-borrowing/ (note that some of the details have changed here, but the thrust of it is very much the same.)
That said, if you can get your job done without this stuff, that's fine too, but the reasons it was pursued specifically involve the above.
That said, if you can get your job done without this stuff, that's fine too, but the reasons it was pursued specifically involve the above.
Ah that's a good point, it's true that the borrow checker can sometimes get in the way for event-driven architectures and async IO borrows.
That being said I can't shake the feeling that going for something like Tokio in such a case is a bit like healing a paper cut by amputating the arm. Sure, technically you don't have the original problem anymore...
That being said I can't shake the feeling that going for something like Tokio in such a case is a bit like healing a paper cut by amputating the arm. Sure, technically you don't have the original problem anymore...
Can you elaborate a bit what it is that you find difficult or undesirable about Tokio? Or async/await + some runtime in general?
So, I can relate to not wanting to pull in the dependency. But otherwise it seems pretty straightforward to me. You just macro-decorate the main function, sprinkle some async/await around, maybe add a join or a mutex somewhere, and then pretty much forget all about event loops, messaging, threads and whatnot. I feel like I must be missing something important here.
So, I can relate to not wanting to pull in the dependency. But otherwise it seems pretty straightforward to me. You just macro-decorate the main function, sprinkle some async/await around, maybe add a join or a mutex somewhere, and then pretty much forget all about event loops, messaging, threads and whatnot. I feel like I must be missing something important here.
> just macro-decorate the main function, sprinkle some async/await around, maybe add a join or a mutex somewhere, and then pretty much forget all about event loops, messaging, threads and whatnot
that is ... not how I have experienced it. I work on building highly concurrent systems every day but async drives me insane. to me the fundamental issue is that although the code now reads linearly, it no longer executes linearly (or reasonably close to linearly), which is 1000x more confusing.
the other thing, when I'm using rust to build something high performance, part of the reason is it provides greater control. I just can't square that with macro-decorating my main function, and handing over the core control-flow to someone else's runtime.
that is ... not how I have experienced it. I work on building highly concurrent systems every day but async drives me insane. to me the fundamental issue is that although the code now reads linearly, it no longer executes linearly (or reasonably close to linearly), which is 1000x more confusing.
the other thing, when I'm using rust to build something high performance, part of the reason is it provides greater control. I just can't square that with macro-decorating my main function, and handing over the core control-flow to someone else's runtime.
Thank you for the perspective! This actually explains it very well. So, in a nutshell, it's the difference between apparent and actual complexity, as well as a trust issue.
With async/await the apparent complexity gets reduced at the cost of vastly increased actual complexity. E.g., now instead of everything being your code that you can look at and reason about, all your concurrent workloads disappear in this void that promises to do the right thing with them. If it works the way you intended, great. If it doesn't, the rabbit hole can now be really deep.
And then, it's also a trust issue. Now you have to trust other people to have done a good job.
Ok, yes, this makes sense.
With async/await the apparent complexity gets reduced at the cost of vastly increased actual complexity. E.g., now instead of everything being your code that you can look at and reason about, all your concurrent workloads disappear in this void that promises to do the right thing with them. If it works the way you intended, great. If it doesn't, the rabbit hole can now be really deep.
And then, it's also a trust issue. Now you have to trust other people to have done a good job.
Ok, yes, this makes sense.
You answered jgilias better than I could, I feel exactly the same way. Async is deceptively simple in my opinion, because while it looks arguably even simpler than an explicit state machine, it makes your program flow nonlinear and I find that a lot harder to work with. With blocking code I can mentally step through the code follow causes and consequences easily, with async I feel like I'm watching a scifi movie involving time travel and parallel universes.
And the loss of control is also an issue for me. I write code for memory-constrained environments, with blocking code and OS threads I can usually bound my memory consumption fairly easily. If I surrender the control to a scheduler runtime I feel like it becomes a lot harder, although here I'm willing to concede that it might have more to do with my lack of experience with Tokio than an objective issue.
And the loss of control is also an issue for me. I write code for memory-constrained environments, with blocking code and OS threads I can usually bound my memory consumption fairly easily. If I surrender the control to a scheduler runtime I feel like it becomes a lot harder, although here I'm willing to concede that it might have more to do with my lack of experience with Tokio than an objective issue.
agree 100%. it honestly kind of baffles me, "async" is like the programming community's white whale, and all of us get to come along for the chase. meanwhile, I long ago grew accustomed to the paradigm of an "event loop" in my programs. after a certain point it becomes very natural. on the subject of memory, recently there was an issue where the async dyn futures were blowing up stacks because a resolved future was > 2mb - what!? I mean, look at the signatures in the aturon article - we are going from this
I recently tried writing a small program that would manually poll a future to get a feel for it - utter disaster. conflicting versions of tokio, compiles but crashes because something is called outside of the tokio runtime context, etc. all the examples have #[tokio::main]-decorated main - it's like, I'm not giving you my #$(&#@(&ing main function! the programs I write have tons of stuff going on! I can't just give some library my entire control flow!
sorry for the rant! felt good to write it though.
fn read(&mut self, buf: &mut [u8]) -> Result<usize, io::Error>
fn read<T: AsMut<[u8]>>(self, buf: T) ->
impl Future<Item = (Self, T, usize), Error = (Self, T, io::Error)>
I recently tried writing a small program that would manually poll a future to get a feel for it - utter disaster. conflicting versions of tokio, compiles but crashes because something is called outside of the tokio runtime context, etc. all the examples have #[tokio::main]-decorated main - it's like, I'm not giving you my #$(&#@(&ing main function! the programs I write have tons of stuff going on! I can't just give some library my entire control flow!
sorry for the rant! felt good to write it though.
I also don't get the async hype.
Maybe it is undesirable because that often times plain mono-thread synchronous is fast enough, easier to read, easier to debug and safe to handle to a junior ? Not everybody in a team has the same level of expertise.
And Rust is not an interpreted language. IMHO, interpreted languages should just drop to a compiled one to keep it KISS. Instead of going the async road, just to discover in production, it is unstable because back-pressure was not taken into account. And, in Rust, it is probably not worth the effort and ultimately bloat most of the time.
Maybe it is undesirable because that often times plain mono-thread synchronous is fast enough, easier to read, easier to debug and safe to handle to a junior ? Not everybody in a team has the same level of expertise.
And Rust is not an interpreted language. IMHO, interpreted languages should just drop to a compiled one to keep it KISS. Instead of going the async road, just to discover in production, it is unstable because back-pressure was not taken into account. And, in Rust, it is probably not worth the effort and ultimately bloat most of the time.
> Maybe it is undesirable because that often times plain mono-thread synchronous is fast enough, easier to read, easier to debug and safe to handle to a junior ? Not everybody in a team has the same level of expertise.
Shared-memory concurrency is pretty much always buggy, IME, even if your team thinks they're experts.
> And Rust is not an interpreted language. IMHO, interpreted languages should just drop to a compiled one to keep it KISS. Instead of going the async road, just to discover in production, it is unstable because back-pressure was not taken into account.
WTF? Switching to a compiled language doesn't magically make your threads nonblocking. Maybe you can serve 10x more users with a compiled language, but if we're talking about slow network requests then async can make your throughput thousands of times higher.
Shared-memory concurrency is pretty much always buggy, IME, even if your team thinks they're experts.
> And Rust is not an interpreted language. IMHO, interpreted languages should just drop to a compiled one to keep it KISS. Instead of going the async road, just to discover in production, it is unstable because back-pressure was not taken into account.
WTF? Switching to a compiled language doesn't magically make your threads nonblocking. Maybe you can serve 10x more users with a compiled language, but if we're talking about slow network requests then async can make your throughput thousands of times higher.
Why WTF ?
You are reading something I did not write. I am not obsessed with the nonblocking mantra. Blocking is not inherently bad. Multi-processus is also a perfectly valid concurrency model.
Nor am I obsessing about being ultra performant to achieve the revered C10K when I don't need to or can get around it. That was my point.
Not everybody is Facebook or Netflix. For the vast majority of small and medium enterprises, it faster, simpler, safer (and possibly cheaper) to quickly develop a blocking program without thread and spawn multiple processes.
You are reading something I did not write. I am not obsessed with the nonblocking mantra. Blocking is not inherently bad. Multi-processus is also a perfectly valid concurrency model.
Nor am I obsessing about being ultra performant to achieve the revered C10K when I don't need to or can get around it. That was my point.
Not everybody is Facebook or Netflix. For the vast majority of small and medium enterprises, it faster, simpler, safer (and possibly cheaper) to quickly develop a blocking program without thread and spawn multiple processes.
> Why WTF ?
Because WTF does using a compiled language have to do with anything?
> Nor am I obsessing about being ultra performant to achieve the revered C10K when I don't need to or can get around it. That was my point.
Then what is it that you imagine using a compiled language would help with?
Because WTF does using a compiled language have to do with anything?
> Nor am I obsessing about being ultra performant to achieve the revered C10K when I don't need to or can get around it. That was my point.
Then what is it that you imagine using a compiled language would help with?
[deleted]
> Shared-memory concurrency is pretty much always buggy, IME, even if your team thinks they're experts.
Writing bug-free shared memory concurrency programs with Go is trivial. Your opinion is based on outdated information.
Writing bug-free shared memory concurrency programs with Go is trivial. Your opinion is based on outdated information.
Go's whole approach to concurrency is about avoiding sharing memory; channels suspend in much the same way as async/await yield points.
I like go, but i don’t understand your point. Go seems to like passing pointers over channels, which is pretty far from avoiding shared memory. Unless you start writing code that looks like actor based concurrency, with channels used to pass messages acrross actors. But this isn’t what i’d call idiomatic go.
> Unless you start writing code that looks like actor based concurrency, with channels used to pass messages acrross actors. But this isn’t what i’d call idiomatic go.
Isn't that exactly what Go people push? "Don't communicate by sharing memory; share memory by communicating" and all that. If you start pushing pointers to shared memory around then I'd expect all of the problems of traditional multithreading to reappear.
Isn't that exactly what Go people push? "Don't communicate by sharing memory; share memory by communicating" and all that. If you start pushing pointers to shared memory around then I'd expect all of the problems of traditional multithreading to reappear.
Passing pointers to shared memory is highly unsafe in Go. While the Rust borrow checker will prevent all data races, there's nothing like that wrt. Go.
> Passing pointers to shared memory is highly unsafe in Go. While the Rust borrow checker will prevent all data races, there's nothing like that wrt. Go
Passing pointers to shared memory is the foundation of a huge number of idiomatic, performant, and productive design patterns and architectures. There exist a number of conventions and tools, like the race detector, which reduce the risks of data races to entirely reasonable levels.
Passing pointers to shared memory is the foundation of a huge number of idiomatic, performant, and productive design patterns and architectures. There exist a number of conventions and tools, like the race detector, which reduce the risks of data races to entirely reasonable levels.
Passing pointers over channels doesn't necessarily mean you're sharing memory, you could be passing ownership. Pointer or value are both equally idiomatic.
I've definitely written buggy goroutine code before. AMA :p
I vaguely remember expecting a reference and getting a copy or vice versa...
I mean, I agree that Go is miles ahead of most other languages when it comes to helping prevent concurrency bugs, but it's sill tricky. Same with Rust.
I vaguely remember expecting a reference and getting a copy or vice versa...
I mean, I agree that Go is miles ahead of most other languages when it comes to helping prevent concurrency bugs, but it's sill tricky. Same with Rust.
Go makes concurrency about as tricky as a reasonably complex data structure. can you still write a bug? Of course. Is it so tricky that bugs are inevitable, or even common? Absolutely not.
Wait, what? What happened to "fearless concurrency"? I thought this was supposed to be one of the borrow checker's selling points!
https://blog.rust-lang.org/2015/04/10/Fearless-Concurrency.h...
https://blog.rust-lang.org/2015/04/10/Fearless-Concurrency.h...
I mean, it is, yes. That post is talking about threads. And the “fearless” name meant that it solves a lot of issues at compile time, which it still does in an async context.
Like any static analysis, it’s a give and take between making sure your analysis is sound, while still allowing useful programs.
Like any static analysis, it’s a give and take between making sure your analysis is sound, while still allowing useful programs.
Whether it's kernel threads or green threads, the same patterns (locks, etc) are possible. Locks are supposed to be the borrow checker's bread and butter, because it can guarantee they are held before accessing shared state. But now you're saying "the borrow checker makes writing code without [async/await] difficult, inefficient, and unergonomic."
I'm not saying locks are better than async/await (although they are[1]). You're saying the borrow checker itself can't handle them in real world use?
[1] https://journal.stuffwithstuff.com/2015/02/01/what-color-is-...
I'm not saying locks are better than async/await (although they are[1]). You're saying the borrow checker itself can't handle them in real world use?
[1] https://journal.stuffwithstuff.com/2015/02/01/what-color-is-...
I am not saying that the borrow checker cannot handle locks. Locks work great.
(The borrow checker does not understand locks as a special construct, to be extra clear.)
Did you read the post I linked? It lays out the details. I am happy to clarify if you don’t get the specifics.
(The borrow checker does not understand locks as a special construct, to be extra clear.)
Did you read the post I linked? It lays out the details. I am happy to clarify if you don’t get the specifics.
I see now, I misunderstood your original post. You were saying async/await is necessary because futures work badly, not because all the alternatives (i.e. locks) work badly.
Sorry, my mistake!
Edit to add: futures work badly in every language, so there's no shame in the borrow checker not working with them.
Edit 2: But in that case we're back to "why would Rust want async/await over (potentially green) threads with its first-class support for locks?"
Sorry, my mistake!
Edit to add: futures work badly in every language, so there's no shame in the borrow checker not working with them.
Edit 2: But in that case we're back to "why would Rust want async/await over (potentially green) threads with its first-class support for locks?"
I believe these are the talks from Steve that he's referring to. It was enlightening for me:
1. Rust's Journey to Async/Await - https://www.youtube.com/watch?v=lJ3NC-R3gSI
2. The Talk You've been Await-ing for - https://www.youtube.com/watch?v=NNwK5ZPAJCk
The first video goes into all the bits you're concerned about and all the things that Rust has tried before arriving where they are now
1. Rust's Journey to Async/Await - https://www.youtube.com/watch?v=lJ3NC-R3gSI
2. The Talk You've been Await-ing for - https://www.youtube.com/watch?v=NNwK5ZPAJCk
The first video goes into all the bits you're concerned about and all the things that Rust has tried before arriving where they are now
Language support for green threads require a heavier runtime (so you'd pay the performance cost even when you didn't write async code).
Tokio basically is green threads as a library.
Tokio basically is green threads as a library.
Regarding your edit 2, I linked two talks I have that go over this in great detail elsewhere in this thread.
Yes. There are some difficult technical issues. In practice, borrow checker works less well on async code than threaded code.
This is partly why I prefer threading over async in Rust. Look, we went through some enormous effort to make threading good and fun again. Why wouldn't you use threading?
This is partly why I prefer threading over async in Rust. Look, we went through some enormous effort to make threading good and fun again. Why wouldn't you use threading?
> Why wouldn't you use threading?
Because the C10^nK problem where n increases periodically is still a thing?
Because the C10^nK problem where n increases periodically is still a thing?
I am not solving C10K problem.
Green threads.
Those are async tasks
Or rather -- async tasks are as close as you can get to green threads in rust without a runtime that would impose overhead on every program
Or rather -- async tasks are as close as you can get to green threads in rust without a runtime that would impose overhead on every program
Because you cannot win artificial benchmarks with threading.
It's still fearless, as in you don't need to worry that you might create data races, but can be clunky to write in some cases.
If the borrow checker has no representation of a memory model, for example relaxed/acquire/release, you can't write a concurrent queue without triple checking for statement ordering, resulting barriers and then formally verify it otherwise you are very likely to introduce data races.
The borrow checker doesn’t understand orderings, as it doesn’t specifically need to. You can get race conditions, but not data races. Yes, you need to be careful when writing this kind of code.
It all comes down to "is it such a common scenario?". Honest answer is no. async is for specialized situations and shouldn't be the default.
Rust is a systems programming language. Modern server software -- a primary use case for a systems language -- is heavily async by default for a long list of compelling architectural reasons. Providing first-class language tooling to support that seems eminently sensible since this is how people will want to use the language.
When you are writing high-performance server code, async is the common scenario.
When you are writing high-performance server code, async is the common scenario.
Depending on the kind of work you're doing, it may or may not be. If you're doing a lot of network IO, then yes, it is very common.
async is not the default. The standard library is 100% blocking, and Rust does not come with a runtime. However, async makes sense for a lot of people, which is why libraries like tokio and async-std are so popular.
async does not make sense for a lot of people and I am deeply worried by hype-driven popularity of Tokio and async-std in Rust.
Especially harmful is that common libraries like reqwest (for HTTP requests) pulls async.
Especially harmful is that common libraries like reqwest (for HTTP requests) pulls async.
Web servers are the quintessential product of async. It’s no surprise that for an industry dominated by web titans spend a lot of time writing web servers and have a huge interest in asynchronous processing. The importance of a sync was cemented way back in 1999 with the c10k problem with nginx vs Apache.
Most people are not writing the next nginx. They are writing web application servers behind nginx.
This, merely, means that the web application code will hold back what Nginx is capable of serving.
If you know that there will be 10k concurrent users in your application, go ahead and use async right from the start.
But for the rest of us, simple, blocking code will do just fine and save us a few headaches.
But for the rest of us, simple, blocking code will do just fine and save us a few headaches.
Alternatively use async everywhere from the start and your headaches go away too. It's mixing blocking code with async code that causes issues.
Just do it if everyone in your team is comfortable with it. I'm just not as productive with async code.
Ah yeah, as someone whose first langauge is javascript where all IO and even things like timers are async, I forget that not everyone groks it. It's really not that complicated (at work we have junior devs with 6 months experience writing async code no problem), but I think there is a certain amount of unlearning that needs to be done if you're used to working with threaded code.
You underestimate how common this use case is. It motivates the Go language, which is much more widely used than Rust.
>But for the rest of us, simple, blocking code will do just fine and save us a few headaches.
I understand your point, but if performance isn't a concern, why use Rust at all? If the intricacies of async is that much of a burden then Rust is probably not the right tool of the job.
I understand your point, but if performance isn't a concern, why use Rust at all? If the intricacies of async is that much of a burden then Rust is probably not the right tool of the job.
And once you get to a limit, the alternative to rewriting everything in non-blocking code might be to put multiple instances of your blocking app on multiple VMs/k8s/whatever behind a load balancer.
Or, you just take the initial leap and write async from the start. For languages with decent abstractions such as async/await, it really isn't hard when you've done it for a while, and I'd make the same argument as one of the parent posters in that it's great "documentation".
K8s is brings way more complexity and headache, so it's kind of funny that you suggest that before using async/await.
K8s is brings way more complexity and headache, so it's kind of funny that you suggest that before using async/await.
> For languages with decent abstractions such as async/await, it really isn't hard when you've done it for a while
https://lucumr.pocoo.org/2016/10/30/i-dont-understand-asynci...
https://lucumr.pocoo.org/2016/10/30/i-dont-understand-asynci...
Setting up k8s once solves the problem for all your apps. In contrast, the additional software complexity of async/await is duplicated across all of them.
If your planning to scale up, k8s is likely something you'll want later. Additional complexity in your apps is not.
If your planning to scale up, k8s is likely something you'll want later. Additional complexity in your apps is not.
You probably shouldn't introduce K8s just to solve that problem. I'm just saying if you're already on k8s, scaling horizontally becomes easy.
Any web application will already be "holding back" Nginx because Nginx doesn't have to do things like database queries...
Maybe in your world!
In fairness it's just the announcement of V1 of the library. There doesn't seem to me to be anyone promoting its use "everywhere".
On the other hand, there has been from day one a sort of hype around Rust's safety features, and an eagerness to promote any new library or framework written in Rust as a savior of programming. This library, as you note, will be used inappropriately (i.e. in contexts where it's not really necessary or reasonable to do) and lead to the worst kinds of bugs--those that lurk in complicated, difficult to understand code, and that are generally worse than any memory-related security vulnerability.
On the other hand, there has been from day one a sort of hype around Rust's safety features, and an eagerness to promote any new library or framework written in Rust as a savior of programming. This library, as you note, will be used inappropriately (i.e. in contexts where it's not really necessary or reasonable to do) and lead to the worst kinds of bugs--those that lurk in complicated, difficult to understand code, and that are generally worse than any memory-related security vulnerability.
If you think OS threads are "better" than async tasks, then use them. Other people want to use async, so they use it. Rust does not have a runtime and provides blocking APIs by default, but gives you the option to use async if you want to.
Shouldn't an HTTP request library make async available? HTTP is the textbook example for async.
It should, but it shouldn't be the default. Right now, reqwest is the default HTTP request library in Rust ecosystem, and async is mandatory for reqwest. This is a bad situation to be in.
> async is mandatory for reqwest
async is not mandatory for reqwest. It provides a blocking API as well.
> reqwest is the default HTTP request library in Rust ecosystem
There is no "default" libraries. There are popular HTTP clients other than reqwest that also provide blocking APIs such as isahc and ureq
async is not mandatory for reqwest. It provides a blocking API as well.
> reqwest is the default HTTP request library in Rust ecosystem
There is no "default" libraries. There are popular HTTP clients other than reqwest that also provide blocking APIs such as isahc and ureq
reqwest provides a blocking API, but reqwest also always depends on Tokio. A blocking API doesn't help when I don't want Tokio in my dependency tree at all.
I am using isahc, but that also doesn't help when (say) Rusoto AWS library pulls reqwest pulls Tokio pulls async.
I am using isahc, but that also doesn't help when (say) Rusoto AWS library pulls reqwest pulls Tokio pulls async.
Doesn't sound like a problem with async, just sounds like you disagreeing about what good deps for your deps are.
Can you explain exactly what that library using Tokio internally exposes to you that's a problem? Because, as written, this sounds like a religious argument.
If you need to use (for the sake of the example) Rusoto, since it is based on tokio, you'll need to set up the Tokio executor or at least add a macro to your main for this to be done for you. I believe Rusoto actually would take care of this for you if you haven't done it yourself however friction arises when you were already using a different version of tokio and Rusoto is built against another.
Basically, it isn't entirely opaque to you how it is handled.
Basically, it isn't entirely opaque to you how it is handled.
Gotcha, makes sense. Thanks!
I had a simple rust program that used reqwest -- it just pulled down a few web pages and parsed some data out of tables in the HTML. At the time reqwest had a simple synchronous API function that made this easy. The version of reqwest which added async support broke compatibility of that function and didn't appear to provide any similarly easy to use equivalent. Luckily my use-case went away (the website I was screen scraping died) so I didn't need to try to actually fix my program to work with newer reqwest versions. But it left a pretty sour taste regarding async...
There's ureq for sync/blocking HTTP requests.
Req west? The best.
Parallelism is complicated, and it is valuable.
[deleted]
> when you have a huge number of very small tasks running concurrently because that's generally where OS-driven parallelism tends to suffer but is it such a common scenario
Web servers are all about I/O and handling small tasks (requests), and are a perfect use case for asynchronous programming.
> That sounds like premature optimization in many situations IMO
Maybe in some cases... but then just don't use futures. Rust does not have a runtime, so it gives you the choice. std is all blocking, so you can spawn threads and do event-driven programming, which might be just fine for a lot of people.
async is more ergonomic for Rust specific reasons, makes sense for a lot of use cases, and was a highly requested language feature, so it was added to the language. OS threads can work just fine for many people. If that includes you, you don't have to use async.
Web servers are all about I/O and handling small tasks (requests), and are a perfect use case for asynchronous programming.
> That sounds like premature optimization in many situations IMO
Maybe in some cases... but then just don't use futures. Rust does not have a runtime, so it gives you the choice. std is all blocking, so you can spawn threads and do event-driven programming, which might be just fine for a lot of people.
async is more ergonomic for Rust specific reasons, makes sense for a lot of use cases, and was a highly requested language feature, so it was added to the language. OS threads can work just fine for many people. If that includes you, you don't have to use async.
> Web servers are all about I/O and handling small tasks (requests), and are a perfect use case for asynchronous programming.
In principle yes, but in practice I disagree. Due to keep-alive you want to be able to handle many idle connections at the same time, but you rarely want to handle many active connections at the same time.
Example: Whenever you talk to another services (e.g. a database) you need to limit the number of connections you have open. You can't just blindly open a new connection per incoming request. This means that your practical level by parallelism is often bounded by your database. If every request talks to Postgres and you have a connection pool with a limit of 50, then there is nothing to gain by having support for 1000s of "active" connections. You'd rather want Postgres to focus on finishing existing requests than opening new ones.
And once you look into Postgres you'll observe the same thing: There's only limited amount of CPU/IO so there's no point in having 1000s of "active" requests going on at the same time.
In principle yes, but in practice I disagree. Due to keep-alive you want to be able to handle many idle connections at the same time, but you rarely want to handle many active connections at the same time.
Example: Whenever you talk to another services (e.g. a database) you need to limit the number of connections you have open. You can't just blindly open a new connection per incoming request. This means that your practical level by parallelism is often bounded by your database. If every request talks to Postgres and you have a connection pool with a limit of 50, then there is nothing to gain by having support for 1000s of "active" connections. You'd rather want Postgres to focus on finishing existing requests than opening new ones.
And once you look into Postgres you'll observe the same thing: There's only limited amount of CPU/IO so there's no point in having 1000s of "active" requests going on at the same time.
You may want async if you are writing the next nginx, but for most web application servers threading is perfectly fine.
Again, depends on if you're doing a lot of IO, especially if you want request-based IO parallelism.
It's worth mentioning that even if you're IO-bound at your DB, running an async application server now means you don't need to tie up a thread waiting on it. Memory usage aside, you more or less don't need to think about threads much, whereas a threadpool (one waiting on IO) is something you have to actively manage.
It's worth mentioning that even if you're IO-bound at your DB, running an async application server now means you don't need to tie up a thread waiting on it. Memory usage aside, you more or less don't need to think about threads much, whereas a threadpool (one waiting on IO) is something you have to actively manage.
But nginx is usually just a proxy for connecting to app servers, and your app servers still need to handle all of those requests.
[deleted]
Rust is intended as a systems programming language, it's for people who are writing "the next nginx". It turns out that there are also a bunch of people who want to write webapp servers in Rust, too, but that's never really been the goal.
Eh, so Rust is not for me? I seem to have heard Rust being inclusive and empowering everyone blah blah. I must have misheard.
It's clear this is one of your hobby horses. Every comment thread here is encumbered with you pointing out that you wouldn't like it if async were the default, fair enough. In fact, you don't seem to like the idea in general.
ctrl-f "sanxiyn" yields 28 instances, most of them restating in every subtree the same point about how you think threads > async.
Since I think most of us tend to read the comments section top to bottom, it seems ideal to limit your opinion to a couple comments and then put your effort into making those comments a good rundown of your position. It would certainly be more interesting to read and consider.
ctrl-f "sanxiyn" yields 28 instances, most of them restating in every subtree the same point about how you think threads > async.
Since I think most of us tend to read the comments section top to bottom, it seems ideal to limit your opinion to a couple comments and then put your effort into making those comments a good rundown of your position. It would certainly be more interesting to read and consider.
You may want to contribute something concrete. I did learn some new advantages of async over threading from replies, besides tired C10K. Yes, async is useful for C10K. No, I am not solving C10K problem.
1. If you use async in single thread mode, you can save thread synchronization.
2. async works better for idle connections and slow connections, even when the absolute number of connections is not large.
3. async task is easier to cancel than thread.
I still won't use async since thread synchronization hasn't been slow for me, thread cancellation hasn't been problematic for me, and I use nginx to handle idle connections and slow connections, but it's useful to know in case I need.
1. If you use async in single thread mode, you can save thread synchronization.
2. async works better for idle connections and slow connections, even when the absolute number of connections is not large.
3. async task is easier to cancel than thread.
I still won't use async since thread synchronization hasn't been slow for me, thread cancellation hasn't been problematic for me, and I use nginx to handle idle connections and slow connections, but it's useful to know in case I need.
I work at a company where I do routinely need to handle C10K problems.
I also routinely interview candidates that want to work at my company. We typically downlevel or turn away candidates who do not have experience solving C10K problems (unless they can appropriately fake that experience).
Even though you may not need to solve C10K problems, (like in any education) it is typically very useful for engineers to think about and attempt solving artificial C10K problems to better educate themselves for when they need to solve those problems.
Meanwhile, if you're the CTO of a company and truly know your business will not require C10K ever in its life, and you know this is the wrong time to educate yourself and your employees, then yes you're correct that async is the wrong abstraction for you right now. Frankly in that case I'd argue Rust may also be the wrong abstraction for right now.
I also routinely interview candidates that want to work at my company. We typically downlevel or turn away candidates who do not have experience solving C10K problems (unless they can appropriately fake that experience).
Even though you may not need to solve C10K problems, (like in any education) it is typically very useful for engineers to think about and attempt solving artificial C10K problems to better educate themselves for when they need to solve those problems.
Meanwhile, if you're the CTO of a company and truly know your business will not require C10K ever in its life, and you know this is the wrong time to educate yourself and your employees, then yes you're correct that async is the wrong abstraction for you right now. Frankly in that case I'd argue Rust may also be the wrong abstraction for right now.
He's multi-threading !
(Sorry could not help myself :-D)
How many threads can you spawn before the system grinds to a halt? If you're processing thousands of requests per second and each request gets its own thread then you will start to queue on thread spawning. Don't forget that each thread gets its own stack taking up megabytes of memory.
The async concept has been used for decades in pretty much every product I've worked on professionally, from enterprise raid controllers to network protocol implementations and telephony software. An engineer I respect once told me that really it's the only way to write services at scale, and anything else is just a step on the road until you reinvent it. He was probably exaggerating, but it is very important, and nearly ubiquitous.
Having used custom frameworks for async code in C and C++, it's really refreshing to have it baked into the language and well supported. It's yet another arrow in Rust's fantastic quiver.
The async concept has been used for decades in pretty much every product I've worked on professionally, from enterprise raid controllers to network protocol implementations and telephony software. An engineer I respect once told me that really it's the only way to write services at scale, and anything else is just a step on the road until you reinvent it. He was probably exaggerating, but it is very important, and nearly ubiquitous.
Having used custom frameworks for async code in C and C++, it's really refreshing to have it baked into the language and well supported. It's yet another arrow in Rust's fantastic quiver.
rouille (my Rust web framework of choice) spawns threads each request, and it handles thousands of requests per second just fine. Computers are fast, and Rust doesn't slow down your computer.
If you can't handle thousands of requests per second with thread per request, that's more about your software stack, not about threading.
I guess it was different in the past when computers were slow. I can believee that.
If you can't handle thousands of requests per second with thread per request, that's more about your software stack, not about threading.
I guess it was different in the past when computers were slow. I can believee that.
How does it handle slow http attacks? If I open 10k TCP connections to your server and drip feed http requests 1 byte at a time on each connection, what happens?
You used to be able to easily DOS apache servers this way, because you just needed enough concurrent connections to exhaust its thread pool and then it wouldn't be able to handle any more requests. And then you need a bit rate on each connection just high enough not to trip apache's connection timeout. (So like, 20 TCP connections each sending 1 byte every 20 seconds would do it. Not sure about today but Apache used to be brought to its knees with 1 bps of bandwidth.)
You could probably mitigate this by putting nginx in front of your server, but this works because nginx uses async internally to handle requests. And that won't work if you ever do proxy passthrough (for SSE, websockets, etc).
You used to be able to easily DOS apache servers this way, because you just needed enough concurrent connections to exhaust its thread pool and then it wouldn't be able to handle any more requests. And then you need a bit rate on each connection just high enough not to trip apache's connection timeout. (So like, 20 TCP connections each sending 1 byte every 20 seconds would do it. Not sure about today but Apache used to be brought to its knees with 1 bps of bandwidth.)
You could probably mitigate this by putting nginx in front of your server, but this works because nginx uses async internally to handle requests. And that won't work if you ever do proxy passthrough (for SSE, websockets, etc).
> it handles thousands of requests per second just fine
It probably doesn't. Spawning threads per request is a lazy pattern.
We use Apache to handle billions of requests per day, and even its thread pool can be an issue.
It probably doesn't. Spawning threads per request is a lazy pattern.
We use Apache to handle billions of requests per day, and even its thread pool can be an issue.
Are you aware that rouille and tiny_http (it's underlying HTTP implementation) have been unmaintained for a while now?
And once starting and stopping threads adds to much delay to your request processing, there could be a thread pool that grows as needed and which will reuse threads that haven't been closed yet.
This mechanism is implemented by Apache httpd, Tomcat and pretty much every classic application server.
This mechanism is implemented by Apache httpd, Tomcat and pretty much every classic application server.
> This mechanism is implemented by Apache httpd, Tomcat and pretty much every classic application server.
True, but there's a reason that Apache usage is declining at the rate that it is.
True, but there's a reason that Apache usage is declining at the rate that it is.
There are some pretty bad usability problems with most async APIs too.
One is they make your functions colored; async functions world best with other async functions while normal blocking functions work best with other blocking functions.
They also introduce a lot of noise; putting async/await everywhere doesn't tell you anything interesting.
Considering a normal-sized Linux server can handle a million threads without much trouble, it really seems like misplaced effort.
One is they make your functions colored; async functions world best with other async functions while normal blocking functions work best with other blocking functions.
They also introduce a lot of noise; putting async/await everywhere doesn't tell you anything interesting.
Considering a normal-sized Linux server can handle a million threads without much trouble, it really seems like misplaced effort.
In the Java world, project Loom[1] is hopefully going to end this situation of async code that is hard to use with blocking code. They introduce a concept called Virtual Threads (previously called Fibers, but they are still looking for the perfect name). This will allow for seamless interoperability between blocking and non-blocking code as everything in Java runs on a Thread and Virtual Threads are just a specialization of the concept that doesn't boil down to OS threads.
I haven't used it yet, so I can only repeat the advertising copy, but nevertheless wanted to give some perspective from other ecosystems.
[1]: https://wiki.openjdk.java.net/display/loom/Main
I haven't used it yet, so I can only repeat the advertising copy, but nevertheless wanted to give some perspective from other ecosystems.
[1]: https://wiki.openjdk.java.net/display/loom/Main
Go and Zig already solved the problem. Java will follow soon with Loom, can't wait.
Go “solved the problem” at the expense of being unable to interface with C libraries without a big performance penalty (and that's you'll have Rust in Chromium and not Go)
> One is they make your functions colored; async functions world best with other async functions while normal blocking functions work best with other blocking functions.
Not in Zig: https://kristoff.it/blog/zig-colorblind-async-await/
Not in Zig: https://kristoff.it/blog/zig-colorblind-async-await/
> One is they make your functions colored
I don't get what's so bad about "colored" function. That color is just about the return type of the function.
How do you return an error from a function that does not return a Result? You must call unwrap (panic) or change the colour of your function by changing the return type to Result and fix all the caller.
Similarly, if you want to use a future from a non-async function, you either call `block_on(...)`, or you change the return type to a future by marking the function async.
I don't think it is that bad. That's just the way explicitly typed programming languages work.
I don't get what's so bad about "colored" function. That color is just about the return type of the function.
How do you return an error from a function that does not return a Result? You must call unwrap (panic) or change the colour of your function by changing the return type to Result and fix all the caller.
Similarly, if you want to use a future from a non-async function, you either call `block_on(...)`, or you change the return type to a future by marking the function async.
I don't think it is that bad. That's just the way explicitly typed programming languages work.
I think you've made me realize why I don't enjoy Rust. Every function is colored.
Zig's "colorblind" async is very exciting for this reason: https://kristoff.it/blog/zig-colorblind-async-await/
I'm skimming your link trying to understand the design. It sounds like there is a global flag for whether the whole program is in evented or blocking mode?
> during compile-time, it’s possible to inspect if the overall program is in evented mode or not, and properly designed code might decide to move to a threaded model when in blocking mode, for example.
> during compile-time, it’s possible to inspect if the overall program is in evented mode or not, and properly designed code might decide to move to a threaded model when in blocking mode, for example.
Sounds like a horribly complex special case. I'd far rather just have higher-kinded types and be able to write sometimes-async code using normal polymorphism (like I do in Scala all the time).
Yep, I was mind blown when I started using futures years ago with this coloring. But then I realized it’s all about types and monoids and applicatives and it started to get clear why I just can’t get the value out of a promise.
> One is they make your functions colored
This is definitely a good thing. All computations should me "marked" as total or effectful with various possible effects (blocking, async, nondeterministic, possibly non-terminating etc etc).
Reasoning about your program is hard when each computation is a blackbox possibly containing any side-effects which could cause unpredictable changes in the control flow and result in a completely incomprehensible way.
Async/await thing is indeed least sound and ergonomic way of doing this. Monads with monad transformers are a bit better. Algebraic effects are the best in terms of composability, ergonomics and mental overhead, but not here yet (though OCaml may be soon become the first industrial-grade language incorporating them [1])
[1] https://www.youtube.com/watch?v=z8SI7WBtlcA
This is definitely a good thing. All computations should me "marked" as total or effectful with various possible effects (blocking, async, nondeterministic, possibly non-terminating etc etc).
Reasoning about your program is hard when each computation is a blackbox possibly containing any side-effects which could cause unpredictable changes in the control flow and result in a completely incomprehensible way.
Async/await thing is indeed least sound and ergonomic way of doing this. Monads with monad transformers are a bit better. Algebraic effects are the best in terms of composability, ergonomics and mental overhead, but not here yet (though OCaml may be soon become the first industrial-grade language incorporating them [1])
[1] https://www.youtube.com/watch?v=z8SI7WBtlcA
> This is definitely a good thing. All computations should me "marked" as total or effectful with various possible effects (blocking, async, nondeterministic, possibly non-terminating etc etc).
Marking functions for side-effects would be a good thing but it isn't what function coloring means in this context.
An async function and a normal function are semantically the same, they just have different syntaxes and you can't easily call one from another.
They can be both be either blocking or non-blocking, especially if they take other functions as arguments.
Marking functions for side-effects would be a good thing but it isn't what function coloring means in this context.
An async function and a normal function are semantically the same, they just have different syntaxes and you can't easily call one from another.
They can be both be either blocking or non-blocking, especially if they take other functions as arguments.
They're not really the same: you know that an async function may potentially suspend and have other code run before its completion, while a normal function (in the absence of threads and signals) is guaranteed to run atomically, at least as far as your process's memory space is concerned. You also know that the only points at which an async function may suspend are an 'await' statement, and so data invariants that don't cross await statements or other async function calls can be reasoned about as if you had purely sequential code.
That's precisely the coloring that makes async useful. Without it you need to explicitly protect all shared data with mutexes or other synchronization primitives. 40+ years of threaded programming has shown that programmers cannot generally be trusted to get this right, and this in an area ripe with bugs.
That's precisely the coloring that makes async useful. Without it you need to explicitly protect all shared data with mutexes or other synchronization primitives. 40+ years of threaded programming has shown that programmers cannot generally be trusted to get this right, and this in an area ripe with bugs.
Programmers cannot get threading right, but Rust can.
Rust won't magically make your threaded code blocks right. It can just provide you tools to ensure that memory won't leak. It's just 1/10 of the solution.
Rust does not ensure that your code is free of memory leaks, to be clear.
Rust solves threading-specific problems. Yes it's partial, but other problems happen in both threaded code and async code, so that's not a reason to choose one over another.
> An async function and a normal function are semantically the same
But they are not, AsyncIO and BlockingIO are different side effects, thus you have different types of computation, that's exactly what I'm talking about. In languages with monads or algebraic effects these would have different types.
You don't say that Lists and Arrays are semantically the same, despite being similar sequential collections, they still have separate types for a reason. Though it's good to be able to abstract over them.
And in languages with monads we can parametrize over various effect types by using tagless final approach, which allows us to write computations which could be interpreted in contexts of various effects (in this case, Async and Sync), just as we parametrize containers with types of content (in [1] there is an example of how we can parametrize computation over various async implementations), but still these are different effects.
[1] https://kubuszok.com/2019/io-monad-which-why-and-how/#typed-...
But they are not, AsyncIO and BlockingIO are different side effects, thus you have different types of computation, that's exactly what I'm talking about. In languages with monads or algebraic effects these would have different types.
You don't say that Lists and Arrays are semantically the same, despite being similar sequential collections, they still have separate types for a reason. Though it's good to be able to abstract over them.
And in languages with monads we can parametrize over various effect types by using tagless final approach, which allows us to write computations which could be interpreted in contexts of various effects (in this case, Async and Sync), just as we parametrize containers with types of content (in [1] there is an example of how we can parametrize computation over various async implementations), but still these are different effects.
[1] https://kubuszok.com/2019/io-monad-which-why-and-how/#typed-...
I don't think considering the blocking strategy an effect is very useful. Even in async context in complex enough applications functions can call other functions including your own, so async is not enough to guarantee reentrancy.
I do agree that parametrizing over the blocking strategy is a great idea, but languages that simply provide an async syntactic marker don't necessarily allow that, and if your language is powerful enough you do not need the annotation in the first place.
I do agree that parametrizing over the blocking strategy is a great idea, but languages that simply provide an async syntactic marker don't necessarily allow that, and if your language is powerful enough you do not need the annotation in the first place.
Yes, the standard library has a bunch of legacy blocking IO stuff. But in general, most modern libraries tend to stick to a convention of:
- async (or explicitly Future/Stream): external effects (I/O, interacts with synchronization, whatever)
- &mut: local effects
- otherwise: pure
- async (or explicitly Future/Stream): external effects (I/O, interacts with synchronization, whatever)
- &mut: local effects
- otherwise: pure
> Considering a normal-sized Linux server can handle a million threads without much trouble, it really seems like misplaced effort
Seems like that would use a lot of memory for all the stacks
Seems like that would use a lot of memory for all the stacks
Based on this, the default value is 2MB: https://unix.stackexchange.com/questions/127602/default-stac...
So that would mean a lot of memory for 1 million threads, 2TB of RAM. But you can change the default. With a 64k stack you'd use up ~68GB of RAM, which doesn't seem like a lot for 1 million threads and 1 million requests happening at the same time.
So that would mean a lot of memory for 1 million threads, 2TB of RAM. But you can change the default. With a 64k stack you'd use up ~68GB of RAM, which doesn't seem like a lot for 1 million threads and 1 million requests happening at the same time.
Also worth noting that the entire stack isn't allocated at once so 1 million threads would be using 2TB/68GB of virtual address space, not 2TB/68GB of physical memory.
That is indeed a very important fact to keep in mind! Thread stack sizes have been a problem with 32 bit systems where you quickly run out of virtual memory because the adress space is not large enough. With 64 bit that is not a problem anymore.
That is also the maximum stack size which shouldn't normally be reached. You'll have to be careful how you use memory when you're handling a million clients, either as async or threaded.
The point is that each stack need to be big enough for the worst case. That means it does not really scales to start many thousands of threads. While the futures themselves used in async code can be kept relatively small, as they only need to contain the state needed while awaiting.
In theory a smart enough OS (or runtime) should be able to reclaim any memory beyond the stack pointer (plus redzone) at any time without preserving its content and shrink back the stack. Because of signal handlers that memory is to be considered volatile anyway.
It might not be worth doing it in practice, but it is something to keep on mind.
It might not be worth doing it in practice, but it is something to keep on mind.
What "normal-sized Linux server" has 70GB of RAM?
One that wants to handle a million requests per second?
Or would you want to do that with a Raspberry Pi? :-)
> What "normal-sized Linux server" has 70GB of RAM?
Also, why are you "quoting" what I did not say?
Or would you want to do that with a Raspberry Pi? :-)
> What "normal-sized Linux server" has 70GB of RAM?
Also, why are you "quoting" what I did not say?
I was quoting the parent comment by @akvadrako :)
Air quotes.
Most servers support at least 128GB; it isn't even very expensive. And if you want to handle a million concurrent users you also need to consider CPU and latency, so for most real-world workloads the memory probably won't even be your bottleneck.
[deleted]
[deleted]
[deleted]
How much does a 68GB cost on the cloud per day? Also, you don't have 1 million cores so quite a bit of your daily server costs will be eaten up by the OS running context switching code.
[deleted]
Pending async waits have stacks to preserve too
The difference, at least in the way this is built in Rust, is that when you create a task, you get a single allocation that's exactly sized. There's no resizing, which means that you aren't getting stacks that are too big or too small, with all of the other runtime shenanigans that that entails.
That said, the future has the size of the biggest state that need to be kept across await. The future might be slightly oversized, but still order of magnitude smaller than a perfectly sized stack.
In some languges, that's true. In Rust, however, that pseudo-stack is typically tiny.
[deleted]
I don't quite understand why one would find async code hard to read.
Basically if you just ignore the async/await keyword, the code should read mostly the same as synchronous code (which is the point of the async/await effort).
Maybe you have a concrete example of convoluted async code?
Basically if you just ignore the async/await keyword, the code should read mostly the same as synchronous code (which is the point of the async/await effort).
Maybe you have a concrete example of convoluted async code?
Async seperates the concurrency from the runtime completely. You have code that returns a Future, and creates more Futures along the way. None of this imposes any constraint on the runtime, except that you need some runtime to evaluate the future. But that runtime could be quite simple and execute in the current process/thread (cf. the CurrentThread runtime), meaning you don't need support for threads at all.
Contrast this with the thread model, where the runtime needs to create and destroy threads where the code asks for it. In other words, your program logic is mixed up with runtime considerations.
For a practical example, let's say you want to use rust to extend some C code to create a network client that calls back into the C code. What if the C code is not thread-safe? With async, no problem, just use the CurrentThread runtime. This is my use case, anyway.
Contrast this with the thread model, where the runtime needs to create and destroy threads where the code asks for it. In other words, your program logic is mixed up with runtime considerations.
For a practical example, let's say you want to use rust to extend some C code to create a network client that calls back into the C code. What if the C code is not thread-safe? With async, no problem, just use the CurrentThread runtime. This is my use case, anyway.
That is an interesting positive for async/await that I had not heard before.
Why are there so many libraries that depend on Tokio then? If only the edges need to actually use an async runtime, the middle-tier libraries can just be plain futures.
Why are there so many libraries that depend on Tokio then? If only the edges need to actually use an async runtime, the middle-tier libraries can just be plain futures.
Because the standard library (and a lot of other libraries) are mostly blocking. You can't use blocking code effectively with async.
In other words with async you have to use code that knows how to yield, but the benefit is that you have a lot of flexibility in the runtime. That means you can scale up, scale down, and fit it into weird environments (like in other programs that aren't expecting concurrency).
Threading (or process model) has a lot of upside though, too. For one, it's pre-emptive, so scheduling can be more "fair". For another, it's a lot easier to debug (erlang does a great job here).
I'd generally default to using threads, unless I have a specific reason for async and/or the code is fairly low-level and might be used in a variety of environments.
Disclaimer: don't take my claims as authoritative. I know a few things from seeing what works and not, but I could be wrong on some of the finer points.
In other words with async you have to use code that knows how to yield, but the benefit is that you have a lot of flexibility in the runtime. That means you can scale up, scale down, and fit it into weird environments (like in other programs that aren't expecting concurrency).
Threading (or process model) has a lot of upside though, too. For one, it's pre-emptive, so scheduling can be more "fair". For another, it's a lot easier to debug (erlang does a great job here).
I'd generally default to using threads, unless I have a specific reason for async and/or the code is fairly low-level and might be used in a variety of environments.
Disclaimer: don't take my claims as authoritative. I know a few things from seeing what works and not, but I could be wrong on some of the finer points.
I think async was inevitable for Rust, for better or for worse. My experience is that just using OS-provided threads isn't good enough for say a high performance webserver—compare the pre-tokio hyper benchmark results to the post-tokio ones for example. And Go-like green threads aren't really possible in Rust given the choice to have no runtime. (Having a stack for every coroutine also probably isn't as good for squeezing out that last bit of performance. Every stack/guard page means more TLB pressure, for example, and programs can spend a lot of time on TLB cache misses.) Rust aims to be suitable for high-performance, low-level environments, so here we are.
I agree async has usability problems. Hopefully they'll get better over time; I'm looking forward to eventually having generators rather than dealing with futures::stream::StreamExt and the like.
I don't think everyone needs to use async all the time. Take a web app, for example. If the webserver is directly Internet-facing, it has to deal with lots of keepalive connections, so the core webserver logic (hyper or equivalent) should be async. But if you're not dealing with too many active requests and don't care about the performance difference, I don't think there's any reason you shouldn't have all your request handling just use threads, sending replies to hyper with a channel and blocking when necessary.
Last I checked I couldn't find an ergonomic and efficient "half-async" bounded channel implementation. By which I mean one that allows you to treat the sender as blocking and the receiver as async, or vice versa. That'd be really useful for writing synchronous programs that use async libraries. I certainly don't see any reason one couldn't exist. Maybe it already does and I missed it.
I agree async has usability problems. Hopefully they'll get better over time; I'm looking forward to eventually having generators rather than dealing with futures::stream::StreamExt and the like.
I don't think everyone needs to use async all the time. Take a web app, for example. If the webserver is directly Internet-facing, it has to deal with lots of keepalive connections, so the core webserver logic (hyper or equivalent) should be async. But if you're not dealing with too many active requests and don't care about the performance difference, I don't think there's any reason you shouldn't have all your request handling just use threads, sending replies to hyper with a channel and blocking when necessary.
Last I checked I couldn't find an ergonomic and efficient "half-async" bounded channel implementation. By which I mean one that allows you to treat the sender as blocking and the receiver as async, or vice versa. That'd be really useful for writing synchronous programs that use async libraries. I certainly don't see any reason one couldn't exist. Maybe it already does and I missed it.
I was trying to look for what channel I used back in the days I needed to write so called half-async code, but I can't find it anymore. If I remember correctly, at least the 0.1 version of futures had a channel with the other end being sync and the other async.
Today I'd maybe take a look into crossbeam and their queue implementations for this kind of synchronization.
https://docs.rs/crossbeam/0.8.0/crossbeam/
Today I'd maybe take a look into crossbeam and their queue implementations for this kind of synchronization.
https://docs.rs/crossbeam/0.8.0/crossbeam/
In a sense, the async API allows you to create green threads. Comparing with the normal threads, which rely on OS schedule and introduce context switches, green threads allow tasks actively yielding the control. This for example can be used to run multiple IO tasks concurrently all in one single OS thread.
See https://en.wikipedia.org/wiki/Green_threads
See https://en.wikipedia.org/wiki/Green_threads
Yes, but why would you want that? Using OS threads is better in most cases.
OS threads cost you stack space, among other things.
There's a reason that most OS native UI frameworks use some sort of event polling structure(WNDPROC[1], Looper/Handler[2], etc) because they're a nice structure for efficient handling of events. Tokio lets you do that across a diverse of events and get the same type of savings.
[1] https://en.wikipedia.org/wiki/Message_loop_in_Microsoft_Wind...
[2] https://developer.android.com/reference/android/os/Looper
There's a reason that most OS native UI frameworks use some sort of event polling structure(WNDPROC[1], Looper/Handler[2], etc) because they're a nice structure for efficient handling of events. Tokio lets you do that across a diverse of events and get the same type of savings.
[1] https://en.wikipedia.org/wiki/Message_loop_in_Microsoft_Wind...
[2] https://developer.android.com/reference/android/os/Looper
OS threads have a high real-world cost that limits throughput, and some things are much simpler to design in an async architecture generally. Context switching is expensive on modern systems, due to both CPU cache thrashing and implied resource cost. In many server applications the CPU will spend more time context switching in massively multithreaded architectures than actually running application code. An async architecture can complete tens of millions of independent operations per second in practice; the OS would have a difficult time just context-switching at that rate even if it was doing nothing else.
Furthermore, some classes of major macro-optimizations can't be implemented effectively if you do everything with kernel threading. This is the reason most modern server software architectures tend be thread-per-core pure async with no real multithreading per se -- it is for the performance.
At a more practical engineering level, these architectures are not more complicated, just different. Some things are much simpler to design because most multithread coordination problems go away. It is nice to be able to write virtually all of your code in single-threaded style where you don't have to worry about locking and consistency, especially as concurrency increases. On the other hand, you have to learn how to design schedules because the OS will no longer be doing that (poorly) for you. It isn't free in that you have to develop expertise in things you may not know but it is often worth it.
Furthermore, some classes of major macro-optimizations can't be implemented effectively if you do everything with kernel threading. This is the reason most modern server software architectures tend be thread-per-core pure async with no real multithreading per se -- it is for the performance.
At a more practical engineering level, these architectures are not more complicated, just different. Some things are much simpler to design because most multithread coordination problems go away. It is nice to be able to write virtually all of your code in single-threaded style where you don't have to worry about locking and consistency, especially as concurrency increases. On the other hand, you have to learn how to design schedules because the OS will no longer be doing that (poorly) for you. It isn't free in that you have to develop expertise in things you may not know but it is often worth it.
OS threads are preemptive and their scheduling is beyond your control, which introduces potential data races. Green threads allow for cooperative scheduling, which is much harder to mess up.
In my experience, past some level of code complexity, you get the pretty much same kind of races with cooperative scheduling as with preemptive scheduling – with the added risk that you did not expect a race with preemptive scheduling.
Still a fan of async programming, but that's a false benefit in my books :)
Still a fan of async programming, but that's a false benefit in my books :)
Most is not all. And some systems (especially embedded) do not have OS threads - they can still make use of green threads / async code.
> I guess I'm trying to understand if it's me who's missing something
No you’re not. There is a similar situation in Kotlin, which supports coroutines. Makes things more complicated and is often used for very questionable reasons.
This is why I’m excited about project Loom, which will use the same old thread abstraction but can be configured to use fibers under the hood instead. Java devs don’t even care that its not traditional threading, its the same API! This simultaneously solves the „coloredness“ problem of functions.
No you’re not. There is a similar situation in Kotlin, which supports coroutines. Makes things more complicated and is often used for very questionable reasons.
This is why I’m excited about project Loom, which will use the same old thread abstraction but can be configured to use fibers under the hood instead. Java devs don’t even care that its not traditional threading, its the same API! This simultaneously solves the „coloredness“ problem of functions.
> Makes things more complicated and is often used for very questionable reasons.
Why is it complicated? Because you didn't bother to learn new API and it is sufficiently different from older one?
Why is it complicated? Because you didn't bother to learn new API and it is sufficiently different from older one?
I think async in this sense leverages concurrency, which isn't necessarily tied to parallelism.
I.e. parallelism means executing several tasks on different compute units (cores) at the same time (multiple threads abstraction). Concurrency simply means that you can execute several tasks over a period of time, but it can happen on the same compute unit (so even within one thread). Tasks could be interleaved and still all progress over time.
I guess some ideal usage is a combination of parallelism and concurrency, but using a separate thread for each task isn't necessarily the most optimal method, because threads have their gotchas like context switching and etc.
I.e. parallelism means executing several tasks on different compute units (cores) at the same time (multiple threads abstraction). Concurrency simply means that you can execute several tasks over a period of time, but it can happen on the same compute unit (so even within one thread). Tasks could be interleaved and still all progress over time.
I guess some ideal usage is a combination of parallelism and concurrency, but using a separate thread for each task isn't necessarily the most optimal method, because threads have their gotchas like context switching and etc.
A very useful thing is that async stuff is easily cancelled in Rust, not so much threads—just making it go out of scope causes it to be cancelled in Rust, while you'd need to add cancellation logic to threads.
> you can just spawn threads, share data through channels or mutexes, use OS-provided async IO primitives to poll file descriptors and do event-driven programming […]
> I have to take an aspirin every time I need to dig into
async-heavy code.
I've seen that a lot on the internet and I guess it must depend where you come from, because I find async/await orders of magnitude easier to reason about than threads+channel (and I'm not even talking about using epoll manually, which is just inscrutable as soon as there is a little bit of complexity involved)
I've seen that a lot on the internet and I guess it must depend where you come from, because I find async/await orders of magnitude easier to reason about than threads+channel (and I'm not even talking about using epoll manually, which is just inscrutable as soon as there is a little bit of complexity involved)
> use OS-provided async IO primitives to poll file descriptors and do event-driven programming etc...
My understanding from reading along with many of these conversations is that the ultimate goal of the async IO frameworks is to amortize system call costs across multiple IO operations, instead of having >1 per.
I've heard some speculation about system call overhead going up in order to guarantee correctness (for Spectre and Meltdown class scenarios, but also for some other classes of concurrency issues). In which case io_uring is the carrot and system call slowdown the stick.
My understanding from reading along with many of these conversations is that the ultimate goal of the async IO frameworks is to amortize system call costs across multiple IO operations, instead of having >1 per.
I've heard some speculation about system call overhead going up in order to guarantee correctness (for Spectre and Meltdown class scenarios, but also for some other classes of concurrency issues). In which case io_uring is the carrot and system call slowdown the stick.
You're talking about the runtime aspects of executing concurrent code (OS threads vs green threads) but these are orthogonal to the idea of async APIs, which are one way of modelling concurrent programming flows.
For example, futures in Rust can be used with both OS threads and lightweight tasks. Tokio is mostly agnostic about the choice of the executor.
It definitely takes some time to grok async Rust (even if you come from C#/JS), but I think it really shines once you get to know it, similar to the benefits you get from learning about iterators & higher level functions as opposed to plain loops.
For instance, I've recently implemented the Raft protocol as part of a distributed algorithms course. Using Tokio and a single threaded executor made the implementation fairly readable, mostly relying on few async constructs(futures, tasks, channels and select loops) to model fairly complex behavior (bidirectional messaging, multiple states, timeouts, etc..) Doing so in a more traditional callback oriented style would've required maintaining a very complex state machine(in addition to the state machine of the algorithm itself)
Also, Async/Await originated in C#, a language which already supports threads(and many other concurrency models), not JavaScript which historically relied on callbacks
For example, futures in Rust can be used with both OS threads and lightweight tasks. Tokio is mostly agnostic about the choice of the executor.
It definitely takes some time to grok async Rust (even if you come from C#/JS), but I think it really shines once you get to know it, similar to the benefits you get from learning about iterators & higher level functions as opposed to plain loops.
For instance, I've recently implemented the Raft protocol as part of a distributed algorithms course. Using Tokio and a single threaded executor made the implementation fairly readable, mostly relying on few async constructs(futures, tasks, channels and select loops) to model fairly complex behavior (bidirectional messaging, multiple states, timeouts, etc..) Doing so in a more traditional callback oriented style would've required maintaining a very complex state machine(in addition to the state machine of the algorithm itself)
Also, Async/Await originated in C#, a language which already supports threads(and many other concurrency models), not JavaScript which historically relied on callbacks
I've not done nearly enough multithreaded programming, so this may be out of my depth, and what I'm saying and asking maybe completely wrong.
Isn't async by design more efficient even when you have multiple threads you can spawn? My understanding is the event loop would do async tasks in the thread's quiet times, and put those tasks to sleep while it's waiting for io and other things, meaning the thread isn't blocked. Comparing that to (my understanding of) threads, while you're not blocking the main thread, you're still blocking the spawned threads while waiting for io.
Isn't this thread blocking something you would want to avoid if you can regardless of whether or not you have additional threads to play with?
Isn't async by design more efficient even when you have multiple threads you can spawn? My understanding is the event loop would do async tasks in the thread's quiet times, and put those tasks to sleep while it's waiting for io and other things, meaning the thread isn't blocked. Comparing that to (my understanding of) threads, while you're not blocking the main thread, you're still blocking the spawned threads while waiting for io.
Isn't this thread blocking something you would want to avoid if you can regardless of whether or not you have additional threads to play with?
Yes! Programs can be much more efficient with non-blocking operations and a scheduler. This doesn't just benefit high performance web servers.
You gain some distinct advantages too by doing this, because you can then just run a single thread in your worker pool and if you ever need to make the program multi-threaded - god forbid - you can add locks to shared resources (or in Rust's case the compiler will help you with this) and then bump up the number of threads.
CPUs are really really fast today, I think engineers generally underestimate the amount of performance you can get with a single-thread and non-blocking I/O.
Async language constructs make this process a bit easier. The only issue IMO is that it is awkward to call async functions synchronously, or that it can have hidden costs to do so. I think languages will improve on this.
I believe on Linux, using non-blocking system calls on threads can help reduce expensive context switches too... whereas spawning multiple threads and having them use blocking system calls can cause more context switches.
I will say though.. I've seen developers just async-ify everything in codebases without thinking about why or if it is beneficial.
You gain some distinct advantages too by doing this, because you can then just run a single thread in your worker pool and if you ever need to make the program multi-threaded - god forbid - you can add locks to shared resources (or in Rust's case the compiler will help you with this) and then bump up the number of threads.
CPUs are really really fast today, I think engineers generally underestimate the amount of performance you can get with a single-thread and non-blocking I/O.
Async language constructs make this process a bit easier. The only issue IMO is that it is awkward to call async functions synchronously, or that it can have hidden costs to do so. I think languages will improve on this.
I believe on Linux, using non-blocking system calls on threads can help reduce expensive context switches too... whereas spawning multiple threads and having them use blocking system calls can cause more context switches.
I will say though.. I've seen developers just async-ify everything in codebases without thinking about why or if it is beneficial.
> Unless you actually think that async code is more expressive and easy to read and write than basic event loops but then you must be a lot smarter than I am because I have to take an aspirin every time I need to dig into async-heavy code.
I don't know about the end of this sentence, but yeah, I'm the kind of person who enjoys very much writing async code and finds that it (sometimes) models the problem much better than sync code and much, much, much better than event-driven/polling programming.
But then, I'm also the kind of person who enjoys coding with CML channels (aka Rust mpsc channels aka Go channels aka Erlang mailboxes aka pi-calculus channels etc.), so I guess I might be a mutant.
I don't know about the end of this sentence, but yeah, I'm the kind of person who enjoys very much writing async code and finds that it (sometimes) models the problem much better than sync code and much, much, much better than event-driven/polling programming.
But then, I'm also the kind of person who enjoys coding with CML channels (aka Rust mpsc channels aka Go channels aka Erlang mailboxes aka pi-calculus channels etc.), so I guess I might be a mutant.
I really like the Golang runtime. It uses coroutines (async) to speed up things (no context switches, low memory overhead) and automatically moves IO to threads where they can wait.
the "async/futures" way of writing code lets you write code that "looks like synchronous code" (write step 1, then step 2, then step 3, etc), while getting really good safety guarantees and not having to manage a state machine yourself.
The model doesn't work for all forms of concurrency, but I think it works for a lot of things that people at the "top of the stack" (application developers) do.
I don't know what kind of code you're looking at, in general, but you should be able to massage most async stuff into a list of things if you're not in callback soup. Granted, lots of people don't try to stay out of the callback soup, but.... I feel like even that is better than just like "try to validate concurrency invariants", which is a much harder problem for arbitrary code IMO?
The model doesn't work for all forms of concurrency, but I think it works for a lot of things that people at the "top of the stack" (application developers) do.
I don't know what kind of code you're looking at, in general, but you should be able to massage most async stuff into a list of things if you're not in callback soup. Granted, lots of people don't try to stay out of the callback soup, but.... I feel like even that is better than just like "try to validate concurrency invariants", which is a much harder problem for arbitrary code IMO?
It's syntactic sugar that makes code a lot easier to reason about.
Synchronous code is even easier to reason about than code using async syntax sugar.
No, multithreaded synchronous code is not easier to reason about than async. I think most would agree.
You probably think async code can't have deadlocks like threaded code can. Async is still multi-threaded it's just that the threads are done in user-space and contain an implicit global lock.
As someone who has criticized Rust for the small standard lib and lack of stability in the ecosystem I'm very, very happy for this. This is one of the biggest milestones for Rust. Rust itself is great enough. The lack of a de facto async runtime and libs for your standard networking needs were a real barrier for non-enthusiasts environments. It will help its adoption in general purpose business apps, I guess and hope.
Can someone explain to a non-rustacean what Tokio introduces that's not part of Rust? It looks like Rust provides the async/await semantics, so I'm guessing this is an event loop and dispatching system?
The Rust language provides the async/await syntax, which can turn imperative code into Future objects, but to run those Future objects, you must repeatedly call their poll method. Tokio is the piece of code that calls that method. It does so in a manner such that futures are only polled if able to continue work, and not e.g. waiting for a timer.
Besides that it provides lots of utilities for working with async code.
Besides that it provides lots of utilities for working with async code.
> but to run those Future objects, you must repeatedly call their poll method
Oh man. C++'s std::future has the same silliness. I've come to the conclusion that futures/promises are a dumb abstraction.
Oh man. C++'s std::future has the same silliness. I've come to the conclusion that futures/promises are a dumb abstraction.
What about this bugs you? You know that it’s not like, a busy loop calling poll all the time, right?
Well, I took the phrase 'repeatedly call' to mean pretty much that!
What bugs me about it, if I'm understanding it correctly, is that you have two options once an async call is issued:
- the calling thread effectively waits for completion. This is fine if a fork/join pattern is useful to you (i.e. issue N async calls and then wait for N completions). This isn't proper asynchrony though.
- the future is pushed on to a thread that does nothing but poll for completions. This effectively imposes an O(n) inefficiency into your code.
What bugs me about it, if I'm understanding it correctly, is that you have two options once an async call is issued:
- the calling thread effectively waits for completion. This is fine if a fork/join pattern is useful to you (i.e. issue N async calls and then wait for N completions). This isn't proper asynchrony though.
- the future is pushed on to a thread that does nothing but poll for completions. This effectively imposes an O(n) inefficiency into your code.
I can't speak to the same level of depth about the C++ model as the Rust one, but, while you could do those things, it's not the usual way that it works, at least, if I'm understanding your terms correctly. I'll admit that I find your terms in the first bullet pretty confusing, and the second, only slightly. Let's back up slightly. You have:
* A future. You can call poll on a future, and it will return you either "not yet" or "done." This API is provided by the standard library. You can create futures with async/await as well, which is provided by the language. These tend to nest, so you can end up with one big future that's composed out of smaller futures.
* A task. Tasks are futures that are being executed, rather than being constructed. Creating a task out of a future may place the future on the heap.
* An executor. This is provided by Tokio. By handing a future to Tokio's executor, you create a task. The job of the executor is to keep track of all tasks, and decide which one to call poll on next.
* A reactor. This is also provided by Tokio. An executor will often employ a reactor to help decide which task to execute and when. This is sometimes called an "event loop," and coordinates with the operating system (or, if you don't have one of those, the hardware) to know when something is ready.
* A Waker. When you call poll on a future, there's one more bit that happens we couldn't talk about until we talked about everything else. If a future is going to return "not yet," it also constructs a Waker. The Waker is the bridge between the task, the reactor, and the executor.
So. You have a task. That task needs to get something from a file descriptor in a non-blocking way. At some point, there's a future way down in the chain whose job it is to handle the file descriptor. When you ask it to be created, it will return "not ready", and construct a waker that uses epoll (or whatever) via the reactor. At some point, the data will be ready, and the reactor will notice, and tell the executor "hey this task is now ready to execute again," and when some time is free, the executor will eventually call poll on it a second time. But until that point, the executor knows it's not ready, and so won't call poll again.
Whew. Does that make sense? I linked my talks in this thread already, but this is kind of a re-hash of them.
* A future. You can call poll on a future, and it will return you either "not yet" or "done." This API is provided by the standard library. You can create futures with async/await as well, which is provided by the language. These tend to nest, so you can end up with one big future that's composed out of smaller futures.
* A task. Tasks are futures that are being executed, rather than being constructed. Creating a task out of a future may place the future on the heap.
* An executor. This is provided by Tokio. By handing a future to Tokio's executor, you create a task. The job of the executor is to keep track of all tasks, and decide which one to call poll on next.
* A reactor. This is also provided by Tokio. An executor will often employ a reactor to help decide which task to execute and when. This is sometimes called an "event loop," and coordinates with the operating system (or, if you don't have one of those, the hardware) to know when something is ready.
* A Waker. When you call poll on a future, there's one more bit that happens we couldn't talk about until we talked about everything else. If a future is going to return "not yet," it also constructs a Waker. The Waker is the bridge between the task, the reactor, and the executor.
So. You have a task. That task needs to get something from a file descriptor in a non-blocking way. At some point, there's a future way down in the chain whose job it is to handle the file descriptor. When you ask it to be created, it will return "not ready", and construct a waker that uses epoll (or whatever) via the reactor. At some point, the data will be ready, and the reactor will notice, and tell the executor "hey this task is now ready to execute again," and when some time is free, the executor will eventually call poll on it a second time. But until that point, the executor knows it's not ready, and so won't call poll again.
Whew. Does that make sense? I linked my talks in this thread already, but this is kind of a re-hash of them.
This is an awesome rundown of the whole stack. It's almost like you've explained this stuff before. ;)
It might sound complicated, but for typical applications almost all of this happens "under the hood". Usually you'll just add an attribute to `main` to start your runtime, then you can compose/await futures without ever needing to think about `poll` and friends.
Here's a small example: https://tokio.rs/tokio/tutorial/hello-tokio.
It might sound complicated, but for typical applications almost all of this happens "under the hood". Usually you'll just add an attribute to `main` to start your runtime, then you can compose/await futures without ever needing to think about `poll` and friends.
Here's a small example: https://tokio.rs/tokio/tutorial/hello-tokio.
Thanks :)
Would it be fair to compare this to Apple's Grand Central Dispatch a.k.a. libdispatch?
It's similar. libdispatch is primarily a task queue to simplify and optimize multithreaded workloads in objc/swift. Tokio uses a similar task queue pattern for scheduling cpu-bound or blocking io, but the main feature is an non-blocking/asynchronous IO runtime. Overall, it's more similar to libuv (the async io library that powers node.js), but with built-in support for Rust's async/await syntax
You mean io-bound
Yes in some regards. Both offer eventloops (in GCD: dispatch queues), which run small chunks of code which belongs to independent tasks on the same thread.
However there are some differences:
- Tokio is focussed on running async/await based code, whereas libdispatch currently mostly targets running callbacks/continuations. This might change once Swift offers async/await support, which for sure could run on top of GCD queues.
- GCD queues provide a lot more fine-grained control. users can exactly specify on which queue to run some code on. And tasks can jump between code. There might also be a dedicated main thread (UI) queue. Tokio just spins up a single queue which runs all code, which might be empowered by a multithreaded executor. This makes it less usable for UI.
However there are some differences:
- Tokio is focussed on running async/await based code, whereas libdispatch currently mostly targets running callbacks/continuations. This might change once Swift offers async/await support, which for sure could run on top of GCD queues.
- GCD queues provide a lot more fine-grained control. users can exactly specify on which queue to run some code on. And tasks can jump between code. There might also be a dedicated main thread (UI) queue. Tokio just spins up a single queue which runs all code, which might be empowered by a multithreaded executor. This makes it less usable for UI.
Yes, but it's strictly userspace. It's more like the concurrency primitives in the JVM. (Eg. work-stealing threadpool, timers, various abstractions over OS/kernel lower level async stuff.)
To me it's kind of like the event loop of node.js (which I believe comes from libuv)?
Yes, Deno uses it as its internal event loop for JS code.
Fun party fact: long ago, Rust had libuv built in too.
... for not-so-fun parties. ;-)
why’d they move past it? does libuv not fit well in rust?
Rust used to have a lot of stuff built into the language, including garbage collection![0] I think that a long time ago they chose to move stuff out into libraries so that Rust could be a serious competitor to C++.
https://pcwalton.github.io/2013/06/02/removing-garbage-colle...
https://pcwalton.github.io/2013/06/02/removing-garbage-colle...
Back then, Rust had a built-in runtime. It was removed, and libuv with it.
In very loose terms: Rust's async/await syntax defines an interface to async programming, not an implementation.
Rust leaves the implementation, and thereby choice of concurrency strategy, to libraries. Tokio is such a library. There's at least one other popular one of note.
Rust leaves the implementation, and thereby choice of concurrency strategy, to libraries. Tokio is such a library. There's at least one other popular one of note.
> There's at least one other popular one of note.
The other popular runtimes are async-std [0] and smol [1].
[0]: https://github.com/async-rs/async-std
[1]: https://github.com/smol-rs/smol
The other popular runtimes are async-std [0] and smol [1].
[0]: https://github.com/async-rs/async-std
[1]: https://github.com/smol-rs/smol
if I understand correctly, smol was created by the async-std folks, and async-std now uses it as its runtime
https://github.com/async-rs/async-std/pull/757
https://github.com/async-rs/async-std/pull/757
From what I've read, smol was created by its author and later on adopted by async-std. And, its author was in Tokio team, and later left and was in async-std team before he left and created smol. So smol was based on quite deep knowledge and experience in async.
AFAIK, the major component is the scheduler.
Rust can't really implement green threading (imagine Golang) by default, because it requires the runtime to be bundled in the executable, and the code to be compiled in a specific way to be managed by the scheduler.
I actually find amusing the thought that _this_ is systems programming. In a low level language one can implement the functionality of a higher level language (ie. Golang), but not the reverse :-)
Rust can't really implement green threading (imagine Golang) by default, because it requires the runtime to be bundled in the executable, and the code to be compiled in a specific way to be managed by the scheduler.
I actually find amusing the thought that _this_ is systems programming. In a low level language one can implement the functionality of a higher level language (ie. Golang), but not the reverse :-)
Both Rust and Go used to have green threads i.e. stackful coroutines / fibers that installed their own execution stack and switched to it.
However everyone who ever did green threads ended up having a lot of difficulties with stack optimization: - Go: https://docs.google.com/document/d/1wAaf1rYoM4S4gtnPh0zOlGzW... - Rust: https://mail.mozilla.org/pipermail/rust-dev/2013-November/00... - Same for Java in the past.
Now only Go is left.
More on fibers woes from the C++ point of view: http://www.open-std.org/JTC1/SC22/WG21/docs/papers/2018/p136...
However everyone who ever did green threads ended up having a lot of difficulties with stack optimization: - Go: https://docs.google.com/document/d/1wAaf1rYoM4S4gtnPh0zOlGzW... - Rust: https://mail.mozilla.org/pipermail/rust-dev/2013-November/00... - Same for Java in the past.
Now only Go is left.
More on fibers woes from the C++ point of view: http://www.open-std.org/JTC1/SC22/WG21/docs/papers/2018/p136...
Replying to myself, I found this bit in the docs explaining how Tokio decorates the main:
> An async fn is used as we want to enter an asynchronous context. However, asynchronous functions must be executed by a runtime. The runtime contains the asynchronous task scheduler, provides evented I/O, timers, etc.
> An async fn is used as we want to enter an asynchronous context. However, asynchronous functions must be executed by a runtime. The runtime contains the asynchronous task scheduler, provides evented I/O, timers, etc.
To provide some more color on why this isn't built in, different runtimes provide different kinds of guarantees and performance profiles. A webapp has very different requirements than an embedded system, and so we don't want to provide a single runtime. The language contains the basic things needed for the ecosystem to exist, and interoperation points, and then leaves the rest to said ecosystem.
(Some of those interoperation points are still being worked out, so it's not perfect yet.)
(Some of those interoperation points are still being worked out, so it's not perfect yet.)
When learning Rust a few months ago, I built a small client library for a REST API using reqwest (which uses Tokio). I then started writing a web app using Tide (https://github.com/http-rs/tide). I eventually realized that it would be difficult to use the library I had built earlier since Tide uses the async-std runtime rather than Tokio. That was very disappointing. Is there any plan to make it easier to write "runtime agnostic" libraries in the future?
Yes, that is what I alluded to at the end. There's a few points here that still need some interop work. The intention is to fix that, but it's non-trivial. We'll get there.
Steve, just want to say thanks for your dedication - I find you’re always around when there’s a Rust thread on HN
You’re welcome!
You can make libraries runtime agnostic, but it requires a bit of design to get right. We tried our best with Tiberius[0], so you just need to provide an object implementing the AsyncRead and AsyncWrite traits from the futures crate, such as the TcpStream from async-std or tokio (using their compat module).
It is not perfect yet, and especially how tokio does not follow the rest of the ecosystem by implementing their own traits is kind of disappointing. We can work around that, but I was hoping they would fix this by version 1.0...
[0] https://github.com/prisma/tiberius
It is not perfect yet, and especially how tokio does not follow the rest of the ecosystem by implementing their own traits is kind of disappointing. We can work around that, but I was hoping they would fix this by version 1.0...
[0] https://github.com/prisma/tiberius
Yup, building libraries on IO traits which then are implemented by the particular runtimes is a good way to have a runtime agnostic library. Ideally the IO traits are defined in the library itself, to make them not again dependent on another moving target. You can provide implementations of the "glue code" for particular runtimes in separate crates to ease integration for users.
Another way to be runtime agnostic is to start a particular runtime as part of your library, which is used internally. The public interface of your library can provide async functions which are agonstic to a particular runtime, since all actions will be deferred/forwarded to an internal runtime. That approach has a bit more overhead, but can ease usage.
Another way to be runtime agnostic is to start a particular runtime as part of your library, which is used internally. The public interface of your library can provide async functions which are agonstic to a particular runtime, since all actions will be deferred/forwarded to an internal runtime. That approach has a bit more overhead, but can ease usage.
I had the exact same experience. I think it can be a pretty big barrier to getting started, as you really have to lock into a sub-set of the ecosystem (i.e. I can only use crates that have support for Tokio).
I understand the reasoning for not wanting this in the core language, but perhaps there could be some standard implementations which would still allow for custom runtimes.
I understand the reasoning for not wanting this in the core language, but perhaps there could be some standard implementations which would still allow for custom runtimes.
You can make async-std mimic a tokio runtime by adding “tokio02” or “tokio03” to the list of features for async_std. At its core, futures and future combinators work under all runtimes, it’s just some features will complain if they don’t detect the tokio runtime.
that's a good point, when playing a bit with rust last year I found that the libraries that deal with "async stuff" (like http clients, db clients etc) are mostly split, some use tokio and some use async-std, and they were incompatible. Not sure if the situation has improved now but it looked like an ecosystem split at the time.
Work is in progress, but the situation hasn't improved yet for end users.
I knew you needed to use a library like Tokio with asynchronous Rust, but I never understood why, so thanks.
That said, the last time I looked at it, Tokio was much too complex for my tastes.
That said, the last time I looked at it, Tokio was much too complex for my tastes.
As always, very elegant. Thanks for the added info.
You're welcome, thanks :)
If you want to dig deeper:
* https://www.infoq.com/presentations/rust-2019/
* https://www.infoq.com/presentations/rust-async-await/
If you want to dig deeper:
* https://www.infoq.com/presentations/rust-2019/
* https://www.infoq.com/presentations/rust-async-await/
Yeah that's exactly right, it's an event loop.
It's hilarious to me that it's hitting "1.0" now. I remember the original release ~4 years ago?? That was way before Rust even had async/await. I imagine there were quite a few refactors. I mean, so much work, to make a library for asynchronous network service programming? If I needed an event loop and a scheduler I think I would just invest in implementing it myself, tailored to the requirements of my project.
It's hilarious to me that it's hitting "1.0" now. I remember the original release ~4 years ago?? That was way before Rust even had async/await. I imagine there were quite a few refactors. I mean, so much work, to make a library for asynchronous network service programming? If I needed an event loop and a scheduler I think I would just invest in implementing it myself, tailored to the requirements of my project.
Tokio is basically the asynchronous standard I/O library for Rust.
Async/await are language features. Tokio is the library for using these to do I/O.
Rust has a really tiny standard library (by modern standards) and a very high standard for moving stuff into the standard library. Right now it has no "standard" asynchronous I/O library ("async-std" bequeathed that name on themselves; it doesn't ship with Rust).
Async/await are language features. Tokio is the library for using these to do I/O.
Rust has a really tiny standard library (by modern standards) and a very high standard for moving stuff into the standard library. Right now it has no "standard" asynchronous I/O library ("async-std" bequeathed that name on themselves; it doesn't ship with Rust).
tokio is to rust what asyncio is to Python.
except say asyncio is pet of standard python
and tokio is pet of rust... I don’t see the value of this argument. Tokio is known as being THE async executor in rust. Asyncio being THE executor in Python. In other languages there’s a number of frameworks to choose from. In this comparison there really is only one.
And nodejs or web eventloop is to JavaScript. They provide the executor and the timeout/IO primitives.
> rustafarian
The community prefers the term "rustacean" to be as inclusive as possible. Please keep that in mind in the future.
The community prefers the term "rustacean" to be as inclusive as possible. Please keep that in mind in the future.
TIL, thanks for the correction
(It's a combination of not wanting to trivialize a religion, and that 'RESTafarian' has a long history of use, and that's just confusing)
One of the issues I have with the rust async story is that colored functions require (at least the way they’re handled in rust and similar languages) a separate standard library (Microsoft really outdid themselves providing a synchronous and asynchronous version of the BCL when they shipped async support, but that was also largely made possible by the fact that the underlying OS APIs were all fairly asynchronous at the lowest levels and had that asynchronocity exposed/available all the way through the BCL for C# devs already).
One problem with this is that an api like tokio’s might appear to be asynchronous but in reality portions of it are “just” synchronous API calls marshaled to a thread pool - i.e. none of the real benefits of async for now, but positioned so that the library can be switched over to real async code and automatically take all the consumers with “it in the future.”
I’m glad to hear mention of io_uring because it means that IOCP on Windows might get some love. For those that don’t know, on Linux there is^H^H was really no such thing as properly async file system access (eg libc faked it in a similar fashion for aio) so libraries like mio didn’t bother with true async for non-network parts of the library (and also partially because the biggest motivation for async development was the web world which doesn’t particularly care about asynchronously listing the contents of a directory or writing a “highest” performance backup product) - even though at least some platforms (like Windows) had very compelling async options available across the board.
One problem with this is that an api like tokio’s might appear to be asynchronous but in reality portions of it are “just” synchronous API calls marshaled to a thread pool - i.e. none of the real benefits of async for now, but positioned so that the library can be switched over to real async code and automatically take all the consumers with “it in the future.”
I’m glad to hear mention of io_uring because it means that IOCP on Windows might get some love. For those that don’t know, on Linux there is^H^H was really no such thing as properly async file system access (eg libc faked it in a similar fashion for aio) so libraries like mio didn’t bother with true async for non-network parts of the library (and also partially because the biggest motivation for async development was the web world which doesn’t particularly care about asynchronously listing the contents of a directory or writing a “highest” performance backup product) - even though at least some platforms (like Windows) had very compelling async options available across the board.
> For those that don’t know, on Linux there is^H^H was really no such thing as properly async file system access (eg libc faked it in a similar fashion for aio)
That's not quite right - there didn't use to be AIO for buffered filesystem IO and for most operations beyond read/write.
But unbuffered reads/writes have been doable asynchronously for quite a long time, via io_submit/libaio. Without falling back to threads.
The restrictions around that can be onerous (e.g. one needs to be careful to not extend file sizes, or risk falling back to synchronous operation).
That's not quite right - there didn't use to be AIO for buffered filesystem IO and for most operations beyond read/write.
But unbuffered reads/writes have been doable asynchronously for quite a long time, via io_submit/libaio. Without falling back to threads.
The restrictions around that can be onerous (e.g. one needs to be careful to not extend file sizes, or risk falling back to synchronous operation).
[deleted]
> But unbuffered reads/writes have been doable asynchronously for quite a long time
That is not quite right because it is dependent on the filesystem. This is why scylla database requires XFS, because they rely on actual async file io using io_submit.
That is not quite right because it is dependent on the filesystem. This is why scylla database requires XFS, because they rely on actual async file io using io_submit.
> there didn't use to be AIO for buffered filesystem IO
Did they change libaio to work without O_DIRECT at some point? Or are you talking about io_uring for async file io?
Did they change libaio to work without O_DIRECT at some point? Or are you talking about io_uring for async file io?
> Did they change libaio to work without O_DIRECT at some point?
No, and I don't really forsee that happening at this point.
> Or are you talking about io_uring for async file io?
Yep. io_uring can do async buffered file IO.
Initially, for buffered file IO, everything not in the page cache (e.g. a cache miss read, or a write without a page cache page already existing) was done via kernel threads inside the kernel, but that's being incrementally improved. Now most buffered reads don't need a kernel thread anymore (instead they are submitted during the io_uring_enter, and completed in task context, avoiding a lot of the overhead of "synchronous" execution in a kernel thread).
No, and I don't really forsee that happening at this point.
> Or are you talking about io_uring for async file io?
Yep. io_uring can do async buffered file IO.
Initially, for buffered file IO, everything not in the page cache (e.g. a cache miss read, or a write without a page cache page already existing) was done via kernel threads inside the kernel, but that's being incrementally improved. Now most buffered reads don't need a kernel thread anymore (instead they are submitted during the io_uring_enter, and completed in task context, avoiding a lot of the overhead of "synchronous" execution in a kernel thread).
> portions of it are “just” synchronous API calls marshaled to a thread pool
That only happens when OS support is missing. As in, the OS does not support doing something asynchronously.
Once io_uring is implemented, there will be no more of these "portions" (at least not on Linux).
That only happens when OS support is missing. As in, the OS does not support doing something asynchronously.
Once io_uring is implemented, there will be no more of these "portions" (at least not on Linux).
They mention working on using io_uring for filesystem calls for 2021. I wonder if there will be an option to use io_uring (instead of epoll) for networking calls as well? Handling network packets and events completely in userspace should allow for lower latency due to no more context switches to and from the Kernel, right?
We are definitely exploring this as well. There are a few possible ways to move forward on this. I'm not sure which is best yet, but with 1.0 out, we are going to be able to put more time into it.
Both epoll and io_uring depend on the kernel to do the "actual IO", if you want really user-space you need DPDK and a userspace network stack (for TCP/UDP).
Both epoll and io_uring have virtually the same performance. Of course uring is a lot more "ergonomic" that's why it already has amazing momentum.
Both epoll and io_uring have virtually the same performance. Of course uring is a lot more "ergonomic" that's why it already has amazing momentum.
You can do network IO completion notification, instead of readiness notification with io_uring but not epoll. Which avoids the need for a separate syscall that needs to copy memory into the userspace buffers. That can be noticable.
It should also, at some point, allow for nice zero copy network receive paths under the right circumstances (i.e. the network card DMAing directly into the userspace buffers, without very weird setup/high op overhead).
It should also, at some point, allow for nice zero copy network receive paths under the right circumstances (i.e. the network card DMAing directly into the userspace buffers, without very weird setup/high op overhead).
There's some interesting bits on the low-level work for using io_uring with the Ringbahn crate - https://boats.gitlab.io/blog/post/ringbahn/
The author of sled[1], an embedded database in Rust which has a number of promising features, has also written parts of rio[2], an underlying pure Rust io_uring library, which is intended to become the core write path for sled. rio has support for files but also has a demo for TCP (on Linux 5.5 and later) and O_DIRECT.
I tested rio recently as I had a Brilliant but Bad Idea™ involving file access and was pleasantly surprised by the API, as I have been with sled's.
I'm excited for the experimentation in the Rust ecosystem and for such low level crates to handle the complex io_uring tasks (relatively) safely!
[1]: https://github.com/spacejam/sled
[2]: https://github.com/spacejam/rio
I tested rio recently as I had a Brilliant but Bad Idea™ involving file access and was pleasantly surprised by the API, as I have been with sled's.
I'm excited for the experimentation in the Rust ecosystem and for such low level crates to handle the complex io_uring tasks (relatively) safely!
[1]: https://github.com/spacejam/sled
[2]: https://github.com/spacejam/rio
Huge thanks to all the contributors! I've been using Tokio in a few projects and it has been a very good experience. Also thank you all for the welcoming community. I once posted on Tokio's Github about a quirky (in my eyes) behavior of a particular edge case and got an answer almost in real time. This really makes Tokio a kind of a project where I could see myself contributing if an appropriate opportunity arises.
Thanks again! Looking forward to all the good things still in the pipeline!
Thanks again! Looking forward to all the good things still in the pipeline!
The changelog can be found here: https://github.com/tokio-rs/tokio/releases/tag/tokio-1.0.0
I'm hopeful that this leads to some focus on the ergonomics of "waiting for async things from sync code". Lots of "handlers" in the universe have synchronous interfaces, so if you want to implement them you end up needing to poll/wait on async from a regular function. I swear that every time I poke at Rust, I seem to find some way to cut my fingers...
My specific example is writing a fuse handler (now with cberner/fuser formerly zargony/rust-fuse) for GCS/S3. If you want to use make any async requests (like via hyper), you currently have to roll your own poller, like reqwest does in blocking mode [1].
The rust async/.await primer [2] offers the reader the seemingly helpful futures::executor::block_on, but basically no two executors can interact (and for good reason!). As others highlight, the ecosystem seems like it's going to end up standardizing on tokio (and/or some flavor thereof) and that hopefully now that it's 1.0, we can have stable enough deps for a while :).
[1] https://github.com/seanmonstar/reqwest/blob/5ee4fe5ab64a2e3d...
[2] https://rust-lang.github.io/async-book/01_getting_started/04...
My specific example is writing a fuse handler (now with cberner/fuser formerly zargony/rust-fuse) for GCS/S3. If you want to use make any async requests (like via hyper), you currently have to roll your own poller, like reqwest does in blocking mode [1].
The rust async/.await primer [2] offers the reader the seemingly helpful futures::executor::block_on, but basically no two executors can interact (and for good reason!). As others highlight, the ecosystem seems like it's going to end up standardizing on tokio (and/or some flavor thereof) and that hopefully now that it's 1.0, we can have stable enough deps for a while :).
[1] https://github.com/seanmonstar/reqwest/blob/5ee4fe5ab64a2e3d...
[2] https://rust-lang.github.io/async-book/01_getting_started/04...
I've encountered the "wait on async things from sync code" issue several times, too. I have found that something like `block_on` from either `futures` or `futures_lite` often does the trick.
https://docs.rs/futures-lite/1.11.3/futures_lite/future/fn.b...
https://docs.rs/futures-lite/1.11.3/futures_lite/future/fn.b...
Right, but depending on the tokio version (maybe) using block_on results in a hang or panic. I’ll see if futures_lite is any different, but I think it’s mostly on tokio’s side.
> depending on the tokio version (maybe) using block_on results in a hang or panic
Isn't it mostly a function of what runtime you're using? `block_on` with a single-threaded runtime can't run tasks elsewhere (since there's only one thread which you're blocking) so depending on the version it would either hang (before detection of that situation was added) or panic, with an error message saying to use a multithreaded runtime.
I completely agree that it's a pain in the ass though, especially since there are situations where tokio must be coerced into using anything but the basic scheduler (e.g. tests).
Isn't it mostly a function of what runtime you're using? `block_on` with a single-threaded runtime can't run tasks elsewhere (since there's only one thread which you're blocking) so depending on the version it would either hang (before detection of that situation was added) or panic, with an error message saying to use a multithreaded runtime.
I completely agree that it's a pain in the ass though, especially since there are situations where tokio must be coerced into using anything but the basic scheduler (e.g. tests).
Tokio has a version of block_on too that you can use with those libraries.
Today also saw the release of Hyper 0.14, the HTTP client built on top of Tokio [0]
[0]: https://news.ycombinator.com/item?id=25521217
[0]: https://news.ycombinator.com/item?id=25521217
Now hyper-tls and hyper-rustls need updating :).
Congratulations on 1.0! Tokio and the ecosystem surrounding it (tracing, hyper, prost, tower) are incredibly well thought out and a pleasure to use for building fast performant services with solid latencies.
Always found these APIs a little hard to work with. For instance, if I tried to use `actix-web`, then using `reqwest` and `tokio` felt like pulling teeth.
If anyone's got minimal code lining up a web framework (any one, not stuck to actix) with some reqwest, I'd be thankful to look over it. Just some trivial stuff so I can add an API gateway that proxies a specific API.
If anyone's got minimal code lining up a web framework (any one, not stuck to actix) with some reqwest, I'd be thankful to look over it. Just some trivial stuff so I can add an API gateway that proxies a specific API.
I've got a few examples of simple web servers within my company, could backport some of it to a public example.
Are you just looking to have something a bit like:
```
#[get('/todo')]
async fn getTodos(...) -> impl Responder {
let todos = reqwest.get('other-api/todos').await?;
Obviously this code won't run, just want to gleam the gist of what you want from an example.
todosObviously this code won't run, just want to gleam the gist of what you want from an example.
Hey, thank you for offering. Yes! Something minimal like that that doesn't use `reqwest::blocking` would be very helpful.
If it could do the Error case too that would be cool.
For Rocket, I have something like:
So if the example is like:
If it could do the Error case too that would be cool.
For Rocket, I have something like:
#[get("/list/<status>")]
fn list_prospects(api_key: ApiKey, status: String) -> Result<Json<PendingList>, Status>
impl<'a, 'r> FromRequest<'a, 'r> for ApiKey
So if the example is like:
#[get('/todo')]
fn get_todos() -> Result<Json<TodoList>, Status>
let todos = reqwest...
Ok(todos)
For Rocket you want to use the master branch which has async support. Then your request handler is an async function and you can just .await the future returned by reqwest.
Oh, I must be behind. Should've kept up and tried it. Thank you!
The examples in the package docs have always been great starting points for me.
Also, I love the fact that Rust will complain if the examples in your comments don't compile. Such a great feature. As a result, copy-and-pasting examples out of rustdoc pages (nearly) always gives you a working starting point to hack from.
Also, I love the fact that Rust will complain if the examples in your comments don't compile. Such a great feature. As a result, copy-and-pasting examples out of rustdoc pages (nearly) always gives you a working starting point to hack from.
Actix web has an example that is basically just a proxy. It does use its own client vs reqwest but the principals are the same.
Congrats! I've fallen off Rust due to pivoting at work plus trying other languages but I'm intrigued again by Tokio's announcement. I'll be trying out your tutorial soon!
what did you pivot to and why, if you don't mind me asking?
Your comment was "dead", but I see nothing wrong with it and vouched for it. This is a valid question asked earnestly.
I'd honestly like to know too.
I'd honestly like to know too.
Can't speak for the commenter, but I used to work for a team couple years ago that used rust and tokio extensively, and some projects were just not a good fit. At the time, futures were well fleshed out but the community hadn't caught up, so we were lacking a futures-compatible postgres and redis client. We wrote the redis client ourselves, and for the main project using rust that was sufficient. But for postgres, that was a show stopper for any other projects we were working on. So we ended up deciding between typescript and go for those.
Just curious, did you consider wrapping the existing sync clients with something like spawn_blocking[1]? If so, what tradeoffs did you find?
[1]: https://docs.rs/tokio/1.0.0/tokio/task/fn.spawn_blocking.htm...
[1]: https://docs.rs/tokio/1.0.0/tokio/task/fn.spawn_blocking.htm...
The ecosystem has moved forward a lot since then. Pretty much all IO libraries provide an asynchronous API now.
Note: now tokio-postgres works fine.
The tokio tutorial is good, but it needs an example of spinning off multiple (2-3 or more) tasks which run forever. I ended up using spawn() multiple times and having the main thread just sleep in a loop and not using #[tokio::main] because I couldn't figure it out.
It is unfortunate, that libraries have to be coded against specific runtime and not generically. There is tokio and there is smol (likely discontinued, since author left rust), maybe other runtimes will emerge, but whole ecosystem is already tied to tokio.
https://github.com/stjepang?tab=overview&from=2020-12-01&to=...
"""smol (likely discontinued, since author left rust)""" doesn't seem well supported and smol is used by async-rs these days iirc.
"""smol (likely discontinued, since author left rust)""" doesn't seem well supported and smol is used by async-rs these days iirc.
[deleted]
async-std [0] is pretty widely used as well.
[0]: https://github.com/async-rs/async-std
Most libraries can be used with different runtimes. Hyper for example, which uses Tokio by default, can be configured to use an async-std executor.
[0]: https://github.com/async-rs/async-std
Most libraries can be used with different runtimes. Hyper for example, which uses Tokio by default, can be configured to use an async-std executor.
It bugs me that that library will spin up a scheduler without being asked to do so.
As I understand it, that difference (vs Tokio) was the main driver behind the projects splitting.
I also think it's a bit presumptuous for them to name themselves "std". It'll be even more ridiculous in the likely event that Tokio becomes the std:: asynchronous I/O library. It's asking for confusion.
As I understand it, that difference (vs Tokio) was the main driver behind the projects splitting.
I also think it's a bit presumptuous for them to name themselves "std". It'll be even more ridiculous in the likely event that Tokio becomes the std:: asynchronous I/O library. It's asking for confusion.
[deleted]
I always get a weird vibe from async-std. I respect the people working on it, but it feels like it's trying to boil the ocean.
I'd be very interested in hearing other opinions, as my Rust project [1] is currently stuck on an older version of Tokio while I wait for deps to update. I'm either going to have to bite the bullet and replace deps or bite more bullets and find a different runtime.
[1] https://github.com/mmastrac/stylus/
I'd be very interested in hearing other opinions, as my Rust project [1] is currently stuck on an older version of Tokio while I wait for deps to update. I'm either going to have to bite the bullet and replace deps or bite more bullets and find a different runtime.
[1] https://github.com/mmastrac/stylus/
Can't find async-std feature here: https://github.com/hyperium/hyper/blob/master/Cargo.toml
do you have an example?
Either way, "can be configured" means that custom code for each runtime must be written, it is not like lets say "Futures", which can be used generically.
do you have an example?
Either way, "can be configured" means that custom code for each runtime must be written, it is not like lets say "Futures", which can be used generically.
Here is an example: https://github.com/async-rs/async-std-hyper/blob/master/READ...
You do have to write a ~50 loc compat layer. However, most of the compat layer is due to the fact that tokio's `AsyncRead` and `AsyncWrite` are different from the standard futures crate, which may change in the future [0]. After that, you just have to implement `hyper::Executor` for async-std's `spawn`, and `hyper::Accept` for async-std's `TcpListener`.
Of course, it is not as generic as "Futures", but it is relatively simple to do. As @steveklabnik mentioned above:
> There's a few points here that still need some interop work. The intention is to fix that, but it's non-trivial. We'll get there.
[0]: https://github.com/tokio-rs/tokio/issues/2716
You do have to write a ~50 loc compat layer. However, most of the compat layer is due to the fact that tokio's `AsyncRead` and `AsyncWrite` are different from the standard futures crate, which may change in the future [0]. After that, you just have to implement `hyper::Executor` for async-std's `spawn`, and `hyper::Accept` for async-std's `TcpListener`.
Of course, it is not as generic as "Futures", but it is relatively simple to do. As @steveklabnik mentioned above:
> There's a few points here that still need some interop work. The intention is to fix that, but it's non-trivial. We'll get there.
[0]: https://github.com/tokio-rs/tokio/issues/2716
> tokio's `AsyncRead` and `AsyncWrite` are different from the standard futures
The standard futures library does not have an AsyncRead or AsyncWrite:
https://doc.rust-lang.org/std/future/index.html
The standard futures library does not have an AsyncRead or AsyncWrite:
https://doc.rust-lang.org/std/future/index.html
Probably in reference to
https://docs.rs/futures/0.3.8/futures/io/trait.AsyncRead.htm...
vs
https://docs.rs/tokio/1.0.0/tokio/io/trait.AsyncRead.html
https://docs.rs/futures/0.3.8/futures/io/trait.AsyncRead.htm...
vs
https://docs.rs/tokio/1.0.0/tokio/io/trait.AsyncRead.html
Yup, that is what I was referring to.
[deleted]
Since when did Stjepang leave Rust?
He switched to Go.
Stjepang is a very smart person and I'm actually quite excited about what he's going to bring to the Go world.
Stjepang is a very smart person and I'm actually quite excited about what he's going to bring to the Go world.
Was there a blogpost? Or when/where was this announced?
Rust/Go really are the Emacs/Vim of programming languages: little in common but commonly mentioned in the same breath.
he posted a goodbye issue on a smol repo, but since then deleted it, so maybe he changed his mind
I found the github issue in Google cache. I'm not sure it's really fair of me to post this link here, but equally I think it's better to give the actual text rather than leave it vague.
https://webcache.googleusercontent.com/search?q=cache:PRjMyv...
https://webcache.googleusercontent.com/search?q=cache:PRjMyv...
Thanks for the pointer! It's really sad to hear this. I'm using smol in my project and really like it. I didn't know the author left until now. It's a great loss for Rust, in my opinion.
> and there is smol (likely discontinued, since author left rust)
Whoa. smol is great. Why’d the author leave?
Whoa. smol is great. Why’d the author leave?
That's not entirely true. If you want to write an entire standalone application that compiles into a binary and starts up a scheduler, then yes, you have to pick a scheduler runtime.
If you're writing a library to be used by others you can very often expose only types which come from std::futures. The result will work with all of the runtimes.
If you're writing a library to be used by others you can very often expose only types which come from std::futures. The result will work with all of the runtimes.
Woah, I didn't know about that. Does this mean Rust only has one maintained runtime now? Namely Tokio.
Async-std is still actively maintained and developed.
I think it's rather that smol is "complete" and doesn't need much maintenance.
Maybe that's true. I don't know how complex these things are, or if they will be affected by future changes in the language, but it feels weird to say that a critical piece of software is complete. I wonder if many new libraries will use Smol.
One of the principles that stjepang was very keen on for smol was that libraries shouldn't depend on it. Libraries should be written in an async-executor generic way and the end binary should then be free to use smol as the executor, or tokio, or anything else.
I'm currently using smol (in an application I'm developing - not just a library) and I don't think this will scare me away. He appears to still be responding to pull requests, and smol consists of a reasonably small amount of very high quality code.
I'm currently using smol (in an application I'm developing - not just a library) and I don't think this will scare me away. He appears to still be responding to pull requests, and smol consists of a reasonably small amount of very high quality code.
If core rust provides an interface for async/await why does usage of it tie you to a specific runtime? Whats the point of that?
Usage of it, on its own, does not tie you to a specific runtime.
Using runtime-specific features and APIs ties you to a specific runtime.
Using runtime-specific features and APIs ties you to a specific runtime.
Is this just an artifact of the current work in progress state of things? Is it just a matter of the interface settling and library authors providing a bit more configuration options to become runtime agnostic?
I am not a super expert in all of the exact details, but my understanding is that generally, there are a few features that don't have a common trait yet. For example, "spawn me a new task." Once those traits exist, libraries can be written against them, and it'll all be agnostic. But you only even run into that kind of problem if you need that specific API in the first place.
I wonder whether there is any ongoing effort to unify the ecosystem between different runtimes. Specially considering that Tokio and futures (by extension async-std) implement their own Async* traits, it seems that it becomes even harder to write runtime agnostic libraries. It would be nice if these fundamental traits were part of rust std library.
This is mentioned downthread, but yes, the intention is still for that to happen. There's more work to be done here.
Congrats to the Tokio team! This is a big deal for the Rust ecosystem.
It's getting close to the point where I can't make my "semver just means prefixing your version number with `0.`" joke anymore. :<
Congrats to the tokio team + contributors. <:)
Congrats to the tokio team + contributors. <:)
Cool stuff. There seems to be some cool improvement every several months.
I'm excited for GATs to land so we can have true async trait methods. The async story in Rust has come a long way, but there's still a lot to be improved.
I'm excited for GATs to land so we can have true async trait methods. The async story in Rust has come a long way, but there's still a lot to be improved.
There seems to be some time loop with network application concurrency approaches. I began my career re-writing servers that had a non-blocking socket model with a state machine to use threads, so we could understand and maintain the code now that OS vendors had come around to supporting threading widely. Fast forward a few decades and the world is teeming with smart people who seem to think it's a good idea to go the other way..
What you were looking though was a syntax to program those things rather than the specific implementation.
[deleted]
This is awesome, congrats on the 1.0!
As a new rustacean it was really difficult to try and use the the async/await syntax with horde of adapters required for tokio 0.1/0.2. I ended up trying out async-std too, but the move to smol had its own issues and I lost interest.
A 1.0 release with stability commitments is exactly what I need to get back into experimenting with Rust.
As a new rustacean it was really difficult to try and use the the async/await syntax with horde of adapters required for tokio 0.1/0.2. I ended up trying out async-std too, but the move to smol had its own issues and I lost interest.
A 1.0 release with stability commitments is exactly what I need to get back into experimenting with Rust.
I see a lot of hate for what amounts to user controlled and scheduled tasks. Do people understand that async/await is effectively user land task scheduling, which for any high performance networking app is of vital importance. Consider cassandra versus scylladb
I have known Tokio for ages and I'm not even a Rust programmer. It seems like a solid project.
> Also, Tokio would not have been possible without Aaron Turon's and Alex Crichton's early involvement.
This feels like a slap in the face for some reason.
Aaron Turon is an extremely talented individual (their PhD thesis was a landmark contributionn). They are super kind and one of the nicest human beings I've ever met. They led the Rust Project until their involvement with Tokio caused them to drop off from Tech.
Alex Crichton is an extremely talented, kind, and hyperproductive individual, who after their involvement with Tokio dropped all async/await work and luckily "refocused" on WebAssembly.
If one is going to recap the road towards Tokio 1.0 and mention all the people that have left the Rust async ecosystem or Rust all together, you might as well spell things out.
This feels like a slap in the face for some reason.
Aaron Turon is an extremely talented individual (their PhD thesis was a landmark contributionn). They are super kind and one of the nicest human beings I've ever met. They led the Rust Project until their involvement with Tokio caused them to drop off from Tech.
Alex Crichton is an extremely talented, kind, and hyperproductive individual, who after their involvement with Tokio dropped all async/await work and luckily "refocused" on WebAssembly.
If one is going to recap the road towards Tokio 1.0 and mention all the people that have left the Rust async ecosystem or Rust all together, you might as well spell things out.
What?
Why did they drop from tech?
Are you sure this isn't something else instead?
Are you sure this isn't something else instead?
I find the domain amusing: https://en.m.wikipedia.org/wiki/Srbija_do_Tokija
Is it a concern that both Tokio and stdlib have AsyncRead+AsyncWrite traits that seem to be incompatible?
Yes, it is a concern, but Tokio people decided transition can happen without breaking changes hence it does not block 1.0. See https://github.com/tokio-rs/tokio/issues/2716 for details.
Thank you!
Rust doesn't have AsyncRead+AsyncWrite in std. You may be thinking about the third-party futures crate though, which does have a currently incompatible implementation.
Can someone tell me why tokio is so damn big and full of transitive dependencies :D?
You could also say that Tokio has a large and mature ecosystem surrounding it.
How big is it?
Here are the compiled-in dependencies:
PS, I think there's a bug in "cargo tree"... the command above actually prints out only one line ("pin-project"). I had to remove the "-p tokio" and then copy-and-paste out the relevant section.
$ cargo tree -e no-dev,no-build --no-dedupe -p tokio
tokio v1.0.0
├── bytes v1.0.0
├── libc v0.2.81
├── memchr v2.3.4
├── mio v0.7.6
│ ├── libc v0.2.81
│ └── log v0.4.11
│ └── cfg-if v0.1.10
├── num_cpus v1.13.0
│ └── libc v0.2.81
├── once_cell v1.5.2
├── parking_lot v0.11.1
│ ├── instant v0.1.9
│ │ └── cfg-if v1.0.0
│ ├── lock_api v0.4.2
│ │ └── scopeguard v1.1.0
│ └── parking_lot_core v0.8.2
│ ├── cfg-if v1.0.0
│ ├── instant v0.1.9 (*)
│ ├── libc v0.2.81
│ └── smallvec v1.4.2
├── pin-project-lite v0.2.0
└── signal-hook-registry v1.3.0
└── libc v0.2.81
PS, I think there's a bug in "cargo tree"... the command above actually prints out only one line ("pin-project"). I had to remove the "-p tokio" and then copy-and-paste out the relevant section.
Most of that is parking_lot, which is an optional feature not even included in the full feature.
the dephell report: https://gallant-ride-cab667.netlify.app/
It irks me that the "async runtime" isn't simply part of the rust runtime. Making it be a separate library simply increases the likelihood of having to deal with libraries expecting different version, or even different async runtimes entirely.
Rust doesn't have a runtime.
That's part of its awesomeness. That's why it can target microcontrollers.
You're probably used to languages with garbage collectors. Having garbage collection forces you to have a "runtime" since that's where the GC code goes. Then more and more stuff accretes onto this unavoidable runtime, and before you know it you're writing Java code...
That's part of its awesomeness. That's why it can target microcontrollers.
You're probably used to languages with garbage collectors. Having garbage collection forces you to have a "runtime" since that's where the GC code goes. Then more and more stuff accretes onto this unavoidable runtime, and before you know it you're writing Java code...
Um, rust does have a runtime, that's why you don't need to include packages for strings, refcounting, allocation, etc.
Anything that has a language keyword should have a default implementation in that runtime, and much like box, ?, !, etc async and await are both language features that I shouldn't need to include from outside of the runtime.
Anything that has a language keyword should have a default implementation in that runtime, and much like box, ?, !, etc async and await are both language features that I shouldn't need to include from outside of the runtime.
> you don't need to include packages for strings, refcounting, allocation, etc.
Oh, but you do!
These are in std::str, std::rc, and std::alloc, respectively. You're absolutely able to choose not to include these packages, with #![no_std]. That's how rust is able to target platforms like 8-bit microcontrollers with 16kbytes of RAM.
https://rust-embedded.github.io/book/intro/no-std.html
Oh, but you do!
These are in std::str, std::rc, and std::alloc, respectively. You're absolutely able to choose not to include these packages, with #![no_std]. That's how rust is able to target platforms like 8-bit microcontrollers with 16kbytes of RAM.
https://rust-embedded.github.io/book/intro/no-std.html
What the OP is probably referring to is that Rust does not have a runtime in the typical sense used by languages such as Java. Rust does have a runtime, as all non-assembly languages do, but it is very very small. Languages with minimal runtimes like C or Rust are commonly referred to as having "no runtime".
Those are all pay-as-you go features; in my mind I associate runtime with a single, constant-ish initialization cost, plus possibly some background processing or inserted hooks that do janitor work for you. Rust doesn't have a runtime in that sense.
I think you are confusing a standard library with a runtime.
A runtime is something you need to start before being able to use a language (typically a garbage collector, an event loop, a threadpool or a VM)
A runtime is something you need to start before being able to use a language (typically a garbage collector, an event loop, a threadpool or a VM)
This is awesome, thanks to all contributors to this incredible tool!
What would be Tokios equivalent in .net/C# ?
In .NET you have the low-level runtime machinery implemented in the CLR (Common Language Runtime), but the transformation from async-await to state machine code is done completely at compile time.
Basically, just like C# (and VB.NET and C++ .NET task/then) provides syntax and semantics for async-await, Rust provides it at language level too. (And it defines how the compiler transforms it into Future objects.)
But, since Rust doesn't have a mandatory runtime, something needs to implement the low-level stuff that knows what to do with these Future objects. (In Tokio you have a work-stealing threadpool, but maybe in smaller runtimes you don't need all that fancy stuff for high-throughput, you just need small binary size, so there's a runtime/library called "smol" that's main feature is that it's a small async runtime.) In the CLR as far as I know there are Task objects, which basically correspond to Rust's Future objects.
One interesting low-level difference (similarity?) is that in the CLR there's an explicit callback support by the runtime (to wake up Task objects - which can lead to deadlocks if they are scheduled on the UI thread), whereas in Rust Futures pass their own callbacks (called Waker) to a thing called the Reactor (which is basically the low-level implementation of the Executor, which binds to the OS/kernel level primitives, such as epoll or IOCP).
And even though it's a "zero cost" abstraction, it still means there's a state machine, just like in .NET. Except it's built and "deadlock checked" at compile time.
https://tooslowexception.com/wp-content/uploads/2020/05/ther...
https://www.red-gate.com/simple-talk/dotnet/net-framework/th...
Basically, just like C# (and VB.NET and C++ .NET task/then) provides syntax and semantics for async-await, Rust provides it at language level too. (And it defines how the compiler transforms it into Future objects.)
But, since Rust doesn't have a mandatory runtime, something needs to implement the low-level stuff that knows what to do with these Future objects. (In Tokio you have a work-stealing threadpool, but maybe in smaller runtimes you don't need all that fancy stuff for high-throughput, you just need small binary size, so there's a runtime/library called "smol" that's main feature is that it's a small async runtime.) In the CLR as far as I know there are Task objects, which basically correspond to Rust's Future objects.
One interesting low-level difference (similarity?) is that in the CLR there's an explicit callback support by the runtime (to wake up Task objects - which can lead to deadlocks if they are scheduled on the UI thread), whereas in Rust Futures pass their own callbacks (called Waker) to a thing called the Reactor (which is basically the low-level implementation of the Executor, which binds to the OS/kernel level primitives, such as epoll or IOCP).
And even though it's a "zero cost" abstraction, it still means there's a state machine, just like in .NET. Except it's built and "deadlock checked" at compile time.
https://tooslowexception.com/wp-content/uploads/2020/05/ther...
https://www.red-gate.com/simple-talk/dotnet/net-framework/th...
The direct equivalent is the async runtime part of the .net/C# runtime, that is, it's built-in there, but is a library here.
C#/.NET has all of this already. You don't have to import anything else to use async/await.
[deleted: man, was that a dumb comment]
Of course it does just because C# comes with all the bells and whistles doesn't mean you can't write your own in it. https://docs.microsoft.com/en-us/dotnet/api/system.threading...
No I think you're mistaken. You can rewrite pretty much every part of the system. You can write your own synchronization, you can write your own scheduling, you can write custom awaiter implementations. Its very pluggable.
You might be able to argue that its even more pluggable than Tokio because the system has the concept of current context and attaching a task to it. Library code can use your custom scheduler.
You can even throw away the Task type and create your own future type that works with the async/await syntax but of course no library would be able to pick that up.
You might be able to argue that its even more pluggable than Tokio because the system has the concept of current context and attaching a task to it. Library code can use your custom scheduler.
You can even throw away the Task type and create your own future type that works with the async/await syntax but of course no library would be able to pick that up.
[deleted]
I really like writing rust, however not having the async runtime be a part of std has hurt the language in my opinion.
Crate consumers have to consider which async lib a crate is using leads to a lot of annoying gotchas that can really confuse less experienced rust devs.
I'm hoping tokio becomes the default and eventually gets merged into std, it really is the best async implementation.
Crate consumers have to consider which async lib a crate is using leads to a lot of annoying gotchas that can really confuse less experienced rust devs.
I'm hoping tokio becomes the default and eventually gets merged into std, it really is the best async implementation.
The first two paragraphs have a lot of guarantees. About stability of version 1.0, length of support for version 1.0 and how long until version 2.0 (at least). "We
"we are committing to providing a stable foundation..."
I am curious: Who is "we"? I have no priors, I really have no idea.
"we are committing to providing a stable foundation..."
I am curious: Who is "we"? I have no priors, I really have no idea.
The tokio maintainers and contributors
Well obviously, but that is not really an answer.
Perhaps it is not answerable but it is a important question, when guarantees are made, who is making them?
I am not sure it is a important question in that it is not a important guarantee. The software is what it is, it is open and modifiable. But if the guarantee mattered then this would be a crucial question and the answer would describe the organisational structure of the "tokio maintainers and contributors" as a group
Perhaps it is not answerable but it is a important question, when guarantees are made, who is making them?
I am not sure it is a important question in that it is not a important guarantee. The software is what it is, it is open and modifiable. But if the guarantee mattered then this would be a crucial question and the answer would describe the organisational structure of the "tokio maintainers and contributors" as a group
"we" here refers to the Tokio team.
But in Rust you can just spawn threads, share data through channels or mutexes, use OS-provided async IO primitives to poll file descriptors and do event-driven programming etc...
I tried looking into Tokio a little while ago and I found that it led to some incredibly complicated, abstracted, hard to think about code for stuff I usually implement (IMO) much more simply with a basic event loop and non-blocking IO for instance.
I'm sure async can get the upper hand when you have a huge number of very small tasks running concurrently because that's generally where OS-driven parallelism tends to suffer but is it such a common scenario? That sounds like premature optimization in many situations IMO. Unless you actually think that async code is more expressive and easy to read and write than basic event loops, but then you must be a lot smarter than I am because I have to take an aspirin every time I need to dig into async-heavy code.
I guess I'm trying to understand if it's me who's missing something, or if it's web-oriented developers who are trying to bring their favourite type of proverbial hammer to system development.