Zig's New Async I/O(kristoff.it)
kristoff.it
Zig's New Async I/O
https://kristoff.it/blog/zig-new-async-io/
289 comments
The key difference to typical async function coloring is that `Io` isn't something you need specifically for asynchronicity; it's something which (unless you make a point to reach into very low-level primitives) you will need in order to perform any IO, including reading a file, sleeping, getting the time, etc. It's also just a value which you can keep wherever you want, rather than a special attribute/property of a function. In practice, these properties solve the coloring problem:
* It's quite rare for a function to unexpectedly gain a dependency on "doing IO" in general. In practice, most of your codebase will have access to an `Io`, and only leaf functions doing pure computation will not need them.
* If a function does start needing to do IO, it almost certainly doesn't need to actually take it as a parameter. As in many languages, it's typical in Zig code to have one type which manages a bunch of core state, and which the whole codebase has easy access to (e.g. in the Zig compiler itself, this is the `Compilation` type). Because of this, despite the perception, Zig code doesn't usually pass (for instance) allocators explicitly all the way down the function call graph! Instead, your "general purpose allocator" is available on that "application state" type, so you can fetch it from essentially wherever. IO will work just the same in practice. So, if you discover that a code path you previously thought was pure actually does need to perform IO, then you don't need to apply some nasty viral change; you just grab `my_thing.io`.
I do agree that in principle, there's still a form of function coloring going on. Arguably, our solution to the problem is just to color every function async-colored (by giving most or all of them access to an `Io`). But it's much like the "coloring" of having to pass `Allocator`s around: it's not a problem in practice, because you'll basically always have easy access to one even if you didn't previously think you'd need it. I think seasoned Zig developers will pretty invariably agree with the statement that explicitly passing `Allocator`s around really does not introduce function coloring annoyances in practice, and I see no reason that `Io` would be particularly different.
* It's quite rare for a function to unexpectedly gain a dependency on "doing IO" in general. In practice, most of your codebase will have access to an `Io`, and only leaf functions doing pure computation will not need them.
* If a function does start needing to do IO, it almost certainly doesn't need to actually take it as a parameter. As in many languages, it's typical in Zig code to have one type which manages a bunch of core state, and which the whole codebase has easy access to (e.g. in the Zig compiler itself, this is the `Compilation` type). Because of this, despite the perception, Zig code doesn't usually pass (for instance) allocators explicitly all the way down the function call graph! Instead, your "general purpose allocator" is available on that "application state" type, so you can fetch it from essentially wherever. IO will work just the same in practice. So, if you discover that a code path you previously thought was pure actually does need to perform IO, then you don't need to apply some nasty viral change; you just grab `my_thing.io`.
I do agree that in principle, there's still a form of function coloring going on. Arguably, our solution to the problem is just to color every function async-colored (by giving most or all of them access to an `Io`). But it's much like the "coloring" of having to pass `Allocator`s around: it's not a problem in practice, because you'll basically always have easy access to one even if you didn't previously think you'd need it. I think seasoned Zig developers will pretty invariably agree with the statement that explicitly passing `Allocator`s around really does not introduce function coloring annoyances in practice, and I see no reason that `Io` would be particularly different.
> It's quite rare for a function to unexpectedly gain a dependency on ...
If this was true in general, the function coloring problem wouldn't be talked about.
However, the second point is more interesting. I think there's a bit of a Stockholm syndrome thing here with Zig programmers and Allocator. It's likely that Zig programmers won't mind passing around an extra param.
If anything, it would make sense to me to have IO contain an allocator too. Allocation is a kind of IO too. But I guess it's going to be 2 params from now on.
If this was true in general, the function coloring problem wouldn't be talked about.
However, the second point is more interesting. I think there's a bit of a Stockholm syndrome thing here with Zig programmers and Allocator. It's likely that Zig programmers won't mind passing around an extra param.
If anything, it would make sense to me to have IO contain an allocator too. Allocation is a kind of IO too. But I guess it's going to be 2 params from now on.
> If anything, it would make sense to me to have IO contain an allocator too. Allocation is a kind of IO too.
Io in zig is for “things that can block execution”. Things that could semantically cause a yield of any kind. Allocation is not one of those things.
Also, it’s perfectly reasonable and sometimes desireable to have 13 different allocators in your program at once. Short lived ones, long lived ones, temporary allocations, super specific allocators to optimize some areas of your game…
There are fewer reasons to want 2 different strategies to handle concurrency at the same time in your program as they could end up deadlocking on each other. Sure, you may want one in debug builds, another in release, another when running tests, but there are much fewer usecases of them running side by side.
Io in zig is for “things that can block execution”. Things that could semantically cause a yield of any kind. Allocation is not one of those things.
Also, it’s perfectly reasonable and sometimes desireable to have 13 different allocators in your program at once. Short lived ones, long lived ones, temporary allocations, super specific allocators to optimize some areas of your game…
There are fewer reasons to want 2 different strategies to handle concurrency at the same time in your program as they could end up deadlocking on each other. Sure, you may want one in debug builds, another in release, another when running tests, but there are much fewer usecases of them running side by side.
> Io in zig is for “things that can block execution”. Things that could semantically cause a yield of any kind. Allocation is not one of those things.
The allocator may yield to the OS when requesting or releasing memory (e.g. sbrk, mmap, munmap)?
The allocator may yield to the OS when requesting or releasing memory (e.g. sbrk, mmap, munmap)?
I don't find that a particularly compelling argument in this case, because so can accessing any memory address if it's not currently swapped in.
Yielding in this context means to a different “thread” in your context, not the OS. If you want to express “this is a point where the program can do something else” it is a yield. If you block and can’t switch to something else… it is not.
So if you’re using an API like mmap like that you should think of it as IO (I don’t think you can, but am not sure).
So if you’re using an API like mmap like that you should think of it as IO (I don’t think you can, but am not sure).
the page allocator which is the root of many allocators calls mmap. of course the fixed buffer allocator does not.
[deleted]
Stockholm syndrome? Many years ago, I was specifically wanting a programming language where I could specify an allocator as a parameter! That's one of Zig's selling points.
> But I guess it's going to be 2 params from now on.
>> So, if you discover that a code path you previously thought was pure actually does need to perform IO, then you don't need to apply some nasty viral change; you just grab `my_thing.io
>> So, if you discover that a code path you previously thought was pure actually does need to perform IO, then you don't need to apply some nasty viral change; you just grab `my_thing.io
Python, for example, will let you call async functions inside non-async functions, you just have to set up the event loop yourself. This isn't conceptually different than the Io thing here.
But the asyncio event loop is not reentrant, so your faux sync functions cannot be safely called from other async functions. It is an extremely leaky abstraction. This is not a theoretical possibility, I stumbled on the issue 15 minutes into my foray into asyncio (turns out that jupyter notebooks use asyncio internally).
There are ways around that (spawn a separate thread with a dedicated event loop, then block), or monkey patch asyncio, but they are all ugly.
There are ways around that (spawn a separate thread with a dedicated event loop, then block), or monkey patch asyncio, but they are all ugly.
except you cant "pass the same event loop in multiple locations". its also not an easy lift. the zig std will provide a few standard implementations which would be trivial to drop in.
I thought it was the exact same; an event loop in Python is just whatever Io is in Zig, make it a param, get it from an import and a lookup (`import asyncio; loop = asyncio.get_running_loop()`). I might be misunderstanding what you're saying though.
hm maybe. i guess ive only used python in situations where it injects it into amain so i could have been confused. i thought python async was a wrapper around generators, and so the mechanism cant be instantiated in multiple places. i stand corrected.
Oh no you're right it's a pain in the ass and weird. But I think it's the way because there's no good reason to have more than one event loop.
Also... maybe it started out as generator wrappers? I think I read something that said that.
Also... maybe it started out as generator wrappers? I think I read something that said that.
> no good reason to have more than one event loop
Per thread—once you start working in multiple threads you have the choice to have one global event loop, which comes at the cost of all async code being effectively serialized as far as threads are concerned*, or one event loop per thread.
* Which can be fine if your program is mostly not async but you have that one stubborn library. Yay async virality.
Per thread—once you start working in multiple threads you have the choice to have one global event loop, which comes at the cost of all async code being effectively serialized as far as threads are concerned*, or one event loop per thread.
* Which can be fine if your program is mostly not async but you have that one stubborn library. Yay async virality.
I can't say there's no good reason to have per thread event loops, but I think I can say if you do know of one you're suffering a terrible curse. I can only imagine the constraints that would force me to do this.
Because you have a main application running a web server and a daemon worker thread performing potentially long running tasks that use async libraries and you don't want to block the responsiveness of your web server. It's really not that bad, at least in Python.
[deleted]
I do something like that with event driven firmware. There is an allocator as part of the context. And the idea that the function is executing under some context seems fine to me.
> I do agree that in principle, there's still a form of function coloring going on. Arguably, our solution to the problem is just to color every function async-colored
I feel like there are two issues with this approach:
- you basically rely on the compiler/stdlib to silently switch the async implementation, effectively implementing a sort of hidden control flow which IMO doesn't really fit Zig
- this only solves the "visible" coloring issue of async vs non-async functions, but does not try to handle the issue of blocking vs non-blocking functions, rather it hides it by making all functions have the same color
- you're limiting the set of async operations to the ones supported in the `Io`'s vtable. This forces it to e.g. include mutexes, even though they are not really I/O, because they might block and hence need async support. But if I wrote my own channel how would this design support it?
I feel like there are two issues with this approach:
- you basically rely on the compiler/stdlib to silently switch the async implementation, effectively implementing a sort of hidden control flow which IMO doesn't really fit Zig
- this only solves the "visible" coloring issue of async vs non-async functions, but does not try to handle the issue of blocking vs non-blocking functions, rather it hides it by making all functions have the same color
- you're limiting the set of async operations to the ones supported in the `Io`'s vtable. This forces it to e.g. include mutexes, even though they are not really I/O, because they might block and hence need async support. But if I wrote my own channel how would this design support it?
Colouring every function async-coloured by default is something that's been attempted in the past; it was called "threads".
The innovation of async over threads is simply to allocate call stack frames on the heap, in linked lists or linked DAGs instead of fixed-size chunks. This sounds inefficient, and it is: indexing a fixed block of memory is much cheaper. It comes with many advantages as well: each "thread" only occupies the amount of memory it actually uses, so you can have a lot more of them; you can have non-linear graphs, like one function that calls two functions at the same time; and by reinventing threading from scratch you avoid a lot of thread-local overhead in libraries because they don't know about your new kind of threads yet. Because it's inefficient (and because for some reason we run the new threading system on top of the old threading system), it also became useful to run CPU-bound functions in the old kind of stack.
If you keep the linked heap activation records but don't colour functions, you might end up with Go, which already does this. Go can handle a large number of goroutines because they use linked activation records (but in chunks, so that not every function call allocates) and every Go function uses them so there is no colour.
You do lose advantages that are specific to coloured async - knowing that a context switch will not occur inside certain function calls.
As usual, we're beating rock with paper in the moment, declaring that paper is clearly the superior strategy, and missing the bigger picture.
The innovation of async over threads is simply to allocate call stack frames on the heap, in linked lists or linked DAGs instead of fixed-size chunks. This sounds inefficient, and it is: indexing a fixed block of memory is much cheaper. It comes with many advantages as well: each "thread" only occupies the amount of memory it actually uses, so you can have a lot more of them; you can have non-linear graphs, like one function that calls two functions at the same time; and by reinventing threading from scratch you avoid a lot of thread-local overhead in libraries because they don't know about your new kind of threads yet. Because it's inefficient (and because for some reason we run the new threading system on top of the old threading system), it also became useful to run CPU-bound functions in the old kind of stack.
If you keep the linked heap activation records but don't colour functions, you might end up with Go, which already does this. Go can handle a large number of goroutines because they use linked activation records (but in chunks, so that not every function call allocates) and every Go function uses them so there is no colour.
You do lose advantages that are specific to coloured async - knowing that a context switch will not occur inside certain function calls.
As usual, we're beating rock with paper in the moment, declaring that paper is clearly the superior strategy, and missing the bigger picture.
And threads are just 100x better than stackless coroutines, period.
Especially when we have actual threads with UMCG.
Especially when we have actual threads with UMCG.
> Arguably, our solution to the problem is just to color every function async-colored.
This is essentially how Golang achived color-blindness.
This is essentially how Golang achived color-blindness.
It's basically how Java does it (circa 17) as well.
It's something you really can't do without a pretty significant language runtime. You also really need people working within your runtime to prefer being in your runtime. Environments that do a lot of FFI don't work well with a colorblind runtime. That's because if the little C library you call does IO then you've got an incongruous interaction that you need to worry about.
It's something you really can't do without a pretty significant language runtime. You also really need people working within your runtime to prefer being in your runtime. Environments that do a lot of FFI don't work well with a colorblind runtime. That's because if the little C library you call does IO then you've got an incongruous interaction that you need to worry about.
Neither Java nor Go make all functions async. What they provide is stackful coroutines (or equivalently one shot continuations) that allow composing async and sync functions transparently.
Golang functions are synchronous but all goroutine code is essentially async or “coloured”.
Golang isn't color-blind. The magic of async/await isn't that the program isn't blocked, it's that the CALLER doesn't have to be blocked. It gives the caller the flexibility to continue and synchronize at its discretion.
In Golang to avoid blocking the CALLER you'd still have to wrap the call in a Goroutine and use something like a channel(or shared mem) to communicate back to the caller.
Guess what ends up happening IRL? People create a set of functions that return channels, and a set of functions that don't for maximum flexibility. Two colors.
And that's viral much like async/await. You block on that channel? Now your caller needs to wrap you in a Goroutine. Or you have to return the/a channel. etc etc etc.
In Golang to avoid blocking the CALLER you'd still have to wrap the call in a Goroutine and use something like a channel(or shared mem) to communicate back to the caller.
Guess what ends up happening IRL? People create a set of functions that return channels, and a set of functions that don't for maximum flexibility. Two colors.
And that's viral much like async/await. You block on that channel? Now your caller needs to wrap you in a Goroutine. Or you have to return the/a channel. etc etc etc.
> It's quite rare for a function to unexpectedly gain a dependency on "doing IO" in general.
From the code sample it looks like printing to stdio will now require an Io param. So won’t you now have to pass that down to wherever you want to do a quick debug printf?
From the code sample it looks like printing to stdio will now require an Io param. So won’t you now have to pass that down to wherever you want to do a quick debug printf?
Zig has specifically a std.debug.print() function for debug printing and a std.log module for logging. Those don't necessarily need to be wired up with the whole stdio machinery.
std.debug.print(..) prints to stderr whuch does not need an io param.
I'm not familiar with Zig, but won't the classic "blocking" APIs still be around? I'd rather a synchronous debug print either way.
Yes. You can always use the blocking syscalls your OS provides and ignore the Io system for stuff like that. No idea how they’d do that by default in the stdlib, but it will definitely be possible.
I think the key is, if you don't have an "io" in your call stack you can always create one. At least I hope that is how it would work. Otherwise it is equally viral to async await.
> * It's quite rare for a function to unexpectedly gain a dependency on "doing IO" in general.
I don't know where you got this, but it's definitely not the case, otherwise async would never cause problems either. (Now the problem in both cases is pretty minor, you just need to change the type signature of the call stack, which isn't generally that big, but it's exactly the same situation)
> In practice, most of your codebase will have access to an `Io`, and only leaf functions doing pure computation will not need them.
So it's exactly similar to making all of your functions async by default…
I don't know where you got this, but it's definitely not the case, otherwise async would never cause problems either. (Now the problem in both cases is pretty minor, you just need to change the type signature of the call stack, which isn't generally that big, but it's exactly the same situation)
> In practice, most of your codebase will have access to an `Io`, and only leaf functions doing pure computation will not need them.
So it's exactly similar to making all of your functions async by default…
I' scratching my head here, because many languages avoid colouring. Effectively all I think you've done is specify an interface for the event loop. Python and I expect a few other languages have pluggable event loops that use the same technique.
Granted some languages like Rust don't, or at least Rust's std library doesn't standardise the event loop interface. That has lead to what can only be described as a giant mess, because there are many async frameworks, and you have to choose. If you implement some marvelous new protocol in Rust, people can't just plug it in unless you have provided the glue for the async framework they use. Zig has managed to avoid Rust's mistake with it's Io interface, but then most async implementations do avoid it in one way or another.
What you haven't avoided is the colouring that occurs between non-async code and async code. Is the trade-off "all code shall be async"? That incurs a cost to single threaded code, as all blocking system calls now become two calls (one to do the operation, and one wait for the outcome).
Long ago Rust avoided that by deciding other whether to do a blocking call, or a schedule call followed by a wait when the system call is done. But making decision also incurs it's over overhead on each and every system call, which Rust decided was too much of an imposition.
For Rust, there is possibly a solution: monomorphisation. The compiler generates one set of code when the OS scheduler is used, and another when the process has it's own event loop. I expect they haven't done that because it's hard and disruptive. I would be impressed if Zig had done it, but I suspect it hasn't.
Granted some languages like Rust don't, or at least Rust's std library doesn't standardise the event loop interface. That has lead to what can only be described as a giant mess, because there are many async frameworks, and you have to choose. If you implement some marvelous new protocol in Rust, people can't just plug it in unless you have provided the glue for the async framework they use. Zig has managed to avoid Rust's mistake with it's Io interface, but then most async implementations do avoid it in one way or another.
What you haven't avoided is the colouring that occurs between non-async code and async code. Is the trade-off "all code shall be async"? That incurs a cost to single threaded code, as all blocking system calls now become two calls (one to do the operation, and one wait for the outcome).
Long ago Rust avoided that by deciding other whether to do a blocking call, or a schedule call followed by a wait when the system call is done. But making decision also incurs it's over overhead on each and every system call, which Rust decided was too much of an imposition.
For Rust, there is possibly a solution: monomorphisation. The compiler generates one set of code when the OS scheduler is used, and another when the process has it's own event loop. I expect they haven't done that because it's hard and disruptive. I would be impressed if Zig had done it, but I suspect it hasn't.
[deleted]
You are skipping the massive point here.
If you are using a library in rust, it has to be async await, tokio, send+sync and all the other crap. Or if it is sync api then it is useless for async application.
This approach of passing IO removes this problem and this is THE main problem.
This way you don’t have to use procedural macros or other bs to implement multi versioning for the functions in your library, which doesn’t work well anyway in the end.
https://nullderef.com/blog/rust-async-sync/
You can find 50 other ones like this by searching.
To be honest I don’t hope they will solve cooperative scheduling, high performance, optionally thread-per-core async soon and the API won’t be that good anyway. But hope it solves all that in the future.
If you are using a library in rust, it has to be async await, tokio, send+sync and all the other crap. Or if it is sync api then it is useless for async application.
This approach of passing IO removes this problem and this is THE main problem.
This way you don’t have to use procedural macros or other bs to implement multi versioning for the functions in your library, which doesn’t work well anyway in the end.
https://nullderef.com/blog/rust-async-sync/
You can find 50 other ones like this by searching.
To be honest I don’t hope they will solve cooperative scheduling, high performance, optionally thread-per-core async soon and the API won’t be that good anyway. But hope it solves all that in the future.
> Or if it is sync api then it is useless for async application.
The rest is true, but this part isn't really an issue. If you're in an async function you can call sync functions still. And if you're worried it'll block and you can afford that, I know tokio offers spawn_blocking for this purpose.
The rest is true, but this part isn't really an issue. If you're in an async function you can call sync functions still. And if you're worried it'll block and you can afford that, I know tokio offers spawn_blocking for this purpose.
> If you are using a library in rust, it has to be async await, tokio, send+sync and all the other crap
Send and sync is only required if you want to access something from multiple threads, which isn't required by async await (parallelism vs concurrency)
1) You can use async await without parallelism and 2) send and sync aren't a product of async/await in Rust, but generally memory safety, i.e. you need Send generally when something can/is allowed to move between threads.
Send and sync is only required if you want to access something from multiple threads, which isn't required by async await (parallelism vs concurrency)
1) You can use async await without parallelism and 2) send and sync aren't a product of async/await in Rust, but generally memory safety, i.e. you need Send generally when something can/is allowed to move between threads.
Yes, but async Rust is basically built on tokio's runtime, which is what most the big async libraries depend on, like hyper/axum/tokio etc. And tokio is a thread-per-core work-stealing architecture, which requires Send + Sync bounds everywhere. You can avoid them if you depend on tokio-proper, but it's more icky when building on something like axum, where your application handlers also require these bounds.
A good article on this: https://emschwartz.me/async-rust-can-be-a-pleasure-to-work-w...
A good article on this: https://emschwartz.me/async-rust-can-be-a-pleasure-to-work-w...
Iirc I had a situation a while back, in which I used async await with tokio with a non Send or Sync type and it compiled when I didn't use spawn[1] (implying multithreading) but a simple loop with sequential processing.
Only when I wanted to enable parallelism using spawn, I got a compilation error.
[1]: https://docs.rs/tokio/latest/tokio/task/fn.spawn.html
Only when I wanted to enable parallelism using spawn, I got a compilation error.
[1]: https://docs.rs/tokio/latest/tokio/task/fn.spawn.html
> Send and sync is only required if you want to access something from multiple threads, which isn't required by async await (parallelism vs concurrency)
In theory this is correct. In practice, a lot of APIs (including many in tokio) require both traits even for single-thread use cases.
In theory this is correct. In practice, a lot of APIs (including many in tokio) require both traits even for single-thread use cases.
I'm not skipping anything. And in fact acknowledge this exact point:
> That being said, I don't think Zig's implementation here is bad. If anything, it does a great job at abstracting the usage from the implementation. This is something Rust fails at spectacularly.
> That being said, I don't think Zig's implementation here is bad. If anything, it does a great job at abstracting the usage from the implementation. This is something Rust fails at spectacularly.
This comment is justly flatly incorrect. You don't need Tokio at all to write an async library. Nor do you need send sync. Not sure what other crap you are speaking of, either.
Sync APIs can be spawned in worker threads as futures, too. Generally executors have helper methods for that.
Sync APIs can be spawned in worker threads as futures, too. Generally executors have helper methods for that.
Here's a trick to make every function red (or blue? I'm colorblind, you decide):
Jokes aside, I agree that there's obviously a non-zero amount of friction to using the `Io` intreface, but it's something qualitatively very different from what causes actual real-world friction around the use of async await.
> but the general problem behind function coloring is that of context
I would disagree, to me the problem seems, from a practical perspective that:
1. Code can't be reused because the async keyword statically colors a function as red (e.g. python's blocking redis client and asyncio-redis). In Zig any function that wants to do Io, be it blue (non-async) or red (async) still has to take in that parameter so from that perspective the Io argument is irrelevant.
2. Using async and await opts you automatically into stackless coroutines with no way of preventing that. With this new I/O system even if you decide to use a library that interally uses async, you can still do blocking I/O, if you want.
To me these seems the real problems of function coloring.
var io: std.Io = undefined;
pub fn main() !void {
var impl = ...;
io = impl.io();
}
Jokes aside, I agree that there's obviously a non-zero amount of friction to using the `Io` intreface, but it's something qualitatively very different from what causes actual real-world friction around the use of async await.
> but the general problem behind function coloring is that of context
I would disagree, to me the problem seems, from a practical perspective that:
1. Code can't be reused because the async keyword statically colors a function as red (e.g. python's blocking redis client and asyncio-redis). In Zig any function that wants to do Io, be it blue (non-async) or red (async) still has to take in that parameter so from that perspective the Io argument is irrelevant.
2. Using async and await opts you automatically into stackless coroutines with no way of preventing that. With this new I/O system even if you decide to use a library that interally uses async, you can still do blocking I/O, if you want.
To me these seems the real problems of function coloring.
Well, it's not really a joke. That's a valid strategy that languages use. In Go, every function is "async". And it basically blocks you from doing FFI (or at least it used to?). I wonder if Zig will run into similar issues here.
> 1. Code can't be reused because the async keyword statically colors a function
This is fair. And it's also a real pain point with Rust. However, it's funny that the "What color is your function?" article doesn't even really mention this.
> 2. Using async and await opts you automatically into stackless coroutines with no way of preventing that
This however I don't think is true. Async/await is mostly syntax sugar.
In Rust and C# it uses stackless coroutines.
In JS it uses callbacks.
There's nothing preventing you from making await suspend a green thread.
> 1. Code can't be reused because the async keyword statically colors a function
This is fair. And it's also a real pain point with Rust. However, it's funny that the "What color is your function?" article doesn't even really mention this.
> 2. Using async and await opts you automatically into stackless coroutines with no way of preventing that
This however I don't think is true. Async/await is mostly syntax sugar.
In Rust and C# it uses stackless coroutines.
In JS it uses callbacks.
There's nothing preventing you from making await suspend a green thread.
I should have specified that better, of course async and await can be lowered to different things (that's what Zig does afterall), what I wanted to say is that that's how it works in general. JS is a good counter example, but for all other mainstream languages, async means stackless coroutines (python, ruby, c#, rust, ...).
Which means that if I want to use a dependency that uses async await, it's stackless coroutines for me too whether I like it or not.
Which means that if I want to use a dependency that uses async await, it's stackless coroutines for me too whether I like it or not.
In ruby async is based on stackful fibers. With https://github.com/socketry/async-debug you can see a tree of all fibers with their full call stack. It also avoids the problem people talk about in this thread with go of passing a context parameter everywhere for cancellation as you can kill or raise any exception inside another fiber. I haven't used them but PHP fibers are also supposedly stackful. And Java and every JVM language has them since project loom in JDK 21.
My mistake, I should have said Python then.
> And it basically blocks you from doing FFI
It doesn't block it. But it does make FFI much more expensive in go than in languages like Rust, because every foreign call needs to set up a c-compatible stack.
It doesn't block it. But it does make FFI much more expensive in go than in languages like Rust, because every foreign call needs to set up a c-compatible stack.
The global io trick would totally be valid if you’re writing an application (i.e. not a library) and don’t have use of two different implementations of io
There are plenty of libraries out there which require users to do an init() call of some sorts at startup. It is perfectly possible to design a library that only works with 1 io instance and gets it at init(). Whether people like or want that… I have no clue.
you could still have a library-global io, let the user set it as desired.
> use of two different implementations of io
functionally rare situation.
> use of two different implementations of io
functionally rare situation.
I'll let a real category theorist get into the details that I'll likely flub, but the IO monad is where you end up if you start on this path. That context can be implicit, but it's there, and if you want any help from the compiler (to, for example, guide Claude Code towards useful outcomes) you've got to reify it as a real thing in the formality of the system.
Async and coroutines are the graveyard of dreams for systems programming languages, and Andrew by independently rediscovering the IO monad and getting it right? Hope of a generation.
Functions in the real world have colors: you can have predictable rules for moving between colors, or you can wing it and get C++ co_await and tokio and please kill me.
This is The Way.
Async and coroutines are the graveyard of dreams for systems programming languages, and Andrew by independently rediscovering the IO monad and getting it right? Hope of a generation.
Functions in the real world have colors: you can have predictable rules for moving between colors, or you can wing it and get C++ co_await and tokio and please kill me.
This is The Way.
it's not a monad, since you can do unholy (for fp) things with it, like stash it in a struct and pass the struct around (even to functions which have no clue theres an io call) or just grab a globalized io object and use that at will arbitrarily at many entrypoints in your function.
most importantly, besides the obvious situations (creating the io object, binding it to another object), it's not generally going to be returned from a function as part of its value.
most importantly, besides the obvious situations (creating the io object, binding it to another object), it's not generally going to be returned from a function as part of its value.
When I write in Haskell, I find myself mentally glossing the returned monadic state, along the lines of, "Oh, an M x is just an x that does monady stuff to get the x". This becomes natural once you get the hang of do-notation and sometimes monad combinators. So I'm not really thinking about the monadic state in the return value a lot.
It's not really any less natural than thinking stateful programming, except now the state is a reified thing, which I think is strictly advantageous once you get used to it.
It's not really any less natural than thinking stateful programming, except now the state is a reified thing, which I think is strictly advantageous once you get used to it.
i'm relatively confident that andrew will happily break the language again (god love him for that) when it becomes clear that you really want that algebra.
though i will say, for a systems language, it's probably better to invert the lift/unlift relationship, default to do-notation and explicitly unlift into pure functions. that's almost what const meant in C++ to begin with but it lost it's way.
though i will say, for a systems language, it's probably better to invert the lift/unlift relationship, default to do-notation and explicitly unlift into pure functions. that's almost what const meant in C++ to begin with but it lost it's way.
The same Andrew who rejected even basic interfaces in favour of duck-typed generics or manually written vtables?
you're just spreading FUD about zig breaking before 1.0. there is zero basis for you to be "relatively confident" about this matter, because
1. the io situation is basically restructuring analgously to the allocator situation, which is at this point battle tested. there are currently no "monads wrapping statefulness" anywhere in zig.
2. It's not in zig's nature to build something because it it satisifies an fp idiom. the abstractions and resulting "thing that the hardware does" (at least in release builds) are generally more or less obvious. the levels of compiler reinterpretation to achieve functional purity are not really the sort of thing that zig does. for example, zig does not have a privileged "iterator" method that the compiler reinterprets in a way that unrolls blocks or lambdas into loops without crossing a frame boundary.
1. the io situation is basically restructuring analgously to the allocator situation, which is at this point battle tested. there are currently no "monads wrapping statefulness" anywhere in zig.
2. It's not in zig's nature to build something because it it satisifies an fp idiom. the abstractions and resulting "thing that the hardware does" (at least in release builds) are generally more or less obvious. the levels of compiler reinterpretation to achieve functional purity are not really the sort of thing that zig does. for example, zig does not have a privileged "iterator" method that the compiler reinterprets in a way that unrolls blocks or lambdas into loops without crossing a frame boundary.
Go also suffers from this form of “subtle coloring”.
If you’re working with goroutines, you would always pass in a context parameter to handle cancellation. Many library functions also require context, which poisons the rest of your functions.
Technically, you don’t have to use context for a goroutine and could stub every dependency with context.Background, but that’s very discouraged.
If you’re working with goroutines, you would always pass in a context parameter to handle cancellation. Many library functions also require context, which poisons the rest of your functions.
Technically, you don’t have to use context for a goroutine and could stub every dependency with context.Background, but that’s very discouraged.
Having all async happen completely transparently is not really logically possible. asynchronous logic is frequently fundamentally different from synchronous logic, and you need to do something different one way or the other. I don't think that's really the same as "function colouring".
And context is used for more than just goroutines. Even a completely synchronous function can (and often does) take a context, and the cancellation is often useful there too.
And context is used for more than just goroutines. Even a completely synchronous function can (and often does) take a context, and the cancellation is often useful there too.
Context is not required in Go and I personally encourage you to avoid it. There is no shame in blazing a different path.
Why do you encourage avoiding it? Afaik it's the only way to early-abort an operation since Goroutines operate in a cooperative, not preemptive, paradigm. To be very clear, I'm asking this completely in good faith looking to learn something new!
> Afaik it's the only way to early-abort an operation since Goroutines operate in a cooperative, not preemptive, paradigm.
I'm not sure what you mean here. Preemptive/coorporative terminology refers to interrupting (not aborting) a CPU-bound task, in which case goroutines are fully preemptive on most platforms since Go 1.14, check the release notes for more info. However, this has nothing to do with context.
If you're referring to early-aborting IO operations, then yes, that's what context is for. However, this doesn't really have anything to do with goroutines, you could do the same if the runtime was built on OS threads.
I'm not sure what you mean here. Preemptive/coorporative terminology refers to interrupting (not aborting) a CPU-bound task, in which case goroutines are fully preemptive on most platforms since Go 1.14, check the release notes for more info. However, this has nothing to do with context.
If you're referring to early-aborting IO operations, then yes, that's what context is for. However, this doesn't really have anything to do with goroutines, you could do the same if the runtime was built on OS threads.
Goroutines are preemptive only to the runtime scheduler. You, the application developer merely using the language, cannot directly preempt a goroutine.
This makes goroutines effectively cooperative still from the perspective of the developer. The preemptive runtime "just" prevents things like user code starving out the garbage collector. To interrupt a goroutine, your options are generally limited to context cancelation and closing the channel or socket being read, if any. And the goroutine may still refuse to exit (or whatever else you want it to do), though that's largely up to how you code it.
This difference is especially stark when compared with Erlang/BEAM where you can directly address, signal, and terminate its lightweight processes.
This makes goroutines effectively cooperative still from the perspective of the developer. The preemptive runtime "just" prevents things like user code starving out the garbage collector. To interrupt a goroutine, your options are generally limited to context cancelation and closing the channel or socket being read, if any. And the goroutine may still refuse to exit (or whatever else you want it to do), though that's largely up to how you code it.
This difference is especially stark when compared with Erlang/BEAM where you can directly address, signal, and terminate its lightweight processes.
Exactly, thank you. I knew that the runtime gained the ability to preempt but the clear fact that you cannot get a handle to a goroutine (eg `gr := go fn()` is proof you have no way to take advantage of this ability as a user.
You are expecting them to actually check the value, there is nothing preemptive.
Another approach is special messages over a side channel.
Another approach is special messages over a side channel.
So instead of a context you need to pass a a channel. Same problem.
Not necessarily, that is one of the reasons OOP exists.
Have a struct representing the set of associated activities, owning the channel.
Have a struct representing the set of associated activities, owning the channel.
soo, a context?
You can take that view, yes.
But if you store your context in a struct (which is not the recommend “best practice” – but which you can do) it's no longer a function coloring issue.
I do that in on of my libraries and I feel that it's the right call (for that library).
But if you store your context in a struct (which is not the recommend “best practice” – but which you can do) it's no longer a function coloring issue.
I do that in on of my libraries and I feel that it's the right call (for that library).
If the struct has a well-scoped and short-lived lifecycle, then it is actually better to put the context in the struct. Many Go libraries including the stdlib do this despite not being "best practice".
An exception to the short-lived rule is to put context in your service struct and pass it as the base context when constructing the HTTP server, so that when you get a service shutdown signal, one can cancel requests gracefully.
An exception to the short-lived rule is to put context in your service struct and pass it as the base context when constructing the HTTP server, so that when you get a service shutdown signal, one can cancel requests gracefully.
It's well scoped, but not short lived; it's an SQLite connection.
But the API surface is huge, with 100s of methods on the connection and derived objects, with it being unclear which might block and be worthy of asynchronous cancellation. You never know when pulling an additional column if that one might be an overflow text/blob that does additional IO.
The solution, while not amazing is a method that you use like this:
But the API surface is huge, with 100s of methods on the connection and derived objects, with it being unclear which might block and be worthy of asynchronous cancellation. You never know when pulling an additional column if that one might be an overflow text/blob that does additional IO.
The solution, while not amazing is a method that you use like this:
old := conn.SetInterrupt(ctx)
defer conn.SetInterrupt(old)
Nope, because I didn't mention it was to be passed around as a compulsory parameter, rather have your logic organised across structs with methods, hide most details behind interfaces.
It's not required, but eschewing it ends up going against the grain, since so much of the ecosystem is written to use contexts, including the standard library.
For example, say you instead of contexts, you use channels for cancellation. You can have a goroutine like this:
Likewise, processResult() may be doing stuff that cannot be aborted just by closing the stop channel. You could pass the stop channel to it, but now you're just reinventing contexts.
Of course, you can choose to only use contexts where you're forced to, and invent wrappers around standard library stuff (e.g. the HTTP client), but at that point you're going pretty far to avoid them.
I do think the context is problematic. For the purposes of cancellation, it's invasive and litters the call graph with parameters and variables. Goroutines really ought to have an implicit context inherited from its parent, since everything is using it anyway.
Contexts are wildly abused for passing data around, leading to bloated contexts and situations where you can't follow the chain of data-passing without carefully reviewing the entire call graph. I always recommend not being extremely discriminating about where to pass values in context. A core principle is that it has to be something that is so pervasive that it would be egregious to pass around explicitly, such as loggers and application-wide feature flags.
For example, say you instead of contexts, you use channels for cancellation. You can have a goroutine like this:
go func() {
for {
select {
case <-stop:
return
case <-time.After(1*time.Second):
resp := fetchURL(url)
processResult(resp.Body) // Simplified, of course
}
}
}()
Likewise, processResult() may be doing stuff that cannot be aborted just by closing the stop channel. You could pass the stop channel to it, but now you're just reinventing contexts.
Of course, you can choose to only use contexts where you're forced to, and invent wrappers around standard library stuff (e.g. the HTTP client), but at that point you're going pretty far to avoid them.
I do think the context is problematic. For the purposes of cancellation, it's invasive and litters the call graph with parameters and variables. Goroutines really ought to have an implicit context inherited from its parent, since everything is using it anyway.
Contexts are wildly abused for passing data around, leading to bloated contexts and situations where you can't follow the chain of data-passing without carefully reviewing the entire call graph. I always recommend not being extremely discriminating about where to pass values in context. A core principle is that it has to be something that is so pervasive that it would be egregious to pass around explicitly, such as loggers and application-wide feature flags.
What would you use in its place? I've never had an issue with it. I use it for 1) early termination 2) carrying custom request metadata.
I don't really think it is fully the coloring problem because you can easily call non-context functions from context functions (but not other way around, so one way coloring issue), but you need to be aware the cancellation chain of course stops then.
I don't really think it is fully the coloring problem because you can easily call non-context functions from context functions (but not other way around, so one way coloring issue), but you need to be aware the cancellation chain of course stops then.
I think the main point is in something like Go, the approach is non-viral. If you are 99 levels deep in synchronous code and need to call something with context, well, you can just create one. With C#, if you need to refactor all 99 levels above (or use bad practices which is of course what everyone does).
Also, in general cancellation is something that you want to optionally have with any asynchronous function so I don't think there really exists an ideal approach that doesn't include it. In my opinion the approach taken by Zig looks pretty good.
Also, in general cancellation is something that you want to optionally have with any asynchronous function so I don't think there really exists an ideal approach that doesn't include it. In my opinion the approach taken by Zig looks pretty good.
Like you said you dont NEED context. Its just something thats available if you need it. I still think Go/Erlang has one of the best concurrency stories out there.
> If you’re working with goroutines, you would always pass in a context parameter to handle cancellation.
The utility of context could be called a subtle coloring. But you do NOT need context at all. If your dealing with data+state (around queue and bus processing) its easy to throw things into a goroutine and let the chips fall where they will.
> which poisons the rest of your functions. You are free to use context dependent functions without a real context: https://pkg.go.dev/context#TODO
The utility of context could be called a subtle coloring. But you do NOT need context at all. If your dealing with data+state (around queue and bus processing) its easy to throw things into a goroutine and let the chips fall where they will.
> which poisons the rest of your functions. You are free to use context dependent functions without a real context: https://pkg.go.dev/context#TODO
The thing about context is it can be a lot more than a cancellation mechanism. You can attach anything to it—metadata, database client, logger, whatever. Even Io and Allocator if you want to. Signatures are future-proof as long as you take a context for everything.
At the end of the day you have to pass something for cooperative multitasking.
Of course it’s also trivial to work around if you don’t like the pattern, “very discouraged” or not.
At the end of the day you have to pass something for cooperative multitasking.
Of course it’s also trivial to work around if you don’t like the pattern, “very discouraged” or not.
Agree that with something like go, there is truly no function coloring at all. However, since most real world async things require cancellation, a context parameter is always present so there is some "coloring" do to that. Still, it is much less viral than C# style async await as if you don't have a context in your call stack you can still create one when needed and call the function. I don't think it is reasonable to abstract cancellation in a way that nothing has to be passed in so perhaps the approach presented here is realistically as good as it gets.
Aside from the ridiculous argument that function parameters color them, the assertion that you can’t call a function that takes IO from inside a function that does not is false, since you can initialize one to pass it in
To me, there's no difference between the IO param and async/await. Adding either one causes it to not be callable from certain places.
As for the second thing:
You can do that, but... You can also do this in Rust. Yet nobody would say Rust has solved function coloring.
Also, check this part of the article:
> In the less common case when a program instantiates more than one Io implementation, virtual calls done through the Io interface will not be de-virtualized, ...
Doing that is an instant performance hit. Not to mention annoying to do.
As for the second thing:
You can do that, but... You can also do this in Rust. Yet nobody would say Rust has solved function coloring.
Also, check this part of the article:
> In the less common case when a program instantiates more than one Io implementation, virtual calls done through the Io interface will not be de-virtualized, ...
Doing that is an instant performance hit. Not to mention annoying to do.
> Doing that is an instant performance hit. Not to mention annoying to do.
The cost of virtual dispatch on IO path is almost always negligible. It is literally one conditional vs syscall. I doubt it you can even measure the difference.
The cost of virtual dispatch on IO path is almost always negligible. It is literally one conditional vs syscall. I doubt it you can even measure the difference.
>To me, there's no difference between the IO param and async/await.
You can't pass around "async/await" as a value attached to another object. You can do that with the IO param. That is very different.
You can't pass around "async/await" as a value attached to another object. You can do that with the IO param. That is very different.
> You can't pass around "async/await" as a value attached to another object
Sure you can? You can just pass e.g. a Task around in C# without awaiting it, it's when you need a result from a task that you must await it.
Sure you can? You can just pass e.g. a Task around in C# without awaiting it, it's when you need a result from a task that you must await it.
[deleted]
Sure you can. An `async` function in Javascript is essentially a completely normal function that returns a promise. The `async`/`await` syntax is a convenient syntax sugar for working with promises, but the issue would still exist if it didn't exist.
More to the point, the issue would still exist even if promises didn't exist — a lot of Node APIs originally used callbacks and a continuation-passing style approach to concurrency, and that had exactly the same issues.
More to the point, the issue would still exist even if promises didn't exist — a lot of Node APIs originally used callbacks and a continuation-passing style approach to concurrency, and that had exactly the same issues.
Other commenters have already provided examples for other languages, and it's the same for Rust: async functions are just regular functions that return an impl Future type. As a sync function, you can call a bunch of async functions and return the futures to your caller to handle, or you can block your current thread with the block_on function typically available through a handle (similar to the Io object here) provided by your favorite async runtime [0].
In other words, you don't need such an Io object upfront: You need it when you want to actually drive its execution and get the result. From this perspective, the Zig approach is actually less flexible than Rust.
[0]: https://docs.rs/tokio/latest/tokio/runtime/struct.Handle.htm...
In other words, you don't need such an Io object upfront: You need it when you want to actually drive its execution and get the result. From this perspective, the Zig approach is actually less flexible than Rust.
[0]: https://docs.rs/tokio/latest/tokio/runtime/struct.Handle.htm...
Conceptually, there's not much of a difference.
If you have a sync/non-IO function that now needs to do IO, it becomes async/IO. And since IO and async are viral, it's callers must also now be IO/async and call it with IO/await. All the way up the call stack.
If you have a sync/non-IO function that now needs to do IO, it becomes async/IO. And since IO and async are viral, it's callers must also now be IO/async and call it with IO/await. All the way up the call stack.
You’re allowed to not like it, but that doesn’t change that your argument that this is a form of coloring is objectively false. I’m not sure what Rust has to do with it.
Sure it is a function coloring. Just in a different form. `async` in other languages is something like an implicit parameter. In zig they made this implicit parameter explicit. Is that more better/more ergonomic? I don't know yet. The sugar is different, but the end result the same. Unless you can show me concrete example of things that the approach zig has taken can do that is not possible in say, rust. Than I don't buy that it's not just another form of function coloring.
> Unless you can show me concrete example
add io to a struct and let the struct keep track of its own io.
add io to a struct and let the struct keep track of its own io.
You can also carry around a runtime and dispatch async function in non-async functions.
Unless I'm misunderstanding, that's effectively implementing Future for the struct
It’s more like adding a runtime handle to the struct.
Modulo that I’m not sure any langage with a sync/async split has an “async” runtime built entirely out of sync operations. So a library can’t take a runtime for a caller and get whatever implementation the caller decided to use.
Modulo that I’m not sure any langage with a sync/async split has an “async” runtime built entirely out of sync operations. So a library can’t take a runtime for a caller and get whatever implementation the caller decided to use.
> I’m not sure any langage with a sync/async split has an “async” runtime built entirely out of sync operations.
You get into hairy problems of definition, but you can definitely create an "async" runtime out of "sync" operations: implement an async runtime with calls to C. C doesn't have a concept of "async", and more or less all async runtime end up like this.
I've implemented Future (Rust) on a struct for a Windows operation based only on C calls into the OS. The struct maintains everything needed to know the state of the IO, and while I coupled the impl to the runtime for efficiency (I've written it too), it's not strictly necessary from memory.
You get into hairy problems of definition, but you can definitely create an "async" runtime out of "sync" operations: implement an async runtime with calls to C. C doesn't have a concept of "async", and more or less all async runtime end up like this.
I've implemented Future (Rust) on a struct for a Windows operation based only on C calls into the OS. The struct maintains everything needed to know the state of the IO, and while I coupled the impl to the runtime for efficiency (I've written it too), it's not strictly necessary from memory.
> You get into hairy problems of definition, but you can definitely create an "async" runtime out of "sync" operations: implement an async runtime with calls to C. C doesn't have a concept of "async", and more or less all async runtime end up like this.
While C doesn't have async OS generally provide APIs which are non-blocking, and that is what async runtimes are implemented on top of.
By sync operations I mean implementing an "async" runtime entirely atop blocking operations, without bouncing them through any sort of worker threads or anything.
While C doesn't have async OS generally provide APIs which are non-blocking, and that is what async runtimes are implemented on top of.
By sync operations I mean implementing an "async" runtime entirely atop blocking operations, without bouncing them through any sort of worker threads or anything.
I've been pondering this, it feels like a brain teaser. Is it possible to create an "async" runtime out of only blocking operations?
It feels like it turns purely on what "blocking operations" are (does setting a lock bit and returning count as non-blocking?)
It feels like it turns purely on what "blocking operations" are (does setting a lock bit and returning count as non-blocking?)
It's funny, but I do actually like it. It's just that it walks like a duck, swims like a duck and quacks like a duck.
I don't have a problem with IO conceptually (but I do have a problem with Zig ergonomics, allocator included). I do have a problem with claiming you defeated function coloring.
Like, look. You didn't even get rid of await ...
> try a_future.await(io);
I don't have a problem with IO conceptually (but I do have a problem with Zig ergonomics, allocator included). I do have a problem with claiming you defeated function coloring.
Like, look. You didn't even get rid of await ...
> try a_future.await(io);
I do want to say that I regretted that comment as nonconstructive after it was too late to edit it. Others in the thread are representing my argument better than I can or care to.
I mean... you use `await` if you've used `async`. It's your choice whether or not you do; and if you don't want to, your callers and callees can still freely `async` and `await` if they want to. I don't understand the point you're trying to make here.
To be clear, where many languages require you to write `const x = await foo()` every time you want to call an async function, in Zig that's just `const x = foo()`. This is a key part of the colorless design; you can't be required to acknowledge that a function is async in order to use it. You'll only use `await` if you first use `async` to explicitly say "I want to run this asynchronously with other code here if possible". If you need the result immediately, that's just a function call. Either way, your caller can make its own choice to call you or other functions as `async`, or not to; as can your callees.
To be clear, where many languages require you to write `const x = await foo()` every time you want to call an async function, in Zig that's just `const x = foo()`. This is a key part of the colorless design; you can't be required to acknowledge that a function is async in order to use it. You'll only use `await` if you first use `async` to explicitly say "I want to run this asynchronously with other code here if possible". If you need the result immediately, that's just a function call. Either way, your caller can make its own choice to call you or other functions as `async`, or not to; as can your callees.
> in Zig that's just ...
Well, no. In zig that's `const x = foo(io)`.
The moment you take or even know about an io, your function is automatically "generic" over the IO interface.
Using stackless coroutines and green threads results in a completely different codegen.
I just noticed this part of the article:
> Stackless Coroutines > > This implementation won’t be available immediately like the previous ones because it depends on reintroducing a special function calling convention and rewriting function bodies into state machines that don’t require an explicit stack to run. > > This execution model is compatible with WASM and other platforms where stack swapping is not available or desireable.
I wonder what will happen if you try to await a future created with a green thread IO using a stackless coroutine IO.
Well, no. In zig that's `const x = foo(io)`.
The moment you take or even know about an io, your function is automatically "generic" over the IO interface.
Using stackless coroutines and green threads results in a completely different codegen.
I just noticed this part of the article:
> Stackless Coroutines > > This implementation won’t be available immediately like the previous ones because it depends on reintroducing a special function calling convention and rewriting function bodies into state machines that don’t require an explicit stack to run. > > This execution model is compatible with WASM and other platforms where stack swapping is not available or desireable.
I wonder what will happen if you try to await a future created with a green thread IO using a stackless coroutine IO.
> Well, no. In zig that's `const x = foo(io)`.
If `foo` needs to do IO, sure. Or, more typically (as I mentioned in a different comment), it's something like `const x = something.foo()`, and `foo` can get its `Io` instance from `something` (in the Zig compiler this would be a `Compilation` or a `Zcu` or a `Sema` or something like that).
> Using stackless coroutines and green threads results in a completely different codegen.
Sure, but that's abstracted away from you. To be clear, stackless coroutines are the only case where the codegen of callers is affected, which is why they require a language feature. Even if your application uses two `Io` implementations for some reason, one of which is based on stackless coroutines, functions using the API are not duplicated.
> I wonder what will happen if you try to await a future created with a green thread IO using a stackless coroutine IO.
Mixing futures from any two different `Io` implementations will typically result in Illegal Behavior -- just like passing a pointer allocated with one `Allocator` into the `free` of a different `Allocator` does. This really isn't a problem. Even with allocators, it's pretty rare for people to mess this up, and with allocators you often do have multiple of them available in one place (e.g. a gpa and an arena). In contrast, it will be extraordinarily rare to have more than one `Io` lying around. Even if you do mess it up, the IB will probably just trip a safety check, so it shouldn't take you too long to realise what you've done.
If `foo` needs to do IO, sure. Or, more typically (as I mentioned in a different comment), it's something like `const x = something.foo()`, and `foo` can get its `Io` instance from `something` (in the Zig compiler this would be a `Compilation` or a `Zcu` or a `Sema` or something like that).
> Using stackless coroutines and green threads results in a completely different codegen.
Sure, but that's abstracted away from you. To be clear, stackless coroutines are the only case where the codegen of callers is affected, which is why they require a language feature. Even if your application uses two `Io` implementations for some reason, one of which is based on stackless coroutines, functions using the API are not duplicated.
> I wonder what will happen if you try to await a future created with a green thread IO using a stackless coroutine IO.
Mixing futures from any two different `Io` implementations will typically result in Illegal Behavior -- just like passing a pointer allocated with one `Allocator` into the `free` of a different `Allocator` does. This really isn't a problem. Even with allocators, it's pretty rare for people to mess this up, and with allocators you often do have multiple of them available in one place (e.g. a gpa and an arena). In contrast, it will be extraordinarily rare to have more than one `Io` lying around. Even if you do mess it up, the IB will probably just trip a safety check, so it shouldn't take you too long to realise what you've done.
I find these two statements to be contradictory
> Sure, but that's abstracted away from you
> Mixing futures from any two different `Io` implementations will typically result in Illegal Behavior
Thinking about it more, you've possibly added even more colors. Each executor adds a different color and while each function is color-agnostic (but not colorless) futures aren't.
> it will be extraordinarily rare to have more than one `Io`
Will it? I can immediately think of a use case where a program might want to block for files on disk, but defer fetching from network to some background async executor.
> Sure, but that's abstracted away from you
> Mixing futures from any two different `Io` implementations will typically result in Illegal Behavior
Thinking about it more, you've possibly added even more colors. Each executor adds a different color and while each function is color-agnostic (but not colorless) futures aren't.
> it will be extraordinarily rare to have more than one `Io`
Will it? I can immediately think of a use case where a program might want to block for files on disk, but defer fetching from network to some background async executor.
Also, I do find it funny that we went from "Zig has completely defeated function coloring" to "Zig has colored objects".
but that's not even the case, because it's certainly possible to write a function that receives an object that holds onto an io (and uses it in its vtable calls) that equally well receives an object that doesn't have anything to do with io [0]. The consumers of those objects don't have to care, so there's no coloring.
[0] and this isn't even really a theoretical matter, having colorblind object passing is extremely useful for say, mocking. Oh, I have a database lookup/remote API call, which obviously requires io, but i want fast tests and I can mock it with an object with preseeded values/expects -- hey, that doesn't require IO.
[0] and this isn't even really a theoretical matter, having colorblind object passing is extremely useful for say, mocking. Oh, I have a database lookup/remote API call, which obviously requires io, but i want fast tests and I can mock it with an object with preseeded values/expects -- hey, that doesn't require IO.
I think in practice the caller still needs to know.
If I call `a.foo()` but `a` has and is using a stackless coroutine IO but the caller is being executed from a green thread IO then as was said before, I'm hitting UB.
But, I do like that you could skip/mock IO for instance. That's pretty neat.
If I call `a.foo()` but `a` has and is using a stackless coroutine IO but the caller is being executed from a green thread IO then as was said before, I'm hitting UB.
But, I do like that you could skip/mock IO for instance. That's pretty neat.
here is example code. you wont "use the wrong io".
const VTable = struct {
f: &fn (*VTable) void,
};
const A = struct {
io: IO,
v: VTable = .{ .f = &A.uses_io },
fn uses_io(this: *VTable) void {
const self: *A = @fieldParentPtr(.v, this);
self.io.some_io_fn(...);
}
};
const B = struct{v: VTable = .{.f = &void_fn}};
fn void_fn(_: *VTable) void {}
pub fn calls_vtable(v: VTable) {
v.f()
}> Adding either one causes it to not be callable from certain places.
you can call a function that requires an io parameter from a function that doesn't have one by passing in a global io instance?
as a trivial example the fn main entrypoint in zig will never take an io parameter... how do you suppose you'd bootstrap the io parameter that you'd eventually need. this is unlike other languages where main might or might not be async.
you can call a function that requires an io parameter from a function that doesn't have one by passing in a global io instance?
as a trivial example the fn main entrypoint in zig will never take an io parameter... how do you suppose you'd bootstrap the io parameter that you'd eventually need. this is unlike other languages where main might or might not be async.
You can call an async function from a function that is not async by passing in a global runtime (/ event loop).
As a trivial example the main entry point in rust is never async. How’d you suppose you’d bootstrap the runtime that you’d eventually need.
This is pretty much like every other langage.
As a trivial example the main entry point in rust is never async. How’d you suppose you’d bootstrap the runtime that you’d eventually need.
This is pretty much like every other langage.
and yet... there are libraries that were written twice, once for async and once for not.
Which should be a pretty big hint that you've misidentified the issue.
People in software really do have poor abilities to see the forest for the trees don't they lol
>you can call a function that requires an io parameter from a function that doesn't have one by passing in a global io instance?
How will that work with code mixing different Io implementations? Say a library pulled in uses a global Io instance while the calling code is using another.
I guess this can just be shot down with "don't do that" but it feels like a new kind of pitfall get get into.
How will that work with code mixing different Io implementations? Say a library pulled in uses a global Io instance while the calling code is using another.
I guess this can just be shot down with "don't do that" but it feels like a new kind of pitfall get get into.
> Say a library pulled in uses a global Io instance while the calling code is using another.
it'll probably carry a stigma like using unsafe does.
it'll probably carry a stigma like using unsafe does.
Zig already has an Allocator interface that gets passed around, and the convention is that libraries don't select an Allocator. Only provide APIs that accept allocators. If there's a certain process that works best with an Arena, then the API may wrap a provided function in an Arena, but not decide on their own underlying allocator for the user.
For Zig users, adopting this same mindset for Io is not really anything new. It's just another parameter that occasionally needs to be passed into an API.
For Zig users, adopting this same mindset for Io is not really anything new. It's just another parameter that occasionally needs to be passed into an API.
while not really idiomatic, as long as you let the user define the Io instance (eg with some kind of init function), then it doesn't really matter how that value is accessed within the library itself.
that's why this isn't really the same as async "coloring"
that's why this isn't really the same as async "coloring"
[deleted]
> you can’t call a function that takes IO from inside a function that does not is false, since you can initialize one to pass it in
that's not true. suppose a function foo(anytype) takes a struct, and expects method bar() on the struct.
you could send foo() the struct type Sync whose bar() does not use io. or you could send foo() the struct type Async whose bar uses an io stashed in the parameter, and there would be no code changes.
if you don't prefer compile time multireification, you can also use type erasure and accomplish the same thing with a vtable.
that's not true. suppose a function foo(anytype) takes a struct, and expects method bar() on the struct.
you could send foo() the struct type Sync whose bar() does not use io. or you could send foo() the struct type Async whose bar uses an io stashed in the parameter, and there would be no code changes.
if you don't prefer compile time multireification, you can also use type erasure and accomplish the same thing with a vtable.
It's hard to parse your comment, but I think we are agreeing? I was refuting the point of the parent. you have given another example of calling an IO-taking function inside a non-IO taking function. the example I gave was initializing an IO inside the non-IO taking function. you could also, as pointed out elsewhere, use global state.
> In order to call such a function you also need to provide the context. Zig hasn't really solved this.
It is much more flexible though since you don't need to pass the IO implementation into each function that needs to do IO. You could pass it once into an init function and then use that IO impl throughout the object or module. Whether that's good style is debatable - the Zig stdlib currently has containers that take an allocator in the init function, but those are on the way out in favour of explicitly taking the allocator in each function that needs to allocate - but the user is still free to write a minimal wrapper to restore the 'pass allocator into init' behaviour.
Odin has an interesting solution in that it passes an implicit context pointer into each function, but I don't know if the compiler is clever enough to remove the overhead for called functions that don't access the context (since it also needs to peek into all called functions - theoretically Zig with it's single-compilation-unit approach could probably solve that problem better).
It is much more flexible though since you don't need to pass the IO implementation into each function that needs to do IO. You could pass it once into an init function and then use that IO impl throughout the object or module. Whether that's good style is debatable - the Zig stdlib currently has containers that take an allocator in the init function, but those are on the way out in favour of explicitly taking the allocator in each function that needs to allocate - but the user is still free to write a minimal wrapper to restore the 'pass allocator into init' behaviour.
Odin has an interesting solution in that it passes an implicit context pointer into each function, but I don't know if the compiler is clever enough to remove the overhead for called functions that don't access the context (since it also needs to peek into all called functions - theoretically Zig with it's single-compilation-unit approach could probably solve that problem better).
You can write a wrapper in other langs, too, e.g. in Rust: block_on(async_fn)
So this is a tangent from the main article, but this comment made me curious and I read the original "What color is Your Function" post.
It was an interesting read, but I guess I came away confused about why "coloring" functions is a problem. Isn't "coloring" just another form of static typing? By giving the compiler (or interpreter) more meta data about your code, it can help you avoid mistakes. But instead of the usual "first argument is an integer" type meta data, "coloring" provides useful information like: "this function behaves in this special way" or "this function can be called in these kinds of contexts." Seems reasonable?
Like the author seems very perturbed that there can be different "colors" of functions, but a function that merely calculates (without any IO or side-effects) is different than one that does perform IO. A function with only synchronous code behaves very differently than one that runs code inside another thread or in a different tick of the event loop. Why is it bad to have functions annotated with this meta data? The functions behave in a fundamentally different way whether you give them special annotations/syntax or not. Shouldn't different things look different?
He mentions 2015 era Java as being ok, but as someone that’s written a lot of multithreaded Java code, it’s easy to mess up and people spam the “synchronized” keyword/“color” everywhere as a result. I don’t feel the lack of colors in Java makes it particularly intuitive or conceptually simpler.
It was an interesting read, but I guess I came away confused about why "coloring" functions is a problem. Isn't "coloring" just another form of static typing? By giving the compiler (or interpreter) more meta data about your code, it can help you avoid mistakes. But instead of the usual "first argument is an integer" type meta data, "coloring" provides useful information like: "this function behaves in this special way" or "this function can be called in these kinds of contexts." Seems reasonable?
Like the author seems very perturbed that there can be different "colors" of functions, but a function that merely calculates (without any IO or side-effects) is different than one that does perform IO. A function with only synchronous code behaves very differently than one that runs code inside another thread or in a different tick of the event loop. Why is it bad to have functions annotated with this meta data? The functions behave in a fundamentally different way whether you give them special annotations/syntax or not. Shouldn't different things look different?
He mentions 2015 era Java as being ok, but as someone that’s written a lot of multithreaded Java code, it’s easy to mess up and people spam the “synchronized” keyword/“color” everywhere as a result. I don’t feel the lack of colors in Java makes it particularly intuitive or conceptually simpler.
Yes, the main character of that article really is mostly JavaScript. The main issue there is that some things must be async, and that doesn't mesh well with things that can't be.
If you're writing a game, and you need to render a new enemy, you might want to reduce performance by blocking rather than being shot by an invisible enemy because you can only load the model async.
But even the article acknowledges that various languages tackle this problem better. Zig does a good job, but claiming it's been defeated completely doesn't really fly for me.
If you're writing a game, and you need to render a new enemy, you might want to reduce performance by blocking rather than being shot by an invisible enemy because you can only load the model async.
But even the article acknowledges that various languages tackle this problem better. Zig does a good job, but claiming it's been defeated completely doesn't really fly for me.
> He mentions 2015 era Java as being ok, but as someone that’s written a lot of multithreaded Java code, it’s easy to mess up and people spam the “synchronized” keyword/“color” everywhere as a result. I don’t feel the lack of colors in Java makes it particularly intuitive or conceptually simpler.
Async as a keyword doesn’t solve this or make writing parallel code any easier. You can still mess this up even if every function is annotated as async.
> A function with only synchronous code behaves very differently than one that runs code inside another thread or in a different tick of the event loop.
I think this is conflating properties of multiple runtimes. This is true in JavaScript because the runtime works on an event loop. In Java an “async” function that reads from a file or makes an http call doesn’t run in a different threads and doesn’t run in a different tick of an event loop. So what value does it have in that type of runtime?
Personally for me I think “async” is putting pain on a lot of developers where 99% of all code is not parallel and doesn’t share memory.
Async as a keyword doesn’t solve this or make writing parallel code any easier. You can still mess this up even if every function is annotated as async.
> A function with only synchronous code behaves very differently than one that runs code inside another thread or in a different tick of the event loop.
I think this is conflating properties of multiple runtimes. This is true in JavaScript because the runtime works on an event loop. In Java an “async” function that reads from a file or makes an http call doesn’t run in a different threads and doesn’t run in a different tick of an event loop. So what value does it have in that type of runtime?
Personally for me I think “async” is putting pain on a lot of developers where 99% of all code is not parallel and doesn’t share memory.
I believe the point is less about "coloring" not having value as a type-system feature, and more about its bad ergonomics, and its viral nature in particular.
> It was an interesting read, but I guess I came away confused about why "coloring" functions is a problem. Isn't "coloring" just another form of static typing?
It is. Function coloring is static typing.
But people never ever agree on what to put in typing system. For example, Java's checked exceptions are a form of typing... and everyone hates them.
Anyway it's always like that. Some people find async painful and say fuck it I'm going to manage threads manually. In the meanwhile another bunch of people work hard to introduce async to their language. Grass is always greener on the other side.
It is. Function coloring is static typing.
But people never ever agree on what to put in typing system. For example, Java's checked exceptions are a form of typing... and everyone hates them.
Anyway it's always like that. Some people find async painful and say fuck it I'm going to manage threads manually. In the meanwhile another bunch of people work hard to introduce async to their language. Grass is always greener on the other side.
> But people never ever agree on what to put in typing system. For example, Java's checked exceptions are a form of typing... and everyone hates them.
I love checked exceptions. Checked errors are fantastic and I think most developers would agree they want errors to be in the type system, but Java as a language just hasn’t provided the language syntax to make them usable. They haven’t made it easy to “uncheck” when you can’t possibly handle an error. You have to write boilerplate:
[0] https://docs.scala-lang.org/scala3/reference/experimental/ca...
I love checked exceptions. Checked errors are fantastic and I think most developers would agree they want errors to be in the type system, but Java as a language just hasn’t provided the language syntax to make them usable. They haven’t made it easy to “uncheck” when you can’t possibly handle an error. You have to write boilerplate:
Something s;
try {
s = something();
} catch (SomethingException e) {
throw new RuntimeException(e);
}
var s = try! something();
try {
someCall(s -> {
try {
another(s);
} catch (CheckedException ex) {
throw new UncheckedException(ex);
}
});
} catch (UncheckedException ex) {
// handle somehow
}
[0] https://docs.scala-lang.org/scala3/reference/experimental/ca...
[deleted]
> Isn't "coloring" just another form of static typing?
In a very direct way. Another example in languages that don't like you ignoring errors, changing a function from infallible to fallible is a breaking change, a la "it's another colour".
I'm glad it is: if a function I call can suddenly fail, at the very least I want to know that it can, even if the only thing I do is ignore it (visibly).
In a very direct way. Another example in languages that don't like you ignoring errors, changing a function from infallible to fallible is a breaking change, a la "it's another colour".
I'm glad it is: if a function I call can suddenly fail, at the very least I want to know that it can, even if the only thing I do is ignore it (visibly).
> Isn't "coloring" just another form of static typing?
Yes, and so is declaring what exceptions a function can throw (checked exceptions in Java).
> Why is it bad to have functions annotated with this meta data? The functions behave in a fundamentally different way whether you give them special annotations/syntax or not. Shouldn't different things look different?
It really isn't a problem. The article makes people think they've discovered some clever gotcha when they first read it, but IMHO people who sit down for a bit and think through the issue come to the same conclusion you have - Function coloring isn't a problem in practice.
Yes, and so is declaring what exceptions a function can throw (checked exceptions in Java).
> Why is it bad to have functions annotated with this meta data? The functions behave in a fundamentally different way whether you give them special annotations/syntax or not. Shouldn't different things look different?
It really isn't a problem. The article makes people think they've discovered some clever gotcha when they first read it, but IMHO people who sit down for a bit and think through the issue come to the same conclusion you have - Function coloring isn't a problem in practice.
> but IMHO people who sit down for a bit and think through the issue come to the same conclusion you have - Function coloring isn't a problem in practice.
I dunno man, have you seen people complain about async virality in Rust being annoying? Have you ever tried to read a backtrace from a program that does stackless coroutines (it's not fun)? Have you seen people do basically duplicate work to maintain a blocking and an async version of the same networking library?
I dunno man, have you seen people complain about async virality in Rust being annoying? Have you ever tried to read a backtrace from a program that does stackless coroutines (it's not fun)? Have you seen people do basically duplicate work to maintain a blocking and an async version of the same networking library?
People do complain, like they do from things like systemd. Then there is us, the silent majority, who just get shit done with these tools.
I respect what zig has done here, and I will want to try it out when it stabilizes. But Rust async is just fine.
I respect what zig has done here, and I will want to try it out when it stabilizes. But Rust async is just fine.
The alternative is to jump through a bunch of hoops to hide the "coloring" behind some opaque abstraction that is complicated and that will still get in the way when things go wrong.
Everyone complained when async IO was done with callbacks, so sugar was added to the callbacks, and now everyone has spent over a decade complaining about what flavor of sugar tastes best.
Y'all at Zig have a solution, I trust Zig's solution will be a good one (zig is lots of fun to use as a language) but at the end of the day, IO is slow, that needs to get hidden somehow, or not.
Everyone should have to do embedded for awhile and setup their own DMA controller operations. Having async IO offloaded to an actual hardware block is... A different type of amusing.
Everyone complained when async IO was done with callbacks, so sugar was added to the callbacks, and now everyone has spent over a decade complaining about what flavor of sugar tastes best.
Y'all at Zig have a solution, I trust Zig's solution will be a good one (zig is lots of fun to use as a language) but at the end of the day, IO is slow, that needs to get hidden somehow, or not.
Everyone should have to do embedded for awhile and setup their own DMA controller operations. Having async IO offloaded to an actual hardware block is... A different type of amusing.
> Well, you don't have async/sync/red/blue anymore, but you now have IO and non-IO functions.
> However, the coloring problem hasn't really been defeated.
Well, yes, but if the only way to do I/O were to have an Io instance to do it with then Io would infect all but pure(ish, non-Io) functions, so calling Io functions would be possible in all but those contexts where calling Io functions is explicitly something you don't want to be possible.
So in a way the color problem is lessened.
And on top of that you get something like Haskell's IO monad (ok, no monad, but an IO interface). Not too shabby, though you're right of course.
Next Zig will want monadic interfaces so that functions only have to have one special argument that can then be hidden.
> However, the coloring problem hasn't really been defeated.
Well, yes, but if the only way to do I/O were to have an Io instance to do it with then Io would infect all but pure(ish, non-Io) functions, so calling Io functions would be possible in all but those contexts where calling Io functions is explicitly something you don't want to be possible.
So in a way the color problem is lessened.
And on top of that you get something like Haskell's IO monad (ok, no monad, but an IO interface). Not too shabby, though you're right of course.
Next Zig will want monadic interfaces so that functions only have to have one special argument that can then be hidden.
Zig's not really about hiding things but you could put it in an options struct that has defaults unless overridden at compile time.
> Technically you could pass in a new executor, but is that really what you want?
why does it have to be new? just use one executor, set it as const in some file, and use that one at every entrypoint that needs io! now your io doesn't propagate downwards.
why does it have to be new? just use one executor, set it as const in some file, and use that one at every entrypoint that needs io! now your io doesn't propagate downwards.
> If anything, it does a great job at abstracting the usage from the implementation. This is something Rust fails at spectacularly.
Could you expand on this? I don't get what you mean
Could you expand on this? I don't get what you mean
I am not very experienced in async Rust, but it seems there are some pieces of async Rust that rely too much on tokio internals, so using an alternative runtime (like pollster) results in broken code.
Searching for comments mentioning "pollster" and "tokio" on HN brings a few results, but not one I recall seeing a while ago where someone demonstrated an example of a library (using async Rust) that crashes when not using tokio as the executor.
Related documentation: https://rust-lang.github.io/async-book/08_ecosystem/00_chapt...
Searching for comments mentioning "pollster" and "tokio" on HN brings a few results, but not one I recall seeing a while ago where someone demonstrated an example of a library (using async Rust) that crashes when not using tokio as the executor.
Related documentation: https://rust-lang.github.io/async-book/08_ecosystem/00_chapt...
There's two details that are important to highlight. tokio is actually 2 components, it's the async scheduler, and it's the IO runtime. Pollster is only a scheduler, and does not offer any IO functionality. You can actually use tokio libraries with pollster, but you need to register the IO runtime (and spawn a thread to manage it) - this is done with Runtime::enter() and it configures the thread local interface so any uses of tokio IO know what runtime to use.
There are ideas to abstract the IO runtime interface into the async machinery (in Rust that's the Context object that schedulers pass into the Future) but so far that hasn't gotten anywhere.
There are ideas to abstract the IO runtime interface into the async machinery (in Rust that's the Context object that schedulers pass into the Future) but so far that hasn't gotten anywhere.
Yep. The old async wars in the Rust ecosystem. It's the AsyncRead and AsyncWrite traits. Tokio has its own, there was a standard brewing at the same time in the futures crate. Tokio did their own thing, people burnt out, these traits were never standardized to std.
So you cannot use most of the async crates easily outside Tokio.
So you cannot use most of the async crates easily outside Tokio.
[deleted]
thanks, got it!
Sure. Let's do an imaginary scenario. Let's say that you are the author of a http request library.
Async hasn't been added yet, so you're using `std::net::TcpStream`.
All is well until async comes along. Now, you have a problem. If you use async, your previous sync users won't be able to (easily) call your functions. You're looking at an API redesign.
So, you swallow your pride and add an async variant of your functionality. Since Tokio is most popular, you use `tokio::net::TcpStream`.
All is well, until a user comes in and says "Hey, I would like to use your library with smol (a different async runtime)". Now what do you do? Add a third variant of your code using `smol::net::TcpStream`? It's getting a bit ridiculous, and smol isn't the only alternative runtime.
One solution is to do what Zig does, but there isn't really an agreed upon solution. The stdlib does not even provide AsyncRead/AsyncWrite so you could invert your code and just work with streams provided from above and keep your libary executor agnostic.
Async hasn't been added yet, so you're using `std::net::TcpStream`.
All is well until async comes along. Now, you have a problem. If you use async, your previous sync users won't be able to (easily) call your functions. You're looking at an API redesign.
So, you swallow your pride and add an async variant of your functionality. Since Tokio is most popular, you use `tokio::net::TcpStream`.
All is well, until a user comes in and says "Hey, I would like to use your library with smol (a different async runtime)". Now what do you do? Add a third variant of your code using `smol::net::TcpStream`? It's getting a bit ridiculous, and smol isn't the only alternative runtime.
One solution is to do what Zig does, but there isn't really an agreed upon solution. The stdlib does not even provide AsyncRead/AsyncWrite so you could invert your code and just work with streams provided from above and keep your libary executor agnostic.
I think of it this way.
Given an `io` you can, technically, build another one from it with the same interface.
For example given an async IO runime, you could create an `io` object that is blocking (awaits every command eagerly). That's not too special - you can call sync functions from async functions. (But in JavaScript you'd have trouble calling a sync function that relies on `await`s inside, so that's still something).
Another thing that is interesting is given a blocking posix I/O that also allows for creating processes or threads, you could build in userspace a truly asynchronous `io` object from that blocking one. It wouldn't be as efficient as one based directly on iouring, and it would be old school, but it would basically work.
Going either way (changing `io` to sync or async) the caller doesn't actually care. Yes the caller needs a context, but most modern apps rely on some form of dependency injection. Most well-factored apps would probably benefit from a more refined and domain-specific "environment" (or set of platform effects, perhaps to use the Roc terminology), not Zig's posix-flavoured standard library `io` thing.
Yes rust achieves this to some extent; you can swap an async runtime for another and your app might still compile and run fine.
Overall I like this alot - I am wondering if Richard Feldmann managed to convince Andrew Kelley that "platforms" are cool and some ideas were borrowed from Roc?
Given an `io` you can, technically, build another one from it with the same interface.
For example given an async IO runime, you could create an `io` object that is blocking (awaits every command eagerly). That's not too special - you can call sync functions from async functions. (But in JavaScript you'd have trouble calling a sync function that relies on `await`s inside, so that's still something).
Another thing that is interesting is given a blocking posix I/O that also allows for creating processes or threads, you could build in userspace a truly asynchronous `io` object from that blocking one. It wouldn't be as efficient as one based directly on iouring, and it would be old school, but it would basically work.
Going either way (changing `io` to sync or async) the caller doesn't actually care. Yes the caller needs a context, but most modern apps rely on some form of dependency injection. Most well-factored apps would probably benefit from a more refined and domain-specific "environment" (or set of platform effects, perhaps to use the Roc terminology), not Zig's posix-flavoured standard library `io` thing.
Yes rust achieves this to some extent; you can swap an async runtime for another and your app might still compile and run fine.
Overall I like this alot - I am wondering if Richard Feldmann managed to convince Andrew Kelley that "platforms" are cool and some ideas were borrowed from Roc?
> but most modern apps rely on some form of dependency injection
Does Zig actually do anything here? If anything, this seems to be anti-Zig, where everything must be explicit.
Does Zig actually do anything here? If anything, this seems to be anti-Zig, where everything must be explicit.
Passing in your dependencies as function arguments is a form of dependency injection. It is the simplest and thus arguably best form of dependency injection.
This is like saying arithmetic is a form of calculus, the simplest form. Ie it reduces the concept (DI) to a meaningless tautology.
Not at all. Dependency injection is the injection of dependencies as logical parameters. The simplest and arguably best way to inject logical parameters to a segment of code is to use function parameters. You can have a complicated DI framework that involves Java class annotations and megabytes of XML, but that’s not the central idea.
> inject logical parameters to a segment of code is to use function parameters. You can have a complicated DI framework that involves Java class annotations and megabytes of XML, but that’s not the central idea.
so the central idea of dependency injection, a concept with a wiki of like 5000 words [1], is just pass parameters to functions...?
i guess i'm happy to accept that (i personally DGAF about DI or whatever) but it certainly means that all the people discussing DI (like yourself) are peddling snake oil...?
[1] https://en.wikipedia.org/wiki/Dependency_injection
so the central idea of dependency injection, a concept with a wiki of like 5000 words [1], is just pass parameters to functions...?
i guess i'm happy to accept that (i personally DGAF about DI or whatever) but it certainly means that all the people discussing DI (like yourself) are peddling snake oil...?
[1] https://en.wikipedia.org/wiki/Dependency_injection
> so the central idea of dependency injection, a concept with a wiki of like 5000 words [1], is just pass parameters to functions...?
Specifically, parameters that refer to a dependency that is being injected.
And yeah, there are a lot of fundamentally simple ideas that can be massively overcomplicated. Let’s take a look at that Wikipedia article:
> There are several ways in which a client can receive injected services:[29]
> * Constructor injection, where dependencies are provided through a client's class constructor.
> * Method Injection, where dependencies are provided to a method only when required for specific functionality.
> * Setter injection, where the client exposes a setter method which accepts the dependency.
> * Interface injection, where the dependency's interface provides an injector method that will inject the dependency into any client passed to it.
You’ll notice that these are more or less an enumeration of ways to pass parameters into functions in object oriented programming. Most of the complexity isn’t the idea of dependency injection itself but rather in building abstractions for doing dependency injection, especially in an object-oriented language.
And yeah, a lot of the complicated versions of DI, like Spring, probably are mostly snake oil. But I object to the notion that I am peddling snake oil because I’m not advocating for anything like Spring or claiming that DI is anything more than parameterization.
Specifically, parameters that refer to a dependency that is being injected.
And yeah, there are a lot of fundamentally simple ideas that can be massively overcomplicated. Let’s take a look at that Wikipedia article:
> There are several ways in which a client can receive injected services:[29]
> * Constructor injection, where dependencies are provided through a client's class constructor.
> * Method Injection, where dependencies are provided to a method only when required for specific functionality.
> * Setter injection, where the client exposes a setter method which accepts the dependency.
> * Interface injection, where the dependency's interface provides an injector method that will inject the dependency into any client passed to it.
You’ll notice that these are more or less an enumeration of ways to pass parameters into functions in object oriented programming. Most of the complexity isn’t the idea of dependency injection itself but rather in building abstractions for doing dependency injection, especially in an object-oriented language.
And yeah, a lot of the complicated versions of DI, like Spring, probably are mostly snake oil. But I object to the notion that I am peddling snake oil because I’m not advocating for anything like Spring or claiming that DI is anything more than parameterization.
> so the central idea of dependency injection, a concept with a wiki of like 5000 words [1], is just pass parameters to functions...
... instead of functions reaching out to obtain those dependencies from globals. Yes, that is exactly what it is about. Like much of 90-00s era OOP design discourse, it's vastly overcomplicated for no good reason.
... instead of functions reaching out to obtain those dependencies from globals. Yes, that is exactly what it is about. Like much of 90-00s era OOP design discourse, it's vastly overcomplicated for no good reason.
I'm sorry but if I read
> most modern apps rely on some form of dependency injection
then plain parameter passing is not what I'm thinking of. Especially not manually doing that to hundreds or thousands of calls.
> most modern apps rely on some form of dependency injection
then plain parameter passing is not what I'm thinking of. Especially not manually doing that to hundreds or thousands of calls.
Most forms of dependency injection are abstractions over parameter passing. If you find yourself passing the same parameters into many different function calls there are ways of abstracting that, even in Zig. It’s not going to look like Spring and if that’s a dealbreaker for you, just use Spring, it’s a free country.
Does Zig have closures? If yes, than at least in that case the IO pointer can be a bound parameter. For languages with the async keyword function coloring also applied to closures.
The original “function colouring” blogpost has done irreparable damage to PL discussions because it’s such a stupid concept to begin with. Of course I want async functions to be “coloured” differently, they do different things! How else is a “normal function” supposed to call a function that gives you a result later——obviously you want to be forced to say what to do with the result; await it, ignore it, .then() in JS terms, etc. these are important decisions that you can’t just ignore because it’s “painful”
There is nothing obvious around that - it is driven by what abstractions the language provides related to concurrency, and with different choices you will end needing different ways to interact with it.
So yes, given how the language designers of C# and JavaScript choose to implement concurrency and the APIs around that, then coloring is necessary. But it is very much implementation driven and implementation of other concurrency models then other ways to do it that don't involve keywords can make sense. So when people complain about function coloring, they are complaining about the choice of concurrency model that a language uses.
So yes, given how the language designers of C# and JavaScript choose to implement concurrency and the APIs around that, then coloring is necessary. But it is very much implementation driven and implementation of other concurrency models then other ways to do it that don't involve keywords can make sense. So when people complain about function coloring, they are complaining about the choice of concurrency model that a language uses.
I have a much longer rant elsethread, but the tl;dr; is:
In some languages red can call blue, but blue cannot call red (JS). In some other languages blue can call red, but the resulting combined function is blue (traditional async with optional blocking). Finally some languages allow blue to call red and having the resulting combined function to be red (lua, scheme, go, and I believe Zig). As color is no longer a n unabstractable restriction in these languages, it no different than other kind of typing.
In some languages red can call blue, but blue cannot call red (JS). In some other languages blue can call red, but the resulting combined function is blue (traditional async with optional blocking). Finally some languages allow blue to call red and having the resulting combined function to be red (lua, scheme, go, and I believe Zig). As color is no longer a n unabstractable restriction in these languages, it no different than other kind of typing.
I think the point is 3 doesn't fully apply anymore. And that was the main pain point. You couldn't call a blue in red even if it didn't use IO without some kind of execution wrapper or waiter. Now you clearly can.
Note that this same concept is "sans io" and was previously discussed for it's use in Rust:
https://www.firezone.dev/blog/sans-io
https://sans-io.readthedocs.io/
https://news.ycombinator.com/item?id=40872020
https://www.firezone.dev/blog/sans-io
https://sans-io.readthedocs.io/
https://news.ycombinator.com/item?id=40872020
If the functions are still calling I/O methods directly rather than the I/O being externally driven, I don't think that qualifies as sans-io, based on my previous exposure / based on your second link:
> For byte-stream based protocols, the protocol implementation can use a single input buffer and a single output buffer. For input (that is, receiving data from the network), the calling code is responsible for delivering code to the implementation via a single input (often via a method called receive_bytes, or something similar). The implementation will then append these bytes to its internal byte buffer. At this point, it can choose to either eagerly process those bytes, or do so lazily at the behest of the calling code.
> When it comes to generating output, a byte-stream based protocol has two options. It can either write its bytes to an internal buffer and provide an API for extracting bytes from that buffer, as done by hyper-h2, or it can return bytes directly when the calling code triggers events (more on this later), as done by h11. The distinction between these two choices is not enormously important, as one can easily be transformed into the other, but using an internal byte buffer is recommended if it is possible that the act of receiving input bytes can cause output bytes to be produced: that is, if the protocol implementation sometimes automatically responds to the peer without user input.
> For byte-stream based protocols, the protocol implementation can use a single input buffer and a single output buffer. For input (that is, receiving data from the network), the calling code is responsible for delivering code to the implementation via a single input (often via a method called receive_bytes, or something similar). The implementation will then append these bytes to its internal byte buffer. At this point, it can choose to either eagerly process those bytes, or do so lazily at the behest of the calling code.
> When it comes to generating output, a byte-stream based protocol has two options. It can either write its bytes to an internal buffer and provide an API for extracting bytes from that buffer, as done by hyper-h2, or it can return bytes directly when the calling code triggers events (more on this later), as done by h11. The distinction between these two choices is not enormously important, as one can easily be transformed into the other, but using an internal byte buffer is recommended if it is possible that the act of receiving input bytes can cause output bytes to be produced: that is, if the protocol implementation sometimes automatically responds to the peer without user input.
Ah good point -- sans I/O as described in that second link is a bit more narrow than what Zig is doing here. The sans I/O discussed there is more for protocols specifically and less for general I/O.
I guess a better name for this approach might be "explicitly managed I/O".
I guess a better name for this approach might be "explicitly managed I/O".
Yep, that would be more like structured concurrency also mentioned in linked blog post.
sans-io is about state machine as interface, but unfortunately does not specify a formal model or how to synthesize/derive one etc.
I'm generally a fan of Zig, but it's a little sad seeing them go all in on green threads (aka fibers, aka stackful coroutines). Rust got rid of their Runtime trait (the rough equivalent of Zig's Io) before 1.0 because it performed badly. Languages and OS's have had to learn this lesson the hard way over and over again:
https://www.open-std.org/JTC1/SC22/WG21/docs/papers/2018/p13...
> While fibers may have looked like an attractive approach to write scalable concurrent code in the 90s, the experience of using fibers, the advances in operating systems, hardware and compiler technology (stackless coroutines), made them no longer a recommended facility.
If they go through with this, Zig will probably top out at "only as fast as Go", instead of being a true performance competitor. I at least hope the old std.fs sticks around for cases where performance matters.
https://www.open-std.org/JTC1/SC22/WG21/docs/papers/2018/p13...
> While fibers may have looked like an attractive approach to write scalable concurrent code in the 90s, the experience of using fibers, the advances in operating systems, hardware and compiler technology (stackless coroutines), made them no longer a recommended facility.
If they go through with this, Zig will probably top out at "only as fast as Go", instead of being a true performance competitor. I at least hope the old std.fs sticks around for cases where performance matters.
I'm not sure how you got the perception that we're going "all in" on green threads, given that the article in OP explicitly mentions that we're hoping to have an implementation based on stackless coroutines, based on this Zig language proposal: https://github.com/ziglang/zig/issues/23446
Performance matters; we're not planning to forget that. If fibers turn out to have unacceptable performance characteristics, then they won't become a widely used implementation. Nothing discussed in this article precludes stackless coroutines from backing the "general purpose" `Io` implementation if that turns out to be the best approach.
Performance matters; we're not planning to forget that. If fibers turn out to have unacceptable performance characteristics, then they won't become a widely used implementation. Nothing discussed in this article precludes stackless coroutines from backing the "general purpose" `Io` implementation if that turns out to be the best approach.
That is lovely to hear. I think the general conscious is that not a single programming language has done Async right. So people are a little sceptical. But Andrew and the team so far seems to have the do it right mentality. So I guess people should be a little more optimistic.
Cant wait for 0.15 coming out soon.
Cant wait for 0.15 coming out soon.
I would argue that Java's async is pretty nifty, especially now that some of the rough edges have been sanded off.
What is wrong with async in JS?
You have to write JS
single threaded, memory requirements, function coloring
> single threaded
The entire language is single thread. But I/O uses a separate thread pool.
> memory usage
Are you talking about extra 120 bytes per Promise?
> function coloring
How does it manifest in JS? You can `await` non-async function without any issues, anything potentially async is awaited, if it doesn't end up doing async inside there is no problem.
The entire language is single thread. But I/O uses a separate thread pool.
> memory usage
Are you talking about extra 120 bytes per Promise?
> function coloring
How does it manifest in JS? You can `await` non-async function without any issues, anything potentially async is awaited, if it doesn't end up doing async inside there is no problem.
Does the BDFL want this though, or is it just one person's opinion that it might be nice? Given how he has been aggressively pruning proposals, I don't put any hope in them anymore unless I see some kind of positive signal from him directly.
e.g. I'd feel a lot more confident if he had made the coroutine language feature a hard dependency of the writergate refactor.
e.g. I'd feel a lot more confident if he had made the coroutine language feature a hard dependency of the writergate refactor.
mlugg is in the Zig core team
I'm aware, but Zig isn't a democracy where the core team votes, right? Has Andrew actually expressed that he wants the proposal? Without that we're left with scraps like this commit message where he seems ambivalent. https://github.com/ziglang/zig/commit/d6c90ceb04f8eda7c6b711...
Andrew, I know you read these threads sometimes, give us a sign so I can go down the mountain with my stone tablets and tell the people whether we'll have coroutines
Andrew, I know you read these threads sometimes, give us a sign so I can go down the mountain with my stone tablets and tell the people whether we'll have coroutines
We don't know whether or not we'll have stackless coroutines; it's possible that we hit design problems we didn't foresee. However, at this moment, the general consensus is that we are interested in pursuing stackless coroutines.
While Andrew has the final say, as Loris points out, we always work to reach a consensus internally. The article lists this an an implementation that will probably exist, because we agree that it probably will; nobody is promising it, because we also agree that it isn't guaranteed.
Also, bear in mind that even if stackless coroutines don't make it into Zig, you can always use a single-threaded blocking implementation of `Io`, so you need not be negatively affected by any potential downsides to fibers either way.
This new `Io` approach has made it strictly more likely than it previously was that stackless coroutines become a part of Zig's final design.
While Andrew has the final say, as Loris points out, we always work to reach a consensus internally. The article lists this an an implementation that will probably exist, because we agree that it probably will; nobody is promising it, because we also agree that it isn't guaranteed.
Also, bear in mind that even if stackless coroutines don't make it into Zig, you can always use a single-threaded blocking implementation of `Io`, so you need not be negatively affected by any potential downsides to fibers either way.
This new `Io` approach has made it strictly more likely than it previously was that stackless coroutines become a part of Zig's final design.
But how will that actually work? Your stackless coroutines proposal talks about explicit primitives for defining a coroutine. But what about a function that's not designed for any particular implementation strategy - it just takes an Io and passes it on to some other functions? Will the compiler have a way to compile it as either sync or async, like apparently it did before? It would have to, if you want to avoid function colors. But your proposal doesn't explain anything about that.
Disclaimer: I'm not actually a Zig user, but I am very interested in the design space.
Disclaimer: I'm not actually a Zig user, but I am very interested in the design space.
Right, the proposal doesn't discuss the implementation details -- I do apologise if that made it seem a little hand-wavey. I opted not to discuss them there, because they're similar-ish to the way we lowered stackless async in its stage1 implementation, and hence not massively interesting to discuss.
The idea is that, yes, the compiler will infer whether or not a function is async (in the stackless async sense) based on whether it has any "suspension point", where a suspension point is either: * Usage of `@asyncSuspend` * A call to another async function
Calls through function pointers (where we typically wouldn't know what we're calling, and hence don't know whether or not it's async!) are handled by a new language feature which has already been accepted; see a comment I left a moment ago [1] for details on that.
If the compiler infers a function to be async, it will lower it differently; with each suspension point becoming a boundary where any stack-local state is saved to the async frame, as well as an integer indicating where we are in the function, and we jump to different code to be resumed once it finishes. The details of this depend on specifics of the proposal (which I'm planning to change soon) and sometimes melt my brain a little, so I'll leave them unexplained for now, but can probably elaborate on them in the issue thread at some point.
Of course, this analysis of whether a function is async is a little bit awkward, because it is a whole-program analysis; a change in a leaf function in a little file in a random helper module could introduce asynchronocity which propagates all the way up to your `pub fn main`. As such, we'll probably have different strategies for this inference in the compiler depending on the release mode:
* In Debug mode, it may be a reasonable strategy to just assume that (almost) all functions are asynchronous (it's safe to lower a synchronous function as asynchronous, just not vice versa). The overhead introduced by the async lowering will probably be fairly minimal in the context of a Debug build, and this will speed up build times by allowing functions to be sent straight to the code generator (like they are today) without having to wait for other functions to be analyzed (and without potentially having to codegen again later if we "guessed wrong").
* In Release[Fast,Small,Safe] mode, we might hold back code generation until we know for sure, based on the parts of the call graph we have analyzed, whether or not a function is async. Vtables might be a bit of a problem here, since we don't know for sure that a vtable call is not async until we've finished literally all semantic analysis. Perhaps we'll make a guess about whether such functions are async and re-do codegen later if that guess was wrong. Or, in the worst case... perhaps we'll literally just defer all codegen until semantic analysis completes! After all, it's a release build, so you're going to be waiting a while for optimizations anyway; you won't mind an extra couple of seconds on delayed codegen.
[1]: https://news.ycombinator.com/item?id=44549131
The idea is that, yes, the compiler will infer whether or not a function is async (in the stackless async sense) based on whether it has any "suspension point", where a suspension point is either: * Usage of `@asyncSuspend` * A call to another async function
Calls through function pointers (where we typically wouldn't know what we're calling, and hence don't know whether or not it's async!) are handled by a new language feature which has already been accepted; see a comment I left a moment ago [1] for details on that.
If the compiler infers a function to be async, it will lower it differently; with each suspension point becoming a boundary where any stack-local state is saved to the async frame, as well as an integer indicating where we are in the function, and we jump to different code to be resumed once it finishes. The details of this depend on specifics of the proposal (which I'm planning to change soon) and sometimes melt my brain a little, so I'll leave them unexplained for now, but can probably elaborate on them in the issue thread at some point.
Of course, this analysis of whether a function is async is a little bit awkward, because it is a whole-program analysis; a change in a leaf function in a little file in a random helper module could introduce asynchronocity which propagates all the way up to your `pub fn main`. As such, we'll probably have different strategies for this inference in the compiler depending on the release mode:
* In Debug mode, it may be a reasonable strategy to just assume that (almost) all functions are asynchronous (it's safe to lower a synchronous function as asynchronous, just not vice versa). The overhead introduced by the async lowering will probably be fairly minimal in the context of a Debug build, and this will speed up build times by allowing functions to be sent straight to the code generator (like they are today) without having to wait for other functions to be analyzed (and without potentially having to codegen again later if we "guessed wrong").
* In Release[Fast,Small,Safe] mode, we might hold back code generation until we know for sure, based on the parts of the call graph we have analyzed, whether or not a function is async. Vtables might be a bit of a problem here, since we don't know for sure that a vtable call is not async until we've finished literally all semantic analysis. Perhaps we'll make a guess about whether such functions are async and re-do codegen later if that guess was wrong. Or, in the worst case... perhaps we'll literally just defer all codegen until semantic analysis completes! After all, it's a release build, so you're going to be waiting a while for optimizations anyway; you won't mind an extra couple of seconds on delayed codegen.
[1]: https://news.ycombinator.com/item?id=44549131
> a change in a leaf function in a little file in a random helper module could introduce asynchronocity which propagates all the way up to your `pub fn main`
If this doesn't make the argument that Zig has certainly not defeated function coloring, I don't know what would.
The fact that this change to how my program runs is hidden away in the compiler instead of somewhere visible is not an improvement.
If this doesn't make the argument that Zig has certainly not defeated function coloring, I don't know what would.
The fact that this change to how my program runs is hidden away in the compiler instead of somewhere visible is not an improvement.
> If this doesn't make the argument that Zig has certainly not defeated function coloring, I don't know what would.
if you're opting into stackless coroutines then yeah you're opting into their viral nature, but the point is that you don't have to. as the application author your dependencies won't opt you forcefully in using stackless coroutines (or any singular execution model), which is currently the case with other languages.
this is what it means to defeat function coloring.
if you're opting into stackless coroutines then yeah you're opting into their viral nature, but the point is that you don't have to. as the application author your dependencies won't opt you forcefully in using stackless coroutines (or any singular execution model), which is currently the case with other languages.
this is what it means to defeat function coloring.
Small marketing suggestion: maybe "limit function coloring" instead of "defeat function coloring". I like Zig's approach so far, but clearer terms would help avoid pointless arguments and disappointments that a certain proglang supremacist community loves.
> it's safe to lower a synchronous function as asynchronous
Is it though? I believe this would be an issue if you want to pass that function as a function pointer to an FFI function, in which case it must be sync.
Is it though? I believe this would be an issue if you want to pass that function as a function pointer to an FFI function, in which case it must be sync.
we do build internal consensus before publishing articles like this one, or doing other public communication.
I’m confused about the assertion that green threads perform badly. 3 of the top platforms for high concurrency servers use or plan to use green threads (Go, Erlang, Java). My understanding was that green threads have limitations with C FFI which is why lower level languages don’t use them (Rust). Rust may also have performance concerns since it has other constraints to deal with.
Green threads have issues with C FFI mostly due to not being able to preempt execution, when the C thing is doing something that blocks. This is a problem when you have one global pool of threads that execute everything. To get around it you essentially need to set up a dedicated thread pool to handle those c calls.
Which may be fine - go doesn't let the user directly create thread pools directly but do create one under the hood for ffi interaction.
Which may be fine - go doesn't let the user directly create thread pools directly but do create one under the hood for ffi interaction.
The problem is when C calls expect to be on a particular thread. Either the main event thread, in a GUI context, or the same thread as some related previous call.
The problem is that C expects to have enough stack to put stuff there, but green threads allocate small stacks to reduce (virtual) memory use.
That’s not a problem when a dedicated thread pool is used as mentioned by GP. However they don’t solve the thread affinity issue.
That's the reason for the thread pool, and the resulting thread affinity issue.
This seems to assume that the same green thread will always run on the same native thread, which I don’t think is universally the case.
Different languages give different level of control over that. There are languages with one main thread pool and perhaps some specialized ones that users don't have control over. Go would be an example of this.
It is also possible for languages to make user creatable thread pools - possibly even with affinity to cores, allowing fibers to run only on a single thread. Crystal is coming along that path. So far it seems to be coming around fairly nicely but I havn't had to battle the GC in anger yet.
It is also possible for languages to make user creatable thread pools - possibly even with affinity to cores, allowing fibers to run only on a single thread. Crystal is coming along that path. So far it seems to be coming around fairly nicely but I havn't had to battle the GC in anger yet.
I don't understand why that matters.
It actually has much the same benefits of Rust removing green threads and replacing them with a generic async runtime.
The point here is that "async stuff is IO stuff is async stuff". So rather than thinking of having pluggable async runtimes (tokio, etc) Zig is going with pluggable IO runtimes (which is kinda the equivalent of "which subset of libc do you want to use?").
But in both moves the idea is to remove the runtime out of the language and into userspace, while still providing a common pluggable interface so everyone shares some common ground.
The point here is that "async stuff is IO stuff is async stuff". So rather than thinking of having pluggable async runtimes (tokio, etc) Zig is going with pluggable IO runtimes (which is kinda the equivalent of "which subset of libc do you want to use?").
But in both moves the idea is to remove the runtime out of the language and into userspace, while still providing a common pluggable interface so everyone shares some common ground.
That paper (P1364R0) was contentious, and I regard it as being severely motivated reasoning, published only to kill off competing approaches to C++ coroutines.
Some discussions of that paper:
* https://old.reddit.com/r/cpp/comments/1jwlur9/stackful_corou...
* https://old.reddit.com/r/programming/comments/dgfxde/fibers_...
Some discussions of that paper:
* https://old.reddit.com/r/cpp/comments/1jwlur9/stackful_corou...
* https://old.reddit.com/r/programming/comments/dgfxde/fibers_...
It's hardly "all-in" if it is merely one choice of many, and the choice is made in the executable not in the library code.
I have definitely gotten the impression that green threads will be the favored implementation, from listening to core team members and hanging around the discord. Stackless coroutines don't even exist in the language currently.
What does "favored" mean if event loop and direct blocking are relatively trivial and provided also/ If I can trivially use them, what do I care what Andrew or someone in core thinks? The control is all mine, and near zero cost (potential vtable indirection).
And would Rust be "all-in" if tokio was in std, so you could use its tasks everywhere? That would be a very similar level of "all-in" to Zig's current plan, but with a seemingly better API.
I understand the benefit of not being in std, but really not a fundamental issue, IMO.
And would Rust be "all-in" if tokio was in std, so you could use its tasks everywhere? That would be a very similar level of "all-in" to Zig's current plan, but with a seemingly better API.
I understand the benefit of not being in std, but really not a fundamental issue, IMO.
In the 2026 roadmap talk Andrew Kelley spoke of the fact that stackless coroutines with iouring is the end goal here (but the requires an orthogonal improvement in the compiler for inlining that data to the stack where possible).
Do you have the timestamp? I watched that video when it came out and don't remember hearing it.
> Stackless coroutines don't even exist in the language currently.
And green thread exists in language?
And green thread exists in language?
[deleted]
Oh man. I think the biggest mistake Rust has ever made is their async model. It’s been nothing short of a disaster. Zig supporting green threads and other options is spectacularly exciting. Thus far no one has “solved” async. So very exciting to see what Zig can come up with.
> I'm generally a fan of Zig, but it's a little sad seeing them go all in on green threads
Read the article, you can use whatever approach you want by writing an implementation for the IO interface, green threading is just one of them.
Read the article, you can use whatever approach you want by writing an implementation for the IO interface, green threading is just one of them.
Seeing a systems language like Zig require runtime polymorphism for something as common as standard IO operations seems off to me -- why force that runtime overhead on everyone when the concrete IO implementation could be known statically in almost all practical cases?
I/O strikes me as one place where dynamic dispatch overhead would likely be negligible in practice. Obviously it depends on the I/O target and would need to be measured, but they don't call them "I/O bound" (as opposed to "CPU bound") programs for no reason.
> why force that runtime overhead on everyone
pretty sure the intent is for systems that only use one io to have a compiler optimization that elides the cost of double indirection... but also, you're doing IO! so usually something else is the bottleneck, one extra indirection is likely to be peanuts.
pretty sure the intent is for systems that only use one io to have a compiler optimization that elides the cost of double indirection... but also, you're doing IO! so usually something else is the bottleneck, one extra indirection is likely to be peanuts.
I think it's just the Zig philosophy to care more about binary size than speed. Allocators have the same tradeoff, ArrayListUnmanaged is not generic over the allocator, so every allocation uses dynamic dispatch. In practice the overhead of allocating or writing a file will dwarf the overhead of an indirect call. Can't argue with those binary sizes.
(And before anyone mentions it, devirtualization is a myth, sorry)
(And before anyone mentions it, devirtualization is a myth, sorry)
> (And before anyone mentions it, devirtualization is a myth, sorry)
In Zig it's going to be a language feature, thanks to its single unit compilation model.
https://github.com/ziglang/zig/issues/23367
In Zig it's going to be a language feature, thanks to its single unit compilation model.
https://github.com/ziglang/zig/issues/23367
Wouldn't this only work if there's only one implementation throughout the entire compliation unit? If you use 2 allocators in your app, your restricted function type has 2 possible callees for each entry, and you're back to the same problem.
> A side effect of proposal #23367, which is needed for determining upper bound stack size, is guaranteed de-virtualization when there is only one Io implementation being used (also in debug builds!).
> In the less common case when a program instantiates more than one Io implementation, virtual calls done through the Io interface will not be de-virtualized, as that would imply doubling the amount of machine code generated, creating massive code bloat.
From the article
> In the less common case when a program instantiates more than one Io implementation, virtual calls done through the Io interface will not be de-virtualized, as that would imply doubling the amount of machine code generated, creating massive code bloat.
From the article
I wonder how massive it actually would be. I'm guessing it really wouldn't be all that massive in practice even if it of course is easy to create massive examples using ways people typically don't write code.
Having a limited number of known callees is already better than a virtual function (unrestricted function pointer). A compiler in theory could devirtualize every two-possible-callee callsite into `if (function_pointer == callee1) callee1() else callee2()` which then can be inlined at compile time or branch-predicted at runtime.
In any case, if you have two different implementations of something then you have to switch between them somewhere -- either at compile-time or link-time or load-time or run-time (or jit-time). The trick is to find an acceptable compromise of performance, (machine)code-bloat and API-simplicity.
In any case, if you have two different implementations of something then you have to switch between them somewhere -- either at compile-time or link-time or load-time or run-time (or jit-time). The trick is to find an acceptable compromise of performance, (machine)code-bloat and API-simplicity.
> Wouldn't this only work if there's only one implementation throughout the entire compliation unit
in practice how often are people using more than one io in a program?
in practice how often are people using more than one io in a program?
I think having a thread pool on top of some evented IO isn't _that_ uncommon.
You might have a thread pool doing some very specific thing. You can do your own threadpool which wont use the Io interface. But if one of the tasks in the threadpool wanted to read a file, I guess you'd have to pass in the blocking Io implementation.
You might have a thread pool doing some very specific thing. You can do your own threadpool which wont use the Io interface. But if one of the tasks in the threadpool wanted to read a file, I guess you'd have to pass in the blocking Io implementation.
one of the io interfaces provided is a standard threadpool io. and if it was really important, you could write your own io interface that selects between std threadpool and std blocking based off of an option (i am guessing, i don't know, but seems reasonable)
In larger Rust applications or servers I find myself doing this very often -- for example, one application I'm working on mostly uses blocking I/O for occasional filesystem access but has a little bit of async networking.
It can also mean faster compilation (and sometimes better performance? https://nical.github.io/posts/rust-custom-allocators.html)
Just templating everything doesn’t mean it will be faster every time
Just templating everything doesn’t mean it will be faster every time
> care more about binary size than speed
That does not seem to be true if you look at how string formatting is implemented.
That does not seem to be true if you look at how string formatting is implemented.
Runtime polymorphism isn’t something inherently bad.
It is bad if you are introducing branching in a tight loop or preventing compiler from inlining things it would inline otherwise and other similar things maybe?
It is bad if you are introducing branching in a tight loop or preventing compiler from inlining things it would inline otherwise and other similar things maybe?
I'm confused. The trouble with "colored" functions is that they either do processing on the stack, or unwind the stack. They claim defeat of function coloring, and describe that IO implementation can use blocking/thread pool/green threads. But... these are all blocking methods, which weren't the problem in the first place! If you keep convention to never do IO using global state, you could do that practically in any language. Stackless coroutines being left for later feels like "draw the rest of the owl" situation.
To actually have truly universal functions, I think there are two solutions:
- Make every function async, and provide extra parameter indicating to not actually unwind the stack and execute synchronously instead. Comes with performance penalty.
- Compile each function twice, picking appropiate variant at call site. Increases code size and requires some hackery with handling function pointers.
To actually have truly universal functions, I think there are two solutions:
- Make every function async, and provide extra parameter indicating to not actually unwind the stack and execute synchronously instead. Comes with performance penalty.
- Compile each function twice, picking appropiate variant at call site. Increases code size and requires some hackery with handling function pointers.
I am not on the core team but i believe the plan is to do exactly what you are talking about, but after the API is nailed down and kinks have been ironed out by users of the existing semiblocking implementation (to possibly include the compiler), as the default LLVM coro state machine compiler has problems (for example: I think I remember Andrew complaining that it has an obligatory libc/malloc dependency).
since the new io interface has userland async/await methods, then dropping in a proper frame jumping solution will be less painful, and easier to debug, and if using coroutines proves to be challenging with the api hopefully changes to io api would be minor, versus going after stackless coroutines NOW and making large API changes often as the warts with the system uncover themselves.
since the new io interface has userland async/await methods, then dropping in a proper frame jumping solution will be less painful, and easier to debug, and if using coroutines proves to be challenging with the api hopefully changes to io api would be minor, versus going after stackless coroutines NOW and making large API changes often as the warts with the system uncover themselves.
> Make every function async, and provide extra parameter indicating to not actually unwind the stack and execute synchronously instead. Comes with performance penalty.
I think ValueTask<T> in C#/.NET can approach this use case - It avoids overhead if the method actually completes synchronously. Otherwise, you can get at the Task<T> if needed. From a code perspective, you await it like you normally would and the compiler/runtime figures out what to do.
I think ValueTask<T> in C#/.NET can approach this use case - It avoids overhead if the method actually completes synchronously. Otherwise, you can get at the Task<T> if needed. From a code perspective, you await it like you normally would and the compiler/runtime figures out what to do.
Although I'm not wild about the new `io` parameter popping up everywhere, I love the fact that it allows multiple implementations (thread based, fiber based, etc.) and avoids forcing the user to know and/or care about the implementation, much like the Allocator interface.
Overall, I think it's a win. Especially if there is a stdlib implementation that is a no overhead, bogstock, synchronous, blocking io implementation. It follows the "don't pay for things you don't use" attitude of the rest of zig.
Overall, I think it's a win. Especially if there is a stdlib implementation that is a no overhead, bogstock, synchronous, blocking io implementation. It follows the "don't pay for things you don't use" attitude of the rest of zig.
Isn’t “don’t pay for what you don’t use” a myth? Some other person will using unless you are a very small team with discipline, and you will pay for it.
Or just passing around an “io” is more work than just calling io functions where you want them.
Or just passing around an “io” is more work than just calling io functions where you want them.
> io.async expresses asynchronicity (the possibility for operations to happen out of order and still be correct) and it does not request concurrency, which in this case is necessary for the code to work correctly.
This is the key point for me. Regardless of whether you’re under an async event loop, you can specify that the order of your io calls does not imply sequencing. Brilliant. Separate what async means from what the io calls do.
This is the key point for me. Regardless of whether you’re under an async event loop, you can specify that the order of your io calls does not imply sequencing. Brilliant. Separate what async means from what the io calls do.
This is very well written, and very exciting! I especially love the implications for WebAssembly -- WASI in userspace? Bring your own IO? Why not both!
I like the IO interface simply for the fact that it would allow me to create language level vfs
Seeing the example code made me wonder if this would allow introducing capability based security. E.g. passing an `io` instance to a library which can only read a subtree of the filesystem.
Edit: not quite https://news.ycombinator.com/item?id=44549430
Edit: not quite https://news.ycombinator.com/item?id=44549430
Only if you are sure all the code uses the IO instance, if you mean language level sandboxing of untrusted code then no, zig code can always call syscalls directly. But you can compile zig to wasm which will give you capability based security.
I wrote a simple ssh server in zig to learn the language in my spare time.
The new design makes the event loop / io much easier to reason about. Thanks Andy
The new design makes the event loop / io much easier to reason about. Thanks Andy
Is there any chance you've published your project?
It would be fun to read the code.
I’ll finish it this summer, hopefully.
> The new design makes the event loop / io much easier to reason about.
Interesting. How so?
Interesting. How so?
Is this in effect introducing algebraic effects by concept? E.g. the io passed in is an effect handler, and it is the effect handler's choice whether to perform stack switching (or other means of non-blocking waiting) to enable asynchronicity?
In my view, algebraic effects enable specifying different kinds of effects (with different interpretations) — e.g. read a file, run DB query, network access — as opposed to just a single 'Io' effect that allows everything.
I think this design is a regression from the previous design, in which you could use compile time introspection to check whether things are actually async (calling convention) or not.
Additionally, I don't necessarily want to delegate the management of the memory backing the futures to an Io, or pass around a blob of syscalls and an associated runtime, which accesses everything via a vtable. I would prefer to have these things be compile time generic only.
Additionally, I don't necessarily want to delegate the management of the memory backing the futures to an Io, or pass around a blob of syscalls and an associated runtime, which accesses everything via a vtable. I would prefer to have these things be compile time generic only.
Your preference to have them be compile time generic shouldn’t come at the cost of those that would want runtime virtualisation.
As the article concludes, you get the best of both worlds here, where the result is effectively compile time generic if you only use one io implementation in your program. In theory it’d also partially compile time generic if you exclusively use one io for one set of libraries/functions and a different io for another set of libraries/functions.
I see this as the objectively correct design based on the existing design decisions in Zig. It follows from the allocator interface decision.
As the article concludes, you get the best of both worlds here, where the result is effectively compile time generic if you only use one io implementation in your program. In theory it’d also partially compile time generic if you exclusively use one io for one set of libraries/functions and a different io for another set of libraries/functions.
I see this as the objectively correct design based on the existing design decisions in Zig. It follows from the allocator interface decision.
Yes, I understand that the designers prefer the Allocator situation and that Reader and Writer being anytype was downstream of the difficulty of using async readers and writers otherwise. So the intention was always to go with the design that I do not prefer. One reason I do not prefer it is that the Reader and Writer interfaces were already staggeringly inefficient, despite the lack or virtualization. We have avoided the issue by reimplementing a bunch of their API in some specific readers and writers and modifying the stdlib Reader and Writer to dispatch to these methods if they are present.
To be honest, I just do not have much faith in the commitment to optimality, when it seems like the team has not spent time doing things like profiling a program that spends a lot of time formatting integers as decimal syrings, and noticing that the vast majority of that formatting runtime is UTF-8 validation. I am happy to continue using the language, because it makes it easy enough to fix these issues oneself.
The only aspect that may not be recoverable by the end user is the "am I async/is this async" reflection issue, though a core team member has clarified in this comment section that the code in the article is a sketch and the design of stackless coroutines is far from done, so we may yet get this.
Some other philosophical point is, like, lua's coroutine.create/resume/yield/clone are control flow primitives for use within a single thread of execution. It's fine to ship an async runtime, which embodies the view they they are not control flow primitives for use within a single thread of execution, for doing I/O. But focusing the primitives for creating and switching between execution contexts too narrowly on the async runtime use case is liable to he harmful to other use cases for these operations. Ideally, we would be able to write things like a prominent SNES emulator that uses stack switching to ensure the simulation of different components proceeds in an order known to be more correct than other orders, and we would be able to do it using native language features, which would compile down to something a bit cheaper than dumping all of our registers onto the stack. Ideally when we do this we would not be asked by the language to consider what it would mean to "cancel" the execution context managing one of the components, in the same way that we do not need to consider what it means to cancel an arbitrary struct, or the function which is calling the function currently executing.
To be honest, I just do not have much faith in the commitment to optimality, when it seems like the team has not spent time doing things like profiling a program that spends a lot of time formatting integers as decimal syrings, and noticing that the vast majority of that formatting runtime is UTF-8 validation. I am happy to continue using the language, because it makes it easy enough to fix these issues oneself.
The only aspect that may not be recoverable by the end user is the "am I async/is this async" reflection issue, though a core team member has clarified in this comment section that the code in the article is a sketch and the design of stackless coroutines is far from done, so we may yet get this.
Some other philosophical point is, like, lua's coroutine.create/resume/yield/clone are control flow primitives for use within a single thread of execution. It's fine to ship an async runtime, which embodies the view they they are not control flow primitives for use within a single thread of execution, for doing I/O. But focusing the primitives for creating and switching between execution contexts too narrowly on the async runtime use case is liable to he harmful to other use cases for these operations. Ideally, we would be able to write things like a prominent SNES emulator that uses stack switching to ensure the simulation of different components proceeds in an order known to be more correct than other orders, and we would be able to do it using native language features, which would compile down to something a bit cheaper than dumping all of our registers onto the stack. Ideally when we do this we would not be asked by the language to consider what it would mean to "cancel" the execution context managing one of the components, in the same way that we do not need to consider what it means to cancel an arbitrary struct, or the function which is calling the function currently executing.
This is a good time to implement "context", a way to pass down the call stack parameters instead of having to add a io argument to every function
I don't know if it's still true in the recent versions of Scala (stopped caring in 2018) but it used to have implicit parameters designed specifically for passing context like this.
A notable example was passing around an implicit ExecutionContext for thread pools, e.g. in Akka :)
A notable example was passing around an implicit ExecutionContext for thread pools, e.g. in Akka :)
What about timers? That's the other resource an event loop must offer, besides I/O.
If you are using I/O with green threads but sleep the OS thread, you block other green threads. Likewise, if you have stackless coroutines (when they exist in Zig) but sleep the OS thread, you block other coroutines.
So is there an io.sleep?
Also - does any of this use io_uring on Linux?
If you are using I/O with green threads but sleep the OS thread, you block other green threads. Likewise, if you have stackless coroutines (when they exist in Zig) but sleep the OS thread, you block other coroutines.
So is there an io.sleep?
Also - does any of this use io_uring on Linux?
Love the no function coloring solution! I am so looking forward to Zig 1.0. Finally, a system programming language that I can actually read and understand without putting in heavy labor. Hell, I could fully follow this blog post, without actually knowing anything much about Zig. Broke my head on async Rust several times before throwing in the towel.
I miss the mention of boost::asio in this thread. At first glance this new Io interface feels not that dissimilar to it:
Both are generic interfaces over an event loop/executor supporting async or blocking operations. Both ship a thread-pool and a stackful coroutine backend and both can be used through their respective language's stackless coroutine implementation (co_yield in cpp and yet-unimplemented in zig).
Both are generic interfaces over an event loop/executor supporting async or blocking operations. Both ship a thread-pool and a stackful coroutine backend and both can be used through their respective language's stackless coroutine implementation (co_yield in cpp and yet-unimplemented in zig).
So you have to do:
Considering there is very little use case for mix and matching different Ios, any chance of having some kind of default / context IO to avoid all these ?
io.async(saveFile, .{io, data, "saveA.txt"}).await(io);
Considering there is very little use case for mix and matching different Ios, any chance of having some kind of default / context IO to avoid all these ?
If you're going to immediately await it, you can just do
So this would be valid:
saveFile(io, data, "saveA.txt");
io.async(saveFile, .{io, data, "saveA.txt"}).await(io);
So this would be valid:
var save_future = io.async(saveFile, .{io, data, "saveA.txt"});
save_future.await(io);
const save_future = io.async(saveFile, .{io, data, "saveA.txt"});
save_future.await(io); // compile errorAs the author of a semi-famous post about how Zig has function colors [1], I decided to read up on this.
I see that blocking I/O is an option:
> The most basic implementation of `Io` is one that maps to blocking I/O operations.
So far, so good, but blocking I/O is not async.
There is a thread pool that uses blocking I/O. Still good so far, but blocking I/O is still not async.
Then there's green threads:
> This implementation uses `io_uring` on Linux and similar APIs on other OSs for performing I/O combined with a thread pool. The key difference is that in this implementation OS threads will juggle multiple async tasks in the form of green threads.
Okay, they went the Go route on this one. Still (sort of) not async, but there is an important limitation:
> This implementation requires having the ability to perform stack swapping on the target platform, meaning that it will not support WASM, for example.
But still no function colors, right?
Unfortunately not:
> This implementation [stackless coroutines] won’t be available immediately like the previous ones because it depends on reintroducing a special function calling convention and rewriting function bodies into state machines that don’t require an explicit stack to run.
(Emphasis added.)
And the function colors appear again.
Now, to be fair, since there are multiple implementation options, you can avoid function colors, especially since `Io` is a value. But those options are either:
* Use blocking I/O.
* Use threads with blocking I/O.
* Use green threads, which Rust removed [2] for good reasons [3]. It only works in Go because of the garbage collector.
In short, the real options are:
* Block (not async).
* Use green threads (with their problems).
* Function colors.
It doesn't appear that the function colors problem has been defeated. Also, it appears to me that the Zig team decided to have every concurrency technique in the hope that it would appear innovative.
[1]: https://gavinhoward.com/2022/04/i-believe-zig-has-function-c...
[2]: https://github.com/aturon/rfcs/blob/remove-runtime/active/00...
[3]: https://www.open-std.org/JTC1/SC22/WG21/docs/papers/2018/p13...
I see that blocking I/O is an option:
> The most basic implementation of `Io` is one that maps to blocking I/O operations.
So far, so good, but blocking I/O is not async.
There is a thread pool that uses blocking I/O. Still good so far, but blocking I/O is still not async.
Then there's green threads:
> This implementation uses `io_uring` on Linux and similar APIs on other OSs for performing I/O combined with a thread pool. The key difference is that in this implementation OS threads will juggle multiple async tasks in the form of green threads.
Okay, they went the Go route on this one. Still (sort of) not async, but there is an important limitation:
> This implementation requires having the ability to perform stack swapping on the target platform, meaning that it will not support WASM, for example.
But still no function colors, right?
Unfortunately not:
> This implementation [stackless coroutines] won’t be available immediately like the previous ones because it depends on reintroducing a special function calling convention and rewriting function bodies into state machines that don’t require an explicit stack to run.
(Emphasis added.)
And the function colors appear again.
Now, to be fair, since there are multiple implementation options, you can avoid function colors, especially since `Io` is a value. But those options are either:
* Use blocking I/O.
* Use threads with blocking I/O.
* Use green threads, which Rust removed [2] for good reasons [3]. It only works in Go because of the garbage collector.
In short, the real options are:
* Block (not async).
* Use green threads (with their problems).
* Function colors.
It doesn't appear that the function colors problem has been defeated. Also, it appears to me that the Zig team decided to have every concurrency technique in the hope that it would appear innovative.
[1]: https://gavinhoward.com/2022/04/i-believe-zig-has-function-c...
[2]: https://github.com/aturon/rfcs/blob/remove-runtime/active/00...
[3]: https://www.open-std.org/JTC1/SC22/WG21/docs/papers/2018/p13...
It’s hard to judge before stackless coroutines are reintroduced. But I think you’re entirely wrong that the next version of it will have colored functions, even according to your definition.
It has been mentioned that it’s possible that the default for debug builds is that every single function is compiled as an async function. I.e. there is canonically only one function color. Changing function color could become an optimisation for release builds. This is really not much different from inlining functions or other tricks the compiler can do with the calling convention if it has perfect knowledge of all callers.
> it appears to me that the Zig team decided to have every concurrency technique in the hope that it would appear innovative.
That’s a really bad take. It’s not much different from what they did to make allocators explicit. It’s an excellent approach for what Zig is supposed to be. Different concurrency models have different performance tradeoffs, just like with allocators. If the can support different IO models without making the language complicated, that’s a huge win, and they seem to be achieving that.
I find this approach the opposite of “appear innovative”. They’ve moved away from designing in a bunch of fancy syntax that locks users into one particular concurrency model, and gone for a more explicit and boring design which puts power in the hands of the user. It may not be right for everyone, but for what Zig is setting out to do it’s perfect. A disciplined decision in my opinion.
Getting stackless coroutines right for a low level language like Zig would be somewhat innovative. But not in a way that’s flashy or super interesting.
It has been mentioned that it’s possible that the default for debug builds is that every single function is compiled as an async function. I.e. there is canonically only one function color. Changing function color could become an optimisation for release builds. This is really not much different from inlining functions or other tricks the compiler can do with the calling convention if it has perfect knowledge of all callers.
> it appears to me that the Zig team decided to have every concurrency technique in the hope that it would appear innovative.
That’s a really bad take. It’s not much different from what they did to make allocators explicit. It’s an excellent approach for what Zig is supposed to be. Different concurrency models have different performance tradeoffs, just like with allocators. If the can support different IO models without making the language complicated, that’s a huge win, and they seem to be achieving that.
I find this approach the opposite of “appear innovative”. They’ve moved away from designing in a bunch of fancy syntax that locks users into one particular concurrency model, and gone for a more explicit and boring design which puts power in the hands of the user. It may not be right for everyone, but for what Zig is setting out to do it’s perfect. A disciplined decision in my opinion.
Getting stackless coroutines right for a low level language like Zig would be somewhat innovative. But not in a way that’s flashy or super interesting.
> It has been mentioned that it’s possible that the default for debug builds is that every single function is compiled as an async function. I.e. there is canonically only one function color.
But then, for those that choose to only use blocking I/O or green threads, they still pay the penalty of async merely existing.
> That’s a really bad take. It’s not much different from what they did to make allocators explicit.
I mean, Zig explicit allocators are really the same thing is Go interfaces, just dressed up as an innovative feature by a specific use case. This is what I mean by "appearing" innovative: they are taking old and tested ideas and presenting them in new ways to make them appear new.
Also, Zig could have had explicit allocators without needing to pass them to every function [1].
> They’ve moved away from designing in a bunch of fancy syntax that locks users into one particular concurrency model, and gone for a more explicit and boring design which puts power in the hands of the user.
Except that if every function is made async, they have actually removed from users the power to choose to not use async.
[1]: https://jai.community/t/context/163
But then, for those that choose to only use blocking I/O or green threads, they still pay the penalty of async merely existing.
> That’s a really bad take. It’s not much different from what they did to make allocators explicit.
I mean, Zig explicit allocators are really the same thing is Go interfaces, just dressed up as an innovative feature by a specific use case. This is what I mean by "appearing" innovative: they are taking old and tested ideas and presenting them in new ways to make them appear new.
Also, Zig could have had explicit allocators without needing to pass them to every function [1].
> They’ve moved away from designing in a bunch of fancy syntax that locks users into one particular concurrency model, and gone for a more explicit and boring design which puts power in the hands of the user.
Except that if every function is made async, they have actually removed from users the power to choose to not use async.
[1]: https://jai.community/t/context/163
Their bet seems to be that they can transparently implement real async inside an IO implementation using compiler magic. But then it means if you use that IO instance with the magic then your function gets transformed into a state machine?
Then this whole thing is useless for implementing cooperative scheduling async like in rust?
Then this whole thing is useless for implementing cooperative scheduling async like in rust?
> But then it means if you use that IO instance with the magic then your function gets transformed into a state machine?
This was essentially like the old async/await implementation in Zig already worked. The same function gets the state-machine treatment if it was called in an async context, otherwise it's compiled as a 'regular' sequential function.
E.g. at runtime there may be two versions of a function, but not in the code base. Not sure though how that same idea would be implemented with the new IO interface, but since Zig strictly uses a single-compilation-unit model the compiler might be able to trace the usage of a specific IO implementation throughout the control flow?
This was essentially like the old async/await implementation in Zig already worked. The same function gets the state-machine treatment if it was called in an async context, otherwise it's compiled as a 'regular' sequential function.
E.g. at runtime there may be two versions of a function, but not in the code base. Not sure though how that same idea would be implemented with the new IO interface, but since Zig strictly uses a single-compilation-unit model the compiler might be able to trace the usage of a specific IO implementation throughout the control flow?
[deleted]
No, it just means the cooperative scheduler needs to provide an io implementation that works with the rest of the scheduler.
> it depends on reintroducing a special function calling convention
This is an internal implementation detail rather than a fact which is usually exposed to the user. This is essentially just explaining that the Zig compiler needs to figure out which functions are async and lower them differently.
We do have an explicit calling convention, `CallingConvention.async`. This was necessary in the old implementation of async functions in order to make runtime function pointer calls work; the idea was that you would cast your `fn () void` to a `fn () callconv(.async) void`, and then you could call the resulting `*const fn () callconv(.async) void` at runtime with the `@asyncCall` builtin function. This was one of the biggest flaws in the design; you could argue that it introduced a form of coloring, but in practice it just made vtables incredibly undesirable to use, because (since nobody was actually doing the `@asyncCall` machinery in their vtable implementations) they effectively just didn't support async.
We're solving this with a new language feature [0]. The idea here is that when you have a virtual function -- for a simple example, let's say `alloc: *const fn (usize) ?[*]u8` -- you instead give it a "restricted function pointer type", e.g. `const AllocFn = @Restricted(*const fn (usize) ?[*]u8);` with `alloc: AllocFn`. The magic bit is that the compiler will track the full set of comptime-known function pointers which are coerced to `AllocFn`, so that it can know the full set of possible `alloc` functions; so, when a call to one is encountered, it knows whether or not the callee is an async function (in the "stackless async" sense). Even if some `alloc` implementations are async and some are not, the compiler can literally lower `vtable.alloc(123)` to `switch (vtable.alloc) { impl1 => impl1(123), impl2 => impl2(123), ... }`; that is, it can look at the pointer, and determine from that whether it needs to dispatch a synchronous or async call.
The end goal is that most function pointers in Zig should be used as restricted function pointers. We'll probably keep normal function pointers around, but they ideally won't be used at all often. If normal function pointers are kept, we might keep `CallingConvention.async` around, giving a way to call them as async functions if you really want to; but to be honest, my personal opinion is that we probably shouldn't do that. We end up with the constraint that unrestricted pointers to functions where the compiler has inferred the function as async (in a stackless sense) cannot become runtime-known, as that would lead to the compiler losing track of the calling convention it is using internally. This would be a very rare case provided we adequately encourage restricted function pointers. Hell, perhaps we'd just ban all unrestricted default-callconv function pointers from becoming runtime-known.
Note also that stackless coroutines do some with a couple of inherent limitations: in particular, they don't play nicely with FFI (you can't suspend across an FFI boundary; in other words, a function with a well-defined calling convention like the C calling convention is not allowed to be inferred as async). This is a limitation which seems perfectly acceptable, and yet I'm very confident that it will impact significantly more code than the calling convention thing might.
TL;DR: depending on where the design ends up, the "calling convention" mentioned is either entirely, or almost entirely, just an implementation detail. Even in the "almost entirely" case, it will be exceptionally rare for anyone to write code which could be affected by it, to the point that I don't think it's a case worth seriously worrying about unless it proves itself to actually be an issue in practice.
[0]: https://github.com/ziglang/zig/issues/23367
This is an internal implementation detail rather than a fact which is usually exposed to the user. This is essentially just explaining that the Zig compiler needs to figure out which functions are async and lower them differently.
We do have an explicit calling convention, `CallingConvention.async`. This was necessary in the old implementation of async functions in order to make runtime function pointer calls work; the idea was that you would cast your `fn () void` to a `fn () callconv(.async) void`, and then you could call the resulting `*const fn () callconv(.async) void` at runtime with the `@asyncCall` builtin function. This was one of the biggest flaws in the design; you could argue that it introduced a form of coloring, but in practice it just made vtables incredibly undesirable to use, because (since nobody was actually doing the `@asyncCall` machinery in their vtable implementations) they effectively just didn't support async.
We're solving this with a new language feature [0]. The idea here is that when you have a virtual function -- for a simple example, let's say `alloc: *const fn (usize) ?[*]u8` -- you instead give it a "restricted function pointer type", e.g. `const AllocFn = @Restricted(*const fn (usize) ?[*]u8);` with `alloc: AllocFn`. The magic bit is that the compiler will track the full set of comptime-known function pointers which are coerced to `AllocFn`, so that it can know the full set of possible `alloc` functions; so, when a call to one is encountered, it knows whether or not the callee is an async function (in the "stackless async" sense). Even if some `alloc` implementations are async and some are not, the compiler can literally lower `vtable.alloc(123)` to `switch (vtable.alloc) { impl1 => impl1(123), impl2 => impl2(123), ... }`; that is, it can look at the pointer, and determine from that whether it needs to dispatch a synchronous or async call.
The end goal is that most function pointers in Zig should be used as restricted function pointers. We'll probably keep normal function pointers around, but they ideally won't be used at all often. If normal function pointers are kept, we might keep `CallingConvention.async` around, giving a way to call them as async functions if you really want to; but to be honest, my personal opinion is that we probably shouldn't do that. We end up with the constraint that unrestricted pointers to functions where the compiler has inferred the function as async (in a stackless sense) cannot become runtime-known, as that would lead to the compiler losing track of the calling convention it is using internally. This would be a very rare case provided we adequately encourage restricted function pointers. Hell, perhaps we'd just ban all unrestricted default-callconv function pointers from becoming runtime-known.
Note also that stackless coroutines do some with a couple of inherent limitations: in particular, they don't play nicely with FFI (you can't suspend across an FFI boundary; in other words, a function with a well-defined calling convention like the C calling convention is not allowed to be inferred as async). This is a limitation which seems perfectly acceptable, and yet I'm very confident that it will impact significantly more code than the calling convention thing might.
TL;DR: depending on where the design ends up, the "calling convention" mentioned is either entirely, or almost entirely, just an implementation detail. Even in the "almost entirely" case, it will be exceptionally rare for anyone to write code which could be affected by it, to the point that I don't think it's a case worth seriously worrying about unless it proves itself to actually be an issue in practice.
[0]: https://github.com/ziglang/zig/issues/23367
From my experience, the calling convention was, in 0.9.x, just an implementation detail, until it wasn't. I think I may still reserve judgment for when async is fully implemented. Then I'll torture it again.
I'm a bit concerned when library authors only test it with blocking Io and the consuming app with a different kind. Wouldn't this potentially lead to bugs?
I don't know Zig, but wouldn't such a change be a major breaking change where all prior Zig code doing Io wouldn't work anymore if upgraded?
Yeah, but why is that a problem? Zig doesn't promise any stability before 1.0, and it's not like we don't need to change code in other language ecosystem frequently for all sorts of reasons (e.g. bumping a dependency version, or a new minor C/C++ compiler implementing new warnings).
Zig's not at 1.0 yet, so there's no stability guarantee at this point.
Large breaking change:
https://github.com/ziglang/zig/pull/24329
https://github.com/ziglang/zig/pull/24329
Breaking changes is just another Tuesday for Zig.
Does this mean that as a side effect, it'll now be possible to enforce functions are pure/deterministic in Zig by not passing in an Io?
Not quite:
* Global variables still exist and can be stored to / loaded from by any code
* Only convention stops a function from constructing its own `Io`
* Only convention stops a function from reaching directly into low-level primitives (e.g. syscalls or libc FFI)
However, in practice, we've found that such conventions tend to be fairly well-respected in most Zig code. I anticipate `Io` being no different. So, if you see a function which doesn't take `Io`, you can be pretty confident (particularly if it's in a somewhat reputable codebase!) that it's not interacting with the system (e.g. doing filesystem accesses, opening sockets, sleeping the thread).
* Global variables still exist and can be stored to / loaded from by any code
* Only convention stops a function from constructing its own `Io`
* Only convention stops a function from reaching directly into low-level primitives (e.g. syscalls or libc FFI)
However, in practice, we've found that such conventions tend to be fairly well-respected in most Zig code. I anticipate `Io` being no different. So, if you see a function which doesn't take `Io`, you can be pretty confident (particularly if it's in a somewhat reputable codebase!) that it's not interacting with the system (e.g. doing filesystem accesses, opening sockets, sleeping the thread).
What about random number generation; is that something that will also fall under Io?
I think random numbers can safely be considered non blocking.
So is that zig becoming a type AND effect system?
Speaking of colors… I like the color scheme of the code snippets, is it a standard scheme available in VSCode?
[deleted]
Interesting. This is a bit reminiscent of how OCaml handles async these days.
[deleted]
I wish Zig had not done async/await. CPS (like you have in Go) is way, way better, and is lower level, making it possible to do you own "async/await" if you really want to.
Read the article, the new Zig async/await interface doesn't imply the typical async/await state-machine code transformation. You can write a simple blocking runtime, or a green-thread implementation, or a thread-pool, or the state-machine approach via stackless coroutines (but AFAIK this needs a couple of language builtins which then must be implemented in an IO implementation).
By CPS do you mean lightweight threads + meeting point channels? (i.e. both the reader and writer get blocked until they meet at the read/write call) Or something else?
Why is CPS better and lower level than async/await?
Why is CPS better and lower level than async/await?
Async/await tend to be IO bound, in zigs case hiw can i know what to use and when? Say i do a db call, thats clearly IO, and later do heavy CPU thing, now how do i know that the CPU thing does not block inside my async io thing?
I guess its pure luck if the io implementation can handle both, but who knows?
I guess its pure luck if the io implementation can handle both, but who knows?
> how do i know that the CPU thing does not block inside my async io thing?
I think it's common sense to not interweave IO with long-running CPU, hence sans IO.
If you want to go that route, we already have solutions: goroutines and beam processes.
I think it's common sense to not interweave IO with long-running CPU, hence sans IO.
If you want to go that route, we already have solutions: goroutines and beam processes.
This was my point. With Go it does not matter, i can do IO or CPU both with Gos concurrency (CSP), but with async/await i usully cannot, i assume this is not something Zig is planning on, as it seems to be all about IO.
Because it allows multiple topologies of producers and consumers.
No idea what that means.. Do you have a concrete example of what CPS allows and async/await doesn't?
I believe async/await means you have a single consumer (caller) and a single producer (callee) and only a single value will be produced (resolved).
With CPS you may send and receive many times over to whomever and from whomever you like.
In JavaScript you may write..
With CPS you may send and receive many times over to whomever and from whomever you like.
In JavaScript you may write..
const fetchData = async () => {
// snip
return data;
}
const data = await fetchData();
channel := make(chan int);
go func() {
// snip
channel <- data;
}()
data := <-channel
go func() {
for { // Infinite loop:
// snip
channel1 <- data;
channel2 <- data;
channel3 <- data;
}
}()The discussion around function color misses the distinction between the typical await approach and other solutions.
For example, consider a library that implements the C preprocessor; it implements a single function that takes a string to be processed and applies the C pre-processing algorithm to it and returns the preprocessed string.
_v1 can of course be implemented in term of _v2 given a default loader definition.
Now you want to implement a preprocessor-as-a-service. It provides a rich API for the user to submit an initial file to your service and for the service to ask the user to submit the additional files on demand. And of course you want to use the c_preprocessor library. You expect your service to have to server hundreds of thousands of concurrent requests, so you want to make it async, in particular you want to make the loading async.
If you are using JS I believe you are screwed: you can't use the library as is: c_preprocessor_v2 and the async loader live in separate worlds: red (async) functions can call blue (sync) functions, but not vice versa; you need to ask the maintainer for a new async c_preprocessor_v3 that takes an async loader.
In some other languages (rust, c#, python) can wrap your async loader with a wrapper that blocks (in a way, closing over the async-ness of the function), but this is hardly ideal, the resulting call to c_preprocessor_v2 would not be async and prevent you from scaling to hundreds of thousands of requests. You might play around with offloading to thread pools, but as the bulk of the work is inside the c_preprocessor function it is never going to work well. In practice your blue functions can call red functions, but the resulting function is blue.
There is a third class of languages that allow you to combine blue and red functions producing red ones (Go, lua, scheme, and I believe this new Zig proposal); in these languages the caller can sandwich calls to sync functions across async domains, while still allowing suspending the whole call stack.
One disadvantage of the third class is that, as side effects are often unrestricted, if c_preprocessor relies on hidden global state, it might not be able to handle reentrancy correctly.
There is then a fourth class of languages where, not only side effects are always explicit (Haskel, some effectful programming languages), it is possible, and indeed idiomatic to be able to abstract over it. So c_processor_v2 might not only be able to call synchronous or asynchronous loaders transparently, but the idiomatic implementation might even be able to extract additional concurrency by not imposing dependencies unless necessary. One interpretation is that in these languages functions are always red, but I think that's reductive and not useful.
[1] this example uses higher order functions, but an OOP example would be of course completely equivalent.
For example, consider a library that implements the C preprocessor; it implements a single function that takes a string to be processed and applies the C pre-processing algorithm to it and returns the preprocessed string.
c_preprocessor_v1(body: string) -> string
c_preprocessor_v2(file: path, loader : path->string) -> string
_v1 can of course be implemented in term of _v2 given a default loader definition.
Now you want to implement a preprocessor-as-a-service. It provides a rich API for the user to submit an initial file to your service and for the service to ask the user to submit the additional files on demand. And of course you want to use the c_preprocessor library. You expect your service to have to server hundreds of thousands of concurrent requests, so you want to make it async, in particular you want to make the loading async.
If you are using JS I believe you are screwed: you can't use the library as is: c_preprocessor_v2 and the async loader live in separate worlds: red (async) functions can call blue (sync) functions, but not vice versa; you need to ask the maintainer for a new async c_preprocessor_v3 that takes an async loader.
In some other languages (rust, c#, python) can wrap your async loader with a wrapper that blocks (in a way, closing over the async-ness of the function), but this is hardly ideal, the resulting call to c_preprocessor_v2 would not be async and prevent you from scaling to hundreds of thousands of requests. You might play around with offloading to thread pools, but as the bulk of the work is inside the c_preprocessor function it is never going to work well. In practice your blue functions can call red functions, but the resulting function is blue.
There is a third class of languages that allow you to combine blue and red functions producing red ones (Go, lua, scheme, and I believe this new Zig proposal); in these languages the caller can sandwich calls to sync functions across async domains, while still allowing suspending the whole call stack.
One disadvantage of the third class is that, as side effects are often unrestricted, if c_preprocessor relies on hidden global state, it might not be able to handle reentrancy correctly.
There is then a fourth class of languages where, not only side effects are always explicit (Haskel, some effectful programming languages), it is possible, and indeed idiomatic to be able to abstract over it. So c_processor_v2 might not only be able to call synchronous or asynchronous loaders transparently, but the idiomatic implementation might even be able to extract additional concurrency by not imposing dependencies unless necessary. One interpretation is that in these languages functions are always red, but I think that's reductive and not useful.
[1] this example uses higher order functions, but an OOP example would be of course completely equivalent.
Et tu, Zig?
> With this last improvement Zig has completely defeated function coloring.
I disagree with this. Let's look at the 5 rules referenced in the famous "What color is your function?" article referenced here.
> 1. Every function has a color
Well, you don't have async/sync/red/blue anymore, but you now have IO and non-IO functions.
> 2. The way you call a function depends on its color.
Now, technically this seems to be solved, but you still need to provide IO as a parameter. Non-IO functions don't need/take it.
It looks like a regular function call, but there's no real difference.
> 3. You can only call a red function from within another red function
This still applies. You can only call IO functions from within other IO functions.
Technically you could pass in a new executor, but is that really what you want? Not to mention that you can also do this in languages that don't claim to solve the coloring problem.
> 4. Red functions are more painful to call
I think the spirit still applies here.
> 5. Some core library functions are red
This one is really about some things being only possible to implement in the language and/or stdlib. I don't think this applies to Zig, but it doesn't apply to Rust either for instance.
Now, I think these rules need some tweaking, but the general problem behind function coloring is that of context. Your function needs some context (an async executor, auth information, an allocator, ...). In order to call such a function you also need to provide the context. Zig hasn't really solved this.
That being said, I don't think Zig's implementation here is bad. If anything, it does a great job at abstracting the usage from the implementation. This is something Rust fails at spectacularly.
However, the coloring problem hasn't really been defeated.