Ruby methods are colorless(jpcamara.com)
jpcamara.com
Ruby methods are colorless
https://jpcamara.com/2024/07/15/ruby-methods-are.html
241 comments
> is a feature
I was quite shocked to read this, as in it never brushed my mind that it could be. I personally doesn't feel like it is and is one of the reasons why I try to avoid working with that stack at all cost.
I don't think the neon sign is a good excuse for the mess colored functions are. You easily can create a synchronous O(n^4) function, and there are probably tons of quick async functions. Moreover, your comment might be read as a free pass for using functions and methods that you're not really understanding the behavior off, it obviously doesn't sound like a good thing
The information is only hidden with colorless methods if you consider the documentation to be a place to hide information (o^o).
I was quite shocked to read this, as in it never brushed my mind that it could be. I personally doesn't feel like it is and is one of the reasons why I try to avoid working with that stack at all cost.
I don't think the neon sign is a good excuse for the mess colored functions are. You easily can create a synchronous O(n^4) function, and there are probably tons of quick async functions. Moreover, your comment might be read as a free pass for using functions and methods that you're not really understanding the behavior off, it obviously doesn't sound like a good thing
The information is only hidden with colorless methods if you consider the documentation to be a place to hide information (o^o).
This is not just about performance.
Unlike Go or Ruby, JavaScript is a single-threaded language. Synchronization constructs such as mutexes and semaphores are uncommon and not part of any standard library. When you are calling a synchronous function, you can be completely assured that no race condition can develop while the function is running, but the same guarantee is not true for asynchronous functions.
That's why knowing which function is running asynchronously is even more useful for languages like JavaScript.
Unlike Go or Ruby, JavaScript is a single-threaded language. Synchronization constructs such as mutexes and semaphores are uncommon and not part of any standard library. When you are calling a synchronous function, you can be completely assured that no race condition can develop while the function is running, but the same guarantee is not true for asynchronous functions.
That's why knowing which function is running asynchronously is even more useful for languages like JavaScript.
"Now you have two problems." Adding async/await was a hack to avoid having to tackle the hard problem of real concurrency, so now you have a) no real concurrency, and b) coloured functions.
There is a parallel universe where JS added almost any other concurrency primitive and got a better trade-off than async/await.
There is a parallel universe where JS added almost any other concurrency primitive and got a better trade-off than async/await.
Having interacted with a bunch of other concurrency models in a bunch of other languages, I call. Concurrency is hard, every concurrency model has flaws, and which flaws you want to live with is largely a matter of personal preference and what type of problem you are working on.
For myself, I happen to agree with OP that single threaded plus async/await has honestly been my favorite concurrency model I've ever worked with for the types of programs I tend to write (IO-bound web apps with lots of network requests and no hot loops). The property that they described of having no race conditions of any kind except where you opt into them by calling out to an explicitly tagged async function is an enormous unnecessary mental burden removed.
In other contexts with higher compute demands, a full threading model or something like Go can make a ton of sense. But it's not worth the overhead for your average web app where what you really want is to just resume when that web request finishes.
Use the right tool for the job, don't get dogmatic about programming language features in the abstract.
For myself, I happen to agree with OP that single threaded plus async/await has honestly been my favorite concurrency model I've ever worked with for the types of programs I tend to write (IO-bound web apps with lots of network requests and no hot loops). The property that they described of having no race conditions of any kind except where you opt into them by calling out to an explicitly tagged async function is an enormous unnecessary mental burden removed.
In other contexts with higher compute demands, a full threading model or something like Go can make a ton of sense. But it's not worth the overhead for your average web app where what you really want is to just resume when that web request finishes.
Use the right tool for the job, don't get dogmatic about programming language features in the abstract.
There is a parallel universe in which Brendan Eich was given more than 10 days to hack on JavaScript, and in which he presented the language with full-blown stackful coroutines instead of hacky callbacks for concurrency. There might be even a parallel universe in which '[] + 0' produces an error rather than the string "0".
Unfortunately, we do not live in that parallel universe, and JavaScript had to evolve the way it did. When callbacks have already been established as the standard concurrency mechanism, promises were a big improvement. When promises became standard, async/await was the natural evolution rather than breaking existing programs by introducing stackful coroutines.
Other languages which chose async/await (Rust, C#, Kotlin) had their own reasons. In all of the cases above, the language designers wanted to give the user more control over the scheduler (Colorless Stackful coroutines require a global scheduler).
In Rust's case, it's async/await state machines are just not wasting memory and unnecessarily copying it like stackful coroutines do (not to mention that Rust doesn't have a garbage collector and pointer rewrites, so it cannot just copy stacks around when they grow). In fact, Rust did have garbage collection and green threads, but they were both removed early on, as the language changed its focus to became a language where you can expect performance that are _at least_ on par with C or C++.
In case of Kotlin, this all mostly had to do with finding an abstraction that can run on pre-Loom JVMs, JavaScript targets and native targets. Kotlin does not own the runtime.
Saying everybody should follow Go here is silly. Not all languages share the same background and design constraints. Go-style Stackful coroutines require garbage collection, full control of the runtime and generally a language that is designed from scratch to have them - or a 6-year project to adapt your language and runtime to have them, like Project project Loom did.
Async/await imposes some costs, but the benefits (compatibility, performance, flexibility) often outweigh these. After working with both async functions and stackful coroutines in multiple languages, I cannot say I ever felt concurrency in Go to be more ergonomic than Kotlin. If anything, it was the reverse for me in that particular case (owing to Go's lack of reification for goroutines and verbose syntax). Deciding in advance what color is my function (i.e. whether it needs to deal with I/O), is pretty simple and it rarely changes during the lifetime of a program (and even when it does, refactoring is not hard unless your design is bad).
The only case where I did feel some pain using async/await, was Rust, and this has more to do with Pin, competing async runtimes (for a while), lack of support for async traits (for a while) and other areas where Async Rust still needs some work.
Unfortunately, we do not live in that parallel universe, and JavaScript had to evolve the way it did. When callbacks have already been established as the standard concurrency mechanism, promises were a big improvement. When promises became standard, async/await was the natural evolution rather than breaking existing programs by introducing stackful coroutines.
Other languages which chose async/await (Rust, C#, Kotlin) had their own reasons. In all of the cases above, the language designers wanted to give the user more control over the scheduler (Colorless Stackful coroutines require a global scheduler).
In Rust's case, it's async/await state machines are just not wasting memory and unnecessarily copying it like stackful coroutines do (not to mention that Rust doesn't have a garbage collector and pointer rewrites, so it cannot just copy stacks around when they grow). In fact, Rust did have garbage collection and green threads, but they were both removed early on, as the language changed its focus to became a language where you can expect performance that are _at least_ on par with C or C++.
In case of Kotlin, this all mostly had to do with finding an abstraction that can run on pre-Loom JVMs, JavaScript targets and native targets. Kotlin does not own the runtime.
Saying everybody should follow Go here is silly. Not all languages share the same background and design constraints. Go-style Stackful coroutines require garbage collection, full control of the runtime and generally a language that is designed from scratch to have them - or a 6-year project to adapt your language and runtime to have them, like Project project Loom did.
Async/await imposes some costs, but the benefits (compatibility, performance, flexibility) often outweigh these. After working with both async functions and stackful coroutines in multiple languages, I cannot say I ever felt concurrency in Go to be more ergonomic than Kotlin. If anything, it was the reverse for me in that particular case (owing to Go's lack of reification for goroutines and verbose syntax). Deciding in advance what color is my function (i.e. whether it needs to deal with I/O), is pretty simple and it rarely changes during the lifetime of a program (and even when it does, refactoring is not hard unless your design is bad).
The only case where I did feel some pain using async/await, was Rust, and this has more to do with Pin, competing async runtimes (for a while), lack of support for async traits (for a while) and other areas where Async Rust still needs some work.
> The information is only hidden with colorless methods if you consider the documentation to be a place to hide information (o^o)
That's exactly the mindset of quite a number of programmers nowadays, sadly. Two anecdotes: First, there was a blog post (probably even featured on HN) about problems in the design of the API of... Python's requests library, I think? It was quite a bizarre reading, complaints made almost no sense to me until I got to about the middle of the article when it clicked: the author was trying to figure out how to use the API by just reading the names of the classes, methods, and parameter names and making guesses as how it all would connect together instead of, you know, reading the docs. Which do exist, by the way.
The second example happened just yesterday at my job. In the essence, some fellow programmer added a call to function foo() which he thought would do some particular thing and return a particular answer while in reality it did not quite do that, and the return value is essentially meaningless (the function basically always returns "yep, I've scheduled do_foo() to run asynchronously some time soon"). Now, this foo() dunction is not documented, and if you read it's source (3 lines of code), it's quite obvious that its return value is useless... so why did they think it works the way they wanted it to work? When asked about it, the answer was "Well, it's named foo(), so I just thought that's what it would do". Sadly, calling strange third-party functions and hoping them to work they way you'd like them to work is no basis for writing the working programs.
That's exactly the mindset of quite a number of programmers nowadays, sadly. Two anecdotes: First, there was a blog post (probably even featured on HN) about problems in the design of the API of... Python's requests library, I think? It was quite a bizarre reading, complaints made almost no sense to me until I got to about the middle of the article when it clicked: the author was trying to figure out how to use the API by just reading the names of the classes, methods, and parameter names and making guesses as how it all would connect together instead of, you know, reading the docs. Which do exist, by the way.
The second example happened just yesterday at my job. In the essence, some fellow programmer added a call to function foo() which he thought would do some particular thing and return a particular answer while in reality it did not quite do that, and the return value is essentially meaningless (the function basically always returns "yep, I've scheduled do_foo() to run asynchronously some time soon"). Now, this foo() dunction is not documented, and if you read it's source (3 lines of code), it's quite obvious that its return value is useless... so why did they think it works the way they wanted it to work? When asked about it, the answer was "Well, it's named foo(), so I just thought that's what it would do". Sadly, calling strange third-party functions and hoping them to work they way you'd like them to work is no basis for writing the working programs.
And of course, these developers will also complain that the project lacks documentation. Often this is the case for projects that are actually well documented, its just that the dev didn't read it.
Was also going to say this. I've seen so many people fumble around figuring something else, then write a doc for how to do it, just to find that someone else wrote effectively the exact same doc 5 months ago.
Async/await is clearly a feature in JS, though not for the reason the previous poster mentions. Async/await wasn't part of the language until 2017. If you don't like it, you can just use Promises. If you do that, there are no "colored function" (functions of color?).
As far as the article goes, I think the pattern in Ruby is great. I prefer it to JS. But the JS approach works fine and the whole controversy about colored functions is a little silly.
I would be interested in understanding why JS can't enable await from inside non-async functions. I (maybe naively) wonder if the compiler couldn't just figure out which functions to treat as async rather than making the programmer do it.
As far as the article goes, I think the pattern in Ruby is great. I prefer it to JS. But the JS approach works fine and the whole controversy about colored functions is a little silly.
I would be interested in understanding why JS can't enable await from inside non-async functions. I (maybe naively) wonder if the compiler couldn't just figure out which functions to treat as async rather than making the programmer do it.
Using promises is basically no different than using async functions, and definitely doesn't "uncolor" your functions.
Any function that calls another that returns a Promise will have to return a Promise to represent completion correctly. That's the color.
Any function that calls another that returns a Promise will have to return a Promise to represent completion correctly. That's the color.
> Any function that calls another that returns a Promise will have to return a Promise to represent completion correctly
No, it doesn't, closures let you update values in the wcope of the parent in .then() callbacks and then you can loop in the function and wait for completion and return a non-Promise value.
No, it doesn't, closures let you update values in the wcope of the parent in .then() callbacks and then you can loop in the function and wait for completion and return a non-Promise value.
This is not true. If your sync function loops until the value is set, then the Promise will never resolve, since it never yields to the event loop.
With a well-configured dev env with typescript, prettier, eslint and the appropriate config, async/await does feel like a feature. The IDE adds the async keyword for you if you forget it. If you enable errors in the whole project, it will tell you of any file/function that has an inconsistency in that regard.
So yes the compiler can figure it out, but only with Typescript. For javascript, a promise it a promise, but it doesn't know that a function will return a promise until it does. You can try compiling some async/await codebase to an old ES5 version and see the mess.
So yes the compiler can figure it out, but only with Typescript. For javascript, a promise it a promise, but it doesn't know that a function will return a promise until it does. You can try compiling some async/await codebase to an old ES5 version and see the mess.
> The information is only hidden with colorless methods if you consider the documentation to be a place to hide information (o^o).
No, source code is where you hide the information, documentation is where you hide misinformation (or accurate information about how the function worked 3 versions ago, which is much the same thing).
No, source code is where you hide the information, documentation is where you hide misinformation (or accurate information about how the function worked 3 versions ago, which is much the same thing).
The code tells you what the code _does_. The documentation (which can include comments in the code) tell you what the code is intended to do. Either one of those can be wrong. If the code doesn't match the documentation, then there is a bug in the system.
Not having documentation just removes the ability to determine if what the code _does_ is what the code is _intended to do_.
Not having documentation just removes the ability to determine if what the code _does_ is what the code is _intended to do_.
> If the code doesn't match the documentation, then there is a bug in the system.
Then I'd say the vast majority of Ruby libraries are buggy.
> Not having documentation just removes the ability to determine if what the code _does_ is what the code is _intended to do_.
But usually there isn't any intent when it comes to async-or-not. Library authors usually just write a method that results in the right value; whether that method can yield or not isn't even something they thought about, much less had a specific design in mind for.
Then I'd say the vast majority of Ruby libraries are buggy.
> Not having documentation just removes the ability to determine if what the code _does_ is what the code is _intended to do_.
But usually there isn't any intent when it comes to async-or-not. Library authors usually just write a method that results in the right value; whether that method can yield or not isn't even something they thought about, much less had a specific design in mind for.
> It's a big neon sign that says "hey, this function call is expensive".
IMO the more important part is that it can yield control. Half the advantage of cooperative multithreading is knowing where you may yield control makes sharing memory more or less workable.
I also like Trio’s model: if it’s async, it will yield control, even if the result is already available so it doesn’t technically have to. Makes it more difficult to accidentally starve other tasks. (I suppose I also have to mention Curio’s model: if it’s async, it’s a scheduler upcall, whether yielding or not. But for all that Curio deserves credit for freeing Python from asyncio hell, I just don’t find myself caring about things being scheduler upcalls or not.)
IMO the more important part is that it can yield control. Half the advantage of cooperative multithreading is knowing where you may yield control makes sharing memory more or less workable.
I also like Trio’s model: if it’s async, it will yield control, even if the result is already available so it doesn’t technically have to. Makes it more difficult to accidentally starve other tasks. (I suppose I also have to mention Curio’s model: if it’s async, it’s a scheduler upcall, whether yielding or not. But for all that Curio deserves credit for freeing Python from asyncio hell, I just don’t find myself caring about things being scheduler upcalls or not.)
I had the same opinion with async/await, that it's nice to know a function performs IO and will wait before continuing. Makes it clearer when to use Promise.all to make multiple requests in parallel and wait for all of them to finish before continuing (faster than making calls sequentially).
I kind of wish the languages I use had Haskell's IO monad too, to separate functions in terms of the type system, but that's slightly different.
You might like this article (which is my personal favourite about function colouring). https://www.tedinski.com/2018/11/13/function-coloring.html
I kind of wish the languages I use had Haskell's IO monad too, to separate functions in terms of the type system, but that's slightly different.
You might like this article (which is my personal favourite about function colouring). https://www.tedinski.com/2018/11/13/function-coloring.html
It's a big neon sign that says "Either this function is expensive, or it isn't but some framework somewhere that sometimes needs to call expensive functions might also need to call this function using the same syntax".
This feels very true in Rust but not sure about JS. In JS you can `await` non-promise values without issue.
No, because in JavaScript/TypeScript you can happily await any function that you'd like on the chance that it sometimes returns a Promise. If a framework demands that you return a Promise unnecessarily, that's on the framework, not the language.
> Also, a nitpick - you can call async functions from sync ones, you just can't access the return value.
Or wait for them to finish.
Or wait for them to finish.
> Knowing ahead of time which functions are async is a feature.
It can be, but this fact doesn't always scale well to larger APIs over time.
If you see that a function is async, any of these could be true:
1. The function does some asynchronous work.
2. The function did some asynchronous work at some point in the past, but does no longer, but changing the signature is a breaking API change, so it still looks async.
3. The function does no asynchronous work, but it's a public API and the library maintainer thinks there is a good chance it may need to do async work in the future so made it async today to reserve the right to make that change without breaking the API.
4. The function may or may not do asynchronous work, but it's part of an interface that others are allowed to implement and the designer of the interface wants to give implementers the freedom to make their implementations async if they want.
5. The function is higher-order and wants to be able to accept callbacks that are themselves async.
There are probably others I'm forgetting.
This all sounds hypothetical but it's really not. If you maintain any widely-used long-lived package, you run into design questions like this all the time.
There is always a tension between an API consumer wanting to know what a function "really" does versus the API author wanting some abstraction so that implementation details can be changed over time without breaking users.
Asynchrony is one kind of effect or observable behavior that a user/maintainer of an API may or may not want to encapsulate, but it's not the only one. The C++ standard library documents and commits to the algorithmic complexity of most functions. Haskell function signatures pin down whether or not a function does IO. Statically typed languages pin down the types parameters must be.
Pinning down more of this stuff gives the API consumer some more information, but it's not always a pure win. It calcifies the behavior of the function in ways that can harm its evolution or interact poorly with higher-order code or polymorphism. There's no silver bullet.
Personally, I've never found "does this function suspend" to be a particularly interesting effect for a function to have to commit to and the fact that it poisons the entire callstack makes it very difficult to work with in practice.
It can be, but this fact doesn't always scale well to larger APIs over time.
If you see that a function is async, any of these could be true:
1. The function does some asynchronous work.
2. The function did some asynchronous work at some point in the past, but does no longer, but changing the signature is a breaking API change, so it still looks async.
3. The function does no asynchronous work, but it's a public API and the library maintainer thinks there is a good chance it may need to do async work in the future so made it async today to reserve the right to make that change without breaking the API.
4. The function may or may not do asynchronous work, but it's part of an interface that others are allowed to implement and the designer of the interface wants to give implementers the freedom to make their implementations async if they want.
5. The function is higher-order and wants to be able to accept callbacks that are themselves async.
There are probably others I'm forgetting.
This all sounds hypothetical but it's really not. If you maintain any widely-used long-lived package, you run into design questions like this all the time.
There is always a tension between an API consumer wanting to know what a function "really" does versus the API author wanting some abstraction so that implementation details can be changed over time without breaking users.
Asynchrony is one kind of effect or observable behavior that a user/maintainer of an API may or may not want to encapsulate, but it's not the only one. The C++ standard library documents and commits to the algorithmic complexity of most functions. Haskell function signatures pin down whether or not a function does IO. Statically typed languages pin down the types parameters must be.
Pinning down more of this stuff gives the API consumer some more information, but it's not always a pure win. It calcifies the behavior of the function in ways that can harm its evolution or interact poorly with higher-order code or polymorphism. There's no silver bullet.
Personally, I've never found "does this function suspend" to be a particularly interesting effect for a function to have to commit to and the fact that it poisons the entire callstack makes it very difficult to work with in practice.
> Knowing ahead of time which functions are async is a feature.
> It's a big neon sign that says "hey, this function call is expensive"
Async and expensive aren't the same thing.
> Also, a nitpick - you can call async functions from sync ones, you just can't access the return value.
That's a quirk of the particular async/await implementation in JS and not generally true of colored-function implementations. (Actually, since async/await is sugar for promises, you can actually, I’m pretty sure, both call and use syntactically async functions from ones which are syntactically not-async if you really want to, the code is just ugly.
> It's a big neon sign that says "hey, this function call is expensive"
Async and expensive aren't the same thing.
> Also, a nitpick - you can call async functions from sync ones, you just can't access the return value.
That's a quirk of the particular async/await implementation in JS and not generally true of colored-function implementations. (Actually, since async/await is sugar for promises, you can actually, I’m pretty sure, both call and use syntactically async functions from ones which are syntactically not-async if you really want to, the code is just ugly.
> Knowing ahead of time which functions are async is a feature.
"Expensive" is subjective and should be up to the programmer to decide.
And, it absolutely should not require creating two identical function definitions to get around the function coloring problem - that's completely indefensible.
> This is a good thing for programmers to easily see and know at the call site.
Yes, and the IDE can show you that information, exactly like it does the types of arguments. We don't redundantly write the types of arguments at the call site because it's a good thing for programmers to see (which it is), we let the IDE do that.
"Expensive" is subjective and should be up to the programmer to decide.
And, it absolutely should not require creating two identical function definitions to get around the function coloring problem - that's completely indefensible.
> This is a good thing for programmers to easily see and know at the call site.
Yes, and the IDE can show you that information, exactly like it does the types of arguments. We don't redundantly write the types of arguments at the call site because it's a good thing for programmers to see (which it is), we let the IDE do that.
> And, it absolutely should not require creating two identical function definitions to get around the function coloring problem - that's completely indefensible.
Oh yes, absolutely. But the solution to that is stop using joke languages that can't do kind polymorphism.
> Yes, and the IDE can show you that information, exactly like it does the types of arguments.
How can it, if the language doesn't give it that information? Either you have a language that can reliably distinguish between sync and async calls in a standard way - which is to say, it has a type system that distinguishes between sync and async, whether it calls it that or not - or your IDE has an ad-hoc informally specified bug-ridden slow implementation of half of one.
Oh yes, absolutely. But the solution to that is stop using joke languages that can't do kind polymorphism.
> Yes, and the IDE can show you that information, exactly like it does the types of arguments.
How can it, if the language doesn't give it that information? Either you have a language that can reliably distinguish between sync and async calls in a standard way - which is to say, it has a type system that distinguishes between sync and async, whether it calls it that or not - or your IDE has an ad-hoc informally specified bug-ridden slow implementation of half of one.
Good luck with that, when we keep using glue languages for full blown applications, and then shelling out to C extensions, instead of ones that could do the job all by themselves.
> And, it absolutely should not require creating two identical function definitions to get around the function coloring problem - that's completely indefensible.
This is very rarely necessary in app code. It's usually only a pattern for higher-level utility functions, and even then if you really wanted to you could unify implementations with generators.
This is very rarely necessary in app code. It's usually only a pattern for higher-level utility functions, and even then if you really wanted to you could unify implementations with generators.
> And, it absolutely should not require creating two identical function definitions to get around the function coloring problem - that's completely indefensible.
I'm afraid I don't follow. It's impossible for you to have two identical functions (one sync and one async), since then they would both be sync. What did you mean?
I'm afraid I don't follow. It's impossible for you to have two identical functions (one sync and one async), since then they would both be sync. What did you mean?
Two functions that are identical up to one of them being async and using "await" to call another function foo_async, and the other that is sync and calling foo_sync (a synchronous version of foo_async) without await?
I sometimes wonder whether this is really an optimization problem pretending to be a coloring problem. In some sense, the sync versions of blocking things are generally better in only one way: they can be more performant (run faster, use less memory, generate shorter code, etc). [0]. If a function can be implemented in async/await style, then ISTM the compiler could treat the sync variant as an automatically generated optimization instead of as a totally different (and differently colored) variant.
Of course, in languages like Rust with multiple colors of async code (single-threaded or multithreading-capable), this would get very messy.
[0] As a major caveat, synchronous code also enforces various state transitions much more efficiently. Want to prove that IO is done (in the sense that the user code is done with it) before closing a file? This is pretty easy in single threaded blocking code.
Of course, in languages like Rust with multiple colors of async code (single-threaded or multithreading-capable), this would get very messy.
[0] As a major caveat, synchronous code also enforces various state transitions much more efficiently. Want to prove that IO is done (in the sense that the user code is done with it) before closing a file? This is pretty easy in single threaded blocking code.
My experience with this is in .NET, which has methods like readFile (which is async) and readFileSync.
.NET doesn't really need to provide two separate utility methods like this though, because you can use Task.wait to block until the async task is done.
.NET doesn't really need to provide two separate utility methods like this though, because you can use Task.wait to block until the async task is done.
File.ReadAllBytes/Text/Lines use synchronous underlying file API.
Their async variants call different OS APIs for asynchronous operation where available (Overlapped IO on Windows, on Linux it is specially scheduled (p)read and write calls).
A similar distinction applies to Socket where asynchronous calls use an internal epoll-based engine while synchronous ones are plain send/recv.
Generally speaking, in synchronous code there is no advantage to calling `.Wait()` or `.GetAwaiter().GetResult()` over synchronous APIs when such are offered, and if you can help it, it is best to update the code using them to be async too if your application is multi-threaded. Luckily it's quite easy in most situations unlike what HN public (that hates better languages like C#) would lead you to believe. But ff you do have to do block on waiting for a task, the impact on throughput is usually negligible if you do so within say threadpool worker - the implementation nowadays can cope with this very well and is more resilient to scenarios that used to be problematic.
Their async variants call different OS APIs for asynchronous operation where available (Overlapped IO on Windows, on Linux it is specially scheduled (p)read and write calls).
A similar distinction applies to Socket where asynchronous calls use an internal epoll-based engine while synchronous ones are plain send/recv.
Generally speaking, in synchronous code there is no advantage to calling `.Wait()` or `.GetAwaiter().GetResult()` over synchronous APIs when such are offered, and if you can help it, it is best to update the code using them to be async too if your application is multi-threaded. Luckily it's quite easy in most situations unlike what HN public (that hates better languages like C#) would lead you to believe. But ff you do have to do block on waiting for a task, the impact on throughput is usually negligible if you do so within say threadpool worker - the implementation nowadays can cope with this very well and is more resilient to scenarios that used to be problematic.
I think the only place that happens is in programming languages where you actually can choose to do something using a thread-blocking operation? Specifically, I've only seen it in Python. And yeah, it's not very ideal there. In other languages, it's typically only possible to do the operation in async mode, so you never write a foo_sync function in the first place.
The "it's typically only possible to do the operation in async mode" is part of the problem. If I'm writing a a batch script that parses a file as part of its operation, I don't want or need the read_file() function to be async! My code should just block until the file opens.
Alternately, I want to be able to designate a computationally-intense function call (just that call, leaving the other calls alone) as async so that control yields to the event loop.
The main problem is that someone got it in their head that the function definition was the right place to designate whether a function was async or not, and it's not. The right place is the call site.
Alternately, I want to be able to designate a computationally-intense function call (just that call, leaving the other calls alone) as async so that control yields to the event loop.
The main problem is that someone got it in their head that the function definition was the right place to designate whether a function was async or not, and it's not. The right place is the call site.
> The main problem is that someone got it in their head
This unnecessarily trivializes the technical problems at hand. This wasn't just something someone got in their head, especially considering...
> that the function definition was the right place to designate whether a function was async or not, and it's not. The right place is the call site.
This is exactly what JavaScript did. The program only yields at `await`ed callsites and doesn't yield at all other callsites.
`async` only tells the VM to create the state machine necessary to resume the function after `awaited` calls return.
This unnecessarily trivializes the technical problems at hand. This wasn't just something someone got in their head, especially considering...
> that the function definition was the right place to designate whether a function was async or not, and it's not. The right place is the call site.
This is exactly what JavaScript did. The program only yields at `await`ed callsites and doesn't yield at all other callsites.
`async` only tells the VM to create the state machine necessary to resume the function after `awaited` calls return.
> This unnecessarily trivializes the technical problems at hand.
Making a language that automatically generates two versions of a function, one that is async and returns a Promise-wrapped object, and another that is sync, is not hard.
> This is exactly what JavaScript did.
No, it isn't. JavaScript requires you to declare functions as async at the function definition, and you can't call async functions from sync ones. This is exactly the thing I am arguing against, and is the opposite of "The right place is the call site."
Making a language that automatically generates two versions of a function, one that is async and returns a Promise-wrapped object, and another that is sync, is not hard.
> This is exactly what JavaScript did.
No, it isn't. JavaScript requires you to declare functions as async at the function definition, and you can't call async functions from sync ones. This is exactly the thing I am arguing against, and is the opposite of "The right place is the call site."
If it wasn't hard, it would have been done. I suppose we should all await your backwards compatible proposal?
> No, it isn't.
Yes, actually, it is. Async functions are a concept that only affects the internal structure of a function - to generate the state machine that allows yielding at `await` keywords and resuming after they resolve. Externally they are no different from "sync" Promise-returning functions.
So they only thing that async functions do is enable yielding at the callsite, the `await` keyword is what actually yields at the callsite and you can await anything, not just async functions.
Again:
> I want to be able to designate a computationally-intense function call (just that call, leaving the other calls alone) as async so that control yields to the event loop.
This is what `await` is - `await` yields. Non-await calls don't yield.
> No, it isn't.
Yes, actually, it is. Async functions are a concept that only affects the internal structure of a function - to generate the state machine that allows yielding at `await` keywords and resuming after they resolve. Externally they are no different from "sync" Promise-returning functions.
So they only thing that async functions do is enable yielding at the callsite, the `await` keyword is what actually yields at the callsite and you can await anything, not just async functions.
Again:
> I want to be able to designate a computationally-intense function call (just that call, leaving the other calls alone) as async so that control yields to the event loop.
This is what `await` is - `await` yields. Non-await calls don't yield.
> If it wasn't hard, it would have been done.
This comment shows a true departure from reality. Computing is filled to the brim with ideas that are easy but are not implemented for reasons other than difficulty.
> I suppose we should all await your backwards compatible proposal?
Your sarcasm merely helps to further demonstrate that you're not interested in a serious discussion.
> Yes, actually, it is.
Then it should be trivial for you to show me an instance of a sync function calling an async one, synchronously waiting for it, then getting back the return value, without clever hacks.
This comment shows a true departure from reality. Computing is filled to the brim with ideas that are easy but are not implemented for reasons other than difficulty.
> I suppose we should all await your backwards compatible proposal?
Your sarcasm merely helps to further demonstrate that you're not interested in a serious discussion.
> Yes, actually, it is.
Then it should be trivial for you to show me an instance of a sync function calling an async one, synchronously waiting for it, then getting back the return value, without clever hacks.
[deleted]
> And, it absolutely should not require creating two identical function definitions to get around the function coloring problem - that's completely indefensible.
Where do you see insanity like this?
Where do you see insanity like this?
Everywhere that hard function coloring exists where the language also supports synchronous calls that do the same thing (JS doesn't support sunchronous calls for lots of things and has only soft function coloring since async is sugar for Promises, so it has two reasons why that doesn't need to happen.)
Everywhere in the C# ecosystem. You’ll see things like x.Read() and x.ReadAsync().
That sounds more like a legacy problem than a problem with the async/await model in the abstract.
If you were designing a language from the ground up, why would you implement a synchronous read operation? I would just assume that all code written in the language will treat async the way that Haskell programmers treat IO and make all IO operations async no matter what.
If you were designing a language from the ground up, why would you implement a synchronous read operation? I would just assume that all code written in the language will treat async the way that Haskell programmers treat IO and make all IO operations async no matter what.
There’s lot of programming models that don’t require async or it’s complexity. Not everything is a web server that needs to serve 100k requests.
Because Read and ReadAsync are usually meaningfully different. There is nothing wrong with that.
See https://news.ycombinator.com/item?id=41055328
This bifurcation is mostly a concern for the standard library. It’s not something you do this way in your regular application code.
See https://news.ycombinator.com/item?id=41055328
This bifurcation is mostly a concern for the standard library. It’s not something you do this way in your regular application code.
Why have Json? We can always use binary to save a lot of space and bandwidth and the IDE will always compile it into plain text for us, why have underscore at the beginning of private variables? We can let the IDE do that, we even have long named functions instead of single letter ones? Most IDEs can read the jsdocs and give us a long description of what every function does, and I could go on and on. At the end it's a balance and subjective preferences of what you want to be immediately visible or something a bit more obscure (e.g. when you move your mouse over a variable)
> Why have Json? We can always use binary to save a lot of space and bandwidth and the IDE will always compile it into plain text for us
Yes, this is a great idea, and people would do it if there was widespread support for a binary structured format.
The difference between this incredibly ignorant sarcasm and my point is that IDEs already support the thing that I'm suggesting. You can already hover over a method and it'll give you information about the types of the arguments and return values that are specific to the call site.
> At the end it's a balance and subjective preferences
...and you completely missed the main point of my comment, which was not about what information to show, but avoiding duplication due to function coloring:
> And, it absolutely should not require creating two identical function definitions to get around the function coloring problem
Yes, this is a great idea, and people would do it if there was widespread support for a binary structured format.
The difference between this incredibly ignorant sarcasm and my point is that IDEs already support the thing that I'm suggesting. You can already hover over a method and it'll give you information about the types of the arguments and return values that are specific to the call site.
> At the end it's a balance and subjective preferences
...and you completely missed the main point of my comment, which was not about what information to show, but avoiding duplication due to function coloring:
> And, it absolutely should not require creating two identical function definitions to get around the function coloring problem
nit: I don't think _expensive_ is quite the right way to think about it. I view it as "hey, this function does IO". The actual cost of doing that IO varies immensely.
Its a bad feature. The thing is it does not matter if its ”obvious” as anything that touches async needs to be async too. Its a bad paradigm, CSP is obviously a better way to do concurrency. As async is usually only IO bound, but how about CPU bound? In the node ecosystem CPU bound tasks are not something you do with async/await.
In node, you generally don't want to do anything CPU bound.
Yep. If you have CPU-bound work then you should use a different language. If you have IO-bound work then explicitly tagged async functions are actually a really nice concurrency model for the only kind of concurrency you need—it's equivalent to having the Haskell IO monad as a first-class language feature.
If explicitly marking expensive IO operations using async is a feature, then why don't languages with async make their "print" functions asynchronous?
What about a language which has both lightweight threads, but in which every I/O operation (including printing and getting a random number) is colored?
https://www.haskell.org/tutorial/io.html
https://www.haskell.org/tutorial/io.html
Not really. What it really means is that some function in this callstack is expensive. Async Await bubbles to the top of your program. Eventually, you might just read this as this application as a whole might be expensive.
I hate that async functions are the ones we are forcing to wait and become synchronous. The naming was a failure imo.
Ruby/rails is full of shortsighted crap like this.
I feel stuck, our whole backend is legacy rails and I can’t escape.
I feel stuck, our whole backend is legacy rails and I can’t escape.
Bit of a strong claim when the Erlang/OTP was designed to handle massive concurrency without colorful methods. Given that both Erlang and Ruby are inspired by the message passing semantics of Smalltalk.
Unicoloured languages are great as long as your code doesn't have to actually do anything (which is lots of modern code, to be fair). Try writing a physics simulator or 3D renderer in Erlang and see how that goes.
Erlang is not great at math performance, also because it uses arbitrary length integers. There is a nice comparison between several languages, including Erlang at https://stackoverflow.com/questions/6964392/speed-comparison...
It all depends on how the code is written. Eventually somebody managed to make the Erlang code faster than the baseline C, then someone else made the C version 8k % faster, which proves your point. However, how is that related to using sync/async vs message passing?
It all depends on how the code is written. Eventually somebody managed to make the Erlang code faster than the baseline C, then someone else made the C version 8k % faster, which proves your point. However, how is that related to using sync/async vs message passing?
> Eventually somebody managed to make the Erlang code faster than the baseline C, then someone else made the C version 8k % faster, which proves your point. However, how is that related to using sync/async vs message passing?
If you want to write high-performance code then you need to be able to write synchronous code and have control over what your yield points are. If you take the "all calls are potentially async, runtime does what it wants" approach (i.e. "no function colouring") then you just will not be able to do that.
If you want to write high-performance code then you need to be able to write synchronous code and have control over what your yield points are. If you take the "all calls are potentially async, runtime does what it wants" approach (i.e. "no function colouring") then you just will not be able to do that.
> … Erlang … inspired by the message passing semantics of Smalltalk.
What makes you think that?
What makes you think that?
The Wikipedia entry says Smalltalk was one of it's influences and Joe Armstrong, co-developer of Erlang, mentions message passing as the fundamental aspect of OOP that Erlang gets right.
Here's something Joe Armstrong's PhD thesis does reference in the context of message passing:
"4.5 Programming Notations Based on Message Passing" p33
"Concepts and Notations for Concurrent Programming", Gregory R. Andrews and Fred B. Scheider, Computing Surveys 15(1) March 1983, pp 3 - 43
[pdf] https://www.cs.cornell.edu/fbs/publications/LangSurv.pdf
"4.5 Programming Notations Based on Message Passing" p33
"Concepts and Notations for Concurrent Programming", Gregory R. Andrews and Fred B. Scheider, Computing Surveys 15(1) March 1983, pp 3 - 43
[pdf] https://www.cs.cornell.edu/fbs/publications/LangSurv.pdf
The Wikipedia "Influenced by Lisp, PLEX,[2] Prolog, Smalltalk" seems to be un-sourced !
> … Joe Armstrong … mentions message passing …
Where?
> … Joe Armstrong … mentions message passing …
Where?
Legacy codebases that are a joy to work with are few and far between, in any language.
I also work on a legacy Ruby/Rails codebase, what I dislike is Ruby as a dynamic language, I'd prefer typed languages, but overall Rails didn't changed too much in the last 14 years that I know it (I didn't used too much in the past), but the concept is still the same to this day, few changes to the API/syntax, but otherwise if you know Rails, if you know Rails, it is most likely that you find very easy to work on any Rails app.
Rails doesn't scale well in my experience. Or maybe rails devs don't scale well.
The language and framework are both centered around developer happiness, which in my experience drops off around 10,000 lines. That's about when projects start getting difficult.
The language and framework are both centered around developer happiness, which in my experience drops off around 10,000 lines. That's about when projects start getting difficult.
A Rails project I'm working on has this LOC
Ruby 12169
ERB 2339
Vue 24895
Js 4526
I've implemented coroutines in C and C++; my preferred multitasking environment is message-passing between processes. I'm not quite sure what the async/await stuff is buying us (I'm thinking C++, here). Like, I get multi-shot stackless coroutines, i.e., function objects, but I don't get why you'd want to orchestrate some sort of temporal Turing pit of async functions bleeding across your code base.
I dunno. Maybe I'm old now?
Anyways; good for Ruby! Async/await just seems very faddish to me: it didn't solve any of the hard multithreading/multiprocessing problems, and introduced a bunch of other issues. My guess is that it was interesting type theory that bled over into Real Life.
I dunno. Maybe I'm old now?
Anyways; good for Ruby! Async/await just seems very faddish to me: it didn't solve any of the hard multithreading/multiprocessing problems, and introduced a bunch of other issues. My guess is that it was interesting type theory that bled over into Real Life.
Coming from a heavy TS background into a go-forward company, I’d say the main thing you get with async is it makes it incredibly obvious when computation can be performed non-sequentially (async…). For example, It’s very common to see the below in go code:
Contrast to TS:
I wouldn’t be bringing this up if I hadn’t made 10x+ performance improvements to critical services within a month of starting at a new company in a new language on a service where the handful of experienced go programmers on the team had no idea why their code was taking so long.
a := &blah{}
rA, err := engine.doSomethingWithA()
b := &bloop{}
rB, err := engine.doSomethingWithB()
Contrast to TS:
let a = {blah}
let [rA, err] = engine.doSomethingWithA()
let b = {bloop}
let [rB, err] engine.doSomethingWithB()
let a = {blah}
let [rA, err] = await engine.doSomethingWithA()
let b = {bloop}
let [rB, err] await engine.doSomethingWithB()
I wouldn’t be bringing this up if I hadn’t made 10x+ performance improvements to critical services within a month of starting at a new company in a new language on a service where the handful of experienced go programmers on the team had no idea why their code was taking so long.
Of course, the other nice thing about the JS example compared to Go is that it's trivial at the callsite to do this:
const [[rA, errA], [rB, errB]] = await Promise.all([
engine.doSomethingWithA(),
engine.doSomethingWithB()
])
Which has to do with the incredible lack of expressivity of Go, not with the concurrency model. Nothing precludes doing exactly the same thing with thread-like constructs in an expressive language.
Not to mention waitgroups are way overkill for this. You’d just use a channel or two. Or an errgroup if you want to be fancy.
Not to mention waitgroups are way overkill for this. You’d just use a channel or two. Or an errgroup if you want to be fancy.
Indeed. And breakpoints and stepping across concurrent context actually works in JS, which is nice.
WaitGroup/ErrGroup doesn't even work here, because the functions return data. I mean, you can use ErrGroup, but it requires additional error-prone concurrency orchestration to work.
Yeah, go's a little boilerplatey, but you have to option to run two sync things concurrently as well with something like:
type result[T any] struct {
el T
err error
}
chanA := make(chan result[aResultType])
chanB := make(chan result[bResultType])
go func() {
defer close(chanA)
a := &blah{}
rA, err := engine.doSomethingWithA()
chanA <- result[aResultType]{
el: rA,
err: err,
}
}()
go func() {
defer close(chanB)
b := &bloop{}
rB, err := engine.doSomethingWithB()
chanB <- result[bResultType]{
el: rB,
err: err,
}
}
resultA := <- chanA
resultB := <- chanBOne could theoretically pull out the shared boilerplate to a utility function like:
func runTask[T any](task func() (T, error)) chan result[T] {
ch := make(chan result[T])
go func() {
defer close(ch)
res, err := task()
ch <- result[T]{ el:res, err:err } }()
return ch }
Yes, it does, and Go is perfectly capable of it, and many libraries exist for you to choose which exact method suits your problem and temperment.
One of the common pasttimes in the threaded versus async debate is to present code in which one side uses all sorts of helpers and patterns and libraries and the other side is presented through writing it "raw". The great-grandparent of my post here is guilty of this. While there are interesting reasons to debate threaded versus async code, this is not one of them. Both of them are absolutely capable of writing the moral equivalent of "output = parallel_map(myMapFunc, input)" and all similar operations to within practical epsilon of each other, and anyone citing this sort of thing as an argument on either side should probably be ignored. And both languages will feature code written by people who don't know that, and it shouldn't count against either.
One of the common pasttimes in the threaded versus async debate is to present code in which one side uses all sorts of helpers and patterns and libraries and the other side is presented through writing it "raw". The great-grandparent of my post here is guilty of this. While there are interesting reasons to debate threaded versus async code, this is not one of them. Both of them are absolutely capable of writing the moral equivalent of "output = parallel_map(myMapFunc, input)" and all similar operations to within practical epsilon of each other, and anyone citing this sort of thing as an argument on either side should probably be ignored. And both languages will feature code written by people who don't know that, and it shouldn't count against either.
No… I fear you’ve missed the entire point of the matter, which is that async/await requires that you must go all the way up the call stack explicitly “await”ing things when you have introduced an “async” call (or similar wide spread changes to that effect). There’s no special magic utility function you can call you hide it away. That’s the whole point – and a very good thing, this thread argues.
No, it is perfectly feasible to abstract around it. It's just that the abstractions are also colored. But there is no more a rule that you can only "await" a promise right in the exact code where you created the promise than there is that the only way to use threads is to spawn them right on the spot and then wait for the result right on the spot. Critics of both async and threads are just dead wrong on this, and observably, objectively so, since libraries in both cases not only exist, but are readily available and abundant.
And I'm admitting this "against interest", as the lawyers say. I'm not striking a disinterested "middle of the road" pose here. I'm hugely on the side of threads. But it is still not a relevant criticism of async. You can easily "parallel map" with either abstraction and you are not stuck unable to abstract the control flow in either case.
And I'm admitting this "against interest", as the lawyers say. I'm not striking a disinterested "middle of the road" pose here. I'm hugely on the side of threads. But it is still not a relevant criticism of async. You can easily "parallel map" with either abstraction and you are not stuck unable to abstract the control flow in either case.
Show me a JS library that allows you to swap a non-async call for an async one in a non-async context^ and I’ll eat my hat.
^Without any non-local changes, obviously.
^Without any non-local changes, obviously.
You can, but I didn't want to introduce any extra constructs from the original example.
And this is a bit of a weird case, at least where I am. I tend to have a bunch of things to process and have one goroutine sending keys/indices/etc to a channel that multiple workers are processing off of.
We did have an abstraction for that at one point, but there were enough edge cases in our domain that we either had to develop a config system or rip it out and go back to writing each one (we went with the latter after an attempt at the former went really bad).
And this is a bit of a weird case, at least where I am. I tend to have a bunch of things to process and have one goroutine sending keys/indices/etc to a channel that multiple workers are processing off of.
We did have an abstraction for that at one point, but there were enough edge cases in our domain that we either had to develop a config system or rip it out and go back to writing each one (we went with the latter after an attempt at the former went really bad).
Now handle the following that is painlessly solved by runtimes with structured concurrency:
If A failed, the whole function is failed, and we don't need B any more. To save resources we should cancel B. And vice versa, cancel A if B failed.
If A failed, the whole function is failed, and we don't need B any more. To save resources we should cancel B. And vice versa, cancel A if B failed.
import github.com/carlmjohnson/flowmatic
func whatever(ctx context.Context) {
err := flowmatic.Race(ctx, taskA, taskB)
}
As I said in another message, this is not a particularly fruitful line of attack in either direction. All the languages in question are perfectly capable of abstractions.
I don't like that the return values of the tasks has to be communicated with side effects, but I'll concede that it's quite painless.
I guess I'm just still salty when someone commented (in another post a long time ago) that golang only has `go` (compared to `launch`, `async`, and `coroutineScope` in Kotlin) and is simpler.
> All the languages in question are perfectly capable of abstractions.
I don't think async functions in JS can be cancelled though.
I guess I'm just still salty when someone commented (in another post a long time ago) that golang only has `go` (compared to `launch`, `async`, and `coroutineScope` in Kotlin) and is simpler.
> All the languages in question are perfectly capable of abstractions.
I don't think async functions in JS can be cancelled though.
"I don't like that the return values of the tasks has to be communicated with side effects, but I'll concede that it's quite painless."
Me neither, however, it is generally the most flexible approach and I can see why a library takes it. If you want to communicate it via the return, you also have to impose a restriction that the tasks all return the same type. I think it makes sense for a library to work this way because you can easily add this around the library call yourself, but it's more difficult to go the other way. (Not impossible, just more difficult.)
"I don't think async functions in JS can be cancelled though."
Poking around, it looks pretty hard.
That said, cancelling in generally is very difficult in imperative languages, so even as someone who finds async in JS quite distasteful I can't dock too many points. Go basically just reifies the JS solution into a standard community practice, which is definitely an improvement since you can largely rely on it being supported everywhere, but one could reasonably debate how good it is. It is occasionally a problem that if you want to cancel an ongoing computation you may have to have your code manually check a context every so often, because there's no "real" connection between a context and a goroutine.
Me neither, however, it is generally the most flexible approach and I can see why a library takes it. If you want to communicate it via the return, you also have to impose a restriction that the tasks all return the same type. I think it makes sense for a library to work this way because you can easily add this around the library call yourself, but it's more difficult to go the other way. (Not impossible, just more difficult.)
"I don't think async functions in JS can be cancelled though."
Poking around, it looks pretty hard.
That said, cancelling in generally is very difficult in imperative languages, so even as someone who finds async in JS quite distasteful I can't dock too many points. Go basically just reifies the JS solution into a standard community practice, which is definitely an improvement since you can largely rely on it being supported everywhere, but one could reasonably debate how good it is. It is occasionally a problem that if you want to cancel an ongoing computation you may have to have your code manually check a context every so often, because there's no "real" connection between a context and a goroutine.
If you’re fine with manually checking a standard interface to see if you should abort, JS’s answer is the AbortController. This is supported by features like the “fetch” function for making cancellable http requests.
https://developer.mozilla.org/en-US/docs/Web/API/AbortContro...
https://developer.mozilla.org/en-US/docs/Web/API/AbortContro...
So, context.Cancel?
[deleted]
Only if those doSomething methods were written as asynchronous to begin with.in your original example, doSomethingA was simple, why would it be an async method. If your answer is write every method async for a rainy day, then whats the point.
No… that’s the whole point. If you change them to be async, the language forces you to go and rethink what implications that has for the callers. This is a good thing, dumbly sequenced operations are terrible UX. And UX is far more important than whatever it is they call “DX”.
I’d rather just stick to languages that don’t have colored functions. You don’t need to be forced to think about this. Your tools manufactured this issue.
Honestly, despite that blog, async coloring is a feature. The pattern enforces implicit critical sections between yields and the coloring is how the dev can know what will yield.
This is a really interesting point. You almost never hear async function coloring being conceptualized as a feature rather than a hindrance. Async function coloring is kind of analogous to the borrow checker in Rust. It makes you think about concurrency the same way that the borrow checker makes you think about memory ownership and lifetimes.
async is a great feature if you use it from square 1. If you start with a legacy codebase using callbacks and try to port it incrementally to async, you're gonna have a bad time. Otherwise, it's definitely a feature
Yeah upgrading a legacy codebase that uses callbacks is not fun, but if the callback functions follow the Node error first value second convention, then it's a little bit easier because you can just use `util.promisify` to convert them to promises in Node. There's also the new Promise.withResolvers method which helps a bit too [1].
[1] https://github.com/tc39/proposal-promise-with-resolvers
[1] https://github.com/tc39/proposal-promise-with-resolvers
Yes. That blog has probably done more to negatively impact the industry than any other written work I know.
For some reason requiring the programmer to use additional syntax at the call site to mark behavioural properties of called functions is not a popular language feature generally. I guess eg TypeScript could add it as a user extensible feature. Would it be useful to be able to require things like this in your internal API?
let gizmos = nocheckperms lookupRequestedGizmos(request);You're trading complexity for expressiveness but the await keyword syntax is essentially unwrap sugar. Dereferencing a pointer is similar syntax.
It's possible to write a language where awaiting a task is done through a method on the task type. I don't think this is ideal because the whole reason you're using explicit yield points is so you can tell when something yields. Using method syntax makes that harder to see at a glance.
It's possible to write a language where awaiting a task is done through a method on the task type. I don't think this is ideal because the whole reason you're using explicit yield points is so you can tell when something yields. Using method syntax makes that harder to see at a glance.
I work on a large multi-threaded ruby code base and it's always a pain to deal with engineers introducing the async usage in it. Most of the time these engineers don't have a grasp on what fibers are good for and we have to painstakingly review their code and provide feedback that no, it's not magical concurrency, we have a limited number of fix sized connection pools for postgres, redis, memcache, etc available in the process and fibers have to respect those limits as much as threaded code. In fact it's better if you don't introduce any additional concurrency below the thread-per-request model if you can. Only in places like concurrent http calls do we allow some async/fiber usage but even then it ends up nominally looking like threaded code and the VM context switching time saved by using fibers instead of threads is trivial. Trying to use fibers instead of threads in typical CRUD workloads feels a little cargo culty to me.
Fibers are cool and performant but usually they should be limited to lightweight work with large concurrency requirements.
Fibers are cool and performant but usually they should be limited to lightweight work with large concurrency requirements.
I wish there were limited forms of true parallelism available with fibers instead of it just being another concurrency construct limited by an interpreter lock. I feel like there should be an in-language construct for stuff that's safely parallel and narrow, and not just Ractors. But I do get that it'd be a footgun for many.
I agree. I've thought about POCing some ractor usage for actual CPU bound workloads but ultimately it would be an academic exercise and most of our engineers aren't even thinking about the code performance on that level.
My beef, if you will, with async (and I realize now I was a little loose in making the distinction between async and fibers) is that engineers reach for async because they are intimidated by threaded code. Instead they are used to the ergonomics of async/await in the browser and think it's basically the same with the async library in ruby and is somehow immune to the considerations you have to make when spawning a thread. This just isn't true. The dragons are mostly the same, or at least they are in our code base.
My beef, if you will, with async (and I realize now I was a little loose in making the distinction between async and fibers) is that engineers reach for async because they are intimidated by threaded code. Instead they are used to the ergonomics of async/await in the browser and think it's basically the same with the async library in ruby and is somehow immune to the considerations you have to make when spawning a thread. This just isn't true. The dragons are mostly the same, or at least they are in our code base.
If you have hundreds or thousands of connection-handlers or scripts-attached-to-game-objects, then it can be useful to write code in those without dividing the code into basic blocks each time I/O might be performed or time might pass in the game.
I generally agree that manually migrating everything to "be async" in order to achieve this exposes at the very least a lack of belief that computers can be used to automate drudge work.
I generally agree that manually migrating everything to "be async" in order to achieve this exposes at the very least a lack of belief that computers can be used to automate drudge work.
I'm with you here, actor model is the way to go.
I thought Go style could be better, but the Go type system is completely inadequate to support that style: it is impossible to guaranteed that a goroutine is panic free, it is not possible to put a recover in the goroutine unless that code is under author's control (could be a lib) and a goroutine panic takes down the whole app.
Suddenly I want a wrapper around each goroutine that bubbles up the panic to the main thread without crashing the app, that sounds a lot like an erlang supervision tree
I thought Go style could be better, but the Go type system is completely inadequate to support that style: it is impossible to guaranteed that a goroutine is panic free, it is not possible to put a recover in the goroutine unless that code is under author's control (could be a lib) and a goroutine panic takes down the whole app.
Suddenly I want a wrapper around each goroutine that bubbles up the panic to the main thread without crashing the app, that sounds a lot like an erlang supervision tree
That sounds a lot like async/await. Any errors thrown in an async context bubble to the callsite awaiting it.
Sorry you are right, that was specific to our case where we were already waiting for a result from the goroutines.
In an actor model, you would let the "goroutine" die and the supervisor would restart it if it has a restart policy, otherwise it will die and stay dead (without bringing the system down).
In erlang you can also "link" the current actor to another actor so that if the linked actor dies, the linker dies too (or receives a message)
In our C++ stack at work we accept HTTP requests, and then spawn internal IO which takes some time. During that time, we yield the handler thread to handle other requests. The arrival of the internal response sets up the next step in the request handler to resume, which needs the response message.
This can be done manually by polling status of descriptors, and stepping into and out of callables / passed continuations by hand, or it can be done with a task scheduling API and typesafe chained async functions.
Pick your poison, I guess, but it probably scales better than using synchronous IO and thread-per-request.
This can be done manually by polling status of descriptors, and stepping into and out of callables / passed continuations by hand, or it can be done with a task scheduling API and typesafe chained async functions.
Pick your poison, I guess, but it probably scales better than using synchronous IO and thread-per-request.
Ever done any UI programming or other paradigms where you have a specific OS thread used for cross runtime synchronization but you need to efficiently call background threads knowing exactly what's running where?
Async/await handles that well and so far the other paradigms just have not targeted these use cases well.
Message passing and promise libraries end up looking a lot like async/await with less sugar. (Compared to C#'s async/await implementation. The level of sugar depends on the language, of course.)
Async/await handles that well and so far the other paradigms just have not targeted these use cases well.
Message passing and promise libraries end up looking a lot like async/await with less sugar. (Compared to C#'s async/await implementation. The level of sugar depends on the language, of course.)
I really like the patterns that it enables for situations with a lot of parallel IO. Easy example is any kind of scraper, where you're fetching large numbers of pages and then immediately processing those to generate more requests for the links and assets referenced within.
> Anyways; good for Ruby! Async/await just seems very faddish to me: it didn't solve any of the hard multithreading/multiprocessing problems, and introduced a bunch of other issues. My guess is that it was interesting type theory that bled over into Real Life.
I think what happened what that JavaScript absolutely necessitated the addition of async/await to avoid callback hell (due to its single-threaded mandate)... just to create a new kind of hell all of its own.
But before the long-term consequences within large code bases could be observed, the cool kids like C# & friends jumped on the bandwagon believing that "more syntax equals more better code".
I think what happened what that JavaScript absolutely necessitated the addition of async/await to avoid callback hell (due to its single-threaded mandate)... just to create a new kind of hell all of its own.
But before the long-term consequences within large code bases could be observed, the cool kids like C# & friends jumped on the bandwagon believing that "more syntax equals more better code".
Async and await appeared in C# well before they were added to JavaScript, so I'm not sure the reasoning of your timeline makes sense.
You are right. I already saw it on other replies. Thanks for pointing it out here as well.
Hopefully I didn't seem too pedantic. I don't think C# having it first necessarily diminishes your point about the use of async and await in JS.
Async and await were nice additions in C# to make working with tasks more convenient, but there were other ways to manage async tasks without ending up with a tower of callbacks. The situation was messier in JS.
Async and await were nice additions in C# to make working with tasks more convenient, but there were other ways to manage async tasks without ending up with a tower of callbacks. The situation was messier in JS.
> I think what happened what that JavaScript absolutely necessitated the addition of async/await to avoid callback hell (due to its single-threaded mandate)... just to create a new kind of hell all of its own.
It’s not really “a new kind of hell”, there’s a logical progression from callbacks to reified callback (promises) to coroutines, and each step makes concurrency more manageable so you do more of it until you hit a new wall.
And client-side JS probably can’t ever be preemptive, so not only would a thread model require adding a ton of tooling at once it would still behave quite strangely.
It’s not really “a new kind of hell”, there’s a logical progression from callbacks to reified callback (promises) to coroutines, and each step makes concurrency more manageable so you do more of it until you hit a new wall.
And client-side JS probably can’t ever be preemptive, so not only would a thread model require adding a ton of tooling at once it would still behave quite strangely.
The alternative isn't preemptive threads, it's coroutines with subroutines that can yield.
Yes, cooperative threading, the worst of all worlds.
imo we should reify more capabilities in the type system. you shouldn't be able to call Math.random without all of your transitive callers declaring at the call site that you plan to do that.
Javascript necessitates async because of its focus on UI. You just can not have good UI while blocking on the main (or UI) thread, you will inevitably have something like async and callbacks, so you might as well embrace it. Whether that is a good tradeoff for a server is a different question.
What you need is concurrency. And async/await is just one form of concurrency.
I don’t see why you would need threads to create a stackful coroutine implementation. However, what you would indeed need is a much more heavy runtime.
I don’t see why you would need threads to create a stackful coroutine implementation. However, what you would indeed need is a much more heavy runtime.
async/await like many similar constructs buys you the ability to defer design decisions you would otherwise have to have made earlier. in a program that stays small, this is good, less work to do. in a program that grows large, this is a hazard and will likely lead to substantial debts down the line.
Async/await is just a sugar that many people happened to like.
As for me, I like Go's CSP better.
As for me, I like Go's CSP better.
JavaScript has async/await because it solves a very specifically JavaScript problem: the language has no threads and all your code lives in the main event loop of the browser tab. If you do the normal thing of pausing your JavaScript until an I/O operation returns, the browser hangs. So you need to write everything with callbacks (continuation passing style), which is a pain in the ass and breaks exception handling. Promises were introduced to fix exceptions and async/await introduced to sugar over CPS.
There's also a very specifically Not JavaScript problem (that happens to also show up in Node.js): exotic concurrency scenarios involving lots of high-latency I/O (read: network sockets). OS kernel/CRT[0] threads are a lot more performant than they used to be but they still require bookkeeping that an application might not need and usually allocate a default stack size that's way too high. There's no language I know of where you can ask for the maximum stack size of a function[1] and then use that to allocate a stack. But if you structure your program as callbacks then you can share one stack allocation across multiple threads and heap-allocate everything that needs to live between yield points.
You can do exotic concurrency without exposing async to the language. For example, Go uses (or at least one point it did) its own thread management and variable-size stacks. This is more invasive to a program than adopting precise garbage collection and requires deep language support[3]. Yes, even moreso than async/await, which just requires the compiler transform imperative code into a state machine on an opt-in basis. You'd never, ever see, say, Rust implement transparent green threading[2].
To be clear, all UI code has to live in an event loop. Any code that lives in that event loop has to be async or the event loop stops servicing user input. But async code doesn't infect UI code in any other language because you can use threads and blocking executors[4] as boundaries between the sync and async worlds. The threading story in JavaScript doesn't exist, it has workers instead. And while workers use threads internally, the structured clone that is done to pass data between workers makes workers a lot more like processes than threads in terms of developer experience.
[0] Critical Runtime Theory: every system call is political
[1] ActionScript 3 had facilities for declaring a stack size limit but I don't think you could get that from the runtime.
async/await syntax in Rust also just so happens to generate an anonymous type whose size is the maximum memory usage of the function across yield points, since Rust promises also store the state of the async function when it's yielded. Yes, this also means recursive yields are forbidden, at least not without strategic use of `Box`. And it also does not cover sync stack usage while a function is being polled.
[2] I say this knowing full well that alpha Rust had transparent green threading.
[3] No, setjmp/longjmp does not count.
[4] i.e. when calling async promise code from sync blocking functions, just sleep the current thread or poll/spinwait until the promise resolves
There's also a very specifically Not JavaScript problem (that happens to also show up in Node.js): exotic concurrency scenarios involving lots of high-latency I/O (read: network sockets). OS kernel/CRT[0] threads are a lot more performant than they used to be but they still require bookkeeping that an application might not need and usually allocate a default stack size that's way too high. There's no language I know of where you can ask for the maximum stack size of a function[1] and then use that to allocate a stack. But if you structure your program as callbacks then you can share one stack allocation across multiple threads and heap-allocate everything that needs to live between yield points.
You can do exotic concurrency without exposing async to the language. For example, Go uses (or at least one point it did) its own thread management and variable-size stacks. This is more invasive to a program than adopting precise garbage collection and requires deep language support[3]. Yes, even moreso than async/await, which just requires the compiler transform imperative code into a state machine on an opt-in basis. You'd never, ever see, say, Rust implement transparent green threading[2].
To be clear, all UI code has to live in an event loop. Any code that lives in that event loop has to be async or the event loop stops servicing user input. But async code doesn't infect UI code in any other language because you can use threads and blocking executors[4] as boundaries between the sync and async worlds. The threading story in JavaScript doesn't exist, it has workers instead. And while workers use threads internally, the structured clone that is done to pass data between workers makes workers a lot more like processes than threads in terms of developer experience.
[0] Critical Runtime Theory: every system call is political
[1] ActionScript 3 had facilities for declaring a stack size limit but I don't think you could get that from the runtime.
async/await syntax in Rust also just so happens to generate an anonymous type whose size is the maximum memory usage of the function across yield points, since Rust promises also store the state of the async function when it's yielded. Yes, this also means recursive yields are forbidden, at least not without strategic use of `Box`. And it also does not cover sync stack usage while a function is being polled.
[2] I say this knowing full well that alpha Rust had transparent green threading.
[3] No, setjmp/longjmp does not count.
[4] i.e. when calling async promise code from sync blocking functions, just sleep the current thread or poll/spinwait until the promise resolves
> For example, Go uses (or at least one point it did) its own thread management and variable-size stacks.
Go developer here: Yup, the language still works like that, and probably always will.
Go developer here: Yup, the language still works like that, and probably always will.
I think the async/await patterns solve one problem really well: UI latency.
My UI background is web testing and C++ game UIs in C++. There are like 3 patterns in multithreaded games. Thread per responsibility (old/bad), Barriers blocking all threads and give each game system all the threads, or something smart. Few games do something smart, and the thread per responsibility is not ideal and we will ignore it for now.
In games often pausing everything and letting the physics system have all the threads for a few milliseconds is "fast enough". Then the graphics systems will try to use all the threads, and so on so that eventually everything will get all the threads even thought everything else rarely needs it. Sometimes two things are both close to single threaded and have no data contention so they might both be given threads, but this is almost always a manually decided things by experts.
This means that once UI, the buttons, the text, cursors, status bars, etc, gets its turn there won't be any race conditions (good), but if it needs to request something from disk that pause will happen on a thread in the UI system (bad and analogous to web sites making web API calls) so UI latency can be a real problem. If any IO small or some resource system has preloaded it then there isn't a detectable slowdown, but there are still plenty of silly periods of waiting. There is also a lot of time when some single threaded part of the game isn't using N-1 hardware threads and all that IO could have been asynchronous. But often game UIs are a frame behind the rest of the game simulation and there is often detectable latency in the UI like the mouse feeling like it drags or similar.
Allowing IO to run in the back while active events are processed can reduce latency and this is the default in web browsers. IO latency in web pages is worse than in games and other computation seems smaller than games, so the the event loop is close to ideal. A function is waiting? throw it on the stack and grab something else to do! This means that all the work that can be done while waiting on IO is done and when does well makes a UI snappy.
If that were available sensibly in games it could allow a game designed appropriately to span IO across multiple frames and be snappy without stutters. With games using the strategy I described above latency in the game simulation or IO can cause the UI to feel sluggish and vice versa. In games caching UI details and trying to pump the frame rate is "good enough". If the UI is a frame behind but we have 200 frames per second, that isn't really a problem. But when it chugs and the mouse stops responding because the player built the whole game world out of dynamite and set it off the game will not process the mouse until that 30 minutes of physics work is done.
There are better scheduling schemes for games. I am a big fan of doing "something smart" but that usually means scheduling heterogenous work with nuanced dependencies and I have written libraries just for that because it isn't actually that hard. But if you don't have the raw compute demands of a game scheduling IO along side your UI computation is often "fast enough" and is any easy enough mental model for JS devs to grok and allow them freedom to speed things up with their own solutions like caching schemes and reworking their UIs.
My UI background is web testing and C++ game UIs in C++. There are like 3 patterns in multithreaded games. Thread per responsibility (old/bad), Barriers blocking all threads and give each game system all the threads, or something smart. Few games do something smart, and the thread per responsibility is not ideal and we will ignore it for now.
In games often pausing everything and letting the physics system have all the threads for a few milliseconds is "fast enough". Then the graphics systems will try to use all the threads, and so on so that eventually everything will get all the threads even thought everything else rarely needs it. Sometimes two things are both close to single threaded and have no data contention so they might both be given threads, but this is almost always a manually decided things by experts.
This means that once UI, the buttons, the text, cursors, status bars, etc, gets its turn there won't be any race conditions (good), but if it needs to request something from disk that pause will happen on a thread in the UI system (bad and analogous to web sites making web API calls) so UI latency can be a real problem. If any IO small or some resource system has preloaded it then there isn't a detectable slowdown, but there are still plenty of silly periods of waiting. There is also a lot of time when some single threaded part of the game isn't using N-1 hardware threads and all that IO could have been asynchronous. But often game UIs are a frame behind the rest of the game simulation and there is often detectable latency in the UI like the mouse feeling like it drags or similar.
Allowing IO to run in the back while active events are processed can reduce latency and this is the default in web browsers. IO latency in web pages is worse than in games and other computation seems smaller than games, so the the event loop is close to ideal. A function is waiting? throw it on the stack and grab something else to do! This means that all the work that can be done while waiting on IO is done and when does well makes a UI snappy.
If that were available sensibly in games it could allow a game designed appropriately to span IO across multiple frames and be snappy without stutters. With games using the strategy I described above latency in the game simulation or IO can cause the UI to feel sluggish and vice versa. In games caching UI details and trying to pump the frame rate is "good enough". If the UI is a frame behind but we have 200 frames per second, that isn't really a problem. But when it chugs and the mouse stops responding because the player built the whole game world out of dynamite and set it off the game will not process the mouse until that 30 minutes of physics work is done.
There are better scheduling schemes for games. I am a big fan of doing "something smart" but that usually means scheduling heterogenous work with nuanced dependencies and I have written libraries just for that because it isn't actually that hard. But if you don't have the raw compute demands of a game scheduling IO along side your UI computation is often "fast enough" and is any easy enough mental model for JS devs to grok and allow them freedom to speed things up with their own solutions like caching schemes and reworking their UIs.
Isn't async/await "scheduling heterogenous work with nuanced dependencies"? Or is that what you were implying?
Although my real guess is ECS but that's more like the "everyone gets every thread for a time."
Although my real guess is ECS but that's more like the "everyone gets every thread for a time."
TLDR; I hadn't meant it that way, but in web pages it really is enough. Web pages generally don't have computation time to worry about, mostly just IO. This simplifies scheduling because whatever is coordinating the event loop in the browser (or other UI) can just background any amount of independent IO tasks. If there is computation screwing with share mutable state something with internal knowledge needs to be involved and that isn't current event loops, but in the long run it could be.
Sorry for the novel.
I meant those nuanced dependencies as a way of managing shared mutable state and complex computations that really do take serious CPU time. Let's make a simple example from a complex game design. This example is ridiculous but conveys the real nature of the problems with CPU and IO. Consider these pieces of work that might exist in a hypothetical game where NPCs and a player are moving around a simulated environment with some physics calculations and that the physics simulation is the source of truth for locations of items in the game. Here are parts of a game:
Physics broad phase: Trivially parallelizable, depends on previous frame. Produces "islands" of physics objects. Imagine two piles of stuff miles apart, they can't interact except with items in the same pile, each island is just a pile of math to do. Perhaps in this game this might take 20 ms of CPU time. Across the minimum target machine with 4 cores that is 5ms apiece.
Physics narrow phase: Each physics island is single threaded but threadsafe from each other, depends on broad phase to produce islands. Each island takes unknown and unequal time, likely between 0 and 2 ms of just math.
Graphics scene/render: Might have a scene graph culling that is parallelizable, and converts game state into a series of commands independent of any specific GPU API. Depends on all physics completing because that is what it is drawing. Likely 1 or 2 ms per island.
Graphics draw calls: Single threaded, sends render results to GPU using directx/opengl/vulkan/metal. This converts the independent render commands to API specific commands. Likely less than 1 ms of actual CPU work, but larger wait on GPU because it is IO.
NPC AI: NPCs are independent but light weight so threading makes no sense if there are fewer than hundreds. Depends on physics to know what NPCs are responding to. Wants to add forces to the physics sim next frame. For this game lets say there are many, I don't know maybe this is dynasty warriors or something, so lets say a 1~3 ms.
User input: Single threaded, will to add forces to the physics sim next frame based on user commands. Can't run at the same time as NPCs because both want to mutate the physics state. Less than 1 ms.
We are ignoring: Sound, Network, Environment, Disk IO, OS events (window resize, etc), UI (not buttons or text positioning), and a few other things.
A first attempt at a real game would likely be coded to give all the threads to each piece of work one at a time in some hand picked order, or at least until this was demonstrate to be slow:
Physics Broad -> Physics Narrow -> Graphics render -> Graphic GPU -> NPC AI -> User input -> Wait/Next frame
But that is likely slow, and I picked our hypothetical math to be slow and marginal. Sending stuff to the GPU is a high latency activity it might take 5 ms to respond, and if this is a 60 FPS game then that is like 1/3 of our time. If we simply add our hypothetical times that is frequently more than 16ms making the game slower than 60fps. Even an ideal frame with just a little physics is right at 15 to 16 ms So a practical game studio might do other work while waiting on the GPU to respond:
Physics Broad -> Physics Narrow -> Graphics render ->
At the same time: { Graphics GPU calls (Uses one thread) NPC AI (Uses all but one thread) -> User input } ->
Wait/Next frame
Most of the time something like this is "fast enough". In this example that 5 ms of CPU time wait on the GPU is now running alongside all that NPC AI so we only need to add the larger of the two. If this takes only a few days of engineer time and keeps the game under the 16ms on most machines then maybe the team makes a business decision to raise the minimum specs just a bit (from 4 to 6 cores would reduce physics time by another ms) and now can they ship this game. There are still needless waits and from a purely GigaFLOPS perspective perhaps much weaker computers could do the work but there is so much waiting that it isn't practical. But this compromise gets all target machines to just about 60 FPS.
Alternatively, if the game is smart enough to make new threads of work for each physics islands (actually super complex and not a great idea in real game, but this all hypothetical but there are similar wins to be had in real games) and manage dependencies carefully based on the the simulation state then something more detailed might be possible:
1. Physics broadphase, create known amount of physics islands.
2. Start a paused GPU thread waiting on known amount of physics islands to be done rendering. This will start step 5 as soon as the last step 4c completes.
3. Add the player's input work to the appropriate group of NPCs
4. Each Physics island gets a thread that does the following: a. Physics narrow phase for this island, b. Partial render for just this island, c. Sets a threadsafe flag this island is done, d. NPC AI is processed for NPCs near this physics island, e. If this is the island with the player process their input.
5. The GPU thread waits for all physics islands threads to get to step 3c then starts sending commands to the GPU! and 3d gets to keep running.
6. When all threads from step 4 and 5 complete pause all game threads to hit the desired frame rate (save battery life for mobile gamers!) or advance to next thread if past frame budget or framerate is uncapped.
This moves all the waits to the end of each thread's frame runtime. This means a bunch of nice things. That last thread can likely do some turbo boosting, a feature of most CPUs where they clock up one CPU if it is the only one working. If the NPCs ever took longer than the GPU they still might complete earlier because they get started earlier. If there are more islands than hardware threads this likely results in better utilization because there are no early pauses.
This would likely a take a ton of engineering time. This might move the frame time down a few more ms and maybe let them lower the minimum requirements perhaps even letting the game run on an older console if market conditions support that. Conceptually, it might be a thing that could be done with async/await, but I don't think that is how most game engines are designed. I also think this makes dependencies implicit and scattered through the code, but likely that could be avoided with careful design.
I am a big fan of libraries that let you provide work units, or functors, and say which depend on each other. They all get to read/write to the global state, but with the dependencies there won't be race conditions. Such libraries locate the threading logic in one place. Then if there is some particularly contentious state that many things need to touch it can be wrapped in a mutex.
I suppose this might just be the iterative vs recursive discussion applied to threading strategies. It just happens that most event loops are single threaded, no reason they need to be in the long run. In the long run I could see making that fastest scenario happen in either paradigm even though the code would look completely different.
Sorry for the novel.
I meant those nuanced dependencies as a way of managing shared mutable state and complex computations that really do take serious CPU time. Let's make a simple example from a complex game design. This example is ridiculous but conveys the real nature of the problems with CPU and IO. Consider these pieces of work that might exist in a hypothetical game where NPCs and a player are moving around a simulated environment with some physics calculations and that the physics simulation is the source of truth for locations of items in the game. Here are parts of a game:
Physics broad phase: Trivially parallelizable, depends on previous frame. Produces "islands" of physics objects. Imagine two piles of stuff miles apart, they can't interact except with items in the same pile, each island is just a pile of math to do. Perhaps in this game this might take 20 ms of CPU time. Across the minimum target machine with 4 cores that is 5ms apiece.
Physics narrow phase: Each physics island is single threaded but threadsafe from each other, depends on broad phase to produce islands. Each island takes unknown and unequal time, likely between 0 and 2 ms of just math.
Graphics scene/render: Might have a scene graph culling that is parallelizable, and converts game state into a series of commands independent of any specific GPU API. Depends on all physics completing because that is what it is drawing. Likely 1 or 2 ms per island.
Graphics draw calls: Single threaded, sends render results to GPU using directx/opengl/vulkan/metal. This converts the independent render commands to API specific commands. Likely less than 1 ms of actual CPU work, but larger wait on GPU because it is IO.
NPC AI: NPCs are independent but light weight so threading makes no sense if there are fewer than hundreds. Depends on physics to know what NPCs are responding to. Wants to add forces to the physics sim next frame. For this game lets say there are many, I don't know maybe this is dynasty warriors or something, so lets say a 1~3 ms.
User input: Single threaded, will to add forces to the physics sim next frame based on user commands. Can't run at the same time as NPCs because both want to mutate the physics state. Less than 1 ms.
We are ignoring: Sound, Network, Environment, Disk IO, OS events (window resize, etc), UI (not buttons or text positioning), and a few other things.
A first attempt at a real game would likely be coded to give all the threads to each piece of work one at a time in some hand picked order, or at least until this was demonstrate to be slow:
Physics Broad -> Physics Narrow -> Graphics render -> Graphic GPU -> NPC AI -> User input -> Wait/Next frame
But that is likely slow, and I picked our hypothetical math to be slow and marginal. Sending stuff to the GPU is a high latency activity it might take 5 ms to respond, and if this is a 60 FPS game then that is like 1/3 of our time. If we simply add our hypothetical times that is frequently more than 16ms making the game slower than 60fps. Even an ideal frame with just a little physics is right at 15 to 16 ms So a practical game studio might do other work while waiting on the GPU to respond:
Physics Broad -> Physics Narrow -> Graphics render ->
At the same time: { Graphics GPU calls (Uses one thread) NPC AI (Uses all but one thread) -> User input } ->
Wait/Next frame
Most of the time something like this is "fast enough". In this example that 5 ms of CPU time wait on the GPU is now running alongside all that NPC AI so we only need to add the larger of the two. If this takes only a few days of engineer time and keeps the game under the 16ms on most machines then maybe the team makes a business decision to raise the minimum specs just a bit (from 4 to 6 cores would reduce physics time by another ms) and now can they ship this game. There are still needless waits and from a purely GigaFLOPS perspective perhaps much weaker computers could do the work but there is so much waiting that it isn't practical. But this compromise gets all target machines to just about 60 FPS.
Alternatively, if the game is smart enough to make new threads of work for each physics islands (actually super complex and not a great idea in real game, but this all hypothetical but there are similar wins to be had in real games) and manage dependencies carefully based on the the simulation state then something more detailed might be possible:
1. Physics broadphase, create known amount of physics islands.
2. Start a paused GPU thread waiting on known amount of physics islands to be done rendering. This will start step 5 as soon as the last step 4c completes.
3. Add the player's input work to the appropriate group of NPCs
4. Each Physics island gets a thread that does the following: a. Physics narrow phase for this island, b. Partial render for just this island, c. Sets a threadsafe flag this island is done, d. NPC AI is processed for NPCs near this physics island, e. If this is the island with the player process their input.
5. The GPU thread waits for all physics islands threads to get to step 3c then starts sending commands to the GPU! and 3d gets to keep running.
6. When all threads from step 4 and 5 complete pause all game threads to hit the desired frame rate (save battery life for mobile gamers!) or advance to next thread if past frame budget or framerate is uncapped.
This moves all the waits to the end of each thread's frame runtime. This means a bunch of nice things. That last thread can likely do some turbo boosting, a feature of most CPUs where they clock up one CPU if it is the only one working. If the NPCs ever took longer than the GPU they still might complete earlier because they get started earlier. If there are more islands than hardware threads this likely results in better utilization because there are no early pauses.
This would likely a take a ton of engineering time. This might move the frame time down a few more ms and maybe let them lower the minimum requirements perhaps even letting the game run on an older console if market conditions support that. Conceptually, it might be a thing that could be done with async/await, but I don't think that is how most game engines are designed. I also think this makes dependencies implicit and scattered through the code, but likely that could be avoided with careful design.
I am a big fan of libraries that let you provide work units, or functors, and say which depend on each other. They all get to read/write to the global state, but with the dependencies there won't be race conditions. Such libraries locate the threading logic in one place. Then if there is some particularly contentious state that many things need to touch it can be wrapped in a mutex.
I suppose this might just be the iterative vs recursive discussion applied to threading strategies. It just happens that most event loops are single threaded, no reason they need to be in the long run. In the long run I could see making that fastest scenario happen in either paradigm even though the code would look completely different.
Dislaimer i work in gamedev. I think what ppl do in gamedev with tasks/jobs ( different ppl call it differently ) and colorless async with functions that may yield at any time are different. Yielding on I/O means you can not meet a deadline ( frame time ). Not on current hardware that has no I/O deadlines. Which means to me that there is no way we can share library code between async web and realtime part of a game. Ofc games have background best effort computations that can call web-like code and it is fine that it runs for unknown amount of time.
Doesn't it mean you can meet the deadline, but you cannot guarantee that your new textures will be loaded/TLS handshake with login server will be completed/etc. before the deadline happens?
Texture loading and TLS can not meet deadline for sure because we rely on APIs that do not support deadlines. They can only be best effort/background code.
The difference I believe is between updating each UI widget and doing something in case of still missing texture or yielding on the texture in some place of UI code and never touching rest of the UI in the frame.
The difference I believe is between updating each UI widget and doing something in case of still missing texture or yielding on the texture in some place of UI code and never touching rest of the UI in the frame.
I've always felt this is fine, just as long as there are API calls to preload. Then on one screen you start preloading the next screens while the user is navigating your menus to hide all this latency as much as possible.
It doesn't sound to me like the engines you dealt with use ECS, which are usually resolved with a job system (your work units and functors), but correct me if I'm wrong.
The good job systems I've dealt with have their dependencies in the functors. So you "wait" on a job to finish, which is really a while loop that plucks and executes other jobs while the dependency job hasn't finished. This kind of job system is nice to deal with as they are generally low overhead which means all threads (processes really) are generally saturated with work at all times.
I don't really remember any global state with contention because that's generally very very slow, but maybe there were bits of our gameplay code I'm not aware of.
The good job systems I've dealt with have their dependencies in the functors. So you "wait" on a job to finish, which is really a while loop that plucks and executes other jobs while the dependency job hasn't finished. This kind of job system is nice to deal with as they are generally low overhead which means all threads (processes really) are generally saturated with work at all times.
I don't really remember any global state with contention because that's generally very very slow, but maybe there were bits of our gameplay code I'm not aware of.
The ECS concerns don't really relate to threading concerns.
I have worked with and without ECS systems both with and without good threading models. ECS writes do create possible issues if write-locks need to be acquired but that isn't usually so big of a deal.
I have worked with and without ECS systems both with and without good threading models. ECS writes do create possible issues if write-locks need to be acquired but that isn't usually so big of a deal.
In the "you're still going to have to wait for something" sense, sure. But the reason ECS exists is because the industry had to change our architecture when we moved to many core CPUs to take advantage.
I'm battling to understand what you want then, sorry. The systems that you say you would like to use (discrete jobs with dependencies) are the kind of systems the industry has been using since the advent of data-oriented architecture, which includes ECS. That is, a job worker process per core plucking off work and doing it.
In the engines I've dealt with, we don't usually have write locks, instead preferring copies of "last frame data" and "next frame data". And all our "read locks" are waits for jobs. Our game code is generally single threaded, but the main loop pretty much just kicks off and waits for jobs.
I guess what is a good threading model to you?
(As a side note I've worked on projects that use ECS on a single core and they still confer benefits there even though that's not what they were invented for)
I'm battling to understand what you want then, sorry. The systems that you say you would like to use (discrete jobs with dependencies) are the kind of systems the industry has been using since the advent of data-oriented architecture, which includes ECS. That is, a job worker process per core plucking off work and doing it.
In the engines I've dealt with, we don't usually have write locks, instead preferring copies of "last frame data" and "next frame data". And all our "read locks" are waits for jobs. Our game code is generally single threaded, but the main loop pretty much just kicks off and waits for jobs.
I guess what is a good threading model to you?
(As a side note I've worked on projects that use ECS on a single core and they still confer benefits there even though that's not what they were invented for)
I have written Ruby, Elixir, and Typescript on Nodejs.
I have yet to see a good reason for async/await, other than syntax sugar on a flawed language architecture. The very thing people like about Nodejs (async reactor), creates a lot of problems in production web and data pipeline code.
As an aside, Elixir’s Task.async and Task.await are function helpers that work with message passing primitives. Code execution can be truly suspended, and messages are queued. Javascript’s async/await queues code execution rather than messages, and I think that leads to error prone code by design.
I have yet to see a good reason for async/await, other than syntax sugar on a flawed language architecture. The very thing people like about Nodejs (async reactor), creates a lot of problems in production web and data pipeline code.
As an aside, Elixir’s Task.async and Task.await are function helpers that work with message passing primitives. Code execution can be truly suspended, and messages are queued. Javascript’s async/await queues code execution rather than messages, and I think that leads to error prone code by design.
My theory is that JavaScript programmers who were forced into thinking this way for decades with their single-threaded runtime have infected other languages with the idea that this style of coding is not only good but furthermore that it needs to be explicit.
Thank goodness we have wiser language developers out there who have resisted this impulse.
Thank goodness we have wiser language developers out there who have resisted this impulse.
Didn’t async/await originate in c#?
async/await may have originated as keywords in C#, I don't know, but the programming style itself is ancient. Almost every GUI ever written is programmed in the style. Network servers were written this way back when threads were just a fever dream of academics yet. The entire Mac OS ecosystem prior to OSX had to be written in this style because time was cooperatively shared across the entire OS.
This is one of the most blatent "failure to learn from our history" debates I know of, when people act as if the question started with Javascript just a few years ago, when in fact the programming community has experience with this style that predates when most of us were born. I'm getting to old fogey myself and it still predates my 1978 birthdate. I was never even tempted to get into Node because I'd already made an async/await-style mess in Perl before Node was even released, and the mess had everything to do with the programming style and little to do with Perl qua Perl, and the libraries I used were already old and well-established then, if not outright long in the tooth. There is nothing new about this, except keywords.
This is one of the most blatent "failure to learn from our history" debates I know of, when people act as if the question started with Javascript just a few years ago, when in fact the programming community has experience with this style that predates when most of us were born. I'm getting to old fogey myself and it still predates my 1978 birthdate. I was never even tempted to get into Node because I'd already made an async/await-style mess in Perl before Node was even released, and the mess had everything to do with the programming style and little to do with Perl qua Perl, and the libraries I used were already old and well-established then, if not outright long in the tooth. There is nothing new about this, except keywords.
F# first actually. Then C#. Then Haskell. Then Python. Then TypeScript. Parent just has an axe to grind.
Anytime this comes up I plug the excellent "Unyielding" (https://glyph.twistedmatrix.com/2014/02/unyielding.html) and "Notes on structured concurrency" (https://vorpus.org/blog/notes-on-structured-concurrency-or-g...) as the counterpoint to "What color is your function". Being able to see structurally what effects your invocation of a function will result in is very helpful in reasoning about the effects of concurrency in a complicated domain.
Related:
What color is your function? (2015) - https://news.ycombinator.com/item?id=28657358 - Sept 2021 (58 comments)
What Color Is Your Function? (2015) - https://news.ycombinator.com/item?id=23218782 - May 2020 (85 comments)
What Color is Your Function? (2015) - https://news.ycombinator.com/item?id=16732948 - April 2018 (45 comments)
What Color Is Your Function? - https://news.ycombinator.com/item?id=8984648 - Feb 2015 (146 comments)
What color is your function? (2015) - https://news.ycombinator.com/item?id=28657358 - Sept 2021 (58 comments)
What Color Is Your Function? (2015) - https://news.ycombinator.com/item?id=23218782 - May 2020 (85 comments)
What Color is Your Function? (2015) - https://news.ycombinator.com/item?id=16732948 - April 2018 (45 comments)
What Color Is Your Function? - https://news.ycombinator.com/item?id=8984648 - Feb 2015 (146 comments)
It fills me with delight that apparently my lasting contribution to computer science is a piece of writing that also contains the phase "Spidermouth the Night Clown".
Don't sell Crafting Interpreters short!
Fair. But that one has a hand-drawn picture of an alligator eating characters and pooping tokens, so it's in about the same category of maturity. :)
Do you still maintain more or less the same opinion from the post?
Yeah, I do, actually.
Dart, which I work on, is still a colored language. And it's like, fine. But I do wish it was colorless. It would make library design a lot easier. There is real friction all the time when doing API design to decide which things should and shouldn't be async.
Dart, which I work on, is still a colored language. And it's like, fine. But I do wish it was colorless. It would make library design a lot easier. There is real friction all the time when doing API design to decide which things should and shouldn't be async.
Translated into simple language, Ruby chose to expose the multithreading paradigm (multple threads over shared data), like Java and others.
Multithreading is strictly more powerful than single threaded event loops. For some kinds of software there is just no alternative - a modern browser engine for example needs to be multithreaded.
The trade off is that you need to make sure your code is thread safe, which is not trivial as the collection of articles explains. That's your function color right there, green functions are verified thread safe, gray functions are not or not sure.
Personally in nearly 30 years of programming I've never needed to write multithreaded code. I still haven't found a business need that could not be met with suitable choices between multiprocessing (i.e fork) and event loops.
I'll definitely take wait/async programming over having to worry about concurrent thread safety any day of the week.
Multithreading is strictly more powerful than single threaded event loops. For some kinds of software there is just no alternative - a modern browser engine for example needs to be multithreaded.
The trade off is that you need to make sure your code is thread safe, which is not trivial as the collection of articles explains. That's your function color right there, green functions are verified thread safe, gray functions are not or not sure.
Personally in nearly 30 years of programming I've never needed to write multithreaded code. I still haven't found a business need that could not be met with suitable choices between multiprocessing (i.e fork) and event loops.
I'll definitely take wait/async programming over having to worry about concurrent thread safety any day of the week.
As they should be.
I object to doing what a computer can do for me (in programming), and manually creating separate versions of functions that are identical up to async absolutely falls into that category.
I object to doing what a computer can do for me (in programming), and manually creating separate versions of functions that are identical up to async absolutely falls into that category.
So use a language that knows how to be polymorphic over async. Just like you don't want to have to write one version of sort() for each possible array element type, but the solution isn't to make all array elements untyped, the solution is to have a language that can abstract over that.
Can you give an example of a language which is polymorphic in this way and how that looks?
Haskell or Scala are the immediate examples. The idiomatic way to write async-style functions in those languages tends to be do notation or for/yield style. E.g.
In Scala you need an explicit type parameter if you want to be polymorphic:
transformFile = do {
x <- readFile(...)
y = process(x)
z <- writeFile(...)
}
async transformFile = {
x = await readFile(...)
y = process(x)
z = await writeFile(...)
}
In Scala you need an explicit type parameter if you want to be polymorphic:
def transformFile = for {
x <- readFile(...)
y = process(x)
z <- writeFile(...)
} yield ()
def transformFile[F[_]: Monad] = for {
x <- readFile(...)
y = process(x)
z <- writeFile(...)
} yield ()
Ah I've got it. Async is a monad and the function is just polymorphic over Monad m. In the sync case m is Identity. Can you point me in the direction of some libraries which offer functions like these?
sttp (HTTP client), http4s (HTTP client/server), quill (database access), fs2 (streaming data processing), I have some vague memory of a gRPC implementation. A lot of stuff that does async in Scala tends to be written in this style because it's almost no extra cost compared to writing it in strictly async fashion.
Yeah, that's the correct solution. I'm not arguing for the solution, I'm arguing for the problem, because a lot of people seem to think that the problem isn't real.
Sure, but if you put it that way it tends to come across as wanting to not track the distinction at all. Like if you said it's good that values are untyped in Ruby because to write a collection type in C you have to reimplement it for every different type of collection you want: yes, but that doesn't mean values having types is bad, it just means you need polymorphism.
They have different signatures because they return different things.
The difference between returning a Foo and Promise<Foo> is utterly irrelevant in this case because the computer is capable of automatically handing the difference.
Great article! I'm looking forward to reading the rest of the series.
I noticed a couple details that seem wrong:
- You are passing `context` to `log_then_get` and `get`, but you never use it. Perhaps that is left over from a previous version of the post?
- In the fiber example you do this inside each fiber:
- Doesn't this leave `responses` with 8 elements instead of 4?
- What does `Fiber.schedule` return anyway? At best it can only be something like a promise, right? It can't be the result of the block. I don't see the answer in the docs: https://ruby-doc.org/3.3.4/Fiber.html#method-c-schedule
- When each fiber internally appends to `responses`, it is asynchronous, so are there concurrency problems? Array is not thread-safe I believe. So with fibers is this safe? If so, how/why? (I assume the answer is "because we are using a single-threaded scheduler", but that would be interesting to put in the post.)
I noticed a couple details that seem wrong:
- You are passing `context` to `log_then_get` and `get`, but you never use it. Perhaps that is left over from a previous version of the post?
- In the fiber example you do this inside each fiber:
responses << log_then_get(URI(url), Fiber.current)
responses << get_http_fiber(...)
- Doesn't this leave `responses` with 8 elements instead of 4?
- What does `Fiber.schedule` return anyway? At best it can only be something like a promise, right? It can't be the result of the block. I don't see the answer in the docs: https://ruby-doc.org/3.3.4/Fiber.html#method-c-schedule
- When each fiber internally appends to `responses`, it is asynchronous, so are there concurrency problems? Array is not thread-safe I believe. So with fibers is this safe? If so, how/why? (I assume the answer is "because we are using a single-threaded scheduler", but that would be interesting to put in the post.)
I feel like I have some intuative understanding of how go achieves colorless concurrency using "go routines" that can park sync/blocking io on a thread "as needed" built into the runtime from the very begining.
I don't understand how Ruby added this after the fact, globally to ALL potential cpu/io blocking libraries/functions without somehow expressing `value = await coro`
Python is currently going through an "coloring" as the stdlib and 3rd-party libraries adapt to the explicit async/await syntax and it's honestly kind of PITA. Curious if there's any more info on how Ruby achived this.
I don't understand how Ruby added this after the fact, globally to ALL potential cpu/io blocking libraries/functions without somehow expressing `value = await coro`
Python is currently going through an "coloring" as the stdlib and 3rd-party libraries adapt to the explicit async/await syntax and it's honestly kind of PITA. Curious if there's any more info on how Ruby achived this.
> I don't understand how Ruby added this after the fact, globally to ALL potential cpu/io blocking libraries/functions without somehow expressing `value = await coro`
In Python gevent can monkeypatch the standard library to do that transparently (mostly). I assume it works the same way except better: have the runtime keep track of the current execution (mostly the stack and ip), when reaching a blocking point register a completion event agains the OS then tell the fiber scheduler to switch off, the fiber scheduler puts away all the running state (stack and instruction pointer) then restores an other one (hopefully one that's ready for execution) and resumes that.
In Python gevent can monkeypatch the standard library to do that transparently (mostly). I assume it works the same way except better: have the runtime keep track of the current execution (mostly the stack and ip), when reaching a blocking point register a completion event agains the OS then tell the fiber scheduler to switch off, the fiber scheduler puts away all the running state (stack and instruction pointer) then restores an other one (hopefully one that's ready for execution) and resumes that.
Java has shown that a sufficiently strong runtime can take the burden onto itself long after the facts have been established.
Python in contrast has a runtime ultimately held back by its deep integration with the C-ecosystem which creates more black-boxes inside the runtime than you can find inside a swiss cheese.
Similar with C# and Rust. No strong runtime, no colorless functions for you.
Python in contrast has a runtime ultimately held back by its deep integration with the C-ecosystem which creates more black-boxes inside the runtime than you can find inside a swiss cheese.
Similar with C# and Rust. No strong runtime, no colorless functions for you.
Greenlet has been available for Python since 2006. Troubles with asyncio are mostly self-inflicted.
> Similar with C# and Rust. No strong runtime, no colorless functions for you.
Zig is the exception here but not sure how well it worked out in practice.
Zig is the exception here but not sure how well it worked out in practice.
Well, the feature is temporarily killed, but users can enjoy zigcoro which exposes a similar API until the old async is back.
C# has a strong runtime.
The C code in Ruby has to yield to the scheduler when it detects a fiber.
For historical context, python had several "colorless" attempts but none achieved widespread adoption.
For historical context, python had several "colorless" attempts but none achieved widespread adoption.
Unless I'm misunderstanding: isn't the JVM's virtual threads another instance of this colorlessness?
I think you're right. People describe Java as being colourless.
> 3. You can only call a red function from within a red function
The base of most arguments against async. And it's false. You can call red from blue. And you should, sometimes.
The base of most arguments against async. And it's false. You can call red from blue. And you should, sometimes.
Yes... But it's a PITA most of the time, right ? I'm not sure for JS, as I can't remember right now but it's a annoying as f*k in python at least
In JS it’s not an issue as long as you just fire and forget. Because calls to async functions essentially fork off tasks. No way to synchronise on them though.
Python (and rust) are coroutine based so calling an async function does essentially nothing, you need to acquire a runtime handle in order to run and resolve the coroutine.
Python (and rust) are coroutine based so calling an async function does essentially nothing, you need to acquire a runtime handle in order to run and resolve the coroutine.
99.9% of these "colored" function articles have an incomplete or even flawed understanding of async/await symantics.
Fibers are not fungible with async/await. This is why structured concurrency is a thing.
Fibers are not fungible with async/await. This is why structured concurrency is a thing.
Kotlin solved this pointless debate long time ago the moment they’ve released coroutines.
Best of both worlds: you no longer have two functions with ReturnType and Promise<ReturnType>. You just mark potentially blocking function with suspend and you’re done.
Best of both worlds: you no longer have two functions with ReturnType and Promise<ReturnType>. You just mark potentially blocking function with suspend and you’re done.
I don’t see how. Only suspending functions can call other suspending functions. You still end up having to mark your call stack all the way up.
I'm confused, and please correct me if I'm wrong.
Aren't all these calls blocking? Doesn't `File.read` still block? Sure it's multithreaded, but it still blocks. Threading vs an event loop are two different concurrency models.
Aren't all these calls blocking? Doesn't `File.read` still block? Sure it's multithreaded, but it still blocks. Threading vs an event loop are two different concurrency models.
> Aren't all these calls blocking?
Only locally, which is pretty much the same as when you `await` a call.
> Threading vs an event loop are two different concurrency models.
The point is that you can build a "threading" (blocking) model on top of an event loop, such that you get a more natural coding style with most of the gain in concurrency.
It loses some, because you only have concurrency between tasks (~threads) leaving aside builtins which might get special cased, but my observation has been that the vast majority of developers don't grok or want to use generalised sub-task concurrency anyway, and are quite confused when that's being done.
For instance in my Javascript experience the average developer just `awaits` things as they get them, the use of concurrent composition (e.g. Promise.all) is an exception which you might get into as an optimisation, e.g. `await`-ing in the middle of a loop is embarrassingly common even when the iterations have no dependency.
Only locally, which is pretty much the same as when you `await` a call.
> Threading vs an event loop are two different concurrency models.
The point is that you can build a "threading" (blocking) model on top of an event loop, such that you get a more natural coding style with most of the gain in concurrency.
It loses some, because you only have concurrency between tasks (~threads) leaving aside builtins which might get special cased, but my observation has been that the vast majority of developers don't grok or want to use generalised sub-task concurrency anyway, and are quite confused when that's being done.
For instance in my Javascript experience the average developer just `awaits` things as they get them, the use of concurrent composition (e.g. Promise.all) is an exception which you might get into as an optimisation, e.g. `await`-ing in the middle of a loop is embarrassingly common even when the iterations have no dependency.
The whole point of async/await is to allow for not blocking the caller though until it's ready for an explicit synchronization point.
If you are blocking the caller you have not "solved" the colored function "problem".
If you are blocking the caller you have not "solved" the colored function "problem".
> The whole point of async/await is to allow for not blocking the caller though until it's ready for an explicit synchronization point.
That is an end not a mean.
Again in my experience the vast majority of devs could not give less of a shit about “not blocking the caller” by default. What most devs want is a reasonably cheap way to get a high amount of concurrency.
If anything not blocking the caller by default is generally an error, because somebody forgot an await.
> If you are blocking the caller you have not "solved" the colored function "problem".
Of course you have: you have solved the actual problem that needs solving without using function colouring. That’s how e.g. Go works. Go has problems up the ass but it doesn’t have that one at least.
Java is also moving back to Userland threads rather than towards async/await.
That is an end not a mean.
Again in my experience the vast majority of devs could not give less of a shit about “not blocking the caller” by default. What most devs want is a reasonably cheap way to get a high amount of concurrency.
If anything not blocking the caller by default is generally an error, because somebody forgot an await.
> If you are blocking the caller you have not "solved" the colored function "problem".
Of course you have: you have solved the actual problem that needs solving without using function colouring. That’s how e.g. Go works. Go has problems up the ass but it doesn’t have that one at least.
Java is also moving back to Userland threads rather than towards async/await.
And it looks pretty much exactly like using threads, which is why they are working so hard on structured concurrency.
Go has NOT solved the problem in a fungible way, as evidence by all the dual APIs; methods that return a channel, and those that don't.
CSP is great at modeling data flow, but IMHO it's lesser than async/await imperative programing for modeling more standard business logic flows.
Go has NOT solved the problem in a fungible way, as evidence by all the dual APIs; methods that return a channel, and those that don't.
CSP is great at modeling data flow, but IMHO it's lesser than async/await imperative programing for modeling more standard business logic flows.
You can build promises with fibers too if you just want to use the value you'd "await" on later on in execution, it's trivial.
You have to watch your definition of "blocking". Node, well, I won't say it created the definition of "blocking" that means "it blocks your entire OS process regardless of how many other things you're trying to do concurrently", but it certainly popularized it, and a lot of people sloppily project the negative aspects of that into threaded languages. In a threaded language, yes, you block on a .Read call until it is complete in that thread, but you don't block the whole OS process; other threads can and do continue on.
By default, File.read does block, yes. As it crosses into libc, it releases Ruby's interpreter lock, allowing another Ruby thread to execute while it blocks.
Ruby 1.9 added Fibers. These are coroutines with their own call stack that yield and resume explicitly. They're like Goroutines but without Go's scheduler. Fibers are commonly used to build Enumerators, internal iterators support external iteration. File.read in a Fiber still blocks by default.
Ruby 3.0 added support for truly asynchronous File.read. The batteries are not included. A fiber scheduler is required to enable this optional feature:
https://docs.ruby-lang.org/en/3.3/Fiber.html#class-Fiber-lab...
https://brunosutic.com/blog/ruby-fiber-scheduler
Ruby 1.9 added Fibers. These are coroutines with their own call stack that yield and resume explicitly. They're like Goroutines but without Go's scheduler. Fibers are commonly used to build Enumerators, internal iterators support external iteration. File.read in a Fiber still blocks by default.
Ruby 3.0 added support for truly asynchronous File.read. The batteries are not included. A fiber scheduler is required to enable this optional feature:
https://docs.ruby-lang.org/en/3.3/Fiber.html#class-Fiber-lab...
https://brunosutic.com/blog/ruby-fiber-scheduler
I loved Ruby as a total beginner but hate it as an experienced programmer.
Colorless brings no meaning when i look at the signature of a method, which is a warning !
Async at the boundary, sync at the core is my favorite paradigm.
Colorless brings no meaning when i look at the signature of a method, which is a warning !
Async at the boundary, sync at the core is my favorite paradigm.
So maybe it's me, but isn't that line mapping on `&:value` in Ruby the exact equivalent of doing `Promise.all` on a bunch of async functions in Javascript, with the downside that you don't explicitly say that the array you calling `value` on is a bunch of asynchronous things that need to be (a)waited for to realize? In other words, since you have color anyway, isn't it better to highlight that upfront rather than hiding it until you need to actually use the asynchronous return values?
> Even more onerous, if it isn’t built into your language core like JavaScript/node.js, adding it later means modifying your entire runtime, libraries and codebases to understand it.
Interestingly, while this has proven true of async/await for many languages it has not at all been true for perl.
The pluggable keywords feature lets us register 'async' and 'await' with the parser as (block scoped) imported keywords and with a little suspend/resume trickery you get https://p3rl.org/Future::AsyncAwait which I've been using happily pretty much since it was released (generally operating on https://p3rl.org/IO::Async::Future and https://p3rl.org/Mojo::Promise objects, often both in the smae process).
I even wrote https://p3rl.org/PerlX::AsyncAwait as a pure perl proof of concept later on, which injects computed gotos as resume points ala the switch/case trick you can use for resumable functions in C (nobody should really be using that one, mind, I wrote it to prove that I could and as potential fodder for https://p3rl.org/App::FatPacker usage later).
I do very much appreciate there are a lot of reasons one might dislike perl (I've been writing it long enough my list is probably longer than most naysayers') but its sheer malleability as a language remains unusually good.
Interestingly, while this has proven true of async/await for many languages it has not at all been true for perl.
The pluggable keywords feature lets us register 'async' and 'await' with the parser as (block scoped) imported keywords and with a little suspend/resume trickery you get https://p3rl.org/Future::AsyncAwait which I've been using happily pretty much since it was released (generally operating on https://p3rl.org/IO::Async::Future and https://p3rl.org/Mojo::Promise objects, often both in the smae process).
I even wrote https://p3rl.org/PerlX::AsyncAwait as a pure perl proof of concept later on, which injects computed gotos as resume points ala the switch/case trick you can use for resumable functions in C (nobody should really be using that one, mind, I wrote it to prove that I could and as potential fodder for https://p3rl.org/App::FatPacker usage later).
I do very much appreciate there are a lot of reasons one might dislike perl (I've been writing it long enough my list is probably longer than most naysayers') but its sheer malleability as a language remains unusually good.
def log_then_get(url, context)
puts "Requesting #{url}..."
get(url, context)
end
def get(uri, context)
response = Net::HTTP.get(uri)
puts caller(0).join("\n")
response
end
def get_http_thread(url)
Thread.new do
log_then_get(URI(url), Thread.current)
end
end
1) get_http_thread takes a url (String) and converts it to a URI object
2) log_then_get defines its parameter as `url`, but really its expecting a URI object
3) get defines its parameter as `uri`, but we're passing it an argument called `url` from within log_then_get.
Lots of traps readily awaiting an unsuspecting programmer or newcomer to a project that contains code like this.
Maybe its Stockholm syndrome after ~4-5 years of TypeScript, but I like knowing "this method call is going to do I/O somewhere" (that its red).
To the point where I consider "colorless functions" to be a leaky abstraction; i.e. I do a lot of ORM stuff, and "I'll just call author.getBooks().get(0) b/c that is a cheap, in-memory, synchronous collection access ... oh wait its actually a colorless SQL call that blocks (sometimes)" imo led to ~majority of ORM backlash/N+1s/etc.
Maybe my preference for "expressing IO in the type system" means in another ~4-5 years, I'll be a Haskell convert, or using Effect.ts to "fix Promise not being a true monad" but so far I feel like the JS Promise/async/await really is just fine.
To the point where I consider "colorless functions" to be a leaky abstraction; i.e. I do a lot of ORM stuff, and "I'll just call author.getBooks().get(0) b/c that is a cheap, in-memory, synchronous collection access ... oh wait its actually a colorless SQL call that blocks (sometimes)" imo led to ~majority of ORM backlash/N+1s/etc.
Maybe my preference for "expressing IO in the type system" means in another ~4-5 years, I'll be a Haskell convert, or using Effect.ts to "fix Promise not being a true monad" but so far I feel like the JS Promise/async/await really is just fine.
I seem to recall a similar argument being made a while ago: https://scholar.harvard.edu/files/waldo/files/waldo-94.pdf
Don't you generally know when you're making an I/O call?
Not when I'm using some library function that's "helpful"
>Because threads share the same memory space they have to be carefully coordinated to safely manage state. Ruby threads cannot run CPU-bound Ruby code in parallel, but they can parallelize for blocking operations
Ugh. I know Ruby (which I used to code in a lot more) has made some real progress toward enabling practical use of parallelism but this sounds still pretty awful.
Is there any effort to make sharing data across threads something that doesn't have to be so "carefully coordinated" (ala Clojure's atom/swap!, ref/dosync)?
Is the inability to parallelize CPU-bound code to do with some sort of GIL?
Ugh. I know Ruby (which I used to code in a lot more) has made some real progress toward enabling practical use of parallelism but this sounds still pretty awful.
Is there any effort to make sharing data across threads something that doesn't have to be so "carefully coordinated" (ala Clojure's atom/swap!, ref/dosync)?
Is the inability to parallelize CPU-bound code to do with some sort of GIL?
That's what Ractor is for, if you want full parallelization without processes.
And, yes, it's to do with a GIL/GVL. The lock is released during blocking IO, and some C extensions etc., so in practice for a lot of uses it's fine.
And, yes, it's to do with a GIL/GVL. The lock is released during blocking IO, and some C extensions etc., so in practice for a lot of uses it's fine.
It all depends on your Ruby runtime.
If you want parallel threads then you can use JRuby and your threads will run in parallel on the JVM. I've used the Concurrent-Ruby gem to coordinate that[1].
It has copies of some of the Clojure data structures.
Otherwise, Ractors the up coming solution for MRI Ruby.
1. https://github.com/ruby-concurrency/concurrent-ruby
If you want parallel threads then you can use JRuby and your threads will run in parallel on the JVM. I've used the Concurrent-Ruby gem to coordinate that[1].
It has copies of some of the Clojure data structures.
Otherwise, Ractors the up coming solution for MRI Ruby.
1. https://github.com/ruby-concurrency/concurrent-ruby
They ditched the GIL a while a ago. But there are smaller locks fighting for resources.
EDIT - I remember when patch notes years ago said the GIL was gone and this says there is a GVL, I guess there is some subtle difference.
then I think for practical purposes, "yes" is your answer, but not in precisely that name.
EDIT - I remember when patch notes years ago said the GIL was gone and this says there is a GVL, I guess there is some subtle difference.
then I think for practical purposes, "yes" is your answer, but not in precisely that name.
> Async code bubbles all the way to the top. If you want to use await, then you have to mark your function as async. Then if someone else calling your function wants to use await, they also have to mark themselves as async, on and on until the root of the call chain. If at any point you don’t then you have to use the async result (in JavaScript’s case a Promise<T>).
I find many descriptions of async code to be confusing, and this kind of description is exactly why.
This description is backwards. You don't choose to use await and then decorate functions with async. Or maybe you do and that's why so many async codebases are a mess.
You don't want to block while a long running operation completes, so you decorate the function that performs that operation with async and return a Promise.
But Promises have exactly the same value as promises in the real world: none until they are fulfilled. You can't do further operations on a promise, you can only wait for it to be done, you have to wait for the promise to be fulfilled to get the result that you actually want to operate on.
The folly of relying on a promise is embodied in the character Whimpy from Popeye: "I'll gladly pay you Tuesday for a hamburger today".
Once you have a promise, you have to await on it, turning the async operation into a synchronous operation.
This example seems crazy to me:
await is the point where an async operation is blocked on to get back into a synchronous flow. Creating promises means you ultimately need to block in a synchronous function to give the single threaded runtime a chance to make progress on all the promises. Done properly, this happens by the event loop. But setting that up requires the actual operation of all your code to be async and thus callback hell and the verbose syntactic salt to even express that in code.
That all being said, this piece is spot on. Threads (in general, but in ruby as the topic of this piece) and go's goroutines encapsulate all this by abstracting over the state management of different threads of execution via stacks. Remove the stacks and async programming requires you to manage that state yourself. Async programming removes a very useful abstraction.
Independent threads of execution, if they are operating system managed threads, operating system managed processes (a special case of OS managed threads), green threads, or go routines, are a scheduler abstraction. Async programming forces you to manage that scheduling. Which may be required if you don't also have an abstraction available for preemption, but async leaks the single threaded implementation into your code, and the syntactic salt necessary to express it.
I find many descriptions of async code to be confusing, and this kind of description is exactly why.
This description is backwards. You don't choose to use await and then decorate functions with async. Or maybe you do and that's why so many async codebases are a mess.
You don't want to block while a long running operation completes, so you decorate the function that performs that operation with async and return a Promise.
But Promises have exactly the same value as promises in the real world: none until they are fulfilled. You can't do further operations on a promise, you can only wait for it to be done, you have to wait for the promise to be fulfilled to get the result that you actually want to operate on.
The folly of relying on a promise is embodied in the character Whimpy from Popeye: "I'll gladly pay you Tuesday for a hamburger today".
Once you have a promise, you have to await on it, turning the async operation into a synchronous operation.
This example seems crazy to me:
async function readFile(): Promise<string> {
return await read();
}
await is the point where an async operation is blocked on to get back into a synchronous flow. Creating promises means you ultimately need to block in a synchronous function to give the single threaded runtime a chance to make progress on all the promises. Done properly, this happens by the event loop. But setting that up requires the actual operation of all your code to be async and thus callback hell and the verbose syntactic salt to even express that in code.
That all being said, this piece is spot on. Threads (in general, but in ruby as the topic of this piece) and go's goroutines encapsulate all this by abstracting over the state management of different threads of execution via stacks. Remove the stacks and async programming requires you to manage that state yourself. Async programming removes a very useful abstraction.
Independent threads of execution, if they are operating system managed threads, operating system managed processes (a special case of OS managed threads), green threads, or go routines, are a scheduler abstraction. Async programming forces you to manage that scheduling. Which may be required if you don't also have an abstraction available for preemption, but async leaks the single threaded implementation into your code, and the syntactic salt necessary to express it.
To be fair to the author, they do mention in the paragraph above that sample:
Async code bubbles all the way to the top. If you want to use await, then you have to mark your function as async. [...] If at any point you don’t then you have to use the async result (in JavaScript’s case a Promise<T>).
> This example seems crazy to me […]
To be fair, this is sort of pointless in JS/TS because an async function returns a promise type by default, so the return value has to be ‘await’ed to access the value anyways. There are linter rules you can use to ban ‘return await’.
The only benefit to ‘return await…’ is when wrapped with a try/catch - you can catch if whatever you ‘await’ed throws an error.
To be fair, this is sort of pointless in JS/TS because an async function returns a promise type by default, so the return value has to be ‘await’ed to access the value anyways. There are linter rules you can use to ban ‘return await’.
The only benefit to ‘return await…’ is when wrapped with a try/catch - you can catch if whatever you ‘await’ed throws an error.
Well, that was a simplistic contrived example, it may make sense to do this if you have other functionality you want to put in the wrapper. But I've seen more of these blind wrappers, the result of stuffing async and await keywords around in some misguided attempt to make things go faster, in code than is really warranted so it's not being made clear that examples like this are for explanatory purposes and shouldn't be cargo-culted (if anything should be cargo-culted).
Only having blue functions is not the same as being colorless.
The argument for "color"ed functions in Javascript is flawed and comes from somebody with a (very) shallow understanding of the language.
Javascript is as "colorless" any other programming language. You can write "async"-style code without using async/await at all while it being functionally equivalent.
Async/await is just syntactic sugar that saves you from writing "thenable" functions and callbacks again and again and again ...
Instead of:
1: https://en.wikipedia.org/wiki/Pyramid_of_doom_(programming)
Javascript is as "colorless" any other programming language. You can write "async"-style code without using async/await at all while it being functionally equivalent.
Async/await is just syntactic sugar that saves you from writing "thenable" functions and callbacks again and again and again ...
Instead of:
function(...).then(function() {
// continue building your pyramid of doom [1]
})
await function()
1: https://en.wikipedia.org/wiki/Pyramid_of_doom_(programming)
If you program with promises and without async/await, then your language is still missing something compared to languages that have coroutines in which functions that suspend and do not suspend may be composed (used by higher-order functions, etc.) in the same ways as each other. You've moved from the situation where both types of functions were present and must be handled differently to the situation where one type of function is entirely missing.
Can you give an example?
An example of a language or of a program? You could try Lua, Python with greenlet, Zig 0.10.0 or Zig master with zigcoro, dozens of libraries that add this sort of capability to C, or becoming a the kind of person who uses search engines when they have questions.
BulletML is not even Turing complete and still has a wait function that does the exact thing mentioned.
BulletML is not even Turing complete and still has a wait function that does the exact thing mentioned.
>or becoming a the kind of person who uses search engines when they have questions
Just FYI, please become familiar with this site's guidelines before posting [1]; try to make @dang's work a bit easier.
>An example of a language or of a program?
A small code snippet would suffice, any language of your choice that gets the point across more meaningfully. Something like:
1: https://news.ycombinator.com/newsguidelines.html
Just FYI, please become familiar with this site's guidelines before posting [1]; try to make @dang's work a bit easier.
>An example of a language or of a program?
A small code snippet would suffice, any language of your choice that gets the point across more meaningfully. Something like:
function() {
// code that shows feature
}
1: https://news.ycombinator.com/newsguidelines.html
I think there's a reasonable motivating example at https://www.chiark.greenend.org.uk/~sgtatham/coroutines.html, but not a reasonable implementation. A reasonable implementation for C is at https://github.com/creationix/libco and for Zig at https://github.com/rsepassi/zigcoro.
For a real world example, https://github.com/alexnask/iguanaTLS is a TLS library written in terms of the stdlib's reader and writer interfaces. If the read and write methods of the provided reader and writer are normal, non-async functions, then the library's functions are too, and they can be used in a program that does not have an async runtime. If they are async functions, then the library's functions inherit this property, and they can be used in a program that uses async I/O. They could also be used in both modes in the same program, though I can't think of a good reason to do this. Normally in Javascript the consumers of your library would all be imprisoned within an event loop provided by node or the browser, so there would be no point exposing a synchronous variant, but for example see https://nullderef.com/blog/rust-async-sync/ for someone's experience trying to write a library that exposes the same functionality both ways in some other language.
For a real world example, https://github.com/alexnask/iguanaTLS is a TLS library written in terms of the stdlib's reader and writer interfaces. If the read and write methods of the provided reader and writer are normal, non-async functions, then the library's functions are too, and they can be used in a program that does not have an async runtime. If they are async functions, then the library's functions inherit this property, and they can be used in a program that uses async I/O. They could also be used in both modes in the same program, though I can't think of a good reason to do this. Normally in Javascript the consumers of your library would all be imprisoned within an event loop provided by node or the browser, so there would be no point exposing a synchronous variant, but for example see https://nullderef.com/blog/rust-async-sync/ for someone's experience trying to write a library that exposes the same functionality both ways in some other language.
I see your point.
What I would do on JS is ...
... on the lib. side, code a single function that behaves the same when meant to run sync or async.
... on the client side, just await the function every time is called; if the sync version is running you don't return a Promise and await-ing on primitives is free, the program will lock automatically if needed.
Obviously, the trivial solution would be two different methods that do the same thing (as is the case now with things like readFile and readFileSync) but I agree that's not elegant.
What I would do on JS is ...
... on the lib. side, code a single function that behaves the same when meant to run sync or async.
... on the client side, just await the function every time is called; if the sync version is running you don't return a Promise and await-ing on primitives is free, the program will lock automatically if needed.
Obviously, the trivial solution would be two different methods that do the same thing (as is the case now with things like readFile and readFileSync) but I agree that's not elegant.
Nope.
- anonymoushnvcxbx(3)
Ruby is like Japanese candlestick charts, both of them are absolute bullshit, idiotic things.
Other languages may handle it differently, but having to manage threads is not a small compromise for going colorless. You're now forced to deal with thread creation, thread pooling, complexities of nested threads, crashes within child or descendant threads, risks of shared state, more difficult unit testing, etc.
I don't like colored function for obvious reasons, but fully colorless for async means you don't know when things are async or not.
There are a lot of things I dislike in JS, but I think the I/O async model is just right from an ergonomics point of view. The event loop is implicit, any async function returns a promise, you can deal with promises from inside sync code without much trouble.
It's just the right balance.
There are a lot of things I dislike in JS, but I think the I/O async model is just right from an ergonomics point of view. The event loop is implicit, any async function returns a promise, you can deal with promises from inside sync code without much trouble.
It's just the right balance.
> fully colorless for async means you don't know when things are async or not
The IDE can tell you.
The IDE can tell you.
> The IDE can tell you.
The only way the IDE can tell you is if the language tells it, or it guesses - and if it guesses then it will get it wrong sometimes. Which things are async or not is exactly the kind of thing that needs to be part of the language definition, so that all tools will agree about it and you won't have functions that are async in one IDE and not another, or async in a profiler but not an IDE, or...
The only way the IDE can tell you is if the language tells it, or it guesses - and if it guesses then it will get it wrong sometimes. Which things are async or not is exactly the kind of thing that needs to be part of the language definition, so that all tools will agree about it and you won't have functions that are async in one IDE and not another, or async in a profiler but not an IDE, or...
Given Ruby culture of monkey patching, not always.
Besides, many people dev Ruby with a lightweight text editor, like text mate, that can't introspect code.
Besides, many people dev Ruby with a lightweight text editor, like text mate, that can't introspect code.
> ...culture of monkey patching...
I haven't seen more than a handful of PRs with monkey-patching in the last decade and even then they are accompanied by long-winded explanations/apologies and plans for removal ASAP (eg monkey-patching an up streamed fix that hasn't been released yet).
Also, ruby classes/methods can tell you all about themselves, so if you haven't got ruby-lsp up and running (and even if you do) you can always open a repl (on its own, in the server process, or in a test process) and ask the interpreter about your method. It's pretty great!
It's definitely the case that the editor's ability to understand code via static analysis is limited compared to other languages, but it's not like we ruby devs are navigating in the dark.
I haven't seen more than a handful of PRs with monkey-patching in the last decade and even then they are accompanied by long-winded explanations/apologies and plans for removal ASAP (eg monkey-patching an up streamed fix that hasn't been released yet).
Also, ruby classes/methods can tell you all about themselves, so if you haven't got ruby-lsp up and running (and even if you do) you can always open a repl (on its own, in the server process, or in a test process) and ask the interpreter about your method. It's pretty great!
It's definitely the case that the editor's ability to understand code via static analysis is limited compared to other languages, but it's not like we ruby devs are navigating in the dark.
If you monkey patch, you get what you paid for - and an annotation at the call site or definition wouldn't help anyway!
If not, then we should be able to use the type annotations that are being added to also indicate async-ness.
If people decide to code Ruby "blind" (without a smart IDE), then that's their choice. There's no reason why someone using an IDE should have to pay for their decisions. We don't force people to manually and redundantly add names and types of parameters to call sites - it makes equally little sense to do the same for async. If someone decides to use a dumb IDE, then they can read the docs, exactly the same as they do for function parameters.
If not, then we should be able to use the type annotations that are being added to also indicate async-ness.
If people decide to code Ruby "blind" (without a smart IDE), then that's their choice. There's no reason why someone using an IDE should have to pay for their decisions. We don't force people to manually and redundantly add names and types of parameters to call sites - it makes equally little sense to do the same for async. If someone decides to use a dumb IDE, then they can read the docs, exactly the same as they do for function parameters.
Can't the text editor open a shell where you can run the repl to do the inspection?
I’m all ears how IDE will determine that function is blocking 20 layers deep into third party library I don’t even have source code of.
Knowing ahead of time which functions are async is a feature.
It's a big neon sign that says "hey, this function call is expensive". This is a good thing for programmers to easily see and know at the call site.
If you make multiple calls with async/await in a row, the performance issues are plainly obvious at the call site. With "colorless" functions, this information is hidden in a deeper layer. You have to know what the function does on the inside to even know what its performance impacts are.
Also, a nitpick - you can call async functions from sync ones, you just can't access the return value. Sometimes, you don't need to.