Code optimizations when using async/await(wolfoliver.medium.com)
wolfoliver.medium.com
Code optimizations when using async/await
https://wolfoliver.medium.com/the-fallacy-of-async-await-c0df0f9dd59d
43 コメント
It looks like thread-per-core stuff like seastar (C++) and glommio (Rust) are still the state of the art, along with userland network drivers.
glommio seems to be like a seda-thread-per-core using multiple rings as staged event buffers while running on dedicated cores.
glommio seems to be like a seda-thread-per-core using multiple rings as staged event buffers while running on dedicated cores.
What's the pros/cons vs Promise.all([...])?
> Assuming the first request is done much faster than the second request, we can already utilize the CPU and doSomeCalculation while waiting for the second request to finish. In this case, the shown code is better than Promise.all([request1, request2]).
Promise.all([request1.then(doSomeWork), request2])
?yes this would work. It's just nested.
The nesting has the advantage of directly expressing the dependency structure of the promises, which is otherwise a bit hard to infer.
This article doesn’t have any depth.
TLDR: avoid blocking unrelated async requests via await.
What’s far more interesting and harder to solve ergonomically is to parallelize async functions - parse responses to validate if they’ve succeeded or failed via Promise.allSettled, and then continue to work with the resolved values while retrying any failed requests.
That’s what I was hoping this would go into but unfortunately it doesn’t.
TLDR: avoid blocking unrelated async requests via await.
What’s far more interesting and harder to solve ergonomically is to parallelize async functions - parse responses to validate if they’ve succeeded or failed via Promise.allSettled, and then continue to work with the resolved values while retrying any failed requests.
That’s what I was hoping this would go into but unfortunately it doesn’t.
The article makes a point that is both a) simple and self-evident and b) ignored by 90% of JS developers I've encountered, who are just trying to write procedural code and adding as many awaits as needed to achieve that.
This is why I'm convinced that async should be opt-in, because most people don't use it effectively even when forced to acknowledge its existence.
This is why I'm convinced that async should be opt-in, because most people don't use it effectively even when forced to acknowledge its existence.
You would think this has no depth, but I have seen code like that waits in serial over and over. So clearly it is deep for the novice, which are in abundance.
Yeah this domain of problems doesn't seem to have a "neat" solution either built in or via some defacto library. I'm guessing you want to easily be able to say "attempt these requests, retry the failures at most these times, using this method of back-off, get back to me with the result" plus whatever other options of configurability for the failure modes.
If you are using ReactJS then react-query does a very good job of handling these concerns for you specifically for syncing up with server side state locally.
If you are using ReactJS then react-query does a very good job of handling these concerns for you specifically for syncing up with server side state locally.
I took one of these out recently and didn’t see much of an improvement. I was already fanning out a lot of requests so the sequential part was barely out of the noise floor. 5% at most.
But I did have to add throttling to that request because of it. It had been previously limited by how fast the follow on request could be run.
Doing more things in parallel doesn’t automatically make you faster. Every new batch of developers seems to have to learn this the hard way. Trying to do everything at once can make you more brittle. And does so, as often as not.
But I did have to add throttling to that request because of it. It had been previously limited by how fast the follow on request could be run.
Doing more things in parallel doesn’t automatically make you faster. Every new batch of developers seems to have to learn this the hard way. Trying to do everything at once can make you more brittle. And does so, as often as not.
I regularly write code in this optimized form and regularly get code review suggestions to change the code to make it sequential "for readability". Yes, it's harder to read, but it cuts runtime in half or more at the most performance critical part of the application. I think we developers too often try to cater to an imaginary lowest common denominator developer. Sometimes it is okay if code takes a little extra effort to read.
I agree - however, I don't think the developers we are imagining are imaginary. They exist and are common, but we shouldn't be catering to them as much as we sometimes do. Maybe that means leaning away from pragmatism and toward purism, but my hope is that software generally moves that way over time.
Edit: Not "moves that way" as in becomes completely purist, just that I hope it lands a little closer to that side of the spectrum in the future.
Edit: Not "moves that way" as in becomes completely purist, just that I hope it lands a little closer to that side of the spectrum in the future.
Coding for aesthetics is purism. Coding for human cognition/DevEx is pragmatism. They looks similar to an outsider, but the hills they choose for dying can be fairly different.
Horses for courses. We have to admit that it takes a team with more skill to build a nuclear submarine than it does to inflate a dinghy, but conversely, no point paying nuclear engineers for your dinghy company. If you are building dinghies, or indeed mass producing dinghies for small clients, sure, pragmatism is not using complex async code. If you are building a nuclear submarine, the team better have a real good idea of how these parallel systems interact, don't deadlock the cooling system, and pragmatism might be "who knows TLA+?".
The smaller the function with the perf tweaks, the less pushback you tend to get in reviews. Near as I can surmise, there’s a threshold below which people feel comfortable refactoring code they don’t like the looks of, but the fact of that comfort discourages them from actually doing it.
You do have to watch out for features being added. Plenty of people don’t understand rewording these three lines results in a 10% response time increase.
You do have to watch out for features being added. Plenty of people don’t understand rewording these three lines results in a 10% response time increase.
That can cut both ways. Sometimes optimizations reduce legibility even for advanced programmers, and often for very marginal improvements in performance...or no improvement in performance at all because they're not on the critical path. Sometimes it's okay to make code slower but more obvious as to what it is doing.
Variable hoisting can make a loop faster and the intentions clearer. As long as you don’t hoist the variable entirely off the page, that is. Declaring at the top of a large scope is confusing. I think people got that from older versions of C as I recall. It’s worse than doing nothing.
Awaiting a promise later on can lead to unhandled promise rejections, which will make Node.js crash in the future (currently prints a warning).
Instead, Promise.all() and .then() should be used:
Instead, Promise.all() and .then() should be used:
let [result1, result2] = await Promise.all([
fetch('/request1').then(doSomeCalculation),
fetch('/request2')
])How can the runtime conclude that the rejection is unhandled when there are outstanding references to the Promise?
In other words, as long as the promise might later be awaited or otherwise handled I don't see why the runtime should care. (That's the mental model I carry from other languages.)
In other words, as long as the promise might later be awaited or otherwise handled I don't see why the runtime should care. (That's the mental model I carry from other languages.)
Normal memory references aren't enough to prove that the promise's rejection is handled. It needs a reference in the form of one or more registered handlers -- which can be achieved by registering them immediately via the Promise.all in the above example.
Correct me if I'm wrong, but I think you can avoid this by doing:
thingToUseLater.catch(() => {})
And just ignoring the result of .catch(). Promises are reusable so thingToUseLater can be waited on later when you want to block for it.
Thanks for thoughts on this. I use promises a lot but this is an edge case I hadn't thought about. They are really pretty handy for build scripts since they make your dependencies explicit.
thingToUseLater.catch(() => {})
And just ignoring the result of .catch(). Promises are reusable so thingToUseLater can be waited on later when you want to block for it.
Thanks for thoughts on this. I use promises a lot but this is an edge case I hadn't thought about. They are really pretty handy for build scripts since they make your dependencies explicit.
Yup. If you call catch, you've handled the promise. You would need to throw from within the callback in order for the promise itself to reject after that.
can you not use try catch blocks for this?
Unless you stop executing code inside the try{} block by using "await" the try/catch is over when the error is thrown asynchronously later on. Leading to an asynchronous promise rejection. Yes your code is located inside a try/catch - but the execution has gone well past it unless you "await" the promise inside the try{}.
That is a serious little trick question/problem for people not deeply familiar with async/await and promises in Javascript, so most newbie and middlish programmers. Either you only use stock-standard code constructs, so you refrain from creating promises and not immediately await-ing them, but passing them around*, or you really need to know and deeply understand promises and async/await.
Relying on async/await but also additionally passing some un-await-ed promises around poses danger and needs a "senior" level understanding. The try/catch tricks people to think any promise created within is caught - but the try/catch exists only at one specific point in time. You have an additional dimension, your lexical code structure shows an incomplete picture.
Whenever you create a promise, if you don't immediately await it, you must make sure that before there is another "await" you must attach a catch handler, or you may get an unhandled promise rejection at some point. Which usually nobody tests for, because on the happy path the program works just fine. Either attach one with .catch(), of if you use try/catch you must "await" the promise inside the try block - it must not resolve before code execution progresses past that block.
That is a serious little trick question/problem for people not deeply familiar with async/await and promises in Javascript, so most newbie and middlish programmers. Either you only use stock-standard code constructs, so you refrain from creating promises and not immediately await-ing them, but passing them around*, or you really need to know and deeply understand promises and async/await.
Relying on async/await but also additionally passing some un-await-ed promises around poses danger and needs a "senior" level understanding. The try/catch tricks people to think any promise created within is caught - but the try/catch exists only at one specific point in time. You have an additional dimension, your lexical code structure shows an incomplete picture.
Whenever you create a promise, if you don't immediately await it, you must make sure that before there is another "await" you must attach a catch handler, or you may get an unhandled promise rejection at some point. Which usually nobody tests for, because on the happy path the program works just fine. Either attach one with .catch(), of if you use try/catch you must "await" the promise inside the try block - it must not resolve before code execution progresses past that block.
Wait, isn’t awaiting inside the try{} exactly how you’re supposed to handle it in the first place? Assuming you’re using await and not .catch(). And if you await without the try/catch you just end up with an unhandled exception at the await, no?
I’ve yet to see any junior devs make the mistake of try-ing an unawaited promise. They’ve all seemed to appreciate the “async” aspect of the semantics up front.
I’ve yet to see any junior devs make the mistake of try-ing an unawaited promise. They’ve all seemed to appreciate the “async” aspect of the semantics up front.
Remember that the context of this discussion is achieving parallelism by not waiting - instead, you create promises in parallel, pass them on, and attach handlers at some later point, e.g. after collecting them in a Promise.all().
As I said, as long as you do standard stuff, which for try/catch in an async function and an "await" of all promises is the standard path.
Sometimes people create a promise without awaiting inside a try/catch, thinking they got errors handled, because they don't want to wait but start another promise-producing function right away. But that other function needs its own catch handler so they write another one below the first one and don't "await" to get both asynchronous promise-returning functions to run simultaneously without the second one having to wait for the first one, because what they do is independent (so far the thoughts are correct and good).
Since they use try/catch instead of the .then() and .catch() handler and they don't fully understand all the implications they run into this problem. It happened to me when I learned promises, after already having programmed with them for well over a year and feeling comfortable, and it just recently happened to some of my people (it no longer happens to me, it's the next generations turn...).
Other issues that make this exact problem insidious are:
- This problem can only be spotted when the promise is rejected. If your tests don't include that you will only see it when someone runs into this issue by chance, much later, maybe in production.
- Often it may not get fixed even after there is a rejected promise showing the problem. What time-limited stressed middle-level developers will do is solve the rejected promise but not the way it isn't caught. The code construct that lead to "uncaught promise rejection" will remain unfixed though, because as soon as the promise no longer rejects the code appears to work fine.
As I said, as long as you do standard stuff, which for try/catch in an async function and an "await" of all promises is the standard path.
Sometimes people create a promise without awaiting inside a try/catch, thinking they got errors handled, because they don't want to wait but start another promise-producing function right away. But that other function needs its own catch handler so they write another one below the first one and don't "await" to get both asynchronous promise-returning functions to run simultaneously without the second one having to wait for the first one, because what they do is independent (so far the thoughts are correct and good).
Since they use try/catch instead of the .then() and .catch() handler and they don't fully understand all the implications they run into this problem. It happened to me when I learned promises, after already having programmed with them for well over a year and feeling comfortable, and it just recently happened to some of my people (it no longer happens to me, it's the next generations turn...).
Other issues that make this exact problem insidious are:
- This problem can only be spotted when the promise is rejected. If your tests don't include that you will only see it when someone runs into this issue by chance, much later, maybe in production.
- Often it may not get fixed even after there is a rejected promise showing the problem. What time-limited stressed middle-level developers will do is solve the rejected promise but not the way it isn't caught. The code construct that lead to "uncaught promise rejection" will remain unfixed though, because as soon as the promise no longer rejects the code appears to work fine.
> Remember that the context of this discussion is achieving parallelism by not waiting - instead, you create promises in parallel, pass them on, and attach handlers at some later point, e.g. after collecting them in a Promise.all().
Aren't "await Promise.all(...)" and "Promise.all(...).then(...)" more or less equivalent? (I'm less familiar with how JavaScript handles it under the hood.) In one case the callback executes once the promise completes, in another case execution of code after the await resumes once the promise completes. The article was about only awaiting once you actually need the data, whether you do it with continuation passing or async/await seems like an unimportant detail.
Aren't "await Promise.all(...)" and "Promise.all(...).then(...)" more or less equivalent? (I'm less familiar with how JavaScript handles it under the hood.) In one case the callback executes once the promise completes, in another case execution of code after the await resumes once the promise completes. The article was about only awaiting once you actually need the data, whether you do it with continuation passing or async/await seems like an unimportant detail.
The discussion is about when you attach handlers, and failure handlers specifically. When someone uses async/await style and try/catch but also adds in handling (passing) actually promises, without await-ing them.
Depends on what you're doing w/ the promise variable. If at some point you start to return promises without awaiting them, then the try/catch might not catch unhandled rejections. This is a source of subtle non-happy-path bugs for code that deals w/ custom deferred objects, e.g. some flavors of promisified ChildProcess'es (promise objects with `stdout`/`stderr` properties).
Naked promises are tricky enough that I've seen experienced developers use hungarian notation to denote that a value is a promise, in order to indicate the presence of non-trivial error propagation semantics in said snippet of code.
Naked promises are tricky enough that I've seen experienced developers use hungarian notation to denote that a value is a promise, in order to indicate the presence of non-trivial error propagation semantics in said snippet of code.
Using a try/catch block won't attach an error handler at the right time (before the next event loop tick):
// promise1 will resolve after one second
let promise1 = new Promise(resolve => setTimeout(resolve, 1000))
// promise2 will wait for the next even loop tick to call its error handler
let promise2 = Promise.reject('something bad')
// At this point, you should await promise2, or set an error handler using .catch
// But you can do some sync stuff if you want to:
someExpensiveCPUStuff()
// But if you do anything async, it'll print a warning in Node.js
let result1 = await promise1
let wrapped = promise
.then(result => ({ type: 'resolved', result }))
.catch(error => ({ type: 'rejected', error }))
await doSomeAsyncStuff()
let result = await wrapped.then(data => {
if(data.type === 'resolved') {
return data.result
} else {
throw data.error
}
})> will wait for the next even loop tick to call its error handler
are you sure about that?
Promise.reject is synchronous and won't wait for the next tick it just returns a rejected promise
try/catch can be used fine
are you sure about that?
Promise.reject is synchronous and won't wait for the next tick it just returns a rejected promise
try/catch can be used fine
async function f() {
throw new Error("asdf");
}
let result1 = f();
let result2 = f();
try {
await result1;
} catch (e) {
console.log("caught the error");
}
try {
await result2;
} catch (e) {
console.log("caught the error");
}The error handler won't be called until the next tick:
Promise
.reject('error')
.catch(error => console.log('Error handler called:', error))
console.log('After reject')
After reject
Error handler called: errorI think the point is that try/catch in async code is tied to the presence of await. This will run happily without falling into the catch clause:
const err = async () => {
throw new Error('async error')
}
try {
const rejected = Promise.reject();
const errored = err();
console.log('ok!'); // in real code, this might have returned a deferred to be handled elsewhere instead
} catch (e) {
console.log('not ok!'); // never gets here
}I don't get a warning for this code:
async function foo() {
try {
// promise1 will resolve after one second
let promise1 = new Promise(resolve => setTimeout(resolve, 1000))
// promise2 will wait for the next even loop tick to call its error handler
let promise2 = Promise.reject('something bad')
// At this point, you should await promise2, or set an error handler using .catch
// But you can do some sync stuff if you want to:
someExpensiveCPUStuff()
// No warning in Node.js
let result2 = await promise2;
let result1 = await promise1;
}
catch (ex) {
console.log(ex)
}
}You should if you await promise1 first:
let result1 = await promise1;
let result2 = await promise2;Yeah, but the goal is to kick off two async calls in parallel. The order in which you await is unimportant for that goal. If awaiting promise2 first solves the problem then that's the solution!
so console.log(ex) is never executed?
you have to execute it like this:
await foo();
Then you will see the error
await foo();
Then you will see the error
Note (for people who might try to mentally generalize this to other languages or situations) that this requires eager coroutines, which not all async/await frameworks support (and is often optional for the ones that do); for libraries that lean into "structured parallelism" you will need to pass these off to something that awaits them in some way (potentially as a group, or maybe to something that adapts the awaitable into an eager promise from a different system) for them to even start executing.
Would love a programming language where optional pre- and post-invariants were a first-class language feature. So each line could be:
<pre-invariants> | <line of code> | <post-invariants>.
You can get something like that in Haskell. Functions are piles of statements that you can generally write in any order, and the compiler will deal with the resolution.
You don't see this effect very strongly in Haskell though because you generally end up with small functions that don't have that many statements anyhow.
(If one is sort of vaguely familiar with Haskell and thinking "what about do notation?" (or, less correctly but to the same point, "what about monads?") the answer is that do notation is a syntax sugar for introducing functions in a slightly different manner, using "<-" and some syntax transformations, so "do notation" is not a function but a function per <- symbol. Do notation affords tons of very small functions.)
"figure out which things can be async automatically"
If you want this, learn Go or Erlang (or Haskell). Personally I find having to manually label and deal with sync vs. "async" functions intolerable, but, fortunately, I don't have to.
At the very least, though, I'd like a type system to help me out. This is something any type system can provide a lot of help with very easily, even sticking with manual async annotation. Getting a compile time error for sticking a promise for an int into an int would help keep a lot of this straight.
You don't see this effect very strongly in Haskell though because you generally end up with small functions that don't have that many statements anyhow.
(If one is sort of vaguely familiar with Haskell and thinking "what about do notation?" (or, less correctly but to the same point, "what about monads?") the answer is that do notation is a syntax sugar for introducing functions in a slightly different manner, using "<-" and some syntax transformations, so "do notation" is not a function but a function per <- symbol. Do notation affords tons of very small functions.)
"figure out which things can be async automatically"
If you want this, learn Go or Erlang (or Haskell). Personally I find having to manually label and deal with sync vs. "async" functions intolerable, but, fortunately, I don't have to.
At the very least, though, I'd like a type system to help me out. This is something any type system can provide a lot of help with very easily, even sticking with manual async annotation. Getting a compile time error for sticking a promise for an int into an int would help keep a lot of this straight.
Does anyone have any good site for ACTUAL state of the art in concurrency that discusses how go, rust, jvm, elixir, etc do things and the advantages?
Node.JS is basically a single CPU using nonblocking I/O, great from the age of two to four core CPUs.
But the future is dozens of CPUs working simultaneously to rapidly process the I/O that comes in from near-terabit networks and super SSDs using main memory bandwidth.