Async Views in Django 3.1(testdriven.io)
testdriven.io
Async Views in Django 3.1
https://testdriven.io/blog/django-async-views/
82 comments
Django agrees with you. The async design for Django is predicated on the idea that the vast majority of cases are better handled by threads - but very occasionally it is useful to run an async view for something that's very IO bound (calling other HTTP APIs most likely).
If you watch Andrew's talk from PyCon AU last week he explicitly calls out that sync views are still better for most cases, which is why he invested so much effort in making sure they were not negatively affected by the addition of async: https://youtu.be/ibAmA4QQDhs
If you watch Andrew's talk from PyCon AU last week he explicitly calls out that sync views are still better for most cases, which is why he invested so much effort in making sure they were not negatively affected by the addition of async: https://youtu.be/ibAmA4QQDhs
But here's the thing. In all my encounters with asyncio, its slower, even for things like HTTP calls
I suspect its because the event loop gets blocked doing trivial things like json/csv parsing, but it could just be python's slowness.
Asyncio seems super hard to profile too..
I suspect its because the event loop gets blocked doing trivial things like json/csv parsing, but it could just be python's slowness.
Asyncio seems super hard to profile too..
This is something that lots of folks seem to be getting confused about, with benchmark posts coming up on HN asking “is Django async faster than X?”.
For the API gateway use case this feature is really useful, and extends the usefulness of Django by letting it serve as a gateway for longer as your app grows and you start extracting services from the monolith.
I think part of it is the idea that in principle calling your DB is IO - but in practice your DB is fast enough that your sync view is not spending much of its time waiting on that IO.
For the API gateway use case this feature is really useful, and extends the usefulness of Django by letting it serve as a gateway for longer as your app grows and you start extracting services from the monolith.
I think part of it is the idea that in principle calling your DB is IO - but in practice your DB is fast enough that your sync view is not spending much of its time waiting on that IO.
This is a weird claim. Database calls where even moderate business logic is done at the database level can have very surprising performance characteristics and can be difficult to optimize. Taking these calls off the main thread isn’t a solution but it’s obviously a significant IO burden that doesn’t need to be shared across all of the happy paths
On my production api about 20% of each request is spent waiting on the DB. That’s with a lot of low hanging fruit remaining for optimization. YMMV. But you don’t get a speed boost by using async to talk to your DB in most common/textbook usecases. Sure, if your DB queries are complex they might take a long time in pathological cases (eg the classic accidental O(N) database query caused by not joining properly in your ORM), but the fix is usually to fix your DB queries, not add async to your API worker.
Note that simonw is a Django core dev, so when they explain what the design goals were for this feature you can be confident they are correct.
I’ll just quote it here for clarity:
> The async design for Django is predicated on the idea that the vast majority of cases are better handled by threads - but very occasionally it is useful to run an async view
This is an empirical question though, and will vary depending on workload; if you are doing something funky with long-running stored procedure calls, then you might get a win with async (make sure your DB driver is async too though, the default ones in Django aren’t last I checked). Again, my claim is just that most use cases are not like that.
Note that simonw is a Django core dev, so when they explain what the design goals were for this feature you can be confident they are correct.
I’ll just quote it here for clarity:
> The async design for Django is predicated on the idea that the vast majority of cases are better handled by threads - but very occasionally it is useful to run an async view
This is an empirical question though, and will vary depending on workload; if you are doing something funky with long-running stored procedure calls, then you might get a win with async (make sure your DB driver is async too though, the default ones in Django aren’t last I checked). Again, my claim is just that most use cases are not like that.
Anything that spawns an unbounded amount of POSIX threads will be bad for performance, so you need an M:N scheduler to execute threads. I feel like this is pretty hard to retrofit onto an existing language (e.g. I don't think you can write a pre-empting scheduler in pure Python as a library either).
AFAIK, only Go does this well (with great results) as a result of it being pretty much baked into the language. I don't think it's an async bandwagon, I think async/await is the best way to get concurrency without requiring the whole language to opt-in (as the Rust devs discovered).
AFAIK, only Go does this well (with great results) as a result of it being pretty much baked into the language. I don't think it's an async bandwagon, I think async/await is the best way to get concurrency without requiring the whole language to opt-in (as the Rust devs discovered).
Elixir does that too (more a BEAM thing to be more precise), the BEAM can spawn millions of processes without killing your server performance.
But I agree with your main point, concurrency is hard if the language isn't built from the ground to be concurrent.
But I agree with your main point, concurrency is hard if the language isn't built from the ground to be concurrent.
I think it's not necessarily POSIX threads related. In theory there is nothing
wrong with spawning tens/hundreds of thousands OS threads, and Linux does a
decent job at this. But in practise you'll run into issues such as:
* Different OS' handle things differently. macOS limits the number of threads per process (somewhere around 1500 I believe). Other OS' may be slower, or impose other limits
* Because of this, sometimes spawning threads is pretty fast (e.g. 10 µsec). Other times it's super slow (I've seen threads take over 500 ms to spawn)
* Context switches are still pretty slow across OS'. There's finally some work being done towards lightweight thread support in Linux, but I suspect it will take a few years to become useful
I _really_ hope that one day we _can_ "just" spawn 100 000 OS threads without issues, as it would make many concurrency problems easier to solve. Sadly, I think it will take another 10-15 years.
* Different OS' handle things differently. macOS limits the number of threads per process (somewhere around 1500 I believe). Other OS' may be slower, or impose other limits
* Because of this, sometimes spawning threads is pretty fast (e.g. 10 µsec). Other times it's super slow (I've seen threads take over 500 ms to spawn)
* Context switches are still pretty slow across OS'. There's finally some work being done towards lightweight thread support in Linux, but I suspect it will take a few years to become useful
I _really_ hope that one day we _can_ "just" spawn 100 000 OS threads without issues, as it would make many concurrency problems easier to solve. Sadly, I think it will take another 10-15 years.
In theory there is nothing wrong with spawning tens/hundreds of thousands OS threads, and Linux does a decent job at this. But in practise you'll run into issues such as:
The other big problem on Linux is stack sizes - "hundreds" or "thousands" of threads isn't a problem - the C10K problem which proved the dominance of nginx's evented model vs. Apache thread-per-conncetion model was broken 20 years ago. Where it matters is when we reach 100s of thousands or millions of connections on a single node. The first issue you run into is the thread stack size - Linux has a hard default of 8MB; at 100,000 you start to hit OOM issues first. Go (and BEAM) has a ton of work to make this use case possible within the runtime.
What I'm unsure about if Linux even cares about solving this problem - the threading model is pretty general and I'm not sure how important high performance connection handling is. At the end of the day, if you are really I/O bound, an async style, co-operative, event queue will always be more performant than a generic threaded scheduler like the one Linux has; meaning if you care about performance (like Django probably does) they would probably go with an async style scheduler even if Linux could support a million threads.
The other big problem on Linux is stack sizes - "hundreds" or "thousands" of threads isn't a problem - the C10K problem which proved the dominance of nginx's evented model vs. Apache thread-per-conncetion model was broken 20 years ago. Where it matters is when we reach 100s of thousands or millions of connections on a single node. The first issue you run into is the thread stack size - Linux has a hard default of 8MB; at 100,000 you start to hit OOM issues first. Go (and BEAM) has a ton of work to make this use case possible within the runtime.
What I'm unsure about if Linux even cares about solving this problem - the threading model is pretty general and I'm not sure how important high performance connection handling is. At the end of the day, if you are really I/O bound, an async style, co-operative, event queue will always be more performant than a generic threaded scheduler like the one Linux has; meaning if you care about performance (like Django probably does) they would probably go with an async style scheduler even if Linux could support a million threads.
> The first issue you run into is the thread stack size - Linux has a hard default of 8MB; at 100,000 you start to hit OOM issues first. Go (and BEAM) has a ton of work to make this use case possible within the runtime.
This is not how thread stack sizes work. Thread stack sizes allocate a certain amount of _virtual_ memory. This means that with a stack size of 8MB you don't actually immediately use 8MB of physical memory. Only when you start filling up that memory do you start using physical memory. We can easily demonstrate this using a simple Rust program:
The virtual memory limits in turn are not much of an issue. I think on most 64 bits systems the virtual memory limit is 256 TB. A 256 TB limit would allow up to 33 554 432 OS threads with a stack size of 8MB. And since you can configure the limit when spawning threads, you can increase this number by spawning threads with a smaller limit (e.g. 4MB).
This is not how thread stack sizes work. Thread stack sizes allocate a certain amount of _virtual_ memory. This means that with a stack size of 8MB you don't actually immediately use 8MB of physical memory. Only when you start filling up that memory do you start using physical memory. We can easily demonstrate this using a simple Rust program:
use std::thread;
use std::time::Duration;
fn main() {
let mut handles = Vec::with_capacity(10_000);
for _ in 0..handles.capacity() {
let handle = thread::spawn(|| thread::sleep(Duration::from_secs(1)));
handles.push(handle);
}
for handle in handles {
handle.join().unwrap();
}
}
/usr/bin/time -f '%M KB RSS' program-name
88492 KB RSS
The virtual memory limits in turn are not much of an issue. I think on most 64 bits systems the virtual memory limit is 256 TB. A 256 TB limit would allow up to 33 554 432 OS threads with a stack size of 8MB. And since you can configure the limit when spawning threads, you can increase this number by spawning threads with a smaller limit (e.g. 4MB).
Not as bad as you think, for an application server anyway. HN actually uses a thread per request according to the docs: https://arclanguage.github.io/ref/srv.html
They also read from disk for most requests. So, thread per request and also reading from disk.
I worked at one company where we had many dozens of services with thrift/spring boot and spawned a request per thread and it worked fairly well. We were collecting and aggregating every review on the internet (for our customers), and had dozens of services... There were a couple in a reactive style. So we took the maintenance cost when we needed to.
They also read from disk for most requests. So, thread per request and also reading from disk.
I worked at one company where we had many dozens of services with thrift/spring boot and spawned a request per thread and it worked fairly well. We were collecting and aggregating every review on the internet (for our customers), and had dozens of services... There were a couple in a reactive style. So we took the maintenance cost when we needed to.
I get where you're coming from, but I disagree.
Async code has its traps, but they are very managable once you've learned them. I haven't found them on the level of manual memory management which is a footgun that even experienced devs keep firing.
The benefits are also there, doing slow things in parallel like HTTP calls or database queries is trivial with async code and trickier in sync code.
Of course you prefer sync code when you don't need it, but async/await style code doesn't prevent sync code. If you only support sync doing anything async because very cumbersome and dangerous, and doing so requires things like managing threads or locks yourself which is an even greater recipe for bugs.
Async code has its traps, but they are very managable once you've learned them. I haven't found them on the level of manual memory management which is a footgun that even experienced devs keep firing.
The benefits are also there, doing slow things in parallel like HTTP calls or database queries is trivial with async code and trickier in sync code.
Of course you prefer sync code when you don't need it, but async/await style code doesn't prevent sync code. If you only support sync doing anything async because very cumbersome and dangerous, and doing so requires things like managing threads or locks yourself which is an even greater recipe for bugs.
You can of course do parallel HTTP calls or database queries with Python's existing threads (IO releases the GIL in Python) or via multiprocessing.
You don't have to use locks either - straight parallelism can be done very easily with the Executors in concurrent.futures. There are a lot of good concurrency primitives in the threading library in the stdlib including conditions, semaphores and barriers. Of course best to avoid parallelism with shared data if at all possible (and IMO it doesn't really come up much).
You don't have to use locks either - straight parallelism can be done very easily with the Executors in concurrent.futures. There are a lot of good concurrency primitives in the threading library in the stdlib including conditions, semaphores and barriers. Of course best to avoid parallelism with shared data if at all possible (and IMO it doesn't really come up much).
The thing is that you can serve a lot of requests on a single core.. (and at least us we pay per core)
You can use multiple threads and multiple processes on a single core - that is the purpose of timesharing operating systems after all.
>A better model already exists - thread per connection
For that you will need a language with good support for threads, which Python definitely isn't (at least not CPython), so Django doesn't have any other option here.
Native threads are huge improvement over async, but still not as good as green threads, which really are the best option here for 99% of usecases, abstracting whole async thing away.
> Thread per connection has gotten a bad name for performance which is pretty false.
With 64 or more cores being the norm, a thread with 2MB stack per connection requires 128MB just to exist (though I believe OS may offer some magick if you are not actually using it). This is probably 3-4x time what's required for green threads.
For that you will need a language with good support for threads, which Python definitely isn't (at least not CPython), so Django doesn't have any other option here.
Native threads are huge improvement over async, but still not as good as green threads, which really are the best option here for 99% of usecases, abstracting whole async thing away.
> Thread per connection has gotten a bad name for performance which is pretty false.
With 64 or more cores being the norm, a thread with 2MB stack per connection requires 128MB just to exist (though I believe OS may offer some magick if you are not actually using it). This is probably 3-4x time what's required for green threads.
>>A better model already exists - thread per connection
> For that you will need a language with good support for threads, which Python definitely isn't (at least not CPython), so Django doesn't have any other option here
The thread per connection is managed by your application server. Why does the language need to have good thread primitives then? In fact I would argue you spawning and managing your own threads make your service less robust.
> For that you will need a language with good support for threads, which Python definitely isn't (at least not CPython), so Django doesn't have any other option here
The thread per connection is managed by your application server. Why does the language need to have good thread primitives then? In fact I would argue you spawning and managing your own threads make your service less robust.
> Why does the language need to have good thread primitives then?
Because: a) we want application server to be written in the language we are using, so that we can control every aspect of how it works, instead of having some silo that developer will avoid because its so different b) those threads will share cache, connections, some preloaded data you may need and so on, you want a language that allows this to be done reliablyl c) those threads may need to create more threads (make parallel requests to 3 services and resize an image at the same time)
Because: a) we want application server to be written in the language we are using, so that we can control every aspect of how it works, instead of having some silo that developer will avoid because its so different b) those threads will share cache, connections, some preloaded data you may need and so on, you want a language that allows this to be done reliablyl c) those threads may need to create more threads (make parallel requests to 3 services and resize an image at the same time)
As if Apache/nginx and wsgi isn’t still the norm...
Uwagi is used for performance reasons only, once you have a language where its not a problem, it immediately goes away. I haven't seen anyone using it with Go for example.
With 64 or more cores being the norm, a thread with 2MB stack per connection requires 128MB just to exist
Sounds good to me!
Can you even find a VM offering that has a 64 core CPU and less than 32GB of RAM?
Sounds good to me!
Can you even find a VM offering that has a 64 core CPU and less than 32GB of RAM?
I tried to search for it and and... well, google is so ad-infested that's its useless, but doesn't seem so. Anything with large core count comes with 192GB or more.
> With 64 or more cores being the norm, a thread with 2MB stack per connection requires 128MB just to exist (though I believe OS may offer some magick if you are not actually using it). This is probably 3-4x time what's required for green threads.
Are you assuming that each thread requires a dedicated stack on each core? That isn't how it works - each thread has a single stack regardless of the number of CPU cores. So, a 2MB stack requires just 2MB. However, that 2MB is 2MB of virtual memory. So, assuming that the stack isn't allocated using a 2MB huge page, it will probably actually be physically allocated in 4K chunks as required. So, most threads are going to use considerably less than 2MB of actual memory. Outside of 32bit systems, virtual memory is close to an unlimited resource.
Green threads also require stacks. How green threads allocate stack space is going to depend on the particular runtime, but it's likely that like with real threads they will also lazily allocate memory as required. A big difference between stack usage with green threads and regular threads is that the runtime _may_ support shrinking the stack used by a green thread if space is only briefly required by a green thread, while a regular thread will hold on to any memory that is physically allocated for its stack until it exits. Reclaiming memory like this isn't free, however - and you run the risk of "stack thrashing" where the point where a stack grows and shrinks is accidentally in the middle of a hot loop and you end up spending basically all your time growing and shrinking the stack as opposed to doing real work.
I _think_ that some green thread runtimes may also support stacks smaller than 4K - but I'm not really sure if / how many actually do.
Are you assuming that each thread requires a dedicated stack on each core? That isn't how it works - each thread has a single stack regardless of the number of CPU cores. So, a 2MB stack requires just 2MB. However, that 2MB is 2MB of virtual memory. So, assuming that the stack isn't allocated using a 2MB huge page, it will probably actually be physically allocated in 4K chunks as required. So, most threads are going to use considerably less than 2MB of actual memory. Outside of 32bit systems, virtual memory is close to an unlimited resource.
Green threads also require stacks. How green threads allocate stack space is going to depend on the particular runtime, but it's likely that like with real threads they will also lazily allocate memory as required. A big difference between stack usage with green threads and regular threads is that the runtime _may_ support shrinking the stack used by a green thread if space is only briefly required by a green thread, while a regular thread will hold on to any memory that is physically allocated for its stack until it exits. Reclaiming memory like this isn't free, however - and you run the risk of "stack thrashing" where the point where a stack grows and shrinks is accidentally in the middle of a hot loop and you end up spending basically all your time growing and shrinking the stack as opposed to doing real work.
I _think_ that some green thread runtimes may also support stacks smaller than 4K - but I'm not really sure if / how many actually do.
> Are you assuming that each thread requires a dedicated stack on each core?
No, I am assuming we run 1 thread per core.
No, I am assuming we run 1 thread per core.
> A better model already exists - thread per connection - for the majority of what we want as web developers.
Haven't we already seen in Apache vs Nginx that thread per connection will OOM the web servers under heavy traffic? (haven't used Apache in many years so this may be obsolete)
Haven't we already seen in Apache vs Nginx that thread per connection will OOM the web servers under heavy traffic? (haven't used Apache in many years so this may be obsolete)
Apache moved from using a thread per connection to fibers years ago [1]. Fibers are simpler than green threads (cooperative multitasking via yielding) which should be illustrative of how good an idea the "thread per connection" model is.
Although it could be argued that fibers are a type of primitive thread--so as long you don't mean OS thread the idea has merit.
[1] http://httpd.apache.org/dev/whiteboard/process-model.html
Although it could be argued that fibers are a type of primitive thread--so as long you don't mean OS thread the idea has merit.
[1] http://httpd.apache.org/dev/whiteboard/process-model.html
As a C#/.NET developer I couldn't agree more - more often than not most of us simply do not need async, but everyone's doing it so it's become a 'best practice.'
As someone who has been in the industry for 20 years everything seems to be getting worse. Like Kubernetes and SPA apps. People rarely need these (OK, high availability via Kubernetes is nice, but few people need elastic scaling).
That was my experience with a spikey server that I had behind an ELB. It was better to just eat it and run a medium server all the time behind a CDN than to try to scale up and down from small to large when a traffic spike came. The scaling isn’t worth the effort.
We apparently saved some server costs by moving to kubernetes, but it cost at least 6 months of dev time. And we have had problems with large uploads and downloads, because the smaller instances can't handle them as easily. I am not convinced it was a wise choice, but it was made before I started.
The new async Razor engine in .NET Core is ridiculous. They threw out view helper functions and support for functional/monadic control flow for what, so you can asynchronously render a string? Insane.
I suspect the actual motivation is that async is virtually required to address performance problems associated with the new tag helpers - which are themselves a terrible, terrible idea. Slow, brittle, surprising, wholly reliant on syntax highlighting, and then redundant because you can implement the same functionality using the existing C# syntax.
I suspect the actual motivation is that async is virtually required to address performance problems associated with the new tag helpers - which are themselves a terrible, terrible idea. Slow, brittle, surprising, wholly reliant on syntax highlighting, and then redundant because you can implement the same functionality using the existing C# syntax.
well the problem is that if you use it once it's probably easier to use it everywhere.
most people understand it if I show them a task explorer or ps and call Thread.Sleep and Task.Delay.
I've never had an Issue with C# async and before I was a big scala user and did not have probems at all.
>I still managed to be bitten by the very common issue of error logging getting lost in a hidden callback that doesn't look like a callback because of the magic of the async keyword (a common issue in node as well).
What do you mean, any example? I have code full of asyncs and never had problem with logging
What do you mean, any example? I have code full of asyncs and never had problem with logging
The biggest culprit of this kind of thing is usually people doing something with Task.Run thinking it’s a free way to queue a background task.
async views aren't there because django is moving in that direction, they are there to make it easy to use async functions that already exist. I write a lot of functions that block on IO and a lot of them benefit from async event loops. Now I have a simple way of integrating that functionality into django views.
Async has a place, but as you say there may be a better option for the majority of developers. The problem is that many developers are not identifying their performance issues and instead just throwing solutions out there. Perf discussions with plain Python often devolve into a GIL discussion before anyone confirms if the problem is CPU-bound.
I see many issues from developers where I have to walk them back to the start and spend time on identifying the problem. There are many reasons for this, but it's not necessarily the most fun or exciting thing for developers to do. There are still many artificial impediments because devs don't universally have the control over, or access to prod environments that startups may have. If this troubleshooting is a hurdle then sprinkling in some async starts to seem much simpler.
I see many issues from developers where I have to walk them back to the start and spend time on identifying the problem. There are many reasons for this, but it's not necessarily the most fun or exciting thing for developers to do. There are still many artificial impediments because devs don't universally have the control over, or access to prod environments that startups may have. If this troubleshooting is a hurdle then sprinkling in some async starts to seem much simpler.
> I see many issues from developers where I have to walk them back to the start and spend time on identifying the problem.
Honestly, in a lot of cases when I have thrown away something and used something else because I just need to show that I'm doing something to develop a reasonably performant application rather than spending all my time optimizing performance. Spending a whole week optimizing your application's performance is surprisingly rare in the modern hustle-driven culture of "lean" startups and such -- cargo-culted by a lot of companies and bosses who have nothing to do with startups.
I recently realized that the entire gunicorn / uwsgi world for example is filled with anecdotes about performance instead of simple, comprehensible benchmarks; and I thought they were one of the better ones. I suspect this kind of lack of performance focus by most of the users is a big part of why.
Honestly, in a lot of cases when I have thrown away something and used something else because I just need to show that I'm doing something to develop a reasonably performant application rather than spending all my time optimizing performance. Spending a whole week optimizing your application's performance is surprisingly rare in the modern hustle-driven culture of "lean" startups and such -- cargo-culted by a lot of companies and bosses who have nothing to do with startups.
I recently realized that the entire gunicorn / uwsgi world for example is filled with anecdotes about performance instead of simple, comprehensible benchmarks; and I thought they were one of the better ones. I suspect this kind of lack of performance focus by most of the users is a big part of why.
>I recently realized that the entire gunicorn / uwsgi world for example is filled with anecdotes about performance instead of simple, comprehensible benchmarks
Its hard to create a simple benchmark for something that has 100+ configuration options and can be run in dozens of fundamentally different ways.
C beats Python though by a large margin.
Its hard to create a simple benchmark for something that has 100+ configuration options and can be run in dozens of fundamentally different ways.
C beats Python though by a large margin.
When you say "threads per connection" do you mean "process per connection"? There isn't a lot shared between the connection in the way that I think of threads.
Anyway, I agree that async seems to be more trouble than it's worth for most cases.
Anyway, I agree that async seems to be more trouble than it's worth for most cases.
This. I really agree with this.
The async afficianados also bring along the async ecosystem split.
I wish they did a better job of bridging this gap.
Languages like rust, which already have amazing support for threads don't need this split, please.
The async afficianados also bring along the async ecosystem split.
I wish they did a better job of bridging this gap.
Languages like rust, which already have amazing support for threads don't need this split, please.
OS thread per connection? Nah. BEAM process per connection - hell yeah!
For anyone looking for an alternative python async web framework, I highly recommend Starlette (https://github.com/encode/starlette).
It was created by the same guy who wrote the excellent & highly popular Django-Rest-Framework. Very lightweight, clean, well-documented, and a breeze to get running with. It also supports async tasks natively.
I spent years working with Django and now I'm currently using it to power the api for https://sqwok.im.
It was created by the same guy who wrote the excellent & highly popular Django-Rest-Framework. Very lightweight, clean, well-documented, and a breeze to get running with. It also supports async tasks natively.
I spent years working with Django and now I'm currently using it to power the api for https://sqwok.im.
We are also using starlette and works quite well.. in combination with asyncpg..
very nice! cool animations on tmpo.io! Right now I'm running things behind aws lambda & using dynamodb etc, have looked into using aioboto3 but haven't got there yet...
Do you use any db connection wrappers around asyncpg? Like encode/databases or gino?
Also important to note that SQLAlchemy 1.4 alpha with asyncpg support is realease and can be tried out(both SQLAlchemy core&orm modes support it).
I myself evaluated all options of async postgress connection wrappers in fastapi project but reverted to... just using sync psycopg2 through sqla core. No very specific gains of async db connection for my use cases and sqlalchemy creator's comments on immaturity of async postgres drivers made my stick to what's battletested.
On one project we use Databases, with a custom opensourced "adapter" for sqlalchemy models: https://github.com/vinissimus/asyncom
Our cms, is based on Guillotina (https://github.com/plone/guillotina) that uses a Datastore more or less equivalent to the Zodb, and plone, but internally uses postgresql and asyncpg. We are active contributors on it, and since latest version it works around asgi (uvicorn). We also have some plans to make it a bit more psqlidiomatic (get rid of pickles, ....)
On other internal services, we use the raw asyncpg.pool, mostly because part of our bussiness logic it's inside prostgresql, and this doesn't feet super well with an ORM. Think on a 15 year old sql schema, that had evolved through different hands, and right now it's multitenant xD
Latest opensourced thing, it's a work queue based on plgsql, with asyncpg bindings. (https://github.com/vinissimus/jobs/).
So, we can say, we use asynpg in all possible forms.
Our cms, is based on Guillotina (https://github.com/plone/guillotina) that uses a Datastore more or less equivalent to the Zodb, and plone, but internally uses postgresql and asyncpg. We are active contributors on it, and since latest version it works around asgi (uvicorn). We also have some plans to make it a bit more psqlidiomatic (get rid of pickles, ....)
On other internal services, we use the raw asyncpg.pool, mostly because part of our bussiness logic it's inside prostgresql, and this doesn't feet super well with an ORM. Think on a 15 year old sql schema, that had evolved through different hands, and right now it's multitenant xD
Latest opensourced thing, it's a work queue based on plgsql, with asyncpg bindings. (https://github.com/vinissimus/jobs/).
So, we can say, we use asynpg in all possible forms.
We use https://github.com/samuelcolvin/buildpg, which is built by the same dude that built pydantic, which is also used in FastAPI.
It's neat, it's a query builder, not an ORM or any nonsense, and it makes queries a lot more manageable (bringing SQL logic into python primitives).
It's neat, it's a query builder, not an ORM or any nonsense, and it makes queries a lot more manageable (bringing SQL logic into python primitives).
Nice one, but.. I prefer sqls that you can debug without rendering.. (it's not so manageable when you deal with ctes, window functions, joins different schemas, custom functions...
Have you tried fastapi? It uses starlette as its base. Such a lovely experience working with it.
I haven't tried it but I did look it over when I was investigating Starlette. May try it out in the future, it seems powerful, and I read about a few projects using it. I ended up just rolling with Starlette because I liked how minimal, yet flexible it was (why FastApi etc built on it).
Some reasons I see for sticking with Django is that it is big and well known. So easier to hire devs that know it, plenty of knowledge on Stack Overflow and elsewhere. Plus a large ecosystem of packages. (We use Tornado at my current workplace, and we would be far better off with something like Django).
Definitely, agree, Django is a great framework and I've used it over the past 5+ years at many companies large and small. When I set off to build my own project I wanted to keep my code as lean as possible and that's one thing I like about Starlette. It gives you the basic blocks and lets you decide how to use it.
I've done asynchronous programming with the reactive paradigm where I specify a chain of function calls I want applied to data and it happens in a thread independent way. I have not done much programming with the new fancy "async/await" keywords. It seems like a worse abstraction because it mixes asynchronous code with synchronous code and is probably harder to reason about. Not to mention you can't do something like application wide backpressure unless everything is asynchronous.
Can anyone who has worked in both types of applications comment on their differences?
Can anyone who has worked in both types of applications comment on their differences?
I would say that there are maybe three different ways of async rather than two. Broadly:
- wholly implicitly (by monkeypatching stuff with gevent typically)
- async/await
- by reactor (twisted)
async/await is not really an alternative to the explicit style of twisted. To paraphase Guy Steele it is an attempt to drag the monkeypatchers halfway to twisted. It doesn't really solve the problem of mixing sync code in with your async code, the only preventative to that is programmer diligence. If you read from a file using open in the middle of your async code you're going to block the process while bytes are gathered for you. However it does prevent you from accidentally using async code in the middle of your sync code.
Problems like backpressure and failure handling are largely unsolved in asyncio.
- wholly implicitly (by monkeypatching stuff with gevent typically)
- async/await
- by reactor (twisted)
async/await is not really an alternative to the explicit style of twisted. To paraphase Guy Steele it is an attempt to drag the monkeypatchers halfway to twisted. It doesn't really solve the problem of mixing sync code in with your async code, the only preventative to that is programmer diligence. If you read from a file using open in the middle of your async code you're going to block the process while bytes are gathered for you. However it does prevent you from accidentally using async code in the middle of your sync code.
Problems like backpressure and failure handling are largely unsolved in asyncio.
I've done both, but 80% in reactive streams.
I like async/await before it has a lot less hard stuff to think about - hot and cold streams take months if not years for devs to grok - and creates code a lot more similar to non-async code. In contrast Reactive Streams requires you to change your complete program.
I like to compare it to Typescript/Kotlin-style null-safety v.s. the Java Optional or Scala Option. If you want make code null safe with option you have to rewrite basically your complete code from "procedural" to chains of (flat)maps. In contrast in Typescript you just change T to T? in a few places and add an if where you want to handle the optionality.
async/await versus reactive streams has this same problem, reactive streams is highly impactful, much more than async/await. In practice I've seen this have the effect of turning applications turn into a small reactive stream outer shell with the app itself being mostly synchronouse/Future, while async/await is used a lot more liberally because the impact is smaller.
> It seems like a worse abstraction because it mixes asynchronous code with synchronous code
It doesn't, at least in Kotlin only an async function can call an async function. This isn't different from using `map` in a stream.
I like to compare it to Typescript/Kotlin-style null-safety v.s. the Java Optional or Scala Option. If you want make code null safe with option you have to rewrite basically your complete code from "procedural" to chains of (flat)maps. In contrast in Typescript you just change T to T? in a few places and add an if where you want to handle the optionality.
async/await versus reactive streams has this same problem, reactive streams is highly impactful, much more than async/await. In practice I've seen this have the effect of turning applications turn into a small reactive stream outer shell with the app itself being mostly synchronouse/Future, while async/await is used a lot more liberally because the impact is smaller.
> It seems like a worse abstraction because it mixes asynchronous code with synchronous code
It doesn't, at least in Kotlin only an async function can call an async function. This isn't different from using `map` in a stream.
As someone who writes + maintains systems with both: They both have their place. Reactive streams (FRP, Rx, ...) are good when you want to do operations over streams like 'fold', 'sample', 'tween', etc., but most code isn't like that. Instead, most code is 'at-most-once' (=> value or exn), where async/await is enough. That matters because adding streams means all of a sudden your code looks like 'one-or-more', which adds an entirely new world of behavior to define & test against. ("Is this a value or a stream? If I feed in a stream... What if I just want one...")
'Behaviors' in FRP help a bit here... but if you can avoid the complexity, why not?
For when we do FRP, I like how we got it in our React code: Mostly synchronous functional code (React::render(), ...), and then we do animations and other stream-like concepts via Rx. Our nodejs code works similarly: mostly async/await route handlers, and now and then we do fancier http streams and whatnot via Rx. (Though even there async/await has been increasingly nice, as we generally aren't doing many fancy temporal operators nor contention handling.) We haven't figured out an equivalent solution for our Python async code, however as async/await is still new & changing, and afaict FRP/Rx never really made it there.
'Behaviors' in FRP help a bit here... but if you can avoid the complexity, why not?
For when we do FRP, I like how we got it in our React code: Mostly synchronous functional code (React::render(), ...), and then we do animations and other stream-like concepts via Rx. Our nodejs code works similarly: mostly async/await route handlers, and now and then we do fancier http streams and whatnot via Rx. (Though even there async/await has been increasingly nice, as we generally aren't doing many fancy temporal operators nor contention handling.) We haven't figured out an equivalent solution for our Python async code, however as async/await is still new & changing, and afaict FRP/Rx never really made it there.
> Writing asynchronous code gives you the ability to speed up your application [...]
Citation needed... It will be slower by definition but it probably advertised higher throughput anyway. I wonder how many people the throughput question is true for. You’d have to be pretty hardware constrained.
They even suggest asyncio could be a replacement for celery... It really is the Wild West out here.
Citation needed... It will be slower by definition but it probably advertised higher throughput anyway. I wonder how many people the throughput question is true for. You’d have to be pretty hardware constrained.
They even suggest asyncio could be a replacement for celery... It really is the Wild West out here.
Yeah the post conflates asynchronous and concurrent like people always do. Asynchronous code can be executed sequentially, people. (Synchronous code can be executed concurrently, too.)
Asynchronous code is absolutely not just for running tasks after returning an HTTP response. A good example of the true power of asynchronous execution would be to do a Web Socket handler _inside a Django view._
Asynchronous code is absolutely not just for running tasks after returning an HTTP response. A good example of the true power of asynchronous execution would be to do a Web Socket handler _inside a Django view._
> They even suggest asyncio could be a replacement for celery
Only if you don't care about reliability.
Only if you don't care about reliability.
> They even suggest asyncio could be a replacement for celery... It really is the Wild West out here.
Yeah this is pretty questionable.
Having state in your app's process and then having to pretty much re-invent queues, job cancellation, uniqueness guarantees, scheduled jobs, periodic jobs on an interval, queue draining and retries with exponential backoffs.
I mean, yeah you could code all of that with enough time but why would you when Celery exists. If anything having the queue's state held in Redis or anywhere outside of your app's process is enough of a reason to pretty much never replace Celery with asyncio. Even in the post's example of sending an email I would still want a lot of what Celery has to offer and wouldn't consider replacing it.
IMO Celery isn't going anywhere soon and I would still use it in every Python project that needs background processing.
Yeah this is pretty questionable.
Having state in your app's process and then having to pretty much re-invent queues, job cancellation, uniqueness guarantees, scheduled jobs, periodic jobs on an interval, queue draining and retries with exponential backoffs.
I mean, yeah you could code all of that with enough time but why would you when Celery exists. If anything having the queue's state held in Redis or anywhere outside of your app's process is enough of a reason to pretty much never replace Celery with asyncio. Even in the post's example of sending an email I would still want a lot of what Celery has to offer and wouldn't consider replacing it.
IMO Celery isn't going anywhere soon and I would still use it in every Python project that needs background processing.
Just based on my C# experience, this sounds like a terrible idea (abandoning a real task queue for Async/await). 90% of the “gotchas” with async/await in C# comes from people thinking that the task scheduler is bonafide background queue.
Same here. We do lot of processing in background jobs, which make no sense moving to async task.
Celery doesn’t guarantee uniqueness. :-/
> They even suggest asyncio could be a replacement for celery... It really is the Wild West out here.
Yes that is complete madness. The last thing I would ever want to do is send an email or use an external API (those are the examples in TFA) in an async thread as part of a request-response cycle. How does error handling or retries work for that background stuff? I don't know the exact ins-and-outs of Djangos new async stuff yet but my guess is that if that mail/API server doesn't respond to your IO then your response to your own HTTP request hangs?
Above all the central and undemonstrated claim is that there are any performance or management advantages to async. I tested sync vs async on this a couple of months ago. The best performance and robustness is with garden variety sync python with a good WSGI server:
http://calpaterson.com/async-python-is-not-faster.html
If anyone has any contradictory benchmarks please let me know. I haven't been made aware of any yet.
Yes that is complete madness. The last thing I would ever want to do is send an email or use an external API (those are the examples in TFA) in an async thread as part of a request-response cycle. How does error handling or retries work for that background stuff? I don't know the exact ins-and-outs of Djangos new async stuff yet but my guess is that if that mail/API server doesn't respond to your IO then your response to your own HTTP request hangs?
Above all the central and undemonstrated claim is that there are any performance or management advantages to async. I tested sync vs async on this a couple of months ago. The best performance and robustness is with garden variety sync python with a good WSGI server:
http://calpaterson.com/async-python-is-not-faster.html
If anyone has any contradictory benchmarks please let me know. I haven't been made aware of any yet.
Techempower benchmark: https://www.techempower.com/benchmarks/#section=data-r19&hw=... makes the difference between async python frameworks and sync frameworks pretty clear.
Async results in better utilization of system resources... since there aren't entire threads with app state blocked on IO requests (which would consume a chunk of system memory.)
Considering that a web app is most commonly waiting for a database operation, a single server can serve many more active requests simultaneously (since each active request is not allocating an app instance.)
Async results in better utilization of system resources... since there aren't entire threads with app state blocked on IO requests (which would consume a chunk of system memory.)
Considering that a web app is most commonly waiting for a database operation, a single server can serve many more active requests simultaneously (since each active request is not allocating an app instance.)
The techempower benchmark predates me and original drafts of my blogpost mention it but I suppose that got cut for various reasons.
The trouble with that benchmark is that (and they're fairly open about this) they aren't keeping things the same between frameworks - there are multiple variables. If you look at the Flask code all the queries are being put through SQLAlchemy but for various async frameworks they're using the database driver directly and using native code libraries for JSON decoding.
I think techempower's benchmarks are interesting but they don't help you do a comparision of async vs sync.
Additionally on the particular benchmark you've linked to #1 is a dead experimental framework that is not in wide use and #2 is Falcon, a sync framework that is in wide use.
The trouble with that benchmark is that (and they're fairly open about this) they aren't keeping things the same between frameworks - there are multiple variables. If you look at the Flask code all the queries are being put through SQLAlchemy but for various async frameworks they're using the database driver directly and using native code libraries for JSON decoding.
I think techempower's benchmarks are interesting but they don't help you do a comparision of async vs sync.
Additionally on the particular benchmark you've linked to #1 is a dead experimental framework that is not in wide use and #2 is Falcon, a sync framework that is in wide use.
[deleted]
I used Celery (as a workaround) for a Django view that should have been in asyncio before asynchio was available.
I'm very familiar with symfony, but not so much with django or other "persistent in memory" web frameworks. Can someone ELI5 what the advantage is to async django? Does it mean the server doesn't block on one request before handling the next?
Yes. Typically to serve several requests at the same time, you'd run multiple processes, each handling an at a time, and if you spend a lot of time waiting for io (database, some service call or slow client), you'll need to use a lot of workers (each being independent python+django instance, so its a lot of memory -> significant investment in hardware). Async is nothing new in Python world, but previously you'd have to resort to pure python to use it, now you have whole django ecosystem that will use it. So main benefit is that you can handle more requests on the same django instance, and pay less for hardware.
I really don't get this anti-async mentality on HN. Maybe I'm doing something wrong but async is a complete game changer for me as there just so much IO wait in every project I touch.
The fact that my program has to stop and wait for something is honestly unacceptable and it's just a logical conclusion to eliminate this blockage.
I worked past 2 years with asyncio and I have to say it still kinda sucks. The syntax is overly verbose and asyncio itself can't decide what it wants to be. However it still works _good_ for the most part and there's an enormous community and effort in this!
I worked past 2 years with asyncio and I have to say it still kinda sucks. The syntax is overly verbose and asyncio itself can't decide what it wants to be. However it still works _good_ for the most part and there's an enormous community and effort in this!
“So much io wait”. What io is your app waiting for? Is it a webserver, or another type of app?
Not the GP, but I'm currently working on a small web app that synchronizes data from 2 other web services, which means that almost every endpoint involves a call to an external API. In some cases, it involves repeated calls to those APIs, but the order in which the responses are received is unimportant, which is an ideal use case for asyncio's gather function. Using a framework (I haven't tried Django's new async views yet, but am using one of the handful of Flask-ish async frameworks) that supports async views has made this a snap, whereas before I would probably have used some sort of out-of-process work queue type setup.
somehow the idea reminds me of continuations based responses (HN used to be wired that way, also Queinnec wrote lisp papers on the topic)
is there a list of real use cases where the async is better than the sync.
it seems that it's ok in very edge casees where there's high load and a lot of I/O (example, calling external APIs, calling a db).
however, is it stable enough and roboust to replace sync methods in production for the benefit of these cases?
however, is it stable enough and roboust to replace sync methods in production for the benefit of these cases?
With async Django views we can use Django with asyncpg... o_o but middlewares are still synch...
Yeah, it will be some more years before you can simply write async stuff in Django and not worry about every single library that you use, whether it supports async or not.
That's why some years ago I gave up on Django altogether and moved to Nodejs. Much better experience writing async code.
That's why some years ago I gave up on Django altogether and moved to Nodejs. Much better experience writing async code.
Our front service it's a nextjs/nodejs project. But I still prefer python :) and our API's are python based, typed async python.. but python..
C# probably has some of the best async primitives of any language. I still managed to be bitten by the very common issue of error logging getting lost in a hidden callback that doesn't look like a callback because of the magic of the async keyword (a common issue in node as well).
I'm convinced that the async style of web servers is largely a step in the wrong direction. A leaky abstraction that we don't need.
A better model already exists - thread per connection - for the majority of what we want as web developers. There are some good things about allowing us to manage our own waits but those things are rare.
It reminds me of the choice of when to manage your own memory - usually we don't need to, but sometimes we do. Most languages programmers use today decided, rightly, that memory management was an exception and not the norm. We are more productive today for the reduction in cognitive load for most applications as we decide not to manage our own memory.
Thread per connection has gotten a bad name for performance which is pretty false. You can create the exact same underlying abstraction with green threads for more concurrency when you absolutely need it (but mostly you don't). The code we get from not using async is reduced in complexity substantially.
I'm disappointed that django is on the async bandwagon. It means that I may have to make another framework choice in the future.