Good question - the question of whether to log the millisecond when the request starts or ends is a great example of how complex these things are to think about accurately, let alone capture.
I'd want to log when the requests start, as I'm mostly concerned with how well-distributed request arrival was at that level of granularity.
I wondered if the network layers in between my client and server were effectively "smoothing" request arrival across each second, or if instead requests were very bursty so that a per-minute spike in a typical graph was dominated by a few seconds or milliseconds within that minute.
Agree that CQRS seems like a useful way to partitions writes from reads, aka slower requests from faster requests, to avoid many fast requests waiting behind a few slow ones in line.
But even if you shifted reads to one or more caches or read replicas, wouldn't those also have queues that will fill up when you are under-provisioned?
Note that I'm using the term "queue" pretty loosely, to include things like Redis' maxclients or tcpbacklog, or client-side queues when all connections are in use.
Author here. That was a great article, thanks for sharing. Especially the part about how your probability of experiencing a p99 latency is much higher than you'd intuit.
I don't agree with all of it, but definitely a few points made directly or indirectly hit home, such as:
- there is no single metric that can accurately represent "latency"
- most of our metrics are misleading in what they unconsciously include or exclude
I can remember once looking at a graph of requests/second and wishing I could see a distribution of requests per millisecond within an individual second. That level of detail is hard to come by, so in the meanwhile, we do what we can with the data we have.
I had a section in the post I cut out about how optimizing queue selection started out as a technical problem, but transformed into more of a business and ethical problem the more I pondered it.
You're effectively deciding how to distribute suffering across a large group of people.
Comes up in any situation where large metric gains can be accomplished by optimizing for specific groups - recommender and personalization systems are another example.
Interesting approach. I make heavy use of Mermaid as I find it easier to make small changes and bulk generate-and-copy for publishing elsewhere. But being able to update the diagram and automatically generate the output files is a nice hybrid.
I'd want to log when the requests start, as I'm mostly concerned with how well-distributed request arrival was at that level of granularity.
I wondered if the network layers in between my client and server were effectively "smoothing" request arrival across each second, or if instead requests were very bursty so that a per-minute spike in a typical graph was dominated by a few seconds or milliseconds within that minute.