Measuring QUIC vs TCP on mobile and desktop(blog.apnic.net)
blog.apnic.net
Measuring QUIC vs TCP on mobile and desktop
https://blog.apnic.net/2018/01/29/measuring-quic-vs-tcp-mobile-desktop/
46 comments
+1 to this. QUIC allows using CUBIC BBR [1], so a comparison based on the exact parameters used is actually comparing the exact parameters used.
The performance effects of QUIC implementing congestion control in userland are more interesting. OTOH, QUIC allows deploying new features to users (through cronet) in an efficient way. TCP does not.
[1] https://chromium.googlesource.com/chromium/src/net/+/master/...
> [Disclaimer: I've worked with some of the people who wrote BBR and QUIC, so I'm biased.] Ditto
The performance effects of QUIC implementing congestion control in userland are more interesting. OTOH, QUIC allows deploying new features to users (through cronet) in an efficient way. TCP does not.
[1] https://chromium.googlesource.com/chromium/src/net/+/master/...
> [Disclaimer: I've worked with some of the people who wrote BBR and QUIC, so I'm biased.] Ditto
minikites(3)
Am I understanding things correctly? Because QUIC ramps up its bandwidth estimate more aggressively, it fundamentally competes unfairly with TCP? Is that an inherent property or something that can be fixed? Definitely it's too late to fix it on the TCP side
Nice to know 7% of Internet traffic isn't playing fair with the rest just so one company's content loads a few ms faster! Really don't know how I feel about TCP in general losing out to QUIC if it saw much wider deployment outside Google
And finally, it's incredibly disappointing to read these results from a third party, rather than having a balanced perspective as part of the original marketing
Nice to know 7% of Internet traffic isn't playing fair with the rest just so one company's content loads a few ms faster! Really don't know how I feel about TCP in general losing out to QUIC if it saw much wider deployment outside Google
And finally, it's incredibly disappointing to read these results from a third party, rather than having a balanced perspective as part of the original marketing
QUIC fundamentally competes on a different level because TCP is broken for modern networks.
From a network usage standpoint, there isn't anything "unfair" about it -- QUIC runs atop UDP, which is also a perfectly acceptable and publicly available Internet protocol. And Chrome will happily use QUIC with non-Google hosts, the real problem is that production-quality QUIC server software is currently rare outside of Google.
From a network usage standpoint, there isn't anything "unfair" about it -- QUIC runs atop UDP, which is also a perfectly acceptable and publicly available Internet protocol. And Chrome will happily use QUIC with non-Google hosts, the real problem is that production-quality QUIC server software is currently rare outside of Google.
One of the major downsides to UDP is that it lacks any form of congestion control, so it's hard to justify QUIC's behaviour due to what it's built atop :)
Meanwhile we're not talking about a small slice of bandwidth spent on SIP video calls playing unfairly, we're talking about 70% or more of all Internet traffic suddenly stampeding over every pre-existing protocol, they're the floodgates I'm worried about
Meanwhile we're not talking about a small slice of bandwidth spent on SIP video calls playing unfairly, we're talking about 70% or more of all Internet traffic suddenly stampeding over every pre-existing protocol, they're the floodgates I'm worried about
Internet traffic or web traffic? Last I saw, Netflix and streaming video were the biggest slices of the pie.
My understanding is that most of that traffic is on the last mile due to them colocating the data at the ISP level[1].
[1] https://openconnect.netflix.com/en/
[1] https://openconnect.netflix.com/en/
If this was actually a problem, it would be trivial for network operators to prioritize TCP packets over UDP packets.
QUIC builds modern congestion control on top of UDP.
QUIC builds modern congestion control on top of UDP.
We really don't want the prioritisation smarts in the network -
End-to-end architecture of the internet, network neutrality and all that.
I think the problem is that currently there is no prioritizing within IP traffic and possibly this would cause TCP to suck while QUIC over UDP ran fine. This is just a guess based on how TCP congestion and sliding window work and I am very curious to see how these two interact at various percentages in the wild. If anyone knows how to simulate QUIC on Spirent or Ixia test gear please drop me a line and I will lab this up.
This thinking is completely divorced from the principles and practice applied by the internet engineering community. The fairness benchmark is TCP, and if built on UDP, protocols are expected to respond to congestion & be fair to TCP streams.
Why is TCP broken?
Use TCP on a lossy network. Keep adding more users till you are using a majority of the bandwidth. Watch the network fail in congestion collapse.
That's actually a misrepresentation. TCP actually works pretty well on crowded networks; a major feature of TCP is to avoid congestion collapse. It doesn't always fully saturate busy/lossy networks, which is an area for improvement, but it's not the same as congestion collapse.
TCP BBR is an attempt to fix TCP congestion control so it can saturate busy/lossy networks more reliably. That doesn't require QUIC.
TCP BBR is an attempt to fix TCP congestion control so it can saturate busy/lossy networks more reliably. That doesn't require QUIC.
Are you proposing a circuit-switched replacement for IP, or do you think the current network would serve better if there was no backoff in the face of congestion?
The ATM effort was a while back, but who knows, maybe it's time to try again.
The ATM effort was a while back, but who knows, maybe it's time to try again.
Does any other protocol do better? The Internet is not a truck.
Use TCP across continents. See how long it takes for the window size to ramp up until your bandwidth is fully used.
I’ve often wondered how much cheating goes on in TCP and who would notice.
Many years ago my company made a VPN which tunneled through TCP† specifically designed for heavily loaded networks. I changed the Linux kernel to cap our back off time to something like 2 seconds because getting to a 2 minute back off just sucks badly when you are encapsulating a bunch of streams. Happy customers. Happy us, no more “the VPN hangs” complaints.
␄
† Yes, "don't do that", but when you have a mission critical unchangeable protocol sending enormous multi fragment UDP datagrams over a network path with a 10% packet loss rate, something has to give.
Many years ago my company made a VPN which tunneled through TCP† specifically designed for heavily loaded networks. I changed the Linux kernel to cap our back off time to something like 2 seconds because getting to a 2 minute back off just sucks badly when you are encapsulating a bunch of streams. Happy customers. Happy us, no more “the VPN hangs” complaints.
␄
† Yes, "don't do that", but when you have a mission critical unchangeable protocol sending enormous multi fragment UDP datagrams over a network path with a 10% packet loss rate, something has to give.
I'd like to know what the efficiency gap is - you can see in the graph that early on, quic is better saturating the connection, while TCP is still scaling up. The other problem is related to how TCP connections work with HTTP/1.1 or similar protocols - one TCP connection per concurrent HTTP request, where with quic any arbitrary number of requests are all part of the same quic stream. It doesn't seem unfair that adding more TCP connections doesn't make quic reduce its share of bandwidth, any more than adding 1000 concurrent http requests into the quic stream should reduce TCP's share of bandwidth.
Google makes QUIC available for others to use if desired. QUIC sits on top of UDP so any other service that uses UDP you could make a similar argument they are optimizing for their service. Which everyone should be doing. Your post sounds more like it is driven more by some issue you have with Google then QUIC.
> Your post sounds more like it is driven more by some issue you have with Google
I'll ignore that and suggest instead you read up on what happened to the Internet in the early 90s, the last time congestion control literally broke everything
I'll ignore that and suggest instead you read up on what happened to the Internet in the early 90s, the last time congestion control literally broke everything
Any chance you can point to a source about what happened in the 90s? I googled "congestion control what happened to the Internet in the early 90s" and it lead me no where.
The relevant keyword fu is 'congestion collapse', and it seems I may be misremembering what you can read about in the opening of Van Jacobsen's paper on congestion control ( http://ee.lbl.gov/papers/congavoid.pdf ) and some new algorithms that were introduced in the early-mid 90s, but perhaps someone with more clue knows what I was thinking of
Finally it's not the first time Google did something foul like that in the protocols department. IIRC they deploy a modified TCP slow start which also doesn't play fair in the face of contention
Finally it's not the first time Google did something foul like that in the protocols department. IIRC they deploy a modified TCP slow start which also doesn't play fair in the face of contention
Congestion collapse is what happened in the mid 80s that led to the creation of Jacobson's congestion control. He's at Google now, working on BBR (which their QUIC uses). Are you thinking of the early P2P days?
There's not much by way of production grade QUIC server software available at present.
If you want to try QUIC yourself, you can use Caddy, which has experimental QUIC support (https://caddyserver.com/docs/cli#quic) powered by quic-go.
(However, QUIC versions are extremely transient and has very limited support in Chrome; the next release of Caddy will bring it back up-to-date again.)
(However, QUIC versions are extremely transient and has very limited support in Chrome; the next release of Caddy will bring it back up-to-date again.)
I'm still waiting for QUIC support for nginx.
I wonder how QUIC compares to KCP[0].
[0]: https://github.com/skywind3000/kcp/blob/master/README.en.md
[0]: https://github.com/skywind3000/kcp/blob/master/README.en.md
[deleted]
What's the current positioning of QUIC by Google? It predates HTTP/2, and Google hasn't been making noises about it lately. Is it being slowly phased out or just in a "let it be" status for now in Chrome?
It's being actively discussed at the IETF. Here's a doc that's last updated 2018-01-28: https://datatracker.ietf.org/doc/draft-ietf-quic-transport/
QUIC replaces TCP and is on IETF standards track.
HTTP/2 replaces HTTP/1.1 (and is a direct descendent of SPDY, which has origins ~2009).
HTTP/2 on TCP gets you some of the benefits that QUIC provides (multiplexing, reduced roundtrips), but HTTP/2 on QUIC is the best of both worlds.
IMO, there's not much noise about QUIC mainly because most people don't care enough about latency to care, and the current lack of production-quality QUIC server software makes caring more complicated than "just enable it", so low demand all around. This should pick up when it gets closer to a ratified standard.
HTTP/2 replaces HTTP/1.1 (and is a direct descendent of SPDY, which has origins ~2009).
HTTP/2 on TCP gets you some of the benefits that QUIC provides (multiplexing, reduced roundtrips), but HTTP/2 on QUIC is the best of both worlds.
IMO, there's not much noise about QUIC mainly because most people don't care enough about latency to care, and the current lack of production-quality QUIC server software makes caring more complicated than "just enable it", so low demand all around. This should pick up when it gets closer to a ratified standard.
The article points out that both the QUIC and TCP implementations they tested are using CUBIC congestion control, but that's not enough information, because the article also points out that QUIC is using "more aggressive parameters." It's tough to say which parameters are better, but what's unsaid is that a TCP implementation could change their parameters and get the same congestion control results as QUIC.
The supposed poor performance of QUIC on resource-limited mobile devices is, as they point out, because it's a user space implementation that is thus more expensive. If QUIC becomes popular, I assume there will be kernel implementations that are as resource efficient as TCP. Meanwhile, it's a lot easier to to experiments (such as tuning CUBIC parameters!) when you don't have to reboot to install a new version.
It's also quite common for various TCP congestion controllers to completely fail to saturate busy links, because of various limitations. If that happens, it might be that QUIC is able to fill the empty space, thereby taking "more than its fair share" only because TCP wasn't going to use that share anyway. Since the article doesn't even say which TCP implementation it's comparing against, and doesn't say what happens when the TCP sessions are competing only amongst themselves with no QUIC present, it's hard to say what's going on.
The funny thing about all this is that, if they use the same congestion control (which seems to be the intention given that they are both using CUBIC and Google is separately trying to fix congestion control via BBR[1]), they should both be about equally fair. The performance benefits of QUIC are not even congestion control related!
[1] https://queue.acm.org/detail.cfm?id=3022184
[Disclaimer: I've worked with some of the people who wrote BBR and QUIC, so I'm biased.]