Decoding Netflix's AV1 Streams(singhkays.com)
singhkays.com
Decoding Netflix's AV1 Streams
https://singhkays.com/blog/netflix-av1-decode/
79 comments
Taking screenshot of netflix on my device results in black square, I dont know if this applies on lower levels of Widevine but if it doesn't then the quality will be much lower as Netflix do not serve 720+ video unless there is protected DRM path
Fascinating, thank you for this analysis! Currently pining for release of an updated Apple TV, which will have an SoC capable of hardware AV1 decode.
It'd be great to hear from someone at Netflix about the unexpected Bojack Horseman results. I'd bet that Netflix just isn't yet taking advantage of AV1 features designed especially for this kind of animation and synthetic content.
It'd be great to hear from someone at Netflix about the unexpected Bojack Horseman results. I'd bet that Netflix just isn't yet taking advantage of AV1 features designed especially for this kind of animation and synthetic content.
> the unexpected Bojack Horseman results. I'd bet that Netflix just isn't yet taking advantage of AV1 features designed especially for this kind of animation and synthetic content.
While the percentages look scary, it's only a slight difference (60kbps!) and still around 1mbps average, but with a significant quality boost (very crisp lines and near perfect quality). I bet Netflix could encode at nearly half that bitrate and stay similar to HEVC in quality, but I'm pleased they seem to have made a good tradeoff here.
It's actually quite amazing the quality that AV1 delivers at such low bitrates across the board. I've said it before, but AV1 is almost magical. Which I think is behind the lack of enthusiasm for VVC/h266; is anyone even using that? I've yet to actually see it in the wild.
While the percentages look scary, it's only a slight difference (60kbps!) and still around 1mbps average, but with a significant quality boost (very crisp lines and near perfect quality). I bet Netflix could encode at nearly half that bitrate and stay similar to HEVC in quality, but I'm pleased they seem to have made a good tradeoff here.
It's actually quite amazing the quality that AV1 delivers at such low bitrates across the board. I've said it before, but AV1 is almost magical. Which I think is behind the lack of enthusiasm for VVC/h266; is anyone even using that? I've yet to actually see it in the wild.
Ditto. Between AV1's momentum and the imminent release of AV2, it sure feels like VVC will find limited adoption outside of broadcast applications.
https://aomedia.org/press%20releases/AOMedia-Announces-Year-...
https://aomedia.org/press%20releases/AOMedia-Announces-Year-...
Some of the torrent releases out of MVGroup are done as h266.
I think as technical tests as much as anything, but they're interesting to see.
I think as technical tests as much as anything, but they're interesting to see.
What benefit would a new appleTV have other than reduced bandwidth usage?
As someone who tried Roku, Android TV, and Fire TV before switching, better hardware offers a vastly better experience. The Apple TV's hardware is fast. The UI doesn't lag. Things feel smooth as butter. Yes, maybe the Apple TV doesn't need more, but more can be helpful.
In terms of AV1 support, YouTube often only does 4K with AV1 so that's an issue for people.
Personally, I'd love to see an Apple TV that was great for gaming. New Apple processors have hardware ray tracing and decent gaming performance.
I think it's also likely that Apple will try and make an Apple TV that will support next-gen Siri and on-device AI stuff. Yes, you can complain about Apple's AI delays, but Apple's probably looking toward an Apple TV that can support their AI models.
In some ways, "what benefit would a new <insert-thing> have?" Sometimes we don't know until we have it and people start using it.
In terms of AV1 support, YouTube often only does 4K with AV1 so that's an issue for people.
Personally, I'd love to see an Apple TV that was great for gaming. New Apple processors have hardware ray tracing and decent gaming performance.
I think it's also likely that Apple will try and make an Apple TV that will support next-gen Siri and on-device AI stuff. Yes, you can complain about Apple's AI delays, but Apple's probably looking toward an Apple TV that can support their AI models.
In some ways, "what benefit would a new <insert-thing> have?" Sometimes we don't know until we have it and people start using it.
Apple just hasn't been able to get traction with gaming on Apple TV. Gaming on Apple TV is so small I couldn't even find an analyst report breaking down the market size.
I don't think new graphics hardware solves the problem. Beyond the friction of the unit not shipping with a controller, tvOS lacks good discovery for games and there is no ad infrastructure comparable to mobile. Most game developers aren't looking to invest in small, closed platforms with bad discovery. It's hard enough to make money on Apple's mobile platforms.
I don't think new graphics hardware solves the problem. Beyond the friction of the unit not shipping with a controller, tvOS lacks good discovery for games and there is no ad infrastructure comparable to mobile. Most game developers aren't looking to invest in small, closed platforms with bad discovery. It's hard enough to make money on Apple's mobile platforms.
I wouldn't mind if they enabled the 120 Hz support with the new chipset. I like my TV but the framerate matching feature causes a few seconds of black screen each time I switch between and also drops the UI to 24 FPS. Would be nice if everything just ran at 120 FPS all the time, and since 24/30/60 are perfect frame multiples that's nice to. NTSTC content at 23.976 I'd hope the player would just speed up at that point, but even if not... judder at 120 Hz is better than at 60 Hz.
Also 6 GHz Wi-Fi would be nice. I had to run a cable to my 2 because the 5 GHz airspace where I am is too crowded to stream high quality movies via Infuse without occasional hitching. Same with the seek speed. Meanwhile my iPhone gets 2.9 Gbps of goodput at solid jitter on 6 GHz Wi-Fi.
There's probably some updates to the HDR standards. For me at least though the current one already supports what my TV does.
Also apps seem to assume "because hardware decode isn't available don't serve AV1" sometimes. As silly as that is with the CPU power in the AppleTV, at least that problem would go away with hardware support and they'd stop trying to serve a "compatible" SDR h.264 stream. Despite internet pessimism, sometimes the quality is also raised with more efficient codecs rather than just "the same quality at less bandwidth".
Also 6 GHz Wi-Fi would be nice. I had to run a cable to my 2 because the 5 GHz airspace where I am is too crowded to stream high quality movies via Infuse without occasional hitching. Same with the seek speed. Meanwhile my iPhone gets 2.9 Gbps of goodput at solid jitter on 6 GHz Wi-Fi.
There's probably some updates to the HDR standards. For me at least though the current one already supports what my TV does.
Also apps seem to assume "because hardware decode isn't available don't serve AV1" sometimes. As silly as that is with the CPU power in the AppleTV, at least that problem would go away with hardware support and they'd stop trying to serve a "compatible" SDR h.264 stream. Despite internet pessimism, sometimes the quality is also raised with more efficient codecs rather than just "the same quality at less bandwidth".
> Also apps seem to assume "because hardware decode isn't available don't serve AV1" sometimes.
This isn't a completely unreasonable decision, since the current 2022 model's software AV1 decode apparently can only sustain 4K AV1 decode (although it handled 1080p content fine in my test) for as little as 45 minutes before thermal throttling kicks in.
This isn't a completely unreasonable decision, since the current 2022 model's software AV1 decode apparently can only sustain 4K AV1 decode (although it handled 1080p content fine in my test) for as little as 45 minutes before thermal throttling kicks in.
"NTSTC content at 23.976 I'd hope the player would just speed up at that point, but even if not... judder at 120 Hz is better than at 60 Hz."
I'd bet money when TVs are advertised at 120 FPS, they're really 119.88 FPS, so no judder showing 23.976 FPS and the other NTSC-off display rates.
I'd bet money when TVs are advertised at 120 FPS, they're really 119.88 FPS, so no judder showing 23.976 FPS and the other NTSC-off display rates.
Some content is truly exactly 24 FPS or 30 FPS though, so whichever path the TV goes (i.e. NTSC rate or integer rate) the same problem will exist. I suppose some TVs might have extremely fancy film mode detection which catches the occasional frame difference, but I doubt mine does :D
The Bojack example shows that it's not just reduced bandwidth; in some cases you can get higher quality with their AV1 encodes. Additionally if you are thinking of average bitrate, that's ignoring variable bitrates (and the extension, the per-shot encoding params that Netflix utilizes.) That is to say, high complexity scenes get more bitrate than low complexity, so very noisy scenes can look way better using AV1 even while still using lower bitrates, but notably some peaks have similar max bitrates.
The promise of new codecs is reduced bandwidth and higher quality. Probably a new device also has a faster processor and/or more ram in general, which helps with incidental jank. Apple TVs are well regarded and maybe have less jank than other products (I don't have personal experience, I'm interested, but my Apple computer is a IIe so I expect account management issues), but it's nice to get a new Roku every once in a while as bloat/jank seems to creep up on them.
The credible rumors beyond AV1 decode include: Wi-Fi 7 via Apple's home-grown N1 chip, a CPU fast enough to support the next-gen Siri release on-device, a RAM bump, improved pass-through for high-end audio formats, potentially a camera (the new square selfie sensor would be perfect for this) for easier group/family FaceTiming, and more aggressive pricing.
The appletv is such a good device that I’m paying it whatever the price may be.
TVs have horrible UIs and are generally ad ridden garbage. Not using anything android based because of the same reason and slow.
TVs have horrible UIs and are generally ad ridden garbage. Not using anything android based because of the same reason and slow.
Google TV UI is fine, only one I accept other than Apple TV
> Device Support: Hardware decoding for AV1 isn’t on every device yet.
By now - it should be in most devices that's aren't outdated by even average standards. And it's worth mentioning that for devices that don't have hardware decoding, dav1d does an excellent job of decoding it on the CPU.
The problem is more with hardware encoding. That's indeed only present in only recent generations (or a couple) of hardware and even with that, AMD for example have an aspect ratio limitation bug in their AV1 hardware encoder (which requires adding black bands to work around) that's only fixed in RDNA 4 which is not available in their APUs, so it won't be fixed in APUs until their UDNA is used for them (they didn't fix it in RDNA 3.5 chips).
By now - it should be in most devices that's aren't outdated by even average standards. And it's worth mentioning that for devices that don't have hardware decoding, dav1d does an excellent job of decoding it on the CPU.
The problem is more with hardware encoding. That's indeed only present in only recent generations (or a couple) of hardware and even with that, AMD for example have an aspect ratio limitation bug in their AV1 hardware encoder (which requires adding black bands to work around) that's only fixed in RDNA 4 which is not available in their APUs, so it won't be fixed in APUs until their UDNA is used for them (they didn't fix it in RDNA 3.5 chips).
Was once out in a remote area on an 800 kbps DSL connection. YouTube couldn't stream, Prime Video couldn't stream. Netflix worked fine. Years later, I remain impressed at their uniqueness.
Okay, so AV1 has lower bitrate. I can encode any video format at arbitrary bitrates, but that metric is not useful on its own. An article about how AV1 requires less bits for the same or improved perceptual quality would have been far more interesting.
This article is a good primer on how AV1 works (and compares with VP8 and VP9): https://www.red5.net/blog/av1-vs-vp9-vs-vp8-comparison-for-l...
Why is this now flagged ?
Perhaps the most important question which I have yet to see anyone pointing out. Those bit rate numbers are appalling! 2-4Mbps average bitrate? For a services you are paying for? I knew Netflix has gotten bad but this is worse than I thought. Even some high end YouTube Content does better than that. It should be 8Mbps minimum. And at this bitrate the difference between H.264 and AV1 wont be so obvious.
Perhaps the most important question which I have yet to see anyone pointing out. Those bit rate numbers are appalling! 2-4Mbps average bitrate? For a services you are paying for? I knew Netflix has gotten bad but this is worse than I thought. Even some high end YouTube Content does better than that. It should be 8Mbps minimum. And at this bitrate the difference between H.264 and AV1 wont be so obvious.
Wondering about the flag myself
>Those bit rate numbers are appalling! 2-4Mbps average bitrate?
While those may sound low, what I'm thinking is Netflix didn't see any benefit in perceptual quality (or VMAF scores) but sending more bits down the pipe and increasing their bandwidth bill.
>Those bit rate numbers are appalling! 2-4Mbps average bitrate?
While those may sound low, what I'm thinking is Netflix didn't see any benefit in perceptual quality (or VMAF scores) but sending more bits down the pipe and increasing their bandwidth bill.
That is the problem. The think VMAF as the gospel, and AV1 is extremely well tuned for it.
VMAF is better than SSIM PSNR but it is still far from perfect. And it is a reason why most torrent are still using HEVC and AVC.
VMAF is better than SSIM PSNR but it is still far from perfect. And it is a reason why most torrent are still using HEVC and AVC.
Very good read, love some of the humor in this article. It helped me get through to the end!!
Also, If anyone was wondering where AV1 stands in comparison to VP8 and VP9... I just looked it up after a few years of not paying attention and I guess Google donated VP8 and VP9 to the alliance for open media foundation (AOMedia) in 2015 and they created AV1 and released it in 2018.
Also, If anyone was wondering where AV1 stands in comparison to VP8 and VP9... I just looked it up after a few years of not paying attention and I guess Google donated VP8 and VP9 to the alliance for open media foundation (AOMedia) in 2015 and they created AV1 and released it in 2018.
Yeah, AV1 is primarily based on what Google was working on for their own successor to VP9, what would have been VP10, with technology contributions from Mozilla/Xiph's Daala and Cisco's Thor codecs.
Thank you. Weaving in humor and wit was a conscious choice :) Based on some of the comments above, I might have gone a little overboard.
https://news.ycombinator.com/item?id=45446589
https://news.ycombinator.com/item?id=45446589
No, it was perfect!
The ads made this unreadable. Is this a thing? Get to hackernews front page & then inject your post with as many ads as possible to “cash in”?
Not doubting your experience, but I can't see a single ad using Safari on iPhone.
I was really impressed with my setup, but after disabling content blockers (Firefox Focus), and turning off using Mullvad's free DNS proxy service, still nothing!
Perhaps the author turned the ads off since you visited?
I was really impressed with my setup, but after disabling content blockers (Firefox Focus), and turning off using Mullvad's free DNS proxy service, still nothing!
Perhaps the author turned the ads off since you visited?
Why aren't you using an adblocker?
Sorry for that. I turned those off while I figure out what's going on.
Looks like latest chatgpt model is not aware of massive battery draining on mobile
This test wasn't done on mobile.
For mobile, I don't know who outside of Netflix is delivering AV1. If they are, I expect them to be leveraging the hardware AV1 decoders for battery life instead of employing a software only solution like dav1d. Saying that, I think Netflix was using dav1d solution where it had a benefit (e.g. low quality cellular networks)
https://netflixtechblog.com/netflix-now-streaming-av1-on-and...
For mobile, I don't know who outside of Netflix is delivering AV1. If they are, I expect them to be leveraging the hardware AV1 decoders for battery life instead of employing a software only solution like dav1d. Saying that, I think Netflix was using dav1d solution where it had a benefit (e.g. low quality cellular networks)
https://netflixtechblog.com/netflix-now-streaming-av1-on-and...
> For mobile, I don't know who outside of Netflix is delivering AV1.
Pretty sure Youtube forces it with dav1d on devices that don’t support hw decoding which is how i became aware of the battery problem
Pretty sure Youtube forces it with dav1d on devices that don’t support hw decoding which is how i became aware of the battery problem
Does anyone know what was used to produce the graphs?
Do you mean charts? If so, it's Datawrapper: https://www.datawrapper.de/charts
One of the quite expensive paid plans, as the free one has to have "Created with Datawrapper" attribution at the bottom. I would guess they've vibe-coded their way to a premium version without paying, as the alternative is definitely outside individual people's budgets (>$500/month).
One of the quite expensive paid plans, as the free one has to have "Created with Datawrapper" attribution at the bottom. I would guess they've vibe-coded their way to a premium version without paying, as the alternative is definitely outside individual people's budgets (>$500/month).
Inspecting the page, I can see some classes "dw-chart" so I looked it up and got to this: https://www.datawrapper.de/charts. Looks a bit different on the page, but I think that's it.
it is indeed Datawrapper like other posters have said. It works well for interactivity on a static blog like Hugo.
the worst screen tearing in old analog movies with camera panning.
I feel like this was copy edited by ChatGPT and it really grates me. I couldn’t help but lose focus after I started seeing telltale signs of AI.
While the topic matter is interesting, I feel like obviously synthetic content falls into the “that which was not worth writing, is not worth reading either” trap.
If the authors tone is extremely ChatGPT-esque, I apologize in advance.
While the topic matter is interesting, I feel like obviously synthetic content falls into the “that which was not worth writing, is not worth reading either” trap.
If the authors tone is extremely ChatGPT-esque, I apologize in advance.
Intolerably ChatGPT-esque. Which is a shame, it seems like a nifty little DIY experiment.
I think what stands out to me is this cartoonishly punchy, faux-dramatic framing.
That, and specialist terms that seem to be thrown in there in an empty way, just to signal subject-matter expertise that’s not even expected of a DIYer’s experiment report:
> It’s a multi-decade, billion-dollar street fight over bytes and pixels, waged in the esoteric battlegrounds of DCT blocks and entropy coding
I think what stands out to me is this cartoonishly punchy, faux-dramatic framing.
That, and specialist terms that seem to be thrown in there in an empty way, just to signal subject-matter expertise that’s not even expected of a DIYer’s experiment report:
> It’s a multi-decade, billion-dollar street fight over bytes and pixels, waged in the esoteric battlegrounds of DCT blocks and entropy coding
LLMs tend to produce ridiculous similes, idioms, euphemisms, and analogues that fall apart upon the slightest scrutiny, as though they are genuine English constructions. I despise it. No one in reality writes like that. Who the hell thinks 'bytes and pixels' are a battleground? if you want to talk about licensing problems, just say they are licensing problems, don't accept shitty similes from LLMs that humanise decidedly non-human constructs. More so if it's a technical discussion: just get to the bloody point.
I like using real idioms that have percolated through culture ('birds of a feather', 'white elephant', 'nip in the bud', etc), not stupid contrivations.
As someone who sweated through hours and hours of English essay-writing in school, LLM output that is misrepresented as genuine human writing is annoying and highly disrespectful of the reader's time and effort. The moment I saw the stupid, contrived headers and dozens of emojis, I closed the tab.
I refuse to waste my time reading the output of a matrix multiplication done in some server farm when I could do the latter myself.
I like using real idioms that have percolated through culture ('birds of a feather', 'white elephant', 'nip in the bud', etc), not stupid contrivations.
As someone who sweated through hours and hours of English essay-writing in school, LLM output that is misrepresented as genuine human writing is annoying and highly disrespectful of the reader's time and effort. The moment I saw the stupid, contrived headers and dozens of emojis, I closed the tab.
I refuse to waste my time reading the output of a matrix multiplication done in some server farm when I could do the latter myself.
Well, journalists often write like that. I presume that's where the LLMs learned. I always find that style annoying, but especially so in a technical post like this.
I don't know which publications you're reading, but the ones I read do not write like this!
Probably the only overlap between AI slop and actual journalistic writing is the obsession with em-dashes
And hallucinations.
Point taken :) In retrospect that metaphor doesn't add any technical clarity. Will address in the rewrite
https://news.ycombinator.com/item?id=45446589
https://news.ycombinator.com/item?id=45446589
> article about compression
> uses slopbot 9000 to explode his point into ten times the "prose"
> mfw
> uses slopbot 9000 to explode his point into ten times the "prose"
> mfw
I laughed. Now only if this was a real greentext.
This didn't trip my AI detector; I instinctively skimmed to look at the numbers and conclusions. Your comment made me go back up to the top and read the opening paragraphs and I see what you are saying. It is always painful to realize you are reading AI product. I think it is less of a problem with this blog post because it is just presenting a handful of tables of numbers and a few graphs, but it seems I am already unconsciously training myself to ignore florid AI writing.
> This didn't trip my AI detector
LLMs seem to love putting stuff in bold, thats an immediate red flag for me.
LLMs seem to love putting stuff in bold, thats an immediate red flag for me.
It's the emoji bullet points and headers that makes me instantly close a page.
"It's not just X, it's Y!"
Heh, didn't have to go far in this article to find this exact construction:
> The data shows it’s not just an incremental improvement; it’s a demolition.
Complete with extra bold to emphasize the second half, sigh.
> The data shows it’s not just an incremental improvement; it’s a demolition.
Complete with extra bold to emphasize the second half, sigh.
Wow, this entire thread is some direct and very valuable feedback. Thank you to everyone who weighed in. I hear you all loud and clear!
To be transparent, I was experimenting with a more "punchy," narrative style to weave in some wit and humor. I didn't want the writing to feel dry and was aiming for a flow that was more entertaining. In retrospect, I clearly overshot the mark and ended up with something that feels inauthentic and distracts from the main point.
The experiment and the data are what I was most excited to share, and the writing shouldn't obscure that. Based on this feedback, I'll revise the article to be more direct, cut the fluff, and let the numbers do the talking.
Seriously, I appreciate the reality check! This is a great lesson in "know your audience." :)
To be transparent, I was experimenting with a more "punchy," narrative style to weave in some wit and humor. I didn't want the writing to feel dry and was aiming for a flow that was more entertaining. In retrospect, I clearly overshot the mark and ended up with something that feels inauthentic and distracts from the main point.
The experiment and the data are what I was most excited to share, and the writing shouldn't obscure that. Based on this feedback, I'll revise the article to be more direct, cut the fluff, and let the numbers do the talking.
Seriously, I appreciate the reality check! This is a great lesson in "know your audience." :)
wtf, is this comment also AI written? this is peak satire
Sorry but don't believe everything is now written by AI. Some people still know how to write addressing feedback and take accountability.
I would 100000% rather read the author's own writing even if English is their 10th language
Rather than this inflated slop that look like I am trying to reach word count in a paper and one sentence becomes 15 useless ones
Edit: This is not so much commentary on AI than it is the core of your post is a few tables. Just post the tables and one or two sentence of conclusion and that is all ! It is so tedious to read through dozens of paragraph of autogenerated unnecessary nonsense -- that contribute nothing of value to the data
Rather than this inflated slop that look like I am trying to reach word count in a paper and one sentence becomes 15 useless ones
Edit: This is not so much commentary on AI than it is the core of your post is a few tables. Just post the tables and one or two sentence of conclusion and that is all ! It is so tedious to read through dozens of paragraph of autogenerated unnecessary nonsense -- that contribute nothing of value to the data
It's unfortunate because there's plenty of good, technical, perfectly readable content in the author's blog archive from 2021 and earlier without the overwrought purple prose.
https://singhkays.com/archives/
https://singhkays.com/archives/
That's kind of you. Thank you for taking the time to look through my older posts. Will be addressing this post based on the feedback here
https://news.ycombinator.com/item?id=45446589
https://news.ycombinator.com/item?id=45446589
I wonder why there is 4 year gap there. Their English is not even bad ! Its better than mine and definitely does not need any kind of GPT copyediting
Life and other priorities :)
But on the bright side I've got a large backlog of ideas now just waiting to be turned into pixels
But on the bright side I've got a large backlog of ideas now just waiting to be turned into pixels
I definitely appreciate the concise nature of the article. Doesn't waste my time at all. Whether that's due to the authors writing ability or some summery tool, I could care less.
[deleted]
Well, use the chatgpt-based compression system for this articl about compression.
By that I mean they might have used ChatGPT to expand this article from simple bullet points and now you can use ChatGPT to summarize this article into succint bullet points for quick digestion.
I feel the same, it's unreadable, which is too bad because the work seems interesting.
I agree that this text in its current style is very hard to read. Feels like the text was ballooned up to 3 or 4 times its original length with pointless "side content"? Lots of distracting noise basically. AI or not AI, this is not very good.
… and so I'll continue to stick with AVC, thanks! :-)
… and so I'll continue to stick with AVC, thanks! :-)
Came to say pretty much the same thing. This slop is unreadable for me at this point.
I keep getting a paragraph or two into something, read one of the terrible "It's not just word - it's massive hyperbole!" sentences, see that there are several more in subsequent paragraphs and can't continue.
However bad the author's original writing that generate this output was, it can't be as awful as this.
I keep getting a paragraph or two into something, read one of the terrible "It's not just word - it's massive hyperbole!" sentences, see that there are several more in subsequent paragraphs and can't continue.
However bad the author's original writing that generate this output was, it can't be as awful as this.
The "it's not this - it's that!" phrasing is everywhere now and is driving me insane.
Presumably nothing jumped out at the author as being worse, but come on how can you have a whole section on why AV1's regression on Bojack is actually a good thing because the quality is way higher, and then not show any quality comparisons?