Firefox brings smooth video playback with the fastest AV1 decoder(hacks.mozilla.org)
hacks.mozilla.org
Firefox brings smooth video playback with the fastest AV1 decoder
https://hacks.mozilla.org/2019/05/firefox-brings-you-smooth-video-playback-with-the-worlds-fastest-av1-decoder/
64 comments
Couple of unscientific observations:
* More CPU but acceptable on a modernish system
* On FF it's dropping zero frames, on chrome a insignificant handful on mine
* Testtube cookie setting this seems to be 3rd party...resets easily depending on your cookie settings
* Seems solid enough to keep it as default & help push adoption
* More CPU but acceptable on a modernish system
* On FF it's dropping zero frames, on chrome a insignificant handful on mine
* Testtube cookie setting this seems to be 3rd party...resets easily depending on your cookie settings
* Seems solid enough to keep it as default & help push adoption
Here on Firefox it just goes up to 480p...
I’m not sure if it’s because I’m on Firefox Nightly, but I get 1080p@60.
Proof: https://i.imgur.com/JoT2wsM.png.
Proof: https://i.imgur.com/JoT2wsM.png.
Not doubting you. I guess is just some hardware detection? Or don't know... https://imgur.com/a/8nTkQEj
Does anything seem amiss at https://www.youtube.com/html5?
Just WebM VP8 and MSE & WebM VP9
Tested AV1 2160p and worked flawlessly, nice quality with YouTube using Firefox.
Can't enable AV1.. clicking that button throws a QuotaExceededError: The quota has been exceeded.
Nice web. I'd like it if such decisions were made in the user agent, but once again all the functionality is baked into webshites that reinvent the wheel badly and the user agent is just a dumb client :(
Nice web. I'd like it if such decisions were made in the user agent, but once again all the functionality is baked into webshites that reinvent the wheel badly and the user agent is just a dumb client :(
On Linux Firefox suffers from lack of hardware decoding and encoding (it doesn't plug into vaapi), which makes performance quite bad on laptops. I suspect something like WebEx, for example, works horribly in result. Or may be it's just Cisco not doing a good job. Conferencing experience with their WebRTC client in Firefox on Linux is horrendous.
We were just talking about VA on Linux. There is a chance for it to be enabled soon.
Oh, that's great to hear! I thought this might come into focus, once WebRender is officially released.
By the way, how is the progress for WebRender over Vulkan?
By the way, how is the progress for WebRender over Vulkan?
Current state of Vulkan port is that the correctness is on par with GL, minus a few driver bugs. We still need to re-architect WebRender a bit in order to get the performance advantage over GL.
I just switched to Firefox again after some shortsighted changes on Chrome. Neither give me GPU decode on Linux, but FF is SIGNIFICANTLY worse than Chrome on CPU decode. I can do a build in the background using all 12 threads and have no issues with video play back on Chrome (well, maybe not 4k60) but FF it's unwatchable, even at 900p60. (i7 8700k if you're wondering hardware wise, not OC'd because horrible chip and not worth it)
So VAAPI support can soon land in Firefox for Linux? Cool. This is great news for me.
Can you give us some details? When? Can I somehow help in testing, for example in Firefox nightly? What hardware can be supported? What formats, h264 or/and vp9?
Can you give us some details? When? Can I somehow help in testing, for example in Firefox nightly? What hardware can be supported? What formats, h264 or/and vp9?
We can't promise anything yet, sorry, but we are looking into it.
The bug report is 4 years old:
https://bugzilla.mozilla.org/show_bug.cgi?id=1210726
It's such a disappointment that this hasn't been fixed yet.
https://bugzilla.mozilla.org/show_bug.cgi?id=1210726
It's such a disappointment that this hasn't been fixed yet.
Sadly Linux has been a 3rd class citizen for years as far as Mozilla is concerned.
Or maybe it's a complex engineering challenge?
The Firefox Devs aren't just sitting there not clicking a button called "enable hardware acceleration"
The Firefox Devs aren't just sitting there not clicking a button called "enable hardware acceleration"
[deleted]
Well at least Chromium has support for this. I don't know why it is taking Mozilla so long honestly, but I'm guessing most of their developers don't use Linux and the org just loves to virtual-signal and not practice what they preach.
Chromium/Chrome does not support HW video decoding on desktop Linux:
https://bugs.chromium.org/p/chromium/issues/detail?id=463440
Patches exist to address it, though.
https://bugs.chromium.org/p/chromium/issues/detail?id=463440
Patches exist to address it, though.
So you're saying it does support it, and is often packaged that way in Linux. Remind me how it doesn't support it again?
Chromium won't take your bug reports on this feature.
google refuses to support this on desktop linux though, it requires a patched chromium build. The only reason vaapi support is there at all is because their chromeOS uses it.
I used Chromium and it is very easy to use on Arch. Not sure why you act like it isn't supported. A lot more than what is available for Firefox.
because it isn't supported, just look at the comments from google employees on the bug: https://bugs.chromium.org/p/chromium/issues/detail?id=463440...
sure the situation is better than it is with firefox, but having to use a patched chromium build isn't exactly ideal
sure the situation is better than it is with firefox, but having to use a patched chromium build isn't exactly ideal
The zealots whining and not respecting the few rules in place regarding Mozilla's Bugzilla sure have helped...
Seems to be limited to playback. Encoding is probably more critical for this.
It's weird, my Firefox seems to use the vaapi (but my iGPU is only able to hw-decode h264 and VP8 and VP9.Too old for AV1.)
I have a not so powerful laptop so I think the HW decoding capability of the iGPU is actually used, otherwise I couldn't play 1080p videos at 60 FPS (h264 and VP9).
I use the static Firefox from Mozilla. Not distro repo's one.
Nope. It's just software decoders from FFmpeg project that are heavily optimized. Also Firefox disables VP9 on slower machines, so websites serve H.264 instead.
At least on YouTube, I use VP9 (we can check that with the "advanced stats" showed when right-clicking on the video).
My laptop is usually noisy when the CPU is in demand. I am a bit surprised because playing HD videos on YT does not make the fan too noisy (comparing to loading a website with a ton of JS,for example).
So, I concluded that some HW decoding was in use, somehow. I can, of course, be wrong.
There is no way that ffmpeg uses a HW decoding decoder under the hood? It may explain why it's so smooth. Besides, when I run VP9 video with MPV, it tells me that it is explicitly using HW decoding via vaapi. And MPV is based on ffmpeg libs for the playback, if I am not mistaken.
Thanks for your light!
My laptop is usually noisy when the CPU is in demand. I am a bit surprised because playing HD videos on YT does not make the fan too noisy (comparing to loading a website with a ton of JS,for example).
So, I concluded that some HW decoding was in use, somehow. I can, of course, be wrong.
There is no way that ffmpeg uses a HW decoding decoder under the hood? It may explain why it's so smooth. Besides, when I run VP9 video with MPV, it tells me that it is explicitly using HW decoding via vaapi. And MPV is based on ffmpeg libs for the playback, if I am not mistaken.
Thanks for your light!
I've just checked Firefox 67.0 source code and found no evidence of hw-accelerated decoding. Firefox uses part of FFmpeg source code under the name "ffvpx", for VP9. There are some remainders of such decode-and-then-download-to-CPU-memory code there, but actual part interfacing with VA-API or any other hw-accelerated API is removed.
Although it's possible to utilize hardware decoders with FFmpeg, frames need to be shown somehow. Downloading decoded frames back to CPU memory just to copy them back to GPU to display is not a good idea. Generally graphics adapters expect a lot more data coming to them, not from them. It may be so slow, that it makes the whole idea infeasible. To make hardware decoding actually useful, one needs to figure out how to combine everything together on GPU side. It's harder to do it in Firefox than in a video player, since browsers have a complex rendering model. And now even run in multiple processes.
> My laptop is usually noisy when the CPU is in demand.
Not only software codecs are highly optimized, CPUs are also power-efficient. 1080p VP9 decode on my Skylake laptop doesn't produce enough heat to start the fan.
Although it's possible to utilize hardware decoders with FFmpeg, frames need to be shown somehow. Downloading decoded frames back to CPU memory just to copy them back to GPU to display is not a good idea. Generally graphics adapters expect a lot more data coming to them, not from them. It may be so slow, that it makes the whole idea infeasible. To make hardware decoding actually useful, one needs to figure out how to combine everything together on GPU side. It's harder to do it in Firefox than in a video player, since browsers have a complex rendering model. And now even run in multiple processes.
> My laptop is usually noisy when the CPU is in demand.
Not only software codecs are highly optimized, CPUs are also power-efficient. 1080p VP9 decode on my Skylake laptop doesn't produce enough heat to start the fan.
> It's weird, my Firefox seems to use the vaapi
Out of curiosity how do you check that? I'm asking because I'd like to check it on my end.
Out of curiosity how do you check that? I'm asking because I'd like to check it on my end.
I am checking that with the "about:support" page. My iGPU and the Intel drivers are correctly recognized so I was assuming that HW decoding was enabled.
But I can certainly be wrong, good platform detection does not necessarily mean that I use HW decoding. I just assumed it was the case.
FWIW I've never had an issue with WebEx on Linux Firefox (besides general WebEx UI/UX things).
Does it work for conferencing too? I often get "low bandwidth / low CPU" warning and no video feed, despite having quite good hardware. The more people are in the room - the worse (it's as if instead of multiplexing on the backend, it tries to multiplex on the client side). Very annoying.
For me appear.in worked perfectly in conferences and AFAIK it uses plain WebRTC.
With Firefox 67 on a quad core Mac I'm getting solid video playback on AV1 on Youtube 1080p
If you want to test this right now, here is how to enable AV1 in Youtube and test videos.
Goto the youtube testtube page and click "Always Prefer AV1" https://www.youtube.com/testtube
AV1 Beta Launch Playlist
https://www.youtube.com/playlist?list=PLyqf6gJt7KuHBmeVzZteZ...
And click "stats for nerds", if it's working it should show "av01" under the Codecs tab.
https://i.imgur.com/eKTFaYU.png
If you want to test this right now, here is how to enable AV1 in Youtube and test videos.
Goto the youtube testtube page and click "Always Prefer AV1" https://www.youtube.com/testtube
AV1 Beta Launch Playlist
https://www.youtube.com/playlist?list=PLyqf6gJt7KuHBmeVzZteZ...
And click "stats for nerds", if it's working it should show "av01" under the Codecs tab.
https://i.imgur.com/eKTFaYU.png
Thanks, I used the Halo video[0] to compare, a couple of notes.
- Youtube seems to have limited the AV1 support to 1080p, 1440/2160p are not available, anyone any idea why?
- The load on my CPU (MB Pro 2018 quad-core/ FF DE 67.0b9) seems about the same but I do seem to be dropping significantly less frames, not sure though.
[0] https://www.youtube.com/watch?v=Fmdb-KmlzD8&list=PLyqf6gJt7K...
[0] https://www.youtube.com/watch?v=Fmdb-KmlzD8&list=PLyqf6gJt7K...
> Youtube seems to have limited the AV1 support to 1080p, 1440/2160p are not available, anyone any idea why?
They probably don't reencode 1440+p (there are 8K videos too) in AV1 as of yet.
They probably don't reencode 1440+p (there are 8K videos too) in AV1 as of yet.
it was probably encoded at a time when they didn't do 4k av1 tests but only 1080p, but since then they also did a 4k one if you wanna test it https://youtu.be/LXb3EKWsInQ
No mention of power efficiency though. Hardware decoders are good for more than just performance.
On the one hand this is great, it's hard to argue that it isn't a good thing to have a very fast CPU-based AV-1 decoder built into Firefox.
On the other hand, I'm wondering how long this decoder will be relevant, especially considering it could still take a while before AV-1 video will be widespread on the web. I fully expect everyone and the kitchen sink will have hardware decoding before long, which is vastly superior over any software implementation.
On the other hand, I'm wondering how long this decoder will be relevant, especially considering it could still take a while before AV-1 video will be widespread on the web. I fully expect everyone and the kitchen sink will have hardware decoding before long, which is vastly superior over any software implementation.
That hardware decoding is more important and will become prevalent sooner on mobile.
This will allow early adopters and video producers to validate their content.
It will also allow people in five years who have a 5-year-old desktop to enjoy the content.
This will allow early adopters and video producers to validate their content.
It will also allow people in five years who have a 5-year-old desktop to enjoy the content.
People don't upgrade their hardware as often as they used to. For example my main machine is nearly ten years old at this point.
> our latest figures showing that 11.8% of video playback in Firefox Beta used AV1
It seems remarkably popular already.
It seems remarkably popular already.
YouTube started testing AV1 - I would not be surprised to learn that’s most of that number alone. Video format market share is tricky to measure because just a few sites like YouTube, Netflix, Vimeo, Twitch, etc. represent a huge fraction of the total streaked bandwidth even though there are so many other sites which publish video on the web.
Yeah, last I remember, Netflix and Youtube alone use something like 50% of the internet's entire bandwidth. Video is pretty data heavy though so I'm sure there's a huge push to switch to something more efficient.
About half of the frames get dropped for me.
https://i.imgur.com/MeLQXST.png
Has anybody published an analysis that clearly demonstrates AV1’s compression efficiency over HEVC?
I’ve seen claims of benefits, and analysis versus h.264, but not the clear evidence that HEVC is going to become obsolete.
I’ve seen claims of benefits, and analysis versus h.264, but not the clear evidence that HEVC is going to become obsolete.
Here's one. See the "High Quality (AV1) Report":
http://www.compression.ru/video/codec_comparison/hevc_2018/
As for HEVC's obsolescence, Leonardo Chiariglione, founder and chairman of MPEG, thinks AV1 has the bigger future because HEVC's licensing is too messy:
http://blog.chiariglione.org/a-crisis-the-causes-and-a-solut...
http://www.compression.ru/video/codec_comparison/hevc_2018/
As for HEVC's obsolescence, Leonardo Chiariglione, founder and chairman of MPEG, thinks AV1 has the bigger future because HEVC's licensing is too messy:
http://blog.chiariglione.org/a-crisis-the-causes-and-a-solut...
still can't get smooth 4k60fps vp9 playback when using speed more than 1.5x on youtube, meanwhile in chrome it works fine
I know you're serious but it's hard to take that complain seriously.
I mean it's the only way u can watch steve from gamers nexus without falling asleep from his voice, great content otherwise...
I don't need to see Steve in 4k60fps though... I mean his hair flips are epic and all but still.
True, but if I am watching videos I'm feeling lazy and I shouldn't have to turn down resolution just for his videos because they're in 60fps instead of 30 when in Chrome it works fine (I'm using tampermonkey script which switches all videos to the highest resolution with 4k as a limit).
Any updates on the free hw decoder by Google? I am hoping to test it on a FPGA (if possible)
So this basically brings Firefox on par with Chromium, which was already using dav1d?
Why isn't there a sample?
Try it on YouTube. Turn on AV1 via TestTube: https://www.youtube.com/testtube
Try the AV1 test playlist: https://www.youtube.com/playlist?list=PLyqf6gJt7KuHBmeVzZteZ...
And you'll find many other YouTube videos encoded in AV1 up to 720p. Try popular music videos or movie trailers for examples. Check that it's AV1 by right clicking on the video and selecting Stats for Nerds.
Try the AV1 test playlist: https://www.youtube.com/playlist?list=PLyqf6gJt7KuHBmeVzZteZ...
And you'll find many other YouTube videos encoded in AV1 up to 720p. Try popular music videos or movie trailers for examples. Check that it's AV1 by right clicking on the video and selecting Stats for Nerds.
Wish they could fix full-range video (for any format) playback first :/
I've filed bugs about this before, part of the problem is that every node in the chain needs to handle video ranges correctly. Once you introduce hardware decoders you can be in bad shape here because the range becomes a guess. Depending on which GPU vendor you have and what OS you're on, there may be a system-wide default output range (and a separate default for videos without range information), and the range assumptions may depend on application. If you use nerdy video compositors like madVR you can get them to tell you how much info they have on video ranges, and in many cases the answer is 'none' so they guess. At least on Windows, it is a fact that some of the range issues are not under Firefox's control, because some of this stuff is delegated to the decoder and to the video driver. (You can configure it in NVIDIA's Control Panel application, fwiw.)
In practice people seem to solve this by just using limited range all the time because it's much better to have limited range content displayed without expansion than to have full range content erroneously limited->full expanded and have a bunch of the range destroyed. If you try to stream to Twitch, for example, they tell you to use limited range, and it will look VERY bad in most players (not just Firefox!) if you disregard the advice. I've also had problems with full range uploads to YouTube that can't be attributed to Firefox.
EDIT: Checked my bug history. For the NVIDIA + Windows + Firefox combo, here is the workaround for range issues along with an explanation of why Mozilla has no way to fix it in this scenario. https://bugzilla.mozilla.org/show_bug.cgi?id=1229304#c8
In practice people seem to solve this by just using limited range all the time because it's much better to have limited range content displayed without expansion than to have full range content erroneously limited->full expanded and have a bunch of the range destroyed. If you try to stream to Twitch, for example, they tell you to use limited range, and it will look VERY bad in most players (not just Firefox!) if you disregard the advice. I've also had problems with full range uploads to YouTube that can't be attributed to Firefox.
EDIT: Checked my bug history. For the NVIDIA + Windows + Firefox combo, here is the workaround for range issues along with an explanation of why Mozilla has no way to fix it in this scenario. https://bugzilla.mozilla.org/show_bug.cgi?id=1229304#c8
Firefox does not support full-range video at all, even with software decoding.
See bug ticket: https://bugzilla.mozilla.org/show_bug.cgi?id=1459526 Or just test this: https://bug1459526.bmoattachments.org/attachment.cgi?id=8984...
Firefox's video support in general is pretty poor compared to Chrome. It doesn't support RGB or 4:4:4/4:2:2 YUV video either.
>here is the workaround
I knew that bug too, but that's a different story. That is about Firefox having problem to playback limited range video properly with nV graphic card (and it's partially nVidia's fault than Firefox).
Chrome used to have that bug too, but they have fixed it.
(Firefox team on the other hand, didn't plan to fix it for Win7 at least: https://bugzilla.mozilla.org/show_bug.cgi?id=1410693#c1 ).
See bug ticket: https://bugzilla.mozilla.org/show_bug.cgi?id=1459526 Or just test this: https://bug1459526.bmoattachments.org/attachment.cgi?id=8984...
Firefox's video support in general is pretty poor compared to Chrome. It doesn't support RGB or 4:4:4/4:2:2 YUV video either.
>here is the workaround
I knew that bug too, but that's a different story. That is about Firefox having problem to playback limited range video properly with nV graphic card (and it's partially nVidia's fault than Firefox).
Chrome used to have that bug too, but they have fixed it.
(Firefox team on the other hand, didn't plan to fix it for Win7 at least: https://bugzilla.mozilla.org/show_bug.cgi?id=1410693#c1 ).
https://www.youtube.com/watch?v=Fmdb-KmlzD8
My system consistently drops frames at the point the jeep drives up the hill and then recovers for the remainder of the video. Other than that it's fine.
You can enable AV1 on YouTube via TestTube (https://www.youtube.com/testtube) and you can check if a video is in AV1 via right clicking on the video and selecting Stats for Nerds. If it's AV1 the codec will be "av01".