An Invisible Tax on the Web: Video Codecs (2018)(blog.mozilla.org)
blog.mozilla.org
An Invisible Tax on the Web: Video Codecs (2018)
https://blog.mozilla.org/blog/2018/07/11/royalty-free-web-video-codecs/
173 comments
> There's a small gotcha in the new MPEG LA license agreement. Footnote 17, which appears in tiny print at the very end of the summary of license terms, notes that “Annual royalty caps are not subject to the 10% limitation.” Although that sounds like a loophole, it really only affects the largest players in the market.
Cisco’s free to use h264 encoder/decoder binaries of its open source OpenH264 are a result of this license cap so potentially it could affect others if those rates rise to the point they withdraw availability.
https://en.m.wikipedia.org/wiki/OpenH264
Cisco’s free to use h264 encoder/decoder binaries of its open source OpenH264 are a result of this license cap so potentially it could affect others if those rates rise to the point they withdraw availability.
https://en.m.wikipedia.org/wiki/OpenH264
the wikipedia page mentions that cisco pays to MPEG LA royalties on behalf of any software project that uses their precompiled binaries. How do they keep track of who downloaded their binaries and how much money they owe to MPEG LA?
AFAIK they pay the max amount, so it doesn't matter how many people use their binaries.
I think it's internally. The article also states that the users of the code would still have to pay the royalties.
No, it's not only internal use, as long as it's Cisco's binary that's being distributed. That's how for example Firefox ships H.264 decoding support.
The license does not cover any other binaries built from the same Cisco source code, though.
The Wikipedia article is also fairly clear about this and says: "any software projects that use Cisco's source code instead of its binaries would be legally responsible for paying all royalties to MPEG LA themselves, however."
(emphasis mine)
The license does not cover any other binaries built from the same Cisco source code, though.
The Wikipedia article is also fairly clear about this and says: "any software projects that use Cisco's source code instead of its binaries would be legally responsible for paying all royalties to MPEG LA themselves, however."
(emphasis mine)
> IIRC, H.264 is one of a few non-Google technologies that is included in Chrome, but is not included in Chromium.
That's interesting, I never missed it and use both Chromium and Firefox. Apparently we can do without this tax. Or might a nonfree codec package in Debian have something to do with it? I did hear someone complain once about my sending a nonfree video format (in a jovial manner), not sure about the details anymore but it played out of the box for me (in Chromium).
That's interesting, I never missed it and use both Chromium and Firefox. Apparently we can do without this tax. Or might a nonfree codec package in Debian have something to do with it? I did hear someone complain once about my sending a nonfree video format (in a jovial manner), not sure about the details anymore but it played out of the box for me (in Chromium).
I believe Firefox will support H.264 for videos only using the operating system's own support (so on Linux, if you have it supported in gstreamer and/or ffmpeg then it's supported in Firefox).
Linux firefox Plugins tab, though I'm not sure if it's used beyond webrtc:
> OpenH264 Video Codec provided by Cisco Systems, Inc.
> This plugin is automatically installed by Mozilla to comply with the WebRTC specification and to enable WebRTC calls with devices that require the H.264 video codec. Visit https://www.openh264.org/ to view the codec source code and learn more about the implementation.
> OpenH264 Video Codec provided by Cisco Systems, Inc.
> This plugin is automatically installed by Mozilla to comply with the WebRTC specification and to enable WebRTC calls with devices that require the H.264 video codec. Visit https://www.openh264.org/ to view the codec source code and learn more about the implementation.
Nope, it isn't used beyond WebRTC, and in Windows it even autodetects if the computer has a capable enough hardware encoder and use that instead (and decoding through native Windows libraries if there isn't a suitable hardware decoder).
If you’re using YouTube, it supports other formats than just H.264. So, in that instance, lack of H.264 isn’t noticeable.
Unless you don't have hardware accelerated decoding for VP8/9. Then it is very noticeable, especially on a laptop.
This completely killed my Tegra K1-based chromebook. What good is a Google laptop with a full HD display, if it can't play Youtube videos in any resolution higher than 480p?
That would be very odd because YouTube definitely supports H.264 videos in higher than 480p resolutions. VP9 is only exclusively used for some formats/resolutions (e.g., HDR is VP9-only, I think).
It's been a while since I sold it, but it would just choke on anything 720p or higher.
this!
h264 is the most widely supported codec to decode in hardware, so the inclusion of a software decoder in the browser is kinda moot.
h264 is the most widely supported codec to decode in hardware, so the inclusion of a software decoder in the browser is kinda moot.
Doesn't Windows bundle an H.264 decoder? If Chrome made use of that, would they have to pay for their own licence?
Windows bundles software decoders (for H.264 and H.265) that are available through MFT and DXVA.[1][2] Chromium leverages both already.[3] It's also a low effort way to support hardware-accelerated decoding as vendors (Intel, AMD, NVIDIA) register MFTs that wrap their propriety SDKs (Quick Sync, AMF, NVDEC).
These provided MFTs have to meet certain minimum requirements for certification by Microsoft, which amounts to "good enough." Results vary in practice and can be incredibly frustrating (especially encode). It's often better to use the underlying SDKs but that comes with a lot of hassle.
(I've been working on a video capture and sharing app inspired by Shadowplay.)
Windows 10 is also rolling out support for AV1, though not bundled.[4]
[1]: https://docs.microsoft.com/en-us/windows/win32/medfound/h-26...
[2]: https://docs.microsoft.com/en-us/windows/win32/medfound/abou...
[3]: https://chromium.googlesource.com/chromium/src/+/master/medi...
[4]: https://www.microsoft.com/en-us/p/av1-video-extension/9mvzqv...
These provided MFTs have to meet certain minimum requirements for certification by Microsoft, which amounts to "good enough." Results vary in practice and can be incredibly frustrating (especially encode). It's often better to use the underlying SDKs but that comes with a lot of hassle.
(I've been working on a video capture and sharing app inspired by Shadowplay.)
Windows 10 is also rolling out support for AV1, though not bundled.[4]
[1]: https://docs.microsoft.com/en-us/windows/win32/medfound/h-26...
[2]: https://docs.microsoft.com/en-us/windows/win32/medfound/abou...
[3]: https://chromium.googlesource.com/chromium/src/+/master/medi...
[4]: https://www.microsoft.com/en-us/p/av1-video-extension/9mvzqv...
> Chromium leverages both already
Why do they need to pay for their own licence then?
Why do they need to pay for their own licence then?
They dont, and neither do Mozilla if they decide to use the OS media playing framework.
But most Browsers tends to implement their own for all sort of reasons.
But most Browsers tends to implement their own for all sort of reasons.
IIRC this is what Firefox tries to do -- use the OS decoder. For many reasons it is not a great user experience.
It's incredible that the MPEG-LA mafia has gotten away with 23 patents on a system of mathematical operations. Math is explicitly outlined as non-patentable. While this extends to all software patents, I'm astounded that even this very narrow case hasn't been challenged on its face. We can't move to unencumbered codecs quickly enough.
Strictly speaking, pretty much any patentable invention is going to use maths. Video codec related patents aren't just "I patent the discrete fourier transform". It's "here's how you can use frequency domain transforms, texture lookups, entropy coding, and other things in order to build a low-bitrate video codec". There's detailed explanations of what the invention does and what is legally being claimed, which will usually be a quite-small improvement on an existing technology.
Many of the standards-essential ITU/MPEG patents have active litigation surrounding them, and the defendents would have been daft to not try to challenge video codec patents on exactly the sort of basis. The fact that the patents still stand would indicate that they are more than just maths.
Many of the standards-essential ITU/MPEG patents have active litigation surrounding them, and the defendents would have been daft to not try to challenge video codec patents on exactly the sort of basis. The fact that the patents still stand would indicate that they are more than just maths.
> The fact that the patents still stand would indicate that they are more than just maths.
I insist that the fact that the patents still stand is rather an indication of the systemic failure of the court system to enforce the non-patentability of what is ultimately still purely mathematical operations.
Sure, any patent-able invention uses math. That's a distraction. These patents are on the formulae (the data transforms) and their use itself. You're on the right track pointing out that it would be ridiculous to patent the discrete Fourier transform.
These kind of mathematical discoveries should not be encumbered, they should be free for anyone to use and implement.
I insist that the fact that the patents still stand is rather an indication of the systemic failure of the court system to enforce the non-patentability of what is ultimately still purely mathematical operations.
Sure, any patent-able invention uses math. That's a distraction. These patents are on the formulae (the data transforms) and their use itself. You're on the right track pointing out that it would be ridiculous to patent the discrete Fourier transform.
These kind of mathematical discoveries should not be encumbered, they should be free for anyone to use and implement.
This is my personal opinion and in no way relates to my employment:
I rather think that the courts have enforced the line the legislators endorse, which supports their ends of promoting industry: It's not a mistake of the court system, inventions that apply maths to achieve technical gains are patentable and maths itself remains un-patentable. The courts seem largely to properly discern where that line lies.
Developing maths that saves bandwidth, storage, and energy, for example, is beneficial and countries use the patent system to encourage that.
I rather think that the courts have enforced the line the legislators endorse, which supports their ends of promoting industry: It's not a mistake of the court system, inventions that apply maths to achieve technical gains are patentable and maths itself remains un-patentable. The courts seem largely to properly discern where that line lies.
Developing maths that saves bandwidth, storage, and energy, for example, is beneficial and countries use the patent system to encourage that.
If I invent a new type of chair that can be stable with two legs, then the way this operates is essentially just a mathematical formula, no? No one would challenge that this would be patentable, and any patent application would describe the mathematics of it, and the usage of this mathematics would be covered in the patent.
I don't really like patents in general, but I don't really see this sharp distinction to be honest and I've always felt that the case against software patents in particular as a special case was rather weak.
I don't really like patents in general, but I don't really see this sharp distinction to be honest and I've always felt that the case against software patents in particular as a special case was rather weak.
A chair is a physical object that may happen to have been designed using maths, a video codec is a set of instructions for what maths to do on your machine that does maths to make it play video.
My understanding (IANAL) is that although maths are not patentable, the use of maths is – much as science is not patentable, but the use of science (technology) is.
Exactly this - otherwise inventions in physics would not be patentable (such as the laser - a purely theoretical construct long before it was engineered, same as the transistor), or chemicals (chemistry follows from physics, which is made of math..)
And on and on.
Using math to engineer solutions to problems is patentable.
And on and on.
Using math to engineer solutions to problems is patentable.
When you patent a laser, the patent doesn't stop someone from calculating the laser's results. With math, you are prohibited from making certain calculations; very different.
You aren't prohibited from doing the calculations per se. You are prohibited from doing them for a particular purpose, specifically video playback, in this case. If you just want to sit down with a pencil and paper and calculate H264 by yourself, there's nothing illegal about that. It only becomes illegal when you embed it into a video player.
Again, IANAL. But my understanding is using ideas in a patent for "experimental purposes" is legally protected: https://nysstlc.syr.edu/wp-content/uploads/2014/09/NYSTAR-Th...
> chemistry follows from physics, which is made of math
Is that a reference to XKCD #435 ? ;)
Is that a reference to XKCD #435 ? ;)
Right, but encoding video is literally just pushing numbers through a function, is it not?
It's more concrete than that there's an input, from a computer, with some sort of physical storage, and an output, again with some storage, that ultimately is for viewing. The goal is to do stuff with those numbers outside of the computer, ie create light patterns on a viewing medium.
> It's more concrete than that there's an input, from a computer, with some sort of physical storage, and an output, again with some storage, that ultimately is for viewing.
None of which is novel, or even directly related to the subject of the patent. The patent isn't about the input mechanism, the computing hardware, the storage media, or the display tech. It's about the abstract math that is carried out mechanically by these existing computing elements.
Evaluating a mathematical formula and visualizing the results as a grid of colored squares—all by hand—should never be found to infringe on anyone's patent. (This appears to be relatively uncontroversial.)
Doing the same, only faster thanks to the aid of a computer, should be similarly exempt from all patent infringement claims. The patent doesn't cover the computer, which already existed. All that's left is the math.
None of which is novel, or even directly related to the subject of the patent. The patent isn't about the input mechanism, the computing hardware, the storage media, or the display tech. It's about the abstract math that is carried out mechanically by these existing computing elements.
Evaluating a mathematical formula and visualizing the results as a grid of colored squares—all by hand—should never be found to infringe on anyone's patent. (This appears to be relatively uncontroversial.)
Doing the same, only faster thanks to the aid of a computer, should be similarly exempt from all patent infringement claims. The patent doesn't cover the computer, which already existed. All that's left is the math.
I have a lot of sympathy with your position and conclusion. Really I was trying to explain the position applied, it's not what I would do. Ultimately, legislatures (or those who control them) want to encourage developments that lead to faster, cheaper (less energy intensive) methods of encryption and encoding and so the law will be wiggled until it fits this need. Hence in Europe we effectively have, AIUI, a variation of the long-standing "technical effect" requirement where if the development can be concretised it can be patented.
Things are slightly tighter in UK law and much looser in USA; I don't follow patent law in other countries.
Things are slightly tighter in UK law and much looser in USA; I don't follow patent law in other countries.
It's the application of the maths for a purpose, not the maths itself.
Maybe it's the future of software will be built in a jurisdiction of the world that doesn't take a shit about software patents.
All software is just a big number after all.
All software is just a big number after all.
Technically all information is just a big number.
And that number is 42. Now the question is how to write all the functions to decode what we want from it.
Use this encoder :
https://www.dangermouse.net/esoteric/lenpeg.html
It allows for compression to a negative, unbounded number of octets. I guess you just have to run it in reverse ?
https://www.dangermouse.net/esoteric/lenpeg.html
It allows for compression to a negative, unbounded number of octets. I guess you just have to run it in reverse ?
Perhaps in China?
I support your sentiment, but most countries with strong software communities appear to have strong software patents (US, EU, Taiwan, SK). Maybe the only outlier is Russia (and possibly India)?
I support your sentiment, but most countries with strong software communities appear to have strong software patents (US, EU, Taiwan, SK). Maybe the only outlier is Russia (and possibly India)?
in the EU software is not patentable, though many companies find loopholes: https://fsfe.org/activities/swpat/swpat.en.html
I'd say China, India, Russia all have pretty strong software communities and are good candidates.
China also has a strong hardware community as well.
China also has a strong hardware community as well.
We can argue about the patents system all day. But we can call it a fact this it is not just "23" patents. I am not even sure how they arrive at the 23 numbers. I was thinking if it was some kind of joke that doesn't work with my /s detector.
Algorithms are not simply math, in the same way mechanical engineering is not simply math.
The patents are the result of engineering to create very precise and complicated tools.
And if it were simply math, then there would be no way to work around those patents, which plenty of open source codecs have done in various directions.
Unfortunately, most open source codecs are not as good at compression, mostly since the patented ones have a massively bigger pool of engineering behind them from dozens of companies banding together their work to develop best of breed solutions.
And eventually the patents expire, and the world gets these codecs free to use.
The patents are the result of engineering to create very precise and complicated tools.
And if it were simply math, then there would be no way to work around those patents, which plenty of open source codecs have done in various directions.
Unfortunately, most open source codecs are not as good at compression, mostly since the patented ones have a massively bigger pool of engineering behind them from dozens of companies banding together their work to develop best of breed solutions.
And eventually the patents expire, and the world gets these codecs free to use.
> And if it were simply math, then there would be no way to work around those patents, which plenty of open source codecs have done in various directions.
What makes you say that? Math usually leaves many ways of calculating the same thing.
> dozens of companies banding together
You could probably get a bigger group if there wasn't a patent vs. no patent conflict.
> And eventually the patents expire, and the world gets these codecs free to use.
How many of these techniques wouldn't have been reinvented a dozen times over in less time, if the original patenter never existed?
What makes you say that? Math usually leaves many ways of calculating the same thing.
> dozens of companies banding together
You could probably get a bigger group if there wasn't a patent vs. no patent conflict.
> And eventually the patents expire, and the world gets these codecs free to use.
How many of these techniques wouldn't have been reinvented a dozen times over in less time, if the original patenter never existed?
>How many of these techniques wouldn't have been reinvented a dozen times over in less time, if the original patenter never existed?
In the exact same details? None. There is an extreme number of parameters and details in each of these codecs.
How many times have you or a small band of people invented a world class video compression codec? They take a huge number of people and a lot of funding to create. That funding, in many cases, was paid for by places like Fraunhofer that has invented many such codecs before anyone else, and they do so because they know they will get a revenue stream to pay for the work.
If these things were easy to make, there would be constant new ones, each improving the last one. This is not the case - they take years and the effort of a lot of people to make new ones that are better enough to slowly replace old ones.
A simple way to see it - if these were so easy that people would invent a dozen if only patents didn't stop them, then people would invent those anyway and publish them for fun. I am unaware of ANY world class video codecs that some hacker invented in their basement, and the only ones close to best in breed were funded to the tune of the prices above by huge companies that want to use their leverage to move markets, for their own gain.
These things are in no way trivial to create.
> You could probably get a bigger group if there wasn't a patent vs. no patent conflict.
Removing a funding channel means there would be less, not more, incentive to pay tens of millions to billions to develop one. If you pay the R&D and anyone else can simply take the work and not pay for it, then less companies will work on it.
In the exact same details? None. There is an extreme number of parameters and details in each of these codecs.
How many times have you or a small band of people invented a world class video compression codec? They take a huge number of people and a lot of funding to create. That funding, in many cases, was paid for by places like Fraunhofer that has invented many such codecs before anyone else, and they do so because they know they will get a revenue stream to pay for the work.
If these things were easy to make, there would be constant new ones, each improving the last one. This is not the case - they take years and the effort of a lot of people to make new ones that are better enough to slowly replace old ones.
A simple way to see it - if these were so easy that people would invent a dozen if only patents didn't stop them, then people would invent those anyway and publish them for fun. I am unaware of ANY world class video codecs that some hacker invented in their basement, and the only ones close to best in breed were funded to the tune of the prices above by huge companies that want to use their leverage to move markets, for their own gain.
These things are in no way trivial to create.
> You could probably get a bigger group if there wasn't a patent vs. no patent conflict.
Removing a funding channel means there would be less, not more, incentive to pay tens of millions to billions to develop one. If you pay the R&D and anyone else can simply take the work and not pay for it, then less companies will work on it.
> In the exact same details? None. There is an extreme number of parameters and details in each of these codecs.
The patent doesn't cover the extreme number of parameters and details, or someone would just change a couple and have a nice patent-free codec. Far more general techniques get snapped up by a single owner.
> If these things were easy to make, there would be constant new ones, each improving the last one. This is not the case - they take years and the effort of a lot of people to make new ones that are better enough to slowly replace old ones.
That's because it takes so long to get implemented, and it's a big cost to re-encode.
If it was just a matter of adding support to a couple programs and sending out an update, you'd see incremental improvements all the time.
We're largely still stuck on jpg and png, for crying out loud, and those use thoroughly obsolete methods.
> Removing a funding channel means there would be less, not more, incentive to pay tens of millions to billions to develop one. If you pay the R&D and anyone else can simply take the work and not pay for it, then less companies will work on it.
There would be less total incentive, but there's still the enormous incentive from everyone working with video to get better compression/quality, and streamlining cooperation would be a big deal.
The patent doesn't cover the extreme number of parameters and details, or someone would just change a couple and have a nice patent-free codec. Far more general techniques get snapped up by a single owner.
> If these things were easy to make, there would be constant new ones, each improving the last one. This is not the case - they take years and the effort of a lot of people to make new ones that are better enough to slowly replace old ones.
That's because it takes so long to get implemented, and it's a big cost to re-encode.
If it was just a matter of adding support to a couple programs and sending out an update, you'd see incremental improvements all the time.
We're largely still stuck on jpg and png, for crying out loud, and those use thoroughly obsolete methods.
> Removing a funding channel means there would be less, not more, incentive to pay tens of millions to billions to develop one. If you pay the R&D and anyone else can simply take the work and not pay for it, then less companies will work on it.
There would be less total incentive, but there's still the enormous incentive from everyone working with video to get better compression/quality, and streamlining cooperation would be a big deal.
>get snapped up by a single owner.
If these things were as trivial as you think, then why doesn't open source simply patent all the good new ideas now?
Even if these easy to create ideas you think exist required some previous work, at least these new simple patents would free up stuff eventually.
But this doesn't happen. Ask yourself why.
Heck, spend some time and try to make a better method your self, and patent it. Then you can hold for-fee patents needing your brilliant idea hostage, and change the playing field.
Or maybe you're mistaken on how hard and non-trivial the needed components are?
And, if they're so non-trivial that open source, or Google, or whomever want them to be free, then the ideas are worth patents.
>That's because it takes so long to get implemented,
You think it takes longer to implement a compression codec than to create it? That's not true for this scale (I've worked on several high end compression codecs and have designed and implemented probably a few dozen compression codes in my career).
Creating one is by far the hardest part - trying tons and tons of ideas (coding them all up for testing), looking for new ideas, working on how various ideas interact, etc.
By the time the spec is solidified probably a few hundred throwaways have been made. Implementation is trivial, then optimization takes some time.
But neither effort, by far, approach the effort it takes to design one on this scale.
> There would be less total incentive, but there's still the enormous incentive from everyone working with video to get better compression/quality, and streamlining cooperation would be a big deal.
If that were true, you'd see all those same actors improving x264 compression, or MP3, or h265, and other compression codecs, but they are not. Improvements, despite being something ANY person can do and push to open libs, is rare. Most all modern compression schemes only define the decoding part, allowing improvements on the encoding side to be continually worked on. The technical skill to do so is non-trivial, and a tiny, tiny fraction of a percent of "everyone working with video" has any idea how to improve it.
This is how x264 kept improving and improving over and over for H.264 compression - there was pretty much one person, Dark Shikari, that worked on it for years and years to improve it.
If you want to see how terribly complex this work is, read his (internet archive) blog on the work. It was fun at the time watching one person make x264 the premier H.264 compressor.
But pretty much zero other besides him made significant improvements to it, despite being something anyone can work on.
So no, the vast majority of people working with video would be useless at making anything better.
If these things were as trivial as you think, then why doesn't open source simply patent all the good new ideas now?
Even if these easy to create ideas you think exist required some previous work, at least these new simple patents would free up stuff eventually.
But this doesn't happen. Ask yourself why.
Heck, spend some time and try to make a better method your self, and patent it. Then you can hold for-fee patents needing your brilliant idea hostage, and change the playing field.
Or maybe you're mistaken on how hard and non-trivial the needed components are?
And, if they're so non-trivial that open source, or Google, or whomever want them to be free, then the ideas are worth patents.
>That's because it takes so long to get implemented,
You think it takes longer to implement a compression codec than to create it? That's not true for this scale (I've worked on several high end compression codecs and have designed and implemented probably a few dozen compression codes in my career).
Creating one is by far the hardest part - trying tons and tons of ideas (coding them all up for testing), looking for new ideas, working on how various ideas interact, etc.
By the time the spec is solidified probably a few hundred throwaways have been made. Implementation is trivial, then optimization takes some time.
But neither effort, by far, approach the effort it takes to design one on this scale.
> There would be less total incentive, but there's still the enormous incentive from everyone working with video to get better compression/quality, and streamlining cooperation would be a big deal.
If that were true, you'd see all those same actors improving x264 compression, or MP3, or h265, and other compression codecs, but they are not. Improvements, despite being something ANY person can do and push to open libs, is rare. Most all modern compression schemes only define the decoding part, allowing improvements on the encoding side to be continually worked on. The technical skill to do so is non-trivial, and a tiny, tiny fraction of a percent of "everyone working with video" has any idea how to improve it.
This is how x264 kept improving and improving over and over for H.264 compression - there was pretty much one person, Dark Shikari, that worked on it for years and years to improve it.
If you want to see how terribly complex this work is, read his (internet archive) blog on the work. It was fun at the time watching one person make x264 the premier H.264 compressor.
But pretty much zero other besides him made significant improvements to it, despite being something anyone can work on.
So no, the vast majority of people working with video would be useless at making anything better.
> If these things were as trivial as you think, then why doesn't open source simply patent all the good new ideas now?[...]But this doesn't happen. Ask yourself why.
Because they didn't find the idea first. I'm not sure what your argument is here?
Lots of patents do go to open source projects. Is "this" a world where open source finds everything first? If so I'm not sure how you got the idea that I was arguing that.
> And, if they're so non-trivial that open source, or Google, or whomever want them to be free, then the ideas are worth patents.
The fact that a patent could be worth money doesn't imply that it should be granted. If I could convince the government to grant me any patent I wanted, I would go patent every word in the dictionary.
> You think it takes longer to implement a compression codec than to create it?
Most codecs aren't made from scratch. If you got rid of the network effects, there definitely would be "constant new ones, each improving the last one".
In a world where implementations have to be nailed down and then it takes years and years to roll out, then creating a codec takes a very long time.
If implementation was as easy as normal software, you'd see organizations that put out a new codec every couple months, featuring the latest ideas and tweaks.
> stuff about compression
I'm talking about the people working on h.265 and AV1. They're already working on codecs, so I know they can work on codecs. If a bunch of people are useless at working on compression, well that sucks but it's a sideline to my argument.
Because they didn't find the idea first. I'm not sure what your argument is here?
Lots of patents do go to open source projects. Is "this" a world where open source finds everything first? If so I'm not sure how you got the idea that I was arguing that.
> And, if they're so non-trivial that open source, or Google, or whomever want them to be free, then the ideas are worth patents.
The fact that a patent could be worth money doesn't imply that it should be granted. If I could convince the government to grant me any patent I wanted, I would go patent every word in the dictionary.
> You think it takes longer to implement a compression codec than to create it?
Most codecs aren't made from scratch. If you got rid of the network effects, there definitely would be "constant new ones, each improving the last one".
In a world where implementations have to be nailed down and then it takes years and years to roll out, then creating a codec takes a very long time.
If implementation was as easy as normal software, you'd see organizations that put out a new codec every couple months, featuring the latest ideas and tweaks.
> stuff about compression
I'm talking about the people working on h.265 and AV1. They're already working on codecs, so I know they can work on codecs. If a bunch of people are useless at working on compression, well that sucks but it's a sideline to my argument.
> Algorithms are not simply math
I don't know about "simply", but algorithms are mathematical. I know so because everyone who matters in the field of algorithms says so (e.g. Knuth's "every algorithm is as mathematical as anything could be.")
Knuth adds: "An algorithm is an abstract concept unrelated to physical laws of the universe." so "in the same way mechanical engineering is not simply math" doesn't hold, or have even much bearing on the truth of the statement "Algorithms are not simply math."
> And if it were simply math, then there would be no way to work around those patents
How does this follow? It seems to be an implication out of nowhere instead of an argument.
> most open source codes are ... since
Another assertion presented as an implication. Following the same line of reasoning one could state: "Open source is not as good at producing kernels, mostly since proprietary vendors have a massively bigger pool of engineering" which by now has been shown to be false, in other words: a post hoc fallacy
I don't know about "simply", but algorithms are mathematical. I know so because everyone who matters in the field of algorithms says so (e.g. Knuth's "every algorithm is as mathematical as anything could be.")
Knuth adds: "An algorithm is an abstract concept unrelated to physical laws of the universe." so "in the same way mechanical engineering is not simply math" doesn't hold, or have even much bearing on the truth of the statement "Algorithms are not simply math."
> And if it were simply math, then there would be no way to work around those patents
How does this follow? It seems to be an implication out of nowhere instead of an argument.
> most open source codes are ... since
Another assertion presented as an implication. Following the same line of reasoning one could state: "Open source is not as good at producing kernels, mostly since proprietary vendors have a massively bigger pool of engineering" which by now has been shown to be false, in other words: a post hoc fallacy
> Algorithms are not simply math, in the same way mechanical engineering is not simply math.
Algorithms are _literally_ mathematical objects. C.f. Church-Turing thesis.
> And if it were simply math, then there would be no way to work around those patents, which plenty of open source codecs have done in various directions.
Show me an H.264 codec that works around the patents in the MPEG LA pool. Also software patents are orthogonal to the question of "open source" (which is about copyright). That there is very often no way to work around the patents in an otherwise independent implementation is precisely the problem with Software patents.
> Unfortunately, most open source codecs are not as good at compression, mostly since the patented ones have a massively bigger pool of engineering behind them from dozens of companies banding together their work to develop best of breed solutions.
Show me the numbers. Because the numbers I know (for example the chart in the blogpost itself) show that x265 is the best codec (or encoder, if it's not a decoder too) for H.265, it self being "open source", better than proprietary alternatives.
> And eventually the patents expire, and the world gets these codecs free to use.
Even after patents expire, copyright expires in 90 years.
Algorithms are _literally_ mathematical objects. C.f. Church-Turing thesis.
> And if it were simply math, then there would be no way to work around those patents, which plenty of open source codecs have done in various directions.
Show me an H.264 codec that works around the patents in the MPEG LA pool. Also software patents are orthogonal to the question of "open source" (which is about copyright). That there is very often no way to work around the patents in an otherwise independent implementation is precisely the problem with Software patents.
> Unfortunately, most open source codecs are not as good at compression, mostly since the patented ones have a massively bigger pool of engineering behind them from dozens of companies banding together their work to develop best of breed solutions.
Show me the numbers. Because the numbers I know (for example the chart in the blogpost itself) show that x265 is the best codec (or encoder, if it's not a decoder too) for H.265, it self being "open source", better than proprietary alternatives.
> And eventually the patents expire, and the world gets these codecs free to use.
Even after patents expire, copyright expires in 90 years.
>Algorithms are _literally_ mathematical objects. C.f. Church-Turing thesis.
The Church-Turing thesis is about computability, and has precisely zero to do with if QuickSort or x264 is math.
Not a single machine on the planet is an actual Turing machine. So in practice the Church-Turing thesis applies to zero actual computers on earth. They don't fit the definition. So throwing out fancy sounding phrases does not help your argument.
As a simple example, theoretically the Halting problem is undecidable on a Turing machine. But on any finite machine the Halting problem is decidable. All physical machines are finite. Thus every physical machine is not a Turing Machine.
You might as well argue that reality is based on physics, and physics is based on math, so nothing can be patented. But reality, and especially law, works differently than that.
The fact remains that algorithms under certain conditions can be patented, precisely because it is engineering, and because some algorithms require massive investment to develop, and it's worthwhile to inspire companies to invest on the R&D.
>> And eventually the patents expire, and the world gets these codecs free to use.
> Even after patents expire, copyright expires in 90 years.
I didn't say the sourcecode was suddenly free to use. I said the codec, as in the algorithm (which is what can be patented - code cannot), is free once the patent expires.
Conflating the two is not useles s -they're orthogonal. I can write code for a non-patented algorithm, and you cannot just take it because that code is automatically copyrighted.
But we were discussing patents.
>Show me the numbers....
You keep confusing codec to mean a particular implementation of a codec. It does not. H.265 is a codec. x265 is an implementation of the codec. MP3 is a a codec. LAME is an implementation. Codecs can be patented, but not copyrighted. Implementations can be copyrighted, but not patented, but they can use patented ideas.
So yes, commercial codecs generally are better than open source. H.264, H.265 were/are both better than any codec released around the same time. MP3 was the same. JPEG was the same.
So you're agreeing with me - H265 is the best codec.
>Show me the numbers.
Here you are: http://compression.ru/
The Church-Turing thesis is about computability, and has precisely zero to do with if QuickSort or x264 is math.
Not a single machine on the planet is an actual Turing machine. So in practice the Church-Turing thesis applies to zero actual computers on earth. They don't fit the definition. So throwing out fancy sounding phrases does not help your argument.
As a simple example, theoretically the Halting problem is undecidable on a Turing machine. But on any finite machine the Halting problem is decidable. All physical machines are finite. Thus every physical machine is not a Turing Machine.
You might as well argue that reality is based on physics, and physics is based on math, so nothing can be patented. But reality, and especially law, works differently than that.
The fact remains that algorithms under certain conditions can be patented, precisely because it is engineering, and because some algorithms require massive investment to develop, and it's worthwhile to inspire companies to invest on the R&D.
>> And eventually the patents expire, and the world gets these codecs free to use.
> Even after patents expire, copyright expires in 90 years.
I didn't say the sourcecode was suddenly free to use. I said the codec, as in the algorithm (which is what can be patented - code cannot), is free once the patent expires.
Conflating the two is not useles s -they're orthogonal. I can write code for a non-patented algorithm, and you cannot just take it because that code is automatically copyrighted.
But we were discussing patents.
>Show me the numbers....
You keep confusing codec to mean a particular implementation of a codec. It does not. H.265 is a codec. x265 is an implementation of the codec. MP3 is a a codec. LAME is an implementation. Codecs can be patented, but not copyrighted. Implementations can be copyrighted, but not patented, but they can use patented ideas.
So yes, commercial codecs generally are better than open source. H.264, H.265 were/are both better than any codec released around the same time. MP3 was the same. JPEG was the same.
So you're agreeing with me - H265 is the best codec.
>Show me the numbers.
Here you are: http://compression.ru/
So are companies in New Zealand not bound by H264 licensing fees? They banned software patents years ago.
They probably don't have to pay for any en/decoders distributed in New Zealand. If they want to distribute anywhere else they have to comply with that place's patent law (as well as all the other laws).
Really? I wasn't aware of that.
French companies are not bound. This is why VLC and other video players are developed by french devs.
Is this true? Software patent is a gray area in the EU and I would not recommend to take for granted that European companies do not have to pay the h.264 license, even if they only distribute in Europe.
Concerning VLC, their FAQ [1] does not mention the EU aspect: they say that they cannot pay for the license because the software is free and that "the end-user becomes responsible for complying with the licensing and royalty requirements" (whether this would be defensible in court is debatable)
[1] https://wiki.videolan.org/Frequently_Asked_Questions/#What_a...
Concerning VLC, their FAQ [1] does not mention the EU aspect: they say that they cannot pay for the license because the software is free and that "the end-user becomes responsible for complying with the licensing and royalty requirements" (whether this would be defensible in court is debatable)
[1] https://wiki.videolan.org/Frequently_Asked_Questions/#What_a...
According to VideoLan's Legal team (in the French section of https://www.videolan.org/legal.html under Brevets):
Patents
According to article L 611-10 of the French Intellectual Property Code, software patents are not allowed under French law. At the [EEA] level, Article 52 of the European Patent Convention has also excluded the protection of computer programs.
As a result, any software supplied by VideoLAN is not subject to licenses on any software patent, regardless of its origin.
The relevant section of the French IP law states that:
1. New inventions, in all fields of technology, involving an innovative approach with industrial application are patentable.
2. The following are not considered as inventions within the meaning of the first Section of this Article in particular: a) The discoveries of scientific theories as well as mathematical methods; b) Aesthetic creations; c) Intellectual plans, principles and methods in the areas of sport or economic activities, as well as computer programs; d) Presentation of information.
3. The provisions of Section 2 of this Article shall exclude the patentability of the elements listed in the said provisions only to the extent that the patent application or patent concerns only one of these elements considered as such.
4. Subject to the provisions of Articles L. 611-16 to L. 611-19, inventions relating to a product consisting wholly or partly of biological material, or to a process for producing, processing or using biological material, shall be patentable under the conditions laid down in Section 1. Biological material is considered to be material that contains genetic information and can reproduce itself or be reproduced in a biological system.
Interpret that as you wish.
Patents
According to article L 611-10 of the French Intellectual Property Code, software patents are not allowed under French law. At the [EEA] level, Article 52 of the European Patent Convention has also excluded the protection of computer programs.
As a result, any software supplied by VideoLAN is not subject to licenses on any software patent, regardless of its origin.
The relevant section of the French IP law states that:
1. New inventions, in all fields of technology, involving an innovative approach with industrial application are patentable.
2. The following are not considered as inventions within the meaning of the first Section of this Article in particular: a) The discoveries of scientific theories as well as mathematical methods; b) Aesthetic creations; c) Intellectual plans, principles and methods in the areas of sport or economic activities, as well as computer programs; d) Presentation of information.
3. The provisions of Section 2 of this Article shall exclude the patentability of the elements listed in the said provisions only to the extent that the patent application or patent concerns only one of these elements considered as such.
4. Subject to the provisions of Articles L. 611-16 to L. 611-19, inventions relating to a product consisting wholly or partly of biological material, or to a process for producing, processing or using biological material, shall be patentable under the conditions laid down in Section 1. Biological material is considered to be material that contains genetic information and can reproduce itself or be reproduced in a biological system.
Interpret that as you wish.
> the end-user becomes responsible for complying with the licensing and royalty requirements
Exactly. If you're using h.264 without a license in a country that requires it, it doesn't matter that some french people wrote the software for you. That's all they're saying.
Exactly. If you're using h.264 without a license in a country that requires it, it doesn't matter that some french people wrote the software for you. That's all they're saying.
Is CocaCola a patent on chemistry?
Coca-Cola is a trade secret. No one has figured out their recipe to date.
The recipe is not patented. This would force them to disclose it. The same goed for WD40. It's not patented because they don't want to disclose the formula.
So with Coca-cola, the ingredients are fully known, it's just the process that's secret?
Also, perhaps, difficult to come by:
"...Though the company removed cocaine from the carbonated concoction over 100 years ago, coca leaves are actually still used to flavor Coke. The soda brand has an exemption with the government allowing them to import coca leaves into a decocanization plant in New Jersey where cocaine is removed so the leaves can be used in Coca-Cola for their natural flavors."
https://www.foodandwine.com/drinks/55-million-worth-cocaine-...
"...Though the company removed cocaine from the carbonated concoction over 100 years ago, coca leaves are actually still used to flavor Coke. The soda brand has an exemption with the government allowing them to import coca leaves into a decocanization plant in New Jersey where cocaine is removed so the leaves can be used in Coca-Cola for their natural flavors."
https://www.foodandwine.com/drinks/55-million-worth-cocaine-...
The ingredients for the Coca-Cola flavor are listed as "natural flavors". That's the extent of disclosure.
Trade secret just means that they don't share the recipe and make efforts to keep it secret. Calling it a "trade secret" is really a marketing strategy as much as anything else. Just like the rumors that coca leaf extract are an ingredient... something that is "neither confirmed nor denied". A marketing strategy.
I think that it's basically flavored with vanilla, cinnamon, a few essential oils (orange, lemon, lavender), and some spices (nutmeg!). One you start adding that many flavors to a recipe, the exact flavors are no longer important, and I'm sure you could replicate something similar at home if you were willing to do enough experimentation. There are a few recipes online that you can use.
Trade secret just means that they don't share the recipe and make efforts to keep it secret. Calling it a "trade secret" is really a marketing strategy as much as anything else. Just like the rumors that coca leaf extract are an ingredient... something that is "neither confirmed nor denied". A marketing strategy.
I think that it's basically flavored with vanilla, cinnamon, a few essential oils (orange, lemon, lavender), and some spices (nutmeg!). One you start adding that many flavors to a recipe, the exact flavors are no longer important, and I'm sure you could replicate something similar at home if you were willing to do enough experimentation. There are a few recipes online that you can use.
A trade secret is a type of intellectual property and has a place in law.
> No one has figured out their recipe to date
> So ... the ingredients are fully known ... ?
No... because nobody has figured out their recipe. They just said that.
https://en.wikipedia.org/wiki/Coca-Cola_formula
> So ... the ingredients are fully known ... ?
No... because nobody has figured out their recipe. They just said that.
https://en.wikipedia.org/wiki/Coca-Cola_formula
Also the proportions. While (in the US at least) manufacturers are required to list their ingredients in the order of greatest proportion to least proportion, they are not required to disclose what those proportions are, which also adds some difficulty to reverse-engineering the recipe.
I think the biggest catch is being able to throw potentially thousands of different substances under the generic “natural flavorings.” Read the Coca Cola ingredients list and you certainly don’t see “coca leaf extract” on it.
Luckily, it seems like the industry is coming along with AV1. I'm not aware of any major companies opposed to AV1.
There's going to be a lag time before enough HW is being used that fully supports it but it'll happen eventually.
There's going to be a lag time before enough HW is being used that fully supports it but it'll happen eventually.
Sisvel AV1 patent pool: https://www.sisvel.com/licensing-programs/audio-and-video-co...
Sisvel isn't directly named in this statement but it's clearly targeted at them: http://aomedia.org/press%20releases/the-alliance-for-open-me...
I trust the AOM will handle this the moment Sisvel actually attempts to file suit. If their patents are as solid as they're claiming, they probably would have sued someone like Netflix for using AV1 already. IIRC, Sisvel did this for both VP8 and VP9 and it didn't end up going anywhere.
I trust the AOM will handle this the moment Sisvel actually attempts to file suit. If their patents are as solid as they're claiming, they probably would have sued someone like Netflix for using AV1 already. IIRC, Sisvel did this for both VP8 and VP9 and it didn't end up going anywhere.
AOMedia claims that's not valid, and will defend you against potential claims: http://aomedia.org/press%20releases/the-alliance-for-open-me...
Is it just my impression that pretty much all sites except youtube/fb/twitter have difficulties with video?
A lot of times video struggles to start, buffers for a long time, stutters, .... I'm not talking about small blogs, but about sites like CNN, WaPo, ...
A lot of times video struggles to start, buffers for a long time, stutters, .... I'm not talking about small blogs, but about sites like CNN, WaPo, ...
It has to do with not having edge nodes. In theory, Cloudflare is in a good position to address this disparity…
My team agrees with you :)
https://developers.cloudflare.com/stream/
Happy to take any critique y'all may have.
https://developers.cloudflare.com/stream/
Happy to take any critique y'all may have.
You skipped the most popular video serving platforms - pr0n companies. Also no problems there, everything works smoothly.
Yes. Not only that but fast forward, backwards, skipping. Crash with Javascript. High CPU loading because some other JS is messing with the Video in a way that shouldn't be. etc etc.
Someone on HN asked me what's wrong with HTML5 video. I forgot to reply as I was gathering examples and informations.
Someone on HN asked me what's wrong with HTML5 video. I forgot to reply as I was gathering examples and informations.
News sites are notoriously resource-heavy. Small blogs have no issue with embedded <video> tag videos.
AV1 have other costs, AFAIK it's so slow to encode that encoding farm cost become much more significant.
For now. When h.264 was new, I remember my dual G5 Mac taking 2-3 times the duration of a 1080p video to encode. Today, my phone can encode video of roughly equal resolution in near real time. It would be unsurprising to see the next generations of processors include optimizations for AV1.
Could that be in part because newer iPhones perform H264 decoding on the GPU?
IIRC there's usually a dedicated piece of hardware for h264 encoding in most phones now (not the gpu per se)
Just realized they were taking about encoding anyway. I’m not sure if that’s usually performed on the same chipset. I don’t know if iPhones have a dedicated chipset for H264, I would be a bit surprised. I could probably look it up later.
> I don’t know if iPhones have a dedicated chipset for H264,
Yes it does, both H.264 ( AVC ) and H.265 ( HEVC ).
But that doesn't really matter because your ( initial ) point still stands, real time H.264 encoding isn't taxing on an A14 anymore. Remember H.264's work started before 2000 and finalised in 2003. That is like playing Doom 3 on your iPhone. No longer technically taxing by any means even if you are playing it at native resolution.
However the industry is well aware of the slow down [1] of Moore's law. The cost increase as well as TDP headroom. So thinking we could continue to throw hardware at the problem simply doesn't work.
[1] Moore's Law actually includes a time frame within it, so it would not be wrong to say the law is "dead" considering it has slowed down way beyond its time limit within margin of error. But modern day's media and computer PR tends to use Moore's law as word for transistor improvement.
Yes it does, both H.264 ( AVC ) and H.265 ( HEVC ).
But that doesn't really matter because your ( initial ) point still stands, real time H.264 encoding isn't taxing on an A14 anymore. Remember H.264's work started before 2000 and finalised in 2003. That is like playing Doom 3 on your iPhone. No longer technically taxing by any means even if you are playing it at native resolution.
However the industry is well aware of the slow down [1] of Moore's law. The cost increase as well as TDP headroom. So thinking we could continue to throw hardware at the problem simply doesn't work.
[1] Moore's Law actually includes a time frame within it, so it would not be wrong to say the law is "dead" considering it has slowed down way beyond its time limit within margin of error. But modern day's media and computer PR tends to use Moore's law as word for transistor improvement.
Okay, thanks. Just got a chance to look it up and indeed, beginning with the Apple A9 processor, there are H.264 and H.265 hardware decoders.
Its going to see adoption is widely distributed content first - its why Google and Netflix are so gung-ho on it, they see the same file downloaded millions of times and in those circumstances bandwidth savings are huge over the encode times.
Eventually the ubiquity of hardware acceleration plus Moore's Law should make even lackluster hardware encoders generally available so average joes can use AV1 without needing to wait a day for a half hour of footage.
Eventually the ubiquity of hardware acceleration plus Moore's Law should make even lackluster hardware encoders generally available so average joes can use AV1 without needing to wait a day for a half hour of footage.
Netflix decided AV1 was worth it:
https://netflixtechblog.com/netflix-now-streaming-av1-on-and...
https://netflixtechblog.com/svt-av1-an-open-source-av1-encod...
https://netflixtechblog.com/netflix-now-streaming-av1-on-and...
https://netflixtechblog.com/svt-av1-an-open-source-av1-encod...
Netflix surely spends less compute power encoding videos than sites like YouTube though, right? Netflix skews more towards fewer videos, higher view counts, more money per video.
YouTube rolled out AV1 support quite some time ago. Many YouTube videos are AV1 encoded. Here's one example:
https://www.youtube.com/watch?v=k3Yk6FrjMkQ
You can tell if it's playing back in AV1 in your browser by right-clicking on the video and selecting "Stats for Nerds" to find out the video and the audio codecs. On my system in Firefox it's AV1 for video and Opus for audio.
For that video the file size savings are a bit hit and miss. AV1 saves a lot versus H.264 at 1080p, saves a little bit at 720p and 480p, and saves a bit more at 360p.
The 1080p results for that video are:
- AV1 50.28MiB
- VP9 69.12MiB
- H.264 106.68MiB
So a clear win at 1080p for AV1 versus VP9 and H.264.
Sometimes YouTube's AV1 encodes can actually end up being bigger than the H.264 versions due to the vagaries of automated transcoding.
When AV1 support was newly added to Firefox I used this 1080p60 video as a benchmark for my system:
https://www.youtube.com/watch?v=Fmdb-KmlzD8
Over the course of the development of Firefox's AV1 support and the switch to the dav1d decoder (https://code.videolan.org/videolan/dav1d) that video has gone from being unwatchable due to too many dropped frames to now playing back smoothly.
My system still struggles with AV1 above 1080p in Firefox though.
https://www.youtube.com/watch?v=k3Yk6FrjMkQ
You can tell if it's playing back in AV1 in your browser by right-clicking on the video and selecting "Stats for Nerds" to find out the video and the audio codecs. On my system in Firefox it's AV1 for video and Opus for audio.
For that video the file size savings are a bit hit and miss. AV1 saves a lot versus H.264 at 1080p, saves a little bit at 720p and 480p, and saves a bit more at 360p.
The 1080p results for that video are:
- AV1 50.28MiB
- VP9 69.12MiB
- H.264 106.68MiB
So a clear win at 1080p for AV1 versus VP9 and H.264.
Sometimes YouTube's AV1 encodes can actually end up being bigger than the H.264 versions due to the vagaries of automated transcoding.
When AV1 support was newly added to Firefox I used this 1080p60 video as a benchmark for my system:
https://www.youtube.com/watch?v=Fmdb-KmlzD8
Over the course of the development of Firefox's AV1 support and the switch to the dav1d decoder (https://code.videolan.org/videolan/dav1d) that video has gone from being unwatchable due to too many dropped frames to now playing back smoothly.
My system still struggles with AV1 above 1080p in Firefox though.
I’m interested in the relative CPU costs of encoding, so this doesn’t really address that.
When you say “many” videos are AV1-encoded, it makes me wonder why other videos aren’t AV1-encoded, if supported. Presumably there is some cost/benefit analysis at Google to decide which formats to use, and it is decided that a particular video should not be encoded using AV1 due to the resulting impact on disk/CPU/network usage (more disk, more CPU, less network egress).
I also suppose YouTube has a massive pool of low-priority batch services for transcoding and can do last-minute encoding for unpopular videos / unpopular formats, so YouTube’s tradeoff will not be the same as your tradeoff. I just wonder what the tradeoff is.
When you say “many” videos are AV1-encoded, it makes me wonder why other videos aren’t AV1-encoded, if supported. Presumably there is some cost/benefit analysis at Google to decide which formats to use, and it is decided that a particular video should not be encoded using AV1 due to the resulting impact on disk/CPU/network usage (more disk, more CPU, less network egress).
I also suppose YouTube has a massive pool of low-priority batch services for transcoding and can do last-minute encoding for unpopular videos / unpopular formats, so YouTube’s tradeoff will not be the same as your tradeoff. I just wonder what the tradeoff is.
> When you say “many” videos are AV1-encoded, it makes me wonder why other videos aren’t AV1-encoded
They will be eventually. The priority goes to the videos with the highest trending view counts because that's where the bandwidth savings are going to be the most significant. This video is five days old and has an AV1 encode:
https://www.youtube.com/watch?v=j8oD9jlCFJw
Whereas this video which has been there for nearly 7 years does not yet have an AV1 encode:
https://www.youtube.com/watch?v=Dqf1BdsV5WE
With only 7,777 views it doesn't matter that much if it has an AV1 encode (or even a VP9 encode really). The bandwidth savings won't be that significant for the number of views it gets.
They will be eventually. The priority goes to the videos with the highest trending view counts because that's where the bandwidth savings are going to be the most significant. This video is five days old and has an AV1 encode:
https://www.youtube.com/watch?v=j8oD9jlCFJw
Whereas this video which has been there for nearly 7 years does not yet have an AV1 encode:
https://www.youtube.com/watch?v=Dqf1BdsV5WE
With only 7,777 views it doesn't matter that much if it has an AV1 encode (or even a VP9 encode really). The bandwidth savings won't be that significant for the number of views it gets.
Also consider that all video have a H264 encoding to play on low end devices.
Any additional encoding, in VP9 or AV1, is an additional cost as the H264 encoding cannot be avoided.
Even VP9 is not used for all video. Only if the video is 4K or 2K7, or has a high number of view.
Any additional encoding, in VP9 or AV1, is an additional cost as the H264 encoding cannot be avoided.
Even VP9 is not used for all video. Only if the video is 4K or 2K7, or has a high number of view.
> Even VP9 is not used for all video. Only if the video is 4K or 2K7, or has a high number of view.
No, the low view count video example I provided above has VP9 encodes for all resolutions, 144p through to 1080p.
Lots of low view count videos are encoded in VP9. But when VP9 was new it followed the same approach AV1 is following now: high trending view count videos get the priority.
No, the low view count video example I provided above has VP9 encodes for all resolutions, 144p through to 1080p.
Lots of low view count videos are encoded in VP9. But when VP9 was new it followed the same approach AV1 is following now: high trending view count videos get the priority.
Those comparisons are meaningless without also factoring in some measure of quality such as VMAF. The advantages of AV1 become clear in that case.
Or you can just look at them with your eyes. The end goal of video is for it to be looked at, after all.
The quality of these different encodes is about the same. With different codecs at about the same quality it becomes a case of picking your errors and artefacts. A given codec will be weak in some areas and strong in others.
The quality of these different encodes is about the same. With different codecs at about the same quality it becomes a case of picking your errors and artefacts. A given codec will be weak in some areas and strong in others.
One of the weird parts of this is just how damn good x264 is. I have a bunch of videos on my website, and because I'm some kind of basement-dwelling cretin, I manually encode them with FFmpeg to MPEG-4 and WebM.
Encoding to MPEG-4: Easy, fast, high-quality.
Encoding to WebM: Much slower to encode, requires tweaking the parameters, video usually looks noticeably worse at similar bit rates.
I don't know how much of this is caused by the encoder, how much is caused by the codec, and how much is caused by poor selection of parameters on my part. I use "recommended settings" that I find online and H.254/HiP.
Encoding to MPEG-4: Easy, fast, high-quality.
Encoding to WebM: Much slower to encode, requires tweaking the parameters, video usually looks noticeably worse at similar bit rates.
I don't know how much of this is caused by the encoder, how much is caused by the codec, and how much is caused by poor selection of parameters on my part. I use "recommended settings" that I find online and H.254/HiP.
If the cost is great enough, why hasn't a new standard been adopted? Is it that there aren't better options, or is it the challenge in getting enough buy in onto one specific option for it to be of use and capable of being seen as a new standard? The article mentions some newer options that are better but cost more, but where is the OSS version? Surely major websites have the incentive to create one that they can then use for free.
Dark Shikari on x264 was like Mike Pall in LuaJIT.
x264 does indeed punch above its weight. Which has been, counterintuitively, a boon for codec freedom.
Nvidia DLSS shows impressive quality upscaling games up to 4x resolution. I wonder if same tech can be applied to video encoding?
What's the issue? Is it very hard making video codecs? Or is it a network effect issue? Why can't any one of these companies take a few months to make one alone and release it and be done with it?
It's both. Making video codecs is hard. Making your own video codec and then convincing video sites and browsers and hardware vendors to adopt it is even harder. Video sites won't adopt your custom codec because people can't play it; browsers aren't interested in supporting it because (a) there's no content (b) there's no hardware acceleration. That's why the AOMedia consortium (behind AV1) includes semiconductor firms, browser vendors, video producers, and video distributors.
Two of the biggest obstacles to deploying a new codec are avoiding patents on existing codecs, and getting hardware decode support added to important mobile platforms. Software decoding is basically a non-starter for battery-powered devices.
Video codecs, like any other software, ultimately do have to compete on merits other than cost. In this case, it might not be "hard" to make a video codec, but it is hard to make a codec that is competitive with anything of the leading codecs from the MPEG LA. Bear in mind that your new codec needs to likely be the best or leading in most of several hundred metrics: power efficiency of encode/decode, compressed size in transmission, video/audio quality measures of many sorts, flexibility of storage in various encapsulating containers, etc. etc. It's a rather long list, and all the while you have to dodge the efforts of many many patent trolls waiting to sue you over infringing their patent.
After having gone through that effort, the costs of it better ultimately justify the outcome, and the MPEG LA licensing rates are very carefully to get you to avoid just this outcome.
After having gone through that effort, the costs of it better ultimately justify the outcome, and the MPEG LA licensing rates are very carefully to get you to avoid just this outcome.
google created their own called webm, released it for use without royalties, simultaneously pushing it hard with Chrome and YouTube. Afaiu this was crucial in reaching a royalty settlement with h264 patent holders.
That's actually happened multiple times. There used to be a company called On2 which basically specialized in making proprietary, royalty-free design-arounds of other patented video codecs. Yes, those words sound confusing. The implementation was proprietary but the codec itself was designed to not bear a patent royalty. They made VP3 (later donated as Theora), VP6 (bought by Adobe to use as the fancy new Flash Player 8 video codec), and then got bought out by Google around the time they were making VP8.
Google basically had them continue what they were doing, but on a completely FOSS basis. They took VP8, bundled it with Ogg Vorbis and the Matroska container, and rebranded it as webm. They released VP9, and then started work on VP10 up until the whole H.265 licensing debacle happened. At this point all the holdouts for ITU/ISO standard codecs threw up their hands and decided to form a new standards consortium to build a new royalty-free video codec. Google, xiph, Cisco, and a few other companies basically put together some of their research projects and built AV1. This codec was more or less funded by their own research, and their business models are such that just having decoder ubiquity is worth more than patent royalties they could otherwise charge.
Video codec research is difficult, but mostly because it requires specialized knowledge of a lot of different related fields of mathematics and computer science. For various historical reasons, the way ITU (H.26x) and ISO (MPEG-n) paid for their research was by deliberately shipping patent-encumbered standards. Essentially, they just used whatever coding tools worked the best and assumed the economics would work out in favor of forming a patent pool under uniform-fee licensing rules. For example, despite the name, MPEG-LA has literally nothing to do with the ISO MPEG working group, or ITU VCEG, despite being the primary means by which their standards are licensed.
This sort of arrangement occasionally works out amazingly well (MPEG-4, H.264) or blows up spectacularly in their faces (H.263, H.265).
If you had the requisite knowledge I mentioned above, you probably could discover at least some small improvement upon the state-of-the-art for video coding techniques. However, I could not guarantee that it would be unencumbered by patents. Hell, MPEG even tried building a patent-free video codec of their own, but it failed completely because of some... rather interesting particularities of how standards bodies handle patent claims. You see, for various legal reasons, people proposing standards that they may have a patent interest in need to be very up front about what parts they will own. In ISO standards speak, these patent declarations come in three flavors:
Option 1, which means you agree to license your patents without a patent royalty (the AV1 patent model).
Option 2, or "FRAND" (Fair, Reasonable, And Non-Discriminatory), which means you agree to license patents for a royalty payment to any entity (the MPEG-LA patent model). (Note that RMS would prefer you call this "Uniform Fee Only" and not FRAND)
Option 3, which means you are refusing to license the patent under any terms suitable for a standard (the "get off my lawn" patent model).
An interesting peculiarity is that ISO will accept patent declarations under Option 1 or Option 2 that do not specifically name the patents potentially being infringed by the standard. This is not the case for Option 3; for obvious reasons. If a standards entity just accepted vague claims of infringement without any offer of licensing, then it would not be able to actually make standards. However, the acceptance of vague Option 2 claims meant that MPEG actually couldn't ascertain what part of their attempt at a royalty-free codec was infringing, so it was a standard dead in the water.
This problem isn't unique to MPEG either; vague claims of infringement are perfectly fine until you actually have to file a lawsuit, and at that point it's far too late to rewrite a standard. So if you do write your own video codec, you still have to have a team of lawyers examine everything about it against basically every patent even remotely relating to video codecs. You don't get the benefit of ISO-style patent disclosures either, as none of the patent owners are working with you and thus have no legal reason to tell you about the patents. Even if you managed to do the legal research yourself, it's entirely possible that another company had a patent application before you wrote your codec that you weren't aware of.
That is known as a "submarine patent" and it's why Microsoft and Apple were initially quite skeptical of VP8 and WebM. Having a large international standards process with basically the entire industry on-board means pretty much everyone with patents will need to disclose and license their patents (albeit only under Option 2). Having all that money into one large standard further creates economic incentives in favor of forming a patent pool. On2/Google going it alone means pretty much any patent claim will be some kind of submarine patent asserted against you far after you had any hope of changing the spec to avoid it. Google gets around this with lots of money and lawyers, which you don't have.
Google basically had them continue what they were doing, but on a completely FOSS basis. They took VP8, bundled it with Ogg Vorbis and the Matroska container, and rebranded it as webm. They released VP9, and then started work on VP10 up until the whole H.265 licensing debacle happened. At this point all the holdouts for ITU/ISO standard codecs threw up their hands and decided to form a new standards consortium to build a new royalty-free video codec. Google, xiph, Cisco, and a few other companies basically put together some of their research projects and built AV1. This codec was more or less funded by their own research, and their business models are such that just having decoder ubiquity is worth more than patent royalties they could otherwise charge.
Video codec research is difficult, but mostly because it requires specialized knowledge of a lot of different related fields of mathematics and computer science. For various historical reasons, the way ITU (H.26x) and ISO (MPEG-n) paid for their research was by deliberately shipping patent-encumbered standards. Essentially, they just used whatever coding tools worked the best and assumed the economics would work out in favor of forming a patent pool under uniform-fee licensing rules. For example, despite the name, MPEG-LA has literally nothing to do with the ISO MPEG working group, or ITU VCEG, despite being the primary means by which their standards are licensed.
This sort of arrangement occasionally works out amazingly well (MPEG-4, H.264) or blows up spectacularly in their faces (H.263, H.265).
If you had the requisite knowledge I mentioned above, you probably could discover at least some small improvement upon the state-of-the-art for video coding techniques. However, I could not guarantee that it would be unencumbered by patents. Hell, MPEG even tried building a patent-free video codec of their own, but it failed completely because of some... rather interesting particularities of how standards bodies handle patent claims. You see, for various legal reasons, people proposing standards that they may have a patent interest in need to be very up front about what parts they will own. In ISO standards speak, these patent declarations come in three flavors:
Option 1, which means you agree to license your patents without a patent royalty (the AV1 patent model).
Option 2, or "FRAND" (Fair, Reasonable, And Non-Discriminatory), which means you agree to license patents for a royalty payment to any entity (the MPEG-LA patent model). (Note that RMS would prefer you call this "Uniform Fee Only" and not FRAND)
Option 3, which means you are refusing to license the patent under any terms suitable for a standard (the "get off my lawn" patent model).
An interesting peculiarity is that ISO will accept patent declarations under Option 1 or Option 2 that do not specifically name the patents potentially being infringed by the standard. This is not the case for Option 3; for obvious reasons. If a standards entity just accepted vague claims of infringement without any offer of licensing, then it would not be able to actually make standards. However, the acceptance of vague Option 2 claims meant that MPEG actually couldn't ascertain what part of their attempt at a royalty-free codec was infringing, so it was a standard dead in the water.
This problem isn't unique to MPEG either; vague claims of infringement are perfectly fine until you actually have to file a lawsuit, and at that point it's far too late to rewrite a standard. So if you do write your own video codec, you still have to have a team of lawyers examine everything about it against basically every patent even remotely relating to video codecs. You don't get the benefit of ISO-style patent disclosures either, as none of the patent owners are working with you and thus have no legal reason to tell you about the patents. Even if you managed to do the legal research yourself, it's entirely possible that another company had a patent application before you wrote your codec that you weren't aware of.
That is known as a "submarine patent" and it's why Microsoft and Apple were initially quite skeptical of VP8 and WebM. Having a large international standards process with basically the entire industry on-board means pretty much everyone with patents will need to disclose and license their patents (albeit only under Option 2). Having all that money into one large standard further creates economic incentives in favor of forming a patent pool. On2/Google going it alone means pretty much any patent claim will be some kind of submarine patent asserted against you far after you had any hope of changing the spec to avoid it. Google gets around this with lots of money and lawyers, which you don't have.
Wow that article was a real eye opener..
It has always pissed me off that popular proprietary / walled garden type platforms like xbox, playstation, smartTVs, smartphones etc. cant play HEVC.
Since they are so popular it means that the most popular and most heavily seeded torrents of films and tv are always crappy x264 encoded.. so 5 or 6 GB for a 1080p movie instead of a nice lean 1GB HEVC file.
I used to think it was simply because those platforms are computers but crippled ie. you cant install codecs (or VLC for example) on them, you can only install whatever is available via the walled garden store.
But now I know that HEVC support is expensive so Sony, MS etc really dont want to support it.
> 5 or 6 GB for a 1080p movie instead of a nice lean 1GB HEVC file
Woah there, HEVC and VP9 are not that good. Both are 25-30% better than h264. See Netflix's benchmarks. We're talking 4gb vs 6gb
A 1gb movie will always look like crap, even with an even better codec like AV1. Pause the movie during an action sequence or explosion and it'll look like you're watching the video on YouTube with all the compression artifacts that entails.
Woah there, HEVC and VP9 are not that good. Both are 25-30% better than h264. See Netflix's benchmarks. We're talking 4gb vs 6gb
A 1gb movie will always look like crap, even with an even better codec like AV1. Pause the movie during an action sequence or explosion and it'll look like you're watching the video on YouTube with all the compression artifacts that entails.
Well, the point about lossy video compression is that you aren't supposed to pause the video…
I think it's important to understand that google wont even let you use any HEVC decoder thats already installed in your OS to decode video in their walled-garden browser. They just deliberately block this technology.
maybe growing up with PAL is hampering my expectations here, but imo 1GB/hour in h264 looks just fine whereas same size HEVEC, while having supperior contrast and sharpness, almost always has fucked up colors so i try to avoid it. also HVEC makes my fans spin up.
(2018).
Given the difficulty and investment involved to engineer and produce codecs - I'd argue those guys deserve every penny of their royalties.
I don't think anyone is saying otherwise. But that doesn't mean people can't come up with a free, better version.
I patently disagree. Why would anyone with the requisite skills create something this complex knowing they'd never be compensated?
Even if they attempted to leverage an open source "enterprise" licensing scheme big corps would likely just use an existing industry proven codec.
Even if they attempted to leverage an open source "enterprise" licensing scheme big corps would likely just use an existing industry proven codec.
Because they want to? Sometimes that's all it takes. Besides, the proposed solution is being developed by multiple corporations who will profit precisely from not having to pay licensing anymore.
MPEG-la did not create those codecs they just use the patents to make money. The people that actually made the codecs are long gone.
I respect the patent system and the justice system that backs it. Of course, it's not without issues - but there are agreements and investments those individuals made to hold said patents. Just because it's "not fair" isn't a reason to invalidate the whole system.
That being said - fuck patent trolls.
That being said - fuck patent trolls.
What kit has AV1 decode? Encode?
The latest gen Intel, Nvidia, and AMD cards have AV1 decode. No general-purpose cards with encode, though.
People would invent more codes if browsers would actually allow MSE.. Safari is today's gate keeper of many things. Just like Microsoft in 90s.
I honestly prefer Safari because the WebKit team deliberates before deciding whether to implement or not. This is evident by their status page.[0]
This has its pros and cons, and is up for debate (Eg. Web Bluetooth API[1], another example is how much time it took Apple to implement Google/YouTube's codec). I personally prefer their slow and steady pace as I feel the browser is lighter and more easy on the battery than Chrome and Chromium based browsers.
[0] https://webkit.org/status/
[1] https://bugs.webkit.org/show_bug.cgi?id=101034
This has its pros and cons, and is up for debate (Eg. Web Bluetooth API[1], another example is how much time it took Apple to implement Google/YouTube's codec). I personally prefer their slow and steady pace as I feel the browser is lighter and more easy on the battery than Chrome and Chromium based browsers.
[0] https://webkit.org/status/
[1] https://bugs.webkit.org/show_bug.cgi?id=101034
What do you mean? from that thread it seams like they have no intention of implementing web Bluetooth and that seams final.
I believe that was what I was trying to highlight by giving that example. Their strategy is not perfect.
This is one of the cons of letting one team decide about the future of the project without letting people challenge their decisions.
This is one of the cons of letting one team decide about the future of the project without letting people challenge their decisions.
What if streaming multimedia on the web was a mistake?
What's wrong with downloading media and viewing it in an external player, like VLC?
What's wrong with downloading media and viewing it in an external player, like VLC?
Companies dont want you to download it... They would much rather charge you every time you enjoy it.
If I'm not mistaken, these days (unlike in old days with RTSP)'streaming' videos are chunk-downloaded instead.
BTW, VLC can read videos while they are being downloaded just fine. It can also stream videos from, for instance, YouTube. Also, youtube-dl is your friend.
BTW, VLC can read videos while they are being downloaded just fine. It can also stream videos from, for instance, YouTube. Also, youtube-dl is your friend.
[deleted]
you dont even need to download it because most players can access an url just fine. mpv with ytdl integration is my goto for viewing sites that just wont give users access to the video-url (youtube/twitch/vimeo...)
[deleted]
djsumdog(3)
Mozilla, with their $50M annual budget, can't pay the fees.
Okay.
Okay.
Firefox has ~220M monthly active users [0], which would easily hit the current maximum royalty cap of $6.5M per year, so more than 10% of their yearly budget going to a single ancient codec.
[0]: https://data.firefox.com/dashboard/user-activity
[0]: https://data.firefox.com/dashboard/user-activity
They say in the post that it would cost almost $10M/year to license H.264, and that HEVC would cost ten times that. Do you expect them to put 20-200% of their budget into a single video codec?
[deleted]
I expect them to negotiate on rates like an adult. Mozilla is taking an activist role, and (to me at least) have been flaunting their incompetence for at least a decade on this topic.
The whole point of these patent pools is that they charge a uniform fee schedule to all users of a particular technology. There isn't much room for Mozilla to negotiate. MPEG-LA demands their pound of flesh.
HN would be up in arms and lobbyists would be running to congress and DOJ to get google and others in major trouble - but if google and apple just stopped supporting H.265 it would go away.
Block uploads and streaming on youtube of 265. No support in chrome.
This would never happen, Apple in particular is in premium / everything costs big $ space so often doesn't mind the fees. But a few big players would make a big difference if they took action.
Block uploads and streaming on youtube of 265. No support in chrome.
This would never happen, Apple in particular is in premium / everything costs big $ space so often doesn't mind the fees. But a few big players would make a big difference if they took action.
But the truth is that while Google might not _like_ paying the MPEG LA for these costs, it would be a foolish business strategy to do so. Obviously Google would rather pay zero dollars for a video codec, but if it means losing out on a profitable source of revenue, it's a no-brainer to just grumble and move on.
As a sibling comment mentions, Apple actually receives money for their role in the MPEG LA, but I doubt that this is anything more than a tiny dribble of money compared to the rest of their business. They're sitting just fine not having to pay out.
Not to mention that getting the entire Web ecosystem to adopt a different codec can take ages - new hardware has to support accelerating it, and software implementions need to be added to thousands of different ecosystems before it becomes worth using on the Web.
As a sibling comment mentions, Apple actually receives money for their role in the MPEG LA, but I doubt that this is anything more than a tiny dribble of money compared to the rest of their business. They're sitting just fine not having to pay out.
Not to mention that getting the entire Web ecosystem to adopt a different codec can take ages - new hardware has to support accelerating it, and software implementions need to be added to thousands of different ecosystems before it becomes worth using on the Web.
Apple is a member of the MPEG-LA patent pool, and receives a cut of the H.265 royalties. They support AV-1 only reluctantly, likely due to pressure from content providers like Netflix and YouTube.
IIRC, H.264 is one of a few non-Google technologies that is included in Chrome, but is not included in Chromium.
> So, how much does those rights cost? Under the terms in place for 2011-2015, the royalty rates are the same regardless of whether a product is part of an OS. There's no royalty for the first 100,000 units of a licensed product; sublicensees pay 20 cents per unit up to 5 million and 10 cents per unit above 5 million. The current agreement includes an annual limit: “The maximum annual royalty (‘cap’) for an Enterprise [is] $6.5 million per year in 2011-2015.”
What happens in 2016? Under the MPEG LA license, the terms are "renewable for successive five-year periods … on reasonable terms and conditions.” The per-copy royalty rates "will not increase by more than 10% at each renewal.” If rates go up by the maximum 10% for that five-year period, the cost of 10 million licenses will increase by $150,000 from a current rate of $1.48 million to $1.63 million.
There's a small gotcha in the new MPEG LA license agreement. Footnote 17, which appears in tiny print at the very end of the summary of license terms, notes that “Annual royalty caps are not subject to the 10% limitation.” Although that sounds like a loophole, it really only affects the largest players in the market.
1: https://www.zdnet.com/article/a-closer-look-at-the-costs-and....