Forging an Alliance for Royalty-Free Video(blog.mozilla.org)
blog.mozilla.org
Forging an Alliance for Royalty-Free Video
https://blog.mozilla.org/blog/2015/09/01/forging-an-alliance-for-royalty-free-video/
60 comments
This came up a few weeks ago.[1] MPEG-LA's patent portfolio isn't that strong any more. MPEG-LA has a long list of patents, but most of them have expired. The key patents on video compression involved motion estimation, and those expired this year. MPEG-LA got their twenty years of patent protection, and it's over.
The remaining patents are mostly on things you don't need for Internet video. US #6,181,712, has to do with multiplexing two unrelated video streams into one. Broadcasters and cable systems do this, but Internet video does not. US #6,160,849 only applies to compression of interlaced video, which nobody uses on line. US #7,627,041 is about dealing with missing header data due to transmission noise. US #5,878,080 is about backwards-compatible multichannel (>2 channels) audio, also seldom used on-line. US #6,185,539 is for video with overcompressed extra-low-quality audio. So is US #6,009,399.
As I pointed out last time, it's time for a hard look at this patent situation. You can probably do a patent-free MPEG-4 video decoder and encoder for Internet video now. You might have to leave out some newer features nobody uses.
[1] https://news.ycombinator.com/item?id=10042469 [2] http://scratchpad.wikia.com/wiki/MPEG_patent_lists
The remaining patents are mostly on things you don't need for Internet video. US #6,181,712, has to do with multiplexing two unrelated video streams into one. Broadcasters and cable systems do this, but Internet video does not. US #6,160,849 only applies to compression of interlaced video, which nobody uses on line. US #7,627,041 is about dealing with missing header data due to transmission noise. US #5,878,080 is about backwards-compatible multichannel (>2 channels) audio, also seldom used on-line. US #6,185,539 is for video with overcompressed extra-low-quality audio. So is US #6,009,399.
As I pointed out last time, it's time for a hard look at this patent situation. You can probably do a patent-free MPEG-4 video decoder and encoder for Internet video now. You might have to leave out some newer features nobody uses.
[1] https://news.ycombinator.com/item?id=10042469 [2] http://scratchpad.wikia.com/wiki/MPEG_patent_lists
It seems that MPEG-1 Layer 3 patents are mostly expired, the core patents for MPEG-2 will expire in 2016-2018, but MPEG-4 MPEGLA patents are safe till 2025-2027.
Also if the patents expire the organizations loses it's ability to collect royalties for the codec it self from what i understand so if that would've been the case there's little need to create a new codec
but MPEG4/AVC patent protection seem to remain quite strong for about another decade.
The MPEG4 licensing scheme is kinda confusing to me it seems that several organizations collect royalties individually even tho it states that MPEGLA collects them in the US, AT&T seems to do so too for their share of the patents.
The MPEG4 licensing scheme is kinda confusing to me it seems that several organizations collect royalties individually even tho it states that MPEGLA collects them in the US, AT&T seems to do so too for their share of the patents.
That's the great thing about Daala, they are exploring alternate ways to build a video codec.I'm really hopeful that one day it will be as successful as Opus.
There was a great talk about it https://www.youtube.com/watch?v=Dmho4gcRvQ4
There was a great talk about it https://www.youtube.com/watch?v=Dmho4gcRvQ4
Opus is tricky, it's royalty free but it's not a "free (as in free software)" standard, it's based on Patents held by the likes of Broadcom and MSFT which make them available without royalties.
This is what I've mentioned earlier there are quite a few royalty free but they are still based on held patents which are offered without royalties.
That said honestly I'm not sure how big royalties are HDMI has quite steep royalties DisplayPort doesn't and with pretty much all devices that come with a single/mono-cultured display interface will use HDMI rather than DisplayPort.
This is what I've mentioned earlier there are quite a few royalty free but they are still based on held patents which are offered without royalties.
That said honestly I'm not sure how big royalties are HDMI has quite steep royalties DisplayPort doesn't and with pretty much all devices that come with a single/mono-cultured display interface will use HDMI rather than DisplayPort.
Do royalty free patents stop something from being free software? That would be surprising.
Opus' patents have patent retaliation clauses. It would be funny if those were incompatible with the GPL, because the GPL is essentially a "copyright retaliation" license.
Opus' patents have patent retaliation clauses. It would be funny if those were incompatible with the GPL, because the GPL is essentially a "copyright retaliation" license.
We ran the Opus license terms by the FSF before publishing them, and they confirmed that the terms were compatible with the GPL (v3 or otherwise).
Wasn't there an issue with a couple of companies filing IPR's with royalties that cover parts of Opus?
Please see the "Other disclosures" section on https://www.opus-codec.org/license/
You may also be interested in https://hacks.mozilla.org/2013/02/defending-opus/ (warning: contains discussion of specific, unexpired patents).
You may also be interested in https://hacks.mozilla.org/2013/02/defending-opus/ (warning: contains discussion of specific, unexpired patents).
> I'm really hopeful that one day it will be as successful as Opus.
I assume you mean more successful.
I have never seen or even heard of an Opus file in the wild.
I assume you mean more successful.
I have never seen or even heard of an Opus file in the wild.
Have you ever used:
- Skype
- Google Hangouts
- Youtube on Chrome
- Mumble
- Teamspeak
- the Playstation 4
- most speech recognition APIs
If so, then you've used Opus.
- Skype
- Google Hangouts
- Youtube on Chrome
- Mumble
- Teamspeak
- the Playstation 4
- most speech recognition APIs
If so, then you've used Opus.
I think it is quite popular for VoIP and it is also part of WebRTC and hence supported by Chrome and Firefox
https://en.wikipedia.org/wiki/Opus_(audio_format)#Software
https://en.wikipedia.org/wiki/Opus_(audio_format)#Software
As others have mentioned, it's heavily in use in streaming. I don't know that it offers enough of an advantage over vorbis as a file-format to overcome the inertia of devices that support vorbis; we'll see in a few years.
Perhaps here success is a measurement of producing a patent free solution that's technically competitive (performance, compression, other metrics), rather than popular and widely adopted.
Edit: Looks like I was mistaken. See: http://aomedia.org/
It's important to note that this isn't a project to create a new codec. It's an organization to share the legal infrastructure for things like patent analysis and licensing agreements.
The Joint Development Foundation's website is not even video-specific, but invites any project to join operate under their legal umbrella.
http://www.jointdevelopment.org/
It's important to note that this isn't a project to create a new codec. It's an organization to share the legal infrastructure for things like patent analysis and licensing agreements.
The Joint Development Foundation's website is not even video-specific, but invites any project to join operate under their legal umbrella.
http://www.jointdevelopment.org/
> It's important to note that this isn't a project to create a new codec.
Are you sure?
The alliance website says "The Open and royalty-free format for next-generation ultra high definition media" and their top news item is "Alliance for Open Media Established to Deliver Next-Generation Open Media Formats".
Are you sure?
The alliance website says "The Open and royalty-free format for next-generation ultra high definition media" and their top news item is "Alliance for Open Media Established to Deliver Next-Generation Open Media Formats".
Oh, good point. I jumped right to the Joint Development Foundation part. Oops!
To be fair, it seems you were technically correct (the best kind!) and the website messaging is slightly confusing. According to xiphmont the main point of this particular group is for sharing IP investigation, and the development output of the members will be directed towards NetVC effort in the IETF. So a codec will be produced, and by these same people, but wearing different hats.
http://xiphmont.livejournal.com/67752.html
http://xiphmont.livejournal.com/67752.html
What Mozilla should do is patent the hell of out everything they are currently developing. One way out is a MAD style codec arms race. You have to get the other side to capitulate, to tap out. If you have no patents you have no leverage. The problem with the licensing "authorities" is that they are borg like in that hundreds of orgs have put patents in the pool. Nocking out a patent or two won't effect them.
Sounds nice and all, but part of me can't help but feel quite "eh" to see names like Netflix, Google and Microsoft in there, all responsible for bringing DRM into HTML and moving the open web more toward a closed one. So while they might champion for the open web in cases like these, it's obvious that the only thing they really care about is their bottom line and not paying license fees on video formats.
Companies definitely look for things that serve themselves. But they are not binary "all good" and "all bad." Those companies have also contributed open technologies as well. And this codec would have nothing in it inherently DRM or "closing."
If successful this will be good for all of us, not just good for them and bad for us.
If successful this will be good for all of us, not just good for them and bad for us.
> So while they might champion for the open web in cases like these, it's obvious that the only thing they really care about is their bottom line and not paying license fees on video formats.
So? Netflix and Google are the two biggest online video players. Without them, adoption is going to be pretty pathetic. You want to start excluding people because of perceived (or actual) motivations?
So? Netflix and Google are the two biggest online video players. Without them, adoption is going to be pretty pathetic. You want to start excluding people because of perceived (or actual) motivations?
>You want to start excluding people because of perceived (or actual) motivations?
Nah, I don't subscribe to the (unfortunately common) mentality of being exclusionary based on ideological purity evaluations. But since they obviously don't really subscribe to the idea of Open Web (as evident by their enthusiastic support for DRM in HTML, which is about as anti-Open Web as it gets), I guess I just wish they could be more honest about it. After all, what good is it that someone uses an "Open Media" format when it's put behind a completely closed black box barrier of DRM that you aren't even allowed to legally poke at thanks to anti-circumvention laws?
I do think this particular development is very much a good thing for the Open Web, and I certainly don't doubt Mozilla commitment to that. But the whole HTML DRM debacle did leave a really bitter taste in my mouth.
Nah, I don't subscribe to the (unfortunately common) mentality of being exclusionary based on ideological purity evaluations. But since they obviously don't really subscribe to the idea of Open Web (as evident by their enthusiastic support for DRM in HTML, which is about as anti-Open Web as it gets), I guess I just wish they could be more honest about it. After all, what good is it that someone uses an "Open Media" format when it's put behind a completely closed black box barrier of DRM that you aren't even allowed to legally poke at thanks to anti-circumvention laws?
I do think this particular development is very much a good thing for the Open Web, and I certainly don't doubt Mozilla commitment to that. But the whole HTML DRM debacle did leave a really bitter taste in my mouth.
This is great, but how does it compare to H.265/HEVC? And any news on hardware decoding support? IMO, hardware support is the thing that really makes a difference from a consumer experience point of view (more than patents). It's what allows one to watch hours of video without the device getting hot or losing too much battery life.
This isn't a codec, it's a group to do patent analysis and development for a future codec.
WRT hardware support, I agree, but it won't be available in the early life of any new codec so it's important that it also works well in software (accelerated by SIMD and the like). The power difference is less than you might think, I have some preliminary measurements here: https://people.xiph.org/~tdaede/power/
WRT hardware support, I agree, but it won't be available in the early life of any new codec so it's important that it also works well in software (accelerated by SIMD and the like). The power difference is less than you might think, I have some preliminary measurements here: https://people.xiph.org/~tdaede/power/
Getting your codec supported in hardware is a tricky business, and as much as I've been a huge fan of the Daala initiative it was always going to be an insurmountable hurdle for Mozilla alone. IMO getting buy-in from Cisco and Youtube actually gives us a fighting chance at an actual royalty-free hardware-supported codec, which is why I'm so excited at this news.
This is aiming to be better than HEVC, because it won't be ready for a couple of years.
Assuming the groups listed actually back up their words with action, then hardware support will happen, basically every smart TV shipped already has VP9, so if Google's Youtube and Netflix and Amazon and Microsoft agree on a codec, it'll be in hardware (both in the sense of it being available, and in the sense of it being in every video device you can buy almost instantly.)
Assuming the groups listed actually back up their words with action, then hardware support will happen, basically every smart TV shipped already has VP9, so if Google's Youtube and Netflix and Amazon and Microsoft agree on a codec, it'll be in hardware (both in the sense of it being available, and in the sense of it being in every video device you can buy almost instantly.)
You're really putting the kart ahead of the horse when you ask questions such as that. First you need a great and highly competitive royalty-free codec. If it satisfies those requirements, the hardware decoding will come later.
It will be interesting to see how much of the "stack" ends up being open and royalty free. Don't get me wrong, open format is great, but only part of the story.
That being said, I'm definitely optimistic this is a step in the right direction given the fact that Mozilla is a part of it.
EDIT: Actually, looks like they may be proposing full-stack will be open: http://aomedia.org/about-us/
That being said, I'm definitely optimistic this is a step in the right direction given the fact that Mozilla is a part of it.
EDIT: Actually, looks like they may be proposing full-stack will be open: http://aomedia.org/about-us/
I wonder what that means for Daala. It's scheduled to be finished by the end of this year. I hope it still gets finised.
Technologies from Daala, Thor, VP10 and contributions from other participants in the group will all be combined to essentially make one uber codec.
Since Thor was released Daala already started incorporating parts of it that made sense (and vice versa for Thor). So this will just expand that pattern.
Since Thor was released Daala already started incorporating parts of it that made sense (and vice versa for Thor). So this will just expand that pattern.
I would guess, as an outsider, that it'll basically stop as an independent project and carry on to whatever degree its tech get absorbed into this new project.
There's lots of cool tech in Daala, but I'm guessing that most people involved would be happy to see a royalty-free video codec succeed even if it wasn't entirely their baby and if the member corporations can throw enough lawyers, engineers and existing patents at the problem, then less sexy, more traditional tech will probably get the job done and make Daala's strategy less relevant.
There's lots of cool tech in Daala, but I'm guessing that most people involved would be happy to see a royalty-free video codec succeed even if it wasn't entirely their baby and if the member corporations can throw enough lawyers, engineers and existing patents at the problem, then less sexy, more traditional tech will probably get the job done and make Daala's strategy less relevant.
Daala came up in the HN thread when Cisco announced Thor:
https://news.ycombinator.com/item?id=10042849
https://news.ycombinator.com/item?id=10042849
From what I've seen that is an unrealistic goal. It's not very close to H.265 yet.
It's definitely cool if we get to a point where Microsoft, Mozilla, and Google aren't each pushing their own video codec in their respective browsers. If this ends that, I'm all for it.
Though as long as it isn't proprietary and DRM'd to crud, you won't see any support from Hollywood.
Though as long as it isn't proprietary and DRM'd to crud, you won't see any support from Hollywood.
Well, DRM is completely independent from the codec. You can put DRM on a free codec and you can have a non-free codec without DRM. As for Hollywood... note that the Alliance also includes Netflix.
DRM has to be kinda an integral part of the Codec in order for it to work efficiently especially with streaming any other type of DRM will not be really viable since it will either require you to get the entire file or to chop the video into smaller blocks.
DRM for the most part is encryption and if you want to have it with all the vector voodoo of video compression it has to be built in.
That said today there is no way in hell to make a codec without DRM and expect it to have wide support, lack of DRM is actually killing more projects (like various open source streamers) than outrage of the free software community will ever be able too.
An open codec is only viable if it's worth to be picked by the big players and those are the big media and content providers who won't pick up a penny if it didn't had DRM support...
DRM for the most part is encryption and if you want to have it with all the vector voodoo of video compression it has to be built in.
That said today there is no way in hell to make a codec without DRM and expect it to have wide support, lack of DRM is actually killing more projects (like various open source streamers) than outrage of the free software community will ever be able too.
An open codec is only viable if it's worth to be picked by the big players and those are the big media and content providers who won't pick up a penny if it didn't had DRM support...
DRM is dying.
CSS, AACS and HDCP are all broken, Flash is on its last legs and music downloads have largely abandoned it to positive consequences. Streams seem to be the last bastion, but that doesn't even make sense -- the reason people subscribe to Netflix is to watch new content every month, not to watch the same content over and over. If they wanted to do that they would just buy the Blu-ray (or, if so inclined, pirate it once). And pirates have no reason to rip a 6Mbps stream when they can rip a 40Mbps Blu-ray.
DRM is a box you have to check when a suit was bamboozled by a DRM snake oil salesman into putting it in a contract. How the DRM works is irrelevant because it won't actually work anyway. A fig leaf doesn't need codec support.
That's what EME is about, and that's why you hear the free software people griping about it but not the actual pirates. EME makes piracy easier. It separates the DRM into its own little box which makes it easier to circumvent. Which is what the pirates will do because they don't care about breaking the law, but which law-abiding free software people are proscribed from doing by DMCA 1201.
But the most fundamental flaw in your argument is that you misunderstand who decides what DRM gets used. DRM is not content. What gets supported is ultimately decided by the platform companies. When Apple says there will be no DRM on iTunes, there will be no DRM on iTunes. The only thing Disney or Universal can do about it is withhold their content, but that costs them more than it costs Apple or Google. Which means it isn't something they can credibly threaten unless the dispute is of a make or break significance, and the particulars of DRM implementation are not on that level.
CSS, AACS and HDCP are all broken, Flash is on its last legs and music downloads have largely abandoned it to positive consequences. Streams seem to be the last bastion, but that doesn't even make sense -- the reason people subscribe to Netflix is to watch new content every month, not to watch the same content over and over. If they wanted to do that they would just buy the Blu-ray (or, if so inclined, pirate it once). And pirates have no reason to rip a 6Mbps stream when they can rip a 40Mbps Blu-ray.
DRM is a box you have to check when a suit was bamboozled by a DRM snake oil salesman into putting it in a contract. How the DRM works is irrelevant because it won't actually work anyway. A fig leaf doesn't need codec support.
That's what EME is about, and that's why you hear the free software people griping about it but not the actual pirates. EME makes piracy easier. It separates the DRM into its own little box which makes it easier to circumvent. Which is what the pirates will do because they don't care about breaking the law, but which law-abiding free software people are proscribed from doing by DMCA 1201.
But the most fundamental flaw in your argument is that you misunderstand who decides what DRM gets used. DRM is not content. What gets supported is ultimately decided by the platform companies. When Apple says there will be no DRM on iTunes, there will be no DRM on iTunes. The only thing Disney or Universal can do about it is withhold their content, but that costs them more than it costs Apple or Google. Which means it isn't something they can credibly threaten unless the dispute is of a make or break significance, and the particulars of DRM implementation are not on that level.
Name one codec with specific support for DRM built into the compression.
There aren't any, the DRM is tied to the 'container' and that's part of the standardization process here.
Everybody 'hates' DRM, it seems but I don't think it's going anywhere and I don't think you can ship a codec standard without it. You usually want hardware support for the DRM encryption. I don't know the specifics of their technology, but Netflix and Hulu and Amazon all stream tons and tons of media and I'd be surprised if it wasn't DRM'd in some capacity, I know the video apple sells/streams all is.
Everybody 'hates' DRM, it seems but I don't think it's going anywhere and I don't think you can ship a codec standard without it. You usually want hardware support for the DRM encryption. I don't know the specifics of their technology, but Netflix and Hulu and Amazon all stream tons and tons of media and I'd be surprised if it wasn't DRM'd in some capacity, I know the video apple sells/streams all is.
I'm not aware of any codec with DRM built in to the compression, but essentially every DRM scheme integrates decompression into the DRM module as closely and inseparably as they can manage. Royalty-free codecs make that quite a lot cheaper to do.
MPEG4 trough part 13 (IPMP was also backported into MPEG2) http://www.iso.org/iso/iso_catalogue/catalogue_tc/catalogue_...
http://www.w3.org/2000/12/drm-ws/pp/koenen.pdf
H.264 (trough IPMP), H.265 also implement in-bistream DRM interfaces directly into the decoder...
http://www.w3.org/2000/12/drm-ws/pp/koenen.pdf
H.264 (trough IPMP), H.265 also implement in-bistream DRM interfaces directly into the decoder...
My understanding is that IPMP was not precisely builtin to MPEG-2 or -4. The fact that it could be backported to MPEG2 is partly indicative of this.
IPMP does not in any meaningful way alter the H.263 or H.264 compression algorithims, even though it does alter the bitstream in ways that require modifications of the encode/decode pipeline
There is nothing stopping a third-party from similarly making a pipeline-invasive DRM for any open codec. As a matter of fact, if the open codec is patent free, then there would be no legal way to stop them.
IPMP does not in any meaningful way alter the H.263 or H.264 compression algorithims, even though it does alter the bitstream in ways that require modifications of the encode/decode pipeline
There is nothing stopping a third-party from similarly making a pipeline-invasive DRM for any open codec. As a matter of fact, if the open codec is patent free, then there would be no legal way to stop them.
Both MPEG-2 and MPEG-4 have several revisions, while not being directly tied to the "compression" which might be the wrong choice of words IPMP hooks into virtually every step in the encoding and decoding pipeline.
You can implement multiple IP controls over every bit of the format from audio streams to the scene constructing language that is used to rebuild the scene it self so you can technically put DRM on individual elements so you can literally force people to pay money to see Kevin Spacey instead of a banana in house of cards without having to encode 2 completely different video streams into the file, but I'm pretty sure people will pay extra to see a banana instead of Kevin Spacey.
And while you can make a DRM free encoder it's still wasn't the argument that i was talking about if there won't be a well established and well integrated DRM mechanism within the design of the Codec it will be dead in the water since for a codec to be widely accepted these days it needs to be adopted by the media/movie/tv/streaming/content delivery whatcha gonna call it industry and that industry needs DRM. Heck even sites like YouTube use DRM these days (paid content trough), Twitch and other similar sites will eventually have to enable DRM too if they want to offer premium content as it's much easier to gate people with DRM than to have some weird session based authorization for streaming which is a nightmare and doesn't really work as DRM. And using multiple codecs (cie? x's? xes?) is probably not a viable approach either.
And while you can make a DRM free encoder it's still wasn't the argument that i was talking about if there won't be a well established and well integrated DRM mechanism within the design of the Codec it will be dead in the water since for a codec to be widely accepted these days it needs to be adopted by the media/movie/tv/streaming/content delivery whatcha gonna call it industry and that industry needs DRM. Heck even sites like YouTube use DRM these days (paid content trough), Twitch and other similar sites will eventually have to enable DRM too if they want to offer premium content as it's much easier to gate people with DRM than to have some weird session based authorization for streaming which is a nightmare and doesn't really work as DRM. And using multiple codecs (cie? x's? xes?) is probably not a viable approach either.
I still maintain that a good royalty-free codec will get a DRM standard (by a third party if the creators don't define one), so having it not builtin to the core standard is a non-issue.
I guess a new web image standard to replace JPEG might be a nice side-effect of this too.
Cisco is working on royalty free open source video codec called Thor. https://github.com/cisco/thor
Read the post.
What does this mean for DRM?
This is a video codec and is a separate topic from DRM.
It will have nothing DRM specific.
http://www.mpegla.com/main/programs/hevc/Documents/hevc-att1...
Considering that many companies patent everything and their mother I wonder if it's possible to actually have a Royalty-Free codec in which the involving parties did not volunteer their IP at no cost because I'm not sure if making one without stepping on the toes of existing patents is possible...