The HTML video element needs to go back on the drawing board(ctrl.blog)
ctrl.blog
The HTML video element needs to go back on the drawing board
https://www.ctrl.blog/entry/html-responsive-video.html
127 comments
It’s not just small websites, though. Government information websites use YouTube too. Even activist websites criticizing the tech monopolies host with YouTube embeds. Even the distributed web/P2P platform IPFS hosts videos on YouTube instead of using it’s own P2P stack.
Anyhow: the point was that it’s too difficult to embed videos even if you’re willing to bear the hosting cost.
It costs roughly 0.0015 USD per hour video in 480p/VP9 hosted with BunnyCDN. The cost is manageable.
Anyhow: the point was that it’s too difficult to embed videos even if you’re willing to bear the hosting cost.
It costs roughly 0.0015 USD per hour video in 480p/VP9 hosted with BunnyCDN. The cost is manageable.
>Anyhow: the point was that it’s too difficult to embed videos even if you’re willing to bear the hosting cost.
That may be one reason but like the gp, I also disagree with the blog's author that it's the "underlying problem".
To further add to gp's point, Amazon AWS has:
+ tech staff with skills to deliver HTML video
+ billions to pay for self-hosting videos on its own infrastructure
+ incentives to avoid a competitor such as Google
... And yet, their official AWS re:invent page of videos points to urls on Youtube:
https://aws.amazon.com/blogs/database/amazon-dynamodb-sessio...
Microsoft is another company with technical chops internally but they also uploaded some (not all) of their Channel9 videos to Youtube instead of self-hosting them.
Yes, the complexity of HLS and DASH is also true but it's way down the list of reasons why many people host on Youtube:
+ $0 hosting costs
+ ad monetization (and by extension, viewership statistics tools)
+ audience reach (via recommendations, etc)
A hypothetical improved <VIDEO> HTML tag does not alter the motivations in the bullet points above.
That may be one reason but like the gp, I also disagree with the blog's author that it's the "underlying problem".
To further add to gp's point, Amazon AWS has:
+ tech staff with skills to deliver HTML video
+ billions to pay for self-hosting videos on its own infrastructure
+ incentives to avoid a competitor such as Google
... And yet, their official AWS re:invent page of videos points to urls on Youtube:
https://aws.amazon.com/blogs/database/amazon-dynamodb-sessio...
Microsoft is another company with technical chops internally but they also uploaded some (not all) of their Channel9 videos to Youtube instead of self-hosting them.
Yes, the complexity of HLS and DASH is also true but it's way down the list of reasons why many people host on Youtube:
+ $0 hosting costs
+ ad monetization (and by extension, viewership statistics tools)
+ audience reach (via recommendations, etc)
A hypothetical improved <VIDEO> HTML tag does not alter the motivations in the bullet points above.
I suspect part of that equation is that it’s going to be posted to the YouTube channel anyway, so why not just embed it from there too?
Exactly. One aspect of the issue is the difficulty of self-hosting video content, but another aspect is everything else YouTube does beyond hosting the file. It's a marketing and distribution platform.
Linking from your website to YouTube will let people find your Channel, getting people to come back after the one video. People can save videos for later, youtube videos will show up in search results. You want views on your website to increment the "Views" count on YouTube because that's a signal of legitimacy. You want to be able to pull one report of "how many people are watching our video content?" without having to add numbers from YouTube and your own hosting.
All of these are benefits (or lock-in) that a YouTube embed provides beyond just hosting the file. A <video> element has no way of getting most of those.
Linking from your website to YouTube will let people find your Channel, getting people to come back after the one video. People can save videos for later, youtube videos will show up in search results. You want views on your website to increment the "Views" count on YouTube because that's a signal of legitimacy. You want to be able to pull one report of "how many people are watching our video content?" without having to add numbers from YouTube and your own hosting.
All of these are benefits (or lock-in) that a YouTube embed provides beyond just hosting the file. A <video> element has no way of getting most of those.
The extra views are probably good for their numbers too, if that's something they care about.
Microsoft spent a lot of money trying to build a competitor to Youtube (And Google video) back in the day. (Soapbox) - But... no one wanted it.
People have this weird habit of just following the trends and at the time, using anything and everything google did without question.
Now they don't like it?
Go use vimeo and pay for distribution with services like Cloudflare
People have this weird habit of just following the trends and at the time, using anything and everything google did without question.
Now they don't like it?
Go use vimeo and pay for distribution with services like Cloudflare
> using anything and everything google did without question.
Google Video[1] launched one month before YouTube, yet the latter won so decisively Google bought them a year later.
> Now they don't like it?
It’s been over a decade. You’re allowed to change your opinion and dislike something you previously liked, especially if the thing changed[2].
[1]: https://en.wikipedia.org/wiki/Google_Video
[2]: https://web.archive.org/web/20150323121117/http://doonesbury...
Google Video[1] launched one month before YouTube, yet the latter won so decisively Google bought them a year later.
> Now they don't like it?
It’s been over a decade. You’re allowed to change your opinion and dislike something you previously liked, especially if the thing changed[2].
[1]: https://en.wikipedia.org/wiki/Google_Video
[2]: https://web.archive.org/web/20150323121117/http://doonesbury...
Since YouTube was already growing significantly before Google bought it, I'm not sure "Google" is the right answer.
A better one is "easy sharing" + "pirated content" + "Lonely Island."
A better one is "easy sharing" + "pirated content" + "Lonely Island."
True, it was a wild west growth though - whereas Google put its PPC monetization behind it when it was purchased and no one else could compete with that prowess.
> tech staff with skills to deliver HTML video
A big one that hits small sites I think is simply tech staff with the ability to properly encode HTML video. You lose them as soon as you say vp9, h264, or av1.
A big one that hits small sites I think is simply tech staff with the ability to properly encode HTML video. You lose them as soon as you say vp9, h264, or av1.
This is true, but I would also suggest that small sites would like be better off simply doing a single H.264 file, which is what comes natively off of many cameras and is an export preset in almost everything. If you're not publishing long high-res videos for many thousands of people, you're almost certainly going to be paying more in staff time than you see from the bandwidth savings.
But it's not only about bandwidth. An h.264 video file off a camera is rarely going to be a good choice to throw up on the web.
Cameras write files with extremely short GOPs and overly high bitrates because they have to capture live video and can't know what the next second of content will look like. They need to use settings such that pretty much anything captured by the lens will be captured in good quality.
An offline encoded h.264 video can have far more processing thrown at it. Typically you'll see longer GOPs and a much tighter tuning of bitrates and more features like b-frames and CABAC encoding enabled.
A file directly off a camera can have bitrates of several tens of megabits a second. Even short videos are huge and won't stream well to many. A lot of devices also have limits on the profiles of video they'll even decode.
Someone throwing a camera's direct output onto YouTube can be guaranteed better than 99% of all devices on the Internet can be served a watchable version of the content.
Cameras write files with extremely short GOPs and overly high bitrates because they have to capture live video and can't know what the next second of content will look like. They need to use settings such that pretty much anything captured by the lens will be captured in good quality.
An offline encoded h.264 video can have far more processing thrown at it. Typically you'll see longer GOPs and a much tighter tuning of bitrates and more features like b-frames and CABAC encoding enabled.
A file directly off a camera can have bitrates of several tens of megabits a second. Even short videos are huge and won't stream well to many. A lot of devices also have limits on the profiles of video they'll even decode.
Someone throwing a camera's direct output onto YouTube can be guaranteed better than 99% of all devices on the Internet can be served a watchable version of the content.
Well, if we are going to make hypothetical tags. I'd have a video tag that includes transcoding and hosting.
The next hypothetical tag would be an <vqgan>A humpback whale in a trench coat stares into the camera and says, "Here is looking at you kid" and the camera reverses to show a goat</vqgan>
The next hypothetical tag would be an <vqgan>A humpback whale in a trench coat stares into the camera and says, "Here is looking at you kid" and the camera reverses to show a goat</vqgan>
Has anyone used Cloudflare Stream [0] [1]?
Because my first thought to your above was "Hmm, who has massive bandwidth and storage and could offer something at a competitive price?"
[0] https://developers.cloudflare.com/stream/
[1] https://developers.cloudflare.com/stream/faq
Because my first thought to your above was "Hmm, who has massive bandwidth and storage and could offer something at a competitive price?"
[0] https://developers.cloudflare.com/stream/
[1] https://developers.cloudflare.com/stream/faq
No one wants to host a 480p video and pay for it. They want to host a 4k video for free. YouTube captured the entire market by giving it away and now it's the expectation.
Not true. Few people — if anyone — wants their entire mobile data plan consumed by a single 4K video. Especially if it’s just a short clip.
That it, right?
Some people want 480p video.
Some people want 1080p video.
Some people want 4k video.
Even if the browser <video> tag did a great job at codec and resolution negotiation you still have to encode _at a minimum_ three copies of the video to hit the qualities. Multiply that each time you need a different codec for different devices.
Or, upload one video to Youtube.
Some people want 480p video.
Some people want 1080p video.
Some people want 4k video.
Even if the browser <video> tag did a great job at codec and resolution negotiation you still have to encode _at a minimum_ three copies of the video to hit the qualities. Multiply that each time you need a different codec for different devices.
Or, upload one video to Youtube.
And target device specific playback resolution also comes for free from YouTube. So it will not be 4k on mobile. But it could be.
That doesn't mean people don't want to serve 4K video.
It means they want the video to be encoded in multiple resolutions for different levels of connection. Another thing that's not necessarily trivial, especially at scale.
It means they want the video to be encoded in multiple resolutions for different levels of connection. Another thing that's not necessarily trivial, especially at scale.
I wasn't talking about consumption, I was talking about the people making the videos. They do not want people to watch it in 480p unless it's the last resort. It's a similar mindset to famous directors hating when people watch their movies on phones.
I've fought and lost this battle many many times, and have used mobile bandwidth as a reason too.
High resolution video hosting/streaming is free, so no one wants to pay to host it. I think there are major downsides to hosting on YouTube, but in my experience the vast majority of people do not care.
I've fought and lost this battle many many times, and have used mobile bandwidth as a reason too.
High resolution video hosting/streaming is free, so no one wants to pay to host it. I think there are major downsides to hosting on YouTube, but in my experience the vast majority of people do not care.
Thankfully YouTube gives you different qualities depending on screen size and network speed, without you having to recode the video a dozen times. And for free. ;)
> "It costs roughly 0.0015 USD per hour video in 480p/VP9 hosted with BunnyCDN. The cost is manageable."
while True:
downloadVideoFromDW2A();
How is that inherently harder to manage than
while True:
downloadIndexHtmlFromCSMPLTN();bandwith
> "bandwith"
Leave it to HackerNews to downvote the only sensible answer in this thread, rooted in a simple technical understanding of how distributed caching works.
Leave it to HackerNews to downvote the only sensible answer in this thread, rooted in a simple technical understanding of how distributed caching works.
It's probably not downvoted because it's wrong, but it's because it doesn't help anyone who don't have "simple technical understanding of how distributed caching works."
Down With the Bandness
I don't disagree but a nice counter example is Chaos Computer Club. They host their content on https://media.ccc.de/. With all the videos from their conferences it is not very small site either.
Agree with your points, but the HTML5 video tag is still broken.
For example, on many browsers (Chrome and Safari at least) if you put a video on loop, with certain sizes, the internal logic makes it re-download the same video ignoring any server cache headers. That's it, if you leave a browser open with the same video in loop, it will suck your bandwidth forever.
I think the same happens if you seek through video playback.
To avoid this you need to put in some javascript that preload the whole video and make it a blob, or something similar.
For example, on many browsers (Chrome and Safari at least) if you put a video on loop, with certain sizes, the internal logic makes it re-download the same video ignoring any server cache headers. That's it, if you leave a browser open with the same video in loop, it will suck your bandwidth forever.
I think the same happens if you seek through video playback.
To avoid this you need to put in some javascript that preload the whole video and make it a blob, or something similar.
Then the <video> element is not to blame but browsers’ handling of video playback/caching.
Playback and caching is part of the implementation of the <video> element.
Can you elaborate on the "with certain sizes" please? If the video file is something in the tens of GBs, it would be quite understandable for the browser to avoid keeping the whole file in memory.
Disk caching is still a thing. Kids watch the same videos on repeat for hours on end, so maybe almost-indiscriminately caching the entirety of the last watched video would be a good idea.
Disk caching probably invokes Flava Works vs Gunter [1] which is a case that distinguishes streaming from copying under copyright law. There are probably later updates to this, but this is the one I'm aware of.
https://www.avvo.com/legal-guides/ugc/copyright-101-is-strea...
https://www.avvo.com/legal-guides/ugc/copyright-101-is-strea...
Leave it to the copyright people to make a huge legal mess out of the simplest concepts. Now people can't even cache data without lawyers showing up. I wonder how long it's gonna take before I see them arguing memcpy should be illegal.
On the other hand, in a typical Netflix binging scenario, this would cause tens of gigabytes of avoidable SSD writes.
Especially for mobile devices with fairly small storage sizes, this can easily represent a sizable fraction of their total write endurance.
Especially for mobile devices with fairly small storage sizes, this can easily represent a sizable fraction of their total write endurance.
So maybe be a bit smarter about it. The browser will know if the video is set to loop unless that is done with JS. In all cases it will know if it has looped and can start caching then.
And just you don't need to limit all devices by the lowest common denominator either.
And just you don't need to limit all devices by the lowest common denominator either.
> unless that is done with JS.
Which is the entire point of the article: Most of the time even trivial video playback is done using JS due to the shortcomings of the <video> tag itself! This also means that browsers are unable to perform such optimizations.
> And just you don't need to limit all devices by the lowest common denominator either.
Unless the video site is intentionally sending cache control headers preventing caching of the video assets, this behavior is presumably already caused by the browser itself, not the site.
Which is the entire point of the article: Most of the time even trivial video playback is done using JS due to the shortcomings of the <video> tag itself! This also means that browsers are unable to perform such optimizations.
> And just you don't need to limit all devices by the lowest common denominator either.
Unless the video site is intentionally sending cache control headers preventing caching of the video assets, this behavior is presumably already caused by the browser itself, not the site.
Sadly the threshold is something around >5Mb, if I remember correctly.
Why? I have 64 GB of RAM. Linux is using most of it to cache file system pages. I already do that kind of stuff anyway when I download videos to tmpfs. The browser should just use the memory.
> That's it, if you leave a browser open... it will suck your bandwidth forever.
Funny you mention that, google / youtube does the same thing to jank'ify its own user metrics on my laptop, regardless of whether it's a 5 minute video or a 3 hour video (by auto-playing the next one). Even if the laptop is asleep.
Funny you mention that, google / youtube does the same thing to jank'ify its own user metrics on my laptop, regardless of whether it's a 5 minute video or a 3 hour video (by auto-playing the next one). Even if the laptop is asleep.
This is correct. I’ve written the JavaScript to select sizes dynamically – it’s 599KB smaller than the 600KB claimed.
The hard part is the transcode infrastructure, and that’s frequently unnecessary for the small videos: if you use multiple sizes and formats, you increase the amount of traffic you need to see cache hits. YouTube has built a ton of infrastructure but most sites won’t see enough return to be worth the ops cost.
The hard part is the transcode infrastructure, and that’s frequently unnecessary for the small videos: if you use multiple sizes and formats, you increase the amount of traffic you need to see cache hits. YouTube has built a ton of infrastructure but most sites won’t see enough return to be worth the ops cost.
I blame Google for making mpeg-dash so inaccessible.
Want/need to use DRM? You won't receive a response from Google/widevine, no matter how often you write to them, despite them claiming otherwise and no matter how much money you already earn them via their advertising network.
They have the defacto monopoly and they want to keep it.
>Want/need to use DRM? You won't receive a response from Google/widevine
Thanks Google!
Thanks Google!
This doesn’t mean DRM is impossible, it means it’s only available to really big players who can cut deals with Google. Which makes it one of the few parts of the Web API that smaller companies and independent devs can’t use (not that I’d personally want to). Keeping it tightly controlled like that also makes it harder to crack.
>Keeping it tightly controlled like that also makes it harder to crack.
We both know that's BS. I never thought I'd say this but Google is doing good by not selling this "product" to anyone that just asks.
We both know that's BS. I never thought I'd say this but Google is doing good by not selling this "product" to anyone that just asks.
>Keeping it tightly controlled like that also makes it harder to crack
Can you elaborate?
Can you elaborate?
> They have the defacto monopoly and they want to keep it.
FairPlay and PlayReady are there too.
FairPlay and PlayReady are there too.
Thanks I didn't know about those.
The previously successful site that needed DRM isn't online anymore because we couldn't find a solution to the streaming audio/video problem with DRM and monetization.
Because of that we lost our access to content so the site had to shut down.
For 9 years it was the leading news source for an, unnamed here, African country.
So I see, Apple and Microsoft have offers, but that's also not open source or easily accessible? It doesn't matter to me anymore personally. Google cut their own source of income by denying us access to widevine, since the site was fully monetized via Google.
So I see, Apple and Microsoft have offers, but that's also not open source or easily accessible? It doesn't matter to me anymore personally. Google cut their own source of income by denying us access to widevine, since the site was fully monetized via Google.
Neither of which are available in Chrome clients though, right?
Even without the storage/bandwidth problem. Why develop internally a system to upload and distribute video which will have, at best, the functionalities YouTube provides for free.
The day YouTube will charge for the embedding, a lot a website will find something like. Like a lot of website started using a solution based on OpenStreetMap when Google maps changed its pricing.
The day YouTube will charge for the embedding, a lot a website will find something like. Like a lot of website started using a solution based on OpenStreetMap when Google maps changed its pricing.
> The reason small websites prefer YouTube embeds is because they want to avoid bearing the storage and bandwidth costs of serving it.
YouTube also works 100% of the time. I've encountered countless other sites, including major ones, like CNN or The Guardian, where half of the time video doesn't work, or it's terrible in some way (impossible to seek or pause, ...)
YouTube also works 100% of the time. I've encountered countless other sites, including major ones, like CNN or The Guardian, where half of the time video doesn't work, or it's terrible in some way (impossible to seek or pause, ...)
Exactly correct. Video is on a different level of cost/complexity than images, html, etc. Video is not one of those things you can have your build pipeline automatically optimize for you as part of deploy (well at least not if you want deploys in a reasonable amount of time).
Disclaimer: Founder of service that lets you use <smartvideo> tag with raw, unfiltered videos, and automatically turns them into optimized, streaming, cdn enabled playback.
Disclaimer: Founder of service that lets you use <smartvideo> tag with raw, unfiltered videos, and automatically turns them into optimized, streaming, cdn enabled playback.
One of the many reasons I use YT or Vimeo is that I simply upload the high-res render, edit some metadata and I'm done with a simple embed in most cases. Alternate versions for lower bandwidth are automatically made and automatically served, streaming is taking care of (YT really excels here), and there is a nice API to do interactive stuff.
I do use the <video> element from time to time, but only if I want to optimise the loading of an autoplaying video or there is some other technical reason.
I do use the <video> element from time to time, but only if I want to optimise the loading of an autoplaying video or there is some other technical reason.
> And the logistical costs of transcoding it to N different resolutions times M different codecs.
If only we had easy access to scalable video (SVC). If a container format supported it, the web browser could perform range queries to get the interesting bits as needed, no need for additional code.
And the uploader would need to transcode only once. This doesn't solve the codecs issue, but I think you could get away with offering one or two common codecs.
If only we had easy access to scalable video (SVC). If a container format supported it, the web browser could perform range queries to get the interesting bits as needed, no need for additional code.
And the uploader would need to transcode only once. This doesn't solve the codecs issue, but I think you could get away with offering one or two common codecs.
Good point. YouTube isn’t the only game in town though. There are competitors such as Mux and Wistia.
Vimeo would've been 1st on my list of yt alts
Youtube is free vs others which are paid. And youtube is the most known video hosting service while I have barely heard of others. Are they really competitors?
Twitch is free and quite popular.
Everyone hangs out on Daily Motion, Vimeo, Cinnamon, D.Tube, and PeerTube all the time. Right?
Everyone hangs out on Daily Motion, Vimeo, Cinnamon, D.Tube, and PeerTube all the time. Right?
twitch isn't a video hosting service. it's a streaming service
...with on-demand replays of previously streamed content.
...which expire in two weeks (or 60 days if you are rich) so only few minutes of highlights remain.
Which everyone are you talking about? 99% of youtube users probably never heard of those services.
After working in video streaming company for several years and writing web player (on top of vjs & shaka) used by millions of users every month I both agree and disagree with the article at the same time.
Streaming video is too hard, 100% agree with that, but <video/> element is the reason nor the biggest problem. Biggest pain in my opinion is DRM. That is huge pain in the ass which is poorly standardised (getting better but still) and is constantly broken. Second is transcoding, which is hard because it is hugely complicated problem. Even after the years I don’t feel I understand more than 10% of what is going on. Luckily there are companies like Bitmovin and Mux that can help with those 2. Distribution of the video is easier with modern CDNs, it’s just insanely expensive. Where we could do better though is to better integrate low level libraries like VideoJS and Shaka to web frameworks like React or Vue. Everything that is out there feels like hobby project someone just played with. Full respect and big thanks to the authors for putting it out though. You did way better than me in supporting open-source.
The html video tag is the smallest bit of the problem, adaptive streaming is much more complex and there isn’t just one file per resolution.
Streaming video is too hard, 100% agree with that, but <video/> element is the reason nor the biggest problem. Biggest pain in my opinion is DRM. That is huge pain in the ass which is poorly standardised (getting better but still) and is constantly broken. Second is transcoding, which is hard because it is hugely complicated problem. Even after the years I don’t feel I understand more than 10% of what is going on. Luckily there are companies like Bitmovin and Mux that can help with those 2. Distribution of the video is easier with modern CDNs, it’s just insanely expensive. Where we could do better though is to better integrate low level libraries like VideoJS and Shaka to web frameworks like React or Vue. Everything that is out there feels like hobby project someone just played with. Full respect and big thanks to the authors for putting it out though. You did way better than me in supporting open-source.
The html video tag is the smallest bit of the problem, adaptive streaming is much more complex and there isn’t just one file per resolution.
My reason for desiring the media attribute on video sources is so you can use different videos in dark mode and light mode, with no JavaScript required, which I have definitely done with image sources. (No public examples, sorry; but it’s stuff like including a screenshot of a third-party web app which has its own light and dark modes, so I take a screenshot of each and have it choose the appropriate one.)
For that matter, I want audio sources to support media queries, so that you can present dark and ominous music in dark mode and light and bouncy music in light mode if you really want to, but mostly just so that <source>’s properties are consistent regardless of the parent element.
As for making poster support multiple sources, I came up with a technique that makes this work a while back. I should finish that pair of blog posts off. Short version: poster="data:image/svg+xml,<svg><foreignObject><picture><source …/>…</picture></foreignObject></svg>". Yep. Seriously! :-)
For that matter, I want audio sources to support media queries, so that you can present dark and ominous music in dark mode and light and bouncy music in light mode if you really want to, but mostly just so that <source>’s properties are consistent regardless of the parent element.
As for making poster support multiple sources, I came up with a technique that makes this work a while back. I should finish that pair of blog posts off. Short version: poster="data:image/svg+xml,<svg><foreignObject><picture><source …/>…</picture></foreignObject></svg>". Yep. Seriously! :-)
> For that matter, I want audio sources to support media queries, so that you can present dark and ominous music in dark mode and
Okay I feel like this computer/internet thing has been a grave error, maybe it's time to pack it in and start over.
Okay I feel like this computer/internet thing has been a grave error, maybe it's time to pack it in and start over.
> I want audio sources to support media queries, so that you can present dark and ominous music in dark mode and light and bouncy music in light mode
You’ll love what they’ve done in Windows 11.
You’ll love what they’ve done in Windows 11.
Huh, I’ve been waiting for years for someone to do something like that. Even if they’ve gone for softer/“calmer” sounds (effectively reducing the contrast) rather than dark and ominous!
You don't need media support in video for prefers-color-scheme. Just include both dark/light versions of the videos and hide/show from CSS.
Much the same is true of picture sources (though I’d guess—without having checked—that browsers are more likely to start downloading images before realising they are display:none than they are videos). The theory behind <source> is that all of the offered sources are equivalent in some way, which for audio and video would strongly imply temporal equivalence, meaning that if the media query match changes, it should swap the source and continue where you’re up to. This, incidentally, is where DASH/HLS would be really helpful, to make it easier to transition from one source to another with no pause.
Whereas if you do two videos and hide the one that doesn’t match, and you have a computer that switches to dark mode at sunset or based on ambient light, your video is going to stop playing and you’ll lose your place as soon as it switches from dark to light or light to dark.
Whereas if you do two videos and hide the one that doesn’t match, and you have a computer that switches to dark mode at sunset or based on ambient light, your video is going to stop playing and you’ll lose your place as soon as it switches from dark to light or light to dark.
And that's precisely the reason it was removed from the specs. There are so many moving things that need to be done right and in the end it all hinges on the media authoring being done right (which is why the HLS/DASH specs are valuable). I went over some of these things a few months ago: https://github.com/filamentgroup/html-video-responsive/issue...
A simpler method is to have a transparent poster and then have a <picture> behind the <video>.
Problems with that: full screen the video and the poster disappears; if the video aspect ratio differs from the poster, or the video has an alpha channel, the poster will be visible underneath; browsers could change their video poster styles so that this would look different or even not work (though at least in Firefox and Chromium they do look the same at present).
> I came up with a technique that makes this work a while back.
It may work, but … ugh. This is a spec issue.
It may work, but … ugh. This is a spec issue.
The <video> element does have some usages which are underutilized, though: short, perhaps looping video, which can be used instead of animated GIFs.
The <video> element is very capable, but the amount of combinations of possible video files has greatly increased. When it was developed, the biggest complexities were codec compatibility, and fallbacks.
While it would be possible to have the ability to select different video sizes/quality rates based on HTML elements and attributes, it is likely impractical to handle, say, adaptive video streaming (where the quality of the video is based on the connection), purely in HTML.
I'm no fan of Javascript for everything, but handling video is a scenario where there's generally a need, due to the complexities of video streaming.
The <video> element is very capable, but the amount of combinations of possible video files has greatly increased. When it was developed, the biggest complexities were codec compatibility, and fallbacks.
While it would be possible to have the ability to select different video sizes/quality rates based on HTML elements and attributes, it is likely impractical to handle, say, adaptive video streaming (where the quality of the video is based on the connection), purely in HTML.
I'm no fan of Javascript for everything, but handling video is a scenario where there's generally a need, due to the complexities of video streaming.
I think it would be better if browsers implemented allowing video formats for src and srcset, just treat them the same as gif files.
Safari does this (at least the src), but nobody can really use it since Chrome doesn’t.
There’s a huge thread somewhere of devs arguing that Chrome should implement. I would fall on the implement side. If we want to replace gifs with better formats browsers should make it easy.
https://bugs.chromium.org/p/chromium/issues/detail?id=791658 https://github.com/whatwg/html/issues/7141
Safari does this (at least the src), but nobody can really use it since Chrome doesn’t.
There’s a huge thread somewhere of devs arguing that Chrome should implement. I would fall on the implement side. If we want to replace gifs with better formats browsers should make it easy.
https://bugs.chromium.org/p/chromium/issues/detail?id=791658 https://github.com/whatwg/html/issues/7141
> The <video> element does have some usages which are underutilized, though: short, perhaps looping video, which can be used instead of animated GIFs.
But then you still want to serve a small video file (320px wide) for mobile and a larger one for desktop (600px wide).
> While it would be possible to have the ability to select different video sizes/quality rates based on HTML elements and attributes, it is likely impractical to handle, say, adaptive video streaming (where the quality of the video is based on the connection), purely in HTML.
Adaptive streaming where the quality changes over time as you watch is only needed for longer videos. You don’t need that for a short one minute video or a GIF equivalent. Just picking a resolution appropriate to the screen size would be enough.
But then you still want to serve a small video file (320px wide) for mobile and a larger one for desktop (600px wide).
> While it would be possible to have the ability to select different video sizes/quality rates based on HTML elements and attributes, it is likely impractical to handle, say, adaptive video streaming (where the quality of the video is based on the connection), purely in HTML.
Adaptive streaming where the quality changes over time as you watch is only needed for longer videos. You don’t need that for a short one minute video or a GIF equivalent. Just picking a resolution appropriate to the screen size would be enough.
Hum. Not so sure about this. A fully qualified video tag would be significantly larger than any iframe tag for embedding video (and therefor also unpopular). Also, it still fails to inform a web browser precisely about the options for handling rate issues (e.g., providing information on data rates). Moreover, providing such hosting specific information is probably beyond what a web developer might know and such information might be also subject to change.
What about addressing the problem from the HTTP-side of things: have a simple video tag (just defining the viewport and an endpoint) and a host response providing options (as a manifest), which may be selected by the browser situationally?
What about addressing the problem from the HTTP-side of things: have a simple video tag (just defining the viewport and an endpoint) and a host response providing options (as a manifest), which may be selected by the browser situationally?
> What about addressing the problem from the HTTP-side of things: have a simple video tag (just defining the viewport and an endpoint) and a host response providing options, which may be selected by the browser situationally?
HTTP Client Hints spec can send the viewport size and pixel ratio and even a low-bandwidth preference as HTTP request headers. It’s only supported in Chrome and requires extra round trips. The other browser vendors have been unwilling to implement the spec.
HTTP Client Hints spec can send the viewport size and pixel ratio and even a low-bandwidth preference as HTTP request headers. It’s only supported in Chrome and requires extra round trips. The other browser vendors have been unwilling to implement the spec.
IMHO, this (client hints) is still not the right way to do it. Any hints (as for preferred quality levels etc) should be part of the tag. Any available options with specific qualifiers should be transparently and extensively provided by the host response (preferably in a human maintainable manifest format). The browser should then select the optimum for the situation, based on any preferences expressed in the tag and available options as provided in the host response. A browser should be also able to adjust situationally (e.g., as network bandwidth, scaling, or orientation changes) and select the new optimum without further negotiations (using standard time offsets).
> The browser should then select the optimum for the situation
I like the idea of a human readable manifest to select for preffered quality based on the browser environment. That would be much preffered over loading the tag. Maybe just a quality attribute and point it to the manifest. A man can dream.
I like the idea of a human readable manifest to select for preffered quality based on the browser environment. That would be much preffered over loading the tag. Maybe just a quality attribute and point it to the manifest. A man can dream.
That’s what the article argues for.
No, the article argues for a very complex tag, which still doesn't provide enough information (like data rate specifications) for a browser to make an educated guess for the available optimum. Note that these are totally diferrent concerns: defining the viewport (which belongs to the tag) and its endpoint, on the one hand, and technical specificity of available media on the other hand.
(Putting all information in the tag is a bit, as if we were to define in an image tag, whether the image was encoded progressive or not, what size the color pallet is, etc. However, we can't put this in a video header, as this comes too late, as we decided for a specific source already. But there's the 300 "Multiple Choices" response, maybe, we may build on this.)
Edit: What really matters to an editor of an embedding document is selecting an endpoint (like "example.com/videos/cute-cat1") and defining its visual properties, probably responsively and by means of CSS. An editor is not concerned with quality levels and bit rates vs bandwidth, actual display size, etc. The editor even isn't in any position to make a suitable decision for every situation beforehand. Available options should be advertised by the endpoint and the browser should select adaptively the optimum to fill the viewport on its own behalf (and not by delegating this to the providing host by means of a handshake, which makes things not only unnecessarily complex, but also leaks information).
(Putting all information in the tag is a bit, as if we were to define in an image tag, whether the image was encoded progressive or not, what size the color pallet is, etc. However, we can't put this in a video header, as this comes too late, as we decided for a specific source already. But there's the 300 "Multiple Choices" response, maybe, we may build on this.)
Edit: What really matters to an editor of an embedding document is selecting an endpoint (like "example.com/videos/cute-cat1") and defining its visual properties, probably responsively and by means of CSS. An editor is not concerned with quality levels and bit rates vs bandwidth, actual display size, etc. The editor even isn't in any position to make a suitable decision for every situation beforehand. Available options should be advertised by the endpoint and the browser should select adaptively the optimum to fill the viewport on its own behalf (and not by delegating this to the providing host by means of a handshake, which makes things not only unnecessarily complex, but also leaks information).
might be a bit off topic than what's discussed, but actually I never cease to be amazed by how easy it is to do image processing in browser. want some pixels? pass an img element. moving pixels? pass a video element. dynamic pixels? pass a canvas element. it just works.
now try the same in any other runtime... go mess with different compressions, encodings, color spaces, list goes on and on...
now try the same in any other runtime... go mess with different compressions, encodings, color spaces, list goes on and on...
Oops! Your forgot to specify the video metadata, and the browser guessed the wrong channel range, and the wrong color space from the video size! And the video colors change when the Youtube player controls appear and disappear! And it won't play your video because it's yuv444 and browsers only support yuv420 with blurry color channels!
Most video editing software does provide nice presets you can use. Then there’s always the FFMPEG wiki.
I found https://trac.ffmpeg.org/wiki/colorspace baffling when trying to understand it.
In Rust you just install the respective Multimedia crate.
https://crates.io/categories/multimedia
https://crates.io/categories/multimedia
But you have to mess with every single one of those things to actually get it working across the major browsers. And then it still won't work at all in older browsers.
If it happens, I only have one wish: make the controls available even if the video is yet to load. I've no idea why I'm unable to press pause or, say, mute the volume before a video loads. Instead, I have to wait around. I think even my old VHS player would let me press pause before the video had finished loading!
Isn't the cardinal rule of the web to not waste people's time?
Isn't the cardinal rule of the web to not waste people's time?
>Isn't the cardinal rule of the web to not waste people's time?
This has certainly not been my experience on the web!
Jokes aside, I agree with your point on controls loading first.
This has certainly not been my experience on the web!
Jokes aside, I agree with your point on controls loading first.
Back to the drawing board feels like an exaggeration: most of what’s outlined here could be added non-destructively.
Personally I love how powerful the video tag is, you can even render it to a canvas tag and do all sorts of manipulations on top of it. But it’s the preparation of videos on the web that’s still a huge pain (and why folks use things like YouTube embeds): you’ve got your h264 standard encoding, but if you care about your users you’ll want to save them bandwidth and make h265 and vp9 encodings as well. What settings you apply to each encoding feels like a dark art to me, thankfully there are cloud services to help you but I wish it was simpler.
Personally I love how powerful the video tag is, you can even render it to a canvas tag and do all sorts of manipulations on top of it. But it’s the preparation of videos on the web that’s still a huge pain (and why folks use things like YouTube embeds): you’ve got your h264 standard encoding, but if you care about your users you’ll want to save them bandwidth and make h265 and vp9 encodings as well. What settings you apply to each encoding feels like a dark art to me, thankfully there are cloud services to help you but I wish it was simpler.
I thought the article would mention my nemesis: ":( No video with supported format and MIME type found".
I’d add that the lack of support for media attributes is an accessibility fail. You effectively cannot honor both:
- prefers-reduced-motion - prefers-reduced-data
… without JS.
Edit to add: also, Safari won’t play <video> at all if your server doesn’t support HLS/respond appropriately to range headers. Which is probably well-meaning but absurd.
- prefers-reduced-motion - prefers-reduced-data
… without JS.
Edit to add: also, Safari won’t play <video> at all if your server doesn’t support HLS/respond appropriately to range headers. Which is probably well-meaning but absurd.
If your server doesn't support range headers you don't support half the features of video playback. Without range requests you can't seek to parts of the video that have yet to download. You also can't support the preload attribute.
Edit:
Without range requests the browser can't play a video that hasn't been saved correctly. The MP4 format supports the header atom written to the end of a file, so in cases where a file is streamed to disk a header isn't written until you know the stream is complete and can fill in the details. "Web Optimized" (there's a variety of names) are MP4s with the header atom written at the beginning of the file with the media atoms following.
Without the header data you can't know much about the media tracks in the file so you can't effectively set up a decoder pipeline. A browser will request a file and if it doesn't find the header atom will use a range request to grab the last few kilobytes of the file to grab the header data. It can then set up the decode pipeline and play back the file.
If the server doesn't support range headers and the file isn't "Web Optimized" a browser needs to download the whole file before it can begin playing it. HLS/DASH streams store a lot of data in the manifest file and the MPEG-TS/MP4 chunk streams contain redundant track metadata to inform the decoder pipeline.
Edit:
Without range requests the browser can't play a video that hasn't been saved correctly. The MP4 format supports the header atom written to the end of a file, so in cases where a file is streamed to disk a header isn't written until you know the stream is complete and can fill in the details. "Web Optimized" (there's a variety of names) are MP4s with the header atom written at the beginning of the file with the media atoms following.
Without the header data you can't know much about the media tracks in the file so you can't effectively set up a decoder pipeline. A browser will request a file and if it doesn't find the header atom will use a range request to grab the last few kilobytes of the file to grab the header data. It can then set up the decode pipeline and play back the file.
If the server doesn't support range headers and the file isn't "Web Optimized" a browser needs to download the whole file before it can begin playing it. HLS/DASH streams store a lot of data in the manifest file and the MPEG-TS/MP4 chunk streams contain redundant track metadata to inform the decoder pipeline.
“My” server admittedly when I discovered this was Cloudflare Pages. I didn’t know at the time video was explicitly disallowed but when I did I yanked it. But otherwise your gripes are better directed to GitHub (with similar restrictions) and Cloudflare.
There's zero incentive to change anything wrt to video for the gatekeepers of the web due to conflict of interest. Just saying.
> The average JavaScript library for handling video resolutions and full-screen mode switching is about 600 KB
Wow, really? You can write a full HTTP 1.0 server in a 15th-20th of that.
Wow, really? You can write a full HTTP 1.0 server in a 15th-20th of that.
The one I’m most familiar with has a lot of legacy code duplicating features which didn’t used to be built-in: when we had to support IE, having a Flash fallback, subtitle parser, etc. made more sense and they built a branded UI with custom themes, and tracking, of course.
If you are just doing standard video playback, something like video.js is an order of magnitude less code (on the order of 50KB minimized + gziped).
If you are just doing standard video playback, something like video.js is an order of magnitude less code (on the order of 50KB minimized + gziped).
This seems massively exaggerated. Likely referring to just one they checked which maybe does many other things. Or is just horribly written. Be nice if that line was cited.
All of VideoJS is 158KB so I doubt it.
I can't see those two interactions needing that much code.
edit: that's gzipped, but still, less than 600KB w/ many more features.
I can't see those two interactions needing that much code.
edit: that's gzipped, but still, less than 600KB w/ many more features.
551.87 KB
https://vjs.zencdn.net/7.15.4/video.min.js
plus
56.14 KB
https://vjs.zencdn.net/7.15.4/video-js.css
https://vjs.zencdn.net/7.15.4/video.min.js
plus
56.14 KB
https://vjs.zencdn.net/7.15.4/video-js.css
I don't know anything about video.js in particular, but many libraries are huge when served from a CDN because they have to account for every possible use case, as well as every locale, whereas in a local build with tree shaking and modular imports, it can be cut down by quite a lot. Depends on how "legacy" the codebase is, though.
True, the CDN size is 158KB gzipped, but it's still under 600KB unpacked.
The point is that's for a feature packed video lib which also covers resolution handling
(not sure if it handles fullscreen resolution changing, but I don't see why they wouldn't.)
The point is that's for a feature packed video lib which also covers resolution handling
(not sure if it handles fullscreen resolution changing, but I don't see why they wouldn't.)
Should also fix seeking if the server doesn't support it. If there is a downloadable video file it should just play. Seeking should provide visual feedback and jump to the selected point only when data is returned while continuing normal playback.
Youtube is not an option as it requires a popup for the General Data Protection Regulation.
The only working thing we have atm is OS specific forced downloads that require downloading the entire file.
Youtube is not an option as it requires a popup for the General Data Protection Regulation.
The only working thing we have atm is OS specific forced downloads that require downloading the entire file.
A server only needs to support range requests to support seeking to parts of the video that haven't yet downloaded.
If something is not available you try to make things work without it. The videos play just fine until the user attempts to seek. At that point playback is terminated.
Thus therefore as a result thereof the reason forwhich one can not put a video on the website is that playback may be terminated unexpectedly. It's just too embarrassing, I don't want to see myself explain why it doesn't work to anyone.
You can disable controls entirely or javascript the seek event but I don't consider those nice solutions. The video player should work by default. The millions of crappy web devs simply want to use it without investigating the weirdness and hacking up some event listener to allow seeking in already downloaded parts. Or is that seeking backwards only? I don't want to know the answer honestly.
Perhaps I should just offer a download? Just like things use to work? But then the browser may play it in a tab and pretend seeking is supported again.
Ah! So I have to use the old forced binary file download with application/octet-stream so that the user can watch the video in a different player. One with expected behavior.
I just want to drop the file in a folder and send the link to someone. Adding a header to it is already asking to much.
I could see myself bother to provide multiple formats in multiple sizes because that is an actual hard problem (I'm lying to be polite here)
I find it amazing how well everything in our world works. We humans are very hard working but we tend to do everything wrong most of the time. Look how much effort it took to get embeded video on a web page. It actually works! Well, almost....
This comment is getting so long, I'm probably wrong in more than one way, made a dozen grammatical boo boo's and misspelled a bunch of words. Still, plenty of room left for mis-interpretations and -understandings.
I hope no one is watching us.
Thus therefore as a result thereof the reason forwhich one can not put a video on the website is that playback may be terminated unexpectedly. It's just too embarrassing, I don't want to see myself explain why it doesn't work to anyone.
You can disable controls entirely or javascript the seek event but I don't consider those nice solutions. The video player should work by default. The millions of crappy web devs simply want to use it without investigating the weirdness and hacking up some event listener to allow seeking in already downloaded parts. Or is that seeking backwards only? I don't want to know the answer honestly.
Perhaps I should just offer a download? Just like things use to work? But then the browser may play it in a tab and pretend seeking is supported again.
Ah! So I have to use the old forced binary file download with application/octet-stream so that the user can watch the video in a different player. One with expected behavior.
I just want to drop the file in a folder and send the link to someone. Adding a header to it is already asking to much.
I could see myself bother to provide multiple formats in multiple sizes because that is an actual hard problem (I'm lying to be polite here)
I find it amazing how well everything in our world works. We humans are very hard working but we tend to do everything wrong most of the time. Look how much effort it took to get embeded video on a web page. It actually works! Well, almost....
This comment is getting so long, I'm probably wrong in more than one way, made a dozen grammatical boo boo's and misspelled a bunch of words. Still, plenty of room left for mis-interpretations and -understandings.
I hope no one is watching us.
> What if the two video files are of different durations? There’s a hornet’s nest of potential issues, but I’d take the occasional stings over the status quo any week of the day.
It's very easy to say things like this when you're not proposing a standards update that browser developers then have to implement. HTML is littered with examples of elements, attributes, whatever, that aren't consistently supported because browser developers disagree with, or just don't know, the details of how they're supposed to work. Case and point, focus management when invoking or moving within a modal created via the native dialog element.
In theory, this single user would be okay with the fact that switching video quality or size might do something weird if the underlying videos aren't compatible because of length. In practice, someone has to develop the code to make or let the "something weird" happen, define what it is, define what conditions trigger it, and so on.
It's very easy to say things like this when you're not proposing a standards update that browser developers then have to implement. HTML is littered with examples of elements, attributes, whatever, that aren't consistently supported because browser developers disagree with, or just don't know, the details of how they're supposed to work. Case and point, focus management when invoking or moving within a modal created via the native dialog element.
In theory, this single user would be okay with the fact that switching video quality or size might do something weird if the underlying videos aren't compatible because of length. In practice, someone has to develop the code to make or let the "something weird" happen, define what it is, define what conditions trigger it, and so on.
>The average JavaScript library for handling video resolutions and full-screen mode switching is about 600 KB. It’s a small overhead for a 15 minute+ video. However, it’s way too much for a short animation or a minute-long presentation.
I would argue it shouldn't even need JS in the first place.
But then again Youtube solves that problem for you with practically zero code.
I would argue it shouldn't even need JS in the first place.
But then again Youtube solves that problem for you with practically zero code.
This is NSFW, but if you want to see how to do HTML video right, check out redgifs.com
The videos play almost instantaneously, in any browser with DRM installed and without, even on, e.g., a 2013 netbook.
I think that, just as with many other things in HTML/JS land, there is a way to do it simply and elegantly, though it requires paging through dozens of StackO posts, random docs, etc.
For example, I used to think that paste-to-upload was difficult to achieve, until I'd paged through all that and discovered it's no more than a couple of lines, including creating the form elements and all.
Dig deep and you will find the way. There's no need to use more complicated options which also don't work in 99% of existing browser base.
The videos play almost instantaneously, in any browser with DRM installed and without, even on, e.g., a 2013 netbook.
I think that, just as with many other things in HTML/JS land, there is a way to do it simply and elegantly, though it requires paging through dozens of StackO posts, random docs, etc.
For example, I used to think that paste-to-upload was difficult to achieve, until I'd paged through all that and discovered it's no more than a couple of lines, including creating the form elements and all.
Dig deep and you will find the way. There's no need to use more complicated options which also don't work in 99% of existing browser base.
redgifs was spun out of Gfycat (with a new owner too), so I assume that the same player is being used at https://gfycat.com/ (or rather the other way around), but can't verify right now as I'm in a working environment.
You're right, I knew it looked familiar.
However, Gfycat features many anti-patterns, such as heavy layout, inaccessible scrolling and cookie notification overlay, while RT omits these.
However, Gfycat features many anti-patterns, such as heavy layout, inaccessible scrolling and cookie notification overlay, while RT omits these.
Porn sites have an impetus to serve content quickly because the content itself is often an advertisement for some other paid service. So even with some small number of conversions they make some money.
Gfycat et all aren't lead-ins to paid services so they're going to festoon pages with low paying ads. The porn content is also usually more fungible than a meme gif. A particular gif of a cat or something is what people are after vs the porn where it's anything that turns them on.
Gfycat et all aren't lead-ins to paid services so they're going to festoon pages with low paying ads. The porn content is also usually more fungible than a meme gif. A particular gif of a cat or something is what people are after vs the porn where it's anything that turns them on.
Cloudflare Stream takes a lot of the headache out of self hosting video. Highly recommended.
If you’re using Cloudflare Stream you’re not “self-hosting”. You are just using a video platform that isn’t YouTube.
It would be great if <video> tag element could work some things out. There are alternatives to this that work better. Still, the tag is broken. There's nothing we can do up until it updates sometimes.
I like to use YouTube so I have a standard player across all browsers and platforms. I never know what the user is going to see when I use <video>.
I don't know, I use the video tag for videos served from an S3 bucket with great performance. Way better than YouTube for videos under 5 min. Seems to be pretty easy. Maybe the author's issues are more for long videos?
Most websites can't even optimize images, how are they going to optimize video based on the devices?
Youtube is the best solution.
Video/audio tags works as good as image tags.
Youtube is the best solution.
Video/audio tags works as good as image tags.
Any kind of video is just a pia. I avoid it at all costs which is why youtube is so popular - video is just annoying.
This seems like a fairly narrow use case to go back to the drawing board over. I just wish Chrome supported h265
H265 is a patent and licensing mess; the industry came up with AV1 instead and it’s better quality at the same bitrate, and already in Chromium and Firefox.
I think apple shipped H265 but they are also part of the AV1 consortium
I think apple shipped H265 but they are also part of the AV1 consortium
> Maybe I’m asking for a faster horse here
Yes.
Keeping things simple is arguably better. MP4 is wonderful. MP3 is also wonderful. Most browsers use these well established codecs as baseline. A few matchmedia queries and rather unreliable navigator - et voilà - my method.
Yes.
Keeping things simple is arguably better. MP4 is wonderful. MP3 is also wonderful. Most browsers use these well established codecs as baseline. A few matchmedia queries and rather unreliable navigator - et voilà - my method.
> You also have to spend time learning and integrating a complicated new library into your documents.
I literally hacked together from scratch a simple HLS stream player in half an hour using VideoJS, not having worked with that one (or video streaming in browsers) before at all, and I'm a backend/ops-focused person. Further two hours saw the addition of a basic playlist for the stream recordings of nginx-rtmp.
I agree that the HTML5 video element lacks features, but the existence of VideoJS alleviates a lot of that.
I literally hacked together from scratch a simple HLS stream player in half an hour using VideoJS, not having worked with that one (or video streaming in browsers) before at all, and I'm a backend/ops-focused person. Further two hours saw the addition of a basic playlist for the stream recordings of nginx-rtmp.
I agree that the HTML5 video element lacks features, but the existence of VideoJS alleviates a lot of that.
The reason small websites prefer YouTube embeds is because they want to avoid bearing the storage and bandwidth costs of serving it. And the logistical costs of transcoding it to N different resolutions times M different codecs. No amount of improvements to HTML alone will change that. (Not that the changes are entirely worthless, but they still do not and cannot address the real issue.)