Time.gif(hookrace.net)
hookrace.net
Time.gif
https://hookrace.net/blog/time.gif
110 comments
I expect that you were using this weird corner of the HTTP spec that is X-Mixed-replace[1]; where the server can keep feeding responses for the same request and the browser is supposed to replace the element with the new-one. A lot of IP cameras are using this to stream JPEG to the browser.
This technology is getting phased out and mostly only works with images.
[1]: https://en.wikipedia.org/wiki/MIME#Mixed-Replace
This technology is getting phased out and mostly only works with images.
[1]: https://en.wikipedia.org/wiki/MIME#Mixed-Replace
bugzilla still uses it to show the "Please wait while your bugs are retrieved." page while executing a search to then replace the page with the results.
What's the replacement?
What's the way to live stream video without javascript?
You can stream video with the HTML5 <video> tag.
Video tag pointing to an MP4 will use byte range requests in chrome, which is pseudo streaming. For real streaming look into something like rtmp or hls, which does require js in some browsers. The main difference is one can handle live feeds the other cannot to my knowledge. Also real streaming has adaptive bitrate (switch to lower quality to avoid stuttering and buffering). The distinction between streaming and pseudo streaming is blurry. My litmus test is it can it handle live feeds? Html5 video tag cannot without hacks, per my understanding
Our app does something like this by loading an image with an xhr then immediately requesting the next image when the previous one finishes
Our app does something like this by loading an image with an xhr then immediately requesting the next image when the previous one finishes
Multipart+JPEG is easily 5x more expensive than MP4 in terms of bandwidth. All new IP cameras come with MP4 as a replacement. Steaming can be done in various ways, usually with the help of some JavaScript + video tag.
This sounds super interesting; a bunch of questions:
Did you just sent multiple images concatenated into the output stream, including file headers? And Mosaic would actually replace the first image with the second and so on? Was the CGI script referenced in an <img> tag?
What image format was this?
Did you just sent multiple images concatenated into the output stream, including file headers? And Mosaic would actually replace the first image with the second and so on? Was the CGI script referenced in an <img> tag?
What image format was this?
Probably jpgs. This used to be how all streaming “webcams” worked until not that long ago
I don't remember the details.
I think they were GIFs. From what I recall, I read the entire images as they were on disk and pushed them down the stream.
Someone mentioned https://en.wikipedia.org/wiki/MIME#Mixed-Replace and that is ringing a tiny bell.
Honestly, it was 23 years ago and I forgot the exact details. I still remember the meeting though. The feeling of dread, excitement, and ridiculousness of knocking out such a powerful machine (at the time) in order to server animated smiley faces.
Nowadays, I knock out powerful machines doing slightly more useful things.
I think they were GIFs. From what I recall, I read the entire images as they were on disk and pushed them down the stream.
Someone mentioned https://en.wikipedia.org/wiki/MIME#Mixed-Replace and that is ringing a tiny bell.
Honestly, it was 23 years ago and I forgot the exact details. I still remember the meeting though. The feeling of dread, excitement, and ridiculousness of knocking out such a powerful machine (at the time) in order to server animated smiley faces.
Nowadays, I knock out powerful machines doing slightly more useful things.
Oh the 90s...
I have often asked in interviews if my job title could be Web Master for nostlolgia.
I never get that title :(
I have often asked in interviews if my job title could be Web Master for nostlolgia.
I never get that title :(
This was what we did before gifs. Then gifs came along and we stopped. Funny to see it again... as an innovation! Remember history, because it repeats.
Oh yes, server push animation. I checked it out a few years ago, and it seems to still work in Mobile Safari:
http://pronoiac.org/misc/2009/10/server-push-animation/
http://pronoiac.org/misc/2009/10/server-push-animation/
how can such an old browser run a C program embedded within it? Can someone elaborate how this could have been done
NCSA httpd introduced the Common Gateway Interface which allows you to execute any binary, and whatever it puts on stdout is sent as the reply. Back then it was typically a Perl script or a compiled C binary.
https://en.m.wikipedia.org/wiki/Common_Gateway_Interface
https://en.m.wikipedia.org/wiki/Common_Gateway_Interface
My company makes a DAS(distributed Antenna System) product with an embedded web interface. All of our web sites have C/C++ cgi back ends. There are more makefiles in the Web folder than the whole rest of system. It is quite mind boggling. But the engineer that has control over this area is stuck in the 90s.
My main objection to such systems is security. It is difficult enough to take care of it in modern frameworks, but in C?
Never saw /cgi-bin/ in a form URL? Or the occasional xycorp_isapi.dll? :)
Found the old guy :). Back then it was quasi-normal to just write your own HTTP server. A TCP socket that just slapped some headers on whatever it was reading from disk. Now everything is so complex...
I remember using server push, and the CGI interface was a hell of a lot simpler than the 8000 levels of abstraction my Typescript Angular apps going through before they hit the wire.
The worst part is that the majority of the added complexity is useless. Instead of replacing the bad parts of HTTP, HTML, JS, and CSS we just build more shit on top of it. This requires more hacks and just up the complexity level even more.
I remember using server push, and the CGI interface was a hell of a lot simpler than the 8000 levels of abstraction my Typescript Angular apps going through before they hit the wire.
The worst part is that the majority of the added complexity is useless. Instead of replacing the bad parts of HTTP, HTML, JS, and CSS we just build more shit on top of it. This requires more hacks and just up the complexity level even more.
“The cgi-bin C program”
That's server side C code speaking HTTP using the common gateway interface protocol.
That's server side C code speaking HTTP using the common gateway interface protocol.
I did something similar long time ago with multipart/x-mixed-replace content type. http://zabasoft.xf.cz/clock.gif
Nice. And this one works well on Safari ! Which language did you use?
Seems that it works on Safari because it uses chunked encoding, and time.gif does not use it.
Usually http video and audio streaming uses chunked encoding.
Usually http video and audio streaming uses chunked encoding.
This is absolutely awesome.
I wonder if this can be used to stream regular GIFs, so that you don't have to wait for the whole thing to load before seeing the animation.
I'm currently working on a project focusing on GIFs (https://www.gifsonic.com), I'll definitely study this to see if it's possible.
I wonder if this can be used to stream regular GIFs, so that you don't have to wait for the whole thing to load before seeing the animation.
I'm currently working on a project focusing on GIFs (https://www.gifsonic.com), I'll definitely study this to see if it's possible.
That is how my browser behaves normally in Chrome at least. Progressive loading of normal .gifs, playing frames as they are available
Same for me FF54 on Kubuntu.
Sometimes it gets stuck and doesn't show the whole gif but I just assume that's my/the website's connection.
Sometimes it gets stuck and doesn't show the whole gif but I just assume that's my/the website's connection.
Sure, but the animation sucks. Extremely choppy and slowed down.
I wonder if this technique can optimize preloading.
For instance, we could send out only 1/2 of the frames (of course, with twice the delay to keep the same animation speed), then load the rest as it's available.
I have no idea what I'm talking about yet, but there seem to be something here.
I wonder if this technique can optimize preloading.
For instance, we could send out only 1/2 of the frames (of course, with twice the delay to keep the same animation speed), then load the rest as it's available.
I have no idea what I'm talking about yet, but there seem to be something here.
I would prefer, as UX, that it would follow this order:
- First load first frame on lowres (may be so lo that is blurred)
- Then add a small rotating loading icon on top of image, possibly with a notation that is a GIF
- Then load first frame completely at correct resolution
- Then load all the other frames in the background, only showing me the first frame still
- When all frames are completely loaded, remove loading rotating icon and run the GIF normally.
- First load first frame on lowres (may be so lo that is blurred)
- Then add a small rotating loading icon on top of image, possibly with a notation that is a GIF
- Then load first frame completely at correct resolution
- Then load all the other frames in the background, only showing me the first frame still
- When all frames are completely loaded, remove loading rotating icon and run the GIF normally.
Awesome suggestions. I'll try to see if it's possible.
It seems like this is the same as with video, where the client is trying to determine how many frames it needs to precache before it starts displaying. I'm not sure how much measurement capacity the server has, here?
I did the same thing about fifteen years ago by generating a mJPEG stream from a PHP-script. As far as I can remember, it seemed more reliable than using GIF. See https://en.wikipedia.org/wiki/Motion_JPEG
MJPEG is indeed much better for this. It uses less bandwidth (GIFs have no compression) and less browser memory (old frames can be discarded). When I was at globo.com, we developed this to serve animated thumbnails for live video streams: https://github.com/jbochi/live_thumb
GIF features lossless LZW compression.
The main advantage is that, because a video format rather than an image that can loop, the player can discard past frames. The GIF method is probably constantly leaking memory.
> The GIF method is probably constantly leaking memory.
If the browser only keeps compressed frames, 16 GB will last for more than a year.
If the browser only keeps compressed frames, 16 GB will last for more than a year.
GIFs can be non looping, but I'm not sure whether that flag is set at the beginning or end of the format.
yeah was wondering at what point chrome would give up, but the gif is only around 250kb per minute (0.004Mb/s)
This technique used to be used to embed "videos" into html emails years ago. Haven't seen it used in a long time maybe email clients block the streaming behavior now (for good reason).
Tried to make it output with chunked-encoding:
- In Chrome it shows nothing (looks like it waits for end of stream)
- In Safari it shows only the first frame
- In Firefox it works (shows animated clock)
Branch with chunked encoding: https://github.com/kolen/time.gif/tree/chunked-encoding
- In Chrome it shows nothing (looks like it waits for end of stream)
- In Safari it shows only the first frame
- In Firefox it works (shows animated clock)
Branch with chunked encoding: https://github.com/kolen/time.gif/tree/chunked-encoding
This is really cool! Completely suboptimal but fascinating none the less. Great project and idea :) would be interested in hearing some people's ideas for real applications of this
One use case for automatically adapting/live updating gifs is emails since HTML in emails can only have a very limited subset of functionality.
One can use automatically generated gifs to add things like countdown timers based on the opening time of the email. (i.e. loading time of the image) See for example this post: https://litmus.com/community/learning/27-how-to-add-a-countd...
One can use automatically generated gifs to add things like countdown timers based on the opening time of the email. (i.e. loading time of the image) See for example this post: https://litmus.com/community/learning/27-how-to-add-a-countd...
My company uses the same technique at scale to serve animations into emails, often at thousands of gifs generated per second and streamed to the user. Interestingly, when you stream gifs frame-by-frame, some size optimizations are not available to you but there are plenty that are. (cropping, transparency, etc)
This won't work with gmail, the images are precached.
Actually, that's not true. GMail proxies images, but doesn't do a lot in the way of caching. They actually cache the DOM when moving between messages for a short while, but a browser reload will reset it.
Been working for 3 years in the past in an email isp I had different information, gmail did cache the images and did it "forever". May be this changed recently?
More info: https://litmus.com/blog/gmail-adds-image-caching-what-you-ne...
More info: https://litmus.com/blog/gmail-adds-image-caching-what-you-ne...
[deleted]
[deleted]
So this was done during a programming paradigms course at KIT? I'm teaching functional programming basics using Haskell at my university. At the end of the semester we aren't quite that far. But I am far from sure whether this exercise, just looking at the given types, has didactic value and I wonder how much of the stuff the instructor has given and how much the students worked out themselves given the course materials.
I would be a bit hesitant (teaching-wise), to talk about any function of the type IO () -> IO Char -> (Frame -> IO ()) -> IO ()
Although, not to be a spoil sport, it seems like a fun thing to do over a long weekend.
I would be a bit hesitant (teaching-wise), to talk about any function of the type IO () -> IO Char -> (Frame -> IO ()) -> IO ()
Although, not to be a spoil sport, it seems like a fun thing to do over a long weekend.
This is just using the framework that I made for the exercise. The students didn't have to do any Crazy IO stuff. You can see the parts under "Aufgabe" here, which is what they were supposed to code: https://github.com/def-/gifstream/blob/master/SnakeFinished....
Exercises in German here: https://github.com/def-/gifstream/blob/master/Aufgabe.pdf
Exercises in German here: https://github.com/def-/gifstream/blob/master/Aufgabe.pdf
This seems interesting. Maybe I can use something similar next year to generate some interest. Without IO (which is monads, which is too advances since we also do functional C# and functional java paradigms in the same course) the subject matter is always a bit dry since the students have nothing but the ghci repl to get any interaction or feedback going.
What if you were to give students some functions like
imShow :: [[Int]] -> IO ()
playSound [Int] -> IO ()
You could also build a "sandbox" monad specifically for the class/framework. Let the students treat the monad as a blackbox Turing machine/state machine, unless they get time/interest/show aptitude to dive deeper. Doing that gives you the benefit that they don't accidentally google "Haskell IO" and fall down the monad rabbit hole without some preparation.
The most fun exercise for me was generating music in Haskell, but it was quite challenging and I remember not many students tried it.
I wonder whether you can even hide the IO in the logic function by providing a random number from the outside which the students can use, so that from their perspective the logic function is pure, which would be quite desirable.
see also raytracing clock on PNG: http://www.ioccc.org/2013/mills/hint.html
I know I'm a bit fatigued... but where / how do I download the doflickymabob to make this go?
The source (and Makefile which defines the camera/light coord. see hint.html) for the winning IOCCC entries are available in the parent directory.
http://www.ioccc.org/2013/mills/
It might be more convenient to download the annual winning entry tarball.
http://www.ioccc.org/2013/2013.tar.bz2
Alternatively, try the index for all of the winning entry files.
http://www.ioccc.org/years.html
edit: By the way, if you download 2013.tar.bz2, I highly recommend checking out the cable3, which implements an entire IBM PC capable of running Windows 3.0 in only 4043 bytes (8086 nibbles).
http://www.ioccc.org/2013/cable3/hint.html
http://www.ioccc.org/2013/mills/
It might be more convenient to download the annual winning entry tarball.
http://www.ioccc.org/2013/2013.tar.bz2
Alternatively, try the index for all of the winning entry files.
http://www.ioccc.org/years.html
edit: By the way, if you download 2013.tar.bz2, I highly recommend checking out the cable3, which implements an entire IBM PC capable of running Windows 3.0 in only 4043 bytes (8086 nibbles).
http://www.ioccc.org/2013/cable3/hint.html
> edit: By the way, if you download 2013.tar.bz2, I highly recommend checking out the cable3, which implements an entire IBM PC capable of running Windows 3.0 in only 4043 bytes (8086 nibbles).
People interested in that emulator may also want to know that the non-obfuscated version (still only 760 physical lines of C, though that's partly achieved by putting opcode information in the BIOS ROM) has moved to GitHub:
https://github.com/adriancable/8086tiny
People interested in that emulator may also want to know that the non-obfuscated version (still only 760 physical lines of C, though that's partly achieved by putting opcode information in the BIOS ROM) has moved to GitHub:
https://github.com/adriancable/8086tiny
When you are running out of time, but still got a decent video player ready:
$ mpv --cache=no https://hookrace.net/time.gifThis reminds me of a project I've created a few years ago that does live video streaming with endless GIFs: https://github.com/jbochi/gifstreaming
Very interesting from a tech perspective, make sure you lazy load the gif if you plan on using it after window load event otherwise it's never going to happen and you'll just get the endless spinner :D
eBay sometimes sends out emails with auction countdowns in them that do something similar.
The first time that I saw an accurate countdown ticking away in an email, it surprised the heck out of me.
The first time that I saw an accurate countdown ticking away in an email, it surprised the heck out of me.
You can do it too with services like motionmail.com
Hey, cool, I wanted to do the exact same thing the other day but as a countdown clock. For those saying this has no practical use, a countdown definitely does :)
Gifs render in email, so you could put a countdown there, for one example
For a countdown, you don't need to stream it, just generate animated countdown gif.
Pretty interesting! Increases in size by ~4KB every second though. Probably would take a toll on your browser if you left it open for a while. :)
The server could finish sending frames and let the gif loop after 12/24 hours have elapsed.
This would fail to account for things like leap seconds
For 24 hours, that's still a 345MB image :)
Looking at the code and the actual file generated, it seems that the image is not compressed at all, using one byte per pixel. It also defines a palette of 128 colors, which is mostly unused. A bit of compression would drastically reduce the image size, this kind of monochrome images of axis-aligned blocks compress very well !
This is great. If I were to serve this over apache is there a limit on the connections? I'm asking because I'm assuming there is only a specific number of concurrent connections per child in apache(?).
So if gif 2 is embedded in 100 sites will it bring my server down since children are not closing connections?
So if gif 2 is embedded in 100 sites will it bring my server down since children are not closing connections?
I had problems with Haskell after 1024 open sockets, but compiling with ghc -threaded fixed them. Other than that, no problems encountered yet. (using nginx as proxy)
Doesn't seem to work in iPhone safari. Displays the time you loaded the page but doesn't update.
Too bad. I only tested with Firefox, Chrome and Android. You could try http://hookrace.net:5002/ directly
Edit: I have now added some cache-control directives, maybe they help
Edit: I have now added some cache-control directives, maybe they help
It worked on my iPhone the first time I tried it. But after reloading, it doesn't seem to work anymore. Maybe slow or too much connection?
Works fine on iPad Safari.
Likewise with desktop Safari.
Also check out https://github.com/ErikvdVen/php-gif
It generates real-time GIF images with PHP, such as countdown, etc.
Am still waiting for it to finish looping ;)
There's probably a cheap optimization in there somewhere. Diffing frames on each tick, but for that size it probably doesn't matter. Thanks for the link to GifStream!
There's probably a cheap optimization in there somewhere. Diffing frames on each tick, but for that size it probably doesn't matter. Thanks for the link to GifStream!
[gifsockets](https://github.com/videlalvaro/gifsockets) is another version of this
The code is constantly 1 second behind.
> It is written in Haskell and works by dynamically generating each frame of the GIF and slowly feeding them over the HTTP connection.
This seems rather difficult. These are dynamically generated. There are 246060=86400 possible slides, but at download time, we have to find the correct starting point. Did I get that right?
This seems rather difficult. These are dynamically generated. There are 246060=86400 possible slides, but at download time, we have to find the correct starting point. Did I get that right?
What would the difficulty be? You pick the first image by looking at the server time then send the next frames once per second...
You may optimize it by sending diffed frames (IIRC the gif format allows one to only update the pixels that change from one frame to the next). You could even pre-generate both sets of images (full frame and diff) and use the script to send the right frame at the right time.
You may optimize it by sending diffed frames (IIRC the gif format allows one to only update the pixels that change from one frame to the next). You could even pre-generate both sets of images (full frame and diff) and use the script to send the right frame at the right time.
No, it's just like video or audio streaming, but for gifs.
Source code for this program is very small, despite it even encodes gif on its own and does http on its own, without using external libraries.
Source code for this program is very small, despite it even encodes gif on its own and does http on its own, without using external libraries.
finally useful haskell project
[deleted]
McAuliffe Counters Critics of Police Response to Charlottesville Violence
http://www.scout.com/Board/102743/Contents/fRee-Colts-Vs-Lio...
I don't understand why this is getting so much attention. From the description:
"[It] works by dynamically generating each frame of the GIF and slowly feeding them over the HTTP connection."
So there is nothing (AFAICT) new or interesting going on here. It's just an animation generated by the server in real time that happens to be formatted as a gif. It's no different from what many low-end web-enabled security cameras do.
"[It] works by dynamically generating each frame of the GIF and slowly feeding them over the HTTP connection."
So there is nothing (AFAICT) new or interesting going on here. It's just an animation generated by the server in real time that happens to be formatted as a gif. It's no different from what many low-end web-enabled security cameras do.
No, low end security cameras send separate JPEG images (sometimes as a MJPEG stream), not GIF frames. GIF frames would be terrible for the cameras because of the limited colour palette.
OK, well, that's true. But is sending an animated gif instead of jpegs really that much of an innovation? If I did the same thing using, say, APNGs would HN go crazy over that?
Nobody is going crazy, they are simply learning things. If you've got something to teach, HN will probably like it.
Though I'm not sure why you even care what HN likes. It certainly isn't measured in units of innovation, as you seem to think it should.
Though I'm not sure why you even care what HN likes. It certainly isn't measured in units of innovation, as you seem to think it should.
Maybe "going crazy" was overstating it. But it has >350 upvotes, and a lot of comments like this:
https://news.ycombinator.com/item?id=14997202
"This is absolutely awesome."
I'm just trying to understand why people think it's "absolutely awesome." It seems, at best, mildly interesting to me.
https://news.ycombinator.com/item?id=14997202
"This is absolutely awesome."
I'm just trying to understand why people think it's "absolutely awesome." It seems, at best, mildly interesting to me.
This is textbook middlebrow dismissal.
Let's turn it around. Why do you think it's so important that we understand how unimpressed and uninterested you are?
This is a simple, accessible project that people think is fun. They upvoted it.
Let's turn it around. Why do you think it's so important that we understand how unimpressed and uninterested you are?
This is a simple, accessible project that people think is fun. They upvoted it.
OK.
interesting concept, but not very practical compared to www.time.is
_Waits for the Node.js implementation_
it is one of those thing you do because you can do that. If you try to open the GIF only it won't work well and if you try to save it as image it won't work.
for me it has no real application that I can think of, it consume lot of bandwidth and processing even for small size gif.
good knowledge for developer, I certainly cannot even think in that line. But no use of it in current form.
for me it has no real application that I can think of, it consume lot of bandwidth and processing even for small size gif.
good knowledge for developer, I certainly cannot even think in that line. But no use of it in current form.
I wrote a C program which would load 6 frames of a smiley face animation and would feed them sequentially, in an endless loop, to anyone who requested the image. Mosaic was happy to animate them.
I get called in to the web mistress's office. The web server is down. We had a donated Silicon Graphics Origin server, if my memory serves me correctly. This was a beefy machine.
The cgi-bin C program would load 6 images, as fast as it possibly could, and dump them into a network socket. It would not throttle. It would not check if the client disconnected. It had one job. It did it efficiently, and ruthlessly.
Poor server.