That means that there were 175574 HTTP GET requests, but my guess is that if someone fast-forwards that that might trigger another request. So the actual number of views may be a little lower.
I see your point now. This post originally pointed to the teaser video and not the project website (I do not know who changed that, must have been an admin): http://content.sniklaus.com/kenburns/promo.mp4
Only very few people in comparison watched the full video. Here are the logs for the content server from yesterday 5 PM PST, covering the previous 24 hours (search for /kenburns/promo.mp4): https://pastebin.com/4zTxL2aw
Thank you for sharing your thoughts! We designed the automatic camera path estimation to minimize the amount of disocclusion which indeed simplifies the problem. As you correctly pointed out, inpainting the background is an additional challenge and while we address it, the inpainted results sometimes lack texture.
It is the result of a lot of work (a lot) and I truly believe everyone in my situation with the right mindset would have been able to achieve it. Being a Ph.D. student and having a chance to do internships at companies like Adobe or Google does sometimes feel like a privilege though, so I understand that not many are in the same situation.
I just got the daily email from my server with some metrics. It has experienced over 500 gigabytes of traffic in the last few hours and the video is only 3 megabytes in size. I definitely did not expect such a HN-effect.
It's complicated but the gist of it is that they are trying to commercialize it, yes. For what it is worth, you might be able to find it in commercial software once they were successful with their business endeavors.
2. Yes and there are two mechanism for handling seams. First, the inpainting which extends the point cloud and can provide a higher sample rate. Second, a postprocessing step that heuristically fills in any seams that may still be present despite inpainting.
4. The downside of it is that one needs a lot of resources in order to try all of these variants, which not everyone is lucky enough to have access to.
I unfortunately did not get the approval to release the context-aware frame interpolation, but one of my older works is open source for research purposes: https://github.com/sniklaus/pytorch-sepconv
1. I am not sure about the scale of the individual loss functions anymore, my apologies. I determined the combination of the two losses via a simple grid search and seeing what works best (plus / minus a magnitude did not make that much of a difference).
2. The points are just splatted to an image plane, more advanced point cloud rendering techniques would be better though. There is no video frame interpolation, each individual frame in the output video is a rendering of the point cloud from a different camera perspective.
3. I am not sure about multi-view stereo, COLMAP still seems like the state of the art for that. But neural networks definitely outperform classic techniques for single image depth estimation.
4. Common architectures just did not do as well as I was hoping for so I tried about 1500 model architectures. I started with an architecture that intuitively seemed right and then gradually explored / refined alterations of it. It ultimately was a lot of trial and error.
Since I have done the work while interning at Adobe, I am afraid that I will not be able to commercialize it myself due to patenting. I agree with you that it has great potential, there are still quite a few failure cases though and it is a little hit and miss.
There definitely is room for improvement. Our proposed framework uses Mask R-CNN for object segmentation. However, Mask R-CNN only as a resolution of 28x28 pixels for the object masks if I remember correctly (which is then resized to the actual resolution of the object in the image). So there are a lot of limitations from that alone.
Yeah, we just are good at seeing artifacts in human renderings, it can easily look wrong or uncanny. In contrast, if a boulder is rendered geometrically wrong we usually do not notice it.
I was thinking about what I should post and decided to make this little teaser since it demonstrates the gist of our work within a few seconds. I understand that the typically Hacker News audience may find the paper more worthwhile though.
The effect itself is actually not too uncommon. It is just very tedious and time-consuming to do manually since it requires segmenting the elements in the scene, arranging them in a 3D space, filling in any holes and specifying a virtual camera trajectory.