Volume.gl – 3D Model from Single Image(volume.gl)
volume.gl
Volume.gl – 3D Model from Single Image
https://volume.gl
29 comments
That sounds interesting. What did you do and why do you think it failed?
We turned a bunch of 2D images into 3D models, for viewing online. Back in 2000.
It partially failed due to the dot com collapse sucking all the oxygen out of the room.
It also failed because the technology wasn't good enough, and the resulting 3D was, frankly, pretty shitty.
It partially failed due to the dot com collapse sucking all the oxygen out of the room.
It also failed because the technology wasn't good enough, and the resulting 3D was, frankly, pretty shitty.
Photogrammetry seems to have advanced since then. Would you agree?
Yes.
But as impressive as creating this from a single image may be, the people viewing it won't care - the output is uselessly bad.
But as impressive as creating this from a single image may be, the people viewing it won't care - the output is uselessly bad.
Volume rendering?
We did lots of ways, then.
In the interim, I worked with true volume rendering of CT, MR, PET data for a decade.
In the interim, I worked with true volume rendering of CT, MR, PET data for a decade.
Incredible. I sincerely believe advances in computer vision will be dependent on creating a 3d 'mental model' of imagery/video.
What happens when you give it images that fool people? For example the infamous Escher drawings.
What happens when you give it images that fool people? For example the infamous Escher drawings.
This was a gear deal more impressive than I expected! I tried a couple of images and it got the depth very close on one and almost perfect on the other. It's weird to be able to change the perspective of an image of a place I know and it be as accurate as it is! Kudos.
Can you link to the images you have tried? My tests were rather disappointing. Yes, I got some 3D point cloud from the image, but nothing that I would consider useful outside computer vision community for now.
I thought uploading classic POV-Ray renders would be easy fodder for it, with no noise, "perfect" lighting, simple shapes, and smooth surfaces. But it was pretty much garble for the heightfield.
They probably didn't train it on 3d renders. Training it on 3d renders would likely hurt its ability to handle real photos.
Needs quite a bit of improvement.
With the current quality, I don't see any possible application.
I agree. I am sure it would be pretty helpfull for models in the background though, which dont need to be perfect, just an approximation is usually enough. At the least its gonna look better than a billboarded image
If the application is gaming, I feel that if a prop is far enough away that this quality of model is acceptable, then you're better off just using a flat texture for efficiency. These generated 3D models aren't necessarily low-poly, despite being somewhat janky.
While I can't personally see much application for this in its current form, the technology is really impressive nonetheless. I'm very interested in seeing how far this can go.
While I can't personally see much application for this in its current form, the technology is really impressive nonetheless. I'm very interested in seeing how far this can go.
It seems sort of unnecessary to do it from a single image. Just get a little motion of the camera (just a second) and you can get way better estimates for depth.
Uploading my own images seems to be broken in Firefox 58.0.2.
Looks like they're using Event.srcElement which isn't part of the spec and Event.target has been supported since IE9. So all they would need to do is replace `event.srcElement` with `event.target || event.srcElement`. Ay!
It doesn't seem to work in Chrome either.
CORS Preflight errors in the console in Chrome
Once tech advances a bit further and we can quickly generate good 3d environments from movies, we will have so many great worlds to experience in VR - it will be amazing.
I think the first thing that will reveal is just how much detail is simply not there, or is inconsistent. E.g. consider the contortions that people often have to go through to make models of TV series sets fit the available facts because the series have not needed to pay perfect attention to room placement. Or how many rooms, or streets will simply be missing, or even walls of rooms because things are only ever shown from one angle.
It may make it easier to generate such environments, but it'll still require a huge amount of extrapolation.
It may make it easier to generate such environments, but it'll still require a huge amount of extrapolation.
Reminds me on disneys research, where you take a single picture of your smile and get a full blown 3D teeth reconstruction.
https://www.disneyresearch.com/publication/model-based-teeth...
https://www.disneyresearch.com/publication/model-based-teeth...
Having seen the state-of-the-art in action (https://arxiv.org/abs/1607.00730) I find the results pretty disappointing. Is there anything I miss?
This does seem pretty poor, I've never seen any results that were impressive from it, but in its defense any published results tend to pick images that work well.
Source: I worked really hard to make 3D less shitty, and we failed at it, and went out of business.