Lytro Announces New Light-Field Camera Photo Features(allthingsd.com)
allthingsd.com
Lytro Announces New Light-Field Camera Photo Features
http://allthingsd.com/20121115/with-new-features-lytro-aims-to-show-its-futuristic-camera-is-no-one-trick-pony/
23 コメント
Man, tough crowd. I have a Lytro, and I love it. I'm quite a fan of the technology. It's not perfect, and it's uses are (for now) somewhat limited, but I feel your criticism of it may be a little overly harsh.
First off, the current Lytro model does not, and will not, replace any traditional camera I own. I like the technology, and it has it's uses even beyond demoing the refocusing abilities, but it's definitely a supplement to my camera arsenal and not a replacement for anything.
You're right that you can "simulate" focus-stacking with a traditional camera, as long as the object you're shooting is sitting still and there's no motion of the camera or in the frame at all. Any motion at all and your simulated focus stacking doesn't hold up well.
Likewise, you can get a perspective shift from multiple images by slight movement with a traditional camera. Assuming again that no actual motion happens and that no observable length of time transpires between the successive offset shots.
Since the Lytro doesn't have these time limitations, and since focus can be done after the fact, I've been able to get a couple of shots of events that I could never get with a traditional camera, just based on the ability to point the camera and snap the shutter without having to even wait for autofocus to happen.
The hardware in the camera to me feels far ahead of the software that drives it. One of the biggest criticisms I have of the Lytro is that the camera captures a vast amount of light field information, and through The Appropriate Application of Mathematics, you should really be able to do a lot more with the captured pictures than you can now. I remain hopeful that as the camera software progresses that more of these possibilities will come to light, and so far other than the time involved to develop the software I haven't been disappointed - there have been pretty regular software updates, and features are being incrementally added.
The Lytro isn't perfect - I'll grant you that - but it's a radical change in technology, and I think that over time the technology can only get better. The price for the current model is a little higher than "what the heck, I'll try one out" typically justifies.
One thing I use my camera for - more than the refocusability - is macro photography. The Lytro has an absolutely insane macro focus capability. It can focus right down to the front of the exterior lens, and it does a great job at shooting small objects at short range - better than any SLR macro lens I have, and I have a couple of "not cheap" ones to compare it to.
First off, the current Lytro model does not, and will not, replace any traditional camera I own. I like the technology, and it has it's uses even beyond demoing the refocusing abilities, but it's definitely a supplement to my camera arsenal and not a replacement for anything.
You're right that you can "simulate" focus-stacking with a traditional camera, as long as the object you're shooting is sitting still and there's no motion of the camera or in the frame at all. Any motion at all and your simulated focus stacking doesn't hold up well.
Likewise, you can get a perspective shift from multiple images by slight movement with a traditional camera. Assuming again that no actual motion happens and that no observable length of time transpires between the successive offset shots.
Since the Lytro doesn't have these time limitations, and since focus can be done after the fact, I've been able to get a couple of shots of events that I could never get with a traditional camera, just based on the ability to point the camera and snap the shutter without having to even wait for autofocus to happen.
The hardware in the camera to me feels far ahead of the software that drives it. One of the biggest criticisms I have of the Lytro is that the camera captures a vast amount of light field information, and through The Appropriate Application of Mathematics, you should really be able to do a lot more with the captured pictures than you can now. I remain hopeful that as the camera software progresses that more of these possibilities will come to light, and so far other than the time involved to develop the software I haven't been disappointed - there have been pretty regular software updates, and features are being incrementally added.
The Lytro isn't perfect - I'll grant you that - but it's a radical change in technology, and I think that over time the technology can only get better. The price for the current model is a little higher than "what the heck, I'll try one out" typically justifies.
One thing I use my camera for - more than the refocusability - is macro photography. The Lytro has an absolutely insane macro focus capability. It can focus right down to the front of the exterior lens, and it does a great job at shooting small objects at short range - better than any SLR macro lens I have, and I have a couple of "not cheap" ones to compare it to.
Very good points. I especially like the macro aspect.
To qualify my remarks: I would not propose that anyone takes several images manually or a photographer moves the camera around physically as an alternative.
I'm suggesting that competitors (mainstream cameras) could take multiple pictures in rapid succession INTERNALLY (very feasible in lower resolutions). The canon 7D can take videos at 720p @ 60FPS. Higher framerates seem likely in the future. Once one reaches 120FPS, slight variations in the focus whilst shooting 1s of video would result in acceptable shutter speeds (1//120th) for most moving objects and (theoretically) a stack of 120 pictures with different focal points that could be "merged" into a "lytro-like" image through in-camera focus-stacking and some "hotspot" mechanism.
Similarly (but less likely) combined with a quick physical circular movement of the sensor, or a lens element (just imagine a "Lytro-lens" for your Canon) would allow us to emulate (match?) the other effect.
Perhaps no movement is needed at all if the Canon Lytro Lens had an internal "microlens" array that subdivided the existing sensor, or uses the "pattern filter" technique.
My point: Lytro is nice, but has limited usability (for me at least) and I see it merely as a stepping stone towards inclusion of similar features in the mainstream. As a company they will have a very tough time defending whatever patents they might have. Theory and technology go back many many years.
To qualify my remarks: I would not propose that anyone takes several images manually or a photographer moves the camera around physically as an alternative.
I'm suggesting that competitors (mainstream cameras) could take multiple pictures in rapid succession INTERNALLY (very feasible in lower resolutions). The canon 7D can take videos at 720p @ 60FPS. Higher framerates seem likely in the future. Once one reaches 120FPS, slight variations in the focus whilst shooting 1s of video would result in acceptable shutter speeds (1//120th) for most moving objects and (theoretically) a stack of 120 pictures with different focal points that could be "merged" into a "lytro-like" image through in-camera focus-stacking and some "hotspot" mechanism.
Similarly (but less likely) combined with a quick physical circular movement of the sensor, or a lens element (just imagine a "Lytro-lens" for your Canon) would allow us to emulate (match?) the other effect.
Perhaps no movement is needed at all if the Canon Lytro Lens had an internal "microlens" array that subdivided the existing sensor, or uses the "pattern filter" technique.
My point: Lytro is nice, but has limited usability (for me at least) and I see it merely as a stepping stone towards inclusion of similar features in the mainstream. As a company they will have a very tough time defending whatever patents they might have. Theory and technology go back many many years.
Let me call out this variation on which was included on the third link in my original response:
http://www.youtube.com/watch?feature=player_embedded&v=Z...!
Also look at the site by the author of said video: http://www.tgeorgiev.net/
http://www.youtube.com/watch?feature=player_embedded&v=Z...!
Also look at the site by the author of said video: http://www.tgeorgiev.net/
It does have limited usability... for now. One review of the Lytro commented that using one was "beta testing the future of photography" and that's really how I view it.
I bought one not for what the current camera could do at the time, but in the hopes that they'd be successful enough to come up with the Lytro 2 which did more.
I bought one not for what the current camera could do at the time, but in the hopes that they'd be successful enough to come up with the Lytro 2 which did more.
To my knowledge, accurately building and calibrating a stereoscopic camera is a significant pain. Saying you could just wave your phone around and extrapolate a 3D scene is easily 10+ years out, because people make terrible tripods, and the requirements for planarity, etc. to do stereoscopic well are pretty strict. That said, there are 3D phones with parallax screens and stereo cameras; they're big and bulky and not that great.
The cool thing I saw about Lytro isn't their flash based player. Its the idea of giving you another degree of freedom after a picture is taken. Like colour correction, you can do focus correction, DOF correction, etc. I would still export one flat image as a result, but gaining that flexibility without carrying around a monopod to take 10 bracketed images is awesome to me.
The cool thing I saw about Lytro isn't their flash based player. Its the idea of giving you another degree of freedom after a picture is taken. Like colour correction, you can do focus correction, DOF correction, etc. I would still export one flat image as a result, but gaining that flexibility without carrying around a monopod to take 10 bracketed images is awesome to me.
See my response above. Did not mean to suggest we should all start waving our cameras. Something could "wave" internally, or this waving isn't even needed of Canon came out with a microlens-adaptor.
By the way, there are several apps already which build 3D scenes from two separate pictures. Some can even generate 3D models. HDR used to require manual exposure bracketing in phones too. Now technology has caught up.
Correct me if I'm wrong, but the Lytro stereoscopic field looks fixed at a relatively short distance. At the moment it seems to be a very subtle parallax effect, one that might be mimicked by an ever so slight movement of a camera sensor.
By the way, there are several apps already which build 3D scenes from two separate pictures. Some can even generate 3D models. HDR used to require manual exposure bracketing in phones too. Now technology has caught up.
Correct me if I'm wrong, but the Lytro stereoscopic field looks fixed at a relatively short distance. At the moment it seems to be a very subtle parallax effect, one that might be mimicked by an ever so slight movement of a camera sensor.
I'd love to have that freedom... combined with everything else that I've come to expect from a DSLR. Lytro's cameras are nice novelty gadgets but it's not clear to me if they're trying to move in the direction of working with a sensor/DSLR maker to enable this. I think that's what they should focus on...
I gave this some thought below, but unfortunately fitting a new, more expensive sensor is probably not going to happen soon. The current 1.8 megapixel is a cheap commodity sensor, and it's still probably costing them buckets to have them fabbed with their custom microlenses. Something like a Kodak 8MP sensor would probably be 20x as expensive, and talking them into making you a custom SKU would be much harder.
Edit: after a bit of digging, it looks like they make the micro lens array themselves and buy the sensors without one. This obviously makes acquisition easier, but as the pixel pitch decreases ( by an order of magnitude, to compete with a modern SLR ) the complexities of building and attaching that array increase significantly.
Edit: after a bit of digging, it looks like they make the micro lens array themselves and buy the sensors without one. This obviously makes acquisition easier, but as the pixel pitch decreases ( by an order of magnitude, to compete with a modern SLR ) the complexities of building and attaching that array increase significantly.
I think Lytro lost some of the magic for me when I realized it was just utilizing a depth map to generate an iris blur. Looks like the same deal for the 3d effect, around edges you can see reconstruction artifacts. I'm impressed by the fact one lens can capture depth, but with two lenses at a narrow aperture they could create the same effect in higher resolution and quality.
My understanding is that the camera's file format is essentially a stack of JPEGs and a depth map. That's how the camera can do it's own refocusing without needing a huge processor. It's not generating the image from the light field, it's combining regions of a series of JPEG images.
I was really disappointed when I found that out. I thought the software on the computer was to process the raw data, but I guess not.
I still might be interested, but the the form factor is odd. Worse is that since they decided to go with that shape, they had to make the screen very very tiny.
It's an interesting product, considering it's the first of it's kind on the consumer market. I hope there is a version 2, because I think it could be very interesting. If it could shoot video, that could be pretty fascinating.
Heck, tell other people how to use the format. A series of focal planes and a depth map? Seems like hackers might be able to come up with some pretty fun stuff.
I was really disappointed when I found that out. I thought the software on the computer was to process the raw data, but I guess not.
I still might be interested, but the the form factor is odd. Worse is that since they decided to go with that shape, they had to make the screen very very tiny.
It's an interesting product, considering it's the first of it's kind on the consumer market. I hope there is a version 2, because I think it could be very interesting. If it could shoot video, that could be pretty fascinating.
Heck, tell other people how to use the format. A series of focal planes and a depth map? Seems like hackers might be able to come up with some pretty fun stuff.
This is actually not the case. I wrote a tool to pull apart the file format: https://github.com/nrpatel/lfptools
The image is stored as a raw Bayer array on the camera. The conversion to stacked JPEG occurs on the computer. The camera has a pretty beefy DSP on board to do live refocusing on the preview.
The image is stored as a raw Bayer array on the camera. The conversion to stacked JPEG occurs on the computer. The camera has a pretty beefy DSP on board to do live refocusing on the preview.
The camera's raw format is indeed an entire light field (I've used nrp's tools, they're great). The camera import software creates a layered JPEG out of the raw data for uploading and sharing, since the size and the amount of math involved in refocusing the raw image on the fly is significant.
It looks like their viewer is JavaScript-based not flash (this may be new, I'm not sure)
Looked flash-based to me; I had to click to allow Flash to play with their new filter effects anyhow.
As someone who works with robotics, I can imagine the applications of a real-time, video version of this. Stereovision would become much simpler and more reliable.
The interesting thing about Lytro is not so much it's current offering, but the potential the technology unlocks.
I would be more excited about a Lytro that would record a second or so of full high speed video (1000fps would be ideal, but it'd probably still be useful with much less), so that you could adjust both the focus and the exposure after the fact. Eliminating the need to take a bunch of photos in a row to get a "good one" would be a legit improvement.
That and fitting it into a phone. :)
That and fitting it into a phone. :)
I would be interested in knowing if their tech could ever be phone-sized. It looks like focusing the light correctly on the sensor is more difficult than with a conventional camera ( their promo video looks to show about 8 lens elements). The entire design is basically a big lens with the sensor and screen on the back, that's why they adopted such a strange form factor. That said, there is significant variability in size with traditional camera lenses, so maybe once the market matures someone will figure out a way to produce cheap plastic light-field lenses.
As far as the high-fps goes, they use a standard sensor with custom microlenses. Procuring a new sensor, either for higher resolution or faster frame rates, is probably more expensive and less likely than continued software updates, and maybe even changes to the enclosure and lenses. I suspect sensor acquisition is a significant part of these cameras high price.
As far as the high-fps goes, they use a standard sensor with custom microlenses. Procuring a new sensor, either for higher resolution or faster frame rates, is probably more expensive and less likely than continued software updates, and maybe even changes to the enclosure and lenses. I suspect sensor acquisition is a significant part of these cameras high price.
people don't have to invent bullshit explanations of how they think this stuff works - the thesis is here http://www.lytro.com/renng-thesis.pdf
it is more than a stack of images.
it is more than a stack of images.
If you were referring to my post. I wasn't explaining how this works (I know how it works). I was outlining how I think the current effects/benefits to the consumer can be mimicked.
The current output format, which is a stack of JPEGs with a depth map, would cover 80% of the result with 20% of the effort/technology. Capturing this stack of JPEGs (e.g. as high speed video frames) is not beyond the realm of our current camera technology.
The current output format, which is a stack of JPEGs with a depth map, would cover 80% of the result with 20% of the effort/technology. Capturing this stack of JPEGs (e.g. as high speed video frames) is not beyond the realm of our current camera technology.
The focus shift feature is nice, but after seeing 50, it seems nothing more than a gimmick, and one that gets darn boring darn fast. This latest feature is cute as well, perhaps more useful even, but again not earth-shattering.
It's just a matter of time before a Samsung simulates the same thing via 'focus bracketing' and storing lots of smaller pictures (they'd probably still be bigger than the Lytro "resolution"), apply a little "focus stacking" algorithm like Photoshop to determine "focal plane" hotspots, and load the focal plane based on the selected hotspot. This little video shows a "fake lytro" image created by taking 10 DSLR pix: http://www.youtube.com/watch?v=KozKofC01_Q
This latest feature could possibly be "simulated" (equalled?) by slight physical movement (circular/conical?) of the sensor in the focal plane.
The light field technology is rather ancient by Internet standards, e.g. see this lovely video on the Stanford Light camera, from 2006 (!) which also explains both features rather well: http://www.youtube.com/watch?NR=1&v=9H7yx31yslM
Finally, if you're still interested in experimenting with this, check out a $10 solution: http://web.media.mit.edu/~raskar//Mask/
So, where does this leave Lytro?