Object classification through a single-pixel detector(phys.org)
phys.org
Object classification through a single-pixel detector
https://phys.org/news/2021-03-classification-single-pixel-detector.html
12 comments
Takeaway from roughly skimming the article: the single pixel detector is a plasmonic nanoantenna, which seems to be tunable. This is placed behind a set of optical diffraction gratings that have been optimized to the task using machine learning and essentially transform the spatially encoded information in the input into a spectral encoding. Both the gratings to encode the information and the decoder have been designed using machine learning. There is no dynamic illumination and the imaged object is static in front of the optics. All of this was done in the THz range.
I interpreted the result as being the convnet equivalent of implementing a multilayer perceptron network as an analog circuit (e.g. using operational amplifiers).
Presumably you could invert it as well and generate images from a single ‘color’. Create a passive spectroscope that converts the observed spectrum to text identiying the chemical compounds it represents.
Alternatively a very complicated way of building a film projector.
Hard-coded "dreaming" of a an optical "net".
The article is confusing. It appears they're leveraging implicit information in the spectrum of the light reflected from objects to settle on the object's identity.
Then they're encoding that information using the light spectrum again - but not the same wavelengths or information as they used to detect them in the first place.
So you have two things being run together. One is the identification of objects and the other is the encoding of information about those objects so that a single pixel detector is all that is needed to read what has been encoded.
Leverging the light coming off objects and processing it using diffusion filters to "learn" or "reason" about what the object is is a sort of side channel attack on the problem of object identification wrt to current approaches.
Encoding thngs into a light so only a single pixel is needed to carry information is an encoding trick.
If I misunderstood anything- entirely possible- feel free to correct.
Then they're encoding that information using the light spectrum again - but not the same wavelengths or information as they used to detect them in the first place.
So you have two things being run together. One is the identification of objects and the other is the encoding of information about those objects so that a single pixel detector is all that is needed to read what has been encoded.
Leverging the light coming off objects and processing it using diffusion filters to "learn" or "reason" about what the object is is a sort of side channel attack on the problem of object identification wrt to current approaches.
Encoding thngs into a light so only a single pixel is needed to carry information is an encoding trick.
If I misunderstood anything- entirely possible- feel free to correct.
My understanding is that it generates parallel to the rgb information, a depth-map and segments encoded also in specific spectrum of colors. This removes the burden of heavy image processing (edge-detection, etc) which improves performance. The segments can be trained for classification.
Any idea if they used any type of nonlinearities in the optics? It's hard to see this being powerful without nonlinearities.
Apparently the idea here is to substitute a bunch of convolution kernels with, well...optics
That would simplify the electronics for applications where electronics need to be radiation hardened.
I'm not so sure, this uses electronics operating in THz range, and the crystals used are also likely vulnerable.
The main gain seems to be energy use and size, thing is very efficient.
The main gain seems to be energy use and size, thing is very efficient.