Yes, I agree. I believe military-grade optics detection systems use multiple wavelengths, and some even use short bursts of visible light, making it harder to block.
Interesting question. Mainly as mentioned above we needed Google Play Services for AR. Nothing else stands out at a high level.
However there is a clear bottleneck with the Google AR team for features that the community really wants. While it's understandable, perhaps the open source libraries aren't the priority or there's short staffing, there are issues like this one - https://github.com/google-ar/arcore-android-sdk/issues/153 - that have been open since Jan 2018. This one asks that we can use the smartphone flashlight simultaneously with ARCore, but there's no visibility on how close we are at the moment. This feature would have likely improved our work's performance greatly, but even in general, many other AR developers are asking for it.
Re: overcoming it, there's not much we can do in this case. We just didn't implement the feature.
Oh, I must have misunderstand, since the root comment was talking about detection methods. But yes, jamming is a good option, and there have been papers to make areas "unrecordable". I don't have my citation manager on hand now, but I think it was an early ACM Sensys / Mobisys research work.
I'm not sure that it's a conclusive yes here. We have a procedure to remove the effects of background reflection saturation by constantly analysing the maximum reflection size. So if there's a large reflection in the scene, we walk the user back til we have all reflections at the same size or smaller than the the larger expected hidden camera reflection. This works fairly well.
However, if the camera reflection is always within a larger reflection for all possible distances and angles, then I doubt this will work.
Funnily enough this is exactly how we started the project, just with a different Thinkpad model.
Separately, Lubos, the creator of ToF Viewer, is very helpful to the ToF community and constantly answers any questions about APIs or sample code. I'd recommend talking to him if anyone wants to get started with ToF-based applications.
Agreed. I expect however that any easily available thermal sensing, especially on smartphones (see FLIR, etc), could be easily defeated with a strong adversary doing good thermal management.
As you say though, extra heat is a physical guarantee, so maybe a smarter technique exists to separate signal from noise in the thermal domain that I don't know of yet.
There's a lot of marketing involved in this naming, with Samsung calling it DepthVision and Apple calling it LIDAR. There may be a difference here, however. My understanding is that Apple LIDAR is doing what we call "direct" ToF, where the round trip travel time of laser pulses is actually measured (this can be in nanoseconds). This lines up with what self driving car (and other expensive) LIDARs do.
Most other ToF sensors use "indirect" ToF, where they measure the phase difference between incoming and outgoing signals to derive distance.
However, it gets murky as cheap 2D LIDARs on say, robot vacuum cleaners, use geometric techniques to find distance (basically return angle of a reflection). I explored this in a previous work.
TLDR: I would recommend not taking any naming at face value and reading the actual datasheet or more commonly, technical marketing materials, since few ToF manufacturers that I see have a public datasheet.
I don't think so. There's "laser-assisted autofocus" which might be slang for an actual ToF sensor, but sometimes that's just a single laser pulse (I.e., no 2D image that we need).
Agreed, and a very valid point that these off the shelf detectors are pretty cheap compared to ToF smartphones. There are a number of unfavourable things about the detectors though:
- No automated detection, so lots of human subjectiveness on whether a bright spot should be investigated
- Background lighting can be an issue, so the view through the detector can be filled with bright reflections
- When I tried this for longer than 30 seconds, there's really some eye strain involved
The idea is that the smartphone has more than this one purpose, and the detection system can improve with software updates, whereas you get what you see with the detector and nothing more.
It could also possibly work on iOS, but there's a catch we have not yet explored. We're relying on something called the "confidence" score in the ToF camera API, which has 8 possible values (3 bits) in Android, but from what I last saw, has only 3 possible values total in iOS. It's not clear how this lack of info exposed by the APi will affect accuracy.
This is something we'd have to separately test, and ultimately make a wholly different app for if it's feasible on iOS.
I don't think this works, mainly because an extremely bright external source of 850 nm light looks incredibly suspicious through the sensor (as we expect the ToF sensor to be the only light source). We could add that check in explicitly.
Thats very interesting, first time hearing of it. Not sure, really depends on what kind of reflection it generates when light hits it, and because of the unique design I'm not sure what that will be.
Thank you! Actually, my understanding when I started this project was that I would get a 640 x 480 image (IMX516 sensor). However, I could only get a 320 x 240 image from the sensor through the Android API, so that was a bit of an oddity.
The free option: use your smartphone's flashlight and try to spot unnaturally bright / colored reflection.
If you want to spend some money and don't mind carrying an extra device: probably any of the "hidden camera detectors" on the market will be at least somewhat useful. K18, CC308+, etc.
We're trying with our work to get better or at least equivalent results without having to use external devices.
So far the biggest issue is if the camera is angled oddly away from the user (basically, outside the 20 degree observable FoV). Another issue would be if they manage to install a larger camera that returns larger reflections (which we filter out).
Regarding using the app to check, I guess that applies for the existing handheld detectors as well. It's definitely something that intelligent attackers can try to plan for, but we havent tested the adversarial robustness of the system right now. That would be a very interesting direction for us as well.