I built a real-time AR plane spotter, here's the math that makes it work
6 comments
It looks super interesting. Have you considered writing a deep-dive article on that?
Developing a similar application has been a long-time occupant on my hobby projects to-do list so I'd love to learn about the topic more!
Developing a similar application has been a long-time occupant on my hobby projects to-do list so I'd love to learn about the topic more!
I actually have written a deep-dive article on this, I have also prepared a comprehensive mathematical reference document as well (linked at the end of the article)
Here is the link to the article on dev.to: https://dev.to/ananddhruv29/from-gps-coordinates-to-screen-p...
[deleted]
Thanks for sharing!
cool
The problem sounds simple, you have a GPS coordinate in the sky and a GPS coordinate in your hand. You want a pixel. But there are four distinct coordinate spaces between those two things, and the transitions between them have sign conventions that fail silently, wrong output with no error.
The pipeline:
Why each transition is non-obvious:
Geodetic → ENU. The East component has a cosine factor that most implementations miss: E = Δλ × (π·RE/180) × cos(φ_user). Meridians converge toward the poles, one degree of longitude is fewer metres at latitude 25° than at the equator. Without it, East-West positions look correct near the equator and quietly diverge as latitude increases.
ENU → Device frame. Android's rotation matrix R maps device axes to ENU world axes. To go the other direction you use R⊤. In Android's row-major FloatArray(9), this means column indices, not row indices:
These produce completely different results. Both compile without complaint.
Device → Camera frame. Android's sensor defines +Zd as pointing out of the screen toward your face. The camera convention requires +Cz to point into the scene. So Cz = −dZ, always. This is the only correction needed for portrait mode.
Camera → Screen. After the perspective divide and FOV normalisation, the Y axis flips: Ypx = (1 − NDCy) × H/2. Camera +Cy is up; screen y=0 is at the top. If we miss this, the aircraft above the horizon appears below screen centre.
Real captured values (ATR72, 18,000 ft):
Phone azimuth 33.0°, aircraft bearing 34.2° → 1.2° right of centre. Phone pitched −4.3°, elevation 29.5° → net 33.8° up, just inside the top edge of the frustum. Physically consistent throughout.
Happy to answer questions about any stage of the pipeline or about anything else, whatever is interesting to anyone.