Apple Maps Flyover Reverse Engineering(github.com)
github.com
Apple Maps Flyover Reverse Engineering
https://github.com/retroplasma/flyover-reverse-engineering
29 comments
The 3d content in Apple Maps comes from their acquisition of C3:
https://9to5mac.com/2011/10/29/apple-acquired-mind-blowing-3...
The technique is detailed in the conference paper:
https://www.spiedigitallibrary.org/conference-proceedings-of...
Essentially they use a rig of DSLR cameras mounted to the bottom of a Cessna to take photographs in a 45 meter grid, and then do stereo reconstruction from that. Crucially they describe using fast phase based disparity maps, which at the time were a recently declassified technique for guiding cruise missiles. C3 was a part of the defense contractor Saab AB.
https://9to5mac.com/2011/10/29/apple-acquired-mind-blowing-3...
The technique is detailed in the conference paper:
https://www.spiedigitallibrary.org/conference-proceedings-of...
Essentially they use a rig of DSLR cameras mounted to the bottom of a Cessna to take photographs in a 45 meter grid, and then do stereo reconstruction from that. Crucially they describe using fast phase based disparity maps, which at the time were a recently declassified technique for guiding cruise missiles. C3 was a part of the defense contractor Saab AB.
Nokia also invested in C3 and had the first 3D maps available to consumers before anyone, completely in WebGL which used to be on https://wego.here.com. I guess Apple had more money to acquire them.
*Disclaimer - I worked for Nokia at the time.
*Disclaimer - I worked for Nokia at the time.
Saab also owns Vricon, basically the same thing as C3 but not aimed at consumers. They sold part of it to DigitalGlobe, and they use both aerial and satellite imagery for 3D models.
https://www.vricon.com/
https://www.vricon.com/
The Apple street view cars also have LIDAR, so I'm curious to see what their plan is for that.
Considering the expense, and especially time, involved in gathering street-level data, if I was Apple, I'd put every sensor I could think of on those cars and then let the data scientists figure out a use for it later.
Didn’t Google get burned by doing that and “accidentally” collecting Wi-Fi network data?
https://www.theguardian.com/technology/2010/may/15/google-ad...
https://www.theguardian.com/technology/2010/may/15/google-ad...
That's probably for making maps used by localization on autonomous cars.
The most immediate use will likely be for the Apple Glasses which are expected to come out in 2020.
LIDAR could definitely assist with identifying what the user is looking at.
LIDAR could definitely assist with identifying what the user is looking at.
Hopefully 3d street view. You'd want that data for good AR overlays, too.
Colossal bandwidth use for very high latency updates? Surely it’s better to get depth information from the device rather that from Apple’s self driving cars/StreetView clones, either with stereo, lidar, or AI estimates?
https://techcrunch.com/2018/06/29/apple-is-rebuilding-maps-f...
tl;dr they're using it to make better maps
tl;dr they're using it to make better maps
It's planes and I think also helicopters. I've written satellite mode in the repo because the Maps app button says "Satellite"
It'll be planes, because the running cost per mile travelled is critical to making the whole thing cost effective.
Power consumption and weight of the rig isn't insubstantial, so the smallest planes and gliders can't be used, and after that cost per hour and cost per mile are the key metrics.
Planes can be 10x cheaper by that metric.
Power consumption and weight of the rig isn't insubstantial, so the smallest planes and gliders can't be used, and after that cost per hour and cost per mile are the key metrics.
Planes can be 10x cheaper by that metric.
makes sense. the region for Phoenix is named "phoenix_heli" though so i figure that one was captured from a helicopter
>I did for the first time see an Apple branded car with a photo sphere on the roof recently and kinda got excited that they are gonna try and bring competition into maps / street view that Google has dominated for the last decade.
Are you in Cali?
Are you in Cali?
Most of the high resolution imagery is from planes.
Previous discussion on how Google Maps' Satellite View was almost named Bird Mode: https://news.ycombinator.com/item?id=19235017
Previous discussion on how Google Maps' Satellite View was almost named Bird Mode: https://news.ycombinator.com/item?id=19235017
> I don't think this is satellite data
It's a metaphor. Like 'bird's eye view'.
It's a metaphor. Like 'bird's eye view'.
I recently asked the founder of a pretty big GIS company how they handle plagiarism. The answer: "we don't bother. [we just monetize services on top of open data.]"
So it looks like the intentional errors method didn't work out in the end. [1]
Wonder if Apple will be as accommodating though :p
https://www.gislounge.com/map-traps-intentional-mapping-erro...
So it looks like the intentional errors method didn't work out in the end. [1]
Wonder if Apple will be as accommodating though :p
https://www.gislounge.com/map-traps-intentional-mapping-erro...
this is the same person who RE'd Google Earth a couple months ago: https://news.ycombinator.com/item?id=18900080
[deleted]
Coincidence: Half an hour after reading this thread, a white Subaru with California tags stopped at the red light outside my office window. Big disco ball on a platform attached to the roof and "Apple Maps maps.apple.com" stenciled on the window.
I took a picture.
https://imgur.com/UwK2wYy
I took a picture.
https://imgur.com/UwK2wYy
As someone who works with GIS data and map meshes this is really awesome. Hats off to the author.
I’d be interested in hearing how they reverse engineered the data!
Mostly used popular tools like Hopper, LLDB, Hex Fiend. Xcode's debugger UI was surprisingly helpful too because it lists current memory allocations of obj-c instances nicely. Also learned about Frida along the way and would have used it more in retrospect.
I just like how the example is the end of the Santa Monica pier!
This is awesome. What are people using it for?
And I'd kinda be curious to what Microsoft has been working on, I remember their birds eye was pretty good a few years ago.
Apple I'd believe the are selectively trying to improve their data in areas that are heavily visited by their users.
Also, I did for the first time see an Apple branded car with a photo sphere on the roof recently and kinda got excited that they are gonna try and bring competition into maps / street view that Google has dominated for the last decade.