Translating OpenStreetMap data to HTML5 Canvas with Rust and WebAssembly(mary.codes)
mary.codes
Translating OpenStreetMap data to HTML5 Canvas with Rust and WebAssembly
https://mary.codes/blog/programming/translating_openstreetmaps_to_HTML5_canvas_rust_wasm/
38 comments
What a clear, step-by-step write-up of getting this POC (proof-of-concept) code running. She has picked as her audience experienced programmers, and one gets the sense that the write-up is as much for herself as for us. Which is great. We get to live vicariously through her and solve a small-ish (but non-trivial) problem with Rust, WASM and an OSM api with a cool visual payoff at the end. It's easy to let yourself not be impressed by straight-forward writing to a clear audience, because it looks so simple. But it's absolutely not simple and I'm glad to have found this blog and author. Thanks again, HN!
The direct plotting from geographic coordinates (e.g., WGS84) can lead to visual distortions, especially for parks located further from the equator, where circles will appear flattened.
A practical solution would be to reproject the data to the Mercator projection or the appropriate local UTM Zone, ensuring accurate distances in cartesian units with no visible distortions of angles.
Alternatively, a quite straightforward method would involve scaling the longitudinal axis by the cosine of the latitude (cos φ [1]) at the center latitude of the area of interest. This adjustment provides a good approximation for small extents, like theme parks, without delving deeply into map projections.
[1] https://en.wikipedia.org/wiki/Longitude#Length_of_a_degree_o...
A practical solution would be to reproject the data to the Mercator projection or the appropriate local UTM Zone, ensuring accurate distances in cartesian units with no visible distortions of angles.
Alternatively, a quite straightforward method would involve scaling the longitudinal axis by the cosine of the latitude (cos φ [1]) at the center latitude of the area of interest. This adjustment provides a good approximation for small extents, like theme parks, without delving deeply into map projections.
[1] https://en.wikipedia.org/wiki/Longitude#Length_of_a_degree_o...
Should not be an issue here as this is based on openstreetmap, which uses wgs84 coordinates everywhere. Changing the projection won't make that much of a difference for small features like parks or buildings.
You are right that this is an issue if you start plotting e.g. circles using a naive algorithm as the x and y scales in degrees are simply not the same in meters. Which is an issue if you want to draw circles and rectangles on a map.
The solution to that is to do it in meters and then translate the points from some origin using e.g. the Haversine distance. A related issue you have there is that distance algorithms aren't accurate either but it's close enough as an approximation over small distances.
I have some algorithms for that in my jillesvangurp/geogeometry library. I also have some UTM coordinate conversion algorithms in there that I recently added.
You are right that this is an issue if you start plotting e.g. circles using a naive algorithm as the x and y scales in degrees are simply not the same in meters. Which is an issue if you want to draw circles and rectangles on a map.
The solution to that is to do it in meters and then translate the points from some origin using e.g. the Haversine distance. A related issue you have there is that distance algorithms aren't accurate either but it's close enough as an approximation over small distances.
I have some algorithms for that in my jillesvangurp/geogeometry library. I also have some UTM coordinate conversion algorithms in there that I recently added.
The distortion is still there, even for small features as seen in the OP examples of circular features of the park. This due to the changing length of longitudinal unit at a given latitude.
Historically this is the reason web mapping picked up Mercator in the first place, since it preserves angles and relatives sizes (conformal) at city scales, which is the primary use case of those.
WGS84 is a CRS first, not meant to be used as cartographic projection for a known local scope. (for some Plate carée is more of a plot than a projection)
I’m not talking about converting between local CRS for surveying an local plate consistency. You just want the display to look not visibly distorted in a way that’s not intentional. Florida is still quite forgiving here. Eurodisney not so much.
https://commons.wikimedia.org/wiki/File:Plate_Carr%C3%A9e_wi...
Historically this is the reason web mapping picked up Mercator in the first place, since it preserves angles and relatives sizes (conformal) at city scales, which is the primary use case of those.
WGS84 is a CRS first, not meant to be used as cartographic projection for a known local scope. (for some Plate carée is more of a plot than a projection)
I’m not talking about converting between local CRS for surveying an local plate consistency. You just want the display to look not visibly distorted in a way that’s not intentional. Florida is still quite forgiving here. Eurodisney not so much.
https://commons.wikimedia.org/wiki/File:Plate_Carr%C3%A9e_wi...
I once experimented with something like this for OpenXcom is to first rotate the geographic coordinate system so that local equator/zero meridian intersection is at the center of the area of interest, then apply merkator or whatever other projection you like.
Otherwise it's a sea of hacks which doesn't get rid of latitude-induced distortions anyway.
After that being very annoyed with lat/long stuff I moved to N-vector representation.
https://en.wikipedia.org/wiki/N-vector
Otherwise it's a sea of hacks which doesn't get rid of latitude-induced distortions anyway.
After that being very annoyed with lat/long stuff I moved to N-vector representation.
https://en.wikipedia.org/wiki/N-vector
I thought all maps were drawn using web Mercator and that things were simply not large enough to be really distorted. Does gmaps and company do projected rendering?
Google Maps on web uses perspective projection, unless WebGL is disabled in your browser. Zoom out and see!
They do, if you zoom out they're a mercator that wraps around, but when they give you latitude and longitude instead of x and y, you're getting the real sphere stuff
I used to know a lot more about rendering than I do today, but (forgive me if this sounds stupid) - I wonder if you could add to a gps coordinate area (say 1 sq. km. to also include that patche's NORMAL from the surface of the globe that you're projecting your map to, and have the 1km tile's image adjusted based on a orthographic view from that tile's NORMAL, so effectively - you're projecting each 1k square more directly from areal imagery?
Or is this already considered in modern projection methods?
Or is this already considered in modern projection methods?
This is already mentioned in the other comments, as the latitudinal correction to longitudinal dimension [i.e. use cos(latitude) as a scale for EW distances].
I call this the tangent-plane projection (which hints at your picture), where distances can be simply calculated by 2D Pythagoras in the EW-scaled coordinates. I recently discovered it is more often known as Equirectangular Projection:
https://en.wikipedia.org/wiki/Equirectangular_projection
More details at this excellent resource for GIS coordinate geometry:
https://www.movable-type.co.uk/scripts/latlong.html
I call this the tangent-plane projection (which hints at your picture), where distances can be simply calculated by 2D Pythagoras in the EW-scaled coordinates. I recently discovered it is more often known as Equirectangular Projection:
https://en.wikipedia.org/wiki/Equirectangular_projection
More details at this excellent resource for GIS coordinate geometry:
https://www.movable-type.co.uk/scripts/latlong.html
The mentioned Overpass API uses sets as a sort of fundamental concept, so the code
The change I propose above adjusts the ">;" to operate on all of the "way[building][!name];" results together, rather than each element one at the time.
way[building][!name];foreach{(._;>;);out;}
way[building][!name];(._;>;);out;
The change I propose above adjusts the ">;" to operate on all of the "way[building][!name];" results together, rather than each element one at the time.
this is a common enough need that
out geom
It doesn't do exactly the same thing, it inserts the lat/long into the parent instead of returning the nodes. Often doesn't matter, but it does if you are using a library that is expecting osm xml.
For anyone interested, I have a similar blog-post [0], in which I explore rendering OSM data using WebGL.
[0]: https://gero.dev/blog/webgl-render-osm-streets
[0]: https://gero.dev/blog/webgl-render-osm-streets
Nice article. It's good to see regl is still being used.
I was looking at that project [1] trying to understand what this article explains in a more simple way. There is some non-trivial complexity of how to put together wasm-pack, web-sys, js-sys and wasm-bindgen functions to work well together with javascript for a web page or a browser extension. Nice article!
[1] https://github.com/drakerossman/hackernews-userscript
[1] https://github.com/drakerossman/hackernews-userscript
This is a cool project; it has just enough meat on its bones to learn a bit about WASM and mapping data, and an attainable goal with a clear, visual result after some effort. I just spent half an hour to try to get this to work myself to see how far WASM has come. Not sure I'll be doing something with this specifically, but good to keep in the back of my head for future challenges along the same vein.
If they have a public API why did she have to crop and manually export stuff? What is she exporting, data? If so what is the API doing?
Looks like she wasn't actually exporting anything, just using the export feature as a convenient way to get the coordinates of the bounding box she wanted to search in.
Author could use that queue time and some ML to forecast waiting times trends and create a real time Next-Best-Ride (NBR) model and plot those in the map.
I'd find that useful the next time I'm pushing my 2 kids around that place...
Wasn't there a post on here recently about OSM going to vector-based tiles? I did a quick search and it seems like this is already a widely-offered option. Can anyone jog our memory on what the change is?
The OSM Foundation is building out a stack to generate 'minutely' vector tiles -- meaning the if you edit the map, it'll be reflected in the vector tiles within a few minutes.
Most vector tile sets today generated from OSM are hours behind the current state of the map data in the best case, much more commonly they are months behind.
Most vector tile sets today generated from OSM are hours behind the current state of the map data in the best case, much more commonly they are months behind.
OSM foundation is going to host vector tiles and build a stack to keep them up to date with ongoing editing.
You are probably talking about https://news.ycombinator.com/item?id=39339182
You are probably talking about https://news.ycombinator.com/item?id=39339182
Thanks for the replies. Sounds like a nice addition.
I have tried to use Canvas on Android to draw some custom view (chart). I still have a PTSD from that experience. I have an exceptionally great amount of respect for people who can do this sort of stuff.
Well, if you're playing with wasm like this post, you could sidestep canvas and render the image on the wasm side.
c.f. https://github.com/plotters-rs/plotters-wasm-demo
I did something similar but used SVG instead - it didn't feel that bad
In case you're hungry after reading this, she has a post on how to make creamy grits:
https://mary.codes/blog/recipes/the_secret_to_the_creamiest_...
I love grits, but have never tried baking soda (or even heard of it). Gonna have to try this.
https://mary.codes/blog/recipes/the_secret_to_the_creamiest_...
I love grits, but have never tried baking soda (or even heard of it). Gonna have to try this.
> While I haven't performed any benchmarks about whether Rust or JavaScript processing is faster, I decided to use Rust and WebAssembly because there will be a lot of coordinates to process, and Rust generally performs faster at large amounts of data processing.
Is this really true for Rust compiled to WebAssembly in the browser?
Is this really true for Rust compiled to WebAssembly in the browser?
It’s an interesting question. When it comes to processing large amounts of data I’d actually be more concerned about JavaScript’s memory usage than strict processing performance. Certainly you could use JS smartly with typed arrays and such to minimise memory impact but you'd basically be manually creating a lot of the convenience that Rust gives you out of the box.
It depends on the use case (e.g., how much you're interacting with web APIs) but I'd generally expect this to be the case for data processing at least.
During compilation from Rust to WebAssembly you have the opportunity to apply ahead-of-time optimizations that would be too expensive or impossible to apply to JavaScript during execution, so this can improve performance pretty significantly over JavaScript.
During compilation from Rust to WebAssembly you have the opportunity to apply ahead-of-time optimizations that would be too expensive or impossible to apply to JavaScript during execution, so this can improve performance pretty significantly over JavaScript.
Depends on the data types and if the WASM code can make use of SIMD.
Here is an idea: use Mapsforge binary format for map data. Map sizes are quite small, especially useful for not to huge areas. Implementing HTTP Range requests might be an interesting exercise in reducing bandwidth usage.
Somewhat bizarrely I ended up doing exactly that as a uni project. Writing a parser for the Mapsforge format in TypeScript was a bit of a weird experience. The funkiest bit was trying to get the range request stuff to work properly offline via service workers.
My only regret is not putting more effort into getting WebGL working, I just used the plain canvas and it was a bit of a bottleneck.
demo: https://files.george.honeywood.org.uk/final-deliverable/#16/...
blog: https://george.honeywood.org.uk/blog/map-from-the-ground-up/
My only regret is not putting more effort into getting WebGL working, I just used the plain canvas and it was a bit of a bottleneck.
demo: https://files.george.honeywood.org.uk/final-deliverable/#16/...
blog: https://george.honeywood.org.uk/blog/map-from-the-ground-up/
An existing solution:
> PMTiles readers use HTTP Range Requests to fetch only the relevant tile or metadata inside a PMTiles archive on-demand.
https://docs.protomaps.com/pmtiles/
> PMTiles readers use HTTP Range Requests to fetch only the relevant tile or metadata inside a PMTiles archive on-demand.
https://docs.protomaps.com/pmtiles/
So the data seems to come from the overpass API as XML, gets a bit of transformation in rust, passed back to javascript for drawing...
To me, that seems like the wrong design for performance... Drawing you probably want to use webgl shaders for so you can do zooming and panning at 60 fps...
And downloading the data as XML seems like a bad idea if you're drawing city-sized areas because the XML will be huge and contain plenty more detail than you need. Instead you need a server-side component to pack simplified and quantized polygon data for the necessary scale into a binary datastructure for sending to the client.
And thats exactly how bing/google maps/apple maps work...
To me, that seems like the wrong design for performance... Drawing you probably want to use webgl shaders for so you can do zooming and panning at 60 fps...
And downloading the data as XML seems like a bad idea if you're drawing city-sized areas because the XML will be huge and contain plenty more detail than you need. Instead you need a server-side component to pack simplified and quantized polygon data for the necessary scale into a binary datastructure for sending to the client.
And thats exactly how bing/google maps/apple maps work...
[deleted]