HackerTrans
TopNewTrendsCommentsPastAskShowJobs

blacha

no profile record

comments

blacha
·3 tahun yang lalu·discuss
Those comments were more at pastmaps.

For elevation data, we store our DEM/DSM in S3 as LERC [1] COGS, LERC has a WASM bundle which I think can be used in the browser. We found LERC COGs to be one of the most space efficient ways of storing highresolution DEM/DSM data [2], If you wanted to you could fetch LERC tiles directly out of a remote COG and use that directly for the terrain heights.

I am more focused on storage/archiving/publishing of our LiDAR capture program [3] than web based visualizations of it though, so I am unsure if a LERC COG would even be better for you than a PNG TerrainRGB.

[1] https://www.npmjs.com/package/lerc

[2] https://github.com/linz/elevation/tree/master/docs/tiff-comp...

[3] https://linz.maps.arcgis.com/apps/MapSeries/index.html?appid...
blacha
·3 tahun yang lalu·discuss
Your browser has a very powerful image decoder built into it, offloading the PNG decoding into Javascript is very resource hungry.

Using maplibre (or any map viewer) you can load blobs of image data out of a tiff and use `Image` or `Canvas` to render the data onto a map.

Its even easier if the tiffs are already Cloud optimized as they perfectly align to a 1-to-1 map tile and they don't need to be rescaled, you can then just render the images onto the map. eg here is a viewer that loads webps out of a 15GB tiff and uses Canvas to render them onto a map [1]

Unless you are trying to layer all your maps together, you also could stop reprojecting them into webmercator, or if your goal is to layer them, then storing them in webmercator would save a ton of user's compute time.

There are a bunch of us that talk web maping and imagery in the #maplibre and #imagery slack channels in OSMUS's slack [2]

[1] https://blayne.chard.com/cogeotiff-web/index.html?view=cog&i...

[2] https://github.com/maplibre/maplibre-gl-js#getting-involved
blacha
·3 tahun yang lalu·discuss
I don't think geojson is a great format for anything with more than a few MB of data.

I wanted to see exactly how bad it is with a largeish datasets, so I exported the New Zealand address dataset[1] with ~2.5M points as a geopackage (750MB) QGIS loads this fine its a little slow when viewing the entire country but when zoomed into city level it is almost instant to pan around.

Using ogr2ogr I converted it to ndgeojson (2.5GB), It crashed my QGIS while trying to load it. Using shuf I created a random 100,000 points geojson (~110MB) it was unbearably slow in QGIS while panning around 5+ seconds.

I currently use and recommend flatgeobuf[2] for most of my working datasets as it is super quick and doesn't need sqlite to read (eg in a browser).

It is also super easy to convert to/from with ogr2ogr

ogr2ogr -f flatgeobuf output.fgb input.geojson

[1] https://data.linz.govt.nz/layer/105689-nz-addresses/data/ [2] https://github.com/flatgeobuf/flatgeobuf
blacha
·4 tahun yang lalu·discuss
if you also wanted to pull in a node framework for deserilization you could import something like `zod` to have the similar level of nice error messages when invalid input is encountered.

If you were in typescript, zod would also automatically generate your interfaces too.
blacha
·4 tahun yang lalu·discuss
New Zealand regularly uses flying companies to take aerial imagery photos of most of the country, which is generally taken between 30cm and ~2cm, Land Information New Zealand (LINZ) then releases this imagery imagery completely free (CC-BY)[1,2] to the public and can be downloaded as GeoTIFFs [3]

I think LINZ decided that <0.05m might have some privacy issues due to being able to distinguish people in it, and have held back releasing some <0.05m or they may have reduced the quality of it.

disclaimer: I work at Land Information New Zealand

[1] https://basemaps.linz.govt.nz/?i=hawkes-bay-urban-2022-0.05m... [2] https://basemaps.linz.govt.nz/?i=christchurch-urban-2021-0.0... [3] https://data.linz.govt.nz/layer/106915-christchurch-005m-urb...
blacha
·5 tahun yang lalu·discuss
This is pretty much exactly what we do to serve aerial/satellite imagery maps.

We convert the imagery into Cloud optimised geo tiffs and store them in S3 https://www.cogeo.org/ then the browser can request the tiles directly from S3.

Even the big imagery providers are now storing their imagery as COGs, eg https://registry.opendata.aws/sentinel-2-l2a-cogs/