I know icechunk and I’m a huge fan of earthmover. But a common binary format like parquet seems nice… with interop for e.g duckdb and geo queries, you can “just load” era5 and do something like get wind direction/speed along the following path for the last 5 years group by day etc…
> This is just the beginning of modernizing geospatial data storage. We’re already looking ahead to other types of geospatial data such as raster, point cloud, spatial indexes…
I’m super exited about the planned support for multidimensional data, I know zarr is there but a long term storage format other than netcdf would be interesting maybe also something to replace grib, geotiff etc to share files
The conclusion: "One interesting idea is that, at their core, AI models are nothing more than compression models that take the corpus of data humanity has and boil it down to a set of weights" is also good, there is a interesting paper about compressing weather date with neuronal networks: https://arxiv.org/abs/2210.12538
What's missing a bit is that the comparison is more for general purpose data, there are some very interesting and super fast compressing algorithms for e.g. numbers (Turbopforc, gorilla, etc...) Daniel Lemires blog is super interesting about the different algorithms and how to make them faster.
Uhhh throwback to simpler times <3 I loved the scene in Basel back in the days
For the adventurous Ishkur also has a nice opinionated guide through the history of electronic music with much of its facets: https://music.ishkur.com/
The thing is most pollution and changes in the ocean are not visible right away, while hiking or on land in general I directly see consequences of littering and environmental impact and its possible to act upon (littering on highways)
in the oceans on the other hand a lot of environmental impact is just not visible, thus it needs to be made visible, I see a nice beach... I don't see particles floating just under the surface, I don't see the destroyed eco systems by trawling, I don't see "death zones" where there is no marine live...
so this is a good step into direction making these things visible
Living in a bigger small town in Germany I completely agree, I can’t imagine living somewhere where I can’t let the kids roam free… if I’m at the bakery I don’t have to lock my bike (and neither do the others) most of the kids at the local skatepark where I skate come and go on their own, on bikes or public transit and are unsupervised… but nobody would suspect neglect… if something happens we’ll help each other and the parents get called by their friends everyone looks after each other, the older ones don’t smoke their weed near the younger :D and so forth… oh and this is in a more “socially strained” neighborhood not the rich villa type of neighborhood
a good source for ERA5 historical data is https://open-meteo.com/en/docs/historical-weather-api (not affiliated, just a happy user) you can also run open-meteo locally, its quite fast for spatial and timeseries queries