HackerTrans
TopNewTrendsCommentsPastAskShowJobs

MrPowers

no profile record

Submissions

2025: The Year of 1,000 DataFusion-Based Systems

influxdata.com
2 points·by MrPowers·8 miesięcy temu·0 comments

Managing spatial tables in Data Lakehouses with Iceberg

sedona.apache.org
2 points·by MrPowers·8 miesięcy temu·0 comments

Is physical world AI the future of autonomous machines?

therobotreport.com
1 points·by MrPowers·8 miesięcy temu·0 comments

SedonaDB: A new geospatial DataFrame library written in Rust

sedona.apache.org
197 points·by MrPowers·10 miesięcy temu·49 comments

comments

MrPowers
·10 miesięcy temu·discuss
Rust is a good language for performant computing in general, but especially for data projects because there are so many great OSS data libraries like DataFusion and Arrow.

SedonaDB currently supports SQL, Python, R, and Rust APIs. We can support APIs for other languages in the future. That's another nice part about Rust. There are lots of libraries to expose other language bindings to Rust projects.
MrPowers
·10 miesięcy temu·discuss
You can generate the dataset with the instructions in this readme: https://github.com/apache/sedona-spatialbench/tree/main

Here are the queries: https://github.com/apache/sedona-spatialbench/blob/main/prin...

They should be fairly easy to replicate!
MrPowers
·10 miesięcy temu·discuss
The "DuckDB is probably the most important geospatial software of the last decade" post has a nice related discussion: https://news.ycombinator.com/item?id=43881468
MrPowers
·10 miesięcy temu·discuss
There is a project called GeoPolars: https://github.com/geopolars/geopolars

From the README:

> Update (August 2024): GeoPolars is blocked on Polars supporting Arrow extension types, which would allow GeoPolars to persist geometry type information and coordinate reference system (CRS) metadata. It's not feasible to create a geopolars. GeoDataFrame as a subclass of a polars. DataFrame (similar to how the geopandas. GeoDataFrame is a subclass of pandas.DataFrame) because polars explicitly does not support subclassing of core data types.
MrPowers
·10 miesięcy temu·discuss
SedonaDB builds on libraries in the Rust ecosystem, like Apache DataFusion, to provide users with a nice geospatial DataFrame experience. It has functions like ST_Intersects that are common in spatial libraries, but not standard in most DataFrame implementations.

There are other good alternatives, such as GeoPandas and DuckDB Spatial. SedonaDB has Python/SQL APIs and is very fast. New features like full raster support and compatibility with lakehouse formats are coming soon!