HackerTrans
トップ新着トレンドコメント過去質問紹介求人

MrPowers

no profile record

投稿

2025: The Year of 1,000 DataFusion-Based Systems

influxdata.com
2 ポイント·投稿者 MrPowers·8 か月前·0 コメント

Managing spatial tables in Data Lakehouses with Iceberg

sedona.apache.org
2 ポイント·投稿者 MrPowers·8 か月前·0 コメント

Is physical world AI the future of autonomous machines?

therobotreport.com
1 ポイント·投稿者 MrPowers·8 か月前·0 コメント

SedonaDB: A new geospatial DataFrame library written in Rust

sedona.apache.org
197 ポイント·投稿者 MrPowers·10 か月前·49 コメント

コメント

MrPowers
·10 か月前·議論
Rust is a good language for performant computing in general, but especially for data projects because there are so many great OSS data libraries like DataFusion and Arrow.

SedonaDB currently supports SQL, Python, R, and Rust APIs. We can support APIs for other languages in the future. That's another nice part about Rust. There are lots of libraries to expose other language bindings to Rust projects.
MrPowers
·10 か月前·議論
You can generate the dataset with the instructions in this readme: https://github.com/apache/sedona-spatialbench/tree/main

Here are the queries: https://github.com/apache/sedona-spatialbench/blob/main/prin...

They should be fairly easy to replicate!
MrPowers
·10 か月前·議論
The "DuckDB is probably the most important geospatial software of the last decade" post has a nice related discussion: https://news.ycombinator.com/item?id=43881468
MrPowers
·10 か月前·議論
There is a project called GeoPolars: https://github.com/geopolars/geopolars

From the README:

> Update (August 2024): GeoPolars is blocked on Polars supporting Arrow extension types, which would allow GeoPolars to persist geometry type information and coordinate reference system (CRS) metadata. It's not feasible to create a geopolars. GeoDataFrame as a subclass of a polars. DataFrame (similar to how the geopandas. GeoDataFrame is a subclass of pandas.DataFrame) because polars explicitly does not support subclassing of core data types.
MrPowers
·10 か月前·議論
SedonaDB builds on libraries in the Rust ecosystem, like Apache DataFusion, to provide users with a nice geospatial DataFrame experience. It has functions like ST_Intersects that are common in spatial libraries, but not standard in most DataFrame implementations.

There are other good alternatives, such as GeoPandas and DuckDB Spatial. SedonaDB has Python/SQL APIs and is very fast. New features like full raster support and compatibility with lakehouse formats are coming soon!