jeadie·3 か月前·議論This is exactly what we found. Ingest rates were tough. We partitioned and ran over multiple duckdb instances too (and wrangled the complexity).We ending up building a Sqlite + vortex file alternative for our use case: https://spice.ai/blog/introducing-spice-cayenne-data-acceler...
jeadie·3 か月前·議論You might find https://github.com/apache/datafusion and https://github.com/datafusion-contrib/datafusion-federation of interest
jeadie·10 か月前·議論We’re building vector indexes into Datafusion for search (starting with S3 vectors).Open source at https://github.com/spiceai/spiceai
We ending up building a Sqlite + vortex file alternative for our use case: https://spice.ai/blog/introducing-spice-cayenne-data-acceler...