8 million rows in. 600,000 out. 5.7 seconds. On a 16GB RAM laptop.
Duckle joined 4 sources at 2M rows each - an ADBC (Arrow) source, a CSV file, a MySQL table, and a second ADBC source - through one visual mapper: a 3-way join, 9 expressions, and a filter, straight to Parquet.
No cloud. No servers. Just Duckle on your laptop/desktop.
This is what local-first data engineering looks like now.
Duckle joined 4 sources at 2M rows each - an ADBC (Arrow) source, a CSV file, a MySQL table, and a second ADBC source - through one visual mapper: a 3-way join, 9 expressions, and a filter, straight to Parquet.
No cloud. No servers. Just Duckle on your laptop/desktop. This is what local-first data engineering looks like now.
Repository: https://github.com/SouravRoy-ETL/duckle