Throughout history, people have believed that new inventions would solve complex social problems. The telegraph was supposed to end war, the telephone to cure loneliness, and the automobile to put education within everyone’s reach.
Techno-Optimism Archive collects clippings from old newspapers and magazines that capture moments when new technologies were met with high hopes and confident predictions about the future.
Author here. Thank you all for the comments. I take full responsibility for stupidly using an image for posting the code snippet. Sorry for that! Also, the article was originally posted almost 2 years ago (and "resurrected" with the recent migration to Medium). This is why a fairly old DuckDB version is referenced there. Some of the issues I observed are now gone too.
Obviously, many things have changed since then. We've experimented extensively and moved back and forth with using DuckDB for our internal cloud processing architecture. We eventually settled on just using it for reading the data and then handling everything else in custom workers. Even using TypeScript, we achieved close to 1M events/s per worker overall with very high scalability.
However, our use-case is quite distinct. We use a custom query engine (for sequence processing), which has driven many design decisions.
Overall, I think DuckDB (both vanilla and WASM version) is absolutely phenomenal. It also matured since my original blog post. I believe we'll only see more and more projects using it as their backbone. For example, MotherDuck is doing some amazing things with it (e.g., https://duckdb.org/2023/03/12/duckdb-ui) but there are also many more exciting initiatives.
Techno-Optimism Archive collects clippings from old newspapers and magazines that capture moments when new technologies were met with high hopes and confident predictions about the future.