Ask HN: Which is the best sync engine?
5 comments
I have been using Powersync, and it works great for mobile apps with offline-first functionality.
I suppose the only part the OP would need to look at is "Being able to form and execute queries in my backend" for the read path, since Sync Rules / Sync Streams need to define those queries. For the write path, you should be able to re-use the existing APIs / permission middleware for the most part.
Then: "Further, I wonder how such a system would work with connection pooling, sharding, replication etc."
These are all topics on their own, but briefly: - Connection Pooling: Not supported in PowerSync Cloud atm, but supported in theory when self-hosting since pgbouncer 1.23 added WAL support. Having said that, clients connect to the PowerSync Service for sync operations, so you only need a single direct connection between the service and your postgres. - Sharding: PowerSync can be used in a sharded setup [1], with better support planned - Replication: PowerSync uses the Postgres WAL for replication [2]
[1] https://docs.powersync.com/usage/sync-rules/advanced-topics/... [2] https://docs.powersync.com/installation/database-setup#other...
Then: "Further, I wonder how such a system would work with connection pooling, sharding, replication etc."
These are all topics on their own, but briefly: - Connection Pooling: Not supported in PowerSync Cloud atm, but supported in theory when self-hosting since pgbouncer 1.23 added WAL support. Having said that, clients connect to the PowerSync Service for sync operations, so you only need a single direct connection between the service and your postgres. - Sharding: PowerSync can be used in a sharded setup [1], with better support planned - Replication: PowerSync uses the Postgres WAL for replication [2]
[1] https://docs.powersync.com/usage/sync-rules/advanced-topics/... [2] https://docs.powersync.com/installation/database-setup#other...
I think you can use Tanstack DB in this case. It fits what you need ideally, if I'm not wrong. It's a wrapper over Electric. However, it's currently in beta, so you can check it out. It's too handy and simple
https://tanstack.com/db/latest/docs/overview
https://tanstack.com/db/latest/docs/overview
Dexie is my favorite, although I haven’t tried them all of course.
For a new app I'm developing I'd like to give it the fast and responsive feel that apps like Linear have. By having real-time querying, syncing and write operations.
I'm no expert on the matter, so feel free to educate me. I have looked into a couple of options such as Convex, ElectricSQL, Zero, Liveblocks, etc..
I feel like they always come short in a couple of ways and the best way to summarize that is their modularity.
What I would love in a sync engine:
- Being able to use my database (Postgres, AWS RDS)
- Being able to form and execute queries in my backend (suppose I have frontend in Next.js (Vercel) with an API route to my FastAPI server (AWS ECS) where I have all of my auth / permission middleware, etc.)
- Have a simple and familiar way to declare the schemas to the sync engine (like re-using SQLalchemy's or Drizzle's schemas)
- A simple SDK to form queries that uses SQL
Further, I wonder how such a system would work with connection pooling, sharding, replication etc.
Does something like this exist? Or what are the major challenges that prevent this from existing?