DocNode: A TypeScript OT library for local-first apps(github.com)
github.com
DocNode: A TypeScript OT library for local-first apps
https://github.com/docnode/docnode
10 comments
Hey German. Congrats on shipping the project!
Reading the "Why another CRDT / OT library?" I like that you seem to have taken a "Pareto approach": going for a simpler solution, even if not theoretically perfect. In the past few months I've been building a local-first app, and I've been a bit overwhelmed by the complexity of CRDTs.
The goal that I have with my app is to allow syncing between devices via Dropbox / Google Drive / iCloud or any another file-syncing server that the user is already using. I don't want to host a sync server for my users, and I don't want my users to need to self-host either.
Do you think it would be possible to use Dropbox as the sync "transport" for DocNode documents? I'm thinking: since a server is needed, one device could be designated as the server, and the others as clients. (Assuming a trusted environment with no rogue clients.)
Reading the "Why another CRDT / OT library?" I like that you seem to have taken a "Pareto approach": going for a simpler solution, even if not theoretically perfect. In the past few months I've been building a local-first app, and I've been a bit overwhelmed by the complexity of CRDTs.
The goal that I have with my app is to allow syncing between devices via Dropbox / Google Drive / iCloud or any another file-syncing server that the user is already using. I don't want to host a sync server for my users, and I don't want my users to need to self-host either.
Do you think it would be possible to use Dropbox as the sync "transport" for DocNode documents? I'm thinking: since a server is needed, one device could be designated as the server, and the others as clients. (Assuming a trusted environment with no rogue clients.)
Thanks! The answer depends on what you want:
1. Do you care about resolving concurrent conflicts? That is, if two users modify the same document simultaneously (or while one is offline), is it acceptable if one of their changes is lost? If that’s not a problem, then using Dropbox or something similar is totally fine, and you don't need a CRDT or OT like DocNode at all. Technologies like Dropbox aren't designed for conflict resolution and can't be integrated with CRDT or OT protocols.
2. If you do want to resolve conflicts, you have two options. (a) Use a CRDT, which doesn’t require a server. One downside is that clients must be connected simultaneously to synchronize changes. Personally, I don’t think most people want software that behaves like that, and that’s one of the reasons I didn’t focus on building a CRDT. If you’re going to end up needing a server anyway, what’s the point? (b) Use a server, either hosted by you or by your users. The good news is that it’s extremely simple. With DocNode Sync, you can literally set it up with one line of code on the server.
That doesn't apply if you're using a CRDT with "a server as an always-present client". But in that scenario, DocNode will be more efficient.
1. Do you care about resolving concurrent conflicts? That is, if two users modify the same document simultaneously (or while one is offline), is it acceptable if one of their changes is lost? If that’s not a problem, then using Dropbox or something similar is totally fine, and you don't need a CRDT or OT like DocNode at all. Technologies like Dropbox aren't designed for conflict resolution and can't be integrated with CRDT or OT protocols.
2. If you do want to resolve conflicts, you have two options. (a) Use a CRDT, which doesn’t require a server. One downside is that clients must be connected simultaneously to synchronize changes. Personally, I don’t think most people want software that behaves like that, and that’s one of the reasons I didn’t focus on building a CRDT. If you’re going to end up needing a server anyway, what’s the point? (b) Use a server, either hosted by you or by your users. The good news is that it’s extremely simple. With DocNode Sync, you can literally set it up with one line of code on the server.
That doesn't apply if you're using a CRDT with "a server as an always-present client". But in that scenario, DocNode will be more efficient.
Thanks for the detailed answer!
I think I do want to solve conflicts. My use case is for a personal database, which simplifies things a bit: sync is between devices of a single person, so it's unlikely for concurrent offline changes to occur.
What I have in mind is a setup like the one from this experiment: https://tonsky.me/blog/crdt-filesync/ . I don't know if it's at all possible in my use case though, or–in case it is possible–if it ends up being practical. As you said, the resulting user experience might be so strange that it's not something users want.
Anyway, thanks again for the info and good luck with DocNode. :)
I think I do want to solve conflicts. My use case is for a personal database, which simplifies things a bit: sync is between devices of a single person, so it's unlikely for concurrent offline changes to occur.
What I have in mind is a setup like the one from this experiment: https://tonsky.me/blog/crdt-filesync/ . I don't know if it's at all possible in my use case though, or–in case it is possible–if it ends up being practical. As you said, the resulting user experience might be so strange that it's not something users want.
Anyway, thanks again for the info and good luck with DocNode. :)
That's an interesting idea.
A "bring your own cloud" provider could be used for DocNode Sync. For example, Dropbox. Dropbox doesn't have the ability to resolve conflicts, but it can be used to deterministically store the order of operations. Then, clients would reconcile and combine the operations into a single state file. Authentication might be a bit tricky, since permissions would have to reside elsewhere. But I think it's doable.
I'll consider it!
A "bring your own cloud" provider could be used for DocNode Sync. For example, Dropbox. Dropbox doesn't have the ability to resolve conflicts, but it can be used to deterministically store the order of operations. Then, clients would reconcile and combine the operations into a single state file. Authentication might be a bit tricky, since permissions would have to reside elsewhere. But I think it's doable.
I'll consider it!
Hey German. I think you should re-submit this as a "Show HN: DocNode, A TypeScript OT library for local-first apps" in a few days to a week. I think it will get more of a chance titled that way.
Thanks for the advice. I'll try it when the post is a week old. I hope it doesn't get rejected.
I don't think it will. I know a few folks who have done the same. It' is part of the T&C's that it is ok to re-submit just not super fast.
Thank you so much for your help! I followed your advice:
https://news.ycombinator.com/item?id=46211551
https://news.ycombinator.com/item?id=46211551
Sorry that it didn't get any traction. It's a bit of a crap shoot with these things. Looks good to me tho. :)
Along the way, I learned a ton. I rewrote the library several times. Sometimes because I was obsessed with the API, other times for technical reasons. I moved from CvRDT to CmRDT, and finally to ID-based OT.
Happy to answer questions!