Automerge: a new foundation for collaboration software [video](youtube.com)
youtube.com
Automerge: a new foundation for collaboration software [video]
https://www.youtube.com/watch?v=Qytg0Ibet2E
32 comments
Thanks for that. I wish they were a little more honest at the top when they claim there are no merge conflicts. Yes, they can automatically merge, but some cases will inevitably produce conflict objects and you are going to have to handle them in some way, or indicate to users that some of their edits have been lost.
The downside of anything like this is that for it to work well everything change must go through this framework because the fine grained change information is needed to effectively replay changes. That can make it hard to use existing domain libraries which might require a round trip data conversion as it may not be possible to differentiate between an insertion plus a deletion versus a complex modification.
The downside of anything like this is that for it to work well everything change must go through this framework because the fine grained change information is needed to effectively replay changes. That can make it hard to use existing domain libraries which might require a round trip data conversion as it may not be possible to differentiate between an insertion plus a deletion versus a complex modification.
How Yjs works from the inside out https://www.youtube.com/watch?v=0l5XgnQ6rB4
I really love Automerge, had some fun in the last months with building some small experiment. Some major problem I see currently is the lacking ecosystem.
I had to roll my own persistence implementation on top of IndexedDB and I still haven’t touched the backend synchronization part.
I would love to see Y.js or Automerge more throughly supported by a wider ecosystem of elements that make building a production app easier.
I had to roll my own persistence implementation on top of IndexedDB and I still haven’t touched the backend synchronization part.
I would love to see Y.js or Automerge more throughly supported by a wider ecosystem of elements that make building a production app easier.
Y.js has pretty good integrations with editors. What would you wanted to see in particular?
BTW Rust port of Y.js might be start of something rather wild. :)
BTW Rust port of Y.js might be start of something rather wild. :)
Are you looking for https://github.com/rust-crdt/rust-crdt and https://github.com/libp2p/rust-libp2p?
I agree but they do seem quite focused and have made a lot of headway over the last year or so when I first started following the project. I think switching to a rust backend was the right move and will open the project up to a lot more platforms than other implementations.
Looks like both Y.js or Automerge communities are busy with benchmarks
Main takeaways from toying with both Yjs and Automerge:
1. Extremely difficult to build backend in other programming languages than Nodejs
Rust implementations https://github.com/automerge/automerge-rs https://github.com/yjs/y-crdt
You will cry looking at source code https://github.com/yjs/y-crdt/blob/main/yffi/src/lib.rs
C-binding, FFI, etc
2. Both communities are great. Before committing recommend to go through issues and discussions
Yjs forum https://discuss.yjs.dev/
Automerge Slack https://join.slack.com/t/automerge/shared_invite/zt-e4p3760n...
3. Watch out implementations of underline libraries. Trace lib0 libraries usage and internals in Yjs for example
https://github.com/yjs/yjs/blob/7bd764fba73fecb8bacd1eb82a4b... https://github.com/dmonad/lib0
JavaScript engines use UTF-16 encoding. Golang (my main backend language) is using UTF-8 ... reimplementing Yjs code in Golang with algorithms and optimization and futher scaling might become impossible for small startups.
Build parallels with Automerge
4. Rich editing similar to Google Doc is very very complicated subject with lot of landmines
BitPhinix seems making a progress on updated example https://github.com/BitPhinix/slate-yjs/tree/next
However there is a surprise waiting for you on backend https://github.com/BitPhinix/slate-yjs/blob/next/examples/ba...
See "@hocuspocus/server" https://tiptap.dev/hocuspocus/
5. There's ProseMirror editor for collaborative editing. However you might not like its internals compare to Slatejs (which is far from perfect and lot of people rely on hard forks)
Feel free to ping me on Telegram @reactima if you are committed to build something similar to Notion.so or thinking to add collaborating editing with Slatejs and Reactjs. I'm really curious how other people plans to live with Yjs and Automerge impressive foundation.
1. Extremely difficult to build backend in other programming languages than Nodejs
Rust implementations https://github.com/automerge/automerge-rs https://github.com/yjs/y-crdt
You will cry looking at source code https://github.com/yjs/y-crdt/blob/main/yffi/src/lib.rs
C-binding, FFI, etc
2. Both communities are great. Before committing recommend to go through issues and discussions
Yjs forum https://discuss.yjs.dev/
Automerge Slack https://join.slack.com/t/automerge/shared_invite/zt-e4p3760n...
3. Watch out implementations of underline libraries. Trace lib0 libraries usage and internals in Yjs for example
https://github.com/yjs/yjs/blob/7bd764fba73fecb8bacd1eb82a4b... https://github.com/dmonad/lib0
JavaScript engines use UTF-16 encoding. Golang (my main backend language) is using UTF-8 ... reimplementing Yjs code in Golang with algorithms and optimization and futher scaling might become impossible for small startups.
Build parallels with Automerge
4. Rich editing similar to Google Doc is very very complicated subject with lot of landmines
BitPhinix seems making a progress on updated example https://github.com/BitPhinix/slate-yjs/tree/next
However there is a surprise waiting for you on backend https://github.com/BitPhinix/slate-yjs/blob/next/examples/ba...
See "@hocuspocus/server" https://tiptap.dev/hocuspocus/
5. There's ProseMirror editor for collaborative editing. However you might not like its internals compare to Slatejs (which is far from perfect and lot of people rely on hard forks)
Feel free to ping me on Telegram @reactima if you are committed to build something similar to Notion.so or thinking to add collaborating editing with Slatejs and Reactjs. I'm really curious how other people plans to live with Yjs and Automerge impressive foundation.
I’m a bit biased, but I think 2022 will be the year of Y.js.
What we’ve done or set up for the next year:
* The Y-Collective to fund more and more related projects, like slate-yjs: https://opencollective.com/y-collective
* The Hocuspocus backend you've discovered, which will become public in April or so
* A cloud offering we’re bootstrapping to hopefully get even more money into the ecosystem
* Y.js core support for the Tiptap editor (900k downloads/month), with a lot more advanced features coming next year
* More and more frontend libraries, like SyncedStore which was on the frontpage a few days ago: https://syncedstore.org/docs/
* The Y.js Rust port: https://github.com/yjs/y-crdt
Exciting times!
What we’ve done or set up for the next year:
* The Y-Collective to fund more and more related projects, like slate-yjs: https://opencollective.com/y-collective
* The Hocuspocus backend you've discovered, which will become public in April or so
* A cloud offering we’re bootstrapping to hopefully get even more money into the ecosystem
* Y.js core support for the Tiptap editor (900k downloads/month), with a lot more advanced features coming next year
* More and more frontend libraries, like SyncedStore which was on the frontpage a few days ago: https://syncedstore.org/docs/
* The Y.js Rust port: https://github.com/yjs/y-crdt
Exciting times!
I agree with Hans, we are going to see an explosion of collaborative tools next year. And Yjs is best placed to enable them.
The ecosystem around Yjs with the Y-collective is so impressive.
The ecosystem around Yjs with the Y-collective is so impressive.
CRDTs (and Y.js especially) will deliver what Google Wave promised. Not as a single unifying UI for communication, rather all existing apps can get collaborative power where it makes sense. And that is a game changer for me – as a user and developer. Exciting times indeed!
I’m biased into Yjs too and bet on it. Yjs Rust port and other simply not ready for the prime and won’t be for a while. Bindings and unsafe Rust is the road to hell
How do the performance and features of Yjs compare to Automerge?
It will depend on your use cases. Better to check updated roadmaps too. The final gap won’t be too big IMHO
https://github.com/dmonad/crdt-benchmarks
Not completely up to date I believe but it shows that Yjs significantly outperforms automerge.
Not completely up to date I believe but it shows that Yjs significantly outperforms automerge.
> You will cry looking at source code https://github.com/yjs/y-crdt/blob/main/yffi/src/lib.rs
Am I missing something? You linked a file that is inherently going to be "un-Rusty" because it's meant to be an FFI shim for _non-Rust_ languages to use through a common C API that this file exposes.
Am I missing something? You linked a file that is inherently going to be "un-Rusty" because it's meant to be an FFI shim for _non-Rust_ languages to use through a common C API that this file exposes.
> 4. Rich editing similar to Google Doc is very very complicated subject with lot of landmines
They (Ink & Switch) are actually working on that too, see https://www.inkandswitch.com/peritext/
They (Ink & Switch) are actually working on that too, see https://www.inkandswitch.com/peritext/
Wow - you weren't kidding - the rust port is total trash. What is even going on there?
I'm one of the maintainers of this project: we're happy to hear some constructive feedback and proposals for improvements, so please don't hesitate to speak your mind at github issues: https://github.com/yjs/y-crdt/issues .
The internals are still very hot and in a state of flux, as we 1st decided to go with porting the Yjs, then leave cleaning and optimizations for 2nd step after we have something, that's compatible with existing Yjs behavior.
The internals are still very hot and in a state of flux, as we 1st decided to go with porting the Yjs, then leave cleaning and optimizations for 2nd step after we have something, that's compatible with existing Yjs behavior.
One sort of issue is that this only directly supports documents/DAGs, so representing many-to-many relationships is a bit iffy. This is similar to how Rust can't directly represent cyclic data structures.
It's not a huge deal, because working with indices works well enough, but it is a bit annoying.
It's not a huge deal, because working with indices works well enough, but it is a bit annoying.
[deleted]
what problem does this solve?
MeteorJS solved this 12 years ago...but good to see more thoughts being put into this.
It did not at all. It solved the problem of how to reflect a source of truth (the server) in somewhat real-time on the client through subscriptions. Also made some attempt to have edits that were persisted to client-side mini-mongo pushed to the server but this was best effort at most.
CRDTs are on a completely different level. They provide deterministic conflict resolution strategies through either/or causal and physical timestamps to ensure it's always safe to persist a write locally and later sync it with the server or not even have a server and sync directly which other peers.
CRDTs are on a completely different level. They provide deterministic conflict resolution strategies through either/or causal and physical timestamps to ensure it's always safe to persist a write locally and later sync it with the server or not even have a server and sync directly which other peers.
I understand the difference.
However, the local mini-mongo is acting a local-state and the JSON/JS all the way is solving the marshalling/ORM issue. I understand it is a centralized way of dealing with this, however, it was operating on the same problem space. Furthermore, automerge can be used as the server, which bring it closer to what Meteor did. I was hinting at the overlap in the problem space, but I should have phrased it better. Anyway, I'm keep to try automerge.
However, the local mini-mongo is acting a local-state and the JSON/JS all the way is solving the marshalling/ORM issue. I understand it is a centralized way of dealing with this, however, it was operating on the same problem space. Furthermore, automerge can be used as the server, which bring it closer to what Meteor did. I was hinting at the overlap in the problem space, but I should have phrased it better. Anyway, I'm keep to try automerge.
You can't just use automerge on the server, it must be used on the client and additionally on the server (or a more cut down version that just replicates deltas between peers).
If you are going to try out automerge you should also take a look at y.js and diamond-types.
If you are going to try out automerge you should also take a look at y.js and diamond-types.
Yes, I get that part. I'll take a look at y.js and diamond-types, thank you!
The key feature of automerge is that it's decentralized like git. Two people could work asynchronously and only occasionally merge their results.
I get it, it is different Thanks for correction.
https://github.com/automerge/automerge