The peculiar event sourced deadlock(jappie.me)
jappie.me
The peculiar event sourced deadlock
https://jappie.me/the-peculiar-event-sourced-deadlock.html
4 comments
I'm confused by the setup: in an event sourced setup, I'd expect a writer to write to the events table, and the projection to then be generated _from those events_ (possibly asynchronously), rather than generating both the table update and the event from the same code. That sounds like a recipe for disaster when your replay code and update code are not actually in sync?
There is so much to dig into for one short article. So much. Too much for an HN comment, I think. A few bullet points:
- It's not an event sourced system, it's a relational system with an absolutely terrible audit log. - The audit log is (was?) serialised and blocking for every transaction, okay for low volume and low latency transactions. - 30 minute transactions!? That is NOT low latency. That needs engineering to manage contention. - The last updated event information is updated in a transaction and definitionally is always the most recent event… what's it actually for? - The solution is exactly equivalent to a boolean "is_applied" column on the event table… and is still always set "true" inside a transaction.
> That sounds like a recipe for disaster when your replay code and update code are not actually in sync?
It reads like their replay code is their update code, but they have absolutely no confidence that their event history is complete and consistent ("A lot of things can go wrong with a large history, foreign keys, exceptions, serialization mismatches, events out of order etc.") so it's unlikely they'll ever risk replaying events.
The bit that I think has the most interest to consider, though, is the granularity of events. The article refers to database transactions as a monoid, which is to be expected since the application's architecture is built around database transactions and not events, but events can also be treated as a monoid. Making events extremely low level (eg, https://vvvvalvalval.github.io/posts/2018-11-12-datomic-even...) provides a ton of flexibility but absolutely needs the higher level semantic information to be explicitly written as well, else a significant portion of the value of event sourcing is lost. Simply recording a transaction ID (as the github account linked to the article does) won't help much as business requirements change over time, exact dates of changes become fuzzy and forgotten, and inferred meanings drift. Event sourcing works best when it's complete and explicit.
- It's not an event sourced system, it's a relational system with an absolutely terrible audit log. - The audit log is (was?) serialised and blocking for every transaction, okay for low volume and low latency transactions. - 30 minute transactions!? That is NOT low latency. That needs engineering to manage contention. - The last updated event information is updated in a transaction and definitionally is always the most recent event… what's it actually for? - The solution is exactly equivalent to a boolean "is_applied" column on the event table… and is still always set "true" inside a transaction.
> That sounds like a recipe for disaster when your replay code and update code are not actually in sync?
It reads like their replay code is their update code, but they have absolutely no confidence that their event history is complete and consistent ("A lot of things can go wrong with a large history, foreign keys, exceptions, serialization mismatches, events out of order etc.") so it's unlikely they'll ever risk replaying events.
The bit that I think has the most interest to consider, though, is the granularity of events. The article refers to database transactions as a monoid, which is to be expected since the application's architecture is built around database transactions and not events, but events can also be treated as a monoid. Making events extremely low level (eg, https://vvvvalvalval.github.io/posts/2018-11-12-datomic-even...) provides a ton of flexibility but absolutely needs the higher level semantic information to be explicitly written as well, else a significant portion of the value of event sourcing is lost. Simply recording a transaction ID (as the github account linked to the article does) won't help much as business requirements change over time, exact dates of changes become fuzzy and forgotten, and inferred meanings drift. Event sourcing works best when it's complete and explicit.
What in the love of God is that header image all about
Wow. I have concluded that it represents a “dead lock”:
Dead = child in a graveyard Lock = overlaid picture of a padlock.
And a Postgres logo thrown in for good measure.
Dead = child in a graveyard Lock = overlaid picture of a padlock.
And a Postgres logo thrown in for good measure.