mrkeen

há 57 minutosdiscuss

Terrific writing. Just terrific. Copied verbatim:

  The Lindy effect in software The longer a tech has been around, the more robust it is seen as compared to more recent ones, we often talk about a technology’s maturity The C language SQL has been around for a while, https://antonz.org/fancy-ql/ JS libraries seem to come and goes

mrkeen

há 15 horasdiscuss

Work is outside.

Workers are demanding to stay inside (with their AC) instead of going outside.

mrkeen

anteontemdiscuss

The problem is you can't choose which side the gun nuts end up on.

mrkeen

anteontemdiscuss

And yet people were happy to call Snowden a traitor for breaking the news of it.

mrkeen

anteontemdiscuss

  in this day and age, are there more useful concepts that we could be giving more attention than CAP, ACID, and friends? Are they useful enough to satisfy the dfsdt principle? I think not.

CAP is useful to the extent that:

- people consciously declare that they have beaten it (in which case you have a very strong launchpad for your rebuttal, e.g. "Are you just pretending P won't happen?")

- or that they are sleepwalking into a software design which implicitly requires CAP capability to function well (in which case you know ahead of time that it will not function well, and so you should choose a different design).

ACID is a useful shorthand for "Can I rely on single-threaded reasoning while writing this multi-threaded program?" E.g. If I transactionally subtract 5 over here, and add 5 over there, then my code will not have changed the total amount. Since we now know DB vendors operate by default in a weaker isolation mode, we have a shorthand for knowing we can't use use single-threaded reasoning, and instead need to consider which commits over there will change the state of the running transaction over here.

mrkeen

anteontemdiscuss

We're currently running a long-term offshore experiment to see if 2A has any measurable impact on dragnet surveillance and the NSA.

mrkeen

há 3 diasdiscuss

They're still right here in sibling comments

mrkeen

há 3 diasdiscuss

Are you arguing that most teams that recognise a need for a distributed system chicken-out and use a single node instead, and if so, is that a good thing?

mrkeen

há 3 diasdiscuss

Agreed. When they describe the advantages over Raft, I can't help but read 'this is Paxos'.

mrkeen

há 4 diasdiscuss

I expected to see mention of the Church-Rosser theorem, which may have just turned 90 years old. Pretty good vintage for a comp sci theorem.

Anyway the way I learned it is:

* for a given lambda expression, all evaluation orders which reduce to a normal form will reduce to the same normal form.

* if a lambda expression has a normal form, the 'lazy' way (normal order, leftmost-outermost-reduction, call-by-name, call-by-need) will find it.

Actually on re-reading, the article did mention confluence of the untyped lambda calculus, but only went as far as saying that the different evaluation order leads to different termination results, which is not quite as strong as saying the lazy way will find the terminating path if it exists.

mrkeen

há 5 diasdiscuss

I mean if I was only going to use 'one thing', it would be tech that doesn't have downtime when you replace its parts. Something like Kafka or Cassandra. Then since postgres is useful, I would likely end up adding it into the system.

mrkeen

há 5 diasdiscuss

The reason for separate systems is that some of them go down. And by "go down", that includes you wanting to deploy a new version. Or restore from backup as the article suggested.

mrkeen

há 5 diasdiscuss

You should try more languages

mrkeen

há 6 diasdiscuss

This is not a bad question!

If you flip it and instead ask "how do I write something that can't fail?" you might find some interesting ground.

The best things I know about are static type-checking, pure functions and totality. Different languages provide more or less help with these things. It's perfectly fine to do 'two things which don't fail or cause other things to fail'.

Forgive the digression, but there is an 'infectious' aspect to the above 3 things (see the function-colouring problem), e.g. you can't build pure functions which call non-pure functions. The Dependency Inversion Principle (of SOLID) gives some help in how to tackle this.

Also, the above things only work within one node (of a distributed system).

For multiple nodes, I use something like Kafka, where you write down one event, and have two systems subscribe to it, each doing one thing. Yes, there's still the obvious issue of them failing independently, but when that happens, you have an authoritative source of truth (in the form of Kafka events). This beats the craps out of developer logs.

You skip the laborious questions of "what happened in the system?" and "what should the correct state be?" Because the events are already the answer - just eyeball them.

Events also machine-readable, so if you diagnose a problem and a fix it in one case, there's a good chance you can build a detector for other cases. You don't have to wait for a support ticket to get escalated to the dev team.

You also divide the debugging space dramatically. If the Kafka log says one thing {Bob bought Minecraft for $10}, then the Ownership service is just wrong if it says Bob doesn't own Minecraft, and the Finance service is just wrong if it doesn't report the $10. Fix each independently. At no point do you need to look at Ownership and Finance together to see which one failed halfway through talking to the other, because they don't talk to each other.

Lastly, events are verifiable; they are their own audit trail. If your boss asks how much money is in the system, would you feel more confident reporting whatever the current balance is set to (i.e. the outcome of whatever code executed the last "UPDATE Balance ..." statement, or would you like to be able to sum over every transaction that you ever recorded?

mrkeen

há 6 diasdiscuss

I don't disagree with anything in particular here, but other developers might fall into a trap with:

  we are not fully enumerating and reasoning about failure cases.

This might put (or keep) a developer in the mindset that they can code a series of imperative instructions to build their minimal viable product, and then come back and tighten things up later.

I expend all my effort in avoiding 'doing two things'. It's bloody difficult, but since I've come around to thinking that recovering from 1-of-2-things-failing is probably impossible in most situations, doing it the bloody difficult way is easier.

mrkeen

há 6 diasdiscuss

I call it the "doing two things" problem.

Your write imperative code, which issues two commands, both of which can fail independently.

There are plenty of ways to pretend to 'deal with it'.

Firstly it will just pass all tests, so most devs can stop thinking about it right away.

A dev might think you can just catch and log the exception. Doesn't fix it.

You could run the code in prod for a while, see if it goes wrong. It will, at which point the dev will try it again, and it will probably work the second time, so they can stop thinking about it.

There was a big outbox pattern discussion a couple of days ago (split thing 1 into two halves, and do them atomically, leave thing 2 as an exercise for the reader.)

I think the reason you encounter this problem in the real world is that devs just exist in some quantum superposition of "it won't happen" and "I fixed it" and "it can't be fixed".

mrkeen

há 6 diasdiscuss

The internet of 2026.

Betteridge's law wrapping a paywall wrapping an AI piece masquerading as domain-specific analysis.

mrkeen

há 8 diasdiscuss

How much does it cost to rent a db that can do CAP?

mrkeen

há 8 diasdiscuss

I hadn't heard of throwing money at it before.

Maybe "Two Generals" doesn't work, but "Two Rich Generals" does.

mrkeen

há 8 diasdiscuss

I somewhat agree with 'just use the db' over 'use the db as the state and the outbox'.

And message "queues" are probably a waste of time too.

Where you get a real benefit is in using a proper append-only ledger. This is a solved problem. Paxos and Raft both give you this on the theoretical side, and systems like Kafka give you practical implementations.

"Pull-based" systems are far, far easier to reason about. E.g., I'm going to update my packages now. I'm going to pull from git now. I'm going to GET news.ycombinator.com now. Imagine the opposite - news.ycombinator deciding to push the frontpage to whichever device I'm using, at the precise time I'm hoping to read it.

So pull is better, but if you can only pull, then how does anyone change any state? Push a new message into Kafka, and let it handle the switch from push to pull.

It may be absurdly complex, but it's the least absurdly complex option if you want to distribute. And if you don't want to distribute, you don't need outbox.

Submissions

[untitled]

Stop Overusing Interfaces (2017)

comments