That’s the question I’ve been mulling over for days, because
I don’t see how this action can make any particular guarantees
about durability, at least not in any portable way.
This part is super easy to clear up. CouchDB in no way relies on an fsync after open for any guarantee on durability. As shown in [1], CouchDB has been running an fsync on file open since extremely early in its development. However, I can easily see how just reading the Neighbourhoodie article would lead here. However, CouchDB is not susceptible to the sad path.
I didn't read the article super closely the first time since I'd been through the background discussions on the finer details, but today I'd probably hedge that a bit with language along the lines of: However, CouchDB is *probably* not susceptible to the sad
path. While we can't guarantee it can't happen due to how
various I/O operations are (not) specified, we're doing as much
as we can to prevent it. Also, don't forget that your storage
device might be lying about fsync anyway.
The underlying logic around that requires considering the original blog post in this chain [3]. That article posits a pathological error condition where we write something, crash, restart, issue read from a dirty page cache, and then hard crashing the entire machine. In this case, the database returned a read that was never committed. Using OpenZFS as an example (hey, it’s what I know), fsync()
always flushes anything outstanding for the underlying object,
regardless of where the writes came from.
AFAIK, this is the norm and, I assume, the reason that the NULL BITMAP article [3] suggests the fsync on open. In CouchDB land, we just went back and said, "Oh nice, we already do that for other reasons anyway." Unfortunately the "we already do it for other reasons" aspect didn't really come through. So in the end, while none of the behavior on fsync-on-open is guaranteed in anyway shape or form, it's not impossible that it's saved our bacon a non-zero number of times. Just because its not guaranteed, its common that filesystems will in fact perform those flushes regardless of which file descriptor is used.
Uh, that is exactly what a derivative work is. You literally specify that Hamlet is an input to your work. I believe you're conflating derivative with transformative. You're certainly creating a transformative derivation of Hamlet, but you are by definition creating a derivative work by training a Markov chain on the text of Hamlet.
The obvious follow up here is whether an LLM is creating transformative derivations or not. A lot of folks argue that yes, an LLM spitting out statistically sampled code that matches existing code is not transformative and is (or might be) infringing the terms of the license it was released under. Others argue that there's not an exact copy of the original source in the LLM's weights so by definition it must be a transformative work. I think it's a pretty obvious "somewhere in the middle" that is gonna make a bunch of lawyers a whole lot of money.
Personally, I don't care one way or the other. I'm one of the folks that thinks software shouldn't be copyright-able in the first place.