I'm not nearly educated enough about this subject to try to summarise it, but the research being carried out by Michael Levin's group into how organisms control anatomy growth is completely fascinating and has the feel of a breakthrough. Almost all the talks I've seen are good, for example:
I agree that reliably writing files is a sneakier problem than it first appears and that the likes of SQLite have already solved it.
Despite that my preference these days is still for a DB, like almost all dependencies, to have to justify its existence in my projects rather than defaulting to it as many seem to favour. It's straightforward to add SQLite but I'd typically rather take a little extra pain around the filesystem upfront to not have to deal with all the extra complexity of an SQL database if I can avoid it.
Naturally, this kind of simplicity is just one factor among many such as the size and shape of the data, expected access patterns and a host of others to weigh up during the trade-off decision.
Is the swap technique you mention writing to a tempfile and then performing an (atomic) rename to overwrite the existing file? I understand how that could cause data loss if the process exited before the rename but I don't understand how it might cause corruption. Are you able to expand on that?
A related thing I discovered recently is the potential non-atomicity of writes using O_APPEND on Linux[1], although I couldn't get the attached test program to fail on my machine. I would love to find some kind of confirmation that this behaviour has been changed and that appends can be relied upon to be atomic.
In the application I'm working on at the moment I've managed to get add more certainty around this stuff since I'm only ever appending to the file in 4096 byte chunks, so can perform a simple file size check to see if data was written completely or if the file needs to truncated to the nearest multiple of 4096. I'm fortunate that it prevents having to do something like open it read only to verify contents as you mentioned.
> I have a deGoogled phone running Lineage, but even with that, no Google Play Services, and some custom settings (like changing the captive portal URLs), there's still network traffic to Google.
I'm running LineageOS without Play Services too and didn't about know this!
Do you have any reference materials (I guess getting busy with Wireshark and the source is my next step)? I found this Reddit thread[1] talking about a connectivity check but am keen to start tracking down any others.
https://www.youtube.com/watch?v=Z-9rLlFgcm0
Creating new species without genetic changes seems wild!