Many thanks for the feedback on the site, perhaps we need to get some more dense technical details either on there directly or adding to the existing docs on the github repo.
Command pattern is definitely one pattern we've seen these tamper-evident logs applied to; you can easily imagine a system where, say, config update "commands" are packaged off and written to a TE log, and then workers apply those commands to a system if and only if they are convinced that the commands were properly logged. If the admins/command-workers/auditors etc. can all prove they're looking at the same set of logged commands (which is the property that these tamper-evident logs are intended to provide), then you have a reliable and tamper-evident audit trail of all changes to your system.
There are a bunch of other scenarios we think these things might be useful for too, and we're trying to explore (and help others explore) these and provide mechanisms for reasoning about the properties of such systems.
[Disclaimer: I work on CT, Trillian, and other related projects]
We're looking in more depth at other use cases, developing a better understanding of which types of problems this might be useful for, and ways to reason about the properties you want/get from using systems like this (e.g. https://github.com/google/trillian/tree/master/docs/claimant...)
[Disclaimer: I work on CT, Trillian, and some other related projects]
I prefer to use the term "tamper-evident" when describing these things - you can totally fiddle with logs, but the property you likely want is that this can be detected.
As you say, Merkle trees are fairly straight-forward, but beyond integrity checking a file they have some other nice properties - as the "operator" of a Merkle tree based log, you can efficiently [O(log N)] convince an observer that:
- you've only grown the log by appending new entries since the last time they looked ("consistency proof")
- a given entry they hold is indeed included in the log ("inclusion proof")
if you change an entry in the log (or claim something is in the log when it's not) it becomes impossible for you to prove both of those properties (unless you're powerful enough to also break what's called the "second pre-image resistance" of the cryptographic hash you're using).
Note that for these two properties, an observer doesn't need to download or inspect the full contents of the log, just a handful of hashes in the Merkle tree which they can then use to recreate the "tree head hash" you mentioned. These "tree heads" are the state they store and are just a few hundred bytes.
There are generally a few types of log client:
- one type is putting things into the log
- one type are wanting to use what's stored in the log to make a decision of some kind ("should I trust this site", "should I install this binary", "am I seeing the true history of events", etc.) and they won't trust that data unless they are convinced it is in the log
- another can verify the accuracy/validity/goodness of things in the log (e.g. in CT it's the domain owner who is in the unique position of saying "yes, I asked for this cert to be created", for binaries it might be "I'm the authorised creator of these binaries and, yup, that was me" or "I'm a respectable malware scanning company, and this binary doesn't match any known signatures", "I'm an auditor and these transactions are complete and correct", etc. - you can probably think of many more examples!)
In this light, you can think of a tamper-evident log as being a sort of "reliable transport" to connect these different types of clients together, and importantly, where they all can prove to each other (using those light weight tree heads + consistency/inclusion proofs) that they're seeing the same set of data in the log - or, if they can't prove it - they have the cryptographic evidence of operator of the log operator having done something unexpected.
Command pattern is definitely one pattern we've seen these tamper-evident logs applied to; you can easily imagine a system where, say, config update "commands" are packaged off and written to a TE log, and then workers apply those commands to a system if and only if they are convinced that the commands were properly logged. If the admins/command-workers/auditors etc. can all prove they're looking at the same set of logged commands (which is the property that these tamper-evident logs are intended to provide), then you have a reliable and tamper-evident audit trail of all changes to your system.
There are a bunch of other scenarios we think these things might be useful for too, and we're trying to explore (and help others explore) these and provide mechanisms for reasoning about the properties of such systems.
[Disclaimer: I work on CT, Trillian, and other related projects]