Sunlight, a Certificate Transparency log implementation(letsencrypt.org)
letsencrypt.org
Sunlight, a Certificate Transparency log implementation
https://letsencrypt.org/2024/03/14/introducing-sunlight.html
47 comments
This is one of the projects I've been most excited about in the last few years. It let me backport to Certificate Transparency some of the modern designs ideas that came after it.
Beyond the Let's Encrypt announcement and the ct-policy thread (which includes a technical and advantages summary), here are a few resources that might be interesting.
- Design document, including architecture and tradeoffs: https://filippo.io/a-different-CT-log
- Implementation: https://github.com/FiloSottile/sunlight
- API specification: https://c2sp.org/sunlight
- Website, including test logs and feedback channels: https://sunlight.dev/
If you’re thinking “oh we could use something similar” please reach out! Sunlight is retrofitting some of the modern tlog designs on a legacy system. With a greenfield deployment you can do even better! I’m working with the Sigsum project on specs, tooling, and a support ecosystem to make deploying tlogs easier and safer.
Beyond the Let's Encrypt announcement and the ct-policy thread (which includes a technical and advantages summary), here are a few resources that might be interesting.
- Design document, including architecture and tradeoffs: https://filippo.io/a-different-CT-log
- Implementation: https://github.com/FiloSottile/sunlight
- API specification: https://c2sp.org/sunlight
- Website, including test logs and feedback channels: https://sunlight.dev/
If you’re thinking “oh we could use something similar” please reach out! Sunlight is retrofitting some of the modern tlog designs on a legacy system. With a greenfield deployment you can do even better! I’m working with the Sigsum project on specs, tooling, and a support ecosystem to make deploying tlogs easier and safer.
Cool!
tlog = transparency log, but not neccessarily for X509 certificates?
tlog = transparency log, but not neccessarily for X509 certificates?
Exactly! It's a growing ecosystem including things like https://transparency.dev, the Go Checksum Database, https://www.sigsum.org, SigStore, and even key transparency solutions like WhatsApp's.
One thing you end up needing to deploy tlogs is a way to reassure clients the tree is not forked, and for that you mostly need witness cosigning, where a quorum of third parties attest that a signed tree head is consistent with all the other ones they've seen. I've worked with the Sigsum project and the Google TrustFabric team on an interoperable specification for witnessing (which Sunlight interoperates with), and I am now working to develop a public, reliable ecosystem of witnesses.
Once you have witnessing, running a log can be as easy as hosting a few files in a GitHub repo or S3 bucket, updated with a batch script. I am very excited to make it possible for any project to get better-than-CT accountability for ~free.
(You might want to catch my RWC 2024 talk about this once it comes out!)
One thing you end up needing to deploy tlogs is a way to reassure clients the tree is not forked, and for that you mostly need witness cosigning, where a quorum of third parties attest that a signed tree head is consistent with all the other ones they've seen. I've worked with the Sigsum project and the Google TrustFabric team on an interoperable specification for witnessing (which Sunlight interoperates with), and I am now working to develop a public, reliable ecosystem of witnesses.
Once you have witnessing, running a log can be as easy as hosting a few files in a GitHub repo or S3 bucket, updated with a batch script. I am very excited to make it possible for any project to get better-than-CT accountability for ~free.
(You might want to catch my RWC 2024 talk about this once it comes out!)
The "standard" list of all known CT logs is maintained by Google at https://www.gstatic.com/ct/log_list/v3/all_logs_list.json and follows the schema at https://www.gstatic.com/ct/log_list/v3/log_list_schema.json.
As far as I can tell, that schema currently only supports RFC6962 logs. Are there plans in train to support enumeration of Sunlight-format logs? Or am I just far too impatient?
As far as I can tell, that schema currently only supports RFC6962 logs. Are there plans in train to support enumeration of Sunlight-format logs? Or am I just far too impatient?
Apple also has a list at https://valid.apple.com/ct/log_list/current_log_list.json
Getting them accepted as trusted lists is the goal, but how exactly that happens is not yet determined. Both the Apple and Chrome CT programs have responded to our announcement email at https://groups.google.com/a/chromium.org/g/ct-policy/c/v9Jzl...
> As the specification, implementations, and ecosystem support evolve, we look forward to being able to include Sunlight logs alongside RFC6962 logs in Chrome. While we're not quite there yet, we encourage the CT community to experiment with the prototype logs and specification as much as possible.
> That said, we’re not planning on accepting new Sunlight logs quite yet in the Apple CT Program (though we hope that is the end result). Before that jump, we’d especially like to hear additional input (even if it’s just “LGTM”) from CT log Monitors and Auditors, as well as CAs and other relying parties submitting certificates to CT logs. Along with that, we invite input from additional SMEs able to perform (informal) security and risk assessments of the proposal — especially if they’re accompanied by filed issues :)
The log list schema is a relatively small problem and should be easily enough to solve.
Getting them accepted as trusted lists is the goal, but how exactly that happens is not yet determined. Both the Apple and Chrome CT programs have responded to our announcement email at https://groups.google.com/a/chromium.org/g/ct-policy/c/v9Jzl...
> As the specification, implementations, and ecosystem support evolve, we look forward to being able to include Sunlight logs alongside RFC6962 logs in Chrome. While we're not quite there yet, we encourage the CT community to experiment with the prototype logs and specification as much as possible.
> That said, we’re not planning on accepting new Sunlight logs quite yet in the Apple CT Program (though we hope that is the end result). Before that jump, we’d especially like to hear additional input (even if it’s just “LGTM”) from CT log Monitors and Auditors, as well as CAs and other relying parties submitting certificates to CT logs. Along with that, we invite input from additional SMEs able to perform (informal) security and risk assessments of the proposal — especially if they’re accompanied by filed issues :)
The log list schema is a relatively small problem and should be easily enough to solve.
Thanks for replying to what was a rather half-baked question, typed out in some excitement.
I guess a more thought-through version is as follows: Currently, the CT logs with state == usable are operated by only six entities (Cloudflare, DigiCert, Google, LE, Sectigo, TrustAsia). In the earlier days of CT logs, there were a larger number of log operators. I think it's a reasonable assumption that we've ended up in the current situation at least partly because of the cost of log operation?
Since Sunlight aims to drastically reduce the cost of log operation, might we expect the number of log operators to rise, and if so would there be likely changes to the ways that log operators are enumerated as "trusted", or would things likely be as they are now?
I guess a more thought-through version is as follows: Currently, the CT logs with state == usable are operated by only six entities (Cloudflare, DigiCert, Google, LE, Sectigo, TrustAsia). In the earlier days of CT logs, there were a larger number of log operators. I think it's a reasonable assumption that we've ended up in the current situation at least partly because of the cost of log operation?
Since Sunlight aims to drastically reduce the cost of log operation, might we expect the number of log operators to rise, and if so would there be likely changes to the ways that log operators are enumerated as "trusted", or would things likely be as they are now?
I do hope we get some more log operators. We've discussed with a couple organizations that are potentially interested, so I'm hopeful we get another 2 or 3 soon.
Cost of log operation is definitely a big part of it, but I think the operational overhead of existing log software is also an issue (which is also cost, but in engineering time).
Cost of log operation is definitely a big part of it, but I think the operational overhead of existing log software is also an issue (which is also cost, but in engineering time).
That's a fun JSON file. It's served with open CORS headers which means I can query it in Datasette Lite like this:
https://lite.datasette.io/?json=https://www.gstatic.com/ct/l...
https://lite.datasette.io/?json=https://www.gstatic.com/ct/l...
I'm super excited about Sunlight. The CT ecosystem is really fragile right now, with current log implemetations being expensive to operate and very difficult to operate correctly, as evidenced by the recent failures of multiple logs[1][2]. And if too many logs fall over, it becomes infeasible to include the requisite number of SCTs in certificates, or worse, already-issued certificates can become effectively untrusted.
With Sunlight reducing costs by a couple orders or magnitude and significantly easing deployment complexity, it will be a huge boon to the whole ecosystem. I really hope log monitors begin crawling sunlight logs and browsers accept them as trusted in the near future.
[1]: https://groups.google.com/a/chromium.org/g/ct-policy/c/6mvSo...
[2]: https://groups.google.com/a/chromium.org/g/ct-policy/c/_dhkS...
With Sunlight reducing costs by a couple orders or magnitude and significantly easing deployment complexity, it will be a huge boon to the whole ecosystem. I really hope log monitors begin crawling sunlight logs and browsers accept them as trusted in the near future.
[1]: https://groups.google.com/a/chromium.org/g/ct-policy/c/6mvSo...
[2]: https://groups.google.com/a/chromium.org/g/ct-policy/c/_dhkS...
[1]: a delete command was wrongly issued to all Cassandra servers during a planned maintenance
[2]: a database server had its disk full that lead to a corrupted database
SRE is tough.
[2]: a database server had its disk full that lead to a corrupted database
SRE is tough.
[deleted]
Is there a specification one can follow to implement a verifier for certificate transparency logs? I looked up RFC 6962, but it does not contain test vectors. I am particularly interested in history tree inclusion and consistency proofs and don’t care how tree root is signed.
We're working on it! RFC 6962 specifies inclusion and consistency proofs, but indeed it's missing test vectors. Keep an eye on https://c2sp.org and https://c2sp.org/CCTV.
Great call out to the Go checksum database.
Go walks a great line between distributed system and central verification.
Curiously, “Sunlight” was the codename for the original CT spec:
https://datatracker.ietf.org/doc/draft-laurie-pki-sunlight/0...
https://datatracker.ietf.org/doc/draft-laurie-pki-sunlight/0...
Purely coincidental! ;-)
I was pretty sure CT was gonna die when Nimbus went down last year. Like, if Digicert or LE's logs went down while Nimbus was down, ha ha
There are six trusted operators, many with multiple logs. So there is some degree of redundancy, but this project is definitely an outcome of those same concerns.
A lot of the benefits I see people proposing for blockchain systems - things like "an immutable log of real estate ownership that no-one can ever tamper with" - look to me like they would be better handled by the certificate transparency log mechanism rather than by a proof-of-work/proof-of-stake blockchain.
Absolutely. The vast majority of such use cases are way better served by a transparency log than a blockchain.
Imagine an embedded system that only accepts firmware updates that are digitally signed and discoverable in a public append-only log. Such a proof would have to be attached to the firmware update just like the digital signature, with a size constraint on the order of kilobytes. To be fair I haven't kept up-to-date with blockchains, but I find it hard to believe that any existing blockchain could solve this. Transparency logs can.
I wonder when businesses that currently use blockchains as immutable logs will realise that there is a more cost-effective solution. One of the reasons it hasn't happened yet is because blockchains have magnitudes more mindshare than transparency logging. In fact, there doesn't seem to be that many people who deeply understand the design parameter space of transparency logs.
Shameless plug: My colleagues and I have spent a few years designing one - Sigsum. As Filippo mentions he has contributed to its design as well.
Imagine an embedded system that only accepts firmware updates that are digitally signed and discoverable in a public append-only log. Such a proof would have to be attached to the firmware update just like the digital signature, with a size constraint on the order of kilobytes. To be fair I haven't kept up-to-date with blockchains, but I find it hard to believe that any existing blockchain could solve this. Transparency logs can.
I wonder when businesses that currently use blockchains as immutable logs will realise that there is a more cost-effective solution. One of the reasons it hasn't happened yet is because blockchains have magnitudes more mindshare than transparency logging. In fact, there doesn't seem to be that many people who deeply understand the design parameter space of transparency logs.
Shameless plug: My colleagues and I have spent a few years designing one - Sigsum. As Filippo mentions he has contributed to its design as well.
> To be fair I haven't kept up-to-date with blockchains, but I find it hard to believe that any existing blockchain could solve this. Transparency logs can.
Any insight as to why you think blockchains can't handle this?
The only fundemental difference between blockchains and transparency logs is the trust assumptions and censorship resistence. Its fairly easy to reason about a transparency log as a permissioned blockchain. Conversely, a blockchain could be seen as a permissionless transparency log.
Any insight as to why you think blockchains can't handle this?
The only fundemental difference between blockchains and transparency logs is the trust assumptions and censorship resistence. Its fairly easy to reason about a transparency log as a permissioned blockchain. Conversely, a blockchain could be seen as a permissionless transparency log.
> blockchain could be seen as a permissionless transparency log
This is not the case. The essence of the transparency logs are thin clients which can do monitoring of the append only nature of the ledger. This allows accountability to be distributed more widelly wheras in blockchain one would need to retrieve the full ledger to do such testing.
On the other hand the responsability of adding records to the ledger in transparency log lies to a single authorithy whereas in permissionless blockchain it lies to individuals who solve the puzzle. However in practice there is centralisation of the computation power making the record inclusion also centralised.
The censorship resistance is somewhat a wicked problem. With transparency logs solve the by counter logging. In cases authorithy does not accepts a record it can be delivered to a counter authorithy. The counter authorithy also keeps records of main authorithy and thus users can eventually switch to it. This is though a manual process, but perhaps one can figure out a protocol to do such thing automatically.
This is not the case. The essence of the transparency logs are thin clients which can do monitoring of the append only nature of the ledger. This allows accountability to be distributed more widelly wheras in blockchain one would need to retrieve the full ledger to do such testing.
On the other hand the responsability of adding records to the ledger in transparency log lies to a single authorithy whereas in permissionless blockchain it lies to individuals who solve the puzzle. However in practice there is centralisation of the computation power making the record inclusion also centralised.
The censorship resistance is somewhat a wicked problem. With transparency logs solve the by counter logging. In cases authorithy does not accepts a record it can be delivered to a counter authorithy. The counter authorithy also keeps records of main authorithy and thus users can eventually switch to it. This is though a manual process, but perhaps one can figure out a protocol to do such thing automatically.
Sort-of. Here's how I think about it.
The fundamental value of a transparency log is to provide trivial discoverability. Say an attacker has compromised a code signing key and is using it to sign a malicious software update. Let's also assume that the attacker can distribute this update to a client that will trust it. Such a targeted attack is not trivially discoverable by the software vendor if the client's trust policy only requires a code signing signature by the compromised key. If the client's trust policy were to also require a proof that the signature is included in a trusted transparency log, the software vendor can discover the attack if it monitors the log. Note that transparency logs doesn't directly prevent attacks - it makes them discoverable.
Transparency systems based on transparency logs have five components: Log, Submitter, Client, Monitor and Auditor or Witness(es). The Log is a Merkle tree with its current root tree hash signed. In the example above the Client would be the software updating mechanism, which has a given Log in its trust policy. The Client will only trust a software update from the Submitter if the Submitter's signature and the update's hash are included in the Log. The Monitor regularly polls the Log for new entries it cares about, in this case software updates signed by the Submitter's public key. The Auditor is another party that regularly polls the Log, but the Auditor's job is not to discover log entries that constitute an attack. Instead, the Auditor's job is to monitor the Log's consistency - i.e. whether the log has removed any previously added entries. This is called a split-view attack, and it is a concern in all widely deployed transparency logs today. An alternative to Auditors is cosigning Witnesses, which proactively witness a Log's consistency, and whose witness signatures are required by the Client in order for it to trust the Log. This results in a much stronger threat model. See sigsum.org for more details on such a design.
Getting back to your comment:
- Yes, Clients are "thin", to use blockchain terminology, but they don't monitor the Log in order to detect attacks. The Log is part of their trust policy for making some trust decision, such as trusting a software update. Monitoring the Log for new entries that indicate e.g. key compromise is the Monitor's job.
- Regarding accountability: The Client's trust policy in conjunction with the Log and the Monitor increases the Submitter's accountability to the Client. Similarly the Auditors or Witnesses increases the Log's accountability to Clients, Submitter and Monitors.
- Regarding inclusion in the Log (or blockchain) I'd use the word "authority" as in who has the power to prevent something from being included. As you point out a transparency log is a central entity which has the sole power to decide what gets included or not. If the log is shut down, or its public key destroyed, nor more inclusions are possible. In this sense a blockchain is far more censorship-resistant than a transparency log. In other words, it is important that Clients trust multiple logs for redundancy. The consensus mechanism of a transparencty log is simply a public-key signature, as opposed to an emergent authority in the form of e.g. a distributed timestamp server based on SHA256. This vast difference in how the database's integrity is defined - the consensus mechanism - is why blockchains work better than transparency logs for censorship-resistant digital currency, whereas transparency logs work better than blockchains for efficient inclusion- and consistency proofs.
The fundamental value of a transparency log is to provide trivial discoverability. Say an attacker has compromised a code signing key and is using it to sign a malicious software update. Let's also assume that the attacker can distribute this update to a client that will trust it. Such a targeted attack is not trivially discoverable by the software vendor if the client's trust policy only requires a code signing signature by the compromised key. If the client's trust policy were to also require a proof that the signature is included in a trusted transparency log, the software vendor can discover the attack if it monitors the log. Note that transparency logs doesn't directly prevent attacks - it makes them discoverable.
Transparency systems based on transparency logs have five components: Log, Submitter, Client, Monitor and Auditor or Witness(es). The Log is a Merkle tree with its current root tree hash signed. In the example above the Client would be the software updating mechanism, which has a given Log in its trust policy. The Client will only trust a software update from the Submitter if the Submitter's signature and the update's hash are included in the Log. The Monitor regularly polls the Log for new entries it cares about, in this case software updates signed by the Submitter's public key. The Auditor is another party that regularly polls the Log, but the Auditor's job is not to discover log entries that constitute an attack. Instead, the Auditor's job is to monitor the Log's consistency - i.e. whether the log has removed any previously added entries. This is called a split-view attack, and it is a concern in all widely deployed transparency logs today. An alternative to Auditors is cosigning Witnesses, which proactively witness a Log's consistency, and whose witness signatures are required by the Client in order for it to trust the Log. This results in a much stronger threat model. See sigsum.org for more details on such a design.
Getting back to your comment:
- Yes, Clients are "thin", to use blockchain terminology, but they don't monitor the Log in order to detect attacks. The Log is part of their trust policy for making some trust decision, such as trusting a software update. Monitoring the Log for new entries that indicate e.g. key compromise is the Monitor's job.
- Regarding accountability: The Client's trust policy in conjunction with the Log and the Monitor increases the Submitter's accountability to the Client. Similarly the Auditors or Witnesses increases the Log's accountability to Clients, Submitter and Monitors.
- Regarding inclusion in the Log (or blockchain) I'd use the word "authority" as in who has the power to prevent something from being included. As you point out a transparency log is a central entity which has the sole power to decide what gets included or not. If the log is shut down, or its public key destroyed, nor more inclusions are possible. In this sense a blockchain is far more censorship-resistant than a transparency log. In other words, it is important that Clients trust multiple logs for redundancy. The consensus mechanism of a transparencty log is simply a public-key signature, as opposed to an emergent authority in the form of e.g. a distributed timestamp server based on SHA256. This vast difference in how the database's integrity is defined - the consensus mechanism - is why blockchains work better than transparency logs for censorship-resistant digital currency, whereas transparency logs work better than blockchains for efficient inclusion- and consistency proofs.
I generally agree with what you have said. However regarding this part
> Yes, Clients are "thin", to use blockchain terminology, but they don't monitor the Log in order to detect attacks
The clients can monitor the ledger via consistency proots and it is highly beneficial as that prevents any chances of split being undetected view without requiring to trust other parties in the process. Clients submiting the record to the chain they can get inclusion proofs, but they can also at random times can query that the previous chain root commitment is consistent with existing one in the process keeping consistency proof chain. If it isn't judiciary actions in the presence of evidence can be taken against the authorithy. This is particulary interesting when ledger hosts votes which allows for a fully centralized deployment which is the basis of an voting system I am developing [1].
> it is important that Clients trust multiple logs for redundancy
Replication solves the issue that the records would would always be available which can be done by anyone who has stakes in the integrity of the ledger. Wheras the censorship reistance is an emergent phenomena where multiple people organize and collect unincluded records and overthrows the authorithy. Thus there is no actually need for Clients to trust multiple logs. Clients only need to trust that they would be able to do incident response in the case of misbehaving authorithy.
[1]: https://janiserdmanis.org/artefacts/EVOTEID-2023-poster.pdf
> Yes, Clients are "thin", to use blockchain terminology, but they don't monitor the Log in order to detect attacks
The clients can monitor the ledger via consistency proots and it is highly beneficial as that prevents any chances of split being undetected view without requiring to trust other parties in the process. Clients submiting the record to the chain they can get inclusion proofs, but they can also at random times can query that the previous chain root commitment is consistent with existing one in the process keeping consistency proof chain. If it isn't judiciary actions in the presence of evidence can be taken against the authorithy. This is particulary interesting when ledger hosts votes which allows for a fully centralized deployment which is the basis of an voting system I am developing [1].
> it is important that Clients trust multiple logs for redundancy
Replication solves the issue that the records would would always be available which can be done by anyone who has stakes in the integrity of the ledger. Wheras the censorship reistance is an emergent phenomena where multiple people organize and collect unincluded records and overthrows the authorithy. Thus there is no actually need for Clients to trust multiple logs. Clients only need to trust that they would be able to do incident response in the case of misbehaving authorithy.
[1]: https://janiserdmanis.org/artefacts/EVOTEID-2023-poster.pdf
> The clients can monitor the ledger via consistency proots and it is highly beneficial as that prevents any chances of split being undetected view without requiring to trust other parties in the process. Clients submiting the record to the chain they can get inclusion proofs, but they can also at random times can query that the previous chain root commitment is consistent with existing one in the process keeping consistency proof chain.
The design you're describing doesn't have those security properties.
A malicious or compromised Log conducts a split-view attack by showing different views of the Log to different parties. This is done by providing different parties with different inclusion proofs and consistency proofs. Thus the Log remains consistent from the respective network vantage points of the target(s). They all see an append-only log.
A Client or Monitor shouldn't merely be concerned with whether the Log is consistent from their vantage point. It should be concerned with whether the Log is globally consistent, i.e. whether all views of the Log are consistent with each other. Historically this has led to designs where the Clients gossip consistency information to each other, the introduction of Log Auditors and witness cosigning. I prefer witness cosigning, although to be fair Clients gossiping can work in some designs. Apple's transparency log design is one example, if I remember correctly. It's been a while since I thought about transparency logs.
The design you're describing doesn't have those security properties.
A malicious or compromised Log conducts a split-view attack by showing different views of the Log to different parties. This is done by providing different parties with different inclusion proofs and consistency proofs. Thus the Log remains consistent from the respective network vantage points of the target(s). They all see an append-only log.
A Client or Monitor shouldn't merely be concerned with whether the Log is consistent from their vantage point. It should be concerned with whether the Log is globally consistent, i.e. whether all views of the Log are consistent with each other. Historically this has led to designs where the Clients gossip consistency information to each other, the introduction of Log Auditors and witness cosigning. I prefer witness cosigning, although to be fair Clients gossiping can work in some designs. Apple's transparency log design is one example, if I remember correctly. It's been a while since I thought about transparency logs.
> A malicious or compromised Log conducts a split-view attack by showing different views of the Log to different parties. This is done by providing different parties with different inclusion proofs and consistency proofs. Thus the Log remains consistent from the respective network vantage points of the target(s). They all see an append-only log.
You are spot on the vantage points. It is therefore at utmost necessity that the server does not know who has made a request, is not able to track clients. This is why requests need to be routed through anonymization service like TOR to preserve a single consitent view. The client only gives away the current root commit index they have stored locally which gives a little wiggle room there to be malicious. The latter can also be improved by first asking for the current tree root commitment and then aksing consitency proof with respect to locally stored commitment.
The global consistency in this design is achieved by the server not knowing who makes requests and thus gosiping is not needed.
You are spot on the vantage points. It is therefore at utmost necessity that the server does not know who has made a request, is not able to track clients. This is why requests need to be routed through anonymization service like TOR to preserve a single consitent view. The client only gives away the current root commit index they have stored locally which gives a little wiggle room there to be malicious. The latter can also be improved by first asking for the current tree root commitment and then aksing consitency proof with respect to locally stored commitment.
The global consistency in this design is achieved by the server not knowing who makes requests and thus gosiping is not needed.
We considered this design and rejected it because it requires an ultimately unwinnable cat-and-mouse game of fingerprint resistance.
Things the log can use to partition the users include their Tor version, HTTP library behavior, contact periodicity... the list goes on and on, and keeps restricting the use cases where it can be deployed securely even just in theory.
Witness cosigning is secure even if the way you fetch the proofs is completely attacker-controlled, which is closer to user expectations of regular digital signatures.
Things the log can use to partition the users include their Tor version, HTTP library behavior, contact periodicity... the list goes on and on, and keeps restricting the use cases where it can be deployed securely even just in theory.
Witness cosigning is secure even if the way you fetch the proofs is completely attacker-controlled, which is closer to user expectations of regular digital signatures.
For my application of the history trees (tansparency log) there are additional measures that help to ensure a global consistency view eventually. The authorithy would need to publish the final tally at the end of the vote which also would contain a tree root. The tree root would be distributed in print, in official anouncements thus any user can check that the tree root shown on the client coresponds to that with anounced tally and thus the authorithy would not know who does the check. Thus it would be unreasonably hard for adversary to deploy a deception attack at such scale.
Additional measure is that the TOR integration with the help of Arti project is deployed with the client. That ensures that every client does make the requests in the same way. It is surelly important to not disclose the server any local data or make identity revealing requests within the same session like giving away clients local commitment index before the server has shown their current commitment. Using anonymous channel for ensuring global consitency for sure is not universal but for some applications it is doable if the problem is approached holistically particularly in situations where anonymous channel is already needed within the protocol.
> Witness cosigning is secure even if the way you fetch the proofs is completely attacker-controlled
Opting for this approach makes sense if the protocol doesn't initially require an anonymous communication channel. However, if the protocol already uses it, introducing an additional assumption for trusted witnesses adds complexity.
Additional measure is that the TOR integration with the help of Arti project is deployed with the client. That ensures that every client does make the requests in the same way. It is surelly important to not disclose the server any local data or make identity revealing requests within the same session like giving away clients local commitment index before the server has shown their current commitment. Using anonymous channel for ensuring global consitency for sure is not universal but for some applications it is doable if the problem is approached holistically particularly in situations where anonymous channel is already needed within the protocol.
> Witness cosigning is secure even if the way you fetch the proofs is completely attacker-controlled
Opting for this approach makes sense if the protocol doesn't initially require an anonymous communication channel. However, if the protocol already uses it, introducing an additional assumption for trusted witnesses adds complexity.
> Any insight as to why you think blockchains can't handle this?
As you point out yourself the trust assumptions are fundamentally different. As a result a transparency log can produce an inclusion proof and a consistency proof that is on the order of a few kilobytes large. A trust policy for such a verifier would be of similar size.
How large would the equivalent inclusion- and consistency proofs for a proof-of-work-based blockchain be? What would the trust policy look like? Presumably the verifier would use some historical block as a starting point, which would constitute the trust policy. It would then need to process the entire blockchain from that point and onwards up until the point of inclusion. That's way more than a few kilobytes.
Again, I haven't paid attention to blockchains for a long time so there might be some consensus mechanism out there that has solved this somehow. I'd love to be proven wrong.
As you point out yourself the trust assumptions are fundamentally different. As a result a transparency log can produce an inclusion proof and a consistency proof that is on the order of a few kilobytes large. A trust policy for such a verifier would be of similar size.
How large would the equivalent inclusion- and consistency proofs for a proof-of-work-based blockchain be? What would the trust policy look like? Presumably the verifier would use some historical block as a starting point, which would constitute the trust policy. It would then need to process the entire blockchain from that point and onwards up until the point of inclusion. That's way more than a few kilobytes.
Again, I haven't paid attention to blockchains for a long time so there might be some consensus mechanism out there that has solved this somehow. I'd love to be proven wrong.
Yep; the only current advantage that blockchains have over current CT logs is trustable global transactions. This is a pretty powerful advantage, but the probabilistic certainty can be shifted from confirmation height to fraction of log inclusion and cost-of-revocation, e.g. transactions can be proposed in one or two logs and then confirmed by a supermajority of acceptable logs (to the transaction) once the ordering of potentially conflicting transactions is established within at least one log.
E.g. double-spends are prevented by announcing a transaction with a TTL that is included in at least one log, such that no conflicting transaction appears within any other log before the TTL expires, at which point most/all logs accept the permanent transaction.
Inclusion in a supermajority of logs provides the probabilistic likelihood of the transaction being durable (e.g. irreversible), and individual CT logs become invalid if they sign a later conflicting transaction.
Economic incentives could be as simple as charging customers for inclusion of their transaction in the log with escrow to refund all customers if the CT log becomes invalid. This can be implemented as its own CT-log transaction because logs have an incentive to cooperate with each other on signing off on the successful completion of a valid CT log (after a year or whatever) so they all keep getting paid.
E.g. double-spends are prevented by announcing a transaction with a TTL that is included in at least one log, such that no conflicting transaction appears within any other log before the TTL expires, at which point most/all logs accept the permanent transaction.
Inclusion in a supermajority of logs provides the probabilistic likelihood of the transaction being durable (e.g. irreversible), and individual CT logs become invalid if they sign a later conflicting transaction.
Economic incentives could be as simple as charging customers for inclusion of their transaction in the log with escrow to refund all customers if the CT log becomes invalid. This can be implemented as its own CT-log transaction because logs have an incentive to cooperate with each other on signing off on the successful completion of a valid CT log (after a year or whatever) so they all keep getting paid.
[deleted]
I agree. I have been toying with the idea of a system where you have a set of participants who each have responsibilities for maintaining some state that users and other participants rely on. For example, a network of banks including a central bank, but the network could also include merchants who rely on the banks for payments and trusted third parties who keep track of agreements such as loan contracts. Instead of putting everything on a blockchain, make each participant be the sole writer for one or more ledgers (e.g. one ledger per account they maintain) and use a non-repudiable two-phase commit protocol to coordinate multiple transactions performed atomically across ledgers.
Using the techniques from certificate transparency, as soon as a participant has published that they have performed a transaction on their ledger, then they cannot take that back again without an auditor finding out. They also cannot branch their ledgers (i.e. double spend) as long as all participants periodically gossip received transactions to the auditors. Or rather, they CAN, but the auditor will detect it eventually and have proof of it. What you get is a sort of multi-party SWIFT-like messaging system which doesn't require banks to change their basic accounting infrastructure and tokenize everything.
Using the techniques from certificate transparency, as soon as a participant has published that they have performed a transaction on their ledger, then they cannot take that back again without an auditor finding out. They also cannot branch their ledgers (i.e. double spend) as long as all participants periodically gossip received transactions to the auditors. Or rather, they CAN, but the auditor will detect it eventually and have proof of it. What you get is a sort of multi-party SWIFT-like messaging system which doesn't require banks to change their basic accounting infrastructure and tokenize everything.
One of the folks behind this effort, Filippo (https://filippo.io/) is now, as he calls it, a full time professional open source maintainer https://words.filippo.io/full-time-maintainer/
> That means I spend most of my time on maintenance, and I offer retainers to companies that benefit from my work and from access to my planning and my expertise.
He is trying a new way of life for highly-skilled OSS maintainers. Hopefully this works out well for him.
> That means I spend most of my time on maintenance, and I offer retainers to companies that benefit from my work and from access to my planning and my expertise.
He is trying a new way of life for highly-skilled OSS maintainers. Hopefully this works out well for him.
I knew I had heard of this guy before! I enjoyed this project of his: https://filippo.io/behindthesofa/
[deleted]
Transparency logging is a fantastic complement to public-key cryptography.
A prioritised list of the most important cryptographic constructions may very well look like this: symmetric ciphers, hash functions, MACs, public-key cryptography, transparency logging.
If you're interested in security and/or cryptography and you don't know what transparency logging is, you're missing out.
A prioritised list of the most important cryptographic constructions may very well look like this: symmetric ciphers, hash functions, MACs, public-key cryptography, transparency logging.
If you're interested in security and/or cryptography and you don't know what transparency logging is, you're missing out.
[deleted]
This is probably a step forward for HTTPS, but it still bugs me that most of the internet is running on a cryptography system that doesn't provide confidentiality or integrity.
Instead, it makes it possible for you to detect that someone's been using a rogue certificate after the fact. It bugs me even more that many security certification folks insist on using this junk to manage SSH keys (out of the box, SSH provides stronger security guarantees than X.509 certificates).
The whole situation reminds me of Crypto AG:
https://en.wikipedia.org/wiki/Crypto_AG
Instead, it makes it possible for you to detect that someone's been using a rogue certificate after the fact. It bugs me even more that many security certification folks insist on using this junk to manage SSH keys (out of the box, SSH provides stronger security guarantees than X.509 certificates).
The whole situation reminds me of Crypto AG:
https://en.wikipedia.org/wiki/Crypto_AG
It probably won't happen, but I'd love to see the concept of channel binding reach the web ecosystem. It allows you to securely verify the certificate, independent of any third party. However it only works when you and the server have a shared secret (which could be a password or something else). It therefore isn't really applicable to casual "anonymous" browsing. But for that kind of browsing, a third-party attestation (e.g. from a CA) is probably good enough anyway.
We use channel binding in the XMPP ecosystem and it's going well.
We use channel binding in the XMPP ecosystem and it's going well.
If you're ok with TOFU ala ssh you can just use self-signed certificates. Skipping the browser warning is the "trust" equivalent.
I'm biased, of course, because I work for Let's Encrypt, and spend a lot of time thinking about the trade-offs and how to make this ecosystem better.
I think claiming that "SSH provides stronger security guarantees than X.509 certificates" is pretty outlandish.
SSH is trust-on-first-use. How many people actually verify the key fingerprints? Maybe you do when you're sshing to your single corporate bastion, or to Github.com, but could you imagine if you had to do that for every website? It simply would not work. It only barely does for SSH because of the (relatively) low amount of use SSH gets. When at a previous company I set up an SSH CA, we found an SSH MITM within 3 days (though benign; an ISP-provided modem/router was resolving all hostnames to itself when the WAN was down), and the user was planning on just accepting the new SSH host key.
Certificate Transparency is an auditing system, so of course it's after-the-fact. It has helped to improve the CA system immensely. While there's still work to do, the TLS ecosystem is unrivalled in terms of actual security benefit delivered to the most people.
I think claiming that "SSH provides stronger security guarantees than X.509 certificates" is pretty outlandish.
SSH is trust-on-first-use. How many people actually verify the key fingerprints? Maybe you do when you're sshing to your single corporate bastion, or to Github.com, but could you imagine if you had to do that for every website? It simply would not work. It only barely does for SSH because of the (relatively) low amount of use SSH gets. When at a previous company I set up an SSH CA, we found an SSH MITM within 3 days (though benign; an ISP-provided modem/router was resolving all hostnames to itself when the WAN was down), and the user was planning on just accepting the new SSH host key.
Certificate Transparency is an auditing system, so of course it's after-the-fact. It has helped to improve the CA system immensely. While there's still work to do, the TLS ecosystem is unrivalled in terms of actual security benefit delivered to the most people.
> When at a previous company I set up an SSH CA, we found an SSH MITM within 3 days (though benign; an ISP-provided modem/router was resolving all hostnames to itself when the WAN was down), and the user was planning on just accepting the new SSH host key.
Wouldn't that be a problem with TOAU? :)
Wouldn't that be a problem with TOAU? :)
Sorry, I'm having trouble understanding that acronym. What do you mean?
In this case it was a TOFU error, as somebody was trying to SSH to a new AWS instance, so they'd have trusted the certificate "on first use", even though they were actually SSHing into their network's router.
In this case it was a TOFU error, as somebody was trying to SSH to a new AWS instance, so they'd have trusted the certificate "on first use", even though they were actually SSHing into their network's router.
> While there's still work to do, the TLS ecosystem is unrivalled in terms of actual security benefit delivered to the most people.
Well said!
Well said!
With SSH, if you care, you just need to check the fingerprints. With HTTPS, you have to implement certificate pinning (or set up a private CA and configure everything to ignore the system roots of trust), which is much more onerous.
Thanks for helping to improve https, since it helps in practice.
edit: Another way to put it is that if you want to reduce your attack surface to just the client and server machine, it's easy enough to scale SSH fingerprint verification to a few dozen machines. It's much, much harder to get SSL to provide that property for even two machines.
As for scaling to 100's of machines, it's not obvious to me that bootstrapping and rsyncing a known_hosts file and /etc/ssh_config that turns off trust on first use is harder than setting up a CA.
For millions of machines, SSH falls over (because known_hosts would be too big), but at that point, you're talking about what percentage of machines are compromised in steady state, not how to keep all attackers out. That's where HTTPS works well.
Thanks for helping to improve https, since it helps in practice.
edit: Another way to put it is that if you want to reduce your attack surface to just the client and server machine, it's easy enough to scale SSH fingerprint verification to a few dozen machines. It's much, much harder to get SSL to provide that property for even two machines.
As for scaling to 100's of machines, it's not obvious to me that bootstrapping and rsyncing a known_hosts file and /etc/ssh_config that turns off trust on first use is harder than setting up a CA.
For millions of machines, SSH falls over (because known_hosts would be too big), but at that point, you're talking about what percentage of machines are compromised in steady state, not how to keep all attackers out. That's where HTTPS works well.
Also see the discussion on the ct-policy mailing list: https://groups.google.com/a/chromium.org/g/ct-policy/c/v9Jzl...
(I am the author of the blog post)