> Anyone know why they used UTF-16 for the message type, instead of UTF-8?
It is an artefact of the original implementation, which was not discovered until the protocol was already in wide use and being independently implemented [1].
> Also, I'm skeptical that two peers can exchange keys without a third party, without being susceptible to man-in-the-middle attacks. But maybe the paper linked in the article proves that it's possible?
SSB uses the Secret Handshake (SHS) protocol in that paper (with some errata [2]). SHS is a authenticated key exchange (key agreement) protocol [3]. The two peers authenticate eachother to their respective public key, and establish a shared secret that is used to bulk-encrypt the rest of the session/connection. With SHS, the client (the peer that initiates the connection) must know the server's public key ahead of time. Both parties must know and previously agree on an additional network capability key (that is usually hard-coded to a specific value in the SSB implementations).
It should be immune to MitM if the party's private keys are kept secret. There are ephemeral keypairs involved, so if a later compromise occurs of the long-term (identity) private keys, that should not reveal previous/existing sessions. SHS has been verified using Tamarin [4].
> I wonder if anyone's made an STM that runs on a readonly transaction log like this, kind of like CouchDB, RethinkDB or Firebase maybe?
I'm not familiar with STM as such, but I am familiar with CouchDB. There are various ways of mutable data structures on SSB. Typically messages are indexed, in general ways (e.g. message type, author, backlinks) and/or application-specific ways. Applications query the indexes to construct some result. Graph processing is often done to handle concurrent operations by different feeds (i.e. using CRDTs).
> Could this be used to build a web of trust? Or is it meant to be more transient, like maybe people broadcast on throwaway identities? Could we drop PGP into this?
Yes, the main SSB network constitutes a web of trust. People do create throwaway identities though, just trying it out and then not returning. But some persist, and people develop and express relationships. Some people share other public keys for PGP, OMEMO, Briar, RetroShare, Dat/Hyper, etc. PGP-signed messages have been published occasionally. Creating a temporary identity is not recommended for broadcasting, because it will not have much reach: message distribution and visibility depends on the social graph.
> Maybe this is more like an RSS feed than something realtime like WebRTC?
Yes. A SSB feed is identified by its public key, and contains an append-only list of messages. Each message is identified by its content hash. However, the RPC protocol used for SSB could be extended to support ephemeral content.
WebRTC DataChannels could be used for gossip connections. But there is still the problem of needing to exchange message to establish the WebRTC connection. Historically SSB has addressed P2P network architecture using Pubs [5], more recently with Rooms [6].
> It seems weird that you would write a formal, generalized protocol specification relying on the idiosyncratic implementation details of JavaScript, for such an important thing as cryptographic signatures, as the Scuttlebutt specification seems to do here.
The Protocol Guide was created after the initial implementation and its protocol were already in wide use, and the quirks were discovered while re-implementing it.
More info about implementations here (Node.js, Go, Rust x2, and Python; additionally there are implementations of varying states in Java, C, Haskell, Erlang and probably others):
https://dev.scuttlebutt.nz/#/?id=implementations
---
If making a new protocol using signatures over JSON objects, one might use this:
JWS Clear Text JSON Signature Option (JWS/CT)
https://datatracker.ietf.org/doc/html/draft-jordan-jws-ct
JWS/CT uses JSON Web Signature (JWS) [RFC7515], JSON Canonicalizion Scheme (JCS) [RFC8785], and I-JSON [RFC7493] (subset of JSON for interoperability)
It is preferable that users bring their own DIDs, rather than you assigning them DIDs as a service provider, to ensure user autonomy. However, you can create DIDs via websites as you are suggesting, using did:web [1].
DIDs are structured or namespaced by DID Methods: the part after "did:" is called the DID Method name, and each Method has its own specification and implementations. There is a registry of DID Methods [2], although it is not a requirement that DID methods be registered. There is a document comparing some DID methods using a Rubric: [3].
> [...] instead of registering a separate account with my website, she can instead type "did:identity.dns.xyz:123456789abcdefghijk" into my website?
It can obviate account registration per website, yes.
For usability reasons it is often considered desirable that users not have to know about DIDs, type them or see them. The functionality could instead be handled by the user's software. There is a browser standard and polyfill in development which can be used for this: Credential Handler API [4].
> Does my site's software then contact identity.dns.xyz to ask it something? Or is all the information my site needs to do its thing contained in the DID itself?
If you have "did:identity.dns.xyz:...", it would depend on what the "identity.dns.xyz" DID method is. If you don't know what the "identity.dns.xyz" DID method is, you could ask a DID resolver that you trust; but they might not know either. If instead you had "did:web:identity.dns.xyz:...", you would indeed contact `identity.dns.xyz` over HTTPS to request the respective DID document. If you instead have a "did:key:..." DID [5], you would indeed have all the information contained in the DID itself, as did:key encodes the public key. In other kinds of DID methods, you may need to contact a peer-to-peer network to resolve the DID.
> Is "123456789abcdefghijk" the hash of some document that's returned to my website by the identity.dns.xyz server?
It depends on the DID method. If you are using did:web, which is based on HTTPS as mentioned, there is no hash, although this could be handled using hashlinks [6], which is mentioned in the did:web specification as a TODO. In other DID methods, the method-specific-id (the "123456789abcdefghijk" part) is a hash which might refer to some static data or initialization state.
> Or is Alice running some identity management browser extension that knows to present a document whose hash is "123456789abcdefghijk" to my website?
Possibly, depending on the DID method and how it is implemented. But DIDs are generally expected to be public, globally resolvable, and highly available.
> [...] is "123456789abcdefghijk" the hash of a public key which signs something
Could be, again depending on the DID method. Some DID methods are based on decentralized ledger technologies where the method-specific-id represents an account id derived from public key hash, e.g. did:ethr [7] and did:tz [8].
> [...] {returned by the identity.dns.xyz server | presented by Alice's software}?
Yes, if you are authenticating the user with a DID, generally you would ask them to sign a challenge using authentication key material from their DID document. The DID document is what the DID resolves to, and it includes public keys. In the case of did:key, the DID document contains a single key as encoded in the DID. In the case of a DID method based on a public key hash, the public key must be provided or looked up somehow, or the signing algorithm must support public key recovery [9].
> Or is "123456789abcdefghijk" just a string your webserver happened to generate to identify Alice uniquely that came from /dev/urandom or a PRIMARY KEY column in your database that has no cryptographic meaning?
The DID method determines the structure and meaning of the method-specific-id (the "123456789abcdefghijk" part). If you are using did:web, the string is not cryptographic but corresponds to a HTTPS URL.
> If Alice's DID document is cryptographically linked to her DID, how does she update it?
She performs a DID document update operation [10]. If the DID method is based on a decentralized ledger technology (e.g. btcr [11], ethr, tz), she might publish a transaction to the corresponding network or make a call on a "smart contract". in the case of a did:key, the DID document cannot be updated.
> If the DID is the hash of the document, does that mean Alice gets a new DID whenever she edits her profile on dns.xyz?
If the DID method is static (basing the DID on the document hash entirely), it would probably have to be a new DID, but if it is only an initial document hash, the DID method could provide for verifying and applying updates somehow. There are also some properties for indicating equivalence between DIDs which might be useful [12].
A DID document could also be updated while still using hashes for integrity protection and referencing if the DID method uses the hash as the DID document version ID. A DID can be resolved at a given version ID using the versionId parameter [13].
For privacy, generally DID documents should not include personal data [14]. Edits to a profile on would then only affect DID documents if they are updating keypairs, or indicating relationships with other DIDs (e.g. using alsoKnownAs), or updating service endpoints, etc.
For decentralization and user autonomy, users should be able to update their DID documents with their software directly, rather than having a website in control of their DID and having to ask the website to update it. In the web context that may mean a browser extension or key management API. But there are institutional use cases where the website is expected to have control.
> How does my website know that the user presenting Alice's DID document is Alice sending her most up-to-date DID document, rather than Eve sending an outdated DID document with the compromised Ethereum address that once belonged to Alice [...]
Often DIDs are resolved publicly or by contacting a network rather than via asking the user (did:peer [15] is a different case). But the concern still applies. Updating a DID document to remove a compromised key is called revocation [16]. The DID method should have some protocol for how updates are performed, which may include how they are ordered, such as by being witnessed or confirmed by some entity or network. The question of the valid state of a DID document should be abstracted by your DID resolver [17].
> Is a DID like a generalized Bitcoin address, a generalized IPNS name, a generalized email address, or something else entirely?
More like a generalized Bitcoin address I think, although there are comparisons to be made with the others. Another idea: generalized PGP keys. It is primarily about identity, rather than about communication, storage or payments.
For more info I recommend checking out the Decentralized Identity Foundation (DIF)'s FAQ: "What is a DID?" [18].
It is an artefact of the original implementation, which was not discovered until the protocol was already in wide use and being independently implemented [1].
> Also, I'm skeptical that two peers can exchange keys without a third party, without being susceptible to man-in-the-middle attacks. But maybe the paper linked in the article proves that it's possible?
SSB uses the Secret Handshake (SHS) protocol in that paper (with some errata [2]). SHS is a authenticated key exchange (key agreement) protocol [3]. The two peers authenticate eachother to their respective public key, and establish a shared secret that is used to bulk-encrypt the rest of the session/connection. With SHS, the client (the peer that initiates the connection) must know the server's public key ahead of time. Both parties must know and previously agree on an additional network capability key (that is usually hard-coded to a specific value in the SSB implementations).
It should be immune to MitM if the party's private keys are kept secret. There are ephemeral keypairs involved, so if a later compromise occurs of the long-term (identity) private keys, that should not reveal previous/existing sessions. SHS has been verified using Tamarin [4].
> I wonder if anyone's made an STM that runs on a readonly transaction log like this, kind of like CouchDB, RethinkDB or Firebase maybe?
I'm not familiar with STM as such, but I am familiar with CouchDB. There are various ways of mutable data structures on SSB. Typically messages are indexed, in general ways (e.g. message type, author, backlinks) and/or application-specific ways. Applications query the indexes to construct some result. Graph processing is often done to handle concurrent operations by different feeds (i.e. using CRDTs).
Here is a document describing threads, a common data structure on SSB: https://hackmd.io/GQ8aTw6STpuSFu6oH5Z63w
> Could this be used to build a web of trust? Or is it meant to be more transient, like maybe people broadcast on throwaway identities? Could we drop PGP into this?
Yes, the main SSB network constitutes a web of trust. People do create throwaway identities though, just trying it out and then not returning. But some persist, and people develop and express relationships. Some people share other public keys for PGP, OMEMO, Briar, RetroShare, Dat/Hyper, etc. PGP-signed messages have been published occasionally. Creating a temporary identity is not recommended for broadcasting, because it will not have much reach: message distribution and visibility depends on the social graph.
> Maybe this is more like an RSS feed than something realtime like WebRTC?
Yes. A SSB feed is identified by its public key, and contains an append-only list of messages. Each message is identified by its content hash. However, the RPC protocol used for SSB could be extended to support ephemeral content.
WebRTC DataChannels could be used for gossip connections. But there is still the problem of needing to exchange message to establish the WebRTC connection. Historically SSB has addressed P2P network architecture using Pubs [5], more recently with Rooms [6].
[1] https://news.ycombinator.com/item?id=29675263
[2] https://github.com/auditdrivencrypto/secret-handshake/issues...
[3] https://en.wikipedia.org/wiki/Authenticated_Key_Exchange
[4] https://github.com/keks/tamarin-shs
[5] https://ssbc.github.io/scuttlebutt-protocol-guide/#pubs
[6] https://ssb-ngi-pointer.github.io/rooms2/