DNS Key Value Storage(dnskv.com)
dnskv.com
DNS Key Value Storage
https://dnskv.com/
32 comments
What is the option 'd to pass a custom domain key'? What is a domain key?
Cool toy:
;; ANSWER SECTION:
pressbutton.dnskv.com. 604740 IN TXT "nannal"
dig TXT nannal.com|grep status
You'd think DNS KV Store's a "nice figment of imagination", but one of the teams I worked for did use DNS TXT records genuinely as a (faster) way to reconcile state between the control plane and the data plane.
It sounded ugly on-paper, but the design was approved by everyone in the chain of command for what I presume was for lack of other viable options given the architecture.
And in ironic turn of events, instead of using DNS for service discovery, another set of engineers in the team opted for EC2 metadata (for example, security groups) instead.
It sounded ugly on-paper, but the design was approved by everyone in the chain of command for what I presume was for lack of other viable options given the architecture.
And in ironic turn of events, instead of using DNS for service discovery, another set of engineers in the team opted for EC2 metadata (for example, security groups) instead.
It should not be surprising to anyone that long-established protocol standards are capable and versatile. DNS and LDAP are robust tree-structured replicated attribute stores; mix in Kerberos for a complete foundation of general purpose directory services. NNTP supplies a straightforward Gossip protocol for eventually-consistent distributed pub/sub. MQTT is still good for lightweight telemetry. I have services I authenticate with via Lamport's mostly forgotten scheme (S/KEY). And so on.
There is a current fad of HTTP+JSON as a generic protocol substrate, but it's almost always a mediocre fit for the problem at hand. Go read the RFC archives, there's diamonds to be found.
On the other hand, and perhaps even by the same token: using EC2 metadata isn't a completely terrible idea if it's the infrastructure you already have and the semantics line up with specific needs.
There is a current fad of HTTP+JSON as a generic protocol substrate, but it's almost always a mediocre fit for the problem at hand. Go read the RFC archives, there's diamonds to be found.
On the other hand, and perhaps even by the same token: using EC2 metadata isn't a completely terrible idea if it's the infrastructure you already have and the semantics line up with specific needs.
I did something similar (for personal use) for a sort of 'peer discovery' where load balancing didn't actually matter, I just wanted to pick one of N without knowing where they are ahead of time, using IPVS: https://github.com/OJFord/ipvs-txt-lb
To be honest I'm not sure if it's a total hack or a reasonable use as intended, or somewhere in between. But it got it done, and is at least more elegant than hard coding something arbitrary which may change (just not too frequently), I think.
To be honest I'm not sure if it's a total hack or a reasonable use as intended, or somewhere in between. But it got it done, and is at least more elegant than hard coding something arbitrary which may change (just not too frequently), I think.
DNS SRV records seem appropriate for that.
Using TXT records for this kind of thing is not that unusual. For example, Kubernetes's external-dns uses TXT records to track which entries it manages.
I've used the AWS metadata trick as well. I kind of liked the solution; better use something that the machine is already relying on than adding something extra.
DNS is also usually a much better option than the framework-specific service discovery mechanism that every framework seems to be destined to reinvent. I think the problem is that most computer scientists aren't very well-versed in networking, and DNS doesn't look like the best match at first glance.
DNS is also usually a much better option than the framework-specific service discovery mechanism that every framework seems to be destined to reinvent. I think the problem is that most computer scientists aren't very well-versed in networking, and DNS doesn't look like the best match at first glance.
I've done the same - tag EBS volumes with metadata rather than having to maintain a separate datastore for a tiny amount of information. Worked well.
The Hesiod system [0] used at MIT in the 80s is based on DNS. It stored things like user (passwd) entries and which is the mail server for an account as DNS TXT records. You could just query the DNS server but there were also dedicated tools. [1]
[0] https://debathena.mit.edu/trac/wiki/Hesiod [1] http://www.mit.edu/afs.new/athena/project/ctraining/presenta...
[0] https://debathena.mit.edu/trac/wiki/Hesiod [1] http://www.mit.edu/afs.new/athena/project/ctraining/presenta...
There is still Hesiod support in GlibC / NSS, etc on most modern systems.
When put together with Kerberos it's a nice way to provision auth on lots of machines automatically.
It's nice to have a dedicated and restricted resolver config so that the zone visibility can be restricted but that makes deployment a little more complex.
It's nice to have a dedicated and restricted resolver config so that the zone visibility can be restricted but that makes deployment a little more complex.
+100 for use Hesiod. It solved this problem well and correctly years ago, and just works. (It was also a key part of the support for Carnegie-Mellon's Andrew networked filesystem.)
The seamless integration of all the Project Athena stuff (Hesiod, Kerberos, etc.) with DEC Ultrix is one of many reasons that Ultrix is still, IMO, the greatest Unix/;Unix-like distro in history.
The seamless integration of all the Project Athena stuff (Hesiod, Kerberos, etc.) with DEC Ultrix is one of many reasons that Ultrix is still, IMO, the greatest Unix/;Unix-like distro in history.
and of course the venerable NIS / yp lived parallel to DNS in most organizations.
With AWS Route 53's 10 000 records/zone, 400 values/record 255 chars/TXT and base64's ~35% overhead, you have a bit over 600 megabytes of binary value storage.
I don’t understand this:
I’m probably missing something about how these records are meant to be used together. Is there a protocol doc available?
A, AAAA, TXT - set record
A, AAAA - check key
TXT - read value
I’m probably missing something about how these records are meant to be used together. Is there a protocol doc available?
The page says:
Read more of the page to find out what else you can do.
Insert:
· value.key.
$ dig txt value.key.dnskv.com
$ dig txt key.dnskv.com
Read more of the page to find out what else you can do.
thanks, the docs could be more verbose.
for those who don't know how does the updating work:
dig txt u[update-secret].value.key.dnskv.com # set for first time
dig txt u[update-secret].new-value.key.dnskv.com # use the same update secret, noone can now use this key without it until expiry
dig @ns.dnskv.com. txt ...
I can't find any docs other than what's on the page but my reading suggests that the idea is that you can get/set values "in-band" using a regular lookup query to a resolver rather than the nsupdate style that you'd normally use to remotely configure zones.
I interpreted this as:
insert with any type of query to value.key.dnskv.com
check if key is set with A or AAAA query to key.dnskv.com
read value of key with TXT query to key.dnskv.com
insert with any type of query to value.key.dnskv.com
check if key is set with A or AAAA query to key.dnskv.com
read value of key with TXT query to key.dnskv.com
Corey Quinn from Last Week in AWS jokes about using Route53 as a db often, including this hilarious twitter thread [0].
[0] https://twitter.com/quinnypig/status/1120653859561459712
[0] https://twitter.com/quinnypig/status/1120653859561459712
Oh the Route 53 team had the same half joke as well. DNS really is a good match for fast globally distributed reads and slowish writes to a hierarchal k/v store. Lots of customers have figured this out to better (or worse) effect.
Source: Principal at AWS, worked on route 53 for a few years.
Source: Principal at AWS, worked on route 53 for a few years.
* DNS really is a good match for fast globally distributed reads and slowish writes to a hierarchal k/v store*
sounds like another good candidate for distributed TimeSeries DB or DHT type of things.
sounds like another good candidate for distributed TimeSeries DB or DHT type of things.
I don't get what the problem is. He used DNS as a name server, which is what it is. The only weird thing is TXT instead of CNAME.
I tend to like Cloudflare's API for key/value storage.
If you don't delete the record, the key will always have the same id-number, so you never need more than a single API call to reference a value or change it.
If you don't delete the record, the key will always have the same id-number, so you never need more than a single API call to reference a value or change it.
We used (route 53) private DNS records for feature toggles in my previous company; worked better than most of the other solutions tbh.
That's a really good use case IMO.
DNS is actually pretty nice for ACL [1]. Just make sure you use it with DNSSEC.
Want to check if Tom has access to /web/mike? Just do a lookup on the user in question with the reversed path you want to check.
"Can Tom read /web/mike?" => tom.mike.web.server.net
Advantages:
- Replication to multiple machines is trivial (piggybacking of DNS, after all).
- Lookups are fast. Can also be easily cached.
- All sane programming languages have built-in DNS resolution in their stdlib out-of-the-box.
- Manual override using /etc/hosts for debugging/emergency.
1. https://itk.samfundet.no/dok/ITKACL2
Want to check if Tom has access to /web/mike? Just do a lookup on the user in question with the reversed path you want to check.
"Can Tom read /web/mike?" => tom.mike.web.server.net
Advantages:
- Replication to multiple machines is trivial (piggybacking of DNS, after all).
- Lookups are fast. Can also be easily cached.
- All sane programming languages have built-in DNS resolution in their stdlib out-of-the-box.
- Manual override using /etc/hosts for debugging/emergency.
1. https://itk.samfundet.no/dok/ITKACL2
I love hacks for DNS, but this one feels a little bit of a stretch. The absolute lack of security is probably the biggest one. Now one can troll DNS for a user who has access to a resource.
Authentication != Authorisation.
You wouldn't use this to determine if the user really is who he says he is. But in determining what a user has access to, DNS is pretty slick. The DNS server doesn't need to be public either.
You wouldn't use this to determine if the user really is who he says he is. But in determining what a user has access to, DNS is pretty slick. The DNS server doesn't need to be public either.
Doesn't necessarily have to be public dns. Could be internal to just your servers...
This is just plain sexy.
The data for the RR was a series of key=value pairs.
IIRC it would look something like this in a zone file...
foo TYPE "foo=bar" "baz=qux"