HTML over DNS: Serving Blog Content over DNS(jacobkiers.net)
jacobkiers.net
HTML over DNS: Serving Blog Content over DNS
https://jacobkiers.net/hod/
76 comments
>Author here: I made it as a proof of concept, just because I could.
Sometimes those are the most satisfying and technologically interesting proof of concepts. Don't let any of the detractors here tell you you're wasting your time!
Sometimes those are the most satisfying and technologically interesting proof of concepts. Don't let any of the detractors here tell you you're wasting your time!
Well the content is stored in a DNS Zone file but it is requested using JavaScript to an external HTTP API. I wouldn't really call that HTML over DNS but rather "DNS Zone as blog database".
Anyway, this made me think of iodine [1], an IP over DNS solution, which I still run on my main server even though it has a lot less use now than it had until a few years ago when there were a lot of open wifi with captive portals and way less 4G available.
[1] https://code.kyro.se/iodine/
Anyway, this made me think of iodine [1], an IP over DNS solution, which I still run on my main server even though it has a lot less use now than it had until a few years ago when there were a lot of open wifi with captive portals and way less 4G available.
[1] https://code.kyro.se/iodine/
Or just call it HTML over CloudFlare? That CloudFlare uses DNS on the backend doesn't validate DNS being in the title IMHO
HTML over DoH maybe; i imagine other providers also have an open CORS policy.
the site is down? - https://code.kyro.se/iodine/
Sorry I was on my smartphone and mistyped the url (it's kryo not kyro): https://code.kryo.se/iodine/
Thanks for pointing it out.
EDIT: As an aside, I still hate how mobile browsers have the bad habits of reloading tabs (and most of the time loosing content) that you get back to with no reason. If not for this crappy behavior, I would have copy-pasted the URL.
Thanks for pointing it out.
EDIT: As an aside, I still hate how mobile browsers have the bad habits of reloading tabs (and most of the time loosing content) that you get back to with no reason. If not for this crappy behavior, I would have copy-pasted the URL.
On a related note, when the autocomplete system sees that you are typing a URL, it could suggest URLs from browser history as completions, with the most recently visited URLs being shown first. Although, some URLs are long. And it’d suck to complete to the wrong URL, both for reasons of embarrassment in some cases and for reasons of privacy on other cases. So I guess it’s better that autocomplete don’t do that. Also it would probably need to show both title and a thumbnail of the page in the preview because many URLs don’t contain info about what’s on the page. Would be hard to fit sensibly on the screen.
The HTML bit is fun, but the more remarkable takeaway for me is that DoH servers accept cross-origin requests from ordinary javascript. This means two things:
- A website can bring its own DoH client and bypass both the OS resolver and the browser's trusted DoH resolver for anything except the initial page request.
- Any website can now access the full DNS information of any domain: Not just A/AAAA records, but also TXT, MX, SRV etc. Record metadata such as TTLs likewise.
All of that without requiring any backend infrastructure or exotic web API. It's literally just a static HTML file and fetch().
That's a genuinely new capability that wasn't available to websites before public DoH servers became available. I'm no security expert, but this smells like it should have some implications for web security.
- A website can bring its own DoH client and bypass both the OS resolver and the browser's trusted DoH resolver for anything except the initial page request.
- Any website can now access the full DNS information of any domain: Not just A/AAAA records, but also TXT, MX, SRV etc. Record metadata such as TTLs likewise.
All of that without requiring any backend infrastructure or exotic web API. It's literally just a static HTML file and fetch().
That's a genuinely new capability that wasn't available to websites before public DoH servers became available. I'm no security expert, but this smells like it should have some implications for web security.
> - A website can bring its own DoH client and bypass both the OS resolver and the browser's trusted DoH resolver for anything except the initial page request.
Not sure about that, since it doesn’t sound like simply requesting answers from the DOH server injects the answer into the DNS cache.
And I don’t think this is novel, since anyone could have ran a dns info api before if they really wanted to.
Not sure about that, since it doesn’t sound like simply requesting answers from the DOH server injects the answer into the DNS cache.
And I don’t think this is novel, since anyone could have ran a dns info api before if they really wanted to.
> Not sure about that, since it doesn’t sound like simply requesting answers from the DOH server injects the answer into the DNS cache.
True about that. While you can find out the real IP address of a host even if the os/browser blackholes it, it's more difficult to actually send requests to it: You can pass IP addresses to fetch(), but those will then also be used for the Host header, SNI and cert validation - not what you'd usually want. To make this work, you'd need a "connect to IP address X but pretend it's host Y" option for fetch - something that is probably not supported for obvious reasons.
I think it should still be possible to connect to WebRTC peers or to implement a tamper detection.
> And I don’t think this is novel, since anyone could have ran a dns info api before if they really wanted to.
If you ran your own API, you'd have server infrastructure to maintain. If you used a 3rd-party API, you'd have to deal with accounts, tokens, rate-limiting, etc. In contrast, this DoH server seems to be open to anyone, without any kind of registration required.
I think this makes this feature significantly cheaper to use.
True about that. While you can find out the real IP address of a host even if the os/browser blackholes it, it's more difficult to actually send requests to it: You can pass IP addresses to fetch(), but those will then also be used for the Host header, SNI and cert validation - not what you'd usually want. To make this work, you'd need a "connect to IP address X but pretend it's host Y" option for fetch - something that is probably not supported for obvious reasons.
I think it should still be possible to connect to WebRTC peers or to implement a tamper detection.
> And I don’t think this is novel, since anyone could have ran a dns info api before if they really wanted to.
If you ran your own API, you'd have server infrastructure to maintain. If you used a 3rd-party API, you'd have to deal with accounts, tokens, rate-limiting, etc. In contrast, this DoH server seems to be open to anyone, without any kind of registration required.
I think this makes this feature significantly cheaper to use.
It shouldn't because this was always possible if someone really cared to just run their own public server with a /dig endpoint. If web security depends on websites not having access to public data that could always be proxied to it then we're already screwed.
> It shouldn't because this was always possible if someone really cared to just run their own public server with a /dig endpoint.
A lot of web features had always been possible by simply running them on your own server: Cross-origin requests, generating images on-demand, rendering vector graphics, etc. Nevertheless, when those features became available in the browser - without any additional effort needed from the developer - it had massive effects on the web ecosystem.
I can't say I have answers, but my suspicion is that it makes a significant difference in friction whether a feature requires you to setup and run your own infrastructure or whether you just need to type in some javascript to use it.
> If web security depends on websites not having access to public data that could always be proxied to it then we're already screwed.
To some extent it does - that's why you cannot access the body cross-origin GET requests or the contents of cross-origin iframes without those sites opting in - even if no cookies or other credentials are sent with the reqest.
A lot of web features had always been possible by simply running them on your own server: Cross-origin requests, generating images on-demand, rendering vector graphics, etc. Nevertheless, when those features became available in the browser - without any additional effort needed from the developer - it had massive effects on the web ecosystem.
I can't say I have answers, but my suspicion is that it makes a significant difference in friction whether a feature requires you to setup and run your own infrastructure or whether you just need to type in some javascript to use it.
> If web security depends on websites not having access to public data that could always be proxied to it then we're already screwed.
To some extent it does - that's why you cannot access the body cross-origin GET requests or the contents of cross-origin iframes without those sites opting in - even if no cookies or other credentials are sent with the reqest.
A script could already bypass DNS by connecting directly to an IP address which could have been either hard coded or come from an API, for example. You could also send it all the other records via some API too. This just means people don't have to implement the service themselves.
Corollary is, spyware can use DNS to exfiltrate data [0]. Or, send out client-side metrics with cleverly drafted DNS requests [1], or use it as a 3p-cookie replacement [2].
[0] https://unit42.paloaltonetworks.com/dns-tunneling-how-dns-ca...
[1] https://github.com/Jigsaw-Code/choir (disclosure: I co-develop hard-forks of two other related Jigsaw-Code projects)
[2] http://dnscookie.com/
[0] https://unit42.paloaltonetworks.com/dns-tunneling-how-dns-ca...
[1] https://github.com/Jigsaw-Code/choir (disclosure: I co-develop hard-forks of two other related Jigsaw-Code projects)
[2] http://dnscookie.com/
Fun story about this. I know one of the former guys at Xobni back in the day when it was still a thing. They had made an enterprise version that was sold to companies like Apple, Intel, etc. But even though the enterprise software no longer made external API calls, the company still wanted usage metrics so they could monitor feature usage. Knowing how locked down the networks were at companies like this, they exfiltrated the usage data via infrequent DNS requests to their DNS server. Everything was going fine until one day a dozen or so users at Intel stopped reporting. Hours later all of Intel stopped reported. Within the next day other companies started coming offline. Apple first, then some other large tech company and then a whole slew of them.
Turns out when you spy on your users word gets around.
Turns out when you spy on your users word gets around.
Packet size is a really good detection for this. DNS packets should only be so big ;)
So I split my packets up into smaller chunks. Now you need to rate limit DNS. But you still need to allow regular DNS traffic somehow, or you break the internet.
Well, allowlist to a known DNS server and block anything on DNS that isn't allowlisted.
The exfiltrated data isn't for DNS resolvers but is leaked to Nameservers through them.
A similar technique forms the basis of how services like dnsleaktest.com and which.nameserve.rs identify DNS resolvers in-use by a particular client.
A similar technique forms the basis of how services like dnsleaktest.com and which.nameserve.rs identify DNS resolvers in-use by a particular client.
And packet rate. Some corporate firewalls may detect this as abusive and block it leading to questions from the security operations team and the person testing this out.
[deleted]
You can even set up a VPN over DNS if you really wanted. https://github.com/yarrick/iodine
I'm not sure why this should be a corollary now. DNS exfiltration has always been possible, just like the article you linked to says. (I also like to point people to https://twitter.com/rsobers/status/1293539543115862016 in this context because it puts the idea so succinctly.)
Interesting, I'll read up on that.
[deleted]
Looks more like HTML via HTTP, specifically DoH.
const dohServer = "https://cloudflare-dns.com/dns-query?ct=application/dns-json&type=TXT&name=";
const baseDomain = "hod.experiments.jacobkiers.net";
Actually JSON via HTTP. Then using Javascript to create HTML from the JSON.
This technique has been a standard exfiltration & C2 (command & control) channel for malware for a long time. Typically malware will make a DNS request for a subdomain where the domain name encodes data or a request, and the response contains e.g. commands.
As someone who has an obsession with base64 encoding (as exemplified by my poorly written shell scripting here: https://miscdotgeek.com/curlytp-every-web-server-is-a-dead-d... ) I love this. It makes me wonder if some CSS and maybe even a highly compressed image or two could be added.
If you're obsessed with base64 here is a fun thing for you:
A few years ago I randomly remarked that repeated base64 prefix converge to a fixed point. I tweeted [1] the first characters of that string out of amusement, without any additional details.
Then someone saw the tweet, found it funny, and did a write-up about it [2], and then there was an interesting discussion on Reddit [3]. There may have been one on HN too, I don't remember.
Have fun :).
[1] https://twitter.com/p4bl0/status/298900842076045312
[2] https://web.archive.org/web/20160313123301/https://fmota.eu/...
[3] https://www.reddit.com/r/compsci/comments/18234a/the_base64_...
A few years ago I randomly remarked that repeated base64 prefix converge to a fixed point. I tweeted [1] the first characters of that string out of amusement, without any additional details.
Then someone saw the tweet, found it funny, and did a write-up about it [2], and then there was an interesting discussion on Reddit [3]. There may have been one on HN too, I don't remember.
Have fun :).
[1] https://twitter.com/p4bl0/status/298900842076045312
[2] https://web.archive.org/web/20160313123301/https://fmota.eu/...
[3] https://www.reddit.com/r/compsci/comments/18234a/the_base64_...
Nice idea. Probably possible, since each resource has an index record, which contains the mime type.
I'll try it sometime soon (though my CSS skills are basically nonexistent...).
I'll try it sometime soon (though my CSS skills are basically nonexistent...).
I modified the dnstxt file from djbdns about twelve years ago to output a MIME header for HTML over UDP DNS. I could store hyperlinks for tiny web pages in a zone file and serve them with tinydns. (This was before EDNS exploded the size of DNS packets.)
regards
https://minimilitiamodapk.info/
Wondering why I can't query the DNS directly.
$ dig posts-2021-08-17-serving-blog-content-over-dns-md.hod.experiments.jacobkiers.net TXT
; <<>> DiG 9.8.3-P1 <<>> posts-2021-08-17-serving-blog-content-over-dns-md.hod.experiments.jacobkiers.net TXT
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 49067
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 0
;; QUESTION SECTION:
;posts-2021-08-17-serving-blog-content-over-dns-md.hod.experiments.jacobkiers.net. IN TXT
;; AUTHORITY SECTION:
hod.experiments.jacobkiers.net. 292 IN SOA home.kie.rs. postmaster.kie.rs. 2021081903 3600 900 604800 3600That's weird, it should work. I'll investigate when I get home.
Just tried it again, now it works like magic. Don't know what was wrong.
Spooky, maybe you were MITMed.
You can look at this line.
> ;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 0
That says that the DNS server didn't answer - for whatever reason - despite the address of that server being determined. It could have been a form of HN's "hug of death" where the website temporarily goes down when many people start to view it in a short period of time.
> ;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 0
That says that the DNS server didn't answer - for whatever reason - despite the address of that server being determined. It could have been a form of HN's "hug of death" where the website temporarily goes down when many people start to view it in a short period of time.
Apparently uses DoH. I don't think dig(1) talks DoH, you'd have to use curl or kdig.
DNS is DNS. You should be able to talk to the NS with DNS over UDP, TCP, or HTTP(s). The protocol used should not matter. The information received should be always the same.
...in an ideal world
> You might not be able to see it immediately
Or at all since the content is entirely injected via JS with no fallback (and the JS uses class fields in case you thought an old browser might be able to load it).
Or at all since the content is entirely injected via JS with no fallback (and the JS uses class fields in case you thought an old browser might be able to load it).
I don't know how to do this without JS.
Also, since this was mostly a DNS-focused proof of concept, I don't particularly care about that. Not in this case, at least.
Also, since this was mostly a DNS-focused proof of concept, I don't particularly care about that. Not in this case, at least.
You don’t have to “do this without JS” but you could have a fallback to tell JS-less users about it. As is they get a blank page and no idea what’s happening.
Fair enough. I have added a fallback.
Been wondering about this, how many people are using old browsers? caniuse.com reports [0] 96% support for ES6 classes. The other 4% is divided between Internet Explorer, Opera Mini and obsolete versions of every other browser. So I'm trying to figure out what kind of people are using such ancient technology and why? What kind of unusual visitors would I be missing out on?
Anecdote: I know someone who has some antiquated games that he couldn't get working in anything other than a real install of Windows XP. The game has a forum, so he frequently uses IE8 installed on the computer to access it.
I guess there are lots of tiny edge cases that look like that.
I guess there are lots of tiny edge cases that look like that.
You're missing on visitors with disabled (or whitelisted) javascript. On HackerNews that'll be pretty common.
It's true that people with JavaScript disabled won't be able to run any JavaScript, but once they turn it on it should have relatively good feature support -- I suspect the sort of people who know how to disable JavaScript aren't using software more than 5 years out of date.
> but once they turn it on it should have relatively good feature support
How could they even know to tuen it on when all they get is an empty page?
How could they even know to tuen it on when all they get is an empty page?
They are used to it? The number of users who have JS disabled is already small. The number of those users who lack the technical expertise to enable it as needed (and understand when that is necessary — e.g. with a blank or broken page) is probably many, many times smaller.
> They are used to it?
And yet most site designers which require javascript turn out to be thoughtful enough to make a note of it rather than leave a completely empty page.
And public sites which absolutely require javascript to do anything turn out to be quite rare.
So there’s no reason to be used to it. “Some buttons don’t work”, yes I do expect i need to allow some JS. “Nothing shows up at all” is usually a network issue.
And yet most site designers which require javascript turn out to be thoughtful enough to make a note of it rather than leave a completely empty page.
And public sites which absolutely require javascript to do anything turn out to be quite rare.
So there’s no reason to be used to it. “Some buttons don’t work”, yes I do expect i need to allow some JS. “Nothing shows up at all” is usually a network issue.
Does anyone have a recommendation for a JavaScript whitelisting extension or method for macOS Safari?
[deleted]
> caniuse.com reports [0] 96% support for ES6 classes.
Class fields are later addition to classes. And completely unnecessary here, it looks. The class fields are just duplicates of the instance fields, which are always filled.
> The other 4% is divided between Internet Explorer, Opera Mini and obsolete versions of every other browser. So I'm trying to figure out what kind of people are using such ancient technology and why?
Unsupported device which can not have recent updates for one reason or an other?
Class fields are later addition to classes. And completely unnecessary here, it looks. The class fields are just duplicates of the instance fields, which are always filled.
> The other 4% is divided between Internet Explorer, Opera Mini and obsolete versions of every other browser. So I'm trying to figure out what kind of people are using such ancient technology and why?
Unsupported device which can not have recent updates for one reason or an other?
That's interesting. I'm thinking about ways how this could be combined with https://dnskv.com - which allows not only reading from dns, but also storing:
dig TXT content.uniquekey.dnskv.com
--> 'ok'
dig TXT uniquekey.dnskv.com
--> 'content'
All sites on the .tel domain used to be rendered from DNS. It seems to have changed in 2017 but Wikipedia still says[1]:
"In contrast to other top-level domains, .tel information is stored directly within the Domain Name System (DNS) [...] as opposed to the DNS simply returning details (such as IP addresses)"
[1] https://en.wikipedia.org/wiki/.tel
"In contrast to other top-level domains, .tel information is stored directly within the Domain Name System (DNS) [...] as opposed to the DNS simply returning details (such as IP addresses)"
[1] https://en.wikipedia.org/wiki/.tel
This looks like it was a fun project!
For anyone who wants to research the subject, the class of security vulnerabilities are called DNS tunneling.
For anyone who wants to research the subject, the class of security vulnerabilities are called DNS tunneling.
Yes. I recently wrote a Lua-scriptable DNS server called redns[1] (which seems to be a pretty popular name for hobbyist DNS servers, BTW) only to test some of those attacks.
[1] https://github.com/luismedel/redns
[1] https://github.com/luismedel/redns
The RR format can be made more efficient. TXT records can contain multiple character strings of up to 255 bytes, and despite their name can contain arbitrary octets. A step further from there would be to use a private use type (65280-65534) so your payload doesn't need to be cut into length byte prefixed chunks.
"100s of requests per second"
dns works fine at those rates or higher, and you'll use caching dns servers of ISPs, scales like no other, geo support etc etc. I think its great idea for public data.
dns works fine at those rates or higher, and you'll use caching dns servers of ISPs, scales like no other, geo support etc etc. I think its great idea for public data.
dig @1.1 TXT +short owo{0..201}.xn--kda.waw.pl | sed -E 's/[" ]//g' | base64 -d | mpv -
A movie served via DNS. Cool!
this would be nice for the gemini protocol or something linke twtxt as it is more text oriented, one could perhaps use base45 or other base for extra space efficiency.
for the ultimate in image compression go avif or svg.
What's the point though? I can think maybe of reduced latency, but then you usually have the DNS already cached locally. And that takes me to a second point, won't you run into problems delivering content updates, given that clients will most likely cache DNS entries?
Highly distributed global CDN for free. You can use a short TTL, though there are lots of misbehaving DNS servers around cache timeouts. "Fetch with DoH" kinda ruins the party, but browsers can't do UDP or non-HTTP TCP.
In my mind this is a neat proof-of-concept that shows how you can serve content through some absurd method (in this case serving HTML over DNS).
It may not have "a point" beyond just being an example of what's possible!
[deleted]
[deleted]
I used DoH because to the best of my knowledge it's not possible to open raw sockets from the browser. Otherwise I'd have done that.
I don't think there's any practical use. And I did not intend it to have any.