That's what the tool is doing - querying a bunch of public resolvers around the world to see the state of what they resolve to. Since end users usually use DNS servers close to their location, this gives an idea, around the world, of who sees what.
Agreed, this is a cache that expires and refreshes from the source DNS server. It just looks like a virus that propagates when the cache expires.
We went from the get go to that infrastructure for multiple reasons in the first place:
* Having a durable buffer before ensures if you have big spikes that gets eaten by the buffer, not OLAP which when it is powering your online dashboard you want to keep responsive. Clickhouse cloud now has compute/compute that addresses that but open source users' don't.
* When we shipped this for the first time, clickhouse did not have the async buffering in place, so not doing some kind of buffered inserts was forwned upon.
* As oatsandsugar mentioned, since them we also shipped direct insert where you don't need a kafka buffer if you don't want it
* From an architecture standpoint, with that architecture you can have multiple consumers
* Finally, having kafka enables having streaming function written in your favorite language vs using SQL. Definitely will be less performance to task ratio, but depending on the task might be faster to setup or even you can do things you couldn't directly in the database.
github.com/callicles