Introducing Nomulus: an open source top-level domain name registry(opensource.googleblog.com)
opensource.googleblog.com
Introducing Nomulus: an open source top-level domain name registry
https://opensource.googleblog.com/2016/10/introducing-nomulus-open-source.html
49 comments
Its great that google is open sourcing this, however, its a bit curious as anyone who is going to be operating a registry really has the resources and expertise to manage this kind of thing (both hw and sw). Unless the end game is to get big resource users (like registries) onto GCE, its a bit odd.
It is a bit odd, but I have friends in African countries with TLDs that need to be registered via paper forms / GPG-signed emails only.
This move by Google just massively simplified the barriers to entry for small countries running their own ccTLDs in a more modern way.
(I work for a registrar, dealing with ccTLD idiosyncracies all day. If more registries adopt common premium price extensions, etc, it will make my life a lot easier, too.)
This move by Google just massively simplified the barriers to entry for small countries running their own ccTLDs in a more modern way.
(I work for a registrar, dealing with ccTLD idiosyncracies all day. If more registries adopt common premium price extensions, etc, it will make my life a lot easier, too.)
I work for a registrar as well. You are right, I hadn't considered ccTLD's What I find really insidious about ccTLD registries is the crazy rules they have (and therefore the crazy business rules I must use to support them) this doesn't fix that, but as you pointed out it might help some of those small ones that are otherwise out of luck.
Well, unlike gTLDs, ICANN have no control over what ccTLD registries do. Personally, I much prefer dealing with ccTLD registries who behave like gTLD registries (.me, .co, &c.), but those are few and far between.
Smaller ccTLDs are unlikely to use this though: it makes more sense for them to contract out the technical side of things than run their own, with .cm being an example of a registry that shouldn't run their own infrastructure.
Smaller ccTLDs are unlikely to use this though: it makes more sense for them to contract out the technical side of things than run their own, with .cm being an example of a registry that shouldn't run their own infrastructure.
Incidentally I find that to be a smart business move for them as well. I mean if you behave like a gTLD you integrate much faster than say .es where you have all kinds of offset rules you must apply to their expiration or .nl who give you a one day window to renew!
You'd think that, but trying to convince registries to go down that route is head wrecking.
And then you have the likes of the .ie domain registry (IEDR) who use their own vaguely EPP-like protocol that works nothing like normal registries with a set of needlessly bizarre policies specific to them.
Ugh.
And then you have the likes of the .ie domain registry (IEDR) who use their own vaguely EPP-like protocol that works nothing like normal registries with a set of needlessly bizarre policies specific to them.
Ugh.
There's one big obvious advantage of open source registries, from the perspective of registrars: You can look at our source code and figure out exactly what we're doing.
You'll be happy to know that we support Gavin Brown's fee extension: https://tools.ietf.org/html/draft-brown-epp-fees-00
The implementation is here: https://github.com/google/nomulus/tree/master/java/google/re...
And a sample domain create EPP message with an attached fee from our test suite: https://github.com/google/nomulus/blob/master/javatests/goog...
The implementation is here: https://github.com/google/nomulus/tree/master/java/google/re...
And a sample domain create EPP message with an attached fee from our test suite: https://github.com/google/nomulus/blob/master/javatests/goog...
From my experience, that's the most common fee extension. Thanks!
I saw that donuts has a test EPP endpoint to try nomulus. I will probably connect to it soon for some integration testing.
My only issues when using this extension:
* It's super flexible. I have to guess if a fee is an EAP fee based on the description, for example. (Registries haven't standardised on anything.)
* Querying for all operations * terms is very verbose.
* The extension itself has multiple versions. We support fee-0.5 - fee-0.8.
I saw that donuts has a test EPP endpoint to try nomulus. I will probably connect to it soon for some integration testing.
My only issues when using this extension:
* It's super flexible. I have to guess if a fee is an EAP fee based on the description, for example. (Registries haven't standardised on anything.)
* Querying for all operations * terms is very verbose.
* The extension itself has multiple versions. We support fee-0.5 - fee-0.8.
Can I just say, anyone who uses the fee extension over crazy price api's or downloaded lists is my friend!
We support downloadable static premium price lists too (it's all that some registrars can handle), but yes, obviously we prefer the price extension too. Here's the entry point into the static premium pricing lists if you're curious: https://github.com/google/nomulus/blob/master/java/google/re...
Said lists are imported into the system from the CSV file format that is given to registrars by this command-line tool: https://github.com/google/nomulus/blob/master/java/google/re...
By the way, can I just say as an aside, that it's so awesome that I can finally link to and discuss our code in the open (I wrote this pricing stuff under stuff).
Said lists are imported into the system from the CSV file format that is given to registrars by this command-line tool: https://github.com/google/nomulus/blob/master/java/google/re...
By the way, can I just say as an aside, that it's so awesome that I can finally link to and discuss our code in the open (I wrote this pricing stuff under stuff).
Yup! And I wish the likes of Rightside would scrap their extension and just use Gavin's work.
> anyone who is going to be operating a registry really has the resources and expertise to manage this kind of thing
Broadly speaking, this only holds true for a limited number of large legacy gTLDs (e.g. .com, .net) and the largest ccTLDs.
Broadly speaking, this only holds true for a limited number of large legacy gTLDs (e.g. .com, .net) and the largest ccTLDs.
Not in practice, the reality is most of the new gTLD's went shopping for industry people - you have to, there are so many unique technical, regulatory, and industry specific items you just buy the expertise, trying to learn on the fly is a monumental task.
FYI, speaking of industry goings-on: http://finance.yahoo.com/news/donuts-collaborates-googles-ne...
I should clarify at this point that I am the author of original linked blog post.
I should clarify at this point that I am the author of original linked blog post.
Can we imagine new gTLD Entrepreneurs with a minimum technical knowlege to become their own back-end Registry AND Registry in round 2 of the ICANN new gTLD program?
OTOH, one of the justifications for why domains cost $7 instead of $2 is all the quadruple-redundant Oracle databases and such that Verisign uses. If a registry can be built cheaper it argues for lower prices.
I disagree. The com/net zone is 144mm records, thats about a billion dollars per year in revenue. The technical requirements to store 144mm records with metadata and have multiple redundancy, backup datacenter failover etc. does not come close to that in costs even if those costs are 10%. The cost is completely divoriced from the technical requirements, its simply what the market bears and is use to. They have momentum on their side.
Some of the new tld's are very inexpensive right now, but that is only the short term business model to get market share. Their renewal prices are often higher and will stay that way.
Some of the new tld's are very inexpensive right now, but that is only the short term business model to get market share. Their renewal prices are often higher and will stay that way.
Besides the post of rdrey, this is also a nice PR/marketing move targeting the open source community.
Registrars.
That is the goal. This runs on App Engine, Google's PAAS, if the Java lock-in is anything like the Python lock-in I have experienced this won't be very useful outside of their infrastructure. At least not without a major rewrite.
I'm not quite sure what I expected from this... I do hope it can turn into something useful that drives down some of the costs/pricing. That said, given the amount of squatting and land grabbing, I'm not sure I want it to all cost all that much less.
I'm not sure what the solution is, but $7-15 for most domains isn't killing anyone's budget. Though having better and faster moving infrastructure in most registrars would be nice.
I'm not sure what the solution is, but $7-15 for most domains isn't killing anyone's budget. Though having better and faster moving infrastructure in most registrars would be nice.
Can anyone explain what is going on with the repo
It looks like they're vendoring all of their third party/first party depedencies e.g.
https://github.com/google/nomulus/tree/master/java/com/googl...
https://github.com/google/nomulus/tree/master/third_party/ja...
Is this a common pattern for modern java projects? I've never seen it done this way....I'm hoping all those BUILD files are autogenerated...
It looks like they're vendoring all of their third party/first party depedencies e.g.
https://github.com/google/nomulus/tree/master/java/com/googl...
https://github.com/google/nomulus/tree/master/third_party/ja...
Is this a common pattern for modern java projects? I've never seen it done this way....I'm hoping all those BUILD files are autogenerated...
I'm responsible for that BUILD file. Here's why I did it.
Every team at Google stores its code in a single shared monolithic repository with petabytes of code: http://research.google.com/pubs/pub45424.html The Nomulus BUILD files you see in the GitHub repository are actually identical to the ones we use internally.
In order to make the BUILD the same both internally and externally, we needed to have some BUILD files in the open source repository, which at first glance, appear to be superfluous. The reason why @guava//jar is mapped to //java/com/google/common/collect is because that's what it's called in the internal repository. By creating this alias, we don't have to update the hundreds of other deps=[...] lines that reference it.
In the future, our tooling will improve and we will no longer need those files. But for the time being, they're a good workaround.
Every team at Google stores its code in a single shared monolithic repository with petabytes of code: http://research.google.com/pubs/pub45424.html The Nomulus BUILD files you see in the GitHub repository are actually identical to the ones we use internally.
In order to make the BUILD the same both internally and externally, we needed to have some BUILD files in the open source repository, which at first glance, appear to be superfluous. The reason why @guava//jar is mapped to //java/com/google/common/collect is because that's what it's called in the internal repository. By creating this alias, we don't have to update the hundreds of other deps=[...] lines that reference it.
In the future, our tooling will improve and we will no longer need those files. But for the time being, they're a good workaround.
Great explanation and interesting insight, thanks!
What stops you from seeing code you aren't allowed (ie. the search algorithm) if its a monolithic repo?
Also, with a repository that large, how do you have it setup locally that is usable?
Also, with a repository that large, how do you have it setup locally that is usable?
OP posted a link to a paper that answers your questions: http://research.google.com/pubs/pub45424.html
[deleted]
I believe they used perforce, and switched to an internal tool called Piper, which has ACLs.
I presume that all employees install custom software that work in a way similar to NFS — a distributed file system that avoids having to download everything locally.
I presume that all employees install custom software that work in a way similar to NFS — a distributed file system that avoids having to download everything locally.
It's essentially an artifact of the way that third-party dependencies are handled in the Google monorepo. If we had started this project from scratch on GitHub, rather than exporting it at a later date, we most likely would not have gone down this path.
> Internet Corporation for Assigned Names and Numbers (ICANN) announced the biggest ever expansion of Internet namespace, aimed at improving choice and spurring innovation for Internet users
Not sure what exactly it was really for, but my guess is not that.
On the other hand, maybe innovation is spurred by letting google own ".dad".
Not sure what exactly it was really for, but my guess is not that.
On the other hand, maybe innovation is spurred by letting google own ".dad".
To go into more detail than I was able to fit into the blog post (and keep in mind that I'm writing from my perspective, not ICANN's):
It definitely increases choice. Before the gTLD expansion, you had your choice of the existing legacy gTLDs, your country's ccTLD, and some ccTLDs from other countries that don't have strict location requirements. That's easily under 100 choices. Contrast with now, post-TLD expansion, in which you have over a thousand choices.
There are more opportunities for technical innovation, though admittedly most of the new TLDs so far have been generic open TLDs. But you can imagine all sorts of new things that are possible with closed or restricted TLDs. Some random ideas include: An entire TLD whose registrations are backed by Namecoin or some other blockchain, a secure TLD with really stringent identity verification requirements to cut down on scamming, and a TLD on which domain names never expire and don't have renewal costs (you'd pay more up front for one, understandably). I'm just throwing random ideas out here; this isn't anything we're necessarily working on. But you can imagine lots more possibilities.
It definitely increases choice. Before the gTLD expansion, you had your choice of the existing legacy gTLDs, your country's ccTLD, and some ccTLDs from other countries that don't have strict location requirements. That's easily under 100 choices. Contrast with now, post-TLD expansion, in which you have over a thousand choices.
There are more opportunities for technical innovation, though admittedly most of the new TLDs so far have been generic open TLDs. But you can imagine all sorts of new things that are possible with closed or restricted TLDs. Some random ideas include: An entire TLD whose registrations are backed by Namecoin or some other blockchain, a secure TLD with really stringent identity verification requirements to cut down on scamming, and a TLD on which domain names never expire and don't have renewal costs (you'd pay more up front for one, understandably). I'm just throwing random ideas out here; this isn't anything we're necessarily working on. But you can imagine lots more possibilities.
Oh there is cool stuff. Even just being able to do to search.google and docs.google or <channel>.youtube is awesome.
I just don't see it as being anything other than a land grab for most of the TLDs google and other companies registered. If you have a trademark on something, or you have a good technical proposal (e.g. .ncd for namecoin backed domain) that you are willing to support then that's fine. .dad, .shop, .buy, etc...
It's just selling out.
I would love to hear technical reasons for their list, https://googleblog.blogspot.com/2012/05/expanding-internet-d...
Anyway, the project is still cool.
I just don't see it as being anything other than a land grab for most of the TLDs google and other companies registered. If you have a trademark on something, or you have a good technical proposal (e.g. .ncd for namecoin backed domain) that you are willing to support then that's fine. .dad, .shop, .buy, etc...
It's just selling out.
I would love to hear technical reasons for their list, https://googleblog.blogspot.com/2012/05/expanding-internet-d...
Anyway, the project is still cool.
How about handling TLDs like second level domains but make them free, instead of requiring money for them get a few big companies and enterpreneurs to fund this "root level free as in beer registrar"
We all know running nameservers doesnt require magical unicorn blood and shouldn't be as obscenely priced as is now.
We all know running nameservers doesnt require magical unicorn blood and shouldn't be as obscenely priced as is now.
What's the process of getting a TLD?
More info here: https://newgtlds.icann.org/en/about/program
But the tl;dr is that you need to wait for the next round of expansion if you didn't already participate in the current round.
But the tl;dr is that you need to wait for the next round of expansion if you didn't already participate in the current round.
This is strictly for operating a registry (ie deploying the infrastructure for your own TLD), getting a TLD granted is through ICANN.
And at the rate the first round is currently going, the next round is probably at least 5+ years down the road. So don't hold your breath.
Is there an easy way to use a non-ICANN sanctioned tld? I kinda like the idea of my own whole tld, even if it takes some work to access it. Is there a browser extension or something that lets me add custom registries?
In principle there's no reason you couldn't. We support an extendable DNS interface, so if you were to integrate that with the DNS system running a local or corporate network, and your DNS provider in your OS set up to use that, it should work. I don't think you'd even need a browser extension. It's one potential use case we've thought of before, but haven't done any specific work towards implementing, so there might be some gotchas or adjustments required in areas where we make certain assumptions that all entities managed by the system resolve at the top level of the Internet.
If anyone does work on something like this, please let me know. I'd be very interested to see the parts in action.
If anyone does work on something like this, please let me know. I'd be very interested to see the parts in action.
what are the other open source options?
FRED by CZ.NIC: https://fred.nic.cz/ . Open sourced as well and it's been around for a long time.
The one used by the New Zealand registry:
https://sourceforge.net/projects/dnrs/
https://sourceforge.net/projects/dnrs/
I am so sorry. I have to write this here: NomNomNomulus
[informative comment about corporate PR disguised as open source news removed as "off-topic"]
It's about an open source TLD registry, and it says it's about an open source TLD registry. That's not clickbait: It's perfectly accurate. The fact that it's Google is something you ought to be able to tell, because it's indicated by the domain, something HN always shows.
And just because the collective disagrees with you doesn't mean the site itself is an echochamber. Case in point: You haven't been censored in any way, even though you expressed a different opinion. The collective has merely decided that your comment isn't worth looking at. People can still read it.
And just because the collective disagrees with you doesn't mean the site itself is an echochamber. Case in point: You haven't been censored in any way, even though you expressed a different opinion. The collective has merely decided that your comment isn't worth looking at. People can still read it.
In my opinion your comment, now removed, wasn't informative. You complained the title was clickbait and that you initially didn't want to click it (ok, then don't). Then followed about Alphabet's ever increasing privacy invasion.
The source "googleblog.com" next to the title on HN is clear enough. It's a company that open sources a project on github (https://github.com/google/nomulus). I don't think the announcement is disguised, spinned or misleading.
My downvote as off-topic came because it didn't address the announcement itself. To be honest I had the impression you read the title only, not the announcement itself.
That said I think the title techcrunch chose is clearer "Google open sources the code that powers its domain registry"
The source "googleblog.com" next to the title on HN is clear enough. It's a company that open sources a project on github (https://github.com/google/nomulus). I don't think the announcement is disguised, spinned or misleading.
My downvote as off-topic came because it didn't address the announcement itself. To be honest I had the impression you read the title only, not the announcement itself.
That said I think the title techcrunch chose is clearer "Google open sources the code that powers its domain registry"