A New Future for Icanhazip(major.io)
major.io
A New Future for Icanhazip
https://major.io/2021/06/06/a-new-future-for-icanhazip/
259 comments
This story is kind of sad. I wonder why the operator didn't blacklist certain netblocks/ASNs who were abusing the service.
or even better: respond with incorrect data
they'll soon learn
they'll soon learn
Why punish a group for one bad actor?
> There were many times where I saw a big traffic jump and I realized the traffic was coming from the same ASN, and likely from the same company. I tried reaching out to these companies when I saw it but they rarely ever replied. Some even became extremely hostile to my emails.
A hostile reply from a netblock operator seems like a perfectly valid reason to block their traffic.
A hostile reply from a netblock operator seems like a perfectly valid reason to block their traffic.
The problem is that you don't know what the source of the traffic is. It could be an incompetent network operator/sysadmin, but it could just as well be something like an IP camera that people bought in good faith. If you block the CGNAT system of an operator that has a hundred million subscribers because it all seems to come from a single IP range you know nothing about, you could be hurting innocent users with the block.
That being said, a service like this doesn't come with any guarantees and if it'd disappear from the net tomorrow, I wouldn't blame the author. Blocking is a perfectly valid solution to this problem, but assuming malice isn't always the right answer.
Were I in this situation, I'd rate limit networks per /24 (maybe even /16?) as much as I could, and work together with antivirus companies to help identify infections of malware known to use the service to discourage criminals from abusing the system. I wouldn't even bother hosting the site on IPv6 since those addresses are supposed to be public anyway. The author clearly has more patience than I do.
That being said, a service like this doesn't come with any guarantees and if it'd disappear from the net tomorrow, I wouldn't blame the author. Blocking is a perfectly valid solution to this problem, but assuming malice isn't always the right answer.
Were I in this situation, I'd rate limit networks per /24 (maybe even /16?) as much as I could, and work together with antivirus companies to help identify infections of malware known to use the service to discourage criminals from abusing the system. I wouldn't even bother hosting the site on IPv6 since those addresses are supposed to be public anyway. The author clearly has more patience than I do.
In some sense, it might not matter. If an ASN/company admin responds to emails in a hostile fashion, does it matter if they bought their devices in good faith? They're still assholes.
Hostility can often come from a place of ignorance or misunderstanding. I can't say much for the former, but the latter can easily go wrong with the cultural and linguistic barrier between operators.
The guy operating the NOC may be a dick, but is taking down the IoT networks for all of their customers unknowingly relying on your services really the right way?
Personally, I'd say yes, it'd help. However, there's an argument to be made that the hostile ASN operator doesn't represent the people behind the network in the slightest. I can understand that someone may give such an asshat the benefit of doubt and drop it despite their abuse.
The guy operating the NOC may be a dick, but is taking down the IoT networks for all of their customers unknowingly relying on your services really the right way?
Personally, I'd say yes, it'd help. However, there's an argument to be made that the hostile ASN operator doesn't represent the people behind the network in the slightest. I can understand that someone may give such an asshat the benefit of doubt and drop it despite their abuse.
This isn’t “one person bought a bad camera”, it was certain ASNs accounting for a huge portion of the traffic. If the operators are unresponsive to the abuse request (making them incompetent network operators), then you absolutely block them. At that point the fallout is the fault of the network operators for operating an abuse friendly network.
This is how cloudflare handles it for normal web services. If you’re coming from trash IPs there is no chance a curl request is going to make it through to a backend without an onerous captcha.
This is how cloudflare handles it for normal web services. If you’re coming from trash IPs there is no chance a curl request is going to make it through to a backend without an onerous captcha.
I wouldn't expect one person with one camera to cause such a load, but popular, cheap internet cameras pull this crap all the time. I remember reading a story here about one company that hardcoded a particular IP address for their NTP bootstrapping in their firmware, with thousands of devices all across the world and no way to easily update them. Such a thing can easily happen with consumer routers and other networking equipment, generating a publicly accessible link for their user's convenience.
If I saw the Time Warner ASN send too many requests, my first thought wouldn't be to just block a huge ISP. Who knows what mihjt be causing these issues and what you could be breaking by interrupting service.
The Time Warner NOC wouldn't be able to completely fix the problem if the source of the issue is the firmware of a certain shitty IoT device. If someone emailed their NOC about some weird IP cams installed by their customers causing load on their servers, they could feel like that's a problem between icanhazip and the camera manufacturer, not something they can fix.
The author is quite tolerant of the obviously malicious behaviour others are attacking his servers with. I'd have taken more aggressive measures instead of scaling up capacities myself. Because the problem is volume and not necessarily anything complex, I'd wager that even a simple block could be quite expensive because that traffic and the associated retries will be going somewhere. Directing the traffic towards the last router in their ASN through DNS would be something I'd consider, making it the problem of the network operators.
If I saw the Time Warner ASN send too many requests, my first thought wouldn't be to just block a huge ISP. Who knows what mihjt be causing these issues and what you could be breaking by interrupting service.
The Time Warner NOC wouldn't be able to completely fix the problem if the source of the issue is the firmware of a certain shitty IoT device. If someone emailed their NOC about some weird IP cams installed by their customers causing load on their servers, they could feel like that's a problem between icanhazip and the camera manufacturer, not something they can fix.
The author is quite tolerant of the obviously malicious behaviour others are attacking his servers with. I'd have taken more aggressive measures instead of scaling up capacities myself. Because the problem is volume and not necessarily anything complex, I'd wager that even a simple block could be quite expensive because that traffic and the associated retries will be going somewhere. Directing the traffic towards the last router in their ASN through DNS would be something I'd consider, making it the problem of the network operators.
Looking at the icanhazip.com site, I wonder how much any kind of rate-limiting per address/block would even help.
At the HTTP level it's probably cheaper to just return the HTTP 200 response. I suppose if you're doing TLS handshakes then a packet-level rate-limit would help significantly, but at the same time I'd be wary of triggering any kind of retry-behavior.
Worst-case scenario for a service like this would be having an error response/timeout trigger some kind of unlimited retry flood.
At the HTTP level it's probably cheaper to just return the HTTP 200 response. I suppose if you're doing TLS handshakes then a packet-level rate-limit would help significantly, but at the same time I'd be wary of triggering any kind of retry-behavior.
Worst-case scenario for a service like this would be having an error response/timeout trigger some kind of unlimited retry flood.
The block route I'd go with is blackholing the entire range into nothing through BGP or similar so the servers wouldn't have to deal with the traffic, similar to how anti DDOS tools often work. Might even redirect the DNS for that subnet to the IP of the people running the network, let them deal with the abuse. That'd be a very offensive approach, though.
I probably wouldn't bother with TLS either, just a plain HTTP 0.1 response with minimum information should be enough.
I probably wouldn't bother with TLS either, just a plain HTTP 0.1 response with minimum information should be enough.
This raises something I've wondered for a while: is there a service or database that can give an indication of how many humans are behind a particular IP address? e.g. with CGNAT, there might be many thousands of people sharing a single IP. For some residential services, it might be 1-2 people.
It feels like this sort of data (even if only providing order of magnitude estimates) would help greatly with deciding on appropriate rate limits for small operators who don't have the time to research all the traffic they're receiving.
It feels like this sort of data (even if only providing order of magnitude estimates) would help greatly with deciding on appropriate rate limits for small operators who don't have the time to research all the traffic they're receiving.
This is not an unimportant or victimless problem, however said problem is the network operator's entire job. Making them deal with this is not uncalled-for.
As a network operator, you would be surprised how many AS operators are hostile or simply don’t respond. It’s unfortunately very common, even Tier-1’s are hostile.
Out of the last month, I sent out 191 abuse reports, of which 10 got replied to, 2 were resolved 6 were “no f** off” style, and 2 were told “can’t fix / won’t fix / don’t know how to fix”.
I’m not just referring to Chinese ASNs either, some US Telco’s, German, Australia even.
Out of the last month, I sent out 191 abuse reports, of which 10 got replied to, 2 were resolved 6 were “no f** off” style, and 2 were told “can’t fix / won’t fix / don’t know how to fix”.
I’m not just referring to Chinese ASNs either, some US Telco’s, German, Australia even.
We're entering an era where the content providers and the major clouds are >50% of demand on new subsea capacity. Feels like this blurs 'Tier 1' and the role of backbones because major eyeball networks now interconnect directly with e.g. Facebook or Google.
Anyway, on abuse@ response rates, my probably unpopular but realistic take based on looking at tens of thousands of such complaints over the years and having worked for ASNs which have received millions, I'd hazard everyone has an SNR and ROI problem with handling these. There's just too many of them and most aren't actionable.
Some examples, "I saw a failed SSH login attempt from 1.2.3.4 and OMG that's a huge issue, you have been compromised, and you must solve this immediately!". OK, well, the subscriber might have: a) Typoed your IP address, b) Been running nmap/zmap over a wide range of IPs for research purposes; c) You're on an IP with a provider who recycled it to you, subscriber has outdated DNS records.
What do you expect a 'Tier 1' to do with your report?
Many ASNs are now just looking at the pattern of reports per IP address or subscriber, are automating scanning for e.g. open mail relays when whatever processes abuse@ determines the person is complaining about spam, or automating looking for anomalies in flows for DDoS complaints, a human may not even see the ticket unless the automation was able to confirm a problem may exist, and the human will probably only engage the subscriber and won't respond to the 1-1000 things received to abuse@ related to the issue.
In Major's case with icanhazip.com it looks like pretty bad behavior from the Chinese ASNs mentioned, but could just be IOT configured to fetch its IP every minute instead of every 60 minutes of 24 hours because someone misunderstood cron. Unfortunate that nobody responded but 30B a day is ~350kRPS (which isn't a lot, in the grand scheme of the internet). I'm sure 30B requests per day is nothing at Cloudflare's scale and they have options to cure these ASNs behavior should they choose, including stuff like IP-based or ASN-based ratelimiting, or even IP/ASN restrictions.
I'm sure Cloudflare will learn some interesting things about both the accidental contributors (e.g. cron) and intentional contributors (e.g. botnets) from analyzing the sources generating the requests, and I'm ultimately glad it is them picking this up, their other initiatives like 1.1.1.1 have had been positive for the internet (IMHO).
Anyway, on abuse@ response rates, my probably unpopular but realistic take based on looking at tens of thousands of such complaints over the years and having worked for ASNs which have received millions, I'd hazard everyone has an SNR and ROI problem with handling these. There's just too many of them and most aren't actionable.
Some examples, "I saw a failed SSH login attempt from 1.2.3.4 and OMG that's a huge issue, you have been compromised, and you must solve this immediately!". OK, well, the subscriber might have: a) Typoed your IP address, b) Been running nmap/zmap over a wide range of IPs for research purposes; c) You're on an IP with a provider who recycled it to you, subscriber has outdated DNS records.
What do you expect a 'Tier 1' to do with your report?
Many ASNs are now just looking at the pattern of reports per IP address or subscriber, are automating scanning for e.g. open mail relays when whatever processes abuse@ determines the person is complaining about spam, or automating looking for anomalies in flows for DDoS complaints, a human may not even see the ticket unless the automation was able to confirm a problem may exist, and the human will probably only engage the subscriber and won't respond to the 1-1000 things received to abuse@ related to the issue.
In Major's case with icanhazip.com it looks like pretty bad behavior from the Chinese ASNs mentioned, but could just be IOT configured to fetch its IP every minute instead of every 60 minutes of 24 hours because someone misunderstood cron. Unfortunate that nobody responded but 30B a day is ~350kRPS (which isn't a lot, in the grand scheme of the internet). I'm sure 30B requests per day is nothing at Cloudflare's scale and they have options to cure these ASNs behavior should they choose, including stuff like IP-based or ASN-based ratelimiting, or even IP/ASN restrictions.
I'm sure Cloudflare will learn some interesting things about both the accidental contributors (e.g. cron) and intentional contributors (e.g. botnets) from analyzing the sources generating the requests, and I'm ultimately glad it is them picking this up, their other initiatives like 1.1.1.1 have had been positive for the internet (IMHO).
So shutting down the entire service would be preferable to blocking one group?
Denying a request is not punishment, it's simply a lack of consent. You're allowed to return 403s for any reason you like.
I'm glad icanhazip will live on! We also see a lot of malware and bot traffic to https://ipinfo.io, but nowhere near these levels!
I've been using ipinfo.io for several years -- checking a dynamic ip address every 10 minutes. My thanks for supplying this service! Is there a reason to change over to icanhazip ?
Awesome!
No of course no reason to switch away from us! We provide many more details than icanhazip and others (geo, asn, anycast etc). We handle 40 billion requests a month,and plan to be around forever!
I'm just glad that icanhazip is staying up - it sucks when useful services go offline, and negatively impact all their users.
No of course no reason to switch away from us! We provide many more details than icanhazip and others (geo, asn, anycast etc). We handle 40 billion requests a month,and plan to be around forever!
I'm just glad that icanhazip is staying up - it sucks when useful services go offline, and negatively impact all their users.
Chinese originated spam and abuse is so outrageously widespread, I don’t understand why there isn’t a conversation going on about cutting them off from the wider internet. They blocked most of it anyway.
It stands to reason that an especially large volume of abuse will originate from the most populous country in the world. I don't think that's a reason to cut them off from the global Internet. If it's true that their government is already oppressing their own people (I don't know what's truth and what's propaganda), then the rest of us shouldn't make it worse for those people by cutting off whatever outside connections they manage to have.
Also, I'm generally bothered by comments like this one that stir up the general human tendency toward xenophobia. We should be fighting that tendency within ourselves, not fighting the out group. Whichever group of people we want to demonize, we should remember that they're people just like us. We shouldn't punish the majority of them for what a minority are doing to us.
Also, I'm generally bothered by comments like this one that stir up the general human tendency toward xenophobia. We should be fighting that tendency within ourselves, not fighting the out group. Whichever group of people we want to demonize, we should remember that they're people just like us. We shouldn't punish the majority of them for what a minority are doing to us.
They are not the most connected country, but a long shot. Why isn’t India no. 2 by your logic? Stop apologizing for unacceptable behavior from a country that openly purports to become “the superpower of the world”.
I'm not apologizing for anyone's bad behavior; I just don't want us to escalate an already tense situation. "The only winning move is not to play", right?
No, you can’t let somebody poke you with a stick and go “the only winning move is not to play” and pretend like it’s raining.
But that's fair they become it. After all they're 20% of humanity...
Yes, they are the most connected country in number of users (https://en.wikipedia.org/wiki/List_of_countries_by_number_of...) and that's what matters: the more users there are, the more malicious activity there will be
I don’t think number of people with internet access is a good measure of connectedness at all. Curiously, you dodged my question on why India isn’t the same when your list also shows they should be.
I mean, if their government doesn’t stop, and often even encourages the behavior, what are we supposed to do? Just roll over and show them the other cheek?
I agree you don’t want to cut them off, but on the other hand, I don’t want 90% of all global malicious traffic to originate from a specific country.
I agree you don’t want to cut them off, but on the other hand, I don’t want 90% of all global malicious traffic to originate from a specific country.
> I don’t want 90% of all global malicious traffic to originate from a specific country.
Is that actually true? I guess I'm inclined to believe that claims like that are more likely to be propaganda from western governments and/or western-owned companies.
If it is true, I wonder why their government isn't stopping it. They must realize that it's giving them a bad reputation in the wider world.
Is that actually true? I guess I'm inclined to believe that claims like that are more likely to be propaganda from western governments and/or western-owned companies.
If it is true, I wonder why their government isn't stopping it. They must realize that it's giving them a bad reputation in the wider world.
No it's not true, the GP just lied out of spite. Also, the OP blog post wrote about Chinese AS' out of spite too /s
To put it rather mildly, I don't really think they care.
It was true (well, in the same ballpark, don’t remember exact numbers) for the website I was in a position to see it for. It may be different for others, but like 98% of the malicious traffic comes from 3 or 4 countries.
It is sometimes explicitly used by the government to bring down sites: see the "Great Cannon of China"[1]
[1] https://en.m.wikipedia.org/wiki/Great_Cannon
[1] https://en.m.wikipedia.org/wiki/Great_Cannon
sorry but this opinion is ridiculous on its face. Blaming china because of population is a complete joke. China (by a large margin), Russia, and poorly configured proxies are 95% of all malicious traffic targeting US-based businesses. It's not even a question.
If you run a small-medium sized business in the US blocking all of countries you can't do business with anyway will save a ton of trouble.
If you run a small-medium sized business in the US blocking all of countries you can't do business with anyway will save a ton of trouble.
> China (by a large margin), Russia, and poorly configured proxies are 95% of all malicious traffic targeting US-based businesses.
My reasoning for these kinds of stats is usually: Of course it makes sense to attack targets in jurisdictions which can't catch you or equally hide in a country which won't extradite you. (But I never looked into it to any depth, so it's baseless reasoning.)
My reasoning for these kinds of stats is usually: Of course it makes sense to attack targets in jurisdictions which can't catch you or equally hide in a country which won't extradite you. (But I never looked into it to any depth, so it's baseless reasoning.)
yup. toxik's comment disgusts me. So much for being against the government and the bad apples and not against the common citizen. Let's just cut off all the devs doing their jobs every day from accessing github, or let's just cut off everyone who was curious enough to bypass the GFW and look around on the outer internet, or let's just cut off someone simply trying to contact an international friend. The truth is: people like toxik could not care less about any person who happens to live inside the borders of "public enemy no. 1".
Sigh.
Sigh.
Your comment contains no information and only presents an extended ad hominem. Go read the site rules please. This isn’t the place for moral grandstanding.
Most Chinese internet users would not miss Western internet for a second, a fact you would be aware of if you actually had any insight into Chinese culture.
This attitude that you cannot give consequences to abuse because THINK OF THE POOR CHINESE is so utterly laughable.
Most Chinese internet users would not miss Western internet for a second, a fact you would be aware of if you actually had any insight into Chinese culture.
This attitude that you cannot give consequences to abuse because THINK OF THE POOR CHINESE is so utterly laughable.
> if you actually had any insight into Chinese culture
Thanks for the blind assumption. I'm Chinese myself.
Thanks for the blind assumption. I'm Chinese myself.
> Also, I'm generally bothered by comments like this one that stir up the general human tendency toward xenophobia.
Most countries cooperate internationally in getting bad actors from hackers over pirates to pedos booted off the Internet and into jail.
The exceptions are China and Russia who won't do anything against any bad actor and India which is a big base for phone scams (as is Turkey for the European Union, but even Erdogan's regime is cooperating with EU police in taking down scammers).
I agree, the line between demanding at least some sort of common decency standards and xenophobia is thin in these days, but we have to get everyone on board to protect everyone else from rampant abuse.
Most countries cooperate internationally in getting bad actors from hackers over pirates to pedos booted off the Internet and into jail.
The exceptions are China and Russia who won't do anything against any bad actor and India which is a big base for phone scams (as is Turkey for the European Union, but even Erdogan's regime is cooperating with EU police in taking down scammers).
I agree, the line between demanding at least some sort of common decency standards and xenophobia is thin in these days, but we have to get everyone on board to protect everyone else from rampant abuse.
The other trending article on HN today is about Chinese fishing boats abusing the world's oceans. Its ok, we'll learn to get used to it.
Wouldn't want to be rude.
Any criticism of China can be construed as being pro-US and we can't have that. Criticism of Chinese traditional medicine, which is a forcing function for the endangering and extinction of species across the globe, is also unacceptable since criticism of any aspect of a culture is just racism (unless it's the one culture that's okay to criticize).
As long as the truth doesn't match what the preferred narrative is we'll continue to suffer the consequences, which is true of so many things beyond just attitudes towards China.
As long as the truth doesn't match what the preferred narrative is we'll continue to suffer the consequences, which is true of so many things beyond just attitudes towards China.
> the one culture that's okay to criticize
That would be one's own culture, whichever one that is.
That would be one's own culture, whichever one that is.
Many of us in China are absolutely saying Chinese Traditional Medicine is garbage and there s no "western medicine", just medicine.
It's not racist in China to say the truth, why is it where you are ? You probably live in an oppressive political regime with a biparty dictating what you can think ? :P
It's not racist in China to say the truth, why is it where you are ? You probably live in an oppressive political regime with a biparty dictating what you can think ? :P
> It's not racist in China to say the truth
mmm, I have had a number of Chinese people explain to me the “truth” about people of various races over the years
mmm, I have had a number of Chinese people explain to me the “truth” about people of various races over the years
> Many of us in China are absolutely saying Chinese Traditional Medicine is garbage and there s no "western medicine", just medicine.
I sure hope that "just medicine" extends to Po Chai pills. So welcome for treating diarrhea symptoms (despite my initial skepticism) when loperamide wasn't available while I was traveling in China!
I sure hope that "just medicine" extends to Po Chai pills. So welcome for treating diarrhea symptoms (despite my initial skepticism) when loperamide wasn't available while I was traveling in China!
It's not even a new trend either. Back in 2003 when I worked at eBay and PayPal doing security, the bulk of the attempts came from China and Romania (Romania at the time had one ISP for the whole country that was fast but didn't care about abuse at all).
15+ or so years ago I worked in the NOC of the 3rd or so largest ISP in the US and a random network engineer did this one evening. We got a big influx of customers complaining about email not working to their family, etc, until I finally figured it out.
That network guy (classic long hair "security" guy) was a lazy asshole for doing it then and the internet needs to have the technology to deal with bad actors beyond AS/geo-level blocking now.
That network guy (classic long hair "security" guy) was a lazy asshole for doing it then and the internet needs to have the technology to deal with bad actors beyond AS/geo-level blocking now.
I would rather have a global network with marginally more spam than a regional network with marginally less.
This false dichotomy is impressive. A single country accounting for for 50+% sets up the choice to be, “a global network with a lot less spam and a regional island with a lot of spam” vs “a global network with a ton of spam barely connected to a regional network that much of the spam originates from”.
That "single country" is of 1.4B people.
Why does that matter?
I don't think every country in the world minus one or two would be "regional".
I believe in reciprocity. China has blocked a lot of the western traffic. So, the west should block China. If they open up, we should welcome them with open arms. Similar spirit as some open source licenses - reciprocity creates fairness and increases collaboration, prevents hawks in a population of doves and improves stability.
We are already doing this with trade. The amount of leeway and free lunch China has gotten from the west is insane. I don’t blame China, I blame the west and the rest of the world for not preventing it. Asymmetrical policies are often exploited by capitalism and governments have been caught off guard.
I’m not an Anti-China lunatic. It’s just common sense.
We are already doing this with trade. The amount of leeway and free lunch China has gotten from the west is insane. I don’t blame China, I blame the west and the rest of the world for not preventing it. Asymmetrical policies are often exploited by capitalism and governments have been caught off guard.
I’m not an Anti-China lunatic. It’s just common sense.
The Chinese government operates their Internet blocks (the “Great Firewall”). But overwhelmingly, it is the Chinese people who are trying to access information on the public Internet.
Blocking the entire country will do little to hurt the government (who can employ state resources to get whatever information they want) and do quite a bit more to harm the Chinese people by reducing whatever level of information independence they still have.
If there is going to be significant change in China, it will have to come from the Chinese people. Cutting them off from the Internet vindictively does not advance that goal.
There are specific people in China doing specific bad things with specific computing resources. It would be far better for the U.S. government to dedicate more resources to finding and partnering with orgs and projects (like icanhazip or Cloudflare) to find the info they need to apply targeted mitigations.
“China does it, so we should do it too” only makes sense as a strategy if our goal is to become exactly like China is today. I don’t think that should be our goal.
Blocking the entire country will do little to hurt the government (who can employ state resources to get whatever information they want) and do quite a bit more to harm the Chinese people by reducing whatever level of information independence they still have.
If there is going to be significant change in China, it will have to come from the Chinese people. Cutting them off from the Internet vindictively does not advance that goal.
There are specific people in China doing specific bad things with specific computing resources. It would be far better for the U.S. government to dedicate more resources to finding and partnering with orgs and projects (like icanhazip or Cloudflare) to find the info they need to apply targeted mitigations.
“China does it, so we should do it too” only makes sense as a strategy if our goal is to become exactly like China is today. I don’t think that should be our goal.
> “China does it, so we should do it too” only makes sense as a strategy if our goal is to become exactly like China is today. I don’t think that should be our goal.
I very strongly disagree. An eye for an eye is exactly what needs to be done and should have been done from the beginning. Unfortunately, it is too late. 1989 massacre should have been condemned more solidly and trade restrictions should have been placed in the 90's. The bet that western alliances made is that China would open up in the 2000s leading into 2010s. That has gone horribly wrong.
The west is finally waking up: https://en.wikipedia.org/wiki/Inter-Parliamentary_Alliance_o...
I very strongly disagree. An eye for an eye is exactly what needs to be done and should have been done from the beginning. Unfortunately, it is too late. 1989 massacre should have been condemned more solidly and trade restrictions should have been placed in the 90's. The bet that western alliances made is that China would open up in the 2000s leading into 2010s. That has gone horribly wrong.
The west is finally waking up: https://en.wikipedia.org/wiki/Inter-Parliamentary_Alliance_o...
I really don't think blocks and embargoes are going to help anyone. They just suck for all the affected people, but I don't think they are very effective at convincing foreign governments to open up.
We're talking about blocking IPs originating from China. How would that hurt the affected people outside of China?
In regards to trade war, HN has discussed this ad-nauseum, I think we should restrict the discussion to internet traffic even though I brought it up as an analogy about asymmetric response from the west in general: https://hn.algolia.com/?q=trade+war
In regards to trade war, HN has discussed this ad-nauseum, I think we should restrict the discussion to internet traffic even though I brought it up as an analogy about asymmetric response from the west in general: https://hn.algolia.com/?q=trade+war
> We're talking about blocking IPs originating from China. How would that hurt the affected people outside of China?
Most of the affected people would obviously be inside China.
Most of the affected people would obviously be inside China.
> I believe in reciprocity. China has blocked a lot of the western traffic. So, the west should block China. If they open up, we should welcome them with open arms.
This sounds like a couple of people I've met, who have a philosophy of "treating people the way they treat me". And what if the other person/side also "believes in reciprocity", what happens then? This seems to rely on other people being nice first, and then always treating them how they treated you, imitating their behaviour, like Tit-for-tat[0]—except Tit-for-tat begins by being nice. It's not easy to put my finger on what seems fishy about that strategy, but it doesn't at all seem the easy solution to being fair and just (or whatever word you most prefer here) its proponents seem to think it is.
[0] https://en.wikipedia.org/wiki/Tit_for_tat (See particularly "Problems" and the next section)
This sounds like a couple of people I've met, who have a philosophy of "treating people the way they treat me". And what if the other person/side also "believes in reciprocity", what happens then? This seems to rely on other people being nice first, and then always treating them how they treated you, imitating their behaviour, like Tit-for-tat[0]—except Tit-for-tat begins by being nice. It's not easy to put my finger on what seems fishy about that strategy, but it doesn't at all seem the easy solution to being fair and just (or whatever word you most prefer here) its proponents seem to think it is.
[0] https://en.wikipedia.org/wiki/Tit_for_tat (See particularly "Problems" and the next section)
One can have a tit-for-tat policies on a geopolitical scale but also excercise forgiveness in most situations in life. I think Tit-for-tat is a terrible strategy on the whole and agree with your philosophical stance - I hope you didn't mean it on the personal level.
Sorry—you hope I didn't mean what on the personal level?
Apologies, I was hoping you didn't mean to judge my personal character from a single data point on China's policy. I think I am pretty forgiving and excercise tolerance because I know when these things are difficult is exactly when it matters the most. Tolerance is the price we pay for freedom and liberty.
Hehe no, sorry to give you that impression. Thanks for explaining!
I’m very glad that my government is probably not allowed to prevent me from accessing content from China. That’s why I live here and not in China.
The lives of people in China trying to use Internet services outside are already miserable; let's not make it worse and alienate others. We should treat them just like the rest: if extensive malicious traffic arrives, we drop it, but we don't ban the entire country.
The internet is built on institutional trust. You can’t have a properly functioning network when a sizable part is just not giving a shit about its users abusing your users.
It very much echoes the problems of intellectual property theft in China.
It very much echoes the problems of intellectual property theft in China.
This supposed 'intellectual property theft' is mostly just reverse engineering of technology.
It's not really a problem anyway. If some capitalists in the US and Europe don't get to skim off a slice of profit from another country's manufacturing output, then so what?
It's not really a problem anyway. If some capitalists in the US and Europe don't get to skim off a slice of profit from another country's manufacturing output, then so what?
> The internet is built on institutional trust.
No it’s not. What is this even supposed to mean?
No it’s not. What is this even supposed to mean?
Autonomous system (AS) operators should not act in bad faith towards other AS networks they peer with. BGP hijacking is an example of abuse of this institutional trust, same for DNS hijacking.
If this trust is repeatedly broken, peering networks may be forced to depeer the AS as a result, like what happened to McColo when they were depeered.
If this trust is repeatedly broken, peering networks may be forced to depeer the AS as a result, like what happened to McColo when they were depeered.
China currently makes some absurdly large percentage of the world's consumer goods, and the discussions about producing them are probably being had over the internet. Cut them off of the internet and we have to rebuild manufacturing capacity everywhere else.
Which might not be a bad thing overall, but it's sure not gonna make any transnational corporation's bottom line happy over the next few quarters, so they'll be waving a lot of money at politicians to make this not happen.
Which might not be a bad thing overall, but it's sure not gonna make any transnational corporation's bottom line happy over the next few quarters, so they'll be waving a lot of money at politicians to make this not happen.
China is an oppressive authoritarian state currently engaged in ethnic cleansing. How is enabling them indefinitely an option?
Because doing so would essentially push China towards a China-only internet, which they're already halfway towards.
The benefits of gobalization and the spread of democracy (or even just alternative governance models) via exposure to other cultures cannot be understated
The benefits of gobalization and the spread of democracy (or even just alternative governance models) via exposure to other cultures cannot be understated
Not a strong reason. I would be shocked if the average Internet user has heard of any of the top ten most visited websites in China. Their entire infrastructure, from the technological layer to the bureaucratic layer, has ensured that the average Chinese Internet user knows very little about the outside world that hasn’t been pre-vetted or filtered out completely by the GFW.
Im in China and met a Xinhua journalist once. At day she would edit propaganda stories that she knew were complete horseshit, at night her and her colleagues would go to their boss watch netflix together because he had a working VPN.
Even if just one port is left open, people will be curious enough to find it and use it. Chinese people are humans too :D
Even if just one port is left open, people will be curious enough to find it and use it. Chinese people are humans too :D
I liked this hypothesis overall—that exposure to democracy through trade is sufficient to breed democracy in China. It’s a confident and peaceful approach, and I’m glad that we tested it. However, in this case, I believe we’ve disproven the hypothesis; continuing to run the same experiment unmodified and expecting improving results is signing up for disappointment.
> continuing to run the same experiment unmodified and expecting improving results is signing up for disappointment.
Maybe that's an acceptable price to pay for not being the ones to take the next step toward war. If war is a game in which "the only winning move is not to play", then maybe it's also true that when it comes to doing the peaceful thing, the only winning move is to keep on playing, even if it hurts us.
Maybe that's an acceptable price to pay for not being the ones to take the next step toward war. If war is a game in which "the only winning move is not to play", then maybe it's also true that when it comes to doing the peaceful thing, the only winning move is to keep on playing, even if it hurts us.
Many British people made similar statements back in the early 1930s.
When do we accept the fact that Xi's ambitions extend far beyond the borders of Mainland China and pose a threat to the very idea of human dignity?
Is it when he invades Taiwan? Floods the Uyghur camps with gas? Bombs Japan? Lands an army at the port of Darwin?
When do we accept the fact that Xi's ambitions extend far beyond the borders of Mainland China and pose a threat to the very idea of human dignity?
Is it when he invades Taiwan? Floods the Uyghur camps with gas? Bombs Japan? Lands an army at the port of Darwin?
I disagree: I m not sure what's making you say it's not working but from a Chinese point of view, this was also an experiment, to try and open up a little.
Are you sure you're doing your part of the bilateral exchange? It cant just be China changing, the US must learn too to adapt and accept a larger, more powerful country, with a widely different model.
Living in China, I can tell you the american model is known, and not particularly impressive to them. They care a lot less about freedom of speech, maybe because they never had it, than they care qbout order, unity and crime rate for instance. And what I always hear is that throwing themselves at the communists in revolt to get the same shitty system as the US is not so seductive.
Maybe become a role model and people will beg to ressemble you ? I have a hard time convincing them voting for their government is gonna work better because "if even idiots can vote, look at who they elect" :s
Are you sure you're doing your part of the bilateral exchange? It cant just be China changing, the US must learn too to adapt and accept a larger, more powerful country, with a widely different model.
Living in China, I can tell you the american model is known, and not particularly impressive to them. They care a lot less about freedom of speech, maybe because they never had it, than they care qbout order, unity and crime rate for instance. And what I always hear is that throwing themselves at the communists in revolt to get the same shitty system as the US is not so seductive.
Maybe become a role model and people will beg to ressemble you ? I have a hard time convincing them voting for their government is gonna work better because "if even idiots can vote, look at who they elect" :s
Is it really working, though? Has there been a push in China (or other country connected to the global internet) for more democracy?
Yeah, just imagine if Chinese internet users were prevented from accessing websites that exposed them to a diversity of opinions
How is keeping the USA online (which by the way actively destabilizes other countries and seeks out war outside its own soil) any different?
It's totally an option if you're in a country effectively ruled by soulless transnational corporations who rank "making lots of money" several orders of magnitude more important than "any kind of ethical concern".
It's probably totally an option if you want to work for one of those corporations, too.
It's probably totally an option if you want to work for one of those corporations, too.
Maybe that is the plan, that we cut them off, so they don't have to
[deleted]
I was using icanhazip to check if my Tor circuit was complete, and probably made 50-100 requests per week. The site was getting slow, and I thought it is just a random site that the author didn't really care too much.
I dropped my jaw when I read it was getting 30B req/day.
Thank you for running this site for so long, and thank you for keeping it up for free, and deciding to not monetize it.
I dropped my jaw when I read it was getting 30B req/day.
Thank you for running this site for so long, and thank you for keeping it up for free, and deciding to not monetize it.
I got a lot of mileage out of neverssl.com before somebody fixed the process to log into various "guest wifi" setups...ones that would intercept/redirect any http request.
I'm somewhat curious what fixed things, as I've not had to use neverssl.com for some time.
I'm somewhat curious what fixed things, as I've not had to use neverssl.com for some time.
From what I can tell, most operating systems will now ping their own version of neverssl as you connect to a network to find out whether they need to show you a login prompt. It looks like they basically just check to see if they get the content they expect from a domain they own, and if not they serve you that page so you can see whatever it is your network injected. (You can usually see the OS domain in the address bar.)
captive.apple.com is what Apple uses
Thanks. I was reaching for an example but don't have a guest WiFi nearby to test.
Apple has at least ten or more of these I’ve seen - on badly configured networks you sometimes see it in the address bar - because cached responses could destroy the utility.
can you name one or two? I've never seen anything else besides captive.apple.com
I found https://community.ui.com/questions/Apple-iPhone-iOS-portals-... which lists some of these domains (www.itools.info, www.thinkdifferent.us, etc.), and this answer on StackOverflow -- https://stackoverflow.com/a/22277933/ -- which reverses a list of domains that all point to the same IP
Google's is http://connectivitycheck.gstatic.com/, and Microsoft's is http://www.msftncsi.com/ncsi.txt
Firefox uses http://detectportal.firefox.com/success.txt
I was always wondering given that firefox has that detectportal.firefox.com why they also put requests into example.com/example.org?
Leave it to Microsoft to use cryptic abbreviations and character limits even in URLs…
Alternatively there's http://www.msftconnecttest.com/ncsi.txt
Ah, that's interesting. I remember it being very broken for a long time...especially for "normal" users that wouldn't understand why navigating to an https site wouldn't work in that captive wifi situation.
Annoyingly, I've seen some networks that try to "fix" problems by letting that go through even when not logged in. I'm pretty sure it had been fixed last time I flew, but Qantas in-flight WiFi used to be one example I'd seen. You'd connect, and then nothing would happen and no SSL connections would work. You'd either need a non-HTTPs site to get the redirect, or go to 'wifi.qantas.com' to accept the terms and conditions before you could browse. I was trying to work out why my iPhone and Mac weren't popping up that page as soon as I connected, and bizarrely captive.apple.com came up with 'Success' instead of redirecting, which means they must have misguidedly put in an explicit rule to let that through, completely breaking the feature!
This exact thing happened to me on an American Airlines flight one time. It seemed to get fixed later.
The Qantas system runs on ViaSat's platform and satellite network, I don't know what American uses but perhaps it's based on the same system.
I set up my own version of both neverssl and icanhazip, with nothing but Nginx on a cheap VPS. I already had the server up for other purposes, and I feel better knowing that I'm not mooching off of other people's effort (and money).
Neverssl has done some pretty nifty work to avoid caching. It redirects you to a random subdomain over plain HTTP just to make sure the browser has a cold cache. Maintaining a domain, the redirects, and making sure to _not_ accidentally obtain a certificate is a burden I wouldn't want to do, although it is not that difficult to do.
I was reading from neverssl maintainer that they get a _lot_ of traffic, questionable ones more than it is not. Its DNS runs on AWS IIRC, and we all know Route53 isn't the cheapest.
I was reading from neverssl maintainer that they get a _lot_ of traffic, questionable ones more than it is not. Its DNS runs on AWS IIRC, and we all know Route53 isn't the cheapest.
Having lived through the debut of Firesheep, which prompted the industry to get serious about using TLS, its an amusing triumph of cybersecurity that today a site has to be careful to not accidentally get issued a certificate. Back in 2010, when certificates cost substantial sums and needed some expertise to apply for and install, I wouldn't have guessed we'd ever get to this point.
Interesting, I haven't encountered either of these issues, but I serve about 1 request a month (to myself) and have Certbot running automatically.
IIRC the neverssl maintainer _works_ at AWS.
Ha, I could've used this ~a year ago when I moved into my current rental apartment. Fiber optic internet is included in the rent (in the form of an RJ45 jack in my wall), but there's a captive portal requiring you to enter your contract ID, which resulted in having to scour my memory and bookmarks for a non-HTTPS site.
My go-to for this is example.com
I’ve seen packages that do ”internet-detection” by calling out to icanhazip.com, and I just thought that was so irresposnible. What if your package got popular, how much money are you costing the hoster? For services like this, people just don’t consider the fact that there’s someone on the other side.
If you want, you can set up a similar service yourself by adding the following lines to an NGINX config:
I'm curious as to what other overused utilities can be trivially done with pure server configs.
location = /ip {
default_type text/plain;
return 200 '$remote_addr';
}
I'm curious as to what other overused utilities can be trivially done with pure server configs.
Is it easy to do the same for Apache? The best solution I found was some hacky way with an ErrorDocument directive which seems pretty gross.
You can use SSI, and echo the remote IP: https://httpd.apache.org/docs/2.4/howto/ssi.html
``` <!--#echo var="REMOTE_ADDR" --> ```
``` <!--#echo var="REMOTE_ADDR" --> ```
Doing this for HN asks you to login as an administrator.
If you want JSON instead:
location /ip {
add_header Content-Type "application/json";
return 200 '{"host":"$server_name","ip":"$remote_addr","port":"$remote_port","server_ip":"$server_addr","server_port":"$server_port"}\n';
}I feel the same about dependency steps in CI, without a cache or any similar structure. Package repos like Rubygems, NPM and PyPi get utterly rinsed by the continual downloading and redownloading of stuff the client should have already stored.
This. And both with GitHub and GitLab it takes quite a bit of an extra effort to setup caching. It hurts to see 'npm ci' download half the internet every time a developer pushes to dev server.
Would be interesting to speculate about the greenhouse effect of all these repeated downloads
It used to be possible to have a squid or even a backwards varnish cache handle lots of this but https everything has made that much harder to do. Still possible, however.
If nothing else, it is patently wasteful and, as a user, you don't really see CI billed in terms of network bandwidth. Just indirectly through the equivalent of mainframe minutes. And even then, that's not enough to discourage anyone from building a suboptimal pipeline.
That's why the first step of CI for me, when possible, is to rsync a .tar.gz file from the server I'm deploying to. The tarball contains statically-linked binaries and other stuff I'll need for the build.
It's also a good reason for CI providers to mirror package repositories.
It's also a good reason for CI providers to mirror package repositories.
Nothing is going to change until the hosters make it a pain to abuse. Rubygems could require an api key to download from and rate limit that key.
Sure you could attempt to generate a bunch of keys and cycle them but it would be easier to just cache your gems.
Sure you could attempt to generate a bunch of keys and cycle them but it would be easier to just cache your gems.
The article was about abusive floods accounting for 90% of the traffic. The author was happy with legitimate use cases like packages doing detection, contrary to your comment.
That’s $8.03 per year right?
I used to use this site until I found https://checkip.amazonaws.com/. Switched because I wasn't sure who was behind icanhazip.com and it's tough to beat AWS. Glad to hear that it will likely be maintained for awhile longer!
Something to be aware of: checkip.amazonaws.com will happily return an X-Forwarded-For address https://stackoverflow.com/questions/52618096/under-what-circ...
Use https
That has nothing to do with an HTTP header. See for yourself:
$ curl -HX-Forwarded-For:127.0.0.1 https://checkip.amazonaws.com
127.0.0.1Headers don’t get injected on you unexpectedly if you encrypt.
I think the point is to prevent middleboxes (eg. caching proxy servers) from interfering with the request. Otherwise I don't really see the issue with the ip address being affected by X-Forwarded-For. You can just... not specify the header.
HTTPS encrypts headers, thereby preventing other people from adding headers to your request. Typically people are not adding X-Forwarded-For to their own requests.
I'm not arguing either point, I just pointed out that headers are independent of whether you use encryption. But now that I'm thinking about it for a sec, you might want to know what the proxy's exit IP is, and if the proxy adds an XFF Header then you just learn your own IP which wasn't what you wanted. If that is what GGGP meant.
I feel like in theory google should be returning this site, instead of the ad-filled sites when one searches "my ip address." But it always seems like Google heavily over-values the domain name and search term matches.
The "first result" in a Google query for "my ip" and other combinations is a box with your public IP. There's no reason to click in any of the ad-filled sites anymore.
This had been true for me for several years, but recently, I have found that Google no longer provides that info box and it's necessary to click into one of the search results. (I am using Firefox, but I am logged into a Google account of it makes any difference.)
Maybe you are getting a/b trapped?
Yes that's true, but besides the point. My point is that obviously relevant (to a human) results like icanhazip don't show up anywhere just because the domain name is... creative. This happens to more verticals than this one, where the top results happen to be websites with some SEO tricks like having a domain name that matches the search term.
[deleted]
Thanks for all your hard work! icanhazip.com / icanhazptr.com have been incredibly useful.
Small feature request: back in the day {ipv4,ipv6}.{icanhazptr,icanhazip}.com only had A / AAAA records, but now it seems they have both and thus a simple "curl ipv4.icanhzptr.com" can also give me a v6 address (of course, "curl -4" works). Would Cloudfare be OK with separating them again?
Small feature request: back in the day {ipv4,ipv6}.{icanhazptr,icanhazip}.com only had A / AAAA records, but now it seems they have both and thus a simple "curl ipv4.icanhzptr.com" can also give me a v6 address (of course, "curl -4" works). Would Cloudfare be OK with separating them again?
http://v4.icanhazip.com/ seems to return just the v4 address, same with http://v6.icanhazip.com/
Yep, same here -- no AAAA record for v4.icanhazip and no A record for v6.icanhazip . The documented interfaces were ipv4/ipv6 though https://major.io/icanhazip-com-faq/ and {v4,v6}.icanhazptr.com have both A and AAAA.
> icanhazptr.com
Cool, I didn't know about that one.
Cool, I didn't know about that one.
Suggest retitle: "Cloudflare acquires Icanhazip".
Glad it will live on!
Glad it will live on!
"Acquires" is a strong word when we are talking about $8 price. Maybe "Cloudflare becomes custodian of Icanhazip"?
This kind of service is exactly what STUN servers are made for. Designed to be used with webrtc, but it works perfectly alright by itself.
There are a plethora of unauthenticated STUN servers around, and while there's still room for abuse, the protocol is a bit more lightweight than full-blown http requests, and faster, too!
I've dabbled with doing this on my own, but I've found `myip` to do the job nicely and without hassle:
https://github.com/Snawoot/myip
There are a plethora of unauthenticated STUN servers around, and while there's still room for abuse, the protocol is a bit more lightweight than full-blown http requests, and faster, too!
I've dabbled with doing this on my own, but I've found `myip` to do the job nicely and without hassle:
https://github.com/Snawoot/myip
Can this be done using the stun command directly?
http://manpages.ubuntu.com/manpages/bionic/man1/stun.1.html
http://manpages.ubuntu.com/manpages/bionic/man1/stun.1.html
It is, but utility above queries multiple public STUN servers concurrently. As soon as quorum of servers replied with matching addresses, result is returned. This way it's more reliable and offers decent latency guarantees.
Wow so this person has been running this site for so many years, paying bills, answering god knows how many idiots and even getting close to trouble with 3 letter agencies and senators for absolutely nothing.. hats off to you sir, any other person would have thrown in the towel a long time ago.
Also i feel little bad you didn't get any money out of it whether the site was designed to make money or not. It would have been a wonderful end to the story if you got something back for all the years of hardwork you put into running it. You do have my appreciation if that means anything though.
P.S. this story is very similar to rawgit which was a wonderful site but also fell prey to malware aholes.
Also i feel little bad you didn't get any money out of it whether the site was designed to make money or not. It would have been a wonderful end to the story if you got something back for all the years of hardwork you put into running it. You do have my appreciation if that means anything though.
P.S. this story is very similar to rawgit which was a wonderful site but also fell prey to malware aholes.
> Also i feel little bad you didn't get any money out of I
Most likely it got them a much higher paying job than they would have otherwise gotten. Walking in and saying you single handedly run a site with billions of requests per day and petabytes of traffic will get you noticed.
Most likely it got them a much higher paying job than they would have otherwise gotten. Walking in and saying you single handedly run a site with billions of requests per day and petabytes of traffic will get you noticed.
Icanhazjob approach!
Major didn't need icanhazip.com to get a good paying and quality job, this man is legend (at RAX when working with him)
Being good is neither necessary or sufficient for getting a high paying job.
Being able to get in front of a hiring manager who is offering a high paying job and convincing them to hire you is the only thing. And having a very popular website with a ton of traffic is more likely to get you in front of a hiring manager with a good job than actually being good.
Being able to get in front of a hiring manager who is offering a high paying job and convincing them to hire you is the only thing. And having a very popular website with a ton of traffic is more likely to get you in front of a hiring manager with a good job than actually being good.
His CVE is good enough to get past any hiring manager for whom hosting the site would be interesting.
Rackspace:
- Principal Architect, OpenStack - Cloud Architect, Cloud Servers
- Chief Security Architect, Director
Rackspace:
- Principal Architect, OpenStack - Cloud Architect, Cloud Servers
- Chief Security Architect, Director
To drive this home, if anyone were to tell me they’re keeping ~400kRPS afloat singlehandedly, that’s “I’ll write you a req, what do you want to do?” territory.
Ditto (London office.)
He hasn't updated this for a while, but it gives you a good picture of the kind of guy Major is: https://majorhayden.com/
Such a fun resume.
I'd love to set one up like that.
Since I have mostly Windows Experience, it would probably need to be a .chm file :)
I'd love to set one up like that.
Since I have mostly Windows Experience, it would probably need to be a .chm file :)
Make sure you do that floating effect (where part of the text 'rises' from the text) IE4 and some other early CHM files had when you opened it to show off that there was a browser engine underneath with full javascript and all :-)
RHCA(rchitect)
That cert is no joke.
RHCA(dminstrator) is a great cert to get (not super difficult but not super easy, but like all RH certs you need to actually know the material).
RHCE is awesome to have, and a good bit harder than RHCAdministrator. But when I grow up I want RHCArchitect...
That cert is no joke.
RHCA(dminstrator) is a great cert to get (not super difficult but not super easy, but like all RH certs you need to actually know the material).
RHCE is awesome to have, and a good bit harder than RHCAdministrator. But when I grow up I want RHCArchitect...
The certs are more just to tick some checkboxes when working with certain enterprise clients (my guess).
I hate certs, but if you want in on certain projects, it doesn't hurt to have the desired certs, especially if you can get your employer to foot the bill.
Anyways, knowing him personally, I can attest he's a really good dev and wholesome person. Sad to see this site get bogged down with malicious activity, because like downforeveryoneorjustme, I kind of have it committed to muscle memory. Hope CloudFlare can keep it going strong!
I hate certs, but if you want in on certain projects, it doesn't hurt to have the desired certs, especially if you can get your employer to foot the bill.
Anyways, knowing him personally, I can attest he's a really good dev and wholesome person. Sad to see this site get bogged down with malicious activity, because like downforeveryoneorjustme, I kind of have it committed to muscle memory. Hope CloudFlare can keep it going strong!
The certs prove you have the knowledge, so you don't have to boast. I've only got one certificate (AWS generic one), and I underestimated how much you need to study for things like that. I mean that was more of a nice to have, I don't actually have aspirations to go into AWS or the "ops" side of things, I just felt like it was something I needed to know more about.
Yeah once your career is somewhat established, they don’t mean all that much. My RHCA (Administrator) aged out in 2017, and I’ve felt little need to renew it.
Yeah he seems like an awesome dude and engineer/leader. Definitely will be following his blog now. Interestingly however, I had never heard of icanhazip before today. Had always used ifconfig.me.
And they got a whopping $7 (nearly) in donations per year... :-/
I used it for a while, then after a couple of failures decided to whip up my own at https://geocode.xyz/myip
Took me 5 minutes of work and exactly one line of code.
Took me 5 minutes of work and exactly one line of code.
There's a lot more helpful info you can return, too. Try https://ifconfig.me. Works great in bash scripts too, as it only returns the IP when called with a curl user agent.
This is cool!
I run a very simple, completely free API service as well. Currently using Google Cloud Run, handling a constant 10 rps for ~$8/mo. Pretty happy with it. I could probably cost optimize more. I sure hope I never have to deal with 30 billion requests per day, though. I'm sure my patience would run thin as well. Thank you to the author for running this site for so many years!
I use this service in my Dynamic DNS script for Cloudflare too: https://github.com/kissgyorgy/cloudflare-dyndns
It's time to put this service to the first place.
It's time to put this service to the first place.
Just checked your python script - looks great! I wrote my own script a few years ago -- yours is way more robust!
> The site went from 1B requests per day to 30-35B requests per day over a weekend.
This is absolutely mind blowing.
This is absolutely mind blowing.
I always type "what is my ip" in google and get a reply. I am amazed that this service gets 30B to 35B hits a day
It will be bots using it. I have a python script that calls it every 5 minutes to get my current IP so that I can update my DNS records on Cloudflare. This is because my server is self hosted on a Pi and I don't have a static IP.
It’s too bad there’s not a simpler way to just ask your router what it’s external IP is or if it has changed. You’d think SNMP could do it.
There was a thread[1] a couple months ago where I discovered a method to get one's IP address relying only on DNS:
[1]: <https://news.ycombinator.com/item?id=26634476>
nslookup myip.opendns.com resolver1.opendns.com
[1]: <https://news.ycombinator.com/item?id=26634476>
[deleted]
For those behind a home router an alternative is to use UPNP, e.g., through the miniupnpc package on Debian which ships the `/usr/bin/external-ip` script that postprocesses the `upnpc -s` output.
I use this https://cloudflare.com/cdn-cgi/trace
I appreciate the x-otter header, and I think I should now have my team add x-rtfm headers to all of our services and apps.
[deleted]
[deleted]
[deleted]
I admire the fact the author didn't snap at some point and start returning 0.0.0.0 or 127.0.0.1 to some or all queries.
Or a rate limit of some kind
Reminds me of `echo $(dig @ns1.google.com o-o.myaddr.l.google.com TXT +short | tr -d \")`. I have no idea where this DNS query came from, because searching all of Google turns up nothing but https://github.com/GoogleCloudPlatform/cloud-self-test-kit/b..., which is never referenced by anyone. I had to track it down myself for a bootstrap.sh, but I don't like using undocumented sources for critical infrastructure.
My use case was needing to set the result of `hostname -f` in /etc/hosts in an automated fashion if a VPS provider didn't already add a line for the public Internet address in that file. You need to do this so that sendmail doesn't fail on `apt install` when it attempts to read your FQDN. So I couldn't use the NGINX example posted elsewhere here.
It seems like https://checkip.amazonaws.com/ is much more "reliable" in that it is publicly documented at https://docs.aws.amazon.com/sdk-for-net/v3/developer-guide/s....
To anyone who needs to read this: please don't use "services" like icanhazip for your provisioning. Even my examples above are bad.
It does strike me as weird that there is seemingly no POSIX-compliant way to get your public Internet address, from my readings.
Edit: Oh goodness... even Amazon's documentation recommends using Google's undocumented DNS query.[1]
[1]: https://aws.amazon.com/premiumsupport/knowledge-center/route...
My use case was needing to set the result of `hostname -f` in /etc/hosts in an automated fashion if a VPS provider didn't already add a line for the public Internet address in that file. You need to do this so that sendmail doesn't fail on `apt install` when it attempts to read your FQDN. So I couldn't use the NGINX example posted elsewhere here.
It seems like https://checkip.amazonaws.com/ is much more "reliable" in that it is publicly documented at https://docs.aws.amazon.com/sdk-for-net/v3/developer-guide/s....
To anyone who needs to read this: please don't use "services" like icanhazip for your provisioning. Even my examples above are bad.
It does strike me as weird that there is seemingly no POSIX-compliant way to get your public Internet address, from my readings.
Edit: Oh goodness... even Amazon's documentation recommends using Google's undocumented DNS query.[1]
[1]: https://aws.amazon.com/premiumsupport/knowledge-center/route...
> It does strike me as weird that there is seemingly no POSIX-compliant way to get your public Internet address, from my readings.
It is not possible to know your public IP address, except by fetching the information from a known entity on the public network.
And in some scenarios, your public IP will change frequently. There is no guarantee that it will be consistent across multiple requests.
It is not possible to know your public IP address, except by fetching the information from a known entity on the public network.
And in some scenarios, your public IP will change frequently. There is no guarantee that it will be consistent across multiple requests.
Yes it is? I'm not talking about a situation where you send a packet to a DHCP server with 0.0.0.0 in the headers, I'm talking about what happens after the fact.
Once you have been allocated an IP address, there should be a way to fetch said address. That's the whole mechanism behind forging UDP packets. If I didn't know what my source IP address was, it would be OK to send 0.0.0.0 out into the world all the time.
Practically speaking, today, it is, considering most, but not all, ISPs won't let you do this and they will rewrite UDP packets that attempt to forge their source.
Edit: OK, since I'm not being clear, let me be explicit: There should be a mechanism in DHCP which allows for the querying of your public Internet address when receiving a packet from a client in one of the Private-Use Networks as defined in RFC 6890. This query should be exposed as a feature of ifconfig or ip, lest a user be forced to manually write such a packet to receive the data.
Once you have been allocated an IP address, there should be a way to fetch said address. That's the whole mechanism behind forging UDP packets. If I didn't know what my source IP address was, it would be OK to send 0.0.0.0 out into the world all the time.
Practically speaking, today, it is, considering most, but not all, ISPs won't let you do this and they will rewrite UDP packets that attempt to forge their source.
Edit: OK, since I'm not being clear, let me be explicit: There should be a mechanism in DHCP which allows for the querying of your public Internet address when receiving a packet from a client in one of the Private-Use Networks as defined in RFC 6890. This query should be exposed as a feature of ifconfig or ip, lest a user be forced to manually write such a packet to receive the data.
> Once you have been allocated an IP address
Well, there’s assumption 1 that isn’t true, often, you aren’t allocated a public IP address…at least in IPv4 world
Well, there’s assumption 1 that isn’t true, often, you aren’t allocated a public IP address…at least in IPv4 world
> querying of your public Internet address
Upnp allows you to do that. But that relies on a lot of assumptions. Your address when connecting to X and Y could be different. It could be different depending on the port. It could be load-balanced and come randomly from a pool. It could vary depending on time of day. And many many other cases.
Upnp allows you to do that. But that relies on a lot of assumptions. Your address when connecting to X and Y could be different. It could be different depending on the port. It could be load-balanced and come randomly from a pool. It could vary depending on time of day. And many many other cases.
you could be behind NAT or reverse proxies or any number of virtual or physical network components. This service helps account for all of that.
Private-use networks connected to the public internet are an ugly hack. If you don't have a public address, you don't have a public address in any meaningful sense, so any effort to figure out "your" address is doomed from the start.
Particularly in these days of IPv6, just give everything a notionally publicly routable address and then every device can know its real address (of course this doesn't have to mean you actually route public traffic to every address if you don't want to).
Particularly in these days of IPv6, just give everything a notionally publicly routable address and then every device can know its real address (of course this doesn't have to mean you actually route public traffic to every address if you don't want to).
I guess you can pick an empty or safe spot from here[1] and send patches to major DHCP server implementations, feels like option 81 "Client FQDN" is already close to that though.
Also do keep in mind that an OS instance can have multiple Ethernet interfaces, and that each interface can have multiple IPs, and that not all DHCP gives out private addresses(University Wi-Fi gave me 133.xx back in the days ... behind NAT!), and that double NAT exists.
1: https://www.iana.org/assignments/bootp-dhcp-parameters/bootp...
Also do keep in mind that an OS instance can have multiple Ethernet interfaces, and that each interface can have multiple IPs, and that not all DHCP gives out private addresses(University Wi-Fi gave me 133.xx back in the days ... behind NAT!), and that double NAT exists.
1: https://www.iana.org/assignments/bootp-dhcp-parameters/bootp...
I'm not sure I follow your model of IP networking, but the problem boils down this:
In live networks, you don't always know which device presents your last-hop routable address to the public network, so it is not clear who can authoritatively answer the whatismyip question.
You can get an instantaneously-correct answer (i.e. correct in the instant, not necessarily quickly-answered) from an empirical test, but you need to choose an known (and trusted) entity on the public network to query. Querying multiple different entities might very well get you multiple different answers, depending on how your packets are routed and manipulated before they reach the public network.
And in all cases, there is no guarantee that your public address will be stable over time or location.
In live networks, you don't always know which device presents your last-hop routable address to the public network, so it is not clear who can authoritatively answer the whatismyip question.
You can get an instantaneously-correct answer (i.e. correct in the instant, not necessarily quickly-answered) from an empirical test, but you need to choose an known (and trusted) entity on the public network to query. Querying multiple different entities might very well get you multiple different answers, depending on how your packets are routed and manipulated before they reach the public network.
And in all cases, there is no guarantee that your public address will be stable over time or location.
This is why running this (completely trivial) service yourself is helpful -- you can get the external IP from the perspective you desire.
Indeed. I like to include two convenience endpoints in my webserver configs (except in cases where I'd need to justify them to compliance/audit, etc):
* /ip responds with the src ip of the request
* /req responds with the full headers and body of the request
> It does strike me as weird that there is seemingly no POSIX-compliant way to get your public Internet address, from my readings.
There is no singular thing called a "public Internet address". Imagine you're writing paper letters to someone. You write a letter, you put your own From address, you drop it in the slot. When the mailperson comes to collect the letter, they replace your mailing address with a special other codeword. And when they receive mail, they replace that codeword back with your original address. You would never know it was intercepted unless you asked around. There's no official protocol to ask for your codeword, it's just a trick the mail service does on your behalf.
Your home router does exactly this; it's known as "Network Address Translation", or NAT. It's not an official part of IPv4, and there's no protocol to ask what it is. Your computer thinks its local IP address (typically some variety of 192.168.0.1) is its real, public address, and your router does the swap behind your back.
There is no singular thing called a "public Internet address". Imagine you're writing paper letters to someone. You write a letter, you put your own From address, you drop it in the slot. When the mailperson comes to collect the letter, they replace your mailing address with a special other codeword. And when they receive mail, they replace that codeword back with your original address. You would never know it was intercepted unless you asked around. There's no official protocol to ask for your codeword, it's just a trick the mail service does on your behalf.
Your home router does exactly this; it's known as "Network Address Translation", or NAT. It's not an official part of IPv4, and there's no protocol to ask what it is. Your computer thinks its local IP address (typically some variety of 192.168.0.1) is its real, public address, and your router does the swap behind your back.
> There is no singular thing called a "public Internet address".
There sort of is, it just doesn't help answer the question.
Both types of addresses have blocks explicitly carved out which are not unicast addresses to be routed over the public Internet. If you have one of those addresses, such as 192.168.0.1, that definitely isn't your "public Internet address" because people can't route stuff to it.
There sort of is, it just doesn't help answer the question.
Both types of addresses have blocks explicitly carved out which are not unicast addresses to be routed over the public Internet. If you have one of those addresses, such as 192.168.0.1, that definitely isn't your "public Internet address" because people can't route stuff to it.
This is explicitly wrong. There absolutely is such a thing as a "public Internet" address, which is the exact terminology used by the IETF. NAT isn't a part of the Internet Protocol, but it is well-defined in other related RFCs.
Yes, I know about what the IETF considers a "public Internet address", but it's sort of ill-defined for a lot of standard network topologies. NAT is a technique, not an explicit protocol. The RFCs simply cover the technique as it existed in practice.
`ip a` will tell you your IP addresses. As far as I can tell, for IPv6 it actually does know it's public address since that is globally unique with no need for a public/private split.
The issue is that ifconfig and ip do not recognize the Private-Use Networks under any flag and thusly query DHCP for the public address, nor does DHCP have such a query. (To my knowledge.)
When the host has a private-use address, there's not necessarily a single public IP (through NAT etc). Packets from my computer will currently appear from two possible public addresses, depending if they've been routed through the work VPN.
When the host has a non-private-use address, there might still be NAT, or there might be no Internet access. I have servers in 128...* without Internet access.
The "public" address might differ depending on the destination or anything else.
"ip ad" shows the address(es) the network interfaces has, nothing more.
When the host has a non-private-use address, there might still be NAT, or there might be no Internet access. I have servers in 128...* without Internet access.
The "public" address might differ depending on the destination or anything else.
"ip ad" shows the address(es) the network interfaces has, nothing more.
dig myip.opendns.com resolver1.opendns.com is another option
> It does strike me as weird that there is seemingly no POSIX-compliant way to get your public Internet address, from my readings.
Because traditionally if you're doing things right, you're not using NAT, which is against IP specs and a nonstandard kludge. So you just take your socket and query its local endpoint address using getsockname and voila.
Because traditionally if you're doing things right, you're not using NAT, which is against IP specs and a nonstandard kludge. So you just take your socket and query its local endpoint address using getsockname and voila.
Why the extra `echo` and subshell $(...) ?
dig... | tr... works fine without it.
also, `dig -4 ... ` to get your IPv4 address, for us dual-stacked folks. Otherwise it returns your V6 address by default.
dig... | tr... works fine without it.
also, `dig -4 ... ` to get your IPv4 address, for us dual-stacked folks. Otherwise it returns your V6 address by default.
Yes, I pulled it out of a shell script which had more concatenated to it and didn’t bother cleaning it up.
What kind of documentation would you like to see for a service like this?
Genuinely asking because I've always used the query
I use Route53 and I either completely missed the checkip link or they simply don't mention it
Genuinely asking because I've always used the query
dig +short myip.opendns.com @resolver1.opendns.com
I use Route53 and I either completely missed the checkip link or they simply don't mention it
Reading the title I expected the article to be about a zipping tool called Icanha Zip. Pleasantly surprised by the great article.
Thanks for your service. Now it will get even more traffic as I realized the existence of the site!
WOW! Nice story. I’m running very similar project for free and for fun. Also this is usually happening to me every day. Besides other things I’m providing also website checks, so almost every second registration is used to wake up bots like repl.co or minecraft bots hosted on such sites. Life isn’t easy, right? :D anyway it is still the fun to run such service and I understand why author want it alive for a such long time :) when you want to try something similar with few more features, give a try to hostbeat.info
This is a perfect illustration of why the DNS system is fundamentally broken. There is zero reason why icanhazip.com needs to or should resolve to a single host or even a single virtual host managed by some SDN operated by a corp that controls thousands of machines.
These queries could load balanced across the whole internet, the code is small enough that you could just whitelist the code by checksum and then compare results from multiple underlying hosts. The fact that you need a giant corporation to be able to practically run the backend for what is essentially urn:asker.public.ip is absurd beyond belief.
The big players aren't going to fix this because the broken protocols give them a major competitive advantage.
These queries could load balanced across the whole internet, the code is small enough that you could just whitelist the code by checksum and then compare results from multiple underlying hosts. The fact that you need a giant corporation to be able to practically run the backend for what is essentially urn:asker.public.ip is absurd beyond belief.
The big players aren't going to fix this because the broken protocols give them a major competitive advantage.
It's worse than that: you shouldn't have to send an HTTP request to another server somewhere else in the world to figure out your public Internet address.
I’m surprised there isn’t something in ICMP that returns the routable host IP. Then you could ping anything in the world that responds to ping and know your IP.
stopped using them a few years ago when they started blocking my VPN. I set up is.gd/icanhazip which points to https://dynupdate.no-ip.com/ip.php who since the switch have been a great substitute
Had the pleasure of working with Major at Rackspace; his professionalism, ethics, and quality of person always impressed me and inspired me to be a better version of myself every day. This move is a very mature decision; one that was probably bittersweet. Kudos Major on taking a step forward and putting the stewardship where it belongs.
I wonder why he did not try to return empty responses to confirmed malware sources, to discourage them from using the site.
You can implement this yourself in nginx like this:
location /ip {
add_header Content-Type "application/json";
return 200 '{"host":"$server_name","ip":"$remote_addr","port":"$remote_port","server_ip":"$server_addr","server_port":"$server_port"}\n';
}
Doesn't seem to work right...
{"ip": "127.0.0.1", ...}
Because you were doing it from localhost, the connection came from localhost. It has to be exposed out on the internet for it to return an internet IP :)
Anyways, I agree for 1 off things, whatever, use icanhazip or whatever you want, it doesn't matter if you make 1 request a day or 1 request a month... But if you are doing anything in code that uses it, you should just host it yourself and be a good net citizen.
Anyways, I agree for 1 off things, whatever, use icanhazip or whatever you want, it doesn't matter if you make 1 request a day or 1 request a month... But if you are doing anything in code that uses it, you should just host it yourself and be a good net citizen.
What qualifies as “abuse” for a website that simply returns your IP? I don’t get it.
You made a free service, and then lots of people started abusing it by... using the service?
You made a free service, and then lots of people started abusing it by... using the service?
Giant amounts of traffic (order of billions of requests per day) from a small number of boxes. It clearly doesn't get them anything (if your IP isn't changing multiple times per second you'd be better off caching, if it is you're probably a moron), so it's reasonable to suspect it's either a mistake or an attempt to take the site offline.
I have admired Major for a long time, however I'm disappointed to see further defacto consolidation of widely used Internet infrastructure (regardless of if it should be getting used this way).
Not really sure why the downvoting. Do people really think private business is the best place for what has become such a heavily relied on free to use resource? Surely it would be better living with the same stewardship as the root DNS zones or similar.
May the great bird of the galaxy roost upon your home planet, Major.
I wonder how much it would've helped if the site was not python. The same thing can be replicated with just nginx something like this:
location /remote_addr {
default_type text/plain;
return 200 "$remote_addr\n";
}
> We made a deal for them to pay me $8.03
I seriously can't imagine NOT negotiating with a company like CloudFlare for a decent chunk of money. As a multi-billion dollar corporation, I wouldn't leave money on the table.
I seriously can't imagine NOT negotiating with a company like CloudFlare for a decent chunk of money. As a multi-billion dollar corporation, I wouldn't leave money on the table.
So it’s a honey pot now? What a lifecycle.
> If you’re curious, Cloudflare did pay me for the site. We made a deal for them to pay me $8.03; the cost of the domain registration. The goal was never to make money from the site (although I did get about $75 in total donations from 2009 to 2021). The goal was to provide a service to the internet. Cloudflare has helped me do that and they will continue to do it as the new owners and operators of icanhazip.com.
Damn, I'm almost mad at you for not hustling. Could have left with a few million easily over that entire time period, including charging Cloudfare a small fortune for the site! But hats off to you for being a true altruist. Too many paper altruists these days.
Damn, I'm almost mad at you for not hustling. Could have left with a few million easily over that entire time period, including charging Cloudfare a small fortune for the site! But hats off to you for being a true altruist. Too many paper altruists these days.
How?
There's a lot of value in owning a website that processes billions of requests per month. A lot of people rely on that website, what if he sold it to a malicious user? Tracking IPs or changing what they return?
Truly selfless service.
It cost him many thousands in money and tens of thousands in time.
And :
"If you’re curious, Cloudflare did pay me for the site. We made a deal for them to pay me $8.03; the cost of the domain registration. The goal was never to make money from the site (although I did get about $75 in total donations from 2009 to 2021). The goal was to provide a service to the internet. Cloudflare has helped me do that and they will continue to do it as the new owners and operators of icanhazip.com."
BTW, speaking as a nerd, he has the best formatted resume that I have ever seen !
See: https://majorhayden.com/
And :
"If you’re curious, Cloudflare did pay me for the site. We made a deal for them to pay me $8.03; the cost of the domain registration. The goal was never to make money from the site (although I did get about $75 in total donations from 2009 to 2021). The goal was to provide a service to the internet. Cloudflare has helped me do that and they will continue to do it as the new owners and operators of icanhazip.com."
BTW, speaking as a nerd, he has the best formatted resume that I have ever seen !
See: https://majorhayden.com/
With that kind of traffic I'd be tempted to take the site offline for a few days to see if some prominent services had an outage due to relying on it.
Cloudflare has an official method to get external IP that uses chaosnet records.
I found a little script for it a while ago.
In python:
https://gist.github.com/czaux/b729a5de8d1334afbbe147a6bcee55...
I read it as "I can has zip" all the while trying to figure out what this has to do with compression.
icanhazip.com is firmly in my debug tool box, I am really grateful that it continues to exist!
There is a weird quirk with icanhazip. It will return an internal cloudflare ip address when using their WARP service with http/3 enabled
Wow ! I would never have guessed icanhazip.com got such an enormous amount of requests per day ! I wonder how this site/service achieved such a notoriety ? I am really not trying to diminish the author work but since it is something pretty simple to build and I know there are lots of others alternative I wonder what makes people choose this one over an other like let's say :
http://ipinfo.io/ip https://test-ipv6.com/ checkip.amazon.com checkip.dyndns.com http://whatismyip.akamai.com/ ifconfig.me ip.zfx.fr // I even have mine ^^ etc...
http://ipinfo.io/ip https://test-ipv6.com/ checkip.amazon.com checkip.dyndns.com http://whatismyip.akamai.com/ ifconfig.me ip.zfx.fr // I even have mine ^^ etc...
As long as it is continuing working, OK. (I can verify that the returned data is correct without too much difficulty)