AWS Route53 is down(twitter.com)
twitter.com
AWS Route53 is down
https://twitter.com/bluekiriTeam/status/988757684966117376
11 comments
Probably the source of outage: https://news.ycombinator.com/item?id=16914698
Route 53 I believe is the only service that AWS promises "100% uptime". So much for that....
The Twitter post says it's Google DNS having issues querying. Are you sure the issue is with AWS Route 53 and not some routing issue between Google and AWS?
Some people are just having dns issues getting there. It's like saying my house is still intact even if you lost your map to it. You loosing the map does not impact the question of if my house is still standing or not.
Is it actually possible for any system to have 100% uptime? Or do they just use that as a marketing gimmick and give out credit whenever there is downtime.
sure, why not? I mean, assuming there isn't some nuclear attack that destroys all networking infrastructure. I'm not suggesting it's easy, but if you distribute it enough and make it able to work when other portions of the system disappear i see no reason you couldn't claim 100% uptime
And then you you have a stupid software bug and the redundant physical systems are not helping.
Azure Storage is a available in a ton of locations and all of them stopped working because Microsoft forgot to roll out their patch to renew the SSL certificate. So they knew about the issue but rolling out the patch was not done.
Another example was when they introduced HTTP2 for CDN endpoints and forgot to test the cipher list with popular browsers and broke compatibility with Firefox ESR.
Azure Storage is a available in a ton of locations and all of them stopped working because Microsoft forgot to roll out their patch to renew the SSL certificate. So they knew about the issue but rolling out the patch was not done.
Another example was when they introduced HTTP2 for CDN endpoints and forgot to test the cipher list with popular browsers and broke compatibility with Firefox ESR.
"works on my machine"
Our infrastructure is heavily using Route53 and we are not seeing any outage. (~1000 domains)
Edit: from http://status.aws.amazon.com/
"5:19 AM PDT We are investigating reports of problems resolving some DNS records hosted on Route53 using the third party DNS resolvers 8.8.8.8 and 8.8.4.4 . DNS resolution using other third-party DNS resolvers or DNS resolution from within EC2 instances using the default EC2 resolvers are not affected at this time.
5:49 AM PDT We have identified the cause for an elevation in DNS resolution errors using third party DNS resolvers 8.8.8.8 / 8.8.4.4 and are working towards resolution. DNS resolution using other third-party DNS resolvers or DNS resolution from within EC2 instances using the default EC2 resolvers continues to work normally.
6:10 AM PDT Between 4:05 AM PDT and 5:56 AM PDT, some customers may have experienced elevated errors resolving DNS records hosted on Route 53 using DNS resolvers 8.8.8.8 / 8.8.4.4 . The issue has been resolved and the service is operating normally."
Our infrastructure is heavily using Route53 and we are not seeing any outage. (~1000 domains)
Edit: from http://status.aws.amazon.com/
"5:19 AM PDT We are investigating reports of problems resolving some DNS records hosted on Route53 using the third party DNS resolvers 8.8.8.8 and 8.8.4.4 . DNS resolution using other third-party DNS resolvers or DNS resolution from within EC2 instances using the default EC2 resolvers are not affected at this time.
5:49 AM PDT We have identified the cause for an elevation in DNS resolution errors using third party DNS resolvers 8.8.8.8 / 8.8.4.4 and are working towards resolution. DNS resolution using other third-party DNS resolvers or DNS resolution from within EC2 instances using the default EC2 resolvers continues to work normally.
6:10 AM PDT Between 4:05 AM PDT and 5:56 AM PDT, some customers may have experienced elevated errors resolving DNS records hosted on Route 53 using DNS resolvers 8.8.8.8 / 8.8.4.4 . The issue has been resolved and the service is operating normally."
This was probably just due to someone being stupid but the conspiracy theorist in me asks: Sabotage by the Google Cloud team?
I saw issues resolving via azure dns servers (i have gitlab runners on there) as well:
ssh 10.0.1.7 ping -c 1 github-production-release-asset-2e65be.s3.amazonaws.com
ping: github-production-release-asset-2e65be.s3.amazonaws.com: Temporary failure in name resolution```
ssh 10.0.1.7 ping -c 1 github-production-release-asset-2e65be.s3.amazonaws.com
ping: github-production-release-asset-2e65be.s3.amazonaws.com: Temporary failure in name resolution```