AWS us-east-1 down
15 comments
UPDATE:
Jun 13 3:42 PM PDT Between 11:49 AM PDT and 3:37 PM PDT, we experienced increased error rates and latencies for multiple AWS Services in the US-EAST-1 Region. Our engineering teams were immediately engaged and began investigating. We quickly narrowed down the root cause to be an issue with a subsystem responsible for capacity management for AWS Lambda, which caused errors directly for customers (including through API Gateway) and indirectly through the use of other AWS services. Additionally, customers may have experienced authentication or sign-in errors when using the AWS Management Console, or authenticating through Cognito or IAM STS. Customers may also have experienced issues when attempting to initiate a Call or Chat to AWS Support. As of 2:47 PM, the issue initiating calls and chats to AWS Support was resolved. By 1:41 PM, the underlying issue with the subsystem responsible for AWS Lambda was resolved. At that time, we began processing the backlog of asynchronous Lambda invocations that accumulated during the event, including invocations from other AWS services. As of 3:37 PM, the backlog was fully processed. The issue has been resolved and all AWS Services are operating normally.
Jun 13 3:42 PM PDT Between 11:49 AM PDT and 3:37 PM PDT, we experienced increased error rates and latencies for multiple AWS Services in the US-EAST-1 Region. Our engineering teams were immediately engaged and began investigating. We quickly narrowed down the root cause to be an issue with a subsystem responsible for capacity management for AWS Lambda, which caused errors directly for customers (including through API Gateway) and indirectly through the use of other AWS services. Additionally, customers may have experienced authentication or sign-in errors when using the AWS Management Console, or authenticating through Cognito or IAM STS. Customers may also have experienced issues when attempting to initiate a Call or Chat to AWS Support. As of 2:47 PM, the issue initiating calls and chats to AWS Support was resolved. By 1:41 PM, the underlying issue with the subsystem responsible for AWS Lambda was resolved. At that time, we began processing the backlog of asynchronous Lambda invocations that accumulated during the event, including invocations from other AWS services. As of 3:37 PM, the backlog was fully processed. The issue has been resolved and all AWS Services are operating normally.
Our apps are working fine, so I don't think "us-east-1" is down. Just the console..
Eh I'm seeing more here. Athena is struggling, IMBD search is down, Vercel is struggling.
Yep, us-east-1 itself is online.
It's more than just console. Lambda or API Gateway is borked. I can't log in to check though.
[deleted]
Same here. But status page is happy.
Lots of alerts in parallel throughout the infrastructure.
"Degraded performance" -- ha! Our Lambdas have been 100% down for an hour!
Several of our production redshift clusters are giving 504s in the console and InternalServerError exceptions related to xen_aws_credentials_mgr.cpp
We're seeing elevated error rates and latency and dramatically reduced throughput on our apps. This is not just Console.
every single time https://downdetector.com/status/aws-amazon-web-services/
Man, when Reddit goes dark it really means it!
https://health.aws.amazon.com/health/status