ButterCMS unreported downtime and security concerns(cside.dev)
cside.dev
ButterCMS unreported downtime and security concerns
https://cside.dev/blog/buttercms-unreported-downtime-and-security-concerns
10 comments
100%. The post cap for that price level is unreasonable, and I don't understand what kind of basic CMS service warrants that pricing.
Unrelated point of contrast:
I got an email a couple weeks ago from a vendor I'm using telling me about 2 hours of downtime I didn't even know about. They certainly could have only written to customers known to have been affected, and maybe just posted on a blog, but I got an email telling me things were down (I didn't even notice) and what was being done to reduce/avoid that.
They will likely grow to a point where they stop being that transparent, but until then... I appreciate the honesty and transparency.
I got an email a couple weeks ago from a vendor I'm using telling me about 2 hours of downtime I didn't even know about. They certainly could have only written to customers known to have been affected, and maybe just posted on a blog, but I got an email telling me things were down (I didn't even notice) and what was being done to reduce/avoid that.
They will likely grow to a point where they stop being that transparent, but until then... I appreciate the honesty and transparency.
> Of course domain change risk is not new to us. It’s a common attack vector in the client-side security world.
The "client-side security world" is just thrown out there!
I always assumed slapping an integrity attribute on script tags was just about the limit of what one could do to block hijacked scripts and otherwise yolo.
This is making me think there's lots more that can be done and there's a discipline to practice.
The "client-side security world" is just thrown out there!
I always assumed slapping an integrity attribute on script tags was just about the limit of what one could do to block hijacked scripts and otherwise yolo.
This is making me think there's lots more that can be done and there's a discipline to practice.
Surely a company that in their own docs suggests using "dangerouslySetInnerHTML" knows how important it is to be transparent about their downtime? This is shocking, thanks for sharing!
There are several red flags in this situation:
1) ButterCMS should have informed their customers/users about what they missed. 2) The way the issue was introduced could have led to a significant supply chain vulnerability. 3) I haven't found any analysis or communication from ButterCMS addressing the issue with their customers. 4) There’s still no downtime indicator on their website.
1) ButterCMS should have informed their customers/users about what they missed. 2) The way the issue was introduced could have led to a significant supply chain vulnerability. 3) I haven't found any analysis or communication from ButterCMS addressing the issue with their customers. 4) There’s still no downtime indicator on their website.
Thanks for sharing this. Unreported downtime is a nasty, nasty look for any service provider. Unfortunately even large providers are often guilty of it given that SLAs can be expensive when they're violated.
[deleted]
I’m Wiktor Walc, the CTO of Tiugo Technologies, which acquired ButterCMS in 2022. I completely understand the frustration and impact this may have had on your operations, and I wanted to personally apologize for how we handled the recent domain transfer and the communication surrounding it.
What Happened:
Domain transfers typically go smoothly, especially since we made sure to keep DNS records and the underlying infrastructure completely intact to minimize any risk. Unfortunately, this time, things didn’t go as planned.
An additional verification email sent by Amazon in August was mistakenly flagged by Gmail as a phishing attempt and placed in the spam folder, so it went unnoticed. As a result, we failed to verify the domain on time, and it was temporarily suspended. The moment we received the suspension notice from AWS, we immediately resolved the issue and restored the domain.
Missteps in Communication:
We recognize that there was a delay in communication after the domain issue had already been resolved, which was due to a breakdown in our internal processes. This misstep caused frustration, and we are committed to improving our communication practices moving forward. This happened because the team was not yet fully integrated into our communication processes. A few days after resolving the issue, we posted an explanation on our status page, which can be found here: https://status.buttercms.com/
Moving Forward:
This incident has been a valuable learning experience for us, and we are taking concrete steps to prevent similar occurrences in the future. Our improvements include:
- Faster responses on social media to keep you informed
- Real-time updates on the status page to ensure transparency
- Resolve outdated code examples to provide better support
We are also making long-term improvements to ButterCMS to ensure stability and security. A dedicated DevOps team has been assigned, and we’ve scaled up our engineering resources. Currently, we are planning the roadmap for the upcoming year and would greatly appreciate any suggestions or feature requests you may have.
Once again, I do apologize for the inconvenience this incident caused, and I truly appreciate your patience and understanding.
If you have any questions or feedback, please don’t hesitate to reach out.
What Happened:
Domain transfers typically go smoothly, especially since we made sure to keep DNS records and the underlying infrastructure completely intact to minimize any risk. Unfortunately, this time, things didn’t go as planned.
An additional verification email sent by Amazon in August was mistakenly flagged by Gmail as a phishing attempt and placed in the spam folder, so it went unnoticed. As a result, we failed to verify the domain on time, and it was temporarily suspended. The moment we received the suspension notice from AWS, we immediately resolved the issue and restored the domain.
Missteps in Communication:
We recognize that there was a delay in communication after the domain issue had already been resolved, which was due to a breakdown in our internal processes. This misstep caused frustration, and we are committed to improving our communication practices moving forward. This happened because the team was not yet fully integrated into our communication processes. A few days after resolving the issue, we posted an explanation on our status page, which can be found here: https://status.buttercms.com/
Moving Forward:
This incident has been a valuable learning experience for us, and we are taking concrete steps to prevent similar occurrences in the future. Our improvements include:
- Faster responses on social media to keep you informed
- Real-time updates on the status page to ensure transparency
- Resolve outdated code examples to provide better support
We are also making long-term improvements to ButterCMS to ensure stability and security. A dedicated DevOps team has been assigned, and we’ve scaled up our engineering resources. Currently, we are planning the roadmap for the upcoming year and would greatly appreciate any suggestions or feature requests you may have.
Once again, I do apologize for the inconvenience this incident caused, and I truly appreciate your patience and understanding.
If you have any questions or feedback, please don’t hesitate to reach out.
[deleted]
Reading this article and then the company not even mentioning it on their site or on any of their social channels is suspicious and just confirms I dodged a bullet.