When your production environment involves thousands of servers and massive databases, at least one testing stage is essential.
You need CI and regression tests running on smaller stages with mock data, because a bad rollout can take some time to revert, even if it didn't include a rogue database query or migration that impacted your data. Knowing how to rollback quickly is important, but a penny of prevention is worth a pound of cure with large services.
And IMO you shouldn't let people "test in production" when the changes involve a production database; those should be off-limits. It's all fun and games until someone forgets which instance they're logged into.
You'll still get plenty of regressions, but the goal is to avoid major issues that impact most of your users/customers/etc.
You need CI and regression tests running on smaller stages with mock data, because a bad rollout can take some time to revert, even if it didn't include a rogue database query or migration that impacted your data. Knowing how to rollback quickly is important, but a penny of prevention is worth a pound of cure with large services.
And IMO you shouldn't let people "test in production" when the changes involve a production database; those should be off-limits. It's all fun and games until someone forgets which instance they're logged into.
You'll still get plenty of regressions, but the goal is to avoid major issues that impact most of your users/customers/etc.