Assuming serverless is out if the question for your use case, have you tried spending a couple of days investigating a managed Kubernetes cluster with node autoscaling enabled? EKS, AKS, GKE...
Honestly it sounds like you could be at the point where K8s is worthwhile.
How hyperbolic. I'm struggling to imagine what you may have experienced, exactly, to put discussing such issues in the UK alongside discussing Tiannamen Square in China.
Honestly it sounds like you could be at the point where K8s is worthwhile.