I think it's a decent architectural decision.
It's a lot easier to develop solid code that handles a single request than code than handles multiple requests in parallel.
Functions are used for a lot more than HTTP, we use them for a lot of DB event handling such as post-processing, in which case cold starts does not really matter.
Maybe we'll one day get to a point where the cloud service knows the best way to run our code?
The NodeJS application is single-threaded, but the Node EventLoop can handle a huge amount of parallel requests, especially since most of the time per request is spent waiting for calls to the DB or external APIs.
I believe the majority of our performance gain is from having far fewer cold starts during large traffic loads.
I believe the team responsible for functions, both for GCP, AWS and Azure, do great work in optimizing functions, far, far better work than we could have done on our own.
The problem in our case is that they are architectured to only support a single request. It's not possible to serve a request faster by starting a new anything than serving it from something that is already running.
The Firebase suite has been _fantastic_ for developing our app, to quote one of the app developers: "[...] but from start until now, I feel we have saved years of shitty bugs, and that it is one of the best decisions we have done".
We still use Firebase for the app, and that's probably not going to change in a long time.
We hit some snags when we started requiring a performant API, which is why we don't use Firebase for that any more, although we still use Firestore and a lot of other stuff for the platform.
As every other technology it's choosing what pains you want, and the current Firesbase pains are quite easy to live with.
It is a full-fledged REST API, but we don't do a lot of caching (other than through Cloudlfare) as most of our endpoints either have volatile data or do various operations.
I agree!
We initially measured in percentiles, but during an internal presentation someone said it was too hard to understand and now we've lost the data :(
I haven't read the entire article, but I assume you need someone to admin the cluster?
How does the savings in billing compare to work needed for maintenance?
Functions still only run a single request at a time, while Cloud Run can run as many as the container can handle. We want to build for multiple smaller requests, so when we get ~10k requests incoming in a short timespan it's a lot more useful to have Cloud Run.
I may be mistaken, but I believe cold start is roughly the same for both.
Functions are used for a lot more than HTTP, we use them for a lot of DB event handling such as post-processing, in which case cold starts does not really matter.
Maybe we'll one day get to a point where the cloud service knows the best way to run our code?