It would be nice if input could be taken from a command line argument or better yet, stdin so that it is fully scriptable.
ollama has a way to do this and lets you play with a bunch of models without being very smart (just do ollama pull <model-name> and it downloads a model and makes it available to ollama run/ollama serve).
Technically yes, the promise can first do an RPC to a distributed key/value store, and only then do the expensive computation (which typically is itself an RPC to some other system, perhaps a traditional database). Promise libraries should make this pretty simple to express (though always uglier than blocking calls).
I say expensive because even in a data center, an RPC is going to take on the order of 1ms. The round-trip from a client like a browser is much greater especially where the client (if this is what you meant by "client") has a poor internet connection.
Therefore this pattern is usually done inside of an API server in the data-center. Additional benefits of the server side approach is that an API server can better control the cache keys (you could put the API server "build number" in the cache key so you can do rolling updates of your jobs), TTL, etc. Also, you don't want a rogue computer you don't control to directly talk to a shared cache since a rogue computer could put dangerous values into your cache, remove elements from the cache they shouldn't, etc.
ollama has a way to do this and lets you play with a bunch of models without being very smart (just do ollama pull <model-name> and it downloads a model and makes it available to ollama run/ollama serve).
(Sounds like I also need to play with LM Studio).