The wide range of different responses to this post illustrates an important point; we can't agree on how to use LLMs in software development, and are still discovering new things.
And in a couple of months we might be doing things completely differently because of some new model or new framework.
Have not had such big outage issues as what's described here, although I have noticed more stability issues lately. Is this worse while Europe is sleeping maybe?
Came back here to say I was wrong! I have been experimenting, and it is doable.
I have been experimenting with setting up a scraping pipeline with LLM enrichment since I wrote the comment above, and have very positive results so far. :)
But LLMs themselves are literally not going away, I think that's the point. Once a model is trained and let out into the open for free download, it's there, and can be used by anyone - and it's only going to get cheaper and easier.
And in a couple of months we might be doing things completely differently because of some new model or new framework.
That's really cool.