do you really need an LLM should be the question. Re smart home devices, I haven't seen anyone using it other than setting an alarm, timer, turning on/off the lights, playing music, checking the weather and so on. Probably there are max 1000 things. you can have an end-to-end spoken language understanding rather than speech-to-text and nlp separately.
other than big tech, smaller speech companies - snips, picovoice, sensory are all doing that.
just because amazon has billions of dollars to acquire user data, or acquire and kill smaller companies, it doesn't mean it's the best or only way.
it's still expensive and not easy to build reliable models. thus open-source is not a sustainable option for maintainers. you have to have talented people and high quality data. talented people have opportunity cost, they can go to big tech with high 6 or 7figure TCs. some former mozilla people did. i'm not criticizing them, we gotta accept the cost of living is crazy. i care about my compensation too and i do not have energy left to contribute to foss. i salut those who can. you cannot compare donations vs. big tech TC unless you have a trust fund i guess. I don't have old money, so not sure about that part.
snips.ai is acquired by sonos. google and sonos are still in patent wars. i'm not sure whether sonos will survive. it's not easy if big tech lawyers come after you, even you are kinda big. mycroft was a part of patent war too and the poor guy dedicated all of his energy instead of trying to build something. https://mycroft.ai/blog/huge-win-for-mycroft-at-the-patent-t...
picovoice.ai tries to do things differently, combining on-device speech processing and subscription based model. it's the only company i know that has good hotword detection and speech-to-text. their free option is enough for my project https://picovoice.ai/pricing/
you may ask why nobody uses their tech to build an alternative voice assistant. again i dont think smart speakers are profitable enough. when you can buy an echo for $20 it's hard to find somebody to pay $20 per month. silicon shortage and logistics costs increased the boms. it's not easy to compete with amazon which can burn $10 billion.
openai api probably feeds their audio streams to microsoft. otherwise why would nuance move to the cloud when they can process on-device, right after microsoft acquires them. to find options not feeding big tech, we gotta start paying for non-big tech.
just because amazon has billions of dollars to acquire user data, or acquire and kill smaller companies, it doesn't mean it's the best or only way.