use tools when Safari loses focus(twitter.com)

1 pointsby ojschwavor 2 Jahren1 comments

iOS web voice assistant that can talk/listen/use tools when Safari loses focus

https://twitter.com/ojschwa/status/1833147554277843103

1 comments

ojschwavor 2 Jahren

I've been working on a voice assistant that runs in your browser, and persist data to localStorage using TinyBase. On mobile it's annoying when the screen switches off, you lose access to the assistant.

I wondered if using Audio Context, I could keep the session going in the background. I naively gave it a go, and with loads of assistance from Claude 3.5, it seems to actually work, and the whole main thread seems active? The UX seems novel to me, and I'm looking forward to experimenting more to understand the limits of background execution, but I'm surprised I can update my localStorage without looking at it. Would you expect this to work?

Try it for yourself here (BYO API keys) -> https://tinytalkingtodos.com/

Some of the code involved.

https://gist.github.com/joshuacrowley/d4879dfe7e39edf86bc07b...