Sony presented a simpler solution to this at CHI https://dl.acm.org/doi/10.1145/3772318.3791397 - check video in that. Also it seems every Cornell student is working on this. Just search on YouTube Cornell silent speech
It's been done for a while - follow the links to who they reference. ie https://www.tolt.tech but it's their integration they've done into the OS is interesting.
I’ve provided a lot of different of the balls. Last one was for someone in bed and needed to control the whole pc with his chin. We adapted the base to fit on a mount. Other uses for people with ALS with reduced hand function - altered the ball shape to hold the hand.
I find that the new "drug" is constantly hunting down new cheaper models.. z.ai/glm, mistral, deepseek.. if you need to get your fix - find the cheaper path..
take a peek at https://github.com/willwade/app-automate?tab=readme-ov-file#... - its early and needs some work -but this is the idea behind this.. (my use case is not agents but actual real disabled people..who need tooling to provide better access to the desktop)
Interesting! I started something - nowhere near as complete as that and quite different but again using accessibility UI elements. The BIG problem I've found is SOOOO much stuff does really poorly having these elements exposed. Here was my approach https://github.com/willwade/app-automate?tab=readme-ov-file#... - What I do here is build UI templates - either using UIAccess OR using a one pass using a vision model.
"my experience is the opposite actually. UIA looks uniform on paper but WPF, WinForms, and Win32 all expose different control patterns and you end up writing per-toolkit handlers anyway. Qt only exposes anything if QAccessible was compiled in and the accessibility plugin is loaded at runtime, which on shipped binaries is basically never. Electron is just as opaque on Windows as on macOS because it's the same chromium underneath drawing into a canvas. the real split isn't OS vs OS, it's native toolkit vs everything else."
It gave me strong vibes too. It’s the writing style. I’ve seen OpenAI write just like this. Doesn’t mean it’s bad. There’s a few other markers. Note “silly” in quotes and over use if that word. Once would be enough. But also this is very very typical. The bolting and short quite direct and a bit repetitive statements “it absolutely does not solve “dumped a bag of candy on a messy kitchen table and took a dramatic iPhone shot.”
Real example programs are where the joke becomes a language
I didn’t want this to stop at “hello world with candy colors.”” The over use of quoting. The bold. It’s not like a human wouldn’t write this. But it’s unusual for a human to do this imho. All the same - it feels novel. And at the end of the day it’s a neat idea. It’s just we enter this new brave world where things written like this give you the ick. “Where do the ai learn this from?” Well I wouldn’t mind betting the author asked it to be written in a hn style post.
this is so cool. I think so much of the logging Saas products are bloated. This looks much simpler. I'd like a python interface.. I've used papertrail and datadog for some of this in the past but dropped it due to cost (and bloat). Nice one.
that to me looks like a error in whatever logic is behind the positional error code. You'd think they would have transformer models based on different layouts but maybe some weighting issues going on.. ie I would have thought its a model that is altering based on likelihood weights and maybe something up with that..
can i have this between my machine and git please.. Like its twice now I've commmited .env* and totally passed me by (usually because its to a private repo..) then later on we/someone clears down the files.. and forgets to rewrite git history before pushing live.. it should never have got there in the first place.. (I wish github did a scan before making a repo public..)
Meta cheated with the mms models. That is they didn’t use a phonemeizsr step. This means they just won’t work or sound very strange. ASR data is usually not quite right for tts. But anyhow - not really answering your question but many of these languages already done in mms. Try them https://huggingface.co/spaces/willwade/sherpa-onnx-tts
We built this for our use case (we create solutions to help people speak who have a disability). This is a prediction model you can run in node or the browser. Next word, next character, word completion.. PPM is old - but still rocks