+1 for OpenVINO, we utilise it for our model. It's quite amazing the inference speed you can get from CPUs that most people would assume are running on a GPU.
> Mine is focused more on long-term project tracking and program management for solo developers or solo builders.
This looks very useful, will download and give it a shot later. It took me a few seconds to find it on your page, and only got to the screenshots in the "navigate" sections after clicking through a lot. I would suggest putting a screenshot or something on the landing page so people can see and understand what it is.
Thank you, it's nice to hear someone else has gone through similar pain (in a good way)!
It's been slow and steady, but it's hard. I've commented previously that whilst the cost to build software has plummeted compared to 2 or 3 years ago, the ability to sell it has got harder and I feel this will keep accelerating.
This was built just for them so I've not spent too much time on the UI (ignore `unstable` in the name, it's just not on a proper release branch) but it's completely free so give it a go if you want. I'm working on the diarisation step so it can tag subtitles to people but that's not ready yet.
It utilises nvidia Parakeet as the ASR model, it is very much European language focused, the supported ones are:
If these languages aren't what you're looking for let me know what you need and I'll see what I can do.
I use subtitles extensively for everything I watch, so if I can help someone make older movies more accessible with them then that would make me happy.
I fell down the rabbit hole of voice transcription about a year ago, always had a love for utilising fine tuned LLMs so have put two and two together and built https://whistle-enterprise.com. The biggest challenge being it all running on CPU with the target device being your low to mid spec office laptop that's a few years old (I5, 8gb RAM). All nicely packaged together in a single completely offline selfcontained app that you just install and run (no environment setups, packages to download, models to download etc).
One of the hardest parts I've found is the diarisation (who said what) side of things. Trying to tune this and have it working in a way that doesn't absolutely grind the laptop to a halt or take forever to complete has been _hard_ but also extremely rewarding.
Another part has been the fine tuning side of the Phi-4 model, I'm on version 10 now, getting that pipeline down was a journey in itself, but I've got some great results. I wrote a bit about it in a comment here - https://news.ycombinator.com/item?id=48385906#48389625
I absolutely love working on this, I still wake up and the first thing I think about is voice transcription pipelines (sad I know), but I'm excited to see how much further performance and utility I can squeeze out.
I hate to do the "you're holding it wrong" trope, but I think you might have something misconfigured somewhere unless you missed a 0, because just past 60k tokens is such a small context window to be seeing issue in.
Do you have any old documentation that it's picking up and referencing? If you set all claude settings back to default do you see the same issue?
Thank you for the assumption, I'm actually not a developer at all.
I'm from a hardware / networking / infrastructure background. I've had extensive exposure to (web) application development as I'm working closely with development teams and I do have the bash/powershell scripting knowledge.
But honestly, if I tried this "the old fashioned way" it probably would have taken me about 6 to 7 years to develop that application, that's an optimistic estimate. You really do have to have a passion for what you're building, I didn't know that voice transcription and local LLMs would be such a driving force for me, but it's all I think about, so much that I find it hard to go to sleep sometimes.
This one works well. I think it's because there's no shine to it, it's just the data, what you need, right there without trying to fluff it all out with rounded edges and superfluous stuff.
I find it such a hard thing to quantify, I know it's not helpful but you can just feel the slop seep through.
I'm not sure if it's because I've iterated through so many sites that LLMs have produced that "slop" is instantly recognisable and it just feels soulless.
Not like web pages ever had a soul, but it's not there on the generic LLM generated sites.
Ahhh ok I totally understand what you mean. Yea the edge cases are absolutely where you start to feel the pain and things look good on the surface until you dig in. I think even in the age of LLMs the adage of 90% of the time is spent of the last 10% will ring true.
Sure an app can be built and spun up in an afternoon, but are you willing to spend another 6 months ironing out all those little bugs, tuning it a bit, testing, tweaking, testing etc.
This absolutely fascinates me. I had a friend who needed subtitle files generating for audio and using in CapCut yesterday yet none of the available stuff was suitable, so he asked if I could adapt some of my software to export subtitles.
2 hours later he's got a fully working piece of local software that does exactly what he wants, yet yours is not able to even sort dates correctly. Feel free to download it if you want to see for yourself, I didn't even do any UI tweaks as this was just a tool for him to use:
This seems so wide reaching if it's catching simple things like explaining a paper. Does this also refuse to help with any already developed training pipelines?
I can kind of understand the generation of synthetic data, but nerfing the assistance of training pipelines just seems like a really shitty thing to do.