went through quite a few iterations of aligning text to speech.
found that ai transcription was really good most of the time but would hallucinate quite a bit towards the start and end of books. which I think might be related to those models being partially trained on audiobooks, and only having the book text itself, without any of the intro or credits.
in the end I landed on extracting text from ebooks, using rule based and language specific segmentation, and espeak based alignment. pretty basic, but it worked wonders in terms of reliability and accuracy.
if you are looking to generate audio from ebooks this is probably not too helpful. it is something I tried to avoid. something about learning a languages from generated audio didn't sit right with me haha.
hey HN!
this is my attempt at language learning with audiobooks. it synchronizes text to speech, and shows translations. the audiobooks themselves are all public domain.
got all of the audio alignment, translation, and asset generation working on my gaming computer. pretty happy with the pipeline, except for the sometimes subpar translations.
if anyone is interested in the details I am happy to write them up!
if you are into language learning, I would love to hear if this could be useful to you!
hey HN! been working on this project on the side for a couple years now.
as a kid i got an audiobook along with a text copy to learn english. i found that to be a very nice combination.
i have been trying to learn more languages, but found that purely reading tends to worsen my pronunciation, but my listening comprehension is not good enough for audiobooks.
so i took some public domain audiobooks, synchronized them with text, paired it up with translations in some of the most spoken languages, and put it all on a website.
as this is for learning, i am avoiding ai generated audio entirely.
looking for feedback and happy to discuss details!
For context: I own licenses for both Affinity Designer, and the full Affinity 2 suite.
Just tried the new affinity application for a couple hours and it's pretty great. Personas are now studios and as far as I can tell features from all apps are now integrated into one.
Giving this away for free is insane value and I am very glad to have this as a photoshop alternative.