You're absolutely right. We used Jordan's Whisper-D, and he was generous enough to offer some guidance along the way.
It's also a valid criticism that we haven’t yet audited the dataset for existing list of tags. That’s something we’ll be improving soon.
As for Dia’s architecture, we largely followed existing models to build the 1.6B version. Since we only started learning about speech AI three months ago, we chose not to innovate too aggressively early on. That said, we're planning to introduce MoE and Sliding Window Attention in our larger models, so we're excited to push the frontier in future iterations.
Thank you so much for the kind words :)
We only support English at the moment, hopefully can do more languages in the future.
We are planning to release a technical report on some of the details, so stay tuned for that!
Thanks for the kind words!
You can try it now on https://huggingface.co/spaces/nari-labs/Dia-1.6B
Also, we'll try to update the Demo Page to something lighter when we have time. Thanks for the feedback :))
Yes! But you would need to put together a LLM system that created scripts from the book content. There is an open source project called OpenNotebookLM (https://github.com/gabrielchua/open-notebooklm) that does something similar. If you hook the Dia model to that kind of system, it will be very possible :) Thanks for the interest!
We just clarified in the README, sorry for the confusion ;(
Note that the model was not fine-tuned on a specific voice. Hence, you will get different voices every time you run the model. You can keep speaker consistency by either adding an audio prompt (a guide coming VERY soon - try it with the second example on Gradio or HF Space for now), or fixing the seed.
It's also a valid criticism that we haven’t yet audited the dataset for existing list of tags. That’s something we’ll be improving soon.
As for Dia’s architecture, we largely followed existing models to build the 1.6B version. Since we only started learning about speech AI three months ago, we chose not to innovate too aggressively early on. That said, we're planning to introduce MoE and Sliding Window Attention in our larger models, so we're excited to push the frontier in future iterations.