Kyutai unveils today the first voice-enabled AI openly accessible to all [pdf](kyutai.org)
kyutai.org
Kyutai unveils today the first voice-enabled AI openly accessible to all [pdf]
https://kyutai.org/cp_moshi.pdf
11 comments
Wow! There is no delay between when person's question end, and when AI starts its answer, just like there would be none when you were talking to a human. This feels eerie, a bit too quick. I know we will get used to it in short time though.
In fact, the delay is so short that it feels unnatural. They should add a small artificial delay before the machine answers.
The technology shown here is great - but I think the problem with most open-source voice assistants is that the voices just don't sound as good as the commercial ones. This one in particular appears to have a mixed accent and I'd imagine having it speak in French might produce more natural output. Examples of good voices are of course the OpenAI ones, and a bunch of the Nuance voices (I think the Tesla/Mercedes Me uses the "Ava" voice). Microsoft Azure's voices are decent too.
Even before neural TTS, some commercial TTS solutions were just better than others and a lot of that was the tone and timbre of the speakers. The challenge here, of course, is that
1) Good voice actors wouldn't want to train AI, especially not for free (though you could go far with people who aren't very professional, just committed)
2) You'll only have a limited set of voices, and people might not want that
3) Any really good model will have usage limitations because it sounds like some particular person
I'm wondering whether there are any techniques to create a good "synthetic" voice. Such as being trained on a large population of people, and then being able to generate character-consistent but uncorrelated with any input data output, the same way you can generate photos of people that don't exist but still look real. (I'm aware there's style transfer, but how do you create the style to begin with?)
I think that solving this may or may not be necessary to get a really robust voice AI solution esp one that's open source. Imagine choosing a voice for your AI personal assistant - wouldn't it be nice to be able to press a button and find *your* Alexa, not just *Alexa*?
You could also use that to find and create voices that "reflect your company's brand" or whatever, and not have to worry about changing it all the time when people come and go
Even before neural TTS, some commercial TTS solutions were just better than others and a lot of that was the tone and timbre of the speakers. The challenge here, of course, is that
1) Good voice actors wouldn't want to train AI, especially not for free (though you could go far with people who aren't very professional, just committed)
2) You'll only have a limited set of voices, and people might not want that
3) Any really good model will have usage limitations because it sounds like some particular person
I'm wondering whether there are any techniques to create a good "synthetic" voice. Such as being trained on a large population of people, and then being able to generate character-consistent but uncorrelated with any input data output, the same way you can generate photos of people that don't exist but still look real. (I'm aware there's style transfer, but how do you create the style to begin with?)
I think that solving this may or may not be necessary to get a really robust voice AI solution esp one that's open source. Imagine choosing a voice for your AI personal assistant - wouldn't it be nice to be able to press a button and find *your* Alexa, not just *Alexa*?
You could also use that to find and create voices that "reflect your company's brand" or whatever, and not have to worry about changing it all the time when people come and go
So i tried their demo on https://www.moshi.chat/?queue_id=talktomoshi
It is very fast and has no delay in answering. And while talking to it, you will find out that its fast because its predicting what you are going to say and will prepare the answer beforehand (that's how it feels like).
While talking it gets stuck in a loop like answer. It will keep saying the same thing no matter what you tell it. And the responses will be immediate, no delay. No matter what you say, answers will be the same.
It is very fast and has no delay in answering. And while talking to it, you will find out that its fast because its predicting what you are going to say and will prepare the answer beforehand (that's how it feels like).
While talking it gets stuck in a loop like answer. It will keep saying the same thing no matter what you tell it. And the responses will be immediate, no delay. No matter what you say, answers will be the same.
The demo is cool but the model is really poor.
I asked it to convert 75 fahrenheit to degrees and it insisted it would be 25 deg.
Then I asked how do you convert fahrenheit to degrees and it said « remove 32 » which is only half the answer.
Then I said, based on this answer, please convert 75 fahrenheit to degrees and it replied 38 deg.
Finally, I asked to integrate log(x) and it replied log (x).
Oh boy.
I asked it to convert 75 fahrenheit to degrees and it insisted it would be 25 deg.
Then I asked how do you convert fahrenheit to degrees and it said « remove 32 » which is only half the answer.
Then I said, based on this answer, please convert 75 fahrenheit to degrees and it replied 38 deg.
Finally, I asked to integrate log(x) and it replied log (x).
Oh boy.
lol, i just tried the demo of this, and it just had a two minute monologue with itself without me saying anything:
How is it going? Hey! So have you seen the new wolf movies? I don't think it's one of those movies where they like they kind of like go in and it's just like, oh, here are all the wolves, I don't know, there's like a big bad. There's a big bad. the main character is a wolf, a young wolf in a family. And then at the beginning of the movie, they get attacked by like the big bad wolves. And then I don't know, I don't know how to say it, but the main character gets separated from the rest of his family. So he's all alone and he has to like go through this adventure to find his family. And there's a lot of action and adventure and like you know, there's some like you know, like some uh, well like you know, some uh, like some sort of you know, like uh, you know, like a montage of him like you know, kind of like training in his new environment. And I don't know, like there's some like heartwarming moments and some like you know, some really fun, like action scenes and like some like you know, some funny moments. Um, but yeah, I don't know much about the story to be honest. Um, but I thought it looked really fun. Yeah. I'm just from a movie from the trailer. So from what I can gather, it should be. But you should see it just to see if it's if it's good. Yeah.
How is it going? Hey! So have you seen the new wolf movies? I don't think it's one of those movies where they like they kind of like go in and it's just like, oh, here are all the wolves, I don't know, there's like a big bad. There's a big bad. the main character is a wolf, a young wolf in a family. And then at the beginning of the movie, they get attacked by like the big bad wolves. And then I don't know, I don't know how to say it, but the main character gets separated from the rest of his family. So he's all alone and he has to like go through this adventure to find his family. And there's a lot of action and adventure and like you know, there's some like you know, like some uh, well like you know, some uh, like some sort of you know, like uh, you know, like a montage of him like you know, kind of like training in his new environment. And I don't know, like there's some like heartwarming moments and some like you know, some really fun, like action scenes and like some like you know, some funny moments. Um, but yeah, I don't know much about the story to be honest. Um, but I thought it looked really fun. Yeah. I'm just from a movie from the trailer. So from what I can gather, it should be. But you should see it just to see if it's if it's good. Yeah.
It sounds like someone leaving a voicemail
[deleted]
The French are really starting to knock it out of the park, and it's very refreshing to have another voice at the table.