But as others have mentioned, there are several problems with audiobooks as an ASR training dataset. First, the language used in literature is often very different from how people actually speak, especially if that language comes from very old texts (which many public domain books are indeed quite old).
Then there is the sound profile, which includes background noise, quality of microphone, speakers distance to device, etc. For recorded audio books, the speaker is often using a somewhat sophisticated setup to make the audio quality as clean as possible. This type of setup is obviously unusual when people want to speak to their devices.
Third, the tone and cadence of read speech is different than that of spontaneous speech (the Common Voice dataset also has this problem, but they are coming up with ideas on how to prompt for spontaneous speech too).
But the goal of Common Voice was never to replace LibreSpeech or other open datasets (like TED talks) as training sets, but rather to compliment them. You mention transfer learning. That is indeed possible. But it's also possible to simply put several datasets together and train on all of them from scratch. That is what Mozilla's DeepSpeech team has been doing since the beginning (you can read the above hacks blog post from Reuben Morais for more context there).
> “But Reitze counters that the complete data from that first run is already available online. According to Shoemaker, this includes the relevant time series data and the programs used, but "it's not a trivial matter to use them." Caltech even held a training workshop on how to deal with gravitational-wave data. That's a pretty far cry from asking the physics community to take its analysis on faith...”
Are you refuting this statement? To me, it looks like the data/analysis are open, but not yet independently verified due to difficult nature of problem space.
Just to add my two cents (I work for Mozilla on Common Voice): without help from linguists, Common Voice would have made some very different and very bad decisions about all sorts of things like: accents, dialect segmentation, corpus curation, licensing, and many other things. Linguistics were absolutely instrumental. We tried to thank some of them at the end of our blog post:
https://medium.com/mozilla-open-innovation/more-common-voice...
We have no plans to allow users to download the "raw" data from s3 (ie. before we perform the train/dev/test split). But we want to eventually build some tools to automate this. See here for some background:
> The other thing is that it's very cool to see the "you helped us reach out x% goal" thing but it locks up all the previous / next shortcuts which means I have to switch back to the mouse after 5 entries.
Just to note, we will never require your email address to contribute. There will always be an anonymous contribution workflow.
But adding new languages to Common Voice is a bit complicated at the moment, and we haven't built a way to do this through the website yet. So for now, we are doing this through a very manual process, and we plan to use email addresses to communicate.
Indeed Common Voice is not for everyone. We try to make it clear in our Privacy Policy [1] what pieces of data we collect and why. We do not publish email address or names with the data, and we even strip speaker identification info (so that a speaker's recordings are not grouped but instead everyone's recordings go into one giant bucket). That said, if this still makes you feel uncomfortable, we understand. And if you would like to contribute without donating your voice, you can always validate the recordings of others.
In the early days of this project, before we shipped the website (ie. ~March of 2017), we did some explorations around Mechanical Turk. The problem with the Mech Turk approach is that for recording your voices you need a lot of different people speaking (ie. 10s of thousands). But for languages other than English, Mech Turk simply doesn't have these kind of numbers. And indeed English is not that interesting to us, since there exists public data already in English (see LibriSpeech). There are of course other micro-task platforms popular in other countries (for instance, there's a myriad in Indonesia), but we didn't have the time to manage jobs on all these different platforms.
However, Mech Turk is better for things like validation, since you only need a handful of people doing the majority of work.
> The other thing is that it's very cool to see the "you helped us reach out x% goal" thing but it locks up all the previous / next shortcuts which means I have to switch back to the mouse after 5 entries.
We used some of the research around Mechanical Turk to find best practices for limiting trolling (e.g. [1]). Our approach thus far has been the two-thirds rule: if two out of three people say the clip is good/bad, we trust that. Also note, that we have seen remarkable low trolling numbers, and that most of the invalid are pronunciation mistakes or saying the wrong word.
I also want to emphasize the importance of listening (validating) as well as recording. Validation is an big part of the puzzle for building machine learning viable data.
Yup, this is an excellent point. We have, and will continue to explore ways to allow Common Voice users to speak more organically (for instance by answering a question, or responding free-form to some other sort of prompt). The problem with this approach is that it requires an extra step, transcription, which at the scale we are trying to achieve is pretty costly in either money or time (ie. tedium for our users). Eventually we hope that speech engines can take care of the transcription part, but for now we need people.
That said, we will definitely be exploring ways to build in organic speech and perhaps transcriptions to the Common Voice app. This will solve another problem for us too, which is getting public domain material for people to read. Doing this obviously requires a much more complex user experience, and we have more work to figure out how to make something that people will want to use and contribute to. Stay tuned for that :)
On the flip side, we hope that these datasets, models, and the tools (ie. DeepSpeech) can get more people (researchers, start-ups, hobbyist) over the hump of building an MVP of something useful in voice. Once you have people using your products, collecting useful in-context voice data becomes much easier.
On that note, another approach we are working on is partnering with universities and socially-aware startups like MyCroft, SNIPS, and Mythic. Imagine if voice products in market allowed their users to opt-in to contributing their utterances to an open resource similar to Common Voice. Of course, sharing your voice publicly is not for everyone, or every product scenario. But it does work for some. And if we pool our resources, our hope is to indeed commoditize speech-to-text so that we can focus on more interesting challenges like building voice experiences people want to use. (For instance, could voice somehow be a "progressive enhancement" to the web?).
This is a bug with our website [1]. We actually are trying to collect non-native speakers (as well as native). We are looking into clarifying this on the site.
But as others have mentioned, there are several problems with audiobooks as an ASR training dataset. First, the language used in literature is often very different from how people actually speak, especially if that language comes from very old texts (which many public domain books are indeed quite old).
Then there is the sound profile, which includes background noise, quality of microphone, speakers distance to device, etc. For recorded audio books, the speaker is often using a somewhat sophisticated setup to make the audio quality as clean as possible. This type of setup is obviously unusual when people want to speak to their devices.
Third, the tone and cadence of read speech is different than that of spontaneous speech (the Common Voice dataset also has this problem, but they are coming up with ideas on how to prompt for spontaneous speech too).
But the goal of Common Voice was never to replace LibreSpeech or other open datasets (like TED talks) as training sets, but rather to compliment them. You mention transfer learning. That is indeed possible. But it's also possible to simply put several datasets together and train on all of them from scratch. That is what Mozilla's DeepSpeech team has been doing since the beginning (you can read the above hacks blog post from Reuben Morais for more context there).