Apple contractors listened to 1k Siri recordings per shift: former employee(irishexaminer.com)
irishexaminer.com
Apple contractors listened to 1k Siri recordings per shift: former employee
https://www.irishexaminer.com/breakingnews/ireland/apple-contractors-listened-to-1000-siri-recordings-per-shift-says-former-employee-945575.html
110 comments
It is surprising to anyone who doesn't work in machine learning.
Personally, I remember Google telling me that they "may use recordings to improve the voice recognition quality" or something like that. What I did not realize is that that meant humans would be listening. Sure, any ML expert will say "Well obviously! A machine can't train itself!", and in retrospect I wonder why this was not obvious to me. But here I am, a software engineer who worked for Google for eight years, and it was not obvious to me, somehow.
These services need to state more clearly how data is being used. When you sign up, it needs to say, clearly: "Human employees of [company] may listen to your recordings for the purpose of improving voice recognition."
I think the reason we are not presented with such clear language is because the people building these products know they'll lose a lot of users. But that's no excuse to mislead people.
Personally, I remember Google telling me that they "may use recordings to improve the voice recognition quality" or something like that. What I did not realize is that that meant humans would be listening. Sure, any ML expert will say "Well obviously! A machine can't train itself!", and in retrospect I wonder why this was not obvious to me. But here I am, a software engineer who worked for Google for eight years, and it was not obvious to me, somehow.
These services need to state more clearly how data is being used. When you sign up, it needs to say, clearly: "Human employees of [company] may listen to your recordings for the purpose of improving voice recognition."
I think the reason we are not presented with such clear language is because the people building these products know they'll lose a lot of users. But that's no excuse to mislead people.
> Well obviously! A machine can't train itself!
In some cases it can. That's what unsupervised learning is.
In some cases it can. That's what unsupervised learning is.
See that's sort of what I thought but now I'm being told it was silly to expect that. ¯\_(ツ)_/¯
one expert in ML told me once: unsupervised ML is a myth. I tend to believe him.
It's better to say that unsupervised ML is merely indirectly supervised ML.
Wasn't there a second version of AlphaGo that was trained only by playing against itself without external input and turned out to be even better than the first version that was trained with external input?
Isn't that unsupervised learning?
Isn't that unsupervised learning?
> don’t see how any of this is surprising
Agree, not a surprise, and already on the record. Here, from 2017:
Siri records your queries too, but she doesn’t catalog them or provide access to the running list of requests. You can’t listen to your history of Siri interactions in Apple’s app universe.
While Apple logs and stores Siri queries, they’re tied to a random string of numbers for each user instead of an Apple ID or email address. Apple deletes the association between those queries and those numerical codes after six months. Your Amazon and Google histories, on the other hand, stay there until you decide to delete them.
http://themillenniumreport.com/2017/03/not-only-are-alexa-si...
From Wired, “Apple finally reveals how long Siri keeps your data”, in 2013:
Once the voice recording is six months old, Apple "disassociates" your user number from the clip, deleting the number from the voice file. But it keeps these disassociated files for up to 18 more months for testing and product improvement purposes.
"Apple may keep anonymized Siri data for up to two years," Muller says "If a user turns Siri off, both identifiers are deleted immediately along with any associated data."
https://www.wired.com/2013/04/siri-two-years/
In general, customers are used to this pattern: “This call may be recorded for quality assurance purposes” clearly means someone will be listening to it later. The same “product improvement purposes” language for voice assist would naturally mean the same thing.
Agree, not a surprise, and already on the record. Here, from 2017:
Siri records your queries too, but she doesn’t catalog them or provide access to the running list of requests. You can’t listen to your history of Siri interactions in Apple’s app universe.
While Apple logs and stores Siri queries, they’re tied to a random string of numbers for each user instead of an Apple ID or email address. Apple deletes the association between those queries and those numerical codes after six months. Your Amazon and Google histories, on the other hand, stay there until you decide to delete them.
http://themillenniumreport.com/2017/03/not-only-are-alexa-si...
From Wired, “Apple finally reveals how long Siri keeps your data”, in 2013:
Once the voice recording is six months old, Apple "disassociates" your user number from the clip, deleting the number from the voice file. But it keeps these disassociated files for up to 18 more months for testing and product improvement purposes.
"Apple may keep anonymized Siri data for up to two years," Muller says "If a user turns Siri off, both identifiers are deleted immediately along with any associated data."
https://www.wired.com/2013/04/siri-two-years/
In general, customers are used to this pattern: “This call may be recorded for quality assurance purposes” clearly means someone will be listening to it later. The same “product improvement purposes” language for voice assist would naturally mean the same thing.
>If you want to improve your voice recognition, then you need to have a ground truth
This is true, but that doesn't automatically make it okay for them to offload this expensive resource to their customers without making it transparent that they were doing so.
>This is literally what google voice was for
Who cares? It's completely expected from Google, they use their customer data to do everything. I expect more from Apple. They have a (so far) great track record of protecting their customer's privacy. I fully expect Google, Facebook, et al to be doing ML on my photos, and that's why I'm bought into the Apple ecosystem instead of Google's. I value that my data is private more than being bombarded with stupid "collections" that the Google Assistant puts together with AI.
I really hope this doesn't indicate a new direction for Apple.
This is true, but that doesn't automatically make it okay for them to offload this expensive resource to their customers without making it transparent that they were doing so.
>This is literally what google voice was for
Who cares? It's completely expected from Google, they use their customer data to do everything. I expect more from Apple. They have a (so far) great track record of protecting their customer's privacy. I fully expect Google, Facebook, et al to be doing ML on my photos, and that's why I'm bought into the Apple ecosystem instead of Google's. I value that my data is private more than being bombarded with stupid "collections" that the Google Assistant puts together with AI.
I really hope this doesn't indicate a new direction for Apple.
As I said at the end of my post, I have a memory of them saying that they took sample for training. But thats hazy, given it must have launched >8 years ago.
but, given their recent drive, they should have asked for permission
but, given their recent drive, they should have asked for permission
> I expect more from Apple.
I expect them to improve their voice recognition which is way behind essentially everyone else's, especially Google's. There's nothing wrong with improving the tech in this manner as long as user's aren't surprised by it.
The fact that people are surprised by it is a problem though.
I expect them to improve their voice recognition which is way behind essentially everyone else's, especially Google's. There's nothing wrong with improving the tech in this manner as long as user's aren't surprised by it.
The fact that people are surprised by it is a problem though.
This is not Python, where the principle of least surprise applies :)
Given the way this is being debated, it's almost certainly undesired by customers and even potentially problematic under the GDPR.
Given the way this is being debated, it's almost certainly undesired by customers and even potentially problematic under the GDPR.
>I expect more from Apple.
They haven't done anything with your data, though, that doesn't show they 100% value your privacy. None of these recordings are identifiable and they're scrapped once they're audited. If you turn off Siri functionality, they're all automatically deleted whether they've been audited or not.
This is all addressed on the Siri screen right before you train Siri to recognize "Hey Siri".
They haven't done anything with your data, though, that doesn't show they 100% value your privacy. None of these recordings are identifiable and they're scrapped once they're audited. If you turn off Siri functionality, they're all automatically deleted whether they've been audited or not.
This is all addressed on the Siri screen right before you train Siri to recognize "Hey Siri".
Can you elaborate on how turning off Siri deletes the recordings?
Could someone use Siri, then right after turn it off to prevent Apple from storing the recording?
>I don't see how any of this is surprising.
Had thay said it before hand, I am pretty sure 99.9% of Apple's user would trust Apple and just click yes.
The problem is they didn't ask, nor did they tell. What privacy entails, in Steve Jobs's own words
Privacy means people know what they’re signing up for, in plain English, and repeatedly. I’m an optimist; I believe people are smart, and some people want to share more data than other people do. Ask them. Ask them every time.
Make them tell you to stop asking them if they get tired of your asking them. Let them know precisely what you’re going to do with their data.
Had thay said it before hand, I am pretty sure 99.9% of Apple's user would trust Apple and just click yes.
The problem is they didn't ask, nor did they tell. What privacy entails, in Steve Jobs's own words
Privacy means people know what they’re signing up for, in plain English, and repeatedly. I’m an optimist; I believe people are smart, and some people want to share more data than other people do. Ask them. Ask them every time.
Make them tell you to stop asking them if they get tired of your asking them. Let them know precisely what you’re going to do with their data.
>Had thay said it before hand
They did and they do.
They did and they do.
Clearly not "in plain English, and repeatedly" if this is news. I mean, when Apple says "What happens on your iPhone stays on your iPhone" [1] in big bold letters, you what do you expect that to mean? I mean, in plain English, that means nothing is sent back to Apple.
[1] https://9to5mac.com/2019/01/05/apple-privacy-billboard-vegas...
Edit: In fact, I went to that link they provided in the billboard, and then followed it to here.
https://www.apple.com/privacy/approach-to-privacy/
Nothing under the Siri and Dictation section says anything about letting contractors listen to your audio. Please show me where it says on these pages in plain English that third parties would be listening to your siri audio?
[1] https://9to5mac.com/2019/01/05/apple-privacy-billboard-vegas...
Edit: In fact, I went to that link they provided in the billboard, and then followed it to here.
https://www.apple.com/privacy/approach-to-privacy/
Nothing under the Siri and Dictation section says anything about letting contractors listen to your audio. Please show me where it says on these pages in plain English that third parties would be listening to your siri audio?
>if this is news
Now the goal posts have moved a little. The reason this is in the news is because a third-party contractor did this. This is not Apple's policy and I expect that the contractor has voided their contract with Apple.
Now the goal posts have moved a little. The reason this is in the news is because a third-party contractor did this. This is not Apple's policy and I expect that the contractor has voided their contract with Apple.
As a very minor point of fact, Google did have a service that (allegedly) existed primarily to collect voice training data, but it was GOOG-411, not Google Voice. Google Voice still exists, is an excellent service, and to my knowledge has not been credibly accused of primarily serving as a data collection tool.
This is just how assisted machine learning works. The model needs to be updated with corrected annotated data that its gotten wrong in the past. It needs to have enough of this data until the model accounts for it properly so that it begins to prioritize the corrected label instead of the incorrect one.
You're not going to be able to go in and hand-fix the model so that it doesn't cause these errors. This is one of the biggest ongoing issues with convolutional neural networks - you cannot debug them to determine why something went wrong.
You're not going to be able to go in and hand-fix the model so that it doesn't cause these errors. This is one of the biggest ongoing issues with convolutional neural networks - you cannot debug them to determine why something went wrong.
Why not just let the users give feedback on the responses? I’ve always been frustrated that I can’t.
Because the user is normally as unreliable as the machine model.
I suspect that when you are dictating, and you correct a blue squiggle, it raises a flag and sends that back as a failed event.
It also makes me think about the predictive text, does that call back home as well?
I suspect that when you are dictating, and you correct a blue squiggle, it raises a flag and sends that back as a failed event.
It also makes me think about the predictive text, does that call back home as well?
Well... the best way to predict text messages is to read lots and lots of text messages.
The difference is that predicting the next word in a text message can be validated without direct human intervention. It takes a human to tell whether Siri interpreted the voice command correctly. It just takes more data to evaluate next-word predictive systems.
The difference is that predicting the next word in a text message can be validated without direct human intervention. It takes a human to tell whether Siri interpreted the voice command correctly. It just takes more data to evaluate next-word predictive systems.
You can. There's an option on the Siri response to edit both your request and the words you used in the request.
[deleted]
The issue really isn't that they are doing it. It's that they were literally advertising the opposite.
https://9to5mac.com/2019/01/05/apple-privacy-billboard-vegas...
I went to that link they provided in the billboard, and then followed it to here.
https://www.apple.com/privacy/approach-to-privacy/
Nothing under the Siri and Dictation section says anything about letting contractors listen to your audio.
The kicker here isn't that Apple was listening to these recordings. The issue is they advertised something else, and then on top of that, Apple sent recordings to 3rd party contractors.
https://9to5mac.com/2019/01/05/apple-privacy-billboard-vegas...
I went to that link they provided in the billboard, and then followed it to here.
https://www.apple.com/privacy/approach-to-privacy/
Nothing under the Siri and Dictation section says anything about letting contractors listen to your audio.
The kicker here isn't that Apple was listening to these recordings. The issue is they advertised something else, and then on top of that, Apple sent recordings to 3rd party contractors.
>Now, ofcourse apple have positioned them selves as a privacy first company, so this is a big no no.
I don't think this is a "no no" based on Apple's privacy stance. The terms do say that they can send the recording in order to perform additional analysis and there's no user data, user ID info, or anything else that would give up a user's privacy included in what's sent over. Unless someone managed to recognize a specific voice in a recording, there's literally no way to tie a specific user to that recording because Apple anonymizes all of it on-device before it gets sent out.
I don't think this is a "no no" based on Apple's privacy stance. The terms do say that they can send the recording in order to perform additional analysis and there's no user data, user ID info, or anything else that would give up a user's privacy included in what's sent over. Unless someone managed to recognize a specific voice in a recording, there's literally no way to tie a specific user to that recording because Apple anonymizes all of it on-device before it gets sent out.
I find it surprising, I would have assumed Apple would simply used subset of informed and paid users to get new data. Just because you need to improve the system does not mean you need to use the recordings from everyone.
Exactly. And it's unreasonable to expect the public to be aware of the sheer amount of data that modern ML consumes, compared to how the more familiar "this call may be recorded for training purposes" is used.
Plus, the latter actually warns you about it every time you are recorded, not somewhere beforehand buried in a TOS. And the experience of "just talking to a computer" (not a person) makes people likely to share all sorts of private stuff.
Plus, the latter actually warns you about it every time you are recorded, not somewhere beforehand buried in a TOS. And the experience of "just talking to a computer" (not a person) makes people likely to share all sorts of private stuff.
I am surprised that Apple was doing it without asking for permission. I would happily allow Apple engineers to listen to failed interactions if I had a chance to pick and choose which ones to give them access to.
I'm sure, as the article says, Apple contractors listened "...without Apple users’ knowledge." However, that doesn't necessarily mean Apple didn't ask permission - I would bet there's something in the terms of service or a similar document that states Apple would listen to improve the quality of the service.
Users likely didn't know about it because, like most people, they didn't read the full TOS. Apple would've been smart to include a conspicuous statement about employees/contractors listening to recordings (if only to preempt this kind of PR debacle)... in addition to the legalese.
Users likely didn't know about it because, like most people, they didn't read the full TOS. Apple would've been smart to include a conspicuous statement about employees/contractors listening to recordings (if only to preempt this kind of PR debacle)... in addition to the legalese.
I personally don’t have a problem either way, but there is a nuanced difference.
If someone presses the home button to actively activate Siri they are purposefully sending messages to Apple and shouldn’t be surprised that the data is being processed and maybe even listened to by humans.
However, I could see someone being slightly more concerned about accidental activations causing data to be sent to Apple.
It would also be easy for Apple not to allow humans to listen to recordings triggered by Hey Siri.
If someone presses the home button to actively activate Siri they are purposefully sending messages to Apple and shouldn’t be surprised that the data is being processed and maybe even listened to by humans.
However, I could see someone being slightly more concerned about accidental activations causing data to be sent to Apple.
It would also be easy for Apple not to allow humans to listen to recordings triggered by Hey Siri.
>This is literally what google voice was for, it was a way of google getting lots and lots of samples of people talking using a phone's microphone.
That's what GOOG411 was for as well.
That's what GOOG411 was for as well.
Why the outcry when Microsoft did the same though?
Of course it makes sense to do this if you want to improve your training set.
Of course it makes sense to do this if you want to improve your training set.
(Edit: before downvoting this post, or any others on HN, please consider reading the whole thing to make sure you actually disagree with it.)
I agree that it shouldn't be surprising, but what does surprise me is that this isn't a massive legal problem for Apple at the criminal level. Every time someone uses Siri in a one-party consent state, isn't that a violation?
Seems like everything could have been fixed with a EULA clause saying that Siri queries and all other communications with Apple are "recorded for quality purposes."
The fact that we aren't talking criminal consequences suggests that there is, in fact, such a clause in there somewhere, and that this entire story has been grossly sensationalized.
I agree that it shouldn't be surprising, but what does surprise me is that this isn't a massive legal problem for Apple at the criminal level. Every time someone uses Siri in a one-party consent state, isn't that a violation?
Seems like everything could have been fixed with a EULA clause saying that Siri queries and all other communications with Apple are "recorded for quality purposes."
The fact that we aren't talking criminal consequences suggests that there is, in fact, such a clause in there somewhere, and that this entire story has been grossly sensationalized.
You're welcome to read the Siri EULA, which is linked from the places where you enable Siri on your devices: https://assets.pcmag.com/media/images/421115-activate-siri.j...
The key sentence is toward the end:
> By using Siri or Dictation, you agree and consent to Apple’s and its subsidiaries’ and agents’ transmission, collection, maintenance, processing, and use of this information, including your voice input and User Data, to provide and improve Siri, Dictation, and other Apple products and services.
So both the data collection and the fact that contractors may be involved are mentioned in the EULA (and have been mentioned for many years, going back to at least iOS 5: http://images.apple.com/legal/sla/docs/ios51.pdf).
Whether people feel they've been adequately briefed on this is a separate question.
The key sentence is toward the end:
> By using Siri or Dictation, you agree and consent to Apple’s and its subsidiaries’ and agents’ transmission, collection, maintenance, processing, and use of this information, including your voice input and User Data, to provide and improve Siri, Dictation, and other Apple products and services.
So both the data collection and the fact that contractors may be involved are mentioned in the EULA (and have been mentioned for many years, going back to at least iOS 5: http://images.apple.com/legal/sla/docs/ios51.pdf).
Whether people feel they've been adequately briefed on this is a separate question.
Right, and even covered in Wired in 2013: https://www.wired.com/2013/04/siri-two-years/
Doesn't matter. I live in a country where GDPR applies. I should be able to opt-in or opt-out of such (without losing the functionality). On my MBP, I opt out of by disabling Siri. Which isn't the way it ought to work. How it ought to work is that I am able to opt-in allowing analysts to listen in.
I'm trying to write a polite response to this, but this just comes off as so cynical to me it's hard not to be outraged that this is the top post in this thread right now. What's to stop you from apply the same argument to medicine? Why shouldn't we test the side effects of new drugs by testing it on people who don't know they're being drugged? We don't just want to improve AI, we also want to improve exercise equipment and parcel delivery. So of course you just have to surreptitiously record people's workout routines and put hidden tacking devices inside the products they buy off amazon. People should just know that if you buy a fancy Japanese toilet with a built-in bidet and heated seats, that it's also streaming video of your anus to HQ for quality control purposes.
Just because you want to improve your product, that doesn't mean you can do anything you want. No one should have to assume that the products they're buying are Trojan horses that are secretly spying on them.
Just because you want to improve your product, that doesn't mean you can do anything you want. No one should have to assume that the products they're buying are Trojan horses that are secretly spying on them.
It’s not a secret and it’s not new. (2013) https://www.wired.com/2013/04/siri-two-years/
Now sure they could say every individual think they do with this data. But, Apple makes it clear they collect and use this information to improve Siri. What else do people think this means?
Now sure they could say every individual think they do with this data. But, Apple makes it clear they collect and use this information to improve Siri. What else do people think this means?
The fact that I have to listen to a "your call may be monitored" spiel every time I talk to customer support leads me to believe tech should at least be annoying on first boot if never again.
That you link an investigative Wired article shows that most people aren't going to know this information. Siri users aren't all 2013 Wired readers, nor are they waiting around to hear what an Apple spokesperson is going to clarify.
Frankly, Apple should at least prompt people even just to be the good guy when it comes to privacy fallout on all these services.
That you link an investigative Wired article shows that most people aren't going to know this information. Siri users aren't all 2013 Wired readers, nor are they waiting around to hear what an Apple spokesperson is going to clarify.
Frankly, Apple should at least prompt people even just to be the good guy when it comes to privacy fallout on all these services.
You get a separate pop up to enable Siri. I read it and said no.
Granted, that might not seem like enough, but it was clear enough IMO.
Granted, that might not seem like enough, but it was clear enough IMO.
No, I agree that if the "your call may be monitored" is played before every monitored call centre phone call, then Siri should also remind people, at least once every other day. I'm not sure how often people reboot their iPhones, but I don't think doing it just on boot is enough, either.
It would drive the point home if Siri just said, every N queries, "anything you say to me can and will be sent to Apple's servers for analysis purposes".
It would drive the point home if Siri just said, every N queries, "anything you say to me can and will be sent to Apple's servers for analysis purposes".
Call centers don’t know who is calling, and thus add the message every time. It’s essentially always the first message until you identify yourself.
As to regular pop ups or messages, that’s simply a worse user experience for zero benefit.
As to regular pop ups or messages, that’s simply a worse user experience for zero benefit.
You are of course, right to be angry.
There is a reason I don't have an echo/home/$other. There is a reason I don't use cloud stuff for my personal life(where practical) I don't have web enabled CCTV, I don't smart locks, I generally don't have stuff with cameras in them.
There is evidence for suggesting that the anonymisation apple uses is quite good. Unlike with alexa, siri recording I don't think have been used in court cases. This means that apple are fairly sure that they can't recover people's recording, on pain of contempt of court.
Why is that important? because it means that its most likely GDPR compliant. If you are an EU citizen, I fully recommend you doing a subject access request, to see what data they hold on you. You are within your rights to have that data deleted, or amended. Or if you so wish, none of your details to be ever processed again.
Now, sadly for the USA, your protections are frankly shite. I suspect you'll have to lobby your local politicians.
There is a reason I don't have an echo/home/$other. There is a reason I don't use cloud stuff for my personal life(where practical) I don't have web enabled CCTV, I don't smart locks, I generally don't have stuff with cameras in them.
There is evidence for suggesting that the anonymisation apple uses is quite good. Unlike with alexa, siri recording I don't think have been used in court cases. This means that apple are fairly sure that they can't recover people's recording, on pain of contempt of court.
Why is that important? because it means that its most likely GDPR compliant. If you are an EU citizen, I fully recommend you doing a subject access request, to see what data they hold on you. You are within your rights to have that data deleted, or amended. Or if you so wish, none of your details to be ever processed again.
Now, sadly for the USA, your protections are frankly shite. I suspect you'll have to lobby your local politicians.
I wouldn't see the problem with it if they were open about it and allowed opting in and out.
I use Android phones, and there are many times I wish there was a human listening, usually when I am cursing at the OK Google lady for being so stupid. (yesterday: "no you idiot, I'm not going to tap one of the options for gas stations, isn't it f*ing obvious I'm driving a car??!!!")
I use Android phones, and there are many times I wish there was a human listening, usually when I am cursing at the OK Google lady for being so stupid. (yesterday: "no you idiot, I'm not going to tap one of the options for gas stations, isn't it f*ing obvious I'm driving a car??!!!")
> I use Android phones, and there are many times I wish there was a human listening
Google does the exact same thing to train the “OK Google” functionality. Apple wasn’t mechanical turk-ing Siri, this is about humans tagging prerecorded data for ML training datasets.
Google does the exact same thing to train the “OK Google” functionality. Apple wasn’t mechanical turk-ing Siri, this is about humans tagging prerecorded data for ML training datasets.
Yes I wasn't suggesting that Apple or Google were having humans actually process the requests real time. My wish that they were listening would be satisfied if, months later, someone heard my frustrated response and there was a way they could register that in the ML training set or otherwise register that so that product designers could see it. (in the example above, it is more of a product design issue than a ML issue)
That's actually what I am hoping for -- I'm not trying to make a minimum wage employee miserable, I would like the actual software to get smarter. (and maybe make the AI Google Lady a bit unhappy, if that is possible)
That's actually what I am hoping for -- I'm not trying to make a minimum wage employee miserable, I would like the actual software to get smarter. (and maybe make the AI Google Lady a bit unhappy, if that is possible)
It seems to me the limitations with assistants aren't voice recognition but NLP. If I have to craft my query for a machine then you've failed to make a 'natural language' input.
I agree (mostly) but not sure how that applies to my comment. Yes my major frustrations with it is that it isn't able to form a helpful response, not that it gets the actual words wrong. And in the case above, it goes beyond that. Designers of the product (Google maps) have it present onscreen options you need to click, in response to a spoken request (e.g. "find gas stations on my route") that is likely to be by a person who is driving and should not have to look at a screen. I would argue that that is not just non-helpful, but irresponsible and unsafe.
But I guess you are right, in that if they have the transcripts, they don't need to actually listen to recordings. Then again they might need to listen to recordings if they are trying to gauge user frustration.
But I guess you are right, in that if they have the transcripts, they don't need to actually listen to recordings. Then again they might need to listen to recordings if they are trying to gauge user frustration.
> It seems to me the limitations with assistants aren't voice recognition but NLP.
You probably speak English with a common American, Canadian, or British accent. Quality for common-accent English is fairly high, but is much lower for people with thick accents or who are speaking languages with smaller available training datasets. These sorts of initiatives are mostly about bringing quality up for those demographics.
You probably speak English with a common American, Canadian, or British accent. Quality for common-accent English is fairly high, but is much lower for people with thick accents or who are speaking languages with smaller available training datasets. These sorts of initiatives are mostly about bringing quality up for those demographics.
That's very true but I wish they'd work on making a better product as well. The growth in that area seems stagnant from both Siri and Google
I've learned that you can say "Feedback" and Google Home will let you dictate some feedback.
I don't know if anyone is actually listening, or if it's just being thrown into a huge dataset somewhere, but it's something.
Recently GHome broke reminders for me. I'm still upset about it.
I don't know if anyone is actually listening, or if it's just being thrown into a huge dataset somewhere, but it's something.
Recently GHome broke reminders for me. I'm still upset about it.
It goes into your Google Feedback account along with any feedback you've left using one of the "send feedback" links on many Google products.
https://www.google.com/tools/feedback/
https://www.google.com/tools/feedback/
You can't opt out anyway, everyone has a smartphone and many have Siri and other services activated. If Siri didn't occasionally activate by accident in meetings I wouldn't even be really aware of that.
How long until these recordings start leaking to the public because of a contractor not being up to security requirements or some employees taping recordings?
I work for a company where we have contractors at the customer service centers. This is pretty standard for customer service workers. They handle reservations over the phone and customer support. The centers where they work are very locked down. They deal with customer names and credit cards as part of their jobs. They go through background checks. (It can be hard to get enough people who both apply and can pass the checks ) They cannot bring in phones, memory sticks, pens, or paper. they have to pass through a scanner to enter and leave the floor. They work on stations with locked down ports and restricted internet access.
I went there for a meeting but was not allowed on the floor because it was not pre-approved by security.
If the Apple centers where Siri errors are reviewed is like that, it would be hard to sneak recordings out.
I went there for a meeting but was not allowed on the floor because it was not pre-approved by security.
If the Apple centers where Siri errors are reviewed is like that, it would be hard to sneak recordings out.
I have worked in similar locked-down environments. Phones, cameras, USB keys, and so on were all checked into lockers before entering the secure area, and this was despite the fact that the computers had all their ports physically glued shut, and there were digital locks and snooping going on as well.
At some level, you have to accept that things like Siri operate just like talking to human customer service representatives, who these days always begin conversations with a rote "This call may be recorded for quality and customer service purposes."
What it comes down to is whether you trust the company to be unreasonably diligent in controlling what happens to these recordings. If there is a criticism to be levelled here, I think it is around the question of how well Apple and everyone else in this space allow consumers to make informed choices.
Fine print in a voluminous user agreement is not nearly the same standard of disclosure as "Your words may be recorded for quality and customer service purposes" every time you say "Hey Siri." The latter may be impractical, of course, but perhaps there is a middle ground. It's not a dichotomy.
At some level, you have to accept that things like Siri operate just like talking to human customer service representatives, who these days always begin conversations with a rote "This call may be recorded for quality and customer service purposes."
What it comes down to is whether you trust the company to be unreasonably diligent in controlling what happens to these recordings. If there is a criticism to be levelled here, I think it is around the question of how well Apple and everyone else in this space allow consumers to make informed choices.
Fine print in a voluminous user agreement is not nearly the same standard of disclosure as "Your words may be recorded for quality and customer service purposes" every time you say "Hey Siri." The latter may be impractical, of course, but perhaps there is a middle ground. It's not a dichotomy.
> you have to accept that things like Siri operate just like talking to human customer service representatives, who these days always begin conversations with a rote "This call may be recorded for quality and customer service purposes."
Then why isn't it mandatory for Siri to start every conversation like that too?
Then why isn't it mandatory for Siri to start every conversation like that too?
It's a bit naive thinking that employees would store their electronics in the lockers. Even if you have spot searches (including cavity search) it still doesn't exclude rogue employee just "forgetting" to store his or hers camera in the locker.
Nobody in that business was naïve. Employees did not "forget," if they were caught with their phone in the secure area, they were fired. I was told right up front that certain things were zero tolerance, and everybody knew what they were.
They took this very seriously.
Now when it comes to a deliberately bad actor, well, nothing is 100% perfect, but there were many other security things going on that I am not going to describe here, plus I know for a fact that there were security measures they did not disclose to me.
But let's face it: Somebody, somewhere, can train themselves to memorize a screen full of information. They could memorize something, go for a smoke break, and upload what they memorized. Lather, rinse, repeat.
The point I made, and am still making, is that some companies care enough to do everything reasonably possible to keep customer data secure, while other companies do not. The company I described here cares. I believe Apple cares too.
I suspect it will always be possible for someone to pull a small data heist, but extraordinarily difficult to set up a regular pipeline to exfiltrate data. The weak point is probably the digital systems. Most attackers would want everything, and the way to get everything is with a vulnerability.
They took this very seriously.
Now when it comes to a deliberately bad actor, well, nothing is 100% perfect, but there were many other security things going on that I am not going to describe here, plus I know for a fact that there were security measures they did not disclose to me.
But let's face it: Somebody, somewhere, can train themselves to memorize a screen full of information. They could memorize something, go for a smoke break, and upload what they memorized. Lather, rinse, repeat.
The point I made, and am still making, is that some companies care enough to do everything reasonably possible to keep customer data secure, while other companies do not. The company I described here cares. I believe Apple cares too.
I suspect it will always be possible for someone to pull a small data heist, but extraordinarily difficult to set up a regular pipeline to exfiltrate data. The weak point is probably the digital systems. Most attackers would want everything, and the way to get everything is with a vulnerability.
User: "Hey Siri"
Siri: "We're listening"
Do they go through phased entry system. For example phase 1 metal detector arch and a 3d scanner like in the airport. Phase 2 strip naked for body and cavity search and then phase 3 - wear company uniform with attached camera and microphones?
If not then I can't see how they couldn't take a pendrive in.
While I'm sure they try their best, if you search online for "leaked security footage", you will quickly find things from prisons, embassies, and other locations that are obviously supposed to be high security.
We live in an age where everyone wanders around with a device for broadcasting video to the world in their pocket.
We live in an age where everyone wanders around with a device for broadcasting video to the world in their pocket.
All of that sounds impressive, until you see red team pros talk about how the whole apparatus melts before the gentlest of social engineering-
https://www.youtube.com/watch?v=JsVtHqICeKE
I'd also be concerned with leaking of mass amounts of records at once due to a security lapse on servers where it's a bit easier for grossly inadequate protections to lurk unnoticed-
https://news.ycombinator.com/item?id=19191241
I think I've read that Google doesn't even store the original recordings, but distorts them randomly to make them unrecognizable but still intelligible by their models. That seems like a pretty reasonable way to protect people, provided they're informed and there's no link back to the original account.
https://www.youtube.com/watch?v=JsVtHqICeKE
I'd also be concerned with leaking of mass amounts of records at once due to a security lapse on servers where it's a bit easier for grossly inadequate protections to lurk unnoticed-
https://news.ycombinator.com/item?id=19191241
I think I've read that Google doesn't even store the original recordings, but distorts them randomly to make them unrecognizable but still intelligible by their models. That seems like a pretty reasonable way to protect people, provided they're informed and there's no link back to the original account.
That cannot be the case when you can listen to a crystal clear record of every voice assistant clip you’ve made, in My Activity.
Wow, thanks for pointing that out. I didn't know that was a thing. It's not even encrypted with a device key or something on my phone - I can listen to a recording my own voice streaming straight from myactivity.google.com to my desktop web browser.
I guess the voice scrambling thing must have been specific to their human-supervised learning program. I expect the unaltered voice data is treated with some special care on Google's backend... but I'm still creeped out.
I guess the voice scrambling thing must have been specific to their human-supervised learning program. I expect the unaltered voice data is treated with some special care on Google's backend... but I'm still creeped out.
This is actually how this whole thing started, a contractor handed a bunch of recordings to a news outlet: https://www.vrt.be/vrtnws/en/2019/07/10/google-employees-are...
So it's not about how long until it happens. It's already happening.
So it's not about how long until it happens. It's already happening.
And what would you do with them if they leaked? It’s thousands of anonymized voices, basically meaningless unless it was some famous person and that person were identifiable. It’s like leaking a list of first names or a conversation recorded at some random bar that nobody has ever heard of with people nobody has met, nor cares about. Just noise. If they were tied to an identity, then it would be worrisome, but they aren’t.
And this is why I've got gear on the way to build a DIY replacement with mycroft & field array microphone & a raspberry. I'll probably throw a blog post on hn despite not being true show hn
I'm looking forward to it!
Look forward to that, something on my list of things to do. Though for the mic array, I want to try out contact microphones, make a whole house far-field array that way, might work, might not, but it will certainly be educational.
One aspect I've worked out is how to make a balanced contact mic using two piezo discs stuck back to back with a rigid layer in between acting as the pickup. Trick is that as the two discs are back to back, they effectively produce a balanced signal and that alone helps eliminate so much of the static/noise that contact mics due to impedance are susceptible too.
One aspect I've worked out is how to make a balanced contact mic using two piezo discs stuck back to back with a rigid layer in between acting as the pickup. Trick is that as the two discs are back to back, they effectively produce a balanced signal and that alone helps eliminate so much of the static/noise that contact mics due to impedance are susceptible too.
Well it's currently shipping so I'm doing this for better or worse
Mic array...pretty sure the tech I ordered (respeaker 6 mic) is towards the top end of DIY mic arrays so hopeful that I'll get good real life results.
Also very keen to work out whether I can use tensorflow to filter out baseline background noise BEFORE feeding the audio stream into mycroft.
Planning on launching a new blog to document my experiments and failures on this ( https://kaizen.today - not yet live cause still waiting for said tech...but SEO links and all that)
Mic array...pretty sure the tech I ordered (respeaker 6 mic) is towards the top end of DIY mic arrays so hopeful that I'll get good real life results.
Also very keen to work out whether I can use tensorflow to filter out baseline background noise BEFORE feeding the audio stream into mycroft.
Planning on launching a new blog to document my experiments and failures on this ( https://kaizen.today - not yet live cause still waiting for said tech...but SEO links and all that)
Yes, they have MEMS, looked at those a while back, was after something that I could add external mics as I need in my case and kinda ended up looking for a 5 mic input solution. Which last I looked entailed an external music input via USB unless I made something from scratch (would like to do that though wanted to work on the software and viability aspect of what I had in mind). But as that is one of my to do projects, I'll look into it again when time and resources prevail as things move fast in tech and might be something more suitable for my needs by the time I get around to this.
https://www.matrix.one/products/voice Just read about that today, no idea if better than the respeaker option - software/support/features..etc, but may be worth having a look in too.
https://www.matrix.one/products/voice Just read about that today, no idea if better than the respeaker option - software/support/features..etc, but may be worth having a look in too.
That board actually looks pretty good too!
To my untrained eye the specs seem superior (higher SNR & sensitivity).
Respeaker appealed to me due to it being fairly mainstream. I've learned not to stray too far from the pack on SBC things. Community support & info falls off a cliff
To my untrained eye the specs seem superior (higher SNR & sensitivity).
Respeaker appealed to me due to it being fairly mainstream. I've learned not to stray too far from the pack on SBC things. Community support & info falls off a cliff
I've used the same mic array and the provided software isn't great. In fact it's terrible, but ODAS[0] is amazing. It's super accurate, can track/isolate multiple audio sources, and has a web ui to visualize it. They also have config specifically to handle the respeaker and raspberry pi.
[0]: https://github.com/introlab/odas
[0]: https://github.com/introlab/odas
Amazing!
Thanks for the helpful link
Thanks for the helpful link
Doing offline voice recognition is pretty challenging on Raspberry Pi since it is relatively underpowered. Using pocketSphinx with RPi 3, it takes 2-3x realtime to do recognition, and the accuracy is underwhelming. Most DIY voice assistants handle voice recognition and speech synthesis with online services, defeating the privacy purpose.
If you would like to really take on Siri and Alexa, I would recommend using beefier hardware.
If you would like to really take on Siri and Alexa, I would recommend using beefier hardware.
>Doing offline voice recognition is pretty challenging on Raspberry Pi
Mycroft is hooked into some sort of mozilla cloud thing (deepspeech) I gather.
So plan is to roll a baseline and then see if I can local host a deep speech server
>I would recommend using beefier hardware
Plan is to ultimately deploy a gpu accelerated tensorflow integration so beefiness of hardware has been considered.
>RPi 3
4 cause true gigabit and usb 3
Mycroft is hooked into some sort of mozilla cloud thing (deepspeech) I gather.
So plan is to roll a baseline and then see if I can local host a deep speech server
>I would recommend using beefier hardware
Plan is to ultimately deploy a gpu accelerated tensorflow integration so beefiness of hardware has been considered.
>RPi 3
4 cause true gigabit and usb 3
anyone aware of low-power ASICs available to DIY to perform inference using pre-trained models?
Yup, Google's Coral and NVIDIA's Jetson Nano.
https://coral.withgoogle.com/products/
https://developer.nvidia.com/embedded/jetson-nano-developer-...
https://coral.withgoogle.com/products/
https://developer.nvidia.com/embedded/jetson-nano-developer-...
Don't think you need that much power for inference. Plus at siri levels you can probably push it to cloud if need be
Sure - you can argue that to improve speech recognition you’ll need to collect ground truth.
But only a tiny fraction of the billion users are tech savvy enough to realize this. Most people are kept in the dark. And we are trusting Apple to be benevolent to not do evil with these recordings.
But only a tiny fraction of the billion users are tech savvy enough to realize this. Most people are kept in the dark. And we are trusting Apple to be benevolent to not do evil with these recordings.
It is not surprising and it is how it works today but people don't fully understand it and can't make a truly infirmed decision.
We, the ones who truly understand both sides if this, must stand in favor of the ones who don't.
We, the ones who truly understand both sides if this, must stand in favor of the ones who don't.
“What happens on your iPhone stays on your iPhone.”
Had they just not doubled down and came out with this statement when the other companies were getting grilled for listening I think I’d have a better opinion of them over all this.
Had they just not doubled down and came out with this statement when the other companies were getting grilled for listening I think I’d have a better opinion of them over all this.
Just remember thousands of icloud nude leaked
I read TFA and still can’t tell - what exactly were the contractors doing?
We’re they grading Siri’s quality responding? Translating the voice to text to help Siri get smarter? Both? Something else?
We’re they grading Siri’s quality responding? Translating the voice to text to help Siri get smarter? Both? Something else?
"What happens in your iPhone, stays in your iPhone"
They should have some option if voice recognition fails like 'Would you like to send this diagnostic voice memo to apple to further improve the user experience ?' that way you'd have a choice if for some reason you were discussing confidential information, I'd send it in 99% of the time. Most user leave that diagnostic shit on by default.
It is just cheaper for tech companies to do this.
Even if unsupervised learning is not doable, they could hire people and pay them to record their voices for supervised learning.
It is just cheaper for them to do this.
It is just cheaper for them to do this.
@jankais3r (Twitter) shared an iOS profile to turn off logging of server-side Siri commands on GitHub: https://github.com/jankais3r/Siri-NoLoggingPLS
On your iOS device, visit the link above, download the “Prevent server-side logging of Siri commands.mobileconfig”
Switch to the Raw view, tap Allow, then download the profile. Complete the profile installation in Settings by reviewing it and tapping Install.
On your iOS device, visit the link above, download the “Prevent server-side logging of Siri commands.mobileconfig”
Switch to the Raw view, tap Allow, then download the profile. Complete the profile installation in Settings by reviewing it and tapping Install.
Every tech company does it as-long-as-they-can / it-goes-public.
[deleted]
Does it clearly say record so when using Siri in regards to GDPR?
Whenever you call a call center they usually says this call may be recorded.
Hidden wire tapping of conversations which some might legal wrangle to it is not legal in many countries. Aldo transfer of personal records to third parties is not legal. If someone speaks “I like political party x” and that is recorded that might be a registration of political opinion.
Whenever you call a call center they usually says this call may be recorded.
Hidden wire tapping of conversations which some might legal wrangle to it is not legal in many countries. Aldo transfer of personal records to third parties is not legal. If someone speaks “I like political party x” and that is recorded that might be a registration of political opinion.
If your data is being uploaded to a remote endpoint, it's gone.
Yeah, we have stuff like GDPR to mitigate the damage, fine people when bad stuff happens.
But ultimately it's _gone_. What actually happens to it is almost immaterial because most of the time, it won't be publicised, and you won't even know. But it's still there, on someone else's server.
Gone.
Yeah, we have stuff like GDPR to mitigate the damage, fine people when bad stuff happens.
But ultimately it's _gone_. What actually happens to it is almost immaterial because most of the time, it won't be publicised, and you won't even know. But it's still there, on someone else's server.
Gone.
Someday the general populace may come to recognize the true cost of demanding the "convenience" of smart voice assistants, automated photo-tagging, relevant product suggestions, etc. and maybe then we'll see outrage.
Or we're already at that point, and the general populace has chosen perceived convenience above all else?
Or we're already at that point, and the general populace has chosen perceived convenience above all else?
The cost will be enormous.
But I expect the outrage will be very similar to the outrage we see now towards industrialism:
Yes, the costs are devastating, and a small minority perceive that devastation acutely (including me). But the vast majority of people think: oh well, would you rather be starving on a farm in the dustbowl?
So I don’t expect any kind of major political unrest regarding privacy ever. Privacy Within Society was a blip in history. After that it was either/or and privacy was a thing you had to construct through separation.
But I expect the outrage will be very similar to the outrage we see now towards industrialism:
Yes, the costs are devastating, and a small minority perceive that devastation acutely (including me). But the vast majority of people think: oh well, would you rather be starving on a farm in the dustbowl?
So I don’t expect any kind of major political unrest regarding privacy ever. Privacy Within Society was a blip in history. After that it was either/or and privacy was a thing you had to construct through separation.
Was anyone’s privacy actually violated? The voices were anonymized and not tied to an identity or device. It’s like someone leaked random paragraphs of random emails with identifiers pulled out.
I see nothing wrong with smart voice assistants etc.
I only see a problem with it all controlled by few corporations and all the proprietary algorithms underneath closed up..
They know all about us and we almost nothing about them. .. or in a dystopian future, even "they" then do not know how all the proprietary libaries, training data and algorithms etc. of all the smart homes and self driving cars and autonomous killer robots anymore, and then some conflicting virus gets out of controll and ... Booom.
I only see a problem with it all controlled by few corporations and all the proprietary algorithms underneath closed up..
They know all about us and we almost nothing about them. .. or in a dystopian future, even "they" then do not know how all the proprietary libaries, training data and algorithms etc. of all the smart homes and self driving cars and autonomous killer robots anymore, and then some conflicting virus gets out of controll and ... Booom.
Someone needs to clarify what those costs actually are. Some people don't value privacy as much as others, and they aren't "wrong", they just have different values and priorities. Personally, I am quite fine having my smart voice assistant send my recordings to humans somewhere to evaluate to train the database. In fact I always assumed they were doing that.
For Apple, auto-photo identification happens on-device.
If you want to improve your voice recognition, then you need to have a ground truth. To continuously improve, you must validate, isolate and update the corner cases.
This means listening to the failed speech to text events, ones with low confidence, and a sampling of high confidence events.
This is literally what google voice was for, it was a way of google getting lots and lots of samples of people talking using a phone's microphone.
Almost certainly google/apple/amazon will be sampling photos, emails anything that goes through a ML pipeline, humans will see.
Now, ofcourse apple have positioned them selves as a privacy first company, so this is a big no no. They should have re-asked for permission (I'm pretty sure they talked about this in the Terms and conditions, there was some noise about it when siri first came out. )