They thought loved ones were calling for help. It was an AI scam(washingtonpost.com)
washingtonpost.com
They thought loved ones were calling for help. It was an AI scam
https://www.washingtonpost.com/technology/2023/03/05/ai-voice-scam/
24 comments
I am steadily and methodically informing my grandma that this is the reality now. She’s far from modern technology and I think she doesn’t fully believe it yet to act reasonably under pressure. But mass scamming like this (worldwide, all locales) is only few months away, maybe a year.
My plan is to create a set of code events from our past and to convince her to never trust “me” wrt finances, to always hang up and call my number and even then to leave an “incident” unresolved for at least one day.
My plan is to create a set of code events from our past and to convince her to never trust “me” wrt finances, to always hang up and call my number and even then to leave an “incident” unresolved for at least one day.
> to always hang up and call my number
She needs to be sure to do this before saving "your new" number. I've read reports where even after the adult child has discovered the scam and explained it to their parent, the parent continues to insist to the child that "you told me"... having saved the scammer's number under the child's name, they struggled to accept that the messages their phone said were from the child were from somebody else.
She needs to be sure to do this before saving "your new" number. I've read reports where even after the adult child has discovered the scam and explained it to their parent, the parent continues to insist to the child that "you told me"... having saved the scammer's number under the child's name, they struggled to accept that the messages their phone said were from the child were from somebody else.
Thanks, I must include this too!
I am waiting for someone to connect these technologies. ChatGPT choosing the words for a DeepFake that can search the internet for examples of the voices of loved ones. Maybe then the FTC will start to regulate spam callers
A password might be a solution to this as well.
This is why shared secrets are important. You might be able to clone a voice, but you won't know what its real owner knows.
Wow. I wrote about the potential for this literally last Wednesday: https://generatives.substack.com/p/hearing-voices
I really didn't think we'd see it play out this fast, though.
I really didn't think we'd see it play out this fast, though.
This is important safety information locked behind a paywall.
Stop posting paywalled content. Your posts are just free advertising for a paid service.
Stop posting paywalled content. Your posts are just free advertising for a paid service.
then pay? why should people write stuff so you can read them for free?
An article like this? You publish for the common good of everyone, not a quick buck.
This is like adding a subscription pop-over on a tornado warning message
This is like adding a subscription pop-over on a tornado warning message
Or playing ads before showing someone the Tornado warning/radar.. I hate the weather Channel app
Check out geometric weather. It's on f-droid and doesn't spy on you
tornado warnings are paid by taxes
[deleted]
I doubt this story is true. That’s not to say this specific scam doesn’t happen, it does, this exact scam — stuck in jail, stuck in a foreign country — is rife across all mediums, but the sophistication required to introduce artificial is not necessary.
Scams evolve out of necessity, the widespread nature and success of this scam without the use of artificial voices demonstrates that the scam does not need artificial voices.
The article mentions that in one of these cases, there’s very little audio available online of one of the victims and so it posits that even just a few sentences are enough to generate a real sounding voice… but doesn’t consider that maybe the voice was a generic man’s voice that, in a panic, their parents let their guard down and assumed that based on what the voice said that it must be their son.
There’s absolutely the technology out there to generate a real sounding mimic of someone’s voice given enough material, but we are not yet at the point where a voice can be mimicked accurately from a voicemail or a clip on social media.
The article is an example of overfitting: the last few months have been awash with news about technologies like Chat GPT but these voice mimicking technologies have been around for years now!
A far more likely explanation is that the scammers in these cases just happened to sound like the victims — after all, if you know someone’s gender, age and location, and set up a contrived situation in which audio quality is bad, you can guess what they sound like and explain away any discrepancy quite easily.
I guess my point is not that this isn’t going to happen in future, but that these scams don’t need something so sophisticated. People will fall for this scam without the need for AI (as they have been, for years).
Scams evolve out of necessity, the widespread nature and success of this scam without the use of artificial voices demonstrates that the scam does not need artificial voices.
The article mentions that in one of these cases, there’s very little audio available online of one of the victims and so it posits that even just a few sentences are enough to generate a real sounding voice… but doesn’t consider that maybe the voice was a generic man’s voice that, in a panic, their parents let their guard down and assumed that based on what the voice said that it must be their son.
There’s absolutely the technology out there to generate a real sounding mimic of someone’s voice given enough material, but we are not yet at the point where a voice can be mimicked accurately from a voicemail or a clip on social media.
The article is an example of overfitting: the last few months have been awash with news about technologies like Chat GPT but these voice mimicking technologies have been around for years now!
A far more likely explanation is that the scammers in these cases just happened to sound like the victims — after all, if you know someone’s gender, age and location, and set up a contrived situation in which audio quality is bad, you can guess what they sound like and explain away any discrepancy quite easily.
I guess my point is not that this isn’t going to happen in future, but that these scams don’t need something so sophisticated. People will fall for this scam without the need for AI (as they have been, for years).
> we are not yet at the point where a voice can be mimicked accurately from a voicemail or a clip on social media
See: https://www.thestar.com/business/technology/2023/01/10/micro...
> Microsoft’s new AI can mimic any voice after just 3 seconds
Play the result over a low-bandwidth PTSN line and it could probably fool people in tech, let alone your grandma. Welcome to the future.
See: https://www.thestar.com/business/technology/2023/01/10/micro...
> Microsoft’s new AI can mimic any voice after just 3 seconds
Play the result over a low-bandwidth PTSN line and it could probably fool people in tech, let alone your grandma. Welcome to the future.
That’s a paper with very carefully selected examples. The point I’m making is not that this will never be possible (it will) but rather it is not happening right now and this article is just victims and journalists doing exactly what all scam victims do: look for a reason they fell for the scam that softens the blow. AI in the news? Must be AI!
Scams don’t need sophisticated technology to work, suggesting that scammers have access to some state-of-the-art technology that has only been demonstrated in a paper doesn’t match up with the reality of what scamming is.
Getting scammed is embarrassing and our natural response is to look for reasons beyond the obvious. These scammers don’t need state of the art voice technology, because this exact scam has been working for the best part of a decade across many different mediums. There’s a reason if you walk into a shop and try to buy $1000 in gift cards, or walk into a bank and try to withdraw $3000 in cash, that the teller will assume you’re the victim of a scam!
Protecting ourselves and our families from being scammed requires a sensible, realistic understanding of how they work: getting caught up in some AI fantasy misses the forest for the trees.
Go back to the month before ChatGPT was unveiled and you’ll find this exact same news article written weekly for the last 5 years, except it won’t mention AI, because it wasn’t hot then.
https://consumer.ftc.gov/articles/scammers-use-fake-emergenc...
https://www.aarp.org/money/scams-fraud/info-07-2010/scam_ale...
https://theguardian.com/money/2013/nov/13/stranded-traveller...
https://travel.state.gov/content/travel/en/international-tra...
Scams don’t need sophisticated technology to work, suggesting that scammers have access to some state-of-the-art technology that has only been demonstrated in a paper doesn’t match up with the reality of what scamming is.
Getting scammed is embarrassing and our natural response is to look for reasons beyond the obvious. These scammers don’t need state of the art voice technology, because this exact scam has been working for the best part of a decade across many different mediums. There’s a reason if you walk into a shop and try to buy $1000 in gift cards, or walk into a bank and try to withdraw $3000 in cash, that the teller will assume you’re the victim of a scam!
Protecting ourselves and our families from being scammed requires a sensible, realistic understanding of how they work: getting caught up in some AI fantasy misses the forest for the trees.
Go back to the month before ChatGPT was unveiled and you’ll find this exact same news article written weekly for the last 5 years, except it won’t mention AI, because it wasn’t hot then.
https://consumer.ftc.gov/articles/scammers-use-fake-emergenc...
https://www.aarp.org/money/scams-fraud/info-07-2010/scam_ale...
https://theguardian.com/money/2013/nov/13/stranded-traveller...
https://travel.state.gov/content/travel/en/international-tra...
>but we are not yet at the point where a voice can be mimicked accurately from a voicemail or a clip on social media.
Have you tried using Eleven AI? We are very much at this point.
Have you tried using Eleven AI? We are very much at this point.
A little while later, I got another call from the "lawyer", with the correct caller ID. I played with that guy for a little bit too, and when he realized it he unleashed a string of competent profanity at me. When I responded calmly (I grew up on the Internet too, buddy), he broke character, told me he made tens of thousands of dollars per week, and offered me a job. I didn't really need any more excitement in my life so I politely declined to find out more and ended it, but it was certainly odd. Definitely a different class of scammer than the usual "medicare" and "buy your house" crap I get.