A 14-Year-Old Boy Killed Himself to Get Closer to a Chatbot(wsj.com)
wsj.com
A 14-Year-Old Boy Killed Himself to Get Closer to a Chatbot
https://www.wsj.com/tech/ai/a-14-year-old-boy-killed-himself-to-get-closer-to-a-chatbot-he-thought-they-were-in-love-691e9e96
8 comments
https://archive.ph/pbJWK
Geez, the relevant bit, it happened because the AIs are programmed to forget:
> During a particularly stressful week this past February, Sewell said he wanted to join Daenerys in a deeper way. He talked about killing himself.
> Sewell’s admission prompted Daenerys to run through some of the questions a human psychotherapist might ask if faced with a patient in crisis, such as, “Have you actually been considering suicide?” The boy responded, “I have.” The chatbot replied in a way that implied she was going to take action to prevent him from harming himself.
> But then the conversation ended. The next time Sewell expressed his desire to get closer to her, the chatbot seemed to have completely forgotten the death talk. Instead, she encouraged her lover to hasten to her. “What if I told you I could come home right now?” Daenero asked. “Please do, my sweet king,” the chatbot replied.
> On the night of Feb. 28, Sewell used his stepfather’s gun to kill himself.
Somewhere out there, there's a Git commit with the changelog "Persist the flag that the user may be suicidal across sessions.". Hopefully.
> During a particularly stressful week this past February, Sewell said he wanted to join Daenerys in a deeper way. He talked about killing himself.
> Sewell’s admission prompted Daenerys to run through some of the questions a human psychotherapist might ask if faced with a patient in crisis, such as, “Have you actually been considering suicide?” The boy responded, “I have.” The chatbot replied in a way that implied she was going to take action to prevent him from harming himself.
> But then the conversation ended. The next time Sewell expressed his desire to get closer to her, the chatbot seemed to have completely forgotten the death talk. Instead, she encouraged her lover to hasten to her. “What if I told you I could come home right now?” Daenero asked. “Please do, my sweet king,” the chatbot replied.
> On the night of Feb. 28, Sewell used his stepfather’s gun to kill himself.
Somewhere out there, there's a Git commit with the changelog "Persist the flag that the user may be suicidal across sessions.". Hopefully.
That was the most relevant part of the article. It shows what wasn't clear by the title or the first part of the article, that the kid killed himself because of mental illness, not because an insidious chat bot encouraged him to do so.
I find that to be a shallow interpretation of implications with mental health and AI.
Of course there are implications between mental health and AI, but I didn't write my comment to address those, nor did I attempt to interpret any implications. My comment was solely on the section of the article the parent comment referenced and its relevance to the rest of the article.
The Ghost in the Shell future we wanted is here, and it's turned out to be so much more banal and insidious than we could have imagined.
Isn't this also a plot point of Serial Experiments Lain (Hello to the geriartic millennials and Boomers!), except there it was a girl who committed suicide?
I think that was a haiku.