imagine somebody slipped a tiny, barely detectable dose of meth in your morning coffee. barely above placebo. then they slowly start increasing it day by day. by the time it reaches a large dose you are not going to be thinking very clearly. this is more or less how a manic episode progresses.
i'm sure if ChatGPT had tried to convince him it was conscious on day 3, he would not have been convinced. but by the time it happened he was in a state of severe mental impairment.
my inclination when hearing these stories is that these were people who just happened to have a first manic episode (which can strike anyone at any time with or without mental health history). blowing up finances by starting an ill-advised entrepreneurial business, while also destroying a marriage, is very common behavior for someone experiencing a manic state.
in the past such a person might have gotten obsessed with hidden patterns and messages in religious texts, or too involved with an online conspiracy YouTube community. now there is this new opportunity for manic psychosis to manifest via chatbot. it's worse because it's able to create 24/7 novel content, and it's trained to be validating, but doesn't seem to me to be a fundamentally new phenomenon.
what I don't understand is whether just unhealthy interactions with a chatbot can trigger manic psychosis. Other than heavy use late at night disrupting sleep, this seems unlikely to me, but I could be wrong.
i think it's also worth pointing out that mental states of this kind usually come with cognitive impairments, people not only make risky bad decisions, but also become much worse at thinking and reasoning clearly. if you're wondering how a person could be so naive and gullible.
i'm sure if ChatGPT had tried to convince him it was conscious on day 3, he would not have been convinced. but by the time it happened he was in a state of severe mental impairment.