Makes me think of the Star Trek episode “A Taste of Armageddon” where a computer simulates the outcome of a war and its causalities, and then those civilizations actually go to murder that number of people in real life.
As it’s been noted, these bots don’t comprehend what they’re saying. But I thought ChatGPT saying ”How can I assist you today?” and “I'd be happy to help with any questions or information you may need.” in the beginning of the conversation really reinforced this. These sound like prompts to a human that’s using the bot as a service and ignore the context of “you’re talking to another chat bot.” You wouldn’t say that if you’re meeting/learning about someone.