Does GPT-4 Know Me Better Than My Girlfriend?(every.to)

3 pointsby dshipper3 lata temu4 comments

Does GPT-4 Know Me Better Than My Girlfriend?

https://every.to/chain-of-thought/does-gpt-4-know-me-better-than-my-girlfriend

4 comments

ftxbro3 lata temu

> "I decided to test GPT-4 as follows: I’d take a personality test. Then, I’d give GPT-4 some information about me and ask it to take the same personality test—but pretend to be me. I’d tell it to attempt to predict the ways I’d answer the questions based on what it knows. Then I’d measure the difference between what GPT-4 predicted my responses would be and my actual responses to see how accurate it was. I’d also ask my girlfriend, Julia, who is unusually perceptive about people and knows me quite well, to do the same task. Then I’d compare GPT-4’s accuracy against hers."

I wonder if he gave his girlfriend the information that he "give GPT-4 some information about me". Maybe if the girlfriend saw what he was emphasizing in that information, she could have better guessed what he would have put on the test. Another possibility is that he asked his girlfriend what she thought he would actually do in those situations, whereas he asked the bot to guess what it thought he would put as answers to hypothetical questions (which wouldn't necessarily match what he would actually do in those situations).

dshipper3 lata temu

To your latter point I did actually two tests with my girlfriend: I asked her to respond "as I would" rather than with what she thought the truth was, and then did another test where I just asked her to say what she thought. She improved when I told her to predict what she thought I would say—and that's the data I used in the article. But she was still significantly worse than the best GPT model

Re the information: I don't think so. The version of GPT that had access to only my most recent tweets performed better than she did, and she reads all of my tweets. In general, I think she has much higher fidelity data on me as an individual than GPT does.

ftxbro3 lata temu

Thanks for those clarifications! I think your article is very good, one of the top ones for this kind of anecdotal AI experimentation.

willdough3 lata temu

This is unsurprising. It was claimed that the algorithms developed by Cambridge Analytica and FaceBook during the Brexit campaign and the American 2016 election were able to predict behavior better than could friends or spouse:

https://www.amazon.com/Mindf-Cambridge-Analytica-Break-Ameri...