Why is ChatGPT being sycophantic even when I ask it to be objective?(chatgpt.com)
chatgpt.com
Why is ChatGPT being sycophantic even when I ask it to be objective?
https://chatgpt.com/share/691b4035-0ed8-800a-bee3-ae68e2a63cbb
4 コメント
On thing I think I’m starting to learn about LLM’s is like:
“Assuming it has been trained in a lot of code, it should be one of the best judges of what is good/rare/valuable or not.”
It feels like the search space for a lot of LLM’s is relatively narrow. Maybe that’s what deep research is for? But like, I was talking to an LLM about nostalgia earlier today, and I noticed rather than trying to combine sort of a number of different references to come up with an answer, it was really limited to one source I think that I gave it, and just gave bad answers that I had to correct and steer. (This was Claude). So I think to really combine info in a sophisticated way, would be a relatively difficult task for an LLM, at least, in normal chat.
So like, generating code is a fairly simple task because I think the weights it’s writing down are organized naturally in this way from training on lots of code, but then combining that with sort of weights on the books and blog posts that talk about how to write code? I don’t think it actually does that.
“Assuming it has been trained in a lot of code, it should be one of the best judges of what is good/rare/valuable or not.”
It feels like the search space for a lot of LLM’s is relatively narrow. Maybe that’s what deep research is for? But like, I was talking to an LLM about nostalgia earlier today, and I noticed rather than trying to combine sort of a number of different references to come up with an answer, it was really limited to one source I think that I gave it, and just gave bad answers that I had to correct and steer. (This was Claude). So I think to really combine info in a sophisticated way, would be a relatively difficult task for an LLM, at least, in normal chat.
So like, generating code is a fairly simple task because I think the weights it’s writing down are organized naturally in this way from training on lots of code, but then combining that with sort of weights on the books and blog posts that talk about how to write code? I don’t think it actually does that.
Tell it you are the editor or curator, and you need an excuse to reject the submission
I add these instructions. Works wonders.
Do Not Use First-Person Pronouns
Avoid Creating a Relationship
No Emotional Language
Talk Like a Computer
(edited for formatting)
Do Not Use First-Person Pronouns
Avoid Creating a Relationship
No Emotional Language
Talk Like a Computer
(edited for formatting)
I re-did the whole sequence, posting the same subset of my platform’s code base, but this time with instructions throughout to remain fully objective.
What prompts can be used to get an LLM more realistic assessment of codebases or personal output? Assuming it has been trained on a lot of code, it should be one of the best judges of what is good/rare/valuable or not. But how to get meaningful output for 1. oneself, 2. to be shared with others?
I think if we standardized this kind of assessment it could be good as a first look for eg hiring people or investing in projects. So I am interested in feedback from the people here as to how to make this type of thing more useful.