HackerTrans
TopNewTrendsCommentsPastAskShowJobs

CBiddulph

no profile record

comments

CBiddulph
·السنة الماضية·discuss
I checked Figure 26 - the way it's presented is a bit confusing, but the model prompt doesn't include the expected output. All the model sees is "Here is the function f, the input provided 'cookie', and we need to predict the output." plus the code. "Input:" and "Output:" are shown for the benefit of the human reader.

The CoT does seem pretty nonsensical. It might be an instance of vestigial reasoning: https://www.lesswrong.com/posts/6AxCwm334ab9kDsQ5/vestigial-... (not to promote my own blog post)

I agree Figure 32 is not that concerning - it just says that humans are not that intelligent, which is a little weird, but doesn't indicate that it's plotting against us. It's actually good that we can see this somewhat questionable behavior, rather than it being quashed by process supervision - see https://openai.com/index/chain-of-thought-monitoring/