DeepSeek R1: Open Weights, Hidden Bias(blog.getplum.ai)
blog.getplum.ai
DeepSeek R1: Open Weights, Hidden Bias
https://blog.getplum.ai/Deepseek-R1-Open-weights-Hidden-bias-18933807f8d08002a18ff42ba343a432
8 comments
The bias is baked into the open weights, namely happening on self-hosted 671B LLM??
Yes -- we observed this behavior on both the open-source open-weights 671B model as well as the DeepSeek web app.
Weird, because I got some deepseek feedback where it was openly critical and explicit about the authoritative regime of china. I really thought it was the "deepseek web app" only.
Then I have mixed signals about this.
Then I have mixed signals about this.
We're working on a follow-up post focused on our analysis of the open-source open-weight 671B model. What we're seeing is that questions related to the Chinese government produce an empty chain-of-thought followed by pro-Chinese-government talking points.
It is too late, I got mixed signals.
This is going to be very hard to trust anything about it anymore, unless running the 671B locally an my own systems.
This is going to be very hard to trust anything about it anymore, unless running the 671B locally an my own systems.
We ran the 671B locally and found a ton of bias. See part 2 of our analysis here: https://news.ycombinator.com/item?id=42918935
Happy to send you the dataset if you'd like! Please reach out to our email linked in the post.
Happy to send you the dataset if you'd like! Please reach out to our email linked in the post.
[deleted]
We evaluated DeepSeek R1 and confirmed that its guardrails deviate significantly from other model providers. We’re currently updating it to behave more in line with Anthropic and OpenAI’s models.