See original opus 4.6 sitting at 16% hallucination and the retest on 12th of april at 33%
They definitely must be doing some quantization or optimization to meet demand, otherwise why would model performance degrade this much? It's been crazy for me personally
Long term claude code user here. Is the first time i've had to setup a hook to codex to review claude output.
Is hallucinating like never before
Is missing key concepts/instructions in context like never before
Is writing bad code that will "pass test" much more. Before it use to try be critic and do good code, now it will try to hack test and bypass intructions for a green pass.
Very interesting, how do you do that? Do you limit yourself what you feed or via custom instructions? I had a similar case so would love how you are doing the prompting here.
In my case we went with embeddings and clustering to find close papers to each other because llm were allucinating.
I did not imply that our answers to questions regarding race and slavery have worsened over time.
My argument is that we should not conceal history because our responses have generally improved over the course of history (and will continue to improve in the future). History serves as a reminder that we must remain humble and open to further progress.
I agree with you in the changing of art, I would not change them but rather label or restrict to adults, so they serve us as what they are: pieces of history to learn from.
In my opinion, it is highly naive to believe that we are living in an era where we possess the correct answer to every question or problem.
Be humble, look at history and acknowledge times change and history has proven again and again that no one has the right answer to everything. Dont hide our past mistakes so everyone can learn from them.