Hi there, Thariq from the Claude team here. Sorry this is happening, we'll fix it ASAP.
We don't want anyone to feel locked into the tool. Claude's designs are HTML/CSS/JS that any editor can handle; we'll make sure it's possible to download them even after you unsubscribe.
We've been on this since the bug surfaced. Everyone affected is getting a full refund and an extra grant of usage credits equal to their monthly subscription as our apology. You can see my original post here: https://x.com/trq212/status/2048495545375990245. We’re still working on sending emails to everyone affected.
Our support flow wasn't set up to route a complex bug like this to engineering. We’re hoping to make this better but will take some time. Sorry to everyone caught up in it.
Sorry to hear, was wondering if you could find a session where this happens and hit /feedback and just say something like stop hook not firing and we'll take a look.
Yes, we do but harnesses are hard to eval, people use them across a huge variety of tasks and sometimes different behaviors tradeoff against each other. We have added some evals to catch this one in particular.
We haven't yet found generalizable "make this model smarter" features, but there is a tradeoff of putting instructions in system prompts, e.g. if you have a chatbot that sometimes generates code, you can give it very specific instructions when it's coding and leave those out of the system prompt otherwise.
I want to build intuition on this by building a logit visualizer for OpenAI outputs. But from what I've seen so far, you can often trace down a hallucination.
I mean, LLMs certainly know representations of what words means and their relationship to each other, that's what the Key and Query matrices hold for example.
But in this case, it means that the underlying point in embedding space doesn't map clearly to only one specific token. That's not too different from when you have an idea in your head but can't think of the word.
Definitely, but if you can detect when you might be in one of those states, you could reflect to see exactly which state you're in.
So far this has mostly been done using Reinforcement Learning, but catching it and doing it inference seems like it could be interesting to explore. And much more approachable for open source, only the big ML labs can do this sort of RL.