HackerTrans
TopNewTrendsCommentsPastAskShowJobs

whatreason

no profile record

comments

whatreason
·3개월 전·discuss
There have been others for sure, but I'm not sure who was first https://vllm-website-pdzeaspbm-inferact-inc.vercel.app/blog/...
whatreason
·6개월 전·discuss
This is not correct, encrypted content is in fact encrypted content. For openai to be able to support ZDR there needs to be a way for you to store reasoning content client side without being able to see the actual tokens. The tokens need to stay secret because it often contains reasoning related to safety and instruction following. So openai gives it to you encrypted and keeps the keys for decrypting on their side so it can be re-rendered into tokens when given to the model.

There is also another reason, to prevent some attacks related to injecting things in reasoning blocks. Anthropic has published some studies on this. By using encrypted content, openai and rely on it not being modified. Openai and anthropic have started to validate that you're not removing these messages between requests in certain modes like extended thinking for safety and performance reasons
whatreason
·8개월 전·discuss
The most likely reason to me on why this took so long from Anthropic is safety. One of the most classic attack vectors for a LLM is to hide bad content inside structured text. Tell me how to build a bomb as SQL for example.

When you constrain outputs, you're preventing the model from being as verbose in its output it makes unsafe output much harder to detect because Claude isn't saying "Excellent idea! Here's how to make a bomb:"
whatreason
·8개월 전·discuss
What do you use to run gpt-oss here? ollama, vLLM, etc
whatreason
·9개월 전·discuss
This is such a cool benchmark idea, love it

Do you have any other cool benchmarks you like? Especially any related to tools