Each token the model outputs requires it to evaluate all of the context it already has (query + existing output). By allowing it more tokens to "reason", you're allowing it to evaluate the context many times over, similar to how a person might turn a problem over in their heads before coming up with an answer. Given the performance of reasoning models on complex tasks, I'm of the opinion that the "more tokens with reasoning prompting" approach is at least a decent model of the process that humans would go through to "reason".
This is a great article -- I really appreciate the author giving specific examples. I have never heard of mise (https://mise.jdx.dev/) before either, and the integration with the saved prompts is a nifty idea -- excited to try it out!
Yeah, as the other commentator mentioned, it practically just means he won't be entering any other competitions until April. He'll probably cover >1000 miles in training over that period :)
Zitron is a blogger whose content/internet personality is centered around being anti-Big Tech, and very much falls in the "AI is dumb/useless and will die any day now".
He's a good writer, but his content is written through an extreme anti-AI lens, so take it with a pretty big grain of salt.
> AGI is a weakly defined term, but generally speaking we mean it to be a system that can tackle increasingly complex problems, at human level, in many fields.
Is it just me, or is that an incredibly weak/vague definition for AGI? Feels like you could make the claim that AI is at this level already if you stretch the terms he used enough.
They appear to basically RLHF a model on a bunch of examples of human/AI output on the same prompt. Not sure how well it works, but I'm guessing Mozilla is doing something similar here.
Everything comes back to social identity theory :) Especially programmers, who are already partially drawn to pattern matching.
I still enjoy articles like these, though -- think the author did a good job of describing the hats as "modes of working that you switch between" instead of "all-encompassing static personality traits that put you in a box".
Yek is fantastic -- I've converted my whole team to using it for prompting. As input context windows keep increasing, I think it'll just keep becoming more and more valuable -- I can put most of my team's code in Gemini 2.0 now.
https://en.wikipedia.org/wiki/2024_California_Proposition_6