The "climate" cares little about carbon intensity of labor or dollars. It cares about absolute tons of green house gases in the atmostphere and if you use more to produce more you still use more.
What prevents a bad actor from posting "easy" problems to the board, solving them and getting quick reputation. Then as a validator they can easily validate their own malicious changes to someones software?
I always wondered how these models would reason correctly. I suppose they are diffusing fixed blocks of text for every step and after the first block comes the next and so on (that is how it looks in the chat interface anyways). But what happens if at the end of the first block it would need information about reasoning at the beginning of the first block? Autoregressive Models can use these tokens to refine the reasoning but I guess that Diffusion Models can only adjust their path after every block? Is there a way maybe to have dynamic block length?
Does anyone know what kind of RL environments they are talking about? They mention they used 15k environments. I can think of a couple hundred maybe that make sense to me, but what is filling that large number?
With the rough numbers from the blog post at ~1k tokens a second in Cerebras it should put it right at the same size as GLM 4.7, which also is available at 1k tokens a second. And they say that it is a smaller model than the normal Codex model
I think this says a lot about the business approach of Anthopic compared to OpenAI. Just the vast amount of free messages you get from OpenAI is crazy that turning a profit with that seems impossible. Anthropic is growing more slowly but it seems like they are not running a crazy deficit. They do not need to put ads or porn in their chatbot
I feel like this comes from the rigorous Reinforcement Learning these models go through now. The token distribution is becoming so narrow, so the models give better answers more often that is stuffles their creativity and ability to break out of the harness. To me, every creative prompt I give them turns into kind of the same mush as output. It is rarely interesting
They trained it in 33 days for ~20m (that includes apparently not only the infrastructure but also the salaries over a 6 month period). And the model is coming close to QWEN and Deepseek. Pretty impressive
You cannot negotiate with a bully. The EU should have never backed down so easily before. I hope someone will soon find some balls and not let the US walk all over everyone
Is it actually possible that nvidia chips will have 50TB/s bandwidth by 2028? Right now it shows they are at 8 TB/s. To me it seems the Nvidia forecast is a very, very optimistic exponential. Nontheless, Huawei not matching the scale of production seems to be the biggest hurdle
To me, this only makes sense on a word level not sentence level. I can understand that words, especially those that are older, have evolved because of energy and comfort constraints of our physiology. But to extend this to sentence level is a rather big step. I would suppose it works for simple, short sentences that had to be efficient in the past. But imagine sentences about computer science where most words are rather new and have been chosen by arbitrary rules.
To me it would be interesting to see, if this hypothesis holds when applied to longer and more complex sentences and "modern" words.
That is partly true. High population density means a lot of roof area. Solar is perfect to put on roofs, you need no extra land. It is basically free (save the investment of the panels which pay off quickly nowadays)
If we think of every generation as a compression step of some form of information into our DNA and early humans existed for ~1.000.000 years and a generation is happening ~20years on average, then we have only ~50.000 compression steps to today. Of course, we have genes from both parents so they is some overlap from others, but especially in the early days the pool of other humans was small. So that still does not look like it is on the order of magnitude anywhere close to modern machine learning. Sure, early humans had already a lot of information in their DNA but still
".. Claude estimates that AI reduces task completion time by 80%. We use Claude to evaluate anonymized Claude.ai transcripts to estimate the productivity impact of AI."
What is this? So they take Claude and ask how much do you think you saved on time here? How can you take this seriously. ChatBots are easy to exaggerate, especially about something positive like this.