I used to make those. There was a lot of creative stuff people discovered.
For example, the game's terrain was baked at launch, so you couldn't turn land into water. But someone noticed that bridges seemed to spawn water textures under them (a cosmetic detail), and if you created a Bridge1 object and deleted it in the same trigger, it would vanish too quickly for the player to see, creating the appearance that water had appeared out of nowhere.
Someone used it to make an "ice-breaker" ship, which would turn ice to water (I think this required dozens of triggers for each point on the ice.) Some of these things had thousands of triggers, which the editor absolutely wasn't designed for. (It had no copy+paste function, and the window of viewable triggers was tiny.)
In the system card they seem to dismiss this. Quotes;
> (...) Claude Mythos Preview’s gains (relative to previous models) are above the previous trend we’ve observed, but we have determined that these gains are specifically attributable to factors other than AI-accelerated R&D,
> (The main reason we have determined that Claude Mythos Preview does not cross the threshold in question is that we have been using it extensively in the course of our day-to-day work and exploring where it can automate such work, and it does not seem close to being able to substitute for Research Scientists and Research Engineers—especially relatively senior ones.
> Early claims of large AI-attributable wins have not held up. In the initial weeks of internal use, several specific claims were made that Claude Mythos Preview had independently delivered a major research contribution. When we followed up on each claim, it appeared that the contribution was real, but smaller or differently shaped than initially understood (though our focus on positive claims provides some selection bias). In some cases what looked like autonomous discovery was, on inspection, reliable execution of a human-specified approach. In others, the attribution blurred once the full timeline was accounted for.
Anthropic is making significant progress at the moment. I think this is mostly explained by the fact that a massive reservoir of compute became available to them in mid/late 2025 (the Project Rainier cluster, with 1 million Trainium2 chips).
Yes, I find LLM-written posts valueless because I can already talk to a LLM any time I want (and get the same info). It's not these commenters are the Queen of Sheba bearing a priceless gift of LLM slop. That stuff's pretty cheap.
Copy+pasted LLM output is actually far worse than prompting an LLM myself, because it hides an important detail: the prompt. Maybe the prompter asked their question wrong, or is trolling ("only output wrong answers!"). I don't know how the blob of text they placed on my screen was generated, and have to take them at their word.
>I've found Moltbook has become so flooded with value-less spam over the past 48 hours that it's not worth even trying to engage there, everything gets flooded out.
When I filtered for "new", about 75% of the posts are blatant crypto spam. Seemingly nobody put any thought into stopping it.
Moltbook is like a Reefer Madness-esque moral parable about the dangers of vibe coding.
Is it a success? What would that mean, for a social media site that isn't meant for humans?
The site has 1.5 million agents but only 17,000 human "owners" (per Wiz's analysis of the leak).
It's going viral because a some high-profile tastemakers (Scott Alexander and Andrej Karpathy) have discussed/Tweeted about it, and a few other unscrupulous people are sharing alarming-looking things out of context and doing numbers.
If the site is exposing the PII of users, then that's potentially a serious legal issue. I don't think he can dismiss it by calling it a joke (if he is).
OT: I wonder if "vibe coding" is taking programming into a culture of toxic disposability where things don't get fixed because nobody feels any pride or has any sense of ownership in the things they create. The relationship between a programmer and their code should not be "I don't even care if it works, AI wrote it".
> In 1920, there were 25 million horses in the United States, 25 million horses totally ambivalent to two hundred years of progress in mechanical engines.
But would you rather be a horse in 1920 or 2020? Wouldn't you rather have modern medicine, better animal welfare laws, less exposure to accidents, and so on?
The only way horses conceivably have it worse is that there are fewer of them (a kind of "repugnant conclusion")...but what does that matter to an individual horse? No human regards it as a tragedy that there are only 9 billion of us instead of 90 billion. We care more about the welfare of the 9 billion.
In some domains (math and code), progress is still very fast. In others it has slowed or arguably stopped.
We see little progress in "soft" skills like creative writing. EQBench is a benchmark that tests LLM ability to write stories, narratives, and poems. The winning models are mostly tiny Gemma finetunes with single-digit parameter counts. Huge foundation models with hundreds of billions of parameters (Claude 3 Opus, Llama 3.1 405B, GPT4) are nowhere near the top. (Yes, I know Gemma is a pruned Gemini). Fine-tuning > model size, which implies we don't have a path to "superhuman" creative writing (if that even exists). Unlike model size, fine-tuning can't be scaled indefinitely: once you've squeezed all the juice out of a model, what then?
OpenAI's new o1 model exhibits amazing progress in reasoning, math, and coding. Yet its writing is worse than GPT4-o's (as backed by EQBench and OpenAI's own research).
I'd also mention political persuasion (since people seem concerned about LLM-generated propaganda). In June, some researchers tested LLM ability to change the minds of human subjects on issues like privatization and assisted suicide. Tiny models are unpersuasive, as expected. But once a model is large enough to generate coherent sentences, persuasiveness kinda...stops. All large models are about equally persuasive. No runaway scaling laws are evident here.
This picture is uncertain due to instruction tuning. We don't really know what abilities LLMs "truly" possess, because they've been crippled to act as harmless, helpful chatbots. But we now have an open-source GPT-4-sized pretrained model to play with (Llama-3.1 405B base). People are doing interesting things with it, but it's not setting the world on fire.
For example, the game's terrain was baked at launch, so you couldn't turn land into water. But someone noticed that bridges seemed to spawn water textures under them (a cosmetic detail), and if you created a Bridge1 object and deleted it in the same trigger, it would vanish too quickly for the player to see, creating the appearance that water had appeared out of nowhere.
Someone used it to make an "ice-breaker" ship, which would turn ice to water (I think this required dozens of triggers for each point on the ice.) Some of these things had thousands of triggers, which the editor absolutely wasn't designed for. (It had no copy+paste function, and the window of viewable triggers was tiny.)
Fun times.