I’ve never played RuneScape before but this was very cool, it wrote lots of scripts as it went and eventually finished a quest to gain the ability to make runes
This was so fun to make with you Morry, amazing job and game vision bringing it through to fruition. Excellent video, and the game is lot of fun to play while making a statement. Highly recommend everyone play for a few minutes! https://doge-decimator--wttdotm.on.websim.ai/
Multiple parts in the workflow stuck out to me:
- "Pair programming" felt a lot more like jamming on ideas than anything
- Copy-pasted real government data into ChatGPT and generated JSON that would be the backbone of the game. Just gave websim that file and copy-pasted the schema, and that was enough to make the bare bones of the project.
- Morry's prompts were super detailed and specific
- When things weren't working, reverting to an earlier version where they did work and writing a different prompt often worked.
- Prompted Claude with a screenshot of the DOGE website, which totally transformed the visual style of Doge Decimator.
- The hardest part of vibe coding a game is to actually think big picture about what the game loop is. Most people vibe code toys and playgrounds, not games.
very exciting! Looking forward to the SOTA for embedding models getting pushed.
A little unclear how important embeddings models will be vs. new methods of semantic search in the coming years (e.g. super fast llm as judge, or emerging methods) but still exciting!
Cofounder of Websim here. Right now it's not clear that there's any eval for a language model's simulation capabilities. Internally, we've (vibe) tested Llama 3, Command R+, WizardLM 8x22b, Mistral Large (first version of Websim came out of a Mistral hackathon) and GPT-4 Turbo and found them all lacking, due to either meh website outputs or mode collapse from reinforcement learning (lack of creativity and flexibility). That also may be a "skill issue" thing because our system prompt is very much optimized for Claude 3's "mind." We'll release functionality in the next week or two that lets users update the system prompt, in which case this may be less of an issue
Claude 3 has a much broader latent space, and seems to "enjoy" imagining things. It hasn’t been banged into too specific of an assistant shape, and doesn’t suffer the same degree of “mode collapse” https://lesswrong.com/posts/t9svvNPNmFf5Qa3TA/mysteries-of-m...
Even Sonnet produces mindblowingly good outputs (https://x.com/RobertHaisfield/status/1774579381132050696). Haiku is capable of producing full websites with insightful and creative content, even if it isn't as capable as Sonnet/Opus. For example, I found Curio, an esolang where every line of code is a living, sentient being with its own unique personality, memories, and goals, mostly by browsing around with Haiku (https://x.com/RobertHaisfield/status/1782586807261233620). Although Haiku tends to perform better when it is few-shot prompted with outputs from Sonnet or Opus earlier in the "browser history."