I think all good apps should strive to achieve a flow-state, and Rosebud is no exception; we aspire to achieve the same. The AI part is to solve a problem rather be a feature. The kind of users we want to serve, and what they want, is to make games. The chat interface is a way to create a flexible interface to do that, not force AI into the solution. "AI build me ___" is only addressing simple asset generation or addition of game features, which is a very limited use of AI to solve the bigger problem we are focused on, which is to decrease the barriers to entry to making games.
There is also a useful distinction between generative AI features of gameplay versus AI in aiding the creator (game dev) to make games. We do both, but they are driven by different user needs. For instance, for the former, there are AI powered characters which gives players a sense of spontaneity in the gameplay.
Thanks for stopping by! That was a fun video to be a part of, Grant’s math videos in general are so well done. I think this video was from 2017. Funny thing is that even back in 2014/2015, which is when I officially changed my PhD from pure math to AI, I had the fear that it was a little late (was originally from Toronto where neural nets were worked on my Hinton and his students —-Ilya, James…etc….so I felt like I had my chance to join much earlier and missed it). Only after 2012 did we start to see super human performance of AI in some tasks (namely, at first, classification). It was the Lee Sedol match with AlphaGo that convinced me that the field became a lot more interesting after I got to Berkeley (and also the generative stuff coming out early days from Alyosha’s lab and Deep RL stuff from Pieter’s).
But look at the world now! It’s never too late, jump in and build if you are excited now!
Thanks for summarizing these and spending a few hours trying out Rosebud!! Some detailed responses below:
"I found getting assets into the game a bit tricky. Sometimes it would try updating the code to reference assets that we hadn't created yet. After that I couldn't get it to generate assets for me."
This is an known issue we are trying to solve. But to give more color on why it's been tricky, since we are relying on a chat interface, our agent has properly determine is your ask includes an ask for asset generation, and if so, it has to determine whether an asset exists, and if it does not, whether to generate an asset, after which it needs to determine where to insert it and whether to modify other parts of the code when inserting. For instance, in the character templates, the AI can successfully modify an asset and character's description in one prompt, but sometimes it doesn't correctly interpret the instruction and pulls existing assets from the Phaser JS library. In short, it's quite finicky to get agents in prod to behave reliably 100% of the time. But we are working on it! Have many ideas of how to improve and are experimenting.
"I thought this would be a useful tool for prototyping, but as a programmer I would probably never use this to release a finished product. Maybe it would be better to target this kind of tool for prototyping instead of being able to make a full game that people will want to pay money for."
&
"I tried making a simple turn-based (american) football game. It's pretty good for getting some boilerplate set up, but sometimes it felt like it would have been easier to modify the code myself at times than try to explain what I wanted modified to the AI."
For users who already know how to develop, we've definitely heard similar validation that this is better for prototyping currently than for final games. I think this stems from the fact that 1) currently we don't have multi-file support and more advanced features like multiplayer yet, which makes it hard to compare with what's achievable by a more advanced game dev. And 2) the chat based approach is not always reliable, so it's frustrating sometimes to get what you want versus coding it up yourself. This is why we are also targeting less technical users, because they don't have many options available to them that can be as flexible as Rosebud. We have been delighted to see what these users create because some of the stuff gets very creative and addictive, even with our current limitations.
"It seems like whenever I did try to modify the code by hand, my changes would get wiped out by the next iteration generated by the AI. Maybe it was referencing the old code?"
This is a bug. If you don't mind sharing in the feedback channel specific cases when it does this, that would be super helpful. We've been fixing aspects of this bug over the last few week.
"Sometimes the AI would get stuck on something (e.g. generating an asset), or it would make a mistake. In that case I couldn't figure out a way to "go back" to a previous iteration of the game."
We have a history feature we are shipping in stages. Right now you can only regenerate, but we want to let users go back to different states of their game. Many users have wanted this feature and we just have to make some choices about what gets saved in the history (all llm changes or also manual, ... etc).
"At one point I tried to get the AI to make a change, which it did, but the code it generated had a large portion of the original code based removed and replaced with something like `// The rest of your code here.` which obviously broke the game."
This stems from a context length issue and is definitely annoying. We also have plans to implement diffs and other solutions that should let users run into this issue less.
Thanks for trying it! One of the more impressive things we have seen so far is how some beta testers with no coding background were able to get deep into debugging by having our AI explain what the issues are and how to fix them. However, as you pointed out, it does not reliably provide working code all the time for the user to apply directly. This is something we are aware of and are trying to make the generated code better and more reliably provide working solutions with no formatting errors. It is basically quite a hard (and thus fun technical) problem we are solving.
Our aim is to make development on Rosebud as easy as possible, which means we are handling various external integrations that are being used by the platform. With that said, some devs may want the flexibility to change what models are used, including switching out internal models we developed with various external ones. If that is true, we’d want to support devs providing their own keys, but only for added flexibility, not as a default. Put it simply, if you don't want to be forced to provide your own keys, we got you covered.
Right now we will try our best not to charge developers for developing on Rosebud. So we will try to optimize on our backend different ways to control costs. With that said, if we have to create some premium tier because the AI inference gets too high to support, we will likely provide options to reduce inference costs in various ways (such as running smaller models for different interactive generative features of a game).
We’d want to provide as much support as possible so that our devs are successful. So if something is not working, let us know! Our devs succeeding is a necessary pre-requisite for our platform succeeding.
See my comment above for another thread and the subsequent comments: "My take is that whenever we can break down barriers to allow more creators to enter, it's ultimately better for the entire industry. As a platform we can work on curating and filtering for high quality game content from our users, but that's a problem of curation rather than artificially limiting creation for fear of people making low quality content. Most content quality like many things are power law distributed anyways."
My optimism is mostly for increasing the number of creators of games --> best games get even better, at the cost of introducing more games, some of which may not be good. Marketplaces being the winners is a symptom of the players being the ultimate beneficiary of increased competition (and thus a surplus of both good and bad games).
I actually don't see this as an AI to replace, but AI to enable more people to create games. So ultimately, it still expresses the desire of the creator, who is human.
We are happy our custom image models are liked by our users and I think helping game devs make good game assets/art is a valuable problem to solve, but not one we are focused on. The first 3-4 years of Rosebud we focused on image generation based app, so I have thought a lot about the opportunity of focusing on a image/asset gen as the main feature of the product. I would argue that it generally has a thinner moat with more capped upside and can more easily be toppled by foundational model companies like OpenAI (with the exception of Midjourney, which is amazing because they have the moat of a community that guides their stronger aesthetics). Furthermore there are a number of other services that focus just on asset gen, and we allow uploads (from assets generated elsewhere) in addition to our own custom models for asset generation for that reason.
The opportunity I am excited most about, and what Rosebud is focused on, is to increase the number of creators of games by many orders of magnitude. That is why we are leaning into an llm native approach for game dev. I also know that the current best models for code gen will be improved dramatically in the next few years and that will have have an even greater impact on the consumer behaviour of who gets to make games and who gets to build software in general. Building a platform around game creation, not just asset gen, will be able to absorb this impending tidal wave of change that I don’t think even incumbent developer tools like Unity or Unreal can address as fast as a startup such as ourselves. The entire game creation work flow is going to get much more intuitive, faster and ultimately be able to generate the quality of games to compete with AAA. Now it is not there, but soon it will be. The game genres we support may look more opinionated and constrained now (ai characters, RPGs, some 3D) but that’s the first step in being able to let our agent based code gen platform perform well in prod. We are making choices that allow us to absorb llm advances later and that generalize well once those improvements happen.
Also, on the IP point, once we let people monetize on the platform, we will be much more strict about what gets to be monetized (i.e only things where it’s ok from an IP point of view).
There's no confusion at all about whether we are trying to a UGC platform to make games (i.e AI Roblox). We are definitely not an asset store. Even though you can create assets in Rosebud, it's a feature of the entire workflow, which is focused on converting user descriptions to games.
I think some confusion may be arising from just going to rosebud.ai and seeing the demo video. Instead go to https://play.rosebud.ai/home to make games and play the games other people already made to test it out! Love to hear your comments about the development experience there!
You are right that there is a developer tool angle here, but I think what's interesting to experiment with respect to the business model is whether developers want to be charged first by using the tool (like unity) versus only when they are successful (like unreal). Roblox is able to collect rent because they help the developers build an audience. What we have to show is that our platform can also help developers build an audience to justify collecting rent.
We have a bunch of trending projects that you can already clone from (see https://play.rosebud.ai/home "trending"), and this was in closed beta, so once we open the app to more users there will be a lot of projects to give new creators projects to start from immediately.
You can already kind of export the code because we show most of the game code. However what we're aiming to do is to make it easier to build in platform without worrying about deployment. Also we are aiming to bring an audience to play your games immediately via the Rosebud platform so you don't have to share elsewhere.
My take is that whenever we can break down barriers to allow more creators to enter, it's ultimately better for the entire industry. As a platform we can work on curating and filtering for high quality game content from our users, but that's a problem of curation rather than artificially limiting creation for fear of people making low quality content. Most content quality like many things are power law distributed anyways.
Thank you! We're so excited about how llms are expanding widely the number of people who can write software, it's the perfect opportunity to apply it to gaming since I think there's far more people who can make games if not limited to technical skills!
Regarding my path with Rosebud, it's such a long journey, I'd love to share. Going way back in 2017 I just finished up a PhD in Deep Learning from Berkeley and did the less usual thing of joining an early stage venture firm. I've always had to compartmentalize my creative work from my technical before. Once the first batch of interesting results in generative came out, primarily in the image space (cycleGAN, StyleGAN …etc), I realized that it was inevitable that most creative work can be changed completely by this research. Who gets to create, how easily they can create and even what is created will change dramatically. I just had very strong conviction on this. The only thing that was hard to predict was whether this was going to happen in the next 2-10 years. Regardless of the uncertainty in when this was going happen, I knew that I wanted to get my hands dirty and build, because it’s just something I wanted to use. So I jumped into the deep-end and founded Rosebud.
Earlier on, the models were quite new and not of the highest quality, so I learned quickly that consumers are a much better audience than using this for businesses. Consumers are still very picky, but also very experimental, whereas business had very specific requirements for what they want that are not easily “replaced” by AI generated stuff. Given these learnings, we iterated many times with different consumer mobile apps, which is a crucible for getting the right intuitive interface down. Long story short, you cannot go viral organically on mobile if your app is hard to use and doesn’t give a sense of magic. My core thesis is that the strength of generative AI is in its potential to make creation widely accessible, on demand and delightful. Naturally, some of our earlier apps were focused on more meme like creation experiences (like Tokkingheads, which organically grew to several million users). We learned that we must in a few clicks as possible allow the creator to achieve impressive results they want to share on social. These experiences helped build a lot of our technical ability to train and productionize models (back then there were no AI inference companies, so we had to manage everything ourselves on cloud or our own machines).
Every year since the founding of Rosebud, I looked at gaming as a target application area. As mentioned, Rosebud was named after the cheat code in the Sims, which allowed me to effectively use the game as a 3D playground to build virtual worlds back when I was a kid. This was my North Star for how powerful generative AI should get, to ultimately allow users to build their immersive virtual worlds and games. I also knew that just doing game asset generation was not enough to be interesting of a platform shift for games. Was waiting on code gen to get good enough to really have an opportunity to change how game dev and creation happens.
There’s definitely a lot to learn from Roblox! The first thing they got right was to make it many orders of magnitude simpler to make games than the incumbents of the time, that’s why it was so easy for Roblox to get many fun games on their platform and build an audience. This is also why we're emphasizing leaning into an AI native approach to have a unique advantage to making creation easier. Following that, as you say, enabling monetization on a browser based platform will be a necessary later step to keep creators incentivized. I think there’s no technical constraint to making monetization possible on our platform, so we are just not choosing to focus on this first because if we can show that the creation experience is far better on our platform, then it will bring a lot more users into it to make the platform even more attractive for creators.
Our belief is also that there’s no platform currently that makes it easy to deploy agent based games, where AI characters and dynamic AI agents are used in game. Given how popular even some basic AI character games are (look at the usage of Character AI!!), if we make such games easier to create, I trust that creators will be able to make very popular games. We’ve been having live game jams biweekly on our discord (next one next week!) and already see some magic being made, even in the closed beta!
PixelVibe by Rosebud.ai (Pixelvibe.com) | Fullstack with Frontend focus , ML Eng | 100% Remote with SF onsite option available | Paid | Fulltime. Internships available for exceptional candidates
Rosebud.ai is building the AI Roblox. We use generative AI workflows to accelerate game development and creative. The next few years of AI are going to define the coming few decades of the world. Some wait a lifetime for a moment like this. For those who have an inner fire to make something great, join us. Let’s build.
We're the team that built Tokkingheads (2 million IOS downloads, all organic) that allows any portrait/photo/face to be animated in seconds with no skill. We’re a team of PhD researchers, creatives and engineers backed by Y Combinator led by Vinod Khosla at Khosla Ventures with participation from Ilya Sutskever (cofounder/chief scientist Open AI), Andrej Karpathy (Director AI, Tesla), Kevin Lin (cofounder COO Twitch), Jon Snoddy SVP @ Walt Disney Imagineering, Jeff Dean SVP Google Research, Adam D'Angelo CEO Quora and Balaji Srinivasan (Former CTO Coinbase and A16Z GP).
Rosebud.ai is building the synthetic media platform for web3. We are helping creators make composable assets for web3 world building.
We're the team that built Tokkingheads (2 million IOS downloads, all organic) that allows any portrait/photo/face to be animated in seconds with no skill. Creators use Tokkingheads to make memes, deepfake parodies, NFTs. We’re a team of PhD researchers, creatives and engineers backed by Y Combinator with our seed led by Vinod Khosla at Khosla Ventures with participation from Balaji Srinivasan *(Former CTO Coinbase and A16Z GP), Ilya Sutskever (cofounder/chief scientist Open AI), Kevin Lin (cofounder COO Twitch), Holly Liu Co-founder Kaba, Kun Gao (cofounder, CEO Crunchyroll), Linda Xie Founder Scalar Capital, Jon Snoddy SVP @ Walt Disney Imagineering, Jeff Dean SVP Google Research, Adam D'Angelo CEO Quora, and Lily Liu. cofounder Earn.com.
https://rosebud.ai/ powers a suite of apps to help creatives make content. https://tokkingheads.com/, our most popular app (2 million IOS downloads, all organic, high retention) allows any portrait/photo/face to be animated in seconds with no skill. Creators use Tokkingheads to make memes, deepfake parodies, NFTs, and most notably make their family and friends feel special with photos of past loved ones animated (1.8M views #tokkingheads hashtags on TikTok).
We are looking for an experienced back-end engineer to bridge the gap between ML pipelines and customer-facing applications:
* maintain and extend business logic, including authentication, billing, access control, caching, user-managed data and settings;
* maintain and extend APIs consumed by our web- and mobile apps;
* in collaboration with ML team integrate business logic with ML pipelines using queues or HTTP;
* create and maintain integrations with 3rd-party services such as Stripe, Mailgun, and others;
* closely follow current and anticipated development of mobile and web applications to support new features with corresponding API endpoints;
* collaborate with front-end developers to shift complex logic to the backend where possible and reuse it across all customer-facing applications;
* document and test API endpoints, help front-end developers troubleshoot and use APIs efficiently.
Our stack: Firestore and GCS for persisting data, Firebase Functions and Firestore Triggers with node.js runtime for hosting APIs and business logic, Firebase Authentication for user management, RabbitMQ for jobs management.
https://rosebud.ai/ powers a suite of apps to help creatives make content. https://tokkingheads.com/, our most popular app (2 million IOS downloads, all organic, high retention) allows any portrait/photo/face to be animated in seconds with no skill. Creators use Tokkingheads to make memes, deepfake parodies, NFTs, and most notably make their family and friends feel special with photos of past loved ones animated (1.8M views #tokkingheads hashtags on TikTok).
We are looking for an experienced back-end engineer to bridge the gap between ML pipelines and customer-facing applications:
* maintain and extend business logic, including authentication, billing, access control, caching, user-managed data and settings;
* maintain and extend APIs consumed by our web- and mobile apps;
* in collaboration with ML team integrate business logic with ML pipelines using queues or HTTP;
* create and maintain integrations with 3rd-party services such as Stripe, Mailgun, and others;
* closely follow current and anticipated development of mobile and web applications to support new features with corresponding API endpoints;
* collaborate with front-end developers to shift complex logic to the backend where possible and reuse it across all customer-facing applications;
* document and test API endpoints, help front-end developers troubleshoot and use APIs efficiently.
Our stack: Firestore and GCS for persisting data, Firebase Functions and Firestore Triggers with node.js runtime for hosting APIs and business logic, Firebase Authentication for user management, RabbitMQ for jobs management.