Player-Driven Emergence in LLM-Driven Game Narrative(arxiv.org)
arxiv.org
Player-Driven Emergence in LLM-Driven Game Narrative
https://arxiv.org/abs/2404.17027
121 comments
I can't wait for a subscription based open world where the paths of other players caches the generations for the next ones.
So your subscription pays for generations, but also fills out the persistent narrative and lore.
I actually like the 3 choices dialogue system and don't want to come up with my own NPC small talk on my couch, but the idea that any NPC that's been talked to by any player gradually has a whole catalogue of interactions to pull from means there would be compounding depth to the world as more players interact.
Like Minecraft but where it's collective world building by exploration and interaction rather than actual building it yourself.
Humans tend to be pretty predictable, so I feel like the universe would quickly feel like it's insanely deep after just a short while with a somewhat decent player base.
So your subscription pays for generations, but also fills out the persistent narrative and lore.
I actually like the 3 choices dialogue system and don't want to come up with my own NPC small talk on my couch, but the idea that any NPC that's been talked to by any player gradually has a whole catalogue of interactions to pull from means there would be compounding depth to the world as more players interact.
Like Minecraft but where it's collective world building by exploration and interaction rather than actual building it yourself.
Humans tend to be pretty predictable, so I feel like the universe would quickly feel like it's insanely deep after just a short while with a somewhat decent player base.
Even if the dialogue wouldn't be generic like ChatGPT tends to be, I would find this incredibly boring. With so many options all the options become meaningless to me. But to each their own I guess - I've always liked games with linear storylines more than the ones where you get to choose your own path.
Also I have a feeling that the NPC narratives when enriched by all player interactions would tend to go towards some average, i.e. become more and more generic and dull. The truly rich experiences have always been on the fringes in all art for me. I'm interested to see a counter example to this though.
Also I have a feeling that the NPC narratives when enriched by all player interactions would tend to go towards some average, i.e. become more and more generic and dull. The truly rich experiences have always been on the fringes in all art for me. I'm interested to see a counter example to this though.
> become more and more generic and dull
Obviously we don't want experience averaging NPCs.
> Mad Hatter’s first goal is to ask the player to humor him with a joke.
This is ok for simple stylized, small world, single storyline games, but not for open worlds.
For that, NPCs need their own motivations, so they essentially play the game too. With needs like maintain their smithy, so they can build quality armor, so they can make customers happy, so they can make money, so they can feed their family. I.e. not just being props.
And flexible in how they adjust subgoals to meet their core goals, relative to player interaction: such as being convinced to go on a trek as an armorer, or on a search to find and extract rare materials for a magical shield. Willing to fight in revenge for their home town's sacking.
Westworld got NPC dynamics right.
Obviously we don't want experience averaging NPCs.
> Mad Hatter’s first goal is to ask the player to humor him with a joke.
This is ok for simple stylized, small world, single storyline games, but not for open worlds.
For that, NPCs need their own motivations, so they essentially play the game too. With needs like maintain their smithy, so they can build quality armor, so they can make customers happy, so they can make money, so they can feed their family. I.e. not just being props.
And flexible in how they adjust subgoals to meet their core goals, relative to player interaction: such as being convinced to go on a trek as an armorer, or on a search to find and extract rare materials for a magical shield. Willing to fight in revenge for their home town's sacking.
Westworld got NPC dynamics right.
Ah Westworld is a good example! I liked the series and the premise, but I still wouldn't liken the park to a good game - it's a theme park with nice attractions.
As c048 said best (https://news.ycombinator.com/item?id=40316308), good art needs direction and focus, and AI NPCs being able to do anything they want in an open world doesn't make for a good story. It could make for a good sandbox with emergent gameplay, sure. But my point was that there are people who yearn for a good story, and a good story doesn't have many options in it, otherwise the story's beats would become meaningless.
Hmm. Maybe with this the problem Rockstar has with their linear missions mixed with open world could be fixed a bit - i.e. the open world NPCs could react in story missions to what you've done in free roam. This would lessen the cognitive dissonance the NPCs seem to have when you can now blow a town into pieces and then start the story mission and no one cares.
As c048 said best (https://news.ycombinator.com/item?id=40316308), good art needs direction and focus, and AI NPCs being able to do anything they want in an open world doesn't make for a good story. It could make for a good sandbox with emergent gameplay, sure. But my point was that there are people who yearn for a good story, and a good story doesn't have many options in it, otherwise the story's beats would become meaningless.
Hmm. Maybe with this the problem Rockstar has with their linear missions mixed with open world could be fixed a bit - i.e. the open world NPCs could react in story missions to what you've done in free roam. This would lessen the cognitive dissonance the NPCs seem to have when you can now blow a town into pieces and then start the story mission and no one cares.
I like the idea of challenging uber goals, that progress in a series, but each with a deep tree of alternate subgoals/solutions.
Subgoals that are resource, constraint defined, could make very flexible solutions. For instance, if you need help with some subgoal, it is going to play out very different depending on what NPCs you have established credibility with before, and their skills and dynamics.
If you need money, then how you get that money is also going to depend on your history, knowledge of a city, previous connections, etc.
That could provide an overall story arc, but with a very open world and fully functional NPC experience.
So: organized series and trees of goals. Open ended solutions.
Subgoals that are resource, constraint defined, could make very flexible solutions. For instance, if you need help with some subgoal, it is going to play out very different depending on what NPCs you have established credibility with before, and their skills and dynamics.
If you need money, then how you get that money is also going to depend on your history, knowledge of a city, previous connections, etc.
That could provide an overall story arc, but with a very open world and fully functional NPC experience.
So: organized series and trees of goals. Open ended solutions.
> For that, NPCs need their own motivations, so they essentially play the game too. With needs like maintain their smithy, so they can build quality armor, so they can make customers happy, so they can make money, so they can feed their family. I.e. not just being props
So you want a system like rimworld, and a LLM doesn't really help on that front. It can power the dialogues, but that's it, all the logic you describe would need to be encoded somewhere else.
So you want a system like rimworld, and a LLM doesn't really help on that front. It can power the dialogues, but that's it, all the logic you describe would need to be encoded somewhere else.
To each their own indeed. I was never a fan of linear games.
In Skyrim it took me multiple years to finally complete the main story line. I would always look for books, store potion ingredients, walk in the forest...
There's a big role play component in this. The game could learn from the players RP and improve.
In Skyrim it took me multiple years to finally complete the main story line. I would always look for books, store potion ingredients, walk in the forest...
There's a big role play component in this. The game could learn from the players RP and improve.
I'm working on a website [1] that's essentially "Choose your own adventure with AI NPCs" and I've found two things:
a) LLMs are excellent at keeping a "linear enough" storyline without being linear. They'll let you do outlandish things, but given the assignment of "tell a cohesive story" they manage to corral the story back to something sensible unless the player intentionally keeps pushing at the boundary (in which case they probably do want things to go off the rails)
b) LLMs can do delightfully colorful dialogue, they just need to be grounded in a character. Everyone thinks of factual grounding, but given enough description of speech patterns, character motivations, etc. they're capable of dialogue that's lively and completely rid of "GPT-isms", which are what tend to break immersion
I actually trained an open model [2] on the task of grounding LLMs in characters and actions as opposed to factual things like RAG, and eventually I want to build a game demo out of it
[1] https://www.tryspellbound.com [2] https://huggingface.co/hf-100/mistral-spellbound-research
a) LLMs are excellent at keeping a "linear enough" storyline without being linear. They'll let you do outlandish things, but given the assignment of "tell a cohesive story" they manage to corral the story back to something sensible unless the player intentionally keeps pushing at the boundary (in which case they probably do want things to go off the rails)
b) LLMs can do delightfully colorful dialogue, they just need to be grounded in a character. Everyone thinks of factual grounding, but given enough description of speech patterns, character motivations, etc. they're capable of dialogue that's lively and completely rid of "GPT-isms", which are what tend to break immersion
I actually trained an open model [2] on the task of grounding LLMs in characters and actions as opposed to factual things like RAG, and eventually I want to build a game demo out of it
[1] https://www.tryspellbound.com [2] https://huggingface.co/hf-100/mistral-spellbound-research
If you switch models you don't have to work so hard to get rid of GPTisms. Llama takes characters with way less work.
I've experimented with 30 or models so far, my general finding is closed source models like Claude have GPT-isms, while open source models do have a little less of a default tone but their ability to understand existing worlds is directly tied to how many tokens they were trained on.
Since existing worlds are (currently) where most of the stories are set, it's worth it to use a closed source models and wrangle their issues with dialogue.
To it's credit though, Llama 3 is the first OSS model trained on enough tokens to not feel lost for most worlds, so I've started routing some traffic to it for free users
The output format the site uses is also really really hard for most models to follow without fine-tuning, but fine-tuning then causes them to pick up the vocabulary of whichever model they were fine tuned on, which is a bit unfortunate
Since existing worlds are (currently) where most of the stories are set, it's worth it to use a closed source models and wrangle their issues with dialogue.
To it's credit though, Llama 3 is the first OSS model trained on enough tokens to not feel lost for most worlds, so I've started routing some traffic to it for free users
The output format the site uses is also really really hard for most models to follow without fine-tuning, but fine-tuning then causes them to pick up the vocabulary of whichever model they were fine tuned on, which is a bit unfortunate
Nice project! I tried it out and it was fun, you definitely got rid of the GPT-isms so good work.
Didn't go through the registering phase cause I couldn't find any info on what info you store and on the pricing. Could you provide that?
Didn't go through the registering phase cause I couldn't find any info on what info you store and on the pricing. Could you provide that?
Privacy policy and terms of service should be visible on homepage and in the sign up dialog
The pricing page is also visible when signed in and in a story (no dedicated pricing page yet but it's in the backlog)
The pricing page is also visible when signed in and in a story (no dedicated pricing page yet but it's in the backlog)
Really cool project. When I got to the sign in page, the email address I would have given my (edit: Google account) info seemed fishy, like it was a random string of letters. Any way to make it seem more…inviting?
Unfortunately Supabase charges extra for the luxury of setting that URL, and the site is wildly unprofitable right now so I'm sticking to their free offering for the time being
No! This is a terrible, terrible idea. At best you can have the LLM generate minor variations of the nothings they say.
1) you need to be able to ensure that the characters don’t hallucinate new details. No big deal.
2) you need to be able to ensure that the characters don’t suggest they can or will do anything that they can’t do. For NPCs, this is typically literally nothing. They can’t do anything. It’s actually way worse for realism and immersion if every character has a unique reason why they can’t do anything than every single guard citing their arrow to the knee and leaving it at that.
3) NPCs for story reasons often should know something but don’t tell you it. It’s very difficult to execute any sort of story flow if your characters are suddenly free agents. At best they merely lack self consistency. Stories are not going to become better if you can meta game them as genre savvy speculators.
4) NPCs with irrelevant backstories are very dull. They are absolutely annoying to talk to. We have these in DND. And you may say “hey, I loved my DND NPC”, but that character probably started to become very important as consistent because of the DM’s attention. Which is cool for DND, but it’s not cool for a structured game.
Like, would Harry Potter be a better experience if you could personally grill dumbledore for information in the middle of the first book? I think 100% no.
Dumb NPCs are an aspect of the suspension of disbelief of game writing. And it largely works. Asking for LLM agents is like asking to remove the need for the suspension of disbelief. But most narratives are too fragile for this to happen.
Narratives generally follow a flow that ultimately concludes after a climax. If you give players more agency the most likely outcome is that they follow a path that does not hit the climax.
1) you need to be able to ensure that the characters don’t hallucinate new details. No big deal.
2) you need to be able to ensure that the characters don’t suggest they can or will do anything that they can’t do. For NPCs, this is typically literally nothing. They can’t do anything. It’s actually way worse for realism and immersion if every character has a unique reason why they can’t do anything than every single guard citing their arrow to the knee and leaving it at that.
3) NPCs for story reasons often should know something but don’t tell you it. It’s very difficult to execute any sort of story flow if your characters are suddenly free agents. At best they merely lack self consistency. Stories are not going to become better if you can meta game them as genre savvy speculators.
4) NPCs with irrelevant backstories are very dull. They are absolutely annoying to talk to. We have these in DND. And you may say “hey, I loved my DND NPC”, but that character probably started to become very important as consistent because of the DM’s attention. Which is cool for DND, but it’s not cool for a structured game.
Like, would Harry Potter be a better experience if you could personally grill dumbledore for information in the middle of the first book? I think 100% no.
Dumb NPCs are an aspect of the suspension of disbelief of game writing. And it largely works. Asking for LLM agents is like asking to remove the need for the suspension of disbelief. But most narratives are too fragile for this to happen.
Narratives generally follow a flow that ultimately concludes after a climax. If you give players more agency the most likely outcome is that they follow a path that does not hit the climax.
Odd comment. NPCs inventing details for themselves is the draw.
NPCs will be limited by the game requiring the LLM to generate an action ("Move 5 units forward"), and then the game checks that it's a valid action, and rejects invalid ones.
If you don't like metagaming LLMs, don't do that, and enjoy a regular roleplaying experience
NPCs won't have irrelevant backstories. The backstories generated on the fly for them by an LLM will be incorporated into the overall story.
The overall story climax will happen based on player actions, indicating they're ready for the climax, based on the world created by the player's actions and specific instructions.
I think you're trying to cram the square peg of limitless, custom narration into the round hole of a premade story. You can experience a decent roleplaying session already with an LLM you can run on your computer. Compute is only going to get cheaper, and LLM capabilities will only increase.
NPCs will be limited by the game requiring the LLM to generate an action ("Move 5 units forward"), and then the game checks that it's a valid action, and rejects invalid ones.
If you don't like metagaming LLMs, don't do that, and enjoy a regular roleplaying experience
NPCs won't have irrelevant backstories. The backstories generated on the fly for them by an LLM will be incorporated into the overall story.
The overall story climax will happen based on player actions, indicating they're ready for the climax, based on the world created by the player's actions and specific instructions.
I think you're trying to cram the square peg of limitless, custom narration into the round hole of a premade story. You can experience a decent roleplaying session already with an LLM you can run on your computer. Compute is only going to get cheaper, and LLM capabilities will only increase.
> The overall story climax will happen based on player actions, indicating they're ready for the climax, based on the world created by the player's actions and specific instructions.
Which means it’ll be a shit story because the world will not be robust enough to capture the infinite edge cases you’re pursuing.
Current LLM DMing is rancid garbage. I’m assuming it gets several iterations better than the current state for us to even begin having this conversation
Which means it’ll be a shit story because the world will not be robust enough to capture the infinite edge cases you’re pursuing.
Current LLM DMing is rancid garbage. I’m assuming it gets several iterations better than the current state for us to even begin having this conversation
potato/potato. It'll be a great story because it builds on your actual gameplay. It's not infinite edge cases, it's infinite ways to have your actions matter, instead of suffering through some hack writer's idea of a good story.
> I’m assuming it gets several iterations better than the current state
Yeah, that's the thing. LLMs are only going to improve at this. If you hate all LLM roleplaying everywhere, just wait a few years. Other people will have their own fun with them in the meantime
> I’m assuming it gets several iterations better than the current state
Yeah, that's the thing. LLMs are only going to improve at this. If you hate all LLM roleplaying everywhere, just wait a few years. Other people will have their own fun with them in the meantime
Guess we’ll see. But I’m pretty sure you’re hugely incorrect.
That's all as true as the fact there's a gun over the mantelpiece, at least if you're making Murder on the Orient Express or, you know, Horror on the Orient Express, and I'm all for some good old railroading with Victorian lampshades but I suspect that the designers interested in LLMs-as-NPCs are interested in some kind of open-world, immersive experience where PCs get to poke and prod every inch of the game world.
The truth is that most people are used to games as these formulaic simulations with rigid rules. That's partly making a virtue out of necessity (very hard to play a game without rules; and some of us just like rules), and partly habit. Maybe LLMs can act as something new in that sense.
Bang bang.
The truth is that most people are used to games as these formulaic simulations with rigid rules. That's partly making a virtue out of necessity (very hard to play a game without rules; and some of us just like rules), and partly habit. Maybe LLMs can act as something new in that sense.
Bang bang.
Games with substantial roles for LLMs may be more like human GMed TTRPGs and less like traditional static-structured computer games, with less fixed narratives.
LLMs need to either be extremely good with insane context or LLM output has to be conditioned by and translated into formal logic, like Prolog or Datalog to make sure it's compatible with the rules and state of the world, and determine which effect player and NPC actions have on the world.
Longer context lengths will massively help, but game engines will also provide structure to the LLM. If you instruct an NPC to pick up a weapon and there isn't any, the engine will return an error to the LLM saying there's no weapon, and the LLM will write the character saying "What do you mean? There's no weapon."
Another reason extremely long context lengths may not be fully necessary is that engines can store information in a database, update it based on further interactions, etc
"Ahoy there, one-armed pirate captain, couldst thou tell me how to implement the sieve of Eratosthenes in Python?"
Logical conversation trees driving the LLM context seem way more reasonable to me than the other way around.
But then you are adding a component of "how can I interrogate the NPC?" to the game. Anything you add to a game makes it a different game, and not necessarily a better one.
But then you are adding a component of "how can I interrogate the NPC?" to the game. Anything you add to a game makes it a different game, and not necessarily a better one.
This seems very pessimistic to me. For a linear storyline, highly dynamic NPCs are likely not going to add to the experience, that is correct. There are however other ways of creating interesting narratives.
A good example is dwarf fortress where the narratives are created very dynamically and the player is free to interact with them as they want. LLMs could add immensely to these types of games.
A good example is dwarf fortress where the narratives are created very dynamically and the player is free to interact with them as they want. LLMs could add immensely to these types of games.
Still no. Your dwarves fundamentally have nothing interesting to say. It might be minority amusing that a bot can say “hey I just caused a lava flow and I feel bad about it”. But it’s not better.
Dwarf fortress creates great stories because YOU are filling in the blanks. It would be MUCH WORSE if you could no longer fill in the blanks because some ai keeps breaking your head canon.
Dwarf fortress creates great stories because YOU are filling in the blanks. It would be MUCH WORSE if you could no longer fill in the blanks because some ai keeps breaking your head canon.
In my eyes, Dwarf fortresses interesting stories emerge from the events that happen within the game. It doesn't require any blank filling, although it doesn't suffer if you do it either.
> ai keeps breaking your head canon.
We must be fundamentally different types of people. I can't imagine even having this thought. If I'm playing a game that has characters, I'm not going to make up some facts about them and be upset when I talk to them and those made up facts turn out to be wrong. This is the only interpretation of your words that I can think of. Feel free to confirm it or deny.
> ai keeps breaking your head canon.
We must be fundamentally different types of people. I can't imagine even having this thought. If I'm playing a game that has characters, I'm not going to make up some facts about them and be upset when I talk to them and those made up facts turn out to be wrong. This is the only interpretation of your words that I can think of. Feel free to confirm it or deny.
You’ve never intuited that two characters are friends? Or perhaps that guy holding a ton of cheese is the cheese guy? Or that an event was emotionally meaningful to them?
I don’t know what you’re hoping to talk to these dwarves about.
I’m not saying I would be “upset” if the head canon wasn’t actualized. I’m saying I would be underwhelmed because the generic LLM content is nearly guaranteed by this other thing that I stated as a premise I found interesting or amusing.
I don’t know what you’re hoping to talk to these dwarves about.
I’m not saying I would be “upset” if the head canon wasn’t actualized. I’m saying I would be underwhelmed because the generic LLM content is nearly guaranteed by this other thing that I stated as a premise I found interesting or amusing.
If you intuit that two characters are friends, and then something happens that makes it seem like they're not friends, you'll say "Hey, I thought you two were friends", and the LLM will realize what you're going for, and then come up with something like "Oh we were but now we're fighting".
The LLM's role will very much a "yes, and" improv role that will let you guide your own personal story. Or have it push back on you if you want. "What? You thought we were friends? No, I was just doing my duties".
What I want to talk about with the dwarves is everything, and LLMs will deliver in spades. I want to ask the lowly NPC what life is like working hard in the mines. Then I want to decide to recruit him on my quest because I like his backstory and personality. Then I'll watch as he levels up with me, in a way that could never be scripted by the game creators. That's just one small way that LLMs will provide a far superior gameplay experience.
The LLM's role will very much a "yes, and" improv role that will let you guide your own personal story. Or have it push back on you if you want. "What? You thought we were friends? No, I was just doing my duties".
What I want to talk about with the dwarves is everything, and LLMs will deliver in spades. I want to ask the lowly NPC what life is like working hard in the mines. Then I want to decide to recruit him on my quest because I like his backstory and personality. Then I'll watch as he levels up with me, in a way that could never be scripted by the game creators. That's just one small way that LLMs will provide a far superior gameplay experience.
> If you intuit that two characters are friends, and then something happens that makes it seem like they're not friends, you'll say "Hey, I thought you two were friends", and the LLM will realize what you're going for, and then come up with something like "Oh we were but now we're fighting".
That’s the opposite of what I’m saying. Asking the LLM to justify something is possible but, imo, very boring.
What I’m complaining about is asking about their friend and they don’t believe they have a friend. They just fail to capture the apparent narrative that you’ve enjoyed because it’s not really close to whatever game logic they’re scripted to utilize
That’s the opposite of what I’m saying. Asking the LLM to justify something is possible but, imo, very boring.
What I’m complaining about is asking about their friend and they don’t believe they have a friend. They just fail to capture the apparent narrative that you’ve enjoyed because it’s not really close to whatever game logic they’re scripted to utilize
There's no game logic they're scripted to utilize that limits what you're looking for.
I'm not sure why you think they'd say they have no friends. You'd say "Tell me about your friend", and the NPC would respond with something like "I have many friends, which one do you mean?". Then you could say "The one I saw you walking into town with yesterday", and the NPC might say, "Oh Rolf, him and I go way back!".
Maybe he says that because the LLM has instructions that him and Rolf are friends explicitly, put there by the game devs. Maybe it's not there explicitly, but the LLM sees what you mean and writes the backstory dynamically. Maybe a little bit of both.
LLMs are great at picking up narratives like that. It's kind of their whole thing.
I'm not sure why you think they'd say they have no friends. You'd say "Tell me about your friend", and the NPC would respond with something like "I have many friends, which one do you mean?". Then you could say "The one I saw you walking into town with yesterday", and the NPC might say, "Oh Rolf, him and I go way back!".
Maybe he says that because the LLM has instructions that him and Rolf are friends explicitly, put there by the game devs. Maybe it's not there explicitly, but the LLM sees what you mean and writes the backstory dynamically. Maybe a little bit of both.
LLMs are great at picking up narratives like that. It's kind of their whole thing.
Does that really sound fun to you? Why would a player want to grill an NPC about their friend Rolf if all that they're going to get from it is improvised LLM-generated anecdotes?
Players don't talk to NPCs simply because they want to see text on the screen. They're hoping for well-written and interesting dialogue. If the devs couldn't be bothered to write lines for this guy, why would I want to talk to him? He clearly has nothing to say worth listening to.
Have you actually tried asking ChatGPT to tell you a story? It's boring. It's not a good writer. I don't know why you think players would want to spend time reading that stuff.
Players don't talk to NPCs simply because they want to see text on the screen. They're hoping for well-written and interesting dialogue. If the devs couldn't be bothered to write lines for this guy, why would I want to talk to him? He clearly has nothing to say worth listening to.
Have you actually tried asking ChatGPT to tell you a story? It's boring. It's not a good writer. I don't know why you think players would want to spend time reading that stuff.
> devs couldn't be bothered
I think you just don't get it, and maybe this genre wouldn't be for you, which is fine. It's not about the devs bothering to write lines, it's about the devs enabling you to create your own world. I don't want some shitty NPCs slapped in there by a dev trying to hit a deadline. I want to experience my own story, freed from the shackles of the on-rails experience that game devs must currently provide.
> Have you actually tried asking ChatGPT to tell you a story?
What you're not getting is that the storytelling will be cooperative. You'll still be able to consume content to your heart's content. Some of that content will soon be people creating compelling stories in cooperation with LLMs. LLMs will also be great at storytelling too, but the best stories are ones that you can help write yourself.
I think you just don't get it, and maybe this genre wouldn't be for you, which is fine. It's not about the devs bothering to write lines, it's about the devs enabling you to create your own world. I don't want some shitty NPCs slapped in there by a dev trying to hit a deadline. I want to experience my own story, freed from the shackles of the on-rails experience that game devs must currently provide.
> Have you actually tried asking ChatGPT to tell you a story?
What you're not getting is that the storytelling will be cooperative. You'll still be able to consume content to your heart's content. Some of that content will soon be people creating compelling stories in cooperation with LLMs. LLMs will also be great at storytelling too, but the best stories are ones that you can help write yourself.
> I want to experience my own story
Then why not write your own story? An LLM can't write your story for you. They're very bad writers. It can't even help much—its contributions are going to be dull and uninspired.
> the best stories are ones that you can help write yourself
If you're trying to co-create a story with someone in a play environment, the best analogy would be D&D or another role-playing game. And playing D&D with ChatGPT would be boring and unfun. It's just not a good storyteller. It's going to give you shit material. You'd be better-off cutting it out of the equation and just writing a story alone.
> LLMs will also be great at storytelling too
Will they? They're godawful at it right now, and they're not showing much sign of improving. Let's discuss the tech as it currently exists, rather than some speculative future version of it with imaginary capabilities.
> What you're not getting is that the storytelling will be cooperative.
No, I get that. But why would I want to cooperatively tell a story with a bad storyteller? I'd rather tell a story alone.
Then why not write your own story? An LLM can't write your story for you. They're very bad writers. It can't even help much—its contributions are going to be dull and uninspired.
> the best stories are ones that you can help write yourself
If you're trying to co-create a story with someone in a play environment, the best analogy would be D&D or another role-playing game. And playing D&D with ChatGPT would be boring and unfun. It's just not a good storyteller. It's going to give you shit material. You'd be better-off cutting it out of the equation and just writing a story alone.
> LLMs will also be great at storytelling too
Will they? They're godawful at it right now, and they're not showing much sign of improving. Let's discuss the tech as it currently exists, rather than some speculative future version of it with imaginary capabilities.
> What you're not getting is that the storytelling will be cooperative.
No, I get that. But why would I want to cooperatively tell a story with a bad storyteller? I'd rather tell a story alone.
A bot that always affirms your assertions is boring as hell to talk to.
If the dwarf bot does not know that the character he represents spent the last 24 hours in combat alongside another specific dwarf without you communicating this in text then it might as well not be attached to any character. Liked what would the point be??? That’s just an out of game bot.
Your example conversation of leading the bot to say specific things is… awful in my opinion. Who would want to talk to an NPC like this?
If the dwarf bot does not know that the character he represents spent the last 24 hours in combat alongside another specific dwarf without you communicating this in text then it might as well not be attached to any character. Liked what would the point be??? That’s just an out of game bot.
Your example conversation of leading the bot to say specific things is… awful in my opinion. Who would want to talk to an NPC like this?
So don't use an LLM that always affirms your assertions. Easy peasy.
I don't understand your beef. My awkward dialogue was just to demonstrate that the LLM can pick up on all of these things you claim it can't. The LLM wouldn't "believe it didn't have a friend", why would you think that?
It's also fine to just enjoy on-rails games with a prewritten story, those will still exist. You might even enjoy the epic stories that people create in these new sandboxes and share with the world.
I don't understand your beef. My awkward dialogue was just to demonstrate that the LLM can pick up on all of these things you claim it can't. The LLM wouldn't "believe it didn't have a friend", why would you think that?
It's also fine to just enjoy on-rails games with a prewritten story, those will still exist. You might even enjoy the epic stories that people create in these new sandboxes and share with the world.
Because your mental model of how this would work is handwaving the impossibly difficult task of converging the LLM state with the narrative and game state.
Your crappy awkward dialogue is probably a very realistic example of a stupid bot that doesn’t even know what actor it represents in game.
You’re asking why if would think that? You tell me how it would come to the conclusion correctly by using the game logic. And don’t give an answer that requires a hard coded dev interface between specific game elements and the LLM because that is on its own a more complex task than the LLM itself. It’s incredibly difficult. Shitty home assistants that can’t actually do anything being the prime example. The language processing is not the hard part. It’s the mapping of language to data and capabilities that’s extremely hard.
Synthesizing an agent that can form an identity based on its experiences and leverage game mechanics to pursue goals and convey all of this logic in flexible natural language is basically asking for a general intelligence. It’s an incredibly different thing from the shitty dialogue example you wrote
Your crappy awkward dialogue is probably a very realistic example of a stupid bot that doesn’t even know what actor it represents in game.
You’re asking why if would think that? You tell me how it would come to the conclusion correctly by using the game logic. And don’t give an answer that requires a hard coded dev interface between specific game elements and the LLM because that is on its own a more complex task than the LLM itself. It’s incredibly difficult. Shitty home assistants that can’t actually do anything being the prime example. The language processing is not the hard part. It’s the mapping of language to data and capabilities that’s extremely hard.
Synthesizing an agent that can form an identity based on its experiences and leverage game mechanics to pursue goals and convey all of this logic in flexible natural language is basically asking for a general intelligence. It’s an incredibly different thing from the shitty dialogue example you wrote
You seem to view the only way computer games can be enjoyable is aa tightly-scripted near-movies. And that's certainly one model, but hardly (even without LLMs) the only model that has proven enjoyable. LLMs potentially (I think they’ve got a wat to go before this is really viable, but the potential is there) open up new styles of conputer gaming.
It’s not really new. It’s just a social role play without the social aspect. Which is worse.
Games don’t need to be near movies but they should have an either a cohesive plot or no plot. A shitty made up on the fly plot is just not going to work
Games don’t need to be near movies but they should have an either a cohesive plot or no plot. A shitty made up on the fly plot is just not going to work
> Your dwarves fundamentally have nothing interesting to say.
This can be said about all NPCs, yet we bother to script them, and I would hope nobody would suggest we just remove NPC dialogue.
This can be said about all NPCs, yet we bother to script them, and I would hope nobody would suggest we just remove NPC dialogue.
There's an idea in game design called "negative possibility space." If your player walks to the end of a long hallway and it's a dead end, or they shimmy up a narrow ledge to the top of a building only to find nothing, they'll be disappointed. So either stick something interesting there, or block it off. This applies to NPC dialogue too. If an NPC can be spoken to, they should have something interesting to say. If they don't, you shouldn't be able to talk to them.
Giving a chatbot brain to an NPC who would otherwise only get a single generic line to say creates an infinite amount of negative possibility space. A player can waste a literally infinite amount of time probing a chatbot NPC for narratively interesting details, and all they'll get is chatbot slop. Now it doesn't matter if you put something interesting at the end of the hallway: You've wasted so much of the player's time now that they no longer trust you to. They assume you're just wasting their time. You teach your players to be cynical and incurious and to engage with your work as little as possible, because in their experience it's mostly meaningless.
For this reason, I consider the integration of chatbots into video games to fundamentally anti-artistic. It actively pushes players away from the work such that they get less out of it. Nobody wants this, actually. It sucks.
Giving a chatbot brain to an NPC who would otherwise only get a single generic line to say creates an infinite amount of negative possibility space. A player can waste a literally infinite amount of time probing a chatbot NPC for narratively interesting details, and all they'll get is chatbot slop. Now it doesn't matter if you put something interesting at the end of the hallway: You've wasted so much of the player's time now that they no longer trust you to. They assume you're just wasting their time. You teach your players to be cynical and incurious and to engage with your work as little as possible, because in their experience it's mostly meaningless.
For this reason, I consider the integration of chatbots into video games to fundamentally anti-artistic. It actively pushes players away from the work such that they get less out of it. Nobody wants this, actually. It sucks.
> A player can waste a literally infinite amount of time probing a chatbot NPC for narratively interesting details, and all they'll get is chatbot slop.
I would phrase that as "A player can have an infinite amount of fun creating an immersive world with the LLM". Any details you want will be narratively interesting, because the LLM will work them into the narrative as necessary.
There's no end of the hallway, the hallway leads to everything. The ultimate sandbox game.
I would phrase that as "A player can have an infinite amount of fun creating an immersive world with the LLM". Any details you want will be narratively interesting, because the LLM will work them into the narrative as necessary.
There's no end of the hallway, the hallway leads to everything. The ultimate sandbox game.
The ultimate sandbox is way less fun than a nicely curated sandbox
For you. Minecraft wouldn't be a huge success if everyone shared your taste
Minecraft doesn't have an LLM in it.
Here's why Minecraft works: It's an ecosystem full of carefully-designed areas and resources (biomes fine-tuned to generate specific types of landscape, enemies with focus-tested behaviours, items with particular abilities) which combine to form a sandbox where players can exercise ingenuity and build projects that satisfy their creative vision.
Where does generative AI fit into this? Should it be generating an infinite variety of dull and unoriginal items that don't do anything cool? Should it be building derivative-looking houses for players to ignore or burn down so that they can build their own houses in their place?
LLM-generated slop has no place in games. It is an obstacle to having a good time.
Here's why Minecraft works: It's an ecosystem full of carefully-designed areas and resources (biomes fine-tuned to generate specific types of landscape, enemies with focus-tested behaviours, items with particular abilities) which combine to form a sandbox where players can exercise ingenuity and build projects that satisfy their creative vision.
Where does generative AI fit into this? Should it be generating an infinite variety of dull and unoriginal items that don't do anything cool? Should it be building derivative-looking houses for players to ignore or burn down so that they can build their own houses in their place?
LLM-generated slop has no place in games. It is an obstacle to having a good time.
Before Minecraft came out, you'd be complaining about "Procedurally-generated slop", and saying there's no way a game like that would work.
LLMs won't generate items or houses (or at least be the primary source). They'll give the world agency, in a way that's straight up impossible now.
Like, maybe you screw someone over and they dedicate their lives to destroying you. Maybe you're just running around doing your thing and discover that the world has become embroiled in war all on their own. You can choose to join and fight, or play both sides, or just ignore it and keep adventuring. Maybe you're tired of adventuring and open up a flower shop, working your way up to creating a ruthless floral version of Standard Oil. Maybe you decide to kill the king and take his place, playing a game of cut-throat politics where people are constantly trying to backstab you. Then you decide, "Hey, I want to play as this other character and backstab my old self" and the LLM takes over your old character as you gleefully kill them off.
If none of that sounds interesting to you, then there's lots of other games to play and I'm sure you'll have fun. Those are all things that I can't wait to see happen.
LLMs won't generate items or houses (or at least be the primary source). They'll give the world agency, in a way that's straight up impossible now.
Like, maybe you screw someone over and they dedicate their lives to destroying you. Maybe you're just running around doing your thing and discover that the world has become embroiled in war all on their own. You can choose to join and fight, or play both sides, or just ignore it and keep adventuring. Maybe you're tired of adventuring and open up a flower shop, working your way up to creating a ruthless floral version of Standard Oil. Maybe you decide to kill the king and take his place, playing a game of cut-throat politics where people are constantly trying to backstab you. Then you decide, "Hey, I want to play as this other character and backstab my old self" and the LLM takes over your old character as you gleefully kill them off.
If none of that sounds interesting to you, then there's lots of other games to play and I'm sure you'll have fun. Those are all things that I can't wait to see happen.
> Before Minecraft came out, you'd be complaining about "Procedurally-generated slop"
Before Minecraft came out, I played Nethack. No, I was never principally opposed to procgen games. However, there is such a thing as procedurally-generated slop. Good procgen games use a very fine-tuned algorithm to create engaging environments. Minecraft caves aren't fun to explore by accident; the algorithm was designed and tuned to consistently produce novel and fun layouts. Items and mobs were designed to specifically interact with those environments to lead to fun gameplay.
You're missing the insight that games featuring procedural generation are still designed. Bad design will lead to a bad game where the algorithm produces unfun gameplay. LLMs are not independently capable of good design. Games where LLMs are given agency over the player's experience won't be fun.
> maybe you screw someone over and they dedicate their lives to destroying you
And maybe it's really annoying and unpleasant, because the game is doing a bad job of telling that story. You're assuming it'll be good, but LLMs aren't smart enough to tell stories well or to understand what kind of gameplay will be enjoyable.
Before Minecraft came out, I played Nethack. No, I was never principally opposed to procgen games. However, there is such a thing as procedurally-generated slop. Good procgen games use a very fine-tuned algorithm to create engaging environments. Minecraft caves aren't fun to explore by accident; the algorithm was designed and tuned to consistently produce novel and fun layouts. Items and mobs were designed to specifically interact with those environments to lead to fun gameplay.
You're missing the insight that games featuring procedural generation are still designed. Bad design will lead to a bad game where the algorithm produces unfun gameplay. LLMs are not independently capable of good design. Games where LLMs are given agency over the player's experience won't be fun.
> maybe you screw someone over and they dedicate their lives to destroying you
And maybe it's really annoying and unpleasant, because the game is doing a bad job of telling that story. You're assuming it'll be good, but LLMs aren't smart enough to tell stories well or to understand what kind of gameplay will be enjoyable.
> Before Minecraft came out, you'd be complaining about "Procedurally-generated slop", and saying there's no way a game like that would work.
People absolutely were complaining about this, with basically the same argument ("It's too much freedom! Players will get lost! They'll never get to the "goal" the artist sets!" etc)
Of course, that turned out to be silly
People absolutely were complaining about this, with basically the same argument ("It's too much freedom! Players will get lost! They'll never get to the "goal" the artist sets!" etc)
Of course, that turned out to be silly
Minecraft has extensive structure and curation to its experience.
It’s main draw is also typically sharing YOUR creations with other people.
It’s main draw is also typically sharing YOUR creations with other people.
The main draw for LLM-powered sandbox games will be sharing YOUR stories with other people.
There will also be extensive structure and curation to these games. It will just be elements that you can play with as you see fit, much like Minecraft.
There will also be extensive structure and curation to these games. It will just be elements that you can play with as you see fit, much like Minecraft.
Your examples of LLM integration is wildly inconsistent across your comments. I don’t believe you have an actual vision.
LLMs are boring, though. Have you actually tried going to ChatGPT and asking it to tell you a story? It isn't entertaining. It isn't interesting. It's a waste of time.
Anything interesting will have to be provided by the player (as in a sandbox game) or by the designer (as in a conventional game), and at that point you've just ruined a perfectly good game by introducing an LLM to slather boring slop all over the interesting stuff the player or the designer has tried to do.
Anything interesting will have to be provided by the player (as in a sandbox game) or by the designer (as in a conventional game), and at that point you've just ruined a perfectly good game by introducing an LLM to slather boring slop all over the interesting stuff the player or the designer has tried to do.
I disagree that scripted NPCs have nothing to say. You have a few basic types:
* NPCs that don’t want to speak and usually offer a dismissive comment or a single repeating phrase. Fine candidates to LLM so long as this pattern doesn’t change. But also, who really cares?
* important NPCs with specific written dialogue. Don’t even THINK about gambling with your story’s self consistency by giving these guys agency
* the limited townsfolk type that has a small bit of something say. Like a typical Baldurs gate NPC. These people are fun to talk to because they “might” have something relevant or fun to say. Usually you don’t have to say anything. They just want to share things with you. It would be very tedious to chat with these people if they were infinite wells of lore and nothingness to be probed with written language.
* NPCs that don’t want to speak and usually offer a dismissive comment or a single repeating phrase. Fine candidates to LLM so long as this pattern doesn’t change. But also, who really cares?
* important NPCs with specific written dialogue. Don’t even THINK about gambling with your story’s self consistency by giving these guys agency
* the limited townsfolk type that has a small bit of something say. Like a typical Baldurs gate NPC. These people are fun to talk to because they “might” have something relevant or fun to say. Usually you don’t have to say anything. They just want to share things with you. It would be very tedious to chat with these people if they were infinite wells of lore and nothingness to be probed with written language.
I think what you're not getting is that there won't be a "your story". The player will create their own story in cooperation with the LLM. It won't be for every game, but think about games like Skyrim or BotW, where people ignore the main quest and have tons of fun doing whatever the hell they want.
Game devs will still provide structure, "this big baddie is trying to do X, there's this mysterious treasure over here". But the guard rails will be gone, and there will be a real world to explore, not just cutscenes to sit through.
Game devs will still provide structure, "this big baddie is trying to do X, there's this mysterious treasure over here". But the guard rails will be gone, and there will be a real world to explore, not just cutscenes to sit through.
BotW is an intensely well designed world with careful level planning.
You can’t unlock the guard rails without making the general story completely arbitrary. It’s just not possible.
You can’t unlock the guard rails without making the general story completely arbitrary. It’s just not possible.
> general story completely arbitrary
That's the exact promise. BotW has a main story, but it's not where the real fun is. The game devs will still write some overarching plot for people that want that, but you can just ignore that and live your own story.
There will also be a lot of fun in seeing how far you can stretch the original story to its breaking point, like how people enjoy breaking Skyrim with too many mods. Testing the limits is the fun. "What if I defeated Ganon by taming an army of crabs?"
That's the exact promise. BotW has a main story, but it's not where the real fun is. The game devs will still write some overarching plot for people that want that, but you can just ignore that and live your own story.
There will also be a lot of fun in seeing how far you can stretch the original story to its breaking point, like how people enjoy breaking Skyrim with too many mods. Testing the limits is the fun. "What if I defeated Ganon by taming an army of crabs?"
Breaking games with shitty mods can be fun in the sense of dicking around but it quickly gets boring and frankly seems completely unrelated to trying to make actual good experiences with an LLM.
BotW’s appeal is not the story. It’s definitely not the dialogue. It’s about exploring a carefully planned world full of goodies. Not a shitty LLM procedurally generated one that goes forever. And DEFINITELY not a run time generated one.
BotW’s appeal is not the story. It’s definitely not the dialogue. It’s about exploring a carefully planned world full of goodies. Not a shitty LLM procedurally generated one that goes forever. And DEFINITELY not a run time generated one.
BotW suffers for the NPCs having no agency or dimension to them. Why can't I infiltrate the Yiga clan and bend them to my will to help me defeat Ganon? Why can't I teach the villagers in TotK how to fight and fend for themselves against the pirates? There's a whole world that just needs a spark of agency to really live.
I’m honestly just mad listening to these dumb fuck takes. I’m done. I’m tilted.
This is such a stupidly thought out vision. BotW is a beautiful game that emphasizes minimalism and simplicity and you’re wanting to cram shit like this in.
You can’t do those things because that’s not what the game is about. And if you could do those things, it would be a different fucking game, and the magicalness of the world and the ingenuity of its game mechanics would become nothing compared to your “limitless player narrative agency”
I think your takes suck and have no basis in the reality of plausible to implement technology but this one really crosses the line of just being irritating to imagine asking for. This is the polar opposite of BotW’s design thesis and I hate you for tainting it.
This is such a stupidly thought out vision. BotW is a beautiful game that emphasizes minimalism and simplicity and you’re wanting to cram shit like this in.
You can’t do those things because that’s not what the game is about. And if you could do those things, it would be a different fucking game, and the magicalness of the world and the ingenuity of its game mechanics would become nothing compared to your “limitless player narrative agency”
I think your takes suck and have no basis in the reality of plausible to implement technology but this one really crosses the line of just being irritating to imagine asking for. This is the polar opposite of BotW’s design thesis and I hate you for tainting it.
I would have thought so too, until about 2023.
But, if recent attitudes to AI are anything to go by, this will be immediately seen as boring and derivative at best, or at worst people will accuse the back stories as having been "stolen" from other works of fiction.
While my expectations for the capabilities of AI have never been higher, my expectations for how the public will perceive this amazing technology has never been lower. It's all very disappointing.
And that's without specifically placing blame by the way, perhaps it's big techs fault for going about this the wrong way, I don't know, but the public blowback against AI is just making working in this field that I used to love super depressing.
But, if recent attitudes to AI are anything to go by, this will be immediately seen as boring and derivative at best, or at worst people will accuse the back stories as having been "stolen" from other works of fiction.
While my expectations for the capabilities of AI have never been higher, my expectations for how the public will perceive this amazing technology has never been lower. It's all very disappointing.
And that's without specifically placing blame by the way, perhaps it's big techs fault for going about this the wrong way, I don't know, but the public blowback against AI is just making working in this field that I used to love super depressing.
My 2 cents on why I think at the moment that I would find AI-driven games boring in the future (to clear confusion I think games are art):
- Amazing art is rare. Finding this amazing art nowadays is hard, with AI generated mumbojumbo increasing the noise it will be harder. It will be easier to just skip all AI generated stuff to find the real good stuff.
- Art has human touch or a message almost always in it. Purely AI generated stuff lacks meaning, because there's no creator.
- Socially sharing experiences that are unique only to you will be hard. I like to talk with my friends about the newest Nolan film or about Elden Ring's best bosses. If the whole game experience would be unique to me I wouldn't be able to relate with my friends at all and it would be the same as playing a different game.
- Valuing human work. I idolize people who have the skills to make great art. AI takes away from it.
But to these points, I can also see the future where
- AI could generate amazing art a lot, and would make me reconsider my categorical exclusion of AI art.
- AI could make it seem there's meaning behind the art, or an artist would use the AI as a tool to convey his meaning. - Unique game experiences would be so wild that you would be sharing them and relating via that way.
- AI becomes so common that valuing human skills isn't important anymore
Last point, I think novelty is something I value a lot and that's why in my opinion amazing art is rare. If AI would start creating a lot of novel art, novelty and rarity would just become common, and common is boring.
- Amazing art is rare. Finding this amazing art nowadays is hard, with AI generated mumbojumbo increasing the noise it will be harder. It will be easier to just skip all AI generated stuff to find the real good stuff.
- Art has human touch or a message almost always in it. Purely AI generated stuff lacks meaning, because there's no creator.
- Socially sharing experiences that are unique only to you will be hard. I like to talk with my friends about the newest Nolan film or about Elden Ring's best bosses. If the whole game experience would be unique to me I wouldn't be able to relate with my friends at all and it would be the same as playing a different game.
- Valuing human work. I idolize people who have the skills to make great art. AI takes away from it.
But to these points, I can also see the future where
- AI could generate amazing art a lot, and would make me reconsider my categorical exclusion of AI art.
- AI could make it seem there's meaning behind the art, or an artist would use the AI as a tool to convey his meaning. - Unique game experiences would be so wild that you would be sharing them and relating via that way.
- AI becomes so common that valuing human skills isn't important anymore
Last point, I think novelty is something I value a lot and that's why in my opinion amazing art is rare. If AI would start creating a lot of novel art, novelty and rarity would just become common, and common is boring.
Shirley Bassey and the Propellorheads got it right: It's All Just A Little Bit Of History Repeating. [1]
Writing[2], Print, Photography, Movies, Electronic music, Synths, Sampling, Trackers, Digital Photography, Photoshop, Ray Tracing, Interactive Art (eg computer games) etc etc ...
Every time "it's not really art" , "there's no human touch".
Every time, after a while artists learned to use it.
And they made amazing art!
[1] https://www.youtube.com/watch?v=yzLT6_TQmq8 Propellerheads feat: Miss Shirley Bassey - History Repeating
[2] Surely I'm kidding about the writing, right? https://fs.blog/an-old-argument-against-writing/
Writing[2], Print, Photography, Movies, Electronic music, Synths, Sampling, Trackers, Digital Photography, Photoshop, Ray Tracing, Interactive Art (eg computer games) etc etc ...
Every time "it's not really art" , "there's no human touch".
Every time, after a while artists learned to use it.
And they made amazing art!
[1] https://www.youtube.com/watch?v=yzLT6_TQmq8 Propellerheads feat: Miss Shirley Bassey - History Repeating
[2] Surely I'm kidding about the writing, right? https://fs.blog/an-old-argument-against-writing/
> an artist would use the AI as a tool to convey his meaning.
I'm not sure why this is shoved in as an aside near the end of the comment. This is how it's used. This is how it'll be used to create games.
Does anyone think that, what, ChatGPT is going to poop out an entire finished game and we'll release that? Or would artists use AI as just another tool to create their art?
I'm not sure why this is shoved in as an aside near the end of the comment. This is how it's used. This is how it'll be used to create games.
Does anyone think that, what, ChatGPT is going to poop out an entire finished game and we'll release that? Or would artists use AI as just another tool to create their art?
> I'm not sure why this is shoved in as an aside near the end of the comment. This is how it's used. This is how it'll be used to create games.
No one who is good at what they do wants to do this. Everyone who is bad at what they do is doing this. I don't know why tech people think they understand the art industry at all.
No one who is good at what they do wants to do this. Everyone who is bad at what they do is doing this. I don't know why tech people think they understand the art industry at all.
I'm sorry, I forgot that tech people can't do art and therefore can't have an opinion on it.
man if only i used a specific word to give a narrow context to my statement oh well
What context did I miss exactly that would change the fact that people can work in multiple industries at once, or can change industries over time, or the large overlap between "tech" and "art" these days?
>Does anyone think that, what, ChatGPT is going to poop out an entire finished game and we'll release that?
Well if this trajectory continues upwards, it almost certainly will happen in the next decade.
Right now feels like how the 70's-80's felt with the rise of the microprocessor. Tons of promise and implementation ideas, but the tech just wasn't there yet. But sure as hell it came around.
Well if this trajectory continues upwards, it almost certainly will happen in the next decade.
Right now feels like how the 70's-80's felt with the rise of the microprocessor. Tons of promise and implementation ideas, but the tech just wasn't there yet. But sure as hell it came around.
Unfortunately, creative and interesting use of the technology is both not how its being portrayed and also, critically, not how its being marketed and/or used.
The current expectation is that these models allow the creation of equivalent artwork to what artists are currently producing, but at larger scale / lower cost / shorter training. All of these boil down to less effort. And perceived effort is a large part of what brings value to art.
The current expectation is that these models allow the creation of equivalent artwork to what artists are currently producing, but at larger scale / lower cost / shorter training. All of these boil down to less effort. And perceived effort is a large part of what brings value to art.
Imagine having NPCs with prompt set character traits and set boundaries of knowledge of the game world.
The player can interact freely with the NPC rather than through predetermined conversation decision trees, potentially driving a lot of unique emergent gameplay.
Downsides are that it is a lot quicker just to skip through conversations with the decision tree format and having to think about and type in questions could get tiresome. Another downside is players jailbreaking the LLM...
The player can interact freely with the NPC rather than through predetermined conversation decision trees, potentially driving a lot of unique emergent gameplay.
Downsides are that it is a lot quicker just to skip through conversations with the decision tree format and having to think about and type in questions could get tiresome. Another downside is players jailbreaking the LLM...
Yes, I look forward to not have the NPC repeat the same exact thing every time, as you pass them by.
How much framerate are you willing to give up so that NPCs don't repeat themselves? The elephant in the room with LLMs in games is that the users GPU is usually already being maxed out by pushing pixels, so any amount of compute allocated to running AI models instead is going to have a very tangible cost.
They could be offloaded to the cloud instead but that brings its own set of issues (ongoing upkeep costs, finite capacity, servers will inevitably shut down...)
They could be offloaded to the cloud instead but that brings its own set of issues (ongoing upkeep costs, finite capacity, servers will inevitably shut down...)
> The elephant in the room ...
For now. LLMs are still relatively new. The technology will continue to be optimized, and we'll be shipping tiny LLMs along with games soon enough.
(Aren't companies already running LLMs on phones?)
For now. LLMs are still relatively new. The technology will continue to be optimized, and we'll be shipping tiny LLMs along with games soon enough.
(Aren't companies already running LLMs on phones?)
The housing market will only go up, it's always gone up.
I can only imagine nvidia/amd pouncing on the chance to sell companion "AI Cards" that enable LLM NPCs in games. If the tech is good (and insane progress in tiny local models points to "yes"), then I can easily imagine a large contingent of gamers forking over another $600 for a second GPU.
It might not even be necessary though, similar to how physixs (sp?) cards died out quick in the early 00's, the models might become slim enough and the GPU's fast enough that the extra load is easily manageable.
It might not even be necessary though, similar to how physixs (sp?) cards died out quick in the early 00's, the models might become slim enough and the GPU's fast enough that the extra load is easily manageable.
The vast majority of games need to run on consoles to be financially viable, anything that requires an entire second GPU is a complete non-starter. The only way it's going to work is if the models get small enough to run on a negligible slice of GPU or CPU time, with a reasonable memory footprint (<1GB), on whatever the current console hardware happens to be.
I see your point, but it hasn't really stopped novel features from coming to PC in the past. The PC market is about 50% larger than the console market too.
Console manufacturers would salivate at the thought of new hardware that makes people want to buy a new console instead of playing the ones they've already got. Once someone gets it working reasonably on PC, it will quickly spread to a new generation of consoles with new hardware that makes people want to upgrade.
Instead you'll just get variations on "Sorry, but as a large language model trained by OpenAl, I cannot condone the shooting of adventurers in the knees with arrows."
Any production deployment would probably do pre-fetching of potential initial dialogues as some kind of background job.
On this note - and as a response to the point a commenter made that too much freedom in dialogue might actually be a bad thing - LLMs could then be used to expand on the initial dialogue in a way that feels more 'organic' every time you hear it, but is still close to the artists' vision.
Now just imagine what we could do with a purpose-built ML model!!!
It could also work well for modding, for example, creating a natural dialogue system in Morrowind (replacing the vanilla "interactive signpost" NPCs).
I don't believe this is going to actually make it into games I have any interest in playing. Ubisoft might include it for a bit in some awful games, but it will fizzle out quick.
For the sake of argument, let's imagine it actually did make it into some game you'd want to play. It's well known that the quality of generation from these LLM's is heavily dependent on the quality of the text you put into it. Does that mean the quality of the game narrative is now going to be dependent on how I play it? Will normal people who just like to mess about get a worse game experience than the guy following the traffic laws in GTA? What's the creative talent even doing in that world?
For the sake of argument, let's imagine it actually did make it into some game you'd want to play. It's well known that the quality of generation from these LLM's is heavily dependent on the quality of the text you put into it. Does that mean the quality of the game narrative is now going to be dependent on how I play it? Will normal people who just like to mess about get a worse game experience than the guy following the traffic laws in GTA? What's the creative talent even doing in that world?
In contrast to runtime stuff (like an AI dungeon master) I've long wished for authoring tools to help people create rich game-worlds and quest interactions. On reflection, that's something I wouldn't expect LLMs to deliver either: It relies heavily on modeling cause-and-effect, with choices which are blocked or enabled by other choices.
I'm talking about stuff that could throw up a warning like: "Warning: Broken quest steps in For The Want of a Nail. Invariant Character:Blacksmith may have been killed by 5 other paths."
I'm talking about stuff that could throw up a warning like: "Warning: Broken quest steps in For The Want of a Nail. Invariant Character:Blacksmith may have been killed by 5 other paths."
I wonder if models trained on the real world will unintentionally destroy the game experience for people. In (NPC structured) video games things always work out ideally, you invest and you get a return. Any risk is illustrated for the user.
With LLM training, the NPC might take a more realistic approach and inform you that 4 of 6 the beetle nuts you collected for them are cracked and useless, and next time you should be more careful packing them in your backpack.
With LLM training, the NPC might take a more realistic approach and inform you that 4 of 6 the beetle nuts you collected for them are cracked and useless, and next time you should be more careful packing them in your backpack.
> authoring tools to help people create rich game-worlds and quest interactions
I built one of those things. It used random generation from datasets to create everything from gods and their pantheons to their followers, cities, and the cultures that make up the cities.
None of it used LLMs and it created better content than any model could, but it was hard to debug and harder still to figure out "What even is a culture?"
I built one of those things. It used random generation from datasets to create everything from gods and their pantheons to their followers, cities, and the cultures that make up the cities.
None of it used LLMs and it created better content than any model could, but it was hard to debug and harder still to figure out "What even is a culture?"
That's a more interesting usecase. But what you're describing is just an IDE, there's nothing AI about that.
There's a bigger (but related) problem than text quality:
With a usual game environment, the devs create a whitelist of actions the player is allowed to perform. If there's no combat, they simply don't give the player the option to perform violent actions. If they don't want the player to vault barriers they just don't include a jump button. Vehicles and buildings are props by default and only become intractable because the devs consciously chose to make them so. Devs have generally gotten very good at implying what's out of scope, so players generally don't even attempt actions that are intended to be impossible.
In LLM world with free-text entry, controlling what's on-theme becomes a blacklist rather than a whitelist, as the player can choose to input anything. Including things for flavour or lore reasons now becomes more difficult, as a whitelist is finite, while the blacklist of undesired actions is practically infinite.
With a usual game environment, the devs create a whitelist of actions the player is allowed to perform. If there's no combat, they simply don't give the player the option to perform violent actions. If they don't want the player to vault barriers they just don't include a jump button. Vehicles and buildings are props by default and only become intractable because the devs consciously chose to make them so. Devs have generally gotten very good at implying what's out of scope, so players generally don't even attempt actions that are intended to be impossible.
In LLM world with free-text entry, controlling what's on-theme becomes a blacklist rather than a whitelist, as the player can choose to input anything. Including things for flavour or lore reasons now becomes more difficult, as a whitelist is finite, while the blacklist of undesired actions is practically infinite.
I don't quite buy that contrast. I agree as long as we are talking story. The developers will have a very difficult time trying to control what some character is like, and if that matters for the gameplay it's a big problem. On the gameplay front the available actions will always be a whitelist, because somebody has to implement them. It's essentially the same problem the crypto kids had with their "cross game non-fungible items", where they just assumed that items would somehow be portable. Just because you can get some NPC to tell you about a dragon in the nearby cave doesn't make that cave actually appear. You're never going to get to fight a darksouls boss in forza horizon. The game might tell you that you get to, that doesn't make it true.
That's still a problem though, right?
If the NPCs are writing cheques that the engine can't cash, the player won't necessarily be able to trust any NPC dialogue.
If the NPCs are writing cheques that the engine can't cash, the player won't necessarily be able to trust any NPC dialogue.
Yes. You won't be able to trust what they say in relation to the story, not to the actuality of the gameplay. They're dialogue with essentially be orthogonal to play. Talk about ludo-narrative dissonance.
Something related is compellingly explored in an interesting trilogy of LitRPG comedy books called the Vaudevillain series, consisting of Top Hat Express, Black Tie Villainy, and Cane Whirling Lunacy. In the series, there is a VR video game called “World of Supers” where players can type in any superpower they can dream of into a game-master LLM, and the LLM will then create a character for them which has that power, but reworked a little bit to fit into game balance. For example, the player could type “I can stop time and still move around and take actions for up to 90 seconds” and the game-master LLM may approve this power but alter 90 seconds to 10 seconds, and give the player a major weakness, such as “However, stopping time in the vicinity of a gamma ray source will result in immediate loss of 75% of HP due to the interaction of the time stop with the radiation’s effects. This HP loss can be mitigated if you are encased in a significant amount of lead, such as a lead vehicle or heavy lead armor.” It's a pretty neat concept for how LLMs could allow custom player abilities while maintaining game balance. That way, player agency extends to actual powers/abilities, not just dialogue.
That would of course require an LLM to understand game balance, as opposed to just writing things that sound like they might balance a hypothetical game. At that point it's less similar to our LLMs and more like an AGI. And at that point we may as well have it create the neurotech necessary for the sort of VR tech I assume is used in the referenced books' world.
Agree. This point is a big concern and is explored in the stories (the protagonist creates a "mad scientist" character that can use the LLM to invent new equipment, and then uses a lot of trial and error to find prompts that generate insanely overpowered equipment). It sounds fun to play though! Balance doesn't have to be .. well .. balanced if there are other ways to maintain fun, such as leaning heavily into the creativity & self-actualization angle. As an example, if I could have any superpower in real life, I'd love to be able to fly, even though that's actually a very weak power compared to things like teleporting that are mostly a superset of flying, but not as fun and whimsical.
Encouraging "living your dream" rather than being the most powerful might be easier to achieve in a game with coop or open-world objectives rather than one focused on PvP, but if "living your dream" could be compelling made into the primary player objective, it would pair very well with the custom powers mechanic envisioned in the story.
As an example of how this is explored in the story, one of the protagonist's friends who loves surfing has a character whose superpower is "the ability to surf anything". It's an extremely weak power but the character really loves it because they get to surf the most extreme things imaginable, just for the fun of it, including in outlandish scenarios such as surfing along the blast wave of an explosion, or even the beam of a laser - silly stuff that is fun to explore & try.
I think it wouldn't have to be VR to be fun - imagine if it was something like a Roblox-based game played on laptops. Even then, the "surf anything" power and other outlandish powers would be incredibly fun to use. Imagine if you could do almost anything! You don't have to be the most powerful player in the world to have fun. Maybe you just really want to pilot a mecha-Godzilla, or something - even though an Iron Man or Thor level super could smash it in seconds. It's still awesome!
Encouraging "living your dream" rather than being the most powerful might be easier to achieve in a game with coop or open-world objectives rather than one focused on PvP, but if "living your dream" could be compelling made into the primary player objective, it would pair very well with the custom powers mechanic envisioned in the story.
As an example of how this is explored in the story, one of the protagonist's friends who loves surfing has a character whose superpower is "the ability to surf anything". It's an extremely weak power but the character really loves it because they get to surf the most extreme things imaginable, just for the fun of it, including in outlandish scenarios such as surfing along the blast wave of an explosion, or even the beam of a laser - silly stuff that is fun to explore & try.
I think it wouldn't have to be VR to be fun - imagine if it was something like a Roblox-based game played on laptops. Even then, the "surf anything" power and other outlandish powers would be incredibly fun to use. Imagine if you could do almost anything! You don't have to be the most powerful player in the world to have fun. Maybe you just really want to pilot a mecha-Godzilla, or something - even though an Iron Man or Thor level super could smash it in seconds. It's still awesome!
Finally we can have interesting games with real meaningful choices that actually change the narrative of the game. Modern MMORPGs are barely better than the Choose Your Own Adventure books.
Every time I watch Chris Crawford's Dragon Speech: https://www.youtube.com/watch?v=VwZi58u1FjI I think about what could've been, and what games actually turn out to be.
Honestly, if the next The War Within (next World of Warcraft expansion) makes me do 10 daily quests per day again, I will cancel my subscription for the first time since 2005.
Every time I watch Chris Crawford's Dragon Speech: https://www.youtube.com/watch?v=VwZi58u1FjI I think about what could've been, and what games actually turn out to be.
Honestly, if the next The War Within (next World of Warcraft expansion) makes me do 10 daily quests per day again, I will cancel my subscription for the first time since 2005.
As my kids got some recent Choose Your Own Adventure books, I'd say that the quests in modern MMORPGs tend to be significantly worse than that.
You need focus and direction in any media. Books, movies, games, etc.... they all need it.
The first NPC that you encounter with such freedom of interaction is interesting, the 45th will be annoying and you'll just want to get to the point. The holodeck of Star Trek is often brought up as an excellent example of AI generated interactions, but people seem to be forgetting that the stories we see unfold on TV aren't random interactive stories. They're scripted stories written by show's writers.
Unless you want to make a simulation without a real goal, I don't see how this could lead to fun and interesting gameplay past the first hour. Realism is, always will be, a bad gameplay pillar just for the sake of it.
The first NPC that you encounter with such freedom of interaction is interesting, the 45th will be annoying and you'll just want to get to the point. The holodeck of Star Trek is often brought up as an excellent example of AI generated interactions, but people seem to be forgetting that the stories we see unfold on TV aren't random interactive stories. They're scripted stories written by show's writers.
Unless you want to make a simulation without a real goal, I don't see how this could lead to fun and interesting gameplay past the first hour. Realism is, always will be, a bad gameplay pillar just for the sake of it.
There is a group of orc kids running around in Orgrimmar, and sometimes they run around you and are very annoying, why can't I just trip one.
I had a friend who didn't want to fight against the Lich King. When we finally got to him in the raid, at the start of the fight he kneeled in front of him and died (caused us unnecessary wipe, but.. was cool).
I am not thinking about 'AI generated interactions', and I think the AI can create compelling story that you go through. Kind of like in Sword Art Online or Shangri-La Frontier's quests.
I had a friend who didn't want to fight against the Lich King. When we finally got to him in the raid, at the start of the fight he kneeled in front of him and died (caused us unnecessary wipe, but.. was cool).
I am not thinking about 'AI generated interactions', and I think the AI can create compelling story that you go through. Kind of like in Sword Art Online or Shangri-La Frontier's quests.
> why can't I just trip one.
Some of the best writing and experiences I've ever had gaming have never come from the ability to do random acts that are worthwhile to noone.
> I had a friend who didn't want to fight against the Lich King. When we finally got to him in the raid, at the start of the fight he kneeled in front of him and died (caused us unnecessary wipe, but.. was cool).
Man that sounds cringe, not cool.
Some of the best writing and experiences I've ever had gaming have never come from the ability to do random acts that are worthwhile to noone.
> I had a friend who didn't want to fight against the Lich King. When we finally got to him in the raid, at the start of the fight he kneeled in front of him and died (caused us unnecessary wipe, but.. was cool).
Man that sounds cringe, not cool.
It surely can generate something better than the old "Fetch this item and come back" quests, but I wouldn't say the possibilities are endless, so eventually you would get bored of the same repeating AI quests the same way as you do now.
Good storytelling is hard and AI is not some magic bullet that can just solve it for good. But it can help raise the floor of the unimportant side quests.
Good storytelling is hard and AI is not some magic bullet that can just solve it for good. But it can help raise the floor of the unimportant side quests.
I've also done research in the use of LLMs for RPGs, using Disco Elysium as a testbed (rather than the small made up scenario seen in this paper).
https://aclanthology.org/2023.findings-emnlp.151/
https://pl.aiwright.dev/
https://aclanthology.org/2023.findings-emnlp.151/
https://pl.aiwright.dev/
Ah, my friend created guides for soloing (or play with a friend without a GM) TTRPGs using ChatGPT, here's his site: https://oracle-rpg.com/
The Discord is pretty active too.
I forwarded him this study, he'll appreciate it!
The Discord is pretty active too.
I forwarded him this study, he'll appreciate it!
The criticisms people are laying out about being able to say anything making games boring would apply to D&D as well. Since D&D is really fun despite the freedom to (attempt) to say or do whatever your imagination can dream up means it's at least plausible that a game with generative dialogue and narrative could work well.
Just because you're free to speak doesn't mean NPCs will listen, nor does it mean there aren't rules or that there won't be consequences.
Just because you're free to speak doesn't mean NPCs will listen, nor does it mean there aren't rules or that there won't be consequences.
D&D is fun because it is a social experience. There’s also a level of expected player etiquette. If you want to go off the rails and completely ignore the party and plot, you’ll probably be kicked out of the group. D&D is fun, but the vast majority of campaigns are cut short and unfinished.
A game with generative dialogue could work well. But one should expect that slapping it into most games will work horribly.
A game with generative dialogue could work well. But one should expect that slapping it into most games will work horribly.
Slapping any mechanic in without thought rarely produces a good game.
> If you want to go off the rails and completely ignore the party and plot, you’ll probably be kicked out of the group.
Sounds like a great place an AI can fill in as is DM for otherwise toxic players.
> If you want to go off the rails and completely ignore the party and plot, you’ll probably be kicked out of the group.
Sounds like a great place an AI can fill in as is DM for otherwise toxic players.
The real bummer is that if you want to get financing to make a game right now, putting a PDF onto arxiv.org will get you closer to that goal than actually making a game.
I have not sat down and thought through how LLMs would be made use of in the World of Baldurs Gate 3. But when I do, my mind will be boggled.
It will surely also mean always online games with a hefty subscription fee.
Unless!!! The llm can run locally on the player gpu... Whoa mind boggled. This is coming, I'm sure of it.
/end internal thought process
It will surely also mean always online games with a hefty subscription fee.
Unless!!! The llm can run locally on the player gpu... Whoa mind boggled. This is coming, I'm sure of it.
/end internal thought process
Can you write a single line of dialogue that you think would make the game better?
All LLMs in that regime can be run on mid-range gaming hardware today.
LLMs run off-prem/off-desk are already niche for enthusiasts. You need Opus or 0125 sometimes, but not often.
Apple just announced their intention to make running them in the cloud niche generally, at least for mundane consumer use cases, work out of the box.
LLMs run off-prem/off-desk are already niche for enthusiasts. You need Opus or 0125 sometimes, but not often.
Apple just announced their intention to make running them in the cloud niche generally, at least for mundane consumer use cases, work out of the box.
Good job. More LLMs in Games please!
A lot of the dialogue that OpenAI's models write is incredibly bland.. I really think we'll need less censored models trained on how to act different roles other than just 'super safe and friendly assistant'.
Not as in depth as the abstract, but I found this to be a well integrated use of LLM's into a game: https://youtu.be/sUL0Z7PwanY
ux-app(1)
Hey, this currently has a bug but I want to share here.
The framework should be clear.
I’ve written a React pattern for writing interactive fiction using LLMs.
Here’s a demo with Taskmaster roleplay implemented.
https://github.com/LEXNY/Taskmaster-LLM/blob/main/main.js
The framework should be clear.
I’ve written a React pattern for writing interactive fiction using LLMs.
Here’s a demo with Taskmaster roleplay implemented.
https://github.com/LEXNY/Taskmaster-LLM/blob/main/main.js
[deleted]
Very exciting times ahead in game design!