My favorite part of the experiment was players realizing that suddenly city guards (one of the types of NPCs in the experiment) would actually react to what you say. Previously, unless you attacked one, or were an enemy of that city, they'd leave you alone. Now, you could say something offensive or disrespectful, completely free-form vs. dialog trees or something, and drive them to violence if they get pissed off about it. It's hilarious and absolutely amazing at the same time.
We even had an NPC spontaneously give a player a quest that would send them to the other side of the world in an apparent attempt to get the player to go away and stop bothering it.
Now we're working on a more permanent design, in which we'll be able to scale all sorts of personality traits that NPCs can have so they'll respond in line with their personalities.
Emphasis mine, because wow. See some of the pretty impressive conversations with ChatGPT-powered NPCs below!
Aetolia is actually a text-based MUD, but Matt tells me this approach could be scaled to run in modern, graphics-based MMOs — with some serious qualifiers.
"I don't see any particular reason it couldn't be scaled from a technical standpoint. You might need to create many accounts with ChatGPT to power a huge number of NPCs, but that's not an inherent barrier. But from a practical usability standpoint, there are probably two concerns there:
"In a graphical game it's expected that you'll see an NPC doing what it's doing. In a MUD, you're just describing it, so the NPC can do anything plausible and no animations are required. 'Saying' something is not really different from acting something out in the sense that it's both just strings of text, which ChatGPT is good for.
"The other big concern is the cost. With ChatGPT, you pay by the token (a token is about 4 text characters)." At the moment, Version 3.5 of ChatGPT is much cheaper to run than version 4.0, but, adds Matt, "4.0 handles extensive backstory/world context much better than 3.5."
GPT has no long-term memory, so to get a good reponse, each API call has to include the character and world's entire backstory. "You can see how that can quickly become potentially expensive, depending on how much info you want to feed it."
For example, say you want the world's city guards to generate dynamic responses via ChatGPT. "That works great until someone asks them about something they don't know, of course, at which point they seem hopelessly ignorant or they just make up answers."
Stereotypically, the average city guard probably isn't very knowledgeable about their world.
"But, if I wanted to make an NPC who knows everything about the game and game world, it'd be essentially impossible. Impossibly expensive because of the amount of info we'd have to pass each time, but also, GPT 4.0 only has a 'context window' of about 8000 words, meaning that if you tell it your name and then spit 8000-ish more words at it, it will forget your name, even in the same API call.
"But," Matt adds, "our implementation here is extremely basic, and there are certainly more sophisticated ways to route queries to generative text AIs. Inworld.AI is a company developing this as a service that looks potentially interesting. If they can do what they say, they add a lot of contextual memory to NPCs compared to what we did with our fairly rudimentary experiment. Its promise, to me, is shown by how much can be achieved so easily."
Another problem is when ChatGPT "hallucinates" in a virtual world (i.e. responds with nonsense), it implies something wrong about the world itself.
"So for instance, when it sent a player on a quest, it was for an item there not instantiated in the game world. We could certainly do special case exceptions and give it lists of instantiated items it could choose to include, but ultimately that’s a very limited patchwork and incomplete way of addressing the vast range of things the AI might talk about."
Amazing stuff, especially for a first attempt. Here's a sample of the info Matt and his team incorporated into the API calls, along with the actual conversations that followed:
