Writeup: https://medium.com/@sean_skimmer/npc-gpt-3f4cb5272773

Leveraging large language models for video games and NPCs. This proof of concept is built out for the game Stardew Valley. The project is made up of 3 parts, described below.

To generate dynamic and unique NPC's, the first model generates a short character bio. The model is based on OpenAI's open source GPT2LMHeadModel. The model was fine tuned using sample character bios that were generated using similar formatting and description styles as the Stardew Valley villager wiki pages. The sample bios used for training data can be found at data/character_bios.csv. The generated character bio is then cleaned and processed to be used as input for the dialogue model.

The cleaned personalities generated by the model above are fed into the dialogue model to create conversations that are both relevant to the game and personality. For this step, the knowledge base of objects in the game was created to ensure the dialogue surrounded relevant items, mobs, and locations. Training data was created with the ChatGPT API created by OpenAI, using prompts that returned item definitions, quests, and general dialogue. This replaced the original method which used the original game dialogue, scraped with VideoGameDialogueCorpusPublic, as the original dialogue was not comprehensive enough to achieve substantial results. Multiple models were trained with guidance from this Medium article, using DialoGPT and GPT2 as base models. The outputs of this model are run through part 3 to determine if there is intent related to a conversation.

To capture the transactional intention of dialogue with the NPC, two Spacy Matchers are used to identify and pull the relevant information from the dialogue. For example, if the user asks the NPC for a quest, the NPC might give the user an item quest or a mob related quest. The Matcher rules can identify that a quest has been given to the user and extract the target items (or mobs), the quantity requested, and the potential reward offered to the user upon completion of the request. This information is used to dynamically implement the transactions in the game, allowing for a more robust and complete user experience.