Creating Tabletop RPG Dialogues via Retrieval-Augmented Generation

• Gabriel Rudan Sales Matos UFC
• José Wellington Franco da Silva UFC
• Artur de Oliveira da Rocha Franco UFC
• José Gilvan Rodrigues Maia UFC
• José Antônio Fernandes de Macêdo UFC

DOI: https://doi.org/10.5753/sbgames.2025.10008

Resumo

Introduction: Tabletop RPGs (TRPGs) rely heavily on narrative, but Game Masters face challenges creating coherent and engaging dialogues while managing extensive rulebooks. Objective: This work investigates the adoption of Retrieval-Augmented Generation (RAG) to improve dialogue generation in TRPGs using Large Language Models. Methodology or Steps: Four dialogues were analyzed: two handwritten and two generated by LLMs—with and without RAG. Eleven participants rated them across five criteria—Engagement, Coherence, Cohesion, Creativity, and Surprise. Results: RAG-based generation outperformed standard LLMs across all categories, improving coherence (+0.18), cohesion (+0.46), creativity (+0.55), engagement (+0.91), and surprise (+0.64). Compared to handwritten dialogues, generated ones were rated higher in cohesion (3.91 vs. 3.68) and matched in coherence (3.82), although handwritten dialogues remained superior in engagement (3.82 vs. 3.09) and creativity (3.68 vs. 3.14).

Palavras-chave: TRPG, Dialogue Generation, RAG, LLM

Referências

Ashby, T., Webb, B. K., Knapp, G., Searle, J., e Fulda, N. (2023). Personalized quest and dialogue generation in role-playing games: A knowledge graph-and language model-based approach. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems, pages 1–20.

Bai, G., Chai, Z., Ling, C., Wang, S., Lu, J., Zhang, N., Shi, T., Yu, Z., Zhu, M., Zhang, Y., et al. (2024). Beyond efficiency: A systematic survey of resource-efficient large language models. arXiv preprint arXiv:2401.00625.

Barton, M. e Stacks, S. (2019). Dungeons and desktops: The history of computer role-playing games 2e. AK Peters/CRC Press.

Bisong, E. e Bisong, E. (2019). Google colaboratory. Building machine learning and deep learning models on google cloud platform: a comprehensive guide for beginners, pages 59–64.

Blohm, I., Leimeister, J. M., e Krcmar, H. (2013). Crowdsourcing: How to benefit from (too) many great ideas. MIS quarterly executive, 12(4).

Bowman, S. L. (2007). The psychological power of the role-playing experience. Journal of Interactive Drama, 2(1):1–15.

Chiu, C.-M., Liang, T.-P., e Turban, E. (2014). What can crowdsourcing do for decision support? Decision Support Systems, 65:40–49.

Cover, J. G. (2014). The creation of narrative in tabletop role-playing games. McFarland.

Cozman, F. G. e Kaufman, D. (2022). Viés no aprendizado de máquina em sistemas de inteligência artificial: a diversidade de origens e os caminhos de mitigação. Revista USP, (135):195–210.

Csepregi, L. M. (2021). The effect of context-aware llm-based npc conversations on player engagement in role-playing video games. Unpublished manuscript.

da Rocha Franco, A. d. O., de Carvalho, W. V., da Silva, J. W. F., Maia, J. G. R., e de Castro, M. F. (2024). Managing and controlling digital role-playing game elements: A current state of affairs. Entertainment Computing, 51:100708.

Daniel, F., Kucherbaev, P., Cappiello, C., Benatallah, B., e Allahbakhsh, M. (2018). Quality control in crowdsourcing: A survey of quality attributes, assessment techniques, and assurance actions. ACM Computing Surveys (CSUR), 51(1):1–40.

Gao, Y., Xiong, Y., Gao, X., Jia, K., Pan, J., Bi, Y., Dai, Y., Sun, J., e Wang, H. (2023). Retrieval-augmented generation for large language models: A survey. arXiv preprint arXiv:2312.10997.

Kaplan, J., McCandlish, S., Henighan, T., Brown, T. B., Chess, B., Child, R., Gray, S., Radford, A., Wu, J., e Amodei, D. (2020). Scaling laws for neural language models. arXiv preprint arXiv:2001.08361.

Kluyver, T., Ragan-Kelley, B., Pérez, F., Granger, B., Bussonnier, M., Frederic, J., Kelley, K., Hamrick, J., Grout, J., Corlay, S., et al. (2016). Jupyter notebooks–a publishing format for reproducible computational workflows. In Positioning and power in academic publishing: Players, agents and agendas, pages 87–90. IOS press.

Lewis, P., Perez, E., Piktus, A., Petroni, F., Karpukhin, V., Goyal, N., Küttler, H., Lewis, M., Yih, W.-t., Rocktäschel, T., et al. (2020). Retrieval-augmented generation for knowledge-intensive nlp tasks. Advances in Neural Information Processing Systems, 33:9459–9474.

Lin, J. C., Younessi, D. N., Kurapati, S. S., Tang, O. Y., e Scott, I. U. (2023). Comparison of gpt-3.5, gpt-4, and human user performance on a practice ophthalmology written examination. Eye, 37(17):3694–3695.

Lin, X., Wang, W., Li, Y., Yang, S., Feng, F., Wei, Y., e Chua, T.-S. (2024). Data-efficient fine-tuning for llm-based recommendation. In Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 365–374.

Luu, Q. K., Deng, X., Van Ho, A., e Nakahira, Y. (2024). Context-aware llm-based safe control against latent risks. arXiv preprint arXiv:2403.11863.

Mäyrä, F. (2017). Dialogue and interaction in role-playing games. Dialogue across Media, 28:271.

Meyer, L.-P., Stadler, C., Frey, J., Radtke, N., Junghanns, K., Meissner, R., Dziwis, G., Bulert, K., e Martin, M. (2023). Llm-assisted knowledge graph engineering: Experiments with chatgpt. In Working conference on Artificial Intelligence Development for a Resilient and Sustainable Tomorrow, pages 103–115. Springer Fachmedien Wiesbaden Wiesbaden.

Moser, C. e Fang, X. (2014). Narrative structure and player experience in role-playing games. International Journal of Human-Computer Interaction, 31:146–156.

Nananukul, N. e Wongkamjan, W. (2024). What if red can talk? dynamic dialogue generation using large language models. arXiv preprint arXiv:2407.20382.

Nye, B. D., Mee, D., e Core, M. G. (2023). Generative large language models for dialog-based tutoring: An early consideration of opportunities and concerns. In LLM@ AIED, pages 78–88.

Patil, R. e Gudivada, V. (2024). A review of current trends, techniques, and challenges in large language models (llms). Applied Sciences, 14(5):2074.

Ray, P. P. (2023). Chatgpt: A comprehensive review on background, applications, key challenges, bias, ethics, limitations and future scope. Internet of Things and Cyber-Physical Systems, 3:121–154.

Rolim, F. M. (2023). Llms e ia generativa em jogos.

Singh, P. N., Talasila, S., e Banakar, S. V. (2023). Analyzing embedding models for embedding vectors in vector databases. In 2023 IEEE International Conference on ICT in Business Industry & Government (ICTBIG), pages 1–7. IEEE.

Tapscott, A., León, C., e Gervás, P. (2018). Generating stories using role-playing games and simulated human-like conversations. In Proceedings of the 3rd Workshop on Computational Creativity in Natural Language Generation (CC-NLG 2018), pages 34–42.

Tychsen, A., Hitchens, M., Brolund, T., e Kavakli, M. (2005). The game master. In ACM International Conference Proceeding Series, volume 123, pages 215–222.

van Stegeren, J. e Myśliwiec, J. (2021). Fine-tuning gpt-2 on annotated rpg quests for npc dialogue generation. In Proceedings of the 16th International Conference on the Foundations of Digital Games, pages 1–8.

Wang, X., Wang, Z., Gao, X., Zhang, F., Wu, Y., Xu, Z., Shi, T., Wang, Z., Li, S., Qian, Q., et al. (2024). Searching for best practices in retrieval-augmented generation. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 17716–17736.

Xian, J., Teofili, T., Pradeep, R., e Lin, J. (2024). Vector search with openai embeddings: Lucene is all you need. In Proceedings of the 17th ACM International Conference on Web Search and Data Mining, pages 1090–1093.

Xue, F., Fu, Y., Zhou, W., Zheng, Z., e You, Y. (2023). To repeat or not to repeat: Insights from scaling llm under token-crisis. Advances in Neural Information Processing Systems, 36:59304–59322.