Memory Architectures for Game Agents: A Systematic Mapping Study Bridging Classical and LLM Paradigms

  • Gabriel Machado Santos UFU
  • Rita Maria Silva Julia UFU
  • Marcelo Nascimento UFU

Resumo


Introduction: Two decades of research on memory for game agents have produced two largely disjoint traditions: classical symbolic game AI (blackboards, GOAP, behavior trees, case-based reasoning, Soar/ACT-R) and LLM-based agents (memory streams, MemGPT, A-MEM, CoALA, Voyager). Despite clear conceptual overlap, no prior survey covers both under a unified lens for game agents. Objective: We bridge these traditions through a systematic mapping that classifies, compares, and identifies open problems across both paradigms. Methodology: Following Petersen et al.’s guidelines, we surveyed 113 works from 1972–2025 across ACM DL, IEEE Xplore, Scopus, and arXiv, with forward and backward snowballing on seminal anchors. Each work was coded along five axes: temporal horizon, representation, lifecycle operations, grounding target, and agent role. Results: The mapping yields (i) a unified five-axis taxonomy for both paradigms, (ii) empty cells exposing under-served combinations (notably crowd-agent memory and explicit forgetting), (iii) an integration lens identifying what LLM systems reinvent from classical game AI, what they add, and what they lose, and (iv) six open problems framing a research agenda for NPC memory.
Palavras-chave: memory architecture, non-player characters, large language models, cognitive architectures, systematic mapping, game AI

Referências

Aha, D. W., Molineaux, M., e Ponsen, M. J. V. (2005). Learning to win: Case-based plan selection in a real-time strategy game. In Proc. ICCBR.

Anderson, J. R., Bothell, D., Byrne, M. D., Douglass, S., Lebiere, C., e Qin, Y. (2004). An integrated theory of the mind. Psychological Review, 111(4):1036–1060.

Ashby, T., Webb, B. K., Knapp, G., Searle, J., e Fulda, N. (2023). Personalized quest and dialogue generation in role-playing games: A knowledge graph- and language model-based approach. In Proc. ACM CHI.

Aylett, R., Louchart, S., Dias, J., Paiva, A., e Vala, M. (2005). FearNot! – an experiment in emergent narrative. In Proc. Intelligent Virtual Agents (IVA).

Baddeley, A. (2000). The episodic buffer: A new component of working memory? Trends in Cognitive Sciences, 4(11):417–423.

Baddeley, A. D. e Hitch, G. (1974). Working memory. In The Psychology of Learning and Motivation, volume 8, pages 47–89. Academic Press.

Baffa, A., Sampaio, M., e Feijó, B. (2017). Dealing with the emotions of non player characters. In Proc. SBGames.

Bates, J. (1994). The role of emotion in believable agents. Communications of the ACM, 37(7):122–125.

Callison-Burch, C., Tomar, G. S., Martin, L. J., Ippolito, D., Bailis, S., e Reitter, D. (2022). Dungeons and dragons as a dialog challenge for artificial intelligence. In Proc. EMNLP.

Champandard, A. J. e Dunstan, P. (2013). The behavior tree starter kit. In Rabin, S., editor, Game AI Pro: Collected Wisdom of Game AI Professionals, pages 73–92. CRC Press.

Colledanchise, M. e Ögren, P. (2017). How behavior trees modularize hybrid control systems and generalize sequential behavior compositions, the subsumption architecture, and decision trees. IEEE Transactions on Robotics, 33(2):372–389.

Colledanchise, M. e Ögren, P. (2018). Behavior Trees in Robotics and AI: An Introduction. CRC Press.

Corbett, A. T. e Anderson, J. R. (1995). Knowledge tracing: Modeling the acquisition of procedural knowledge. User Modeling and User-Adapted Interaction, 4(4):253–278.

Dias, J., Mascarenhas, S., e Paiva, A. (2014). FAtiMA Modular: Towards an agent architecture with a generic appraisal framework. In Emotion Modeling: Towards Pragmatic Computational Models of Affective Processes. Springer.

Dias, J. e Paiva, A. (2005). Feeling and reasoning: A computational model for emotional characters. In Proc. EPIA.

Gallotta, R., Todd, G., Zammit, M., Earle, S., Liapis, A., Togelius, J., e Yannakakis, G. N. (2024). Large language models and games: A survey and roadmap. IEEE Transactions on Games.

Gorski, N. A. e Laird, J. E. (2011). Learning to use episodic memory. Cognitive Systems Research, 12(2):144–153.

Guu, K., Lee, K., Tung, Z., Pasupat, P., e Chang, M.-W. (2020). REALM: Retrieval-augmented language model pre-training. In Proc. ICML.

Isla, D. (2005). Handling complexity in the Halo 2 AI. In Game Developers Conference (GDC).

Isla, D. (2008). Building a better battle: The Halo 3 AI objectives system. In Game Developers Conference (GDC).

Isla, D. e Blumberg, B. (2002). Object persistence for synthetic characters. In Proc. AAMAS.

Justesen, N., Bontrager, P., Togelius, J., e Risi, S. (2020). Deep learning for video game playing. IEEE Transactions on Games, 12(1):1–20.

Kallmann, M. e Thalmann, D. (1999). Direct 3D interaction with smart objects. In Proc. ACM VRST.

Kallmann, M. e Thalmann, D. (2002). Modeling behaviors of interactive objects for real-time virtual environments. Journal of Visual Languages and Computing, 13(2):177–195.

Karpukhin, V., Oğuz, B., Min, S., Lewis, P., Wu, L., Edunov, S., Chen, D., e Yih, W.-t. (2020). Dense passage retrieval for open-domain question answering. In Proc. EMNLP.

Kotseruba, I. e Tsotsos, J. K. (2020). 40 years of cognitive architectures: Core cognitive abilities and practical applications. Artificial Intelligence Review, 53:17–94.

Laird, J. E. (2001). It knows what you’re going to do: Adding anticipation to a Quakebot. In Proc. Int. Conf. on Autonomous Agents (AGENTS), pages 385–392.

Laird, J. E. (2012). The Soar Cognitive Architecture. MIT Press.

Laird, J. E. e Duchi, J. C. (2000). Creating human-like synthetic characters with multiple skill levels: A case study using the Soar Quakebot. In AAAI Fall Symposium on Simulating Human Agents.

Langley, P., Laird, J. E., e Rogers, S. (2009). Cognitive architectures: Research issues and challenges. Cognitive Systems Research, 10(2):141–160.

Lewis, P., Perez, E., Piktus, A., Petroni, F., Karpukhin, V., Goyal, N., Küttler, H., et al. (2020). Retrieval-augmented generation for knowledge-intensive NLP tasks. In Proc. NeurIPS.

Lim, M. Y., Dias, J., Aylett, R., e Paiva, A. (2012). Creating adaptive affective autonomous NPCs. Autonomous Agents and Multi-Agent Systems, 24:287–311.

Lima, E., Feijó, B., e Furtado, A. L. (2016). Player behavior modeling for interactive storytelling in games. In Proc. SBGames.

Loyall, A. B. (1997). Believable Agents: Building Interactive Personalities. PhD thesis, Carnegie Mellon University.

Loyall, A. B. e Bates, J. (1991). Hap: A reactive, adaptive architecture for agents. In CMU-CS-91-147. Carnegie Mellon University.

Madaan, A., Tandon, N., Gupta, P., Hallinan, S., Gao, L., Wiegreffe, S., et al. (2023). Self-Refine: Iterative refinement with self-feedback. In Proc. NeurIPS.

Marsella, S. C. e Gratch, J. (2009). EMA: A process model of appraisal dynamics. Cognitive Systems Research, 10(1):70–90.

Mateas, M. e Stern, A. (2002). A behavior language for story-based believable agents. IEEE Intelligent Systems, 17(4):39–47.

Mateas, M. e Stern, A. (2005). Structuring content in the Façade interactive drama architecture. In Proc. AIIDE.

Mehta, M., Ontañón, S., e Ram, A. (2009). Using meta-reasoning to improve the performance of case-based planning. In Proc. ICCBR.

Modarressi, A. et al. (2024). MemLLM: Finetuning LLMs to use an explicit read-write memory. arXiv preprint arXiv:2404.11672.

Mohtashami, A. e Jaggi, M. (2023). Landmark attention: Random-access infinite context length for transformers. In Proc. NeurIPS.

Munkhdalai, T., Faruqui, M., e Gopal, S. (2024). Leave no context behind: Efficient infinite context transformers with Infini-attention. arXiv preprint arXiv:2404.07143.

Nelson, M. J., Roberts, D. L., Isbell, C. L., e Mateas, M. (2006). Reinforcement learning for declarative optimization-based drama management. In Proc. AAMAS.

Nuxoll, A. M. e Laird, J. E. (2007). Extending cognitive architecture with episodic memory. In Proc. AAAI, pages 1560–1565.

Nuxoll, A. M. e Laird, J. E. (2012). Enhancing intelligent agents with episodic memory. Cognitive Systems Research, 17–18:34–48.

Ontañón, S., Mishra, K., Sugandh, N., e Ram, A. (2007). Case-based planning and execution for real-time strategy games. In Proc. ICCBR.

Ontañón, S., Mishra, K., Sugandh, N., e Ram, A. (2010). On-line case-based planning. Computational Intelligence, 26(1):84–119.

Ontañón, S., Synnaeve, G., Uriarte, A., Richoux, F., Churchill, D., e Preuss, M. (2013). A survey of real-time strategy game AI research and competition in StarCraft. IEEE Trans. on Computational Intelligence and AI in Games, 5(4):293–311.

Orkin, J. (2004). Applying goal-oriented action planning to games. In AI Game Programming Wisdom 2, pages 217–228. Charles River Media.

Orkin, J. (2006). Three states and a plan: The AI of F.E.A.R. In Game Developers Conference (GDC).

Orkin, J. e Roy, D. (2007). The restaurant game: Learning social behavior and language from thousands of players online. Journal of Game Development, 3(1):39–60.

Packer, C., Fang, V., Patil, S., Lin, K., Wooders, S., e Gonzalez, J. (2023). Memgpt: towards llms as operating systems.

Park, J. S., O’Brien, J. C., Cai, C. J., Morris, M. R., Liang, P., e Bernstein, M. S. (2023). Generative agents: Interactive simulacra of human behavior. In Proc. ACM UIST.

Petersen, K., Feldt, R., Mujtaba, S., e Mattsson, M. (2008). Systematic mapping studies in software engineering. In Proceedings of the 12th International Conference on Evaluation and Assessment in Software Engineering (EASE), pages 68–77.

Petersen, K., Vakkalanka, S., e Kuzniarz, L. (2015). Guidelines for conducting systematic mapping studies in software engineering: An update. Information and Software Technology, 64:1–18.

Riedl, M. O., Stern, A., Dini, D., e Alderman, J. (2008). Dynamic experience management in virtual worlds for entertainment, education, and training. Int. Trans. on Systems Science and Applications, 4(2):23–42.

Rowe, J. P., Shores, L. R., Mott, B. W., e Lester, J. C. (2011). Integrating learning, problem solving, and engagement in narrative-centered learning environments. International Journal of Artificial Intelligence in Education, 21(1–2):115–133.

Sharma, M., Ontañón, S., Mehta, M., e Ram, A. (2010). Drama management and player modeling for interactive fiction games. Computational Intelligence, 26(2):183–211.

Shinn, N., Cassano, F., Gopinath, A., Narasimhan, K., e Yao, S. (2023). Reflexion: Language agents with verbal reinforcement learning. Advances in neural information processing systems, 36:8634–8652.

Shute, V. J. e Ventura, M. (2013). Stealth Assessment: Measuring and Supporting Learning in Video Games. MIT Press.

Squire, L. R. (2004). Memory systems of the brain: A brief history and current perspective. Neurobiology of Learning and Memory, 82(3):171–177.

Sumers, T. R., Yao, S., Narasimhan, K., e Griffiths, T. L. (2024). Cognitive architectures for language agents. Transactions on Machine Learning Research.

Sweetser, P. e Rogers, K. (2024). Large language models and video games: A preliminary scoping review. In Proc. ACM Conversational User Interfaces (CUI).

Tulving, E. (1972). Episodic and semantic memory. In Tulving, E. e Donaldson, W., editors, Organization of Memory, pages 381–403. Academic Press.

Tulving, E. (1985). How many memory systems are there? American Psychologist, 40(4):385–398.

VanLehn, K. (2006). The behavior of tutoring systems. International Journal of Artificial Intelligence in Education, 16(3):227–265.

Wang, G., Xie, Y., Jiang, Y., Mandlekar, A., Xiao, C., Zhu, Y., Fan, L., e Anandkumar, A. (2024a). Voyager: An open-ended embodied agent with large language models. Transactions on Machine Learning Research.

Wang, Y. et al. (2025). From human memory to AI memory: A survey on memory mechanisms in the era of LLMs. arXiv preprint arXiv:2504.15965.

Wang, Z., Cai, S., Liu, A., Jin, Y., Hou, J., Zhang, B., Lin, H., et al. (2024b). JARVIS-1: Open-world multi-task agents with memory-augmented multimodal language models. IEEE Transactions on Pattern Analysis and Machine Intelligence.

Weber, B. G., Mateas, M., e Jhala, A. (2010). Applying goal-driven autonomy to StarCraft. In Proc. AIIDE.

Wu, Y., Min, S. Y., Prabhumoye, S., Bisk, Y., Salakhutdinov, R., Azaria, A., Mitchell, T., e Li, Y. (2023). SPRING: Studying papers and reasoning to play games. In Proc. NeurIPS.

Xu, W., Liang, Z., Mei, K., Gao, H., Tan, J., e Zhang, Y. (2025). A-mem: Agentic memory for llm agents. arXiv preprint arXiv:2502.12110.

Yannakakis, G. N. e Togelius, J. (2015). A panorama of artificial and computational intelligence in games. IEEE Trans. on Computational Intelligence and AI in Games, 7(4):317–335.

Yannakakis, G. N. e Togelius, J. (2018). Artificial Intelligence and Games. Springer.

Zhang, Z. et al. (2025). A survey on the memory mechanism of large language model-based agents. ACM Transactions on Information Systems. arXiv:2404.13501, 2024.

Zhao, A., Huang, D., Xu, Q., Lin, M., Liu, Y.-J., e Huang, G. (2024). ExpeL: LLM agents are experiential learners. In Proc. AAAI.

Zhong, W., Guo, L., Gao, Q., Ye, H., e Wang, Y. (2024). MemoryBank: Enhancing large language models with long-term memory. In Proc. AAAI.

Zhou, P., Zhu, A., Hu, J., Pujara, J., Ren, X., Callison-Burch, C., Choi, Y., e Ammanabrolu, P. (2023). I Cast Detect Thoughts: Learning to converse and guide with intents and theory-of-mind in dungeons and dragons. In Proc. ACL.

Zhu, A., Aggarwal, K., Feng, A., Martin, L. J., e Callison-Burch, C. (2023). FIREBALL: A dataset of dungeons and dragons actual-play with structured game state information. In Proc. ACL.
Publicado
29/09/2026
SANTOS, Gabriel Machado; JULIA, Rita Maria Silva; NASCIMENTO, Marcelo. Memory Architectures for Game Agents: A Systematic Mapping Study Bridging Classical and LLM Paradigms. In: SIMPÓSIO BRASILEIRO DE JOGOS E ENTRETENIMENTO DIGITAL (SBGAMES), 25. , 2026, Goiânia/GO. Anais [...]. Porto Alegre: Sociedade Brasileira de Computação, 2026 . p. 818-832. DOI: https://doi.org/10.5753/sbgames.2026.25637.