How do LLMs analyze and interpret data from educational games? A study with GLA experts
Resumo
Game Learning Analytics (GLA) holds promise for extracting insights into player learning, especially with visual analytics techniques such as dashboards. However, data analysis and interpretation are not always straightforward. An emerging alternative is the use of Large Language Models (LLMs). In this work, we investigated ChatGPT, Gemini, DeepSeek, and GLA Specialist in analyzing data from an educational game collected by the GLBoard model. We conducted an empirical study in which two GLA experts evaluated the results. In its two tested versions, Gemini performed best, excelling in profile analysis, generating pedagogical insights, and providing detailed assessments.Referências
Agarwal, N. S. and Sonbhadra, S. K. (2025). A review on large language models for visual analytics. arXiv preprint arXiv:2503.15176.
AlAli, R. and Wardat, Y. (2024). Opportunities and challenges of integrating generative artificial intelligence in education. International Journal of Religion, 5(7):784–793.
Alhadad, S. (2016). Attentional and cognitive processing of analytics visualisations: Can design features affect interpretations and decisions about learning and teaching? In ASCILITE 2016. Australasian Society for Computers in Learning in Tertiary Education (ASCILITE).
Alonso-Fernández, C., Calvo-Morata, A., Freire, M., Martínez-Ortiz, I., and Manjón, B. F. (2021). Data science meets standardized game learning analytics. In 2021 IEEE Global Engineering Education Conference (EDUCON), pages 1546–1552. IEEE.
Bang, Y., Cahyawijaya, S., Lee, N., Dai, W., Su, D., Wilie, B., Lovenia, H., Ji, Z., Yu, T., Chung, W., et al. (2023). A multitask, multilingual, multimodal evaluation of chatgpt on reasoning, hallucination, and interactivity. arXiv preprint arXiv:2302.04023.
Bardin, L. (2015). Análise de conteúdo (la reto & a. pinheiro, tradução)(6ª edição). Lisboa, Portugal: Edições, 70.
Davalos, E., Zhang, Y., Srivastava, N., Salas, J. A., McFadden, S., Cho, S.-J., Biswas, G., and Goodwin, A. (2025). Llms as educational analysts: Transforming multimodal data traces into actionable reading assessment reports. arXiv preprint arXiv:2503.02099.
El-Nasr, M. S., Drachen, A., and Canossa, A. (2016). Game analytics. Springer.
Few, S. (2006). Information dashboard design: The effective visual communication of data. O’Reilly Media, Inc.
Freire, M., Serrano-Laguna, Á., Manero Iglesias, B., Martínez-Ortiz, I., Moreno-Ger, P., and Fernández-Manjón, B. (2016). Game learning analytics: Learning analytics for serious games. In Learning, design, and technology: An international compendium of theory, research, practice, and policy, pages 3475–3502. Springer.
Genesio, N. O. S., de Oliveira, A. M., Oliveira, E. W., and Valle, P. H. D. (2024). Panorama de estudos sobre jogos educacionais digitais em educação em computação. In Workshop sobre Educação em Computação (WEI), pages 737–749. SBC.
Guo, D., Zhu, Q., Yang, D., Xie, Z., Dong, K., Zhang, W., Chen, G., Bi, X., Wu, Y., Li, Y., et al. (2024). Deepseek-coder: When the large language model meets programming– the rise of code intelligence. arXiv preprint arXiv:2401.14196.
Honda, F., Pires, F., Pessoa, M., and Oliveira, E. H. (2024). Building a specialist agent to assist in the implementation of game learning analytics techniques. In Simpósio Brasileiro de Informática na Educação (SBIE), pages 2856–2865. SBC.
Hutchinson, M., Jianu, R., Slingsby, A., and Madhyastha, P. (2024). Llm-assisted visual analytics: Opportunities and challenges. arXiv preprint arXiv:2409.02691.
Imran, M. and Almusharraf, N. (2024). Google gemini as a next generation ai educational tool: a review of emerging educational technology. Smart Learning Environments, 11(1):22.
Kim, M., Kim, S., Lee, S., Yoon, Y., Myung, J., Yoo, H., Lim, H., Han, J., Kim, Y., Ahn, S.-Y., et al. (2024). Llm-driven learning analytics dashboard for teachers in efl writing education. arXiv preprint arXiv:2410.15025.
Larusson, J. A. and White, B. (2014). Learning analytics. From Research to Practice. Nueva York: Springer.
Liu, Y., Pozdniakov, S., and Martinez-Maldonado, R. (2024). The effects of visualisation literacy and data storytelling dashboards on teachers’ cognitive load. Australasian Journal of Educational Technology, 40(1):78–93.
Lo, L. S. (2023). The art and science of prompt engineering: a new literacy in the information age. Internet Reference Services Quarterly, 27(4):203–210.
Macfadyen, L. P. and Dawson, S. (2012). Numbers are not enough. why e-learning analytics failed to inform an institutional strategic plan. Journal of Educational Technology & Society, 15(3):149–163.
Melo, D., de Sousa Pires, F. G., Melo, R., and Júnior, R. J. d. R. S. (2018). Robô euroi: Game de estratégia matemática para exercitar o pensamento computacional. In Brazilian Symposium on Computers in Education (Simpósio Brasileiro de Informática na Educaçao-SBIE), volume 29, page 685.
Pang, R. Y., Schroeder, H., Smith, K. S., Barocas, S., Xiao, Z., Tseng, E., and Bragg, D. (2025). Understanding the llm-ification of chi: Unpacking the impact of llms at chi through a systematic literature review. In Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems, pages 1–20.
Petri, G. and von Wangenheim, C. G. (2017). How games for computing education are evaluated? a systematic literature review. Computers & education, 107:68–90.
Pires, F. G. d. S., Melo, R., Machado, J., Silva, M. S., Franzoia, F., and de Freitas, R. (2018). Ecologic: um jogo de estratégia para o desenvolvimento do pensamento computacional e da consciência ambiental. In Anais dos Workshops do Congresso Brasileiro de Informática na Educação, volume 7, page 629.
Plass, J. L., Homer, B. D., and Kinzer, C. K. (2015). Foundations of game-based learning. Educational psychologist, 50(4):258–283.
Silva, D., Melo, R., Pires, F., and Pessoa, M. (2021). Avaliacão de objetos digitais de aprendizagem: como os licenciados em computação analisam jogos educacionais? RENOTE, 19(2):111–121.
Silva, D., Pires, F., Melo, R., and Pessoa, M. (2022). Glboard: um sistema para auxiliar na captura e análise de dados em jogos educacionais. In Simpósio Brasileiro de Jogos e Entretenimento Digital (SBGames), pages 959–968. SBC.
Susnjak, T., Ramaswami, G. S., and Mathrani, A. (2022). Learning analytics dashboard: a tool for providing actionable insights to learners. International Journal of Educational Technology in Higher Education, 19(1):12.
Verbert, K., Duval, E., Klerkx, J., Govaerts, S., and Santos, J. L. (2013). Learning analytics dashboard applications. American Behavioral Scientist, 57(10):1500–1509.
Wall, E., Blaha, L. M., Paul, C. L., Cook, K., and Endert, A. (2018). Four perspectives on human bias in visual analytics. In Cognitive biases in visualizations, pages 29–42. Springer.
Yao, Y., Duan, J., Xu, K., Cai, Y., Sun, Z., and Zhang, Y. (2024). A survey on large language model (llm) security and privacy: The good, the bad, and the ugly. High-Confidence Computing, page 100211.
Ye, Q., Axmed, M., Pryzant, R., and Khani, F. (2023). Prompt engineering a prompt engineer. arXiv preprint arXiv:2311.05661.
Zhao, Y., Zhang, Y., Zhang, Y., Zhao, X., Wang, J., Shao, Z., Turkay, C., and Chen, S. (2024). Leva: Using large language models to enhance visual analytics. IEEE transactions on visualization and computer graphics, 31(3):1830–1847.
AlAli, R. and Wardat, Y. (2024). Opportunities and challenges of integrating generative artificial intelligence in education. International Journal of Religion, 5(7):784–793.
Alhadad, S. (2016). Attentional and cognitive processing of analytics visualisations: Can design features affect interpretations and decisions about learning and teaching? In ASCILITE 2016. Australasian Society for Computers in Learning in Tertiary Education (ASCILITE).
Alonso-Fernández, C., Calvo-Morata, A., Freire, M., Martínez-Ortiz, I., and Manjón, B. F. (2021). Data science meets standardized game learning analytics. In 2021 IEEE Global Engineering Education Conference (EDUCON), pages 1546–1552. IEEE.
Bang, Y., Cahyawijaya, S., Lee, N., Dai, W., Su, D., Wilie, B., Lovenia, H., Ji, Z., Yu, T., Chung, W., et al. (2023). A multitask, multilingual, multimodal evaluation of chatgpt on reasoning, hallucination, and interactivity. arXiv preprint arXiv:2302.04023.
Bardin, L. (2015). Análise de conteúdo (la reto & a. pinheiro, tradução)(6ª edição). Lisboa, Portugal: Edições, 70.
Davalos, E., Zhang, Y., Srivastava, N., Salas, J. A., McFadden, S., Cho, S.-J., Biswas, G., and Goodwin, A. (2025). Llms as educational analysts: Transforming multimodal data traces into actionable reading assessment reports. arXiv preprint arXiv:2503.02099.
El-Nasr, M. S., Drachen, A., and Canossa, A. (2016). Game analytics. Springer.
Few, S. (2006). Information dashboard design: The effective visual communication of data. O’Reilly Media, Inc.
Freire, M., Serrano-Laguna, Á., Manero Iglesias, B., Martínez-Ortiz, I., Moreno-Ger, P., and Fernández-Manjón, B. (2016). Game learning analytics: Learning analytics for serious games. In Learning, design, and technology: An international compendium of theory, research, practice, and policy, pages 3475–3502. Springer.
Genesio, N. O. S., de Oliveira, A. M., Oliveira, E. W., and Valle, P. H. D. (2024). Panorama de estudos sobre jogos educacionais digitais em educação em computação. In Workshop sobre Educação em Computação (WEI), pages 737–749. SBC.
Guo, D., Zhu, Q., Yang, D., Xie, Z., Dong, K., Zhang, W., Chen, G., Bi, X., Wu, Y., Li, Y., et al. (2024). Deepseek-coder: When the large language model meets programming– the rise of code intelligence. arXiv preprint arXiv:2401.14196.
Honda, F., Pires, F., Pessoa, M., and Oliveira, E. H. (2024). Building a specialist agent to assist in the implementation of game learning analytics techniques. In Simpósio Brasileiro de Informática na Educação (SBIE), pages 2856–2865. SBC.
Hutchinson, M., Jianu, R., Slingsby, A., and Madhyastha, P. (2024). Llm-assisted visual analytics: Opportunities and challenges. arXiv preprint arXiv:2409.02691.
Imran, M. and Almusharraf, N. (2024). Google gemini as a next generation ai educational tool: a review of emerging educational technology. Smart Learning Environments, 11(1):22.
Kim, M., Kim, S., Lee, S., Yoon, Y., Myung, J., Yoo, H., Lim, H., Han, J., Kim, Y., Ahn, S.-Y., et al. (2024). Llm-driven learning analytics dashboard for teachers in efl writing education. arXiv preprint arXiv:2410.15025.
Larusson, J. A. and White, B. (2014). Learning analytics. From Research to Practice. Nueva York: Springer.
Liu, Y., Pozdniakov, S., and Martinez-Maldonado, R. (2024). The effects of visualisation literacy and data storytelling dashboards on teachers’ cognitive load. Australasian Journal of Educational Technology, 40(1):78–93.
Lo, L. S. (2023). The art and science of prompt engineering: a new literacy in the information age. Internet Reference Services Quarterly, 27(4):203–210.
Macfadyen, L. P. and Dawson, S. (2012). Numbers are not enough. why e-learning analytics failed to inform an institutional strategic plan. Journal of Educational Technology & Society, 15(3):149–163.
Melo, D., de Sousa Pires, F. G., Melo, R., and Júnior, R. J. d. R. S. (2018). Robô euroi: Game de estratégia matemática para exercitar o pensamento computacional. In Brazilian Symposium on Computers in Education (Simpósio Brasileiro de Informática na Educaçao-SBIE), volume 29, page 685.
Pang, R. Y., Schroeder, H., Smith, K. S., Barocas, S., Xiao, Z., Tseng, E., and Bragg, D. (2025). Understanding the llm-ification of chi: Unpacking the impact of llms at chi through a systematic literature review. In Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems, pages 1–20.
Petri, G. and von Wangenheim, C. G. (2017). How games for computing education are evaluated? a systematic literature review. Computers & education, 107:68–90.
Pires, F. G. d. S., Melo, R., Machado, J., Silva, M. S., Franzoia, F., and de Freitas, R. (2018). Ecologic: um jogo de estratégia para o desenvolvimento do pensamento computacional e da consciência ambiental. In Anais dos Workshops do Congresso Brasileiro de Informática na Educação, volume 7, page 629.
Plass, J. L., Homer, B. D., and Kinzer, C. K. (2015). Foundations of game-based learning. Educational psychologist, 50(4):258–283.
Silva, D., Melo, R., Pires, F., and Pessoa, M. (2021). Avaliacão de objetos digitais de aprendizagem: como os licenciados em computação analisam jogos educacionais? RENOTE, 19(2):111–121.
Silva, D., Pires, F., Melo, R., and Pessoa, M. (2022). Glboard: um sistema para auxiliar na captura e análise de dados em jogos educacionais. In Simpósio Brasileiro de Jogos e Entretenimento Digital (SBGames), pages 959–968. SBC.
Susnjak, T., Ramaswami, G. S., and Mathrani, A. (2022). Learning analytics dashboard: a tool for providing actionable insights to learners. International Journal of Educational Technology in Higher Education, 19(1):12.
Verbert, K., Duval, E., Klerkx, J., Govaerts, S., and Santos, J. L. (2013). Learning analytics dashboard applications. American Behavioral Scientist, 57(10):1500–1509.
Wall, E., Blaha, L. M., Paul, C. L., Cook, K., and Endert, A. (2018). Four perspectives on human bias in visual analytics. In Cognitive biases in visualizations, pages 29–42. Springer.
Yao, Y., Duan, J., Xu, K., Cai, Y., Sun, Z., and Zhang, Y. (2024). A survey on large language model (llm) security and privacy: The good, the bad, and the ugly. High-Confidence Computing, page 100211.
Ye, Q., Axmed, M., Pryzant, R., and Khani, F. (2023). Prompt engineering a prompt engineer. arXiv preprint arXiv:2311.05661.
Zhao, Y., Zhang, Y., Zhang, Y., Zhao, X., Wang, J., Shao, Z., Turkay, C., and Chen, S. (2024). Leva: Using large language models to enhance visual analytics. IEEE transactions on visualization and computer graphics, 31(3):1830–1847.
Publicado
24/11/2025
Como Citar
BASTOS, Manuela; HONDA, Fabrizio; LIMA, Márcia; PESSOA, Marcela; PIRES, Fernanda.
How do LLMs analyze and interpret data from educational games? A study with GLA experts. In: SIMPÓSIO BRASILEIRO DE INFORMÁTICA NA EDUCAÇÃO (SBIE), 36. , 2025, Curitiba/PR.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2025
.
p. 1317-1330.
DOI: https://doi.org/10.5753/sbie.2025.12887.
