Building a specialist agent to assist in the implementation of Game Learning Analytics techniques

Fabrizio Honda; Fernanda Pires; Marcela Pessoa; Elaine H. T. Oliveira

doi:10.5753/sbie.2024.244936

Fabrizio Honda Universidade do Estado do Amazonas / Universidade Federal do Amazonas https://orcid.org/0009-0003-8174-1002
Fernanda Pires Universidade do Estado do Amazonas https://orcid.org/0000-0002-8180-9850
Marcela Pessoa Universidade do Estado do Amazonas https://orcid.org/0000-0002-3064-5585
Elaine H. T. Oliveira Universidade Federal do Amazonas https://orcid.org/0000-0003-2884-9359

DOI: https://doi.org/10.5753/sbie.2024.244936

Resumo

Game Learning Analytics (GLA) involves capturing and analyzing data from educational games, enabling the identification of evidence of learning. A fundamental step before implementing GLA techniques is data modeling, which is not trivial. Using large language models (LLMs) can help in this context, as they can generate text like humans. Therefore, considering Chat-GPT and its customizable functionality, “MyGPTs,” this work proposes creating a specialist agent to assist learning designers in data modeling and implementing GLA techniques based on the GLBoard system. Preliminary results with GLA specialists were positive, indicating the agent’s potential.

Referências

Almasre, M. (2024). Development and evaluation of a custom gpt for the assessment of students’ designs in a typography course. Education Sciences, 14(2):148.

Alonso-Fernandez, C., Calvo, A., Freire, M., Martinez-Ortiz, I., and Fernandez-Manjon, B. (2017). Systematizing game learning analytics for serious games. In 2017 IEEE Global Engineering Education Conference (EDUCON), pages 1111–1118. IEEE.

Alonso-Fernández, C., Calvo-Morata, A., Freire, M., Martínez-Ortiz, I., and Manjón, B. F. (2021). Data science meets standardized game learning analytics. In 2021 IEEE Global Engineering Education Conference (EDUCON), pages 1546–1552. IEEE.

Banihashem, S. K., Dehghanzadeh, H., Clark, D., Noroozi, O., and Biemans, H. J. (2023). Learning analytics for online game-based learning: A systematic literature review. Behaviour and Information Technology, pages 1–28.

Bardin, L. (2015). Analise de conteudo. Lisboa, Portugal: Edicoes 70.

Brown, A. L. (1992). Design experiments: Theoretical and methodological challenges in creating complex interventions in classroom settings. The journal of the learning sciences, 2(2):141–178.

DemandSage (2024). Chatgpt statistics 2024: Users, market share, and key insights. Accessed 2024-08-18.

Fraefel, U. (2014). Professionalization of pre-service teachers through university-school partnerships. In Conference Proceedings of WERA Focal Meeting, Edinburgh.

Freire, M., Serrano-Laguna, A., Manero Iglesias, B., Martínez-Ortiz, I., Moreno-Ger, P., and Fernández-Manjón, B. (2023). Game learning analytics: Learning analytics for serious games. In Learning, design, and technology: An international compendium of theory, research, practice, and policy, pages 3475–3502. Springer.

Gagnon, J. C. and Barber, B. R. (2018). The sage encyclopedia of educational research, measurement, and evaluation. In The SAGE encyclopedia of educational research, measurement and evaluation, page 668. Sage.

Jovanovic, M. and Campbell, M. (2022). Generative artificial intelligence: Trends and prospects. Computer, 55(10):107–112.

Kasneci, E., Seßler, K., Küchemann, S., Bannert, M., Dementieva, D., Fischer, F., Gasser, U., Groh, G., Günnemann, S., Hüllermeier, E., et al. (2023). Chatgpt for good? on opportunities and challenges of large language models for education. Learning and individual differences, 103:102274.

Kitto, K., Whitmer, J., Silvers, A., and Webb, M. (2020). Creating data for learning analytics ecosystems.

Kıyak, Y. S. and Kononowicz, A. A. (2024). Case-based mcq generator: a custom chatgpt based on published prompts in the literature for automatic item generation. Medical teacher, pages 1–3.

Liu, P., Yuan, W., Fu, J., Jiang, Z., Hayashi, H., and Neubig, G. (2023). Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing. ACM Computing Surveys, 55(9):1–35.

Lo, L. S. (2023). The art and science of prompt engineering: a new literacy in the information age. Internet Reference Services Quarterly, 27(4):203–210.

OpenAI (2023). Introducing gpts. [link]. Accessed 2024-08-12.

Reimann, P. (2011). Design-based research. In Methodological choice and design, pages 37–50. Springer.

Sahoo, P., Singh, A. K., Saha, S., Jain, V., Mondal, S., and Chadha, A. (2024). A systematic survey of prompt engineering in large language models: Techniques and applications. arXiv preprint arXiv:2402.07927.

Sathe, T. S., Roshal, J., Naaseh, A., L’Huillier, J. C., Navarro, S. M., and Silvestri, C. (2024). How i gpt it: Development of custom artificial intelligence (ai) chatbots for surgical education. Journal of Surgical Education, 81(6):772–775.

Saveski, G. L., Westera, W., Yuan, L., Hollins, P., Manjón, B. F., Ger, P. M., and Stefanov, K. (2016). What serious game studios want from ict research: identifying developers’ needs. In Games and Learning Alliance: 4th International Conference, GALA 2015, Rome, Italy, December 9-11, 2015, Revised Selected Papers 4, pages 32–41. Springer.

Serrano-Laguna, A., Martínez-Ortiz, I., Haag, J., Regan, D., Johnson, A., and Fernández-Manjón, B. (2017). Applying standards to systematize learning analytics in serious games. Computer Standards and Interfaces, 50:116–123.

Silva, D., Pires, F., Melo, R., and Pessoa, M. (2022). Glboard: um sistema para auxiliar na captura e analise de dados em jogos educacionais. In Anais Estendidos do XXI Simposio Brasileiro de Jogos e Entretenimento Digital, pages 959–968. SBC.