Engajamento em Jogos Educacionais Digitais: Análise com um Modelo Híbrido de Detecção
Resumo
Este estudo investigou como os desafios propostos em um Jogo Educacional Digital (JED) e as habilidades dos estudantes em superá-los influenciam seus estados afetivos e comportamentais, afetando o engajamento e a aprendizagem. Foi desenvolvido um modelo híbrido que integra dados emocionais e comportamentais coletados automaticamente, como emoções faciais e movimentos dos olhos e cabeça. Esses dados foram combinados para avaliar o engajamento dos estudantes durante a interação com o JED. Além disso, relatos dos estudantes por meio de pré e pós-questionários foram utilizados para validar os resultados obtidos pelo modelo automático e para fornecer uma análise qualitativa das percepções dos estudantes sobre os desafios enfrentados e a aprendizagem proporcionada pelo jogo. Dez estudantes participaram do estudo e, na maioria do tempo, permaneceram engajados. Observou-se que o desengajamento ocorreu principalmente quando os estudantes encontravam dificuldades no controle do jogo. A partir dos resultados, foi proposta uma representação da dinâmica das emoções dos estudantes durante os jogos.
Palavras-chave:
Jogo Educacional Digital (JED), Engajamento, Estados afetivos, Detecção de emoções por face
Referências
Ahmed, A. A. and Goodwin, M. S. (2017). Automated detection of facial expressions during computer-assisted instruction in individuals on the autism spectrum. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, pages 6050–6055, [S.l.]. s.n.
Csikszentmihalyi, M. (1998). Finding Flow: The Psychology of Engagement with Everyday Life. Basic Books (AZ), New York, NY, USA, revised ed. edition.
Csikszentmihalyi, M. (2008). Flow: The Psychology of Optimal Experience. Harper Perennial, New York, English edition.
D’Mello, S. and Graesser, A. (2012). Dynamics of affective states during complex learning. Learning and Instruction, 22(2):145–157.
Ekman, P., Friesen, W. V., O’Sullivan, M., Chan, A., Diacoyanni-Tarlatzis, I., Heider, K., Krause, R., LeCompte, W. A., Pitcairn, T., Ricci-Bitti, P. E., et al. (1987). Universals and cultural differences in the judgments of facial expressions of emotion. Journal of Personality and Social Psychology, 53(4):712.
Gonzalez Sánchez, J. L., Padilla Zea, N., and Gutiérrez, F. L. (2009). From usability to playability: Introduction to player-centered video game development process. In Human Centered Design: First International Conference, HCD 2009, Held as Part of HCI International 2009, San Diego, CA, USA, July 19-24, 2009 Proceedings 1, pages 65–74. Springer.
Gupta, S. K., Ashwin, T., and Guddeti, R. M. R. (2019). Students’ affective content analysis in smart classroom environment using deep learning techniques. Multimedia Tools and Applications, 78:25321–25348.
Isbister, K. and Schaffer, N. (2008). “Gamenics” and its Potential - Interview with Akihiro Saito, Professor, Ritsumeikan University, College of Image Arts and Sciences; Director, Bmat Japan, pages 357–379. O’Reilly Media, Inc.
Kiili, K., De Freitas, S., Arnab, S., and Lainema, T. (2012). The design principles for flow experience in educational games. Procedia Computer Science, 15:78–91.
Klein, R. and Celik, T. (2017). The WITS Intelligent Teaching System: Detecting student engagement during lectures using convolutional neural networks. In 2017 IEEE International Conference on Image Processing (ICIP), pages 2856–2860. IEEE.
Mansoor Iqbal (2024). Fortnite usage and revenue statistics (2024). Disponível em: [link]. Acesso em: 10 fevereiro 2024.
Ninaus, M., Greipl, S., Kiili, K., Lindstedt, A., Huber, S., Klein, E., Karnath, H.-O., and Moeller, K. (2019). Increased emotional engagement in game-based learning – a machine learning approach on facial emotion detection data. Computers & Education, 142:103641.
Psaltis, A., Apostolakis, K. C., Dimitropoulos, K., and Daras, P. (2017). Multimodal student engagement recognition in prosocial games. IEEE Transactions on Games, 10(3):292–303.
Sell, K., Lillie, T., and Taylor, J. (2008). Energy expenditure during physically interactive video game playing in male college students with different playing experience. Journal of American College Health, 56(5):505–512.
Sharma, P., Joshi, S., Gautam, S., Maharjan, S., Khanal, S. R., Reis, M. C., Barroso, J., and de Jesus Filipe, V. M. (2022). Student engagement detection using emotion analysis, eye tracking and head movement with machine learning. In International Conference on Technology and Innovation in Learning, Teaching and Education, pages 52–68. Springer.
Thiruthuvanathan, M. M., Krishnan, B., and Rangaswamy, M. (2021). Engagement detection through facial emotional recognition using shallow residual convolutional neural networks. International Journal of Intelligent Engineering and Systems, 14(2):236–247.
Wan, C.-S. and Chiou, W.-B. (2007). The motivations of adolescents who are addicted to online games: A cognitive perspective. Adolescence, 42:179–97.
Whitehill, J., Serpell, Z., Lin, Y.-C., Foster, A., and Movellan, J. R. (2014). The faces of engagement: Automatic recognition of student engagement from facial expressions. IEEE Transactions on Affective Computing, 5(1):86–98.
Zaletelj, J. and Košir, A. (2017). Predicting students’ attention in the classroom from Kinect facial and body features. EURASIP Journal on Image and Video Processing, 2017(1):1–12.
Csikszentmihalyi, M. (1998). Finding Flow: The Psychology of Engagement with Everyday Life. Basic Books (AZ), New York, NY, USA, revised ed. edition.
Csikszentmihalyi, M. (2008). Flow: The Psychology of Optimal Experience. Harper Perennial, New York, English edition.
D’Mello, S. and Graesser, A. (2012). Dynamics of affective states during complex learning. Learning and Instruction, 22(2):145–157.
Ekman, P., Friesen, W. V., O’Sullivan, M., Chan, A., Diacoyanni-Tarlatzis, I., Heider, K., Krause, R., LeCompte, W. A., Pitcairn, T., Ricci-Bitti, P. E., et al. (1987). Universals and cultural differences in the judgments of facial expressions of emotion. Journal of Personality and Social Psychology, 53(4):712.
Gonzalez Sánchez, J. L., Padilla Zea, N., and Gutiérrez, F. L. (2009). From usability to playability: Introduction to player-centered video game development process. In Human Centered Design: First International Conference, HCD 2009, Held as Part of HCI International 2009, San Diego, CA, USA, July 19-24, 2009 Proceedings 1, pages 65–74. Springer.
Gupta, S. K., Ashwin, T., and Guddeti, R. M. R. (2019). Students’ affective content analysis in smart classroom environment using deep learning techniques. Multimedia Tools and Applications, 78:25321–25348.
Isbister, K. and Schaffer, N. (2008). “Gamenics” and its Potential - Interview with Akihiro Saito, Professor, Ritsumeikan University, College of Image Arts and Sciences; Director, Bmat Japan, pages 357–379. O’Reilly Media, Inc.
Kiili, K., De Freitas, S., Arnab, S., and Lainema, T. (2012). The design principles for flow experience in educational games. Procedia Computer Science, 15:78–91.
Klein, R. and Celik, T. (2017). The WITS Intelligent Teaching System: Detecting student engagement during lectures using convolutional neural networks. In 2017 IEEE International Conference on Image Processing (ICIP), pages 2856–2860. IEEE.
Mansoor Iqbal (2024). Fortnite usage and revenue statistics (2024). Disponível em: [link]. Acesso em: 10 fevereiro 2024.
Ninaus, M., Greipl, S., Kiili, K., Lindstedt, A., Huber, S., Klein, E., Karnath, H.-O., and Moeller, K. (2019). Increased emotional engagement in game-based learning – a machine learning approach on facial emotion detection data. Computers & Education, 142:103641.
Psaltis, A., Apostolakis, K. C., Dimitropoulos, K., and Daras, P. (2017). Multimodal student engagement recognition in prosocial games. IEEE Transactions on Games, 10(3):292–303.
Sell, K., Lillie, T., and Taylor, J. (2008). Energy expenditure during physically interactive video game playing in male college students with different playing experience. Journal of American College Health, 56(5):505–512.
Sharma, P., Joshi, S., Gautam, S., Maharjan, S., Khanal, S. R., Reis, M. C., Barroso, J., and de Jesus Filipe, V. M. (2022). Student engagement detection using emotion analysis, eye tracking and head movement with machine learning. In International Conference on Technology and Innovation in Learning, Teaching and Education, pages 52–68. Springer.
Thiruthuvanathan, M. M., Krishnan, B., and Rangaswamy, M. (2021). Engagement detection through facial emotional recognition using shallow residual convolutional neural networks. International Journal of Intelligent Engineering and Systems, 14(2):236–247.
Wan, C.-S. and Chiou, W.-B. (2007). The motivations of adolescents who are addicted to online games: A cognitive perspective. Adolescence, 42:179–97.
Whitehill, J., Serpell, Z., Lin, Y.-C., Foster, A., and Movellan, J. R. (2014). The faces of engagement: Automatic recognition of student engagement from facial expressions. IEEE Transactions on Affective Computing, 5(1):86–98.
Zaletelj, J. and Košir, A. (2017). Predicting students’ attention in the classroom from Kinect facial and body features. EURASIP Journal on Image and Video Processing, 2017(1):1–12.
Publicado
04/11/2024
Como Citar
NASCIMENTO JUNIOR, Nelson; BRAGA, Juliana Cristina; JAQUES, Patricia A.; GOIS, João Paulo.
Engajamento em Jogos Educacionais Digitais: Análise com um Modelo Híbrido de Detecção. In: SIMPÓSIO BRASILEIRO DE INFORMÁTICA NA EDUCAÇÃO (SBIE), 35. , 2024, Rio de Janeiro/RJ.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2024
.
p. 896-909.
DOI: https://doi.org/10.5753/sbie.2024.242572.
