Hybrid Method for Game Assessment Based on Automatic Detection of Student Emotions

• Nelson N. Nascimento Federal University of ABC
• Francinete F. da Cunha Federal University of ABC
• Juliana C. Braga Federal University of ABC
• Joao Paulo Gois Federal University of ABC

DOI: https://doi.org/10.5753/sbie.2021.218276

Abstract

One of the approaches to evaluate the entertainment in educational games is to analyze their ability to provoke positive emotions in students. This evaluation is usually conducted through questionnaires applied before and after the match. This work proposes a method to evaluate educational games, based on the association of students' report data with data collected automatically from their emotional states during the gameplay. A literature review was performed to support the research and an experiment to validate the proposed method. The results indicate that recognizing students' emotional states during the game can support the development of motivating educational games.

Keywords: assessment, convolutional, emotion, student, educational game, facial recognition

References

A. Drachen, P. Mirza-Babaei, L. N. (2018). Games User Research. Oxford Scholarship Online.

Bilal, D. and Bachir, I. (2007). Children’s interaction with cross-cultural and multilingual digital libraries. ii. information seeking, success, and affective experience. Information Processing and Management, 43(1):65 – 80.

Cao, H., Verma, R. and Nenkova, A. (2015). Speaker-sensitive emotion recognition via ranking: Studies on acted and spontaneous speech. Computer Speech and Language, 29(1):186–202.

Darrell, T., Gordon, G., Harville, M., and Woodfill, J. (2000). Integrated person tracking using stereo, color, and pattern detection. International Journal of Computer Vision, 37(2):175–185.

Drachen, A., Mirza-Babaei, P., and Nacke, L. E. (2018). Games User Research. Oxford Scholarship Online.

Ekman, P., Friesen, W., O’Sullivan, M., Chan, A., Diacoyanni-Tarlatzis, I., Heider, K., Krause, R., LeCompte, W., Pitcairn, T., and Bitti, P. R. (1987). Universals and cultural differences in the judgments of facial expressions of emotion. Journal of personality and social psychology, 53:712–7.

Elsattar, H. K. H. A. (2017). Designing for game-based learning model: The effective integration of flow experience and game elements to support learning. In 2017 14th International Conference on Computer Graphics, Imaging and Visualization (CGiV), pages 34–43, Los Alamitos, CA, USA. IEEE Computer Society.

Eseryel, D., Guo, Y., and Law, V. (2012). Interactivity3 Design and Assessment Framework for Educational Games to Promote Motivation and Complex Problem-Solving Skills, pages 257–285.

Hernandez, J., Paredes, P., Roseway, A., and Czerwinski, M. (2014). Under pressure: sensing stress of computer users. In Proceedings of the SIGCHI conference on Human factors in computing systems, pages 51–60.

Jabbar, A. I. A. and Felicia, P. (2016). Towards a conceptual framework of GBL Design for engagement and learning of curriculum-based content. International Journal of Game-Based Learning, 6:87–108.

Jovanovic, M., Starcevic, D., Minovic, M., and Stavljanin, V. (2011). Motivation and multimodal interaction in model-driven educational game design. Systems, Man and Cybernetics, Part A: Systems and Humans, IEEE Transactions on, 41:817 – 824.

Kapoor, A., Burleson, W., and Picard, R. W. (2007). Automatic prediction of frustration. International Journal of Human-Computer Studies, 65(8):724 – 736.

Kardan, A. and Einavypour, Y. (2008). Involving learner’s emotional behaviors in learning process as a temporary learner model. In Proceeding of International Conference on Virtual Learning (ICVL), Bucurest, Romania.

Kiili, K., de Freitas, S., Arnab, S., and Lainema, T. (2012). The design principles for flow experience in educational games. Procedia Computer Science, 15:78–91.

Lopatovska, I. (2009). Emotional aspects of the online information retrieval process. PhD thesis, Rutgers University-Graduate School-New Brunswick.

Mysirlaki, S. and Paraskeva, F. (2010). Intrinsic motivation and the sense of community in multiplayer games: An extended framework for educational game design. Informatics, Panhellenic Conference on, 0:223–227.

Nickel, K. and Stiefelhagen, R. (2007). Visual recognition of pointing gestures for human robot interaction. Image and Vision Computing, 25(12):1875–1884. The age of human computer interaction.

Pekrun, R., Goetz, T., Frenzel, A. C., Barchfeld, P., and Perry, R. P. (2011). Measuring emotions in students’ learning and performance: The achievement emotions questionnaire (AEQ). Contemporary Educational Psychology, 36(1):36–48. Students’ Emotions and Academic Engagement.

Shimizu, M., Yoshizuka, T., and Miyamoto, H. (2007). A gesture recognition system using stereo vision and arm model fitting. International Congress Series, 1301:89– 92. Brain-Inspired IT III. Invited and selected papers of the 3rd International Conference on Brain-Inspired Information Technology "BrainIT 2006" held in Hibikino, Kitakyushu, Japan between 27 and 29 September 2006.

Truong, K. P., van Leeuwen, D. A., and de Jong, F. M. (2012). Speech-based recognition of self-reported and observed emotion in a dimensional space. Speech Communication, 54(9):1049 – 1063.

Verma, G. K. and Tiwary, U. S. (2014). Multimodal fusion framework: A multiresolution approach for emotion classification and recognition from physiological signals. NeuroImage, 102:162 – 172. Multimodal Data Fusion.