User Engagement Evaluation in digital games: Comparing self-reported and psychophysiological data
Abstract
In the field of Games User Research, engagement is the consequence of a motivating interaction between the user and a digital game. Measuring engagement is relevant and useful for all types of applications, but in games, this construct is critical to understanding player behavior and measuring game efficiency. Due to the relevance of this subject, several theoretical and empirical models have been proposed in the literature, from physiological measurements, such as monitoring heart beats, to self-reported methods, such as the UES-Br attitude scale. Therefore, the objective of this work is to verify the correlation of the engagement constructs of the self-reported methods with the psychophysiological measurements and how one can complement the other. To conduct this research, the following steps are being carried out: Literature review, where a tertiary study of works that map engagement and well-being assessment methods is being conducted; Project writing for the Ethics Committee, describing the experiments planned for the evaluation of self-reported and psychophysiological evaluation methods; Application of experiments, where a sample of participants will be previously selected to play certain digital games; Finally, the collection and analysis of the results obtained in the experiments will be carried out.
References
Brockmyer, J. H., Fox, C. M., Curtiss, K. A., McBroom, E., Burkhart, K. M., and Pidruzny, J. N. (2009). The development of the game engagement questionnaire: A measure of engagement in video game-playing. Journal of experimental social psychology, 45(4):624–634
Bruhlmann, F. and Schmid, G.-M. (2015). How to measure the game experience? analysis of the factor structure of two questionnaires. In Proceedings of the 33rd annual acm conference extended abstracts on human factors in computing systems, pages 1181–1186.
Deci, E. L. and Ryan, R. M. (1985). Intrinsic motivation and self-determination in human behavior. Perspectives in social psychology.
Doherty, K. and Doherty, G. (2018). Engagement in hci: conception, theory and measurement. ACM Computing Surveys (CSUR), 51(5):1–39.
Glasnapp, J. and Brdiczka, O. (2009). A human-centered model for detecting technology engagement. In International Conference on Human-Computer Interaction, pages 621–630. Springer.
Kim, H. and Kim, B. (2019). The evaluation of visitor experiences using the peak-end rule. Journal of Heritage Tourism, 14(5-6):561–573.
Maia, C. L. B. and Furtado, E. S. (2019). An approach to analyze user’s emotion in hci experiments using psychophysiological measures. IEEE Access, 7:36471–36480.
Miranda, D., Li, C., and Darin, T. (2021). Ues-br: Translation and cross-cultural adaptation of the user engagement scale for brazilian portuguese. Proceedings of the ACM on Human-Computer Interaction, 5(CHI PLAY):1–22.
Ntoumanis, N., Ng, J. Y., Prestwich, A., Quested, E., Hancox, J. E., Thøgersen-Ntoumani, C., Deci, E. L., Ryan, R. M., Lonsdale, C., and Williams, G. C. (2021). A meta-analysis of self-determination theory-informed intervention studies in the health domain: Effects on motivation, health behavior, physical, and psychological health. Health psychology review, 15(2):214–244.
O’Brien, H. and Cairns, P. (2016). Why engagement matters. Cham: Springer International Publishing. doi, 10:978–3.
O’Brien, H. L. and Toms, E. G. (2008). What is user engagement? a conceptual framework for defining user engagement with technology. Journal of the American society for Information Science and Technology, 59(6):938–955.
O’Brien, H. L. and Toms, E. G. (2010). The development and evaluation of a survey to measure user engagement. Journal of the American Society for Information Science and Technology, 61(1):50–69.
Peters, D., Calvo, R. A., and Ryan, R. M. (2018). Designing for motivation, engagement and wellbeing in digital experience. Frontiers in psychology, 9:797.
Proulx, J.-N., Romero, M., and Arnab, S. (2017). Learning mechanics and game mechanics under the perspective of self-determination theory to foster motivation in digital game based learning. Simulation & Gaming, 48(1):81–97.
Resolução, N. 466/2012 do conselho nacional de saúde (2012). Diretrizes e normas. Ryan, R. M. and Deci, E. L. (2000). Self-determination theory and the facilitation of intrinsic motivation, social development, and well-being. American psychologist, 55(1):68.
Ryan, R. M. and Deci, E. L. (2017). Self-determination theory: Basic psychological needs in motivation, development, and wellness. Guilford Publications.
Schoenau-Fog, H. (2011). Hooked!–evaluating engagement as continuation desire in interactive narratives. In International Conference on Interactive Digital Storytelling, pages 219–230. Springer.
Song, Y., Shi, X., and Fu, X. (2013). Evaluating and predicting user engagement change with degraded search relevance. In Proceedings of the 22nd international conference on World Wide Web, pages 1213–1224.
Sun, J. C.-Y. (2014). Influence of polling technologies on student engagement: An analysis of student motivation, academic performance, and brainwave data. Computers & education, 72:80–89.
Vasconcellos, D., Parker, P. D., Hilland, T., Cinelli, R., Owen, K. B., Kapsal, N., Lee, J., Antczak, D., Ntoumanis, N., Ryan, R. M., et al. (2020). Self-determination theory applied to physical education: A systematic review and meta-analysis. Journal of Educational Psychology, 112(7):1444.
