"Every move you make in every game you play": can game logs determine player's engagement?

• Lucas Mendes Sales Universidade de São Paulo
• Leonardo Nogueira Cordeiro Universidade de São Paulo
• Renan Vinicius Aranha Universidade de São Paulo
• Fátima L. S. Nunes Universidade de São Paulo

Resumo

Context: Metrics and instruments available to measure player's engagement usually involve the application of questionnaires or interviews. Problem: Despite their importance and validity, these approaches can affect the player experience by interrupting the interaction process. In addition, the literature reports a growing interest in the development of games capable of real-time self-adapting based on the player state. This type of game requires strategies and tools to allow an automatic and non-noticeable evaluation of the user's experience, which is a challenging and open research field. Solution: Game analytics, which embraces techniques that can automatically collect and analyze data, can contribute in this context by providing data that can be interpreted in order to establish eventual relation with players’ experience. IS Theory: Flow theory. Methods: This study investigates whether there is a correlation between interaction logs with the self-assessed engagement. An experimental evaluation was performed with 37 volunteers interacting with two games. The correlations were analyzed considering each log individually and their classification in three categories: neutral, positive and negative. Results: The results of the experimental evaluation reveal that the correlation coefficients found are considered weak (about 0.44). On the other hand, it is interesting that the categorization of logs between neutral, positive and negative presented the main significant results in both games. Contributions and Impact in the IS area: The experiment showed weak correlation between game logs and engagement in the most cases. However, they suggest that the categorization of logs in neutral, positive and negative logs can be an interesting approach.

Palavras-chave: games, player experience, engagement, evaluation

Referências

Vero Vanden Abeele, Lennart E. Nacke, Elisa D. Mekler, and Daniel Johnson. 2016. Design and Preliminary Validation of The Player Experience Inventory. In Proceedings of the 2016 Annual Symposium on Computer-Human Interaction in Play Companion Extended Abstracts. ACM. https://doi.org/10.1145/2968120.2987744

Vero Vanden Abeele, Katta Spiel, Lennart Nacke, Daniel Johnson, and Kathrin Gerling. 2020. Development and validation of the player experience inventory: A scale to measure player experiences at the level of functional and psychosocial consequences. International Journal of Human-Computer Studies 135 (March 2020), 102370. https://doi.org/10.1016/j.ijhcs.2019.102370

Renan Vinicius Aranha, Leonardo Nogueira Cordeiro, Lucas Mendes Sales, and Fátima L. S. Nunes. 2021. Engagement and Discrete Emotions in Game Scenario: Is There a Relation Among Them?In Human-Computer Interaction – INTERACT 2021. Springer International Publishing, 22–42. https://doi.org/10.1007/978-3-030-85613-7_3

Oussama Bonnor. 2020. Color Switch Replica - GitHub. https://github.com/oussamabonnor1/Color_Switch_Replica

João Bosco Borges, Carmen Li Juy, Izac Sidarta De Andrade Matos, Paulo Vitor Angelo Silveira, and Ticianne De Gois Ribeiro Darin. 2020. Player Experience Evaluation: Which instrument should I use?Journal on Interactive Systems 11, 1 (Dec. 2020), 74–91. https://doi.org/10.5753/jis.2020.765

Barbaros Bostan (Ed.). 2020. Game User Experience And Player-Centered Design. Springer International Publishing. https://doi.org/10.1007/978-3-030-37643-7

Elizabeth A. Boyle, Thomas M. Connolly, Thomas Hainey, and James M. Boyle. 2012. Engagement in digital entertainment games: A systematic review. Computers in Human Behavior 28, 3 (May 2012), 771–780. https://doi.org/10.1016/j.chb.2011.11.020

Emily Brown and Paul Cairns. 2004. A Grounded Investigation of Game Immersion. In CHI ’04 Extended Abstracts on Human Factors in Computing Systems (Vienna, Austria) (CHI EA ’04). Association for Computing Machinery, New York, NY, USA, 1297–1300. https://doi.org/10.1145/985921.986048

Luciene Chagas de Oliveira, Edgard Afonso Lamounier, Adriano Oliveira Andrade, Renato Aquino Lopes, Samila Carolina da Costa, Igor Silvano de Oliveira, Jose Antonio Serra Carneiro, Pedro Daibert, and Alexandre Cardoso. 2020. Application of Serious Games based on Virtual Reality for Rehabilitation of Patients with Parkinson's Disease through a Wrist Orthosis. In 2020 22nd Symposium on Virtual and Augmented Reality (SVR). IEEE. https://doi.org/10.1109/svr51698.2020.00053

Leandro Agostini do Amaral, Renata Pontin de Mattos Fortes, Marcos Hortes Nisihara Chagas, Paula Costa Castro, Thiago Jabur Bittar, and Luanna Lopes Lobato. 2019. Towards adopting serious games for cognitive tests with Brazilian older people. Journal on Interactive Systems 10, 1 (Dec. 2019), 1. https://doi.org/10.5753/jis.2019.715

Anders Drachen, Magy Seif El-Nasr, and Alessandro Canossa. 2013. Game Analytics – The Basics. In Game Analytics. Springer London, 13–40. https://doi.org/10.1007/978-1-4471-4769-5_2

Cassandra Folkins, Emily Read, Jeff Mundee, Max V. Birk, and Scott Bateman. 2020. A Serious Game for Promoting Positive Attitudes Towards Nursing Homes Among Youth. In Proceedings of the Annual Symposium on Computer-Human Interaction in Play (Virtual Event, Canada) (CHI PLAY ’20). Association for Computing Machinery, New York, NY, USA, 484–498. https://doi.org/10.1145/3410404.3414253

Stefan Göbel and Viktor Wendel. 2016. Personalization and Adaptation. Springer International Publishing, Cham, 161–210. https://doi.org/10.1007/978-3-319-40612-1_7

Christian Guckelsberger, Christoph Salge, Jeremy Gow, and Paul Cairns. 2017. Predicting Player Experience without the Player.. In Proceedings of the Annual Symposium on Computer-Human Interaction in Play. ACM. https://doi.org/10.1145/3116595.3116631

Geoffrey Hookham and Keith Nesbitt. 2019. A Systematic Review of the Definition and Measurement of Engagement in Serious Games. In Proceedings of the Australasian Computer Science Week Multiconference. ACM. https://doi.org/10.1145/3290688.3290747

Ruud S. Jacobs. 2021. Winning Over the Players: Investigating the Motivations to Play and Acceptance of Serious Games. Media and Communication 9, 1 (Jan. 2021), 28–38. https://doi.org/10.17645/mac.v9i1.3308

Shin-Jin Kang, Young Bin Kim, Taejung Park, and Chang-Hun Kim. 2012. Automatic player behavior analysis system using trajectory data in a massive multiplayer online game. Multimedia Tools and Applications 66, 3 (March 2012), 383–404. https://doi.org/10.1007/s11042-012-1052-x

Raphaël Marczak, Jasper van Vught, Lennart E. Nacke, and Gareth Schott. 2012. Feedback-based gameplay metrics. In Proceedings of The 8th Australasian Conference on Interactive Entertainment Playing the System - IE '12. ACM Press. https://doi.org/10.1145/2336727.2336733

Rosa Mikeal Martey, Kate Kenski, James Folkestad, Laurie Feldman, Elana Gordis, Adrienne Shaw, Jennifer Stromer-Galley, Ben Clegg, Hui Zhang, Nissim Kaufman, Ari N. Rabkin, Samira Shaikh, and Tomek Strzalkowski. 2014. Measuring Game Engagement. Simulation & Gaming 45, 4-5 (Aug. 2014), 528–547. https://doi.org/10.1177/1046878114553575

Gevã Martins, Welington Veiga, Fernanda Campos, Victor Ströele, José Maria N. David, and Regina Braga. 2018. Building Educational Games from a Feature Model. In Proceedings of the XIV Brazilian Symposium on Information Systems (Caxias do Sul, Brazil) (SBSI’18). Association for Computing Machinery, New York, NY, USA, Article 3, 7 pages. https://doi.org/10.1145/3229345.3229349

Caitlin Mills, Sidney D'Mello, Blair Lehman, Nigel Bosch, Amber Strain, and Art Graesser. 2013. What Makes Learning Fun? Exploring the Influence of Choice and Difficulty on Mind Wandering and Engagement during Learning. In Artificial Intelligence in Education, H. Chad Lane, Kalina Yacef, Jack Mostow, and Philip Pavlik (Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg, 71–80. 

Belkacem Mostefai, Amar Balla, and Philippe Trigano. 2019. A generic and efficient emotion-driven approach toward personalized assessment and adaptation in serious games. Cognitive Systems Research 56 (aug 2019), 82–106. https://doi.org/10.1016/j.cogsys.2019.03.006

Teresa M. Ober, Corinne J. Brenner, Alvaro Olsen, Bruce D. Homer, and Jan L. Plass. 2021. Detecting patterns of engagement in a digital cognitive skills training game. Computers & Education 165 (May 2021), 104144. https://doi.org/10.1016/j.compedu.2021.104144

Heather L. O'Brien, Paul Cairns, and Mark Hall. 2018. A practical approach to measuring user engagement with the refined user engagement scale (UES) and new UES short form. International Journal of Human-Computer Studies 112 (April 2018), 28–39. https://doi.org/10.1016/j.ijhcs.2018.01.004

Wilk Oliveira, Luiz Rodrigues, Armando Toda, Paula Palomino, and Seiji Isotani. 2019. Automatic Game Experience Identification in Educational Games. In Anais do XXX Simpósio Brasileiro de Informática na Educação (SBIE 2019). Brazilian Computer Society (Sociedade Brasileira de Computação - SBC). https://doi.org/10.5753/cbie.sbie.2019.952

Raquel Robinson, John Murray, and Katherine Isbister. 2018. ”You're Giving Me Mixed Signals!”. In Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems. ACM. https://doi.org/10.1145/3170427.3188469

Raquel Robinson, Katelyn Wiley, Amir Rezaeivahdati, Madison Klarkowski, and Regan L. Mandryk. 2020. ”Let's Get Physiological, Physiological!”: A Systematic Review of Affective Gaming. In Proceedings of the Annual Symposium on Computer-Human Interaction in Play (Virtual Event, Canada) (CHI PLAY ’20). Association for Computing Machinery, New York, NY, USA, 132–147. https://doi.org/10.1145/3410404.3414227

Henrique S. Rossi, Sibele M. dos Santos, Raquel O. Prates, and Renato A. C. Ferreira. 2018. Imaginator: A virtual reality based game for the treatment of sensory processing disorders. In 2018 IEEE 6th International Conference on Serious Games and Applications for Health (SeGAH). IEEE. https://doi.org/10.1109/segah.2018.8401355

Josef Wiemeyer, Lennart Nacke, Christiane Moser, and Florian ‘Floyd’ Mueller. 2016. Player Experience. In Serious Games. Springer International Publishing, 243–271. https://doi.org/10.1007/978-3-319-40612-1_9