Classificação de Gamers por Padrões em suas Expressões Faciais
Resumo
Este trabalho apresenta como principais contribuições um modelo de mineração dos dados das expressões faciais de jogadores de jogos digitais e o agrupamento desses jogadores por padrões. A base de dados autoral foi gerada utilizando um sistema de visão computacional capaz de auditar reações emocionais contendo um híbrido de sentimentos. As técnicas de mineração utilizadas foram aplicadas nos dados com as variações das 6 emoções básicas felicidade, tristeza, medo, surpresa, raiva e nojo, coletados durante o uso dos jogos, e nos questionários respondidos depois do uso desses mesmos jogos. Após a extração e o tratamento desses dados, foi possível agrupar os jogadores de acordo com padrões em suas expressões faciais e com as respostas dos questionários. Por fim, como forma de visualização de cada grupo de jogadores, os resultados gerados a partir dos centroides, após equalizados de acordo com o histograma dos valores das suas 6 emoções básicas, foram inseridos em um simulador de expressões faciais.
Palavras-chave:
Computação Afetiva, Jogos Digitais, Interação Humano Computador, Mineração de Dados, Reconhecimento de Padrões
Referências
R. W. Picard, "Affective Computing," MIT Press: Cambridge, MA, USA, 2000.
F. Bevilacqua, H. Engström, P. Backlund, "Game-calibrated and usertailored remote detection of stress and boredom in games," Sensors (Switzerland), vol. 19, no. 13, Jul. 2019, doi: 10.3390/s19132877.
A. Dzedzickis, A. Kaklauskas, V. Bucinskas, "Human Emotion Recognition: Review of Sensors and Methods," Sensors. 2020; 20(3):592. https://doi.org/10.3390/s20030592
E. D. Mekler, J. A. Bopp, A. N. Tuch, K. Opwis, "A systematic review of quantitative studies on the enjoyment of digital entertainment games," in Proceedings of the 32nd Annual ACM Conference on Human Factors in Computing Systems, CHI 2014, pp. 927-936, May 2014.
S. Jerritta, M. Murugappan, R. Nagarajan, K. Wan, "Physiological signals based human emotion Recognition: A review," In Proceedings of the 2011 IEEE 7th International Colloquium on Signal Processing and its Applications, Penang, Malaysia, 4-6 March 2011; Institute of Electrical and Electronics Engineers (IEEE): Piscataway Township, NJ, USA, 2011; pp. 410-415.
R. L. Mandryk, M. S. Atkins, K. M. Inkpen, "A continuous and objective evaluation of emotional experience with interactive play environments," In Proceedings of the SIGCHI conference on Human Factors in computing systems, Montréal, QC, Canada, 22-27 April 2006; ACM: New York, NY, USA, 2006; pp. 1027-1036.
F. Bousefsaf, C. Maaoui, A. Pruski, "Remote assessment of the Heart Rate Variability to detect mental stress," In Proceedings of the 2013 7th International Conference on Pervasive Computing Technologies for Healthcare and Workshops, Venice, Italy, 5-8 May 2013; pp. 348-351.
C. Yun, D. Shastri, I. Pavlidis, Z. Deng, "O'game, can you feel my frustration?," In Proceedings of the 27th International Conference on Human Factors in Computing Systems-CHI 09, Boston, MA, USA, 4-9 April 2009; Association for Computing Machinery (ACM): New York, NY, USA, 2009; pp. 2195-2204.
P. Rani, C. Liu, N. Sarkar, E. Vanman, "An empirical study of machine learning techniques for affect recognition in human-robot interaction," Pattern Anal. Appl. 2006, 9, 58-69.
T.J.W. Tijs, D. Brokken, W.A. IJsselsteijn, "Dynamic game balancing by recognizing afect," in Fun and Games, vol. 5294 of Lecture Notes in Computer Science, pp. 88-93, Springer, Berlin, Germany, 2008.
D. Kukolja, S. Popovic, M. Horvat, B. Kovac, K. Cosic, "Comparative analysis of emotion estimation' methods based on physiological measurements for real-time applications," Int. J. Hum. Comput. Stud. 2014, 72, 717-727.
H. Zacharatos, C. Gatzoulis, Y. L. Chrysanthou, "Automatic emotion recognition based on body movement analysis: A survey," IEEE Computer Graphics and Applications, vol. 34, no. 6, article no. 106, pp. 35-45, 2014.
T. Yamakoshi, K. Yamakoshi, S. Tanaka et al., "A preliminary study on driver's stress index using a new method based on differential skin temperature measurement," in Proceedings of the 29th Annual International Conference of IEEE-EMBS, Engineering in Medicine and Biology Society, EMBC'07, pp. 722-725, August 2007.
M. Yamaguchi, J. Wakasugi, J. Sakakima, "Evaluation of driver stress using biomarker in motor-vehicle driving simulator," in Proceedings of the Conference Proceedings. Annual International Conference of the IEEE Engineering in Medicine and Biology Society, pp. 1834-1837, New York, NY, August 2006.
J. A. Healey, R. W. Picard, "Detecting stress during real-world driving tasks using physiological sensors," IEEE Transactions on Intelligent Transportation Systems, vol. 6, no. 2, pp. 156-166, 2005.
J. F. Cohn, F. De la Torre, "Automated Face Analysis for Affective," The Oxford handbook of affective computing, 2014.
C. T. Tan, S. Bakkes, Y. Pisan, "Inferring player experiences using facial expressions analysis," in Proceedings of the 10th Australian Conference on Interactive Entertainment, IE 2014, pp. 1-8, ACM Press, December 2014.
C. T. Tan, S. Bakkes, Y. Pisan, "Correlation between facial expressions and the game experience questionnaire," in Proceedings of the Entertainment Computing-ICEC 2014: 13th International Conference, vol. 8770, p. 229, Springer, Sydney, Australia, October 2014.
X. Zhou, X. Huang, Y. Wang, "Real-time facial expression recognition in the interactive game based on embedded hidden markov model," in Proceedings of the Computer Graphics, Imaging and Visualization, pp. 144-148, Penang, Malaysia, 2004.
C. Zhan, W. Li, P. Ogunbona, F. Safaei, "A real-time facial expression recognition system for online games," International Journal of Computer Games Technology, vol. 2008, pp. 1-7, 2008.
J. Schell, "The Art of Game Design: A book of Lenses," CRC Press: Boca Raton, FL, USA, 2014.
P. Sweetser, P. Wyeth, "GameFlow: a model for evaluating player enjoyment in games," Comput. Entertain. 2005. DOI=10.1145/1077246.1077253. Disponível em: <http://doi.acm.org/10.1145/1077246.1077253> Acesso em 03/05/2021.
J. Chen, "Flow in games (and everything else)," Commun. ACM, p. 31-34. 2007.
R. Koster, "The Theory of Fun for Game Design," 2a. ed. O'Reilly Media, p. 300. 2013.
C. Schrader, J. Brich, J. Frommel, V. Riemer, K. Rogers, "Rising to the challenge: an emotion-driven approach toward adaptive serious games," in Serious Games and Edutainment Applications, pp. 3-28, Springer, 2017.
R. L. Hazlett, "Measuring emotional valence during interactive experiences: boys at video game play," in Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 1023-1026, 2006.
N. Ravaja, T. Saari, M. Salminen, J. Laarni, K. Kallinen, "Phasic emotional reactions to video game events: a psychophysiological investigation," Media Psychology, vol. 8, no. 4, pp. 343-367, 2006.
A. A. Salah, N. Sebe, T. Gevers, "Communication and automatic interpretation of affect from facial expressions," Affective Computing and Interaction: Psychological, Cognitive and Neuroscientific Perspectives, pp. 157-183, 2010.
A. Samara, L. Galway, R. Bond, H. Wang, "Sensing affective states using facial expression analysis," in Ubiquitous Computing and Ambient Intelligence, Lecture Notes in Computer Science, pp. 341-352, Springer International Publishing, 2016.
C. Y. Chang, J. S. Tsai, C. J. Wang, P. C. Chung, "Emotion recognition with consideration of facial expression and physiological signals," in Proceedings of the 2009 IEEE Symposium on Computational Intelligence in Bioinformatics and Computational Biology, CIBCB 2009, pp. 278-283, April 2009.
Z. Hammal, L. Couvreur, A. Caplier, and M. Rombaut, "Facial expression classification: An approach based on the fusion of facial deformations using the transferable belief model," International Journal of Approximate Reasoning, vol. 46, no. 3, pp. 542-567, 2007
G. A. Abrantes, F. Pereira, "MPEG-4 facial animation technology: survey, implementation, and results," IEEE Transactions on Circuits and Systems for Video Technology, vol. 9, no. 2, pp. 290-305, 1999.
H. Tang, T. S. Huang, "3D Facial expression recognition based on automatically selected features," in Proceedings of the 2008 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, CVPR Workshops, pp. 1-8, June 2008.
H. Tang, T. S. Huang, "3d facial expression recognition based on properties of line segments connecting facial feature points," in Proceedings of the Automatic Face & Gesture Recognition, 8th IEEE International Conference on IEEE, pp. 1-6, 2008.
I. Hupont, S. Baldassarri, E. Cerezo, "Facial emotional classification: from a discrete perspective to a continuous emotional space," PAA. Pattern Analysis and Applications, vol. 16, no. 1, pp. 41-54, 2013.
H. Ç. Akakn, B. Sankur, "Spatiotemporal-boosted DCT features for head and face gesture analysis," in Human Behavior Understanding, pp. 64-74, Springer Nature, 2010.
M. S. Bartlett, J. C. Hager, P. Ekman, T. J. Sejnowski, "Measuring facial expressions by computer image analysis," Psychophysiology, vol. 36, no. 2, pp. 253-263, 1999.
S. Kaiser, T. Wehrle, P. Edwards, "Multi-modal emotion measurement in an interactive computer game: A pilot-study," in in Proceedings of the VIII conference of the international society for research on emotions, pp. 275-279, ISRE Publications Storrs, 1994.
T. Wehrle, S. Kaiser, "Emotion and facial expression," in Affective Interactions, vol. 1814 of Lecture Notes in Computer Science, pp. 49-63, Springer, Berlin, Germany, 2000.
J. F. Grafsgaard, J. B. Wiggins, K. E. Boyer, E. N. Wiebe, J. C. Lester, "Automatically recognizing facial expression: Predicting engagement and frustration," in EDM, pp. 43-50, 2013.
D. Heylen, M. Ghijsen, A. Nijholt, R. Op Den Akker, "Facial signs of affect during tutoring sessions," Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics): Preface, vol. 3784, pp. 24-31, 2005
R. E. Jack, "Culture and facial expressions of emotion," Visual Cognition, vol. 21, no. 9-10, pp. 1248-1286, 2013.
J. M. Saragih, S. Lucey, J. F. Cohn, "Deformable model fitting by regularized landmark mean-shift," Int J Comput Vis 91(2):200-215. 2011. Disponível em: https://dl.acm.org/doi/10.1007/s11263-010-0380-4. Acesso em: 27 abr. 2021 às 14hs50min.
M. Prensky, "Digital Natives, Digital Immigrants," On the Horizon, Vol. 9 No. 5, pp. 1-6. https://doi.org/10.1108/10748120110424816, 2001.
P. Nerurkar, et al. "Empirical Analysis of Data Clustering Algorithms," Procedia Computer Science. Volume 125, Pages 770-779. doi: https://doi.org/10.1016/j.procs.2017.12.099, 2018.
S. M. L. Lima, A. G. Silva-Filho, W. P. Santos, "Morphological Decomposition to Detect and Classify Lesions in Mammograms," In: Wellington Pinheiro dos Santos; Maíra Araújo de Santana; Washington Wagner Azevedo da Silva. (Org.). Understanding a Cancer Diagnosis, New York: Nova Science. https://novapublishers.com/shop/understanding-a-cancer-diagnosis/, 2020 a.
F. Bevilacqua, H. Engström, P. Backlund, "Game-calibrated and usertailored remote detection of stress and boredom in games," Sensors (Switzerland), vol. 19, no. 13, Jul. 2019, doi: 10.3390/s19132877.
A. Dzedzickis, A. Kaklauskas, V. Bucinskas, "Human Emotion Recognition: Review of Sensors and Methods," Sensors. 2020; 20(3):592. https://doi.org/10.3390/s20030592
E. D. Mekler, J. A. Bopp, A. N. Tuch, K. Opwis, "A systematic review of quantitative studies on the enjoyment of digital entertainment games," in Proceedings of the 32nd Annual ACM Conference on Human Factors in Computing Systems, CHI 2014, pp. 927-936, May 2014.
S. Jerritta, M. Murugappan, R. Nagarajan, K. Wan, "Physiological signals based human emotion Recognition: A review," In Proceedings of the 2011 IEEE 7th International Colloquium on Signal Processing and its Applications, Penang, Malaysia, 4-6 March 2011; Institute of Electrical and Electronics Engineers (IEEE): Piscataway Township, NJ, USA, 2011; pp. 410-415.
R. L. Mandryk, M. S. Atkins, K. M. Inkpen, "A continuous and objective evaluation of emotional experience with interactive play environments," In Proceedings of the SIGCHI conference on Human Factors in computing systems, Montréal, QC, Canada, 22-27 April 2006; ACM: New York, NY, USA, 2006; pp. 1027-1036.
F. Bousefsaf, C. Maaoui, A. Pruski, "Remote assessment of the Heart Rate Variability to detect mental stress," In Proceedings of the 2013 7th International Conference on Pervasive Computing Technologies for Healthcare and Workshops, Venice, Italy, 5-8 May 2013; pp. 348-351.
C. Yun, D. Shastri, I. Pavlidis, Z. Deng, "O'game, can you feel my frustration?," In Proceedings of the 27th International Conference on Human Factors in Computing Systems-CHI 09, Boston, MA, USA, 4-9 April 2009; Association for Computing Machinery (ACM): New York, NY, USA, 2009; pp. 2195-2204.
P. Rani, C. Liu, N. Sarkar, E. Vanman, "An empirical study of machine learning techniques for affect recognition in human-robot interaction," Pattern Anal. Appl. 2006, 9, 58-69.
T.J.W. Tijs, D. Brokken, W.A. IJsselsteijn, "Dynamic game balancing by recognizing afect," in Fun and Games, vol. 5294 of Lecture Notes in Computer Science, pp. 88-93, Springer, Berlin, Germany, 2008.
D. Kukolja, S. Popovic, M. Horvat, B. Kovac, K. Cosic, "Comparative analysis of emotion estimation' methods based on physiological measurements for real-time applications," Int. J. Hum. Comput. Stud. 2014, 72, 717-727.
H. Zacharatos, C. Gatzoulis, Y. L. Chrysanthou, "Automatic emotion recognition based on body movement analysis: A survey," IEEE Computer Graphics and Applications, vol. 34, no. 6, article no. 106, pp. 35-45, 2014.
T. Yamakoshi, K. Yamakoshi, S. Tanaka et al., "A preliminary study on driver's stress index using a new method based on differential skin temperature measurement," in Proceedings of the 29th Annual International Conference of IEEE-EMBS, Engineering in Medicine and Biology Society, EMBC'07, pp. 722-725, August 2007.
M. Yamaguchi, J. Wakasugi, J. Sakakima, "Evaluation of driver stress using biomarker in motor-vehicle driving simulator," in Proceedings of the Conference Proceedings. Annual International Conference of the IEEE Engineering in Medicine and Biology Society, pp. 1834-1837, New York, NY, August 2006.
J. A. Healey, R. W. Picard, "Detecting stress during real-world driving tasks using physiological sensors," IEEE Transactions on Intelligent Transportation Systems, vol. 6, no. 2, pp. 156-166, 2005.
J. F. Cohn, F. De la Torre, "Automated Face Analysis for Affective," The Oxford handbook of affective computing, 2014.
C. T. Tan, S. Bakkes, Y. Pisan, "Inferring player experiences using facial expressions analysis," in Proceedings of the 10th Australian Conference on Interactive Entertainment, IE 2014, pp. 1-8, ACM Press, December 2014.
C. T. Tan, S. Bakkes, Y. Pisan, "Correlation between facial expressions and the game experience questionnaire," in Proceedings of the Entertainment Computing-ICEC 2014: 13th International Conference, vol. 8770, p. 229, Springer, Sydney, Australia, October 2014.
X. Zhou, X. Huang, Y. Wang, "Real-time facial expression recognition in the interactive game based on embedded hidden markov model," in Proceedings of the Computer Graphics, Imaging and Visualization, pp. 144-148, Penang, Malaysia, 2004.
C. Zhan, W. Li, P. Ogunbona, F. Safaei, "A real-time facial expression recognition system for online games," International Journal of Computer Games Technology, vol. 2008, pp. 1-7, 2008.
J. Schell, "The Art of Game Design: A book of Lenses," CRC Press: Boca Raton, FL, USA, 2014.
P. Sweetser, P. Wyeth, "GameFlow: a model for evaluating player enjoyment in games," Comput. Entertain. 2005. DOI=10.1145/1077246.1077253. Disponível em: <http://doi.acm.org/10.1145/1077246.1077253> Acesso em 03/05/2021.
J. Chen, "Flow in games (and everything else)," Commun. ACM, p. 31-34. 2007.
R. Koster, "The Theory of Fun for Game Design," 2a. ed. O'Reilly Media, p. 300. 2013.
C. Schrader, J. Brich, J. Frommel, V. Riemer, K. Rogers, "Rising to the challenge: an emotion-driven approach toward adaptive serious games," in Serious Games and Edutainment Applications, pp. 3-28, Springer, 2017.
R. L. Hazlett, "Measuring emotional valence during interactive experiences: boys at video game play," in Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 1023-1026, 2006.
N. Ravaja, T. Saari, M. Salminen, J. Laarni, K. Kallinen, "Phasic emotional reactions to video game events: a psychophysiological investigation," Media Psychology, vol. 8, no. 4, pp. 343-367, 2006.
A. A. Salah, N. Sebe, T. Gevers, "Communication and automatic interpretation of affect from facial expressions," Affective Computing and Interaction: Psychological, Cognitive and Neuroscientific Perspectives, pp. 157-183, 2010.
A. Samara, L. Galway, R. Bond, H. Wang, "Sensing affective states using facial expression analysis," in Ubiquitous Computing and Ambient Intelligence, Lecture Notes in Computer Science, pp. 341-352, Springer International Publishing, 2016.
C. Y. Chang, J. S. Tsai, C. J. Wang, P. C. Chung, "Emotion recognition with consideration of facial expression and physiological signals," in Proceedings of the 2009 IEEE Symposium on Computational Intelligence in Bioinformatics and Computational Biology, CIBCB 2009, pp. 278-283, April 2009.
Z. Hammal, L. Couvreur, A. Caplier, and M. Rombaut, "Facial expression classification: An approach based on the fusion of facial deformations using the transferable belief model," International Journal of Approximate Reasoning, vol. 46, no. 3, pp. 542-567, 2007
G. A. Abrantes, F. Pereira, "MPEG-4 facial animation technology: survey, implementation, and results," IEEE Transactions on Circuits and Systems for Video Technology, vol. 9, no. 2, pp. 290-305, 1999.
H. Tang, T. S. Huang, "3D Facial expression recognition based on automatically selected features," in Proceedings of the 2008 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, CVPR Workshops, pp. 1-8, June 2008.
H. Tang, T. S. Huang, "3d facial expression recognition based on properties of line segments connecting facial feature points," in Proceedings of the Automatic Face & Gesture Recognition, 8th IEEE International Conference on IEEE, pp. 1-6, 2008.
I. Hupont, S. Baldassarri, E. Cerezo, "Facial emotional classification: from a discrete perspective to a continuous emotional space," PAA. Pattern Analysis and Applications, vol. 16, no. 1, pp. 41-54, 2013.
H. Ç. Akakn, B. Sankur, "Spatiotemporal-boosted DCT features for head and face gesture analysis," in Human Behavior Understanding, pp. 64-74, Springer Nature, 2010.
M. S. Bartlett, J. C. Hager, P. Ekman, T. J. Sejnowski, "Measuring facial expressions by computer image analysis," Psychophysiology, vol. 36, no. 2, pp. 253-263, 1999.
S. Kaiser, T. Wehrle, P. Edwards, "Multi-modal emotion measurement in an interactive computer game: A pilot-study," in in Proceedings of the VIII conference of the international society for research on emotions, pp. 275-279, ISRE Publications Storrs, 1994.
T. Wehrle, S. Kaiser, "Emotion and facial expression," in Affective Interactions, vol. 1814 of Lecture Notes in Computer Science, pp. 49-63, Springer, Berlin, Germany, 2000.
J. F. Grafsgaard, J. B. Wiggins, K. E. Boyer, E. N. Wiebe, J. C. Lester, "Automatically recognizing facial expression: Predicting engagement and frustration," in EDM, pp. 43-50, 2013.
D. Heylen, M. Ghijsen, A. Nijholt, R. Op Den Akker, "Facial signs of affect during tutoring sessions," Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics): Preface, vol. 3784, pp. 24-31, 2005
R. E. Jack, "Culture and facial expressions of emotion," Visual Cognition, vol. 21, no. 9-10, pp. 1248-1286, 2013.
J. M. Saragih, S. Lucey, J. F. Cohn, "Deformable model fitting by regularized landmark mean-shift," Int J Comput Vis 91(2):200-215. 2011. Disponível em: https://dl.acm.org/doi/10.1007/s11263-010-0380-4. Acesso em: 27 abr. 2021 às 14hs50min.
M. Prensky, "Digital Natives, Digital Immigrants," On the Horizon, Vol. 9 No. 5, pp. 1-6. https://doi.org/10.1108/10748120110424816, 2001.
P. Nerurkar, et al. "Empirical Analysis of Data Clustering Algorithms," Procedia Computer Science. Volume 125, Pages 770-779. doi: https://doi.org/10.1016/j.procs.2017.12.099, 2018.
S. M. L. Lima, A. G. Silva-Filho, W. P. Santos, "Morphological Decomposition to Detect and Classify Lesions in Mammograms," In: Wellington Pinheiro dos Santos; Maíra Araújo de Santana; Washington Wagner Azevedo da Silva. (Org.). Understanding a Cancer Diagnosis, New York: Nova Science. https://novapublishers.com/shop/understanding-a-cancer-diagnosis/, 2020 a.
Publicado
18/10/2021
Como Citar
JOFILSAN, Nicolau Calado; LIMA, Sidney Marlon Lopes de; ALENCAR, Renan Costa; SILVA, Sthéfano Henrique M. Tavares; ALBUQUERQUE, João Victor Oliveira de; SOARES FILHO, Odilon Herculano; MACIEL, Alexandre Magno Andrade; SANTOS, Wellington Pinheiro dos.
Classificação de Gamers por Padrões em suas Expressões Faciais. In: TRILHA DE INDÚSTRIA – ARTIGOS COMPLETOS - SIMPÓSIO BRASILEIRO DE JOGOS E ENTRETENIMENTO DIGITAL (SBGAMES), 20. , 2021, Online.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2021
.
p. 796-805.
DOI: https://doi.org/10.5753/sbgames_estendido.2021.21516.
