Testes com usuários para análise de emoções em conteúdos audiovisuais utilizando EEG e eye tracking
Resumo
This paper describes methods for testing and analyzing brain waves during the consumption of audiovisual content, carried out with 10 individuals, in order to identify patterns of unconscious emotions that can influence an individual’s decision, particularly whether they enjoyed a specific content and if there is a predisposition to watch it in a movie theather. This research aims to define user testing methods within an emotion identification system using EEG (electroencephalography), with the goal of assessing the accuracy and usability of the system in detecting and interpreting users’ emotions based on brain activities captured by EEG. This testing approach is crucial for validating the practical applicability of the system and understanding its behavior in a real-world environment with real users. It’s based on the Design Science Research (DSR) method and utilizes the Emotiv Insight EEG headset for brain wave capture, incorporating eye tracking to map individuals’ eye movements. For the tests, two questionnaires were administered. A preliminary one was used to gather participants’ psychological and physical states, and a post-test was conducted to collect feedback on the content viewed and self-assessments of emotional states. In conclusion, the results demonstrate the effectiveness of the techniques within the applied context, indicating progress in the evaluation of audiovisual content by reflecting unconsciously generated emotions and providing insight into the perceived content.
Referências
Sarah N. Abdulkader, Ayman Atia, and Mostafa-Sami M. Mostafa. 2015. Brain computer interfacing: Applications and challenges. Egyptian Informatics Journal 16, 2 (July 2015), 213–230. https://doi.org/10.1016/j.eij.2015.06.002
Bryan C. ; PEREIRA JUNIOR Antônio ; MEDEIROS Adelardo A. D.de . BARBOSA, André F. ; SOUZA. 2009. Implementação de classificador de tarefas mentais baseado em EEG. Anais do IX Congresso Brasileiro de Redes Neurais, Inteligência Computacional (IX CBRN), Ouro Preto, MG, Brasil. [link]. [Accessed 30-07-2023].
Valdecir Becker, Daniel Gambaro, Thais Saraiva Ramos, and Rafael Moura Toscano. 2018. Audiovisual Design: Introducing ‘Media Affordances’ as a Relevant Concept for the Development of a New Communication Model. In Applications and Usability of Interactive Television. Springer International Publishing, 17–31. https://doi.org/10.1007/978-3-319-90170-1_2
Margaret M. Bradley and Peter J. Lang. 1994. Measuring emotion: The self-assessment manikin and the semantic differential. Journal of Behavior Therapy and Experimental Psychiatry 25, 1 (1994), 49–59. https://doi.org/10.1016/0005-7916(94)90063-9
Blain Brown. 2016. Cinematography: theory and practice: image making for cinematographers and directors. Taylor & Francis.
Thiago Henrique Coelho Tavares da Silva, Matheus Dantas Cavalcanti, Isaac Nóbrega Marinho, and Valdecir Becker. 2022. Developing a System for Graphical Analysis of Brainwaves During Media Consumption. In Anais Estendidos do XXVIII Simpósio Brasileiro de Sistemas Multimídia e Web (WebMedia 2022). Sociedade Brasileira de Computação - SBC. https://doi.org/10.5753/webmedia_estendido.2022.227061.
Aline Dresch, Daniel Pacheco Lacerda, and Junico Antunes. 2015. Design Science Research: Método de Pesquisa para Avanço da Ciência e Tecnologia. Bookman. https://doi.org/10.13140/2.1.2264.2885
Jérémy Frey, Jelena Mladenović, Fabien Lotte, Camille Jeunet, and Léa Pillette. 2017. When HCI Meets Neurotechnologies. In Proceedings of the 2017 CHI Conference Extended Abstracts on Human Factors in Computing Systems. ACM. https://doi.org/10.1145/3027063.3027100
Crystal A Gabert-Quillen, Ellen E Bartolini, Benjamin T Abravanel, and Charles A Sanislow. 2015. Ratings for emotion film clips. Behavior research methods 47 (2015), 773–787. https://doi.org/10.3758/s13428-014-0500-0
Zhen Gao and Shangfei Wang. 2015. Emotion recognition from EEG signals using hierarchical Bayesian network with privileged information. In Proceedings of the 5th ACM on International Conference on Multimedia Retrieval. 579–582. https://doi.org/10.1145/2671188.2749364
James J Gross and RobertWLevenson. 1995. Emotion elicitation using films. Cognition & emotion 9, 1 (1995), 87–108. https://doi.org/10.1080/02699939508408966
AR Hevner, ST March, J Park, and S Ram. 2004. Design science in information systems research. MIS Q 28 (1): 75–105.
Yang Li, Wenming Zheng, Yuan Zong, Zhen Cui, Tong Zhang, and Xiaoyan Zhou. 2021. A Bi-Hemisphere Domain Adversarial Neural Network Model for EEG Emotion Recognition. IEEE Transactions on Affective Computing 12 (2021), 494–504. https://doi.org/10.1109/TAFFC.2018.2885474
Juan-Miguel López-Gil, Jordi Virgili-Gomá, Rosa Gil, and Roberto García. 2016. Method for Improving EEG Based Emotion Recognition by Combining It with Synchronized Biometric and Eye Tracking Technologies in a Non-invasive and Low CostWay. Frontiers in Computational Neuroscience 10 (Aug. 2016). https://doi.org/10.3389/fncom.2016.00085
Yifei Lu, Wei-Long Zheng, Binbin Li, and Bao-Liang Lu. 2015. Combining eye movements and EEG to enhance emotion recognition. In Twenty-Fourth International Joint Conference on Artificial Intelligence.
Wagner Rodrigues Miranda. 2017. Netflix: Big Data e os algoritmos de recomendação1. Anais do XXXII INTERCOM, Rio de Janeiro (2017).
Gelareh Mohammadi and Patrik Vuilleumier. 2020. A multicomponential approach to emotion recognition and the effect of personality. IEEE Transactions on Affective Computing 13, 3 (2020), 1127–1139.
Rose Marie Santini. 2020. O Algoritmo do Gosto: Os Sistemas de Recomendação On-Line e seus Impactos no Mercado Cultural;: Volume 1. Editora Appris.
Alexandre Schaefer, Frédéric Nils, Xavier Sanchez, and Pierre Philippot. 2010. Assessing the effectiveness of a large database of emotion-eliciting films: A new tool for emotion researchers. Cognition and emotion 24, 7 (2010), 1153–1172. https://doi.org/10.1080/02699930903274322
Jan van Erp, Fabien Lotte, and Michael Tangermann. 2012. Brain-Computer Interfaces: Beyond Medical Applications. Computer 45, 4 (2012), 26–34. https://doi.org/10.1109/MC.2012.107
Vamsi Vijay Mohan Dattada andMJeevan. 2019. Analysis of Concealed Anger Emotion in a Neutral Speech Signal. In 2019 IEEE International Conference on Distributed Computing, VLSI, Electrical Circuits and Robotics (DISCOVER). 1–5. https://doi.org/10.1109/DISCOVER47552.2019.9008037
