A Virtual Class Tool for Facial Expressions Telemonitoring through Embedded Systems

  • Murilo de Souza Preto UFABC
  • Fernando Teubl Ferreira UFABC
  • Celso Setsuo Kurashima UFABC

Resumo


In recent years, there has been a transition from face-to-face class meetings to videoconferences, which has been further expanded during the COVID-19 pandemic. In the area of education, remote classes entailed new advantages and some challenges – in one hand, allowing for an extended time and location flexibility – while, in the other hand, operating in a inexpressive, distanced, manner. Aiming to increase non-verbal communication in virtual class meetings, this research explores the technical feasibility and potential implementation challenges of developing a virtual class tool for facial expressions telemonitoring through embedded systems. As a suggested solution, a system which allows for a teacher to remotely follow the facial expressions of students through real-time rendered graphical data was developed. Technical experimental results include: an average recognition accuracy of 57%, for all facial expressions, peaking at 80% for happiness, in a median processing time of 2.85 seconds with a Raspberry Pi 4B, and of 0.572 seconds with a commodity computer. These findings highlight the technical viability of the implemented system, possibly allowing to decrease the lack of non-verbal communication in distanced learning.
Palavras-chave: Facial Expression, Image Processing, Embedded Systems, Computer Networks Based Systems

Referências

Cristina Bustos, Neska Elhaouij, Albert Sole-Ribalta, Javier Borge-Holthoefer, Agata Lapedriza, and Rosalind Picard. 2021. Predicting Driver Self-Reported Stress by Analyzing the Road Scene. 2021 9th International Conference on Affective Computing and Intelligent Interaction, ACII 2021 (9 2021). https://doi.org/10.1109/ACII52823.2021.9597438

Ulf Dimberg, Monika Thunberg, and Kurt Elmehed. 2000. Unconscious Facial Reactions to Emotional Facial Expressions. https://doi.org/10.1111/1467-9280.00221 11 (1 2000), 86–89. Issue 1. https://doi.org/10.1111/1467-9280.00221

C.N.W. Geraets, S. Klein Tuente, B.P. Lestestuiver, M. van Beilen, S.A. Nijman, J.B.C. Marsman, and W. Veling. 2021. Virtual reality facial emotion recognition in social environments: An eye-tracking study. Internet Interventions 25 (2021), 100432. https://doi.org/10.1016/j.invent.2021.100432

Zuheir N. Khlaif, Soheil Salha, and Bochra Kouraichi. 2021. Emergency remote learning during COVID-19 crisis: Students’ engagement. Education and Information Technologies 26, 6 (Nov. 2021), 7033–7055. https://doi.org/10.1007/s10639-021-10566-4

Ömer Koçak and İdris Göksu. 2023. Engagement of Higher Education Students in Live Online Classes: Scale Development and Validation. TechTrends 67, 3 (2023), 534–549. https://doi.org/10.1007/s11528-023-00849-7

Krithika L.B and Lakshmi Priya GG. 2016. Student Emotion Recognition System (SERS) for e-learning Improvement Based on Learner Concentration Metric. Procedia Computer Science 85 (2016), 767–776. https://doi.org/10.1016/j.procs.2016.05.264 International Conference on Computational Modelling and Security (CMS 2016).

Shan Li and Weihong Deng. 2022. Deep Facial Expression Recognition: A Survey. IEEE Transactions on Affective Computing 13, 3 (jul 2022), 1195–1215. https://doi.org/10.1109/taffc.2020.2981446

Patrick Lucey, Jeffrey F. Cohn, Takeo Kanade, Jason Saragih, Zara Ambadar, and Iain Matthews. 2010. The Extended Cohn-Kanade Dataset (CK+): A complete dataset for action unit and emotion-specified expression. In 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Workshops. 94– 101. https://doi.org/10.1109/CVPRW.2010.5543262

Qi Meng, Xuejun Hu, Jian Kang, and Yue Wu. 2020. On the effectiveness of facial expression recognition for evaluation of urban sound perception. Science of The Total Environment 710 (2020), 135484. https://doi.org/10.1016/j.scitotenv.2019.135484

Ali Mollahosseini, Behzad Hasani, and Mohammad H. Mahoor. 2019. AffectNet: A Database for Facial Expression, Valence, and Arousal Computing in the Wild. IEEE Transactions on Affective Computing 10, 1 (jan 2019), 18–31. https://doi.org/10.1109/taffc.2017.2740923

Simić Nataša, Kristina Mojović Zdravković, and Natalija Ignjatović. 2022. Student engagement in online and face-to-face classes in times of pandemic. Nastava i vaspitanje 71, 3 (2022), 347–362. https://doi.org/10.5937/nasvas2203347S

Luan Pham, The Huynh Vu, and Tuan Anh Tran. 2021. Facial Expression Recognition Using Residual Masking Network. In 2020 25th International Conference on Pattern Recognition (ICPR). 4513–4519. https://doi.org/10.1109/ICPR48806.2021.9411919
Publicado
06/11/2023
PRETO, Murilo de Souza; FERREIRA, Fernando Teubl; KURASHIMA, Celso Setsuo. A Virtual Class Tool for Facial Expressions Telemonitoring through Embedded Systems. In: SIMPÓSIO DE REALIDADE VIRTUAL E AUMENTADA (SVR), 25. , 2023, Rio Grande/RS. Anais [...]. Porto Alegre: Sociedade Brasileira de Computação, 2023 . p. 228–232.