A Virtual Class Tool for Facial Expressions Telemonitoring through Embedded Systems
Resumo
In recent years, there has been a transition from face-to-face class meetings to videoconferences, which has been further expanded during the COVID-19 pandemic. In the area of education, remote classes entailed new advantages and some challenges – in one hand, allowing for an extended time and location flexibility – while, in the other hand, operating in a inexpressive, distanced, manner. Aiming to increase non-verbal communication in virtual class meetings, this research explores the technical feasibility and potential implementation challenges of developing a virtual class tool for facial expressions telemonitoring through embedded systems. As a suggested solution, a system which allows for a teacher to remotely follow the facial expressions of students through real-time rendered graphical data was developed. Technical experimental results include: an average recognition accuracy of 57%, for all facial expressions, peaking at 80% for happiness, in a median processing time of 2.85 seconds with a Raspberry Pi 4B, and of 0.572 seconds with a commodity computer. These findings highlight the technical viability of the implemented system, possibly allowing to decrease the lack of non-verbal communication in distanced learning.
Palavras-chave:
Facial Expression, Image Processing, Embedded Systems, Computer Networks Based Systems
Referências
Cristina Bustos, Neska Elhaouij, Albert Sole-Ribalta, Javier Borge-Holthoefer, Agata Lapedriza, and Rosalind Picard. 2021. Predicting Driver Self-Reported Stress by Analyzing the Road Scene. 2021 9th International Conference on Affective Computing and Intelligent Interaction, ACII 2021 (9 2021). https://doi.org/10.1109/ACII52823.2021.9597438
Ulf Dimberg, Monika Thunberg, and Kurt Elmehed. 2000. Unconscious Facial Reactions to Emotional Facial Expressions. https://doi.org/10.1111/1467-9280.00221 11 (1 2000), 86–89. Issue 1. https://doi.org/10.1111/1467-9280.00221
C.N.W. Geraets, S. Klein Tuente, B.P. Lestestuiver, M. van Beilen, S.A. Nijman, J.B.C. Marsman, and W. Veling. 2021. Virtual reality facial emotion recognition in social environments: An eye-tracking study. Internet Interventions 25 (2021), 100432. https://doi.org/10.1016/j.invent.2021.100432
Zuheir N. Khlaif, Soheil Salha, and Bochra Kouraichi. 2021. Emergency remote learning during COVID-19 crisis: Students’ engagement. Education and Information Technologies 26, 6 (Nov. 2021), 7033–7055. https://doi.org/10.1007/s10639-021-10566-4
Ömer Koçak and İdris Göksu. 2023. Engagement of Higher Education Students in Live Online Classes: Scale Development and Validation. TechTrends 67, 3 (2023), 534–549. https://doi.org/10.1007/s11528-023-00849-7
Krithika L.B and Lakshmi Priya GG. 2016. Student Emotion Recognition System (SERS) for e-learning Improvement Based on Learner Concentration Metric. Procedia Computer Science 85 (2016), 767–776. https://doi.org/10.1016/j.procs.2016.05.264 International Conference on Computational Modelling and Security (CMS 2016).
Shan Li and Weihong Deng. 2022. Deep Facial Expression Recognition: A Survey. IEEE Transactions on Affective Computing 13, 3 (jul 2022), 1195–1215. https://doi.org/10.1109/taffc.2020.2981446
Patrick Lucey, Jeffrey F. Cohn, Takeo Kanade, Jason Saragih, Zara Ambadar, and Iain Matthews. 2010. The Extended Cohn-Kanade Dataset (CK+): A complete dataset for action unit and emotion-specified expression. In 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Workshops. 94– 101. https://doi.org/10.1109/CVPRW.2010.5543262
Qi Meng, Xuejun Hu, Jian Kang, and Yue Wu. 2020. On the effectiveness of facial expression recognition for evaluation of urban sound perception. Science of The Total Environment 710 (2020), 135484. https://doi.org/10.1016/j.scitotenv.2019.135484
Ali Mollahosseini, Behzad Hasani, and Mohammad H. Mahoor. 2019. AffectNet: A Database for Facial Expression, Valence, and Arousal Computing in the Wild. IEEE Transactions on Affective Computing 10, 1 (jan 2019), 18–31. https://doi.org/10.1109/taffc.2017.2740923
Simić Nataša, Kristina Mojović Zdravković, and Natalija Ignjatović. 2022. Student engagement in online and face-to-face classes in times of pandemic. Nastava i vaspitanje 71, 3 (2022), 347–362. https://doi.org/10.5937/nasvas2203347S
Luan Pham, The Huynh Vu, and Tuan Anh Tran. 2021. Facial Expression Recognition Using Residual Masking Network. In 2020 25th International Conference on Pattern Recognition (ICPR). 4513–4519. https://doi.org/10.1109/ICPR48806.2021.9411919
Ulf Dimberg, Monika Thunberg, and Kurt Elmehed. 2000. Unconscious Facial Reactions to Emotional Facial Expressions. https://doi.org/10.1111/1467-9280.00221 11 (1 2000), 86–89. Issue 1. https://doi.org/10.1111/1467-9280.00221
C.N.W. Geraets, S. Klein Tuente, B.P. Lestestuiver, M. van Beilen, S.A. Nijman, J.B.C. Marsman, and W. Veling. 2021. Virtual reality facial emotion recognition in social environments: An eye-tracking study. Internet Interventions 25 (2021), 100432. https://doi.org/10.1016/j.invent.2021.100432
Zuheir N. Khlaif, Soheil Salha, and Bochra Kouraichi. 2021. Emergency remote learning during COVID-19 crisis: Students’ engagement. Education and Information Technologies 26, 6 (Nov. 2021), 7033–7055. https://doi.org/10.1007/s10639-021-10566-4
Ömer Koçak and İdris Göksu. 2023. Engagement of Higher Education Students in Live Online Classes: Scale Development and Validation. TechTrends 67, 3 (2023), 534–549. https://doi.org/10.1007/s11528-023-00849-7
Krithika L.B and Lakshmi Priya GG. 2016. Student Emotion Recognition System (SERS) for e-learning Improvement Based on Learner Concentration Metric. Procedia Computer Science 85 (2016), 767–776. https://doi.org/10.1016/j.procs.2016.05.264 International Conference on Computational Modelling and Security (CMS 2016).
Shan Li and Weihong Deng. 2022. Deep Facial Expression Recognition: A Survey. IEEE Transactions on Affective Computing 13, 3 (jul 2022), 1195–1215. https://doi.org/10.1109/taffc.2020.2981446
Patrick Lucey, Jeffrey F. Cohn, Takeo Kanade, Jason Saragih, Zara Ambadar, and Iain Matthews. 2010. The Extended Cohn-Kanade Dataset (CK+): A complete dataset for action unit and emotion-specified expression. In 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Workshops. 94– 101. https://doi.org/10.1109/CVPRW.2010.5543262
Qi Meng, Xuejun Hu, Jian Kang, and Yue Wu. 2020. On the effectiveness of facial expression recognition for evaluation of urban sound perception. Science of The Total Environment 710 (2020), 135484. https://doi.org/10.1016/j.scitotenv.2019.135484
Ali Mollahosseini, Behzad Hasani, and Mohammad H. Mahoor. 2019. AffectNet: A Database for Facial Expression, Valence, and Arousal Computing in the Wild. IEEE Transactions on Affective Computing 10, 1 (jan 2019), 18–31. https://doi.org/10.1109/taffc.2017.2740923
Simić Nataša, Kristina Mojović Zdravković, and Natalija Ignjatović. 2022. Student engagement in online and face-to-face classes in times of pandemic. Nastava i vaspitanje 71, 3 (2022), 347–362. https://doi.org/10.5937/nasvas2203347S
Luan Pham, The Huynh Vu, and Tuan Anh Tran. 2021. Facial Expression Recognition Using Residual Masking Network. In 2020 25th International Conference on Pattern Recognition (ICPR). 4513–4519. https://doi.org/10.1109/ICPR48806.2021.9411919
Publicado
06/11/2023
Como Citar
PRETO, Murilo de Souza; FERREIRA, Fernando Teubl; KURASHIMA, Celso Setsuo.
A Virtual Class Tool for Facial Expressions Telemonitoring through Embedded Systems. In: SIMPÓSIO DE REALIDADE VIRTUAL E AUMENTADA (SVR), 25. , 2023, Rio Grande/RS.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2023
.
p. 228–232.
