Automatic Emotion Detection in the Learning of Algorithms

• Paulo Victor Borges Oliveira Lima UFMA
• Djefferson Maranhão UFMA
• Carlos De Salles Soares UFMA

Resumo

Problem-based learning is a methodology seen repeatedly in the literature for teaching algorithms and data structures. There are numerous online judge platforms available, where the student solves computational problems and their codes are automatically corrected by comparing whether the output response matches the expected feedback for that problem. Despite its popularity, there are few works that associate the use of emotion detection techniques as a diagnostic tool to assess the quality of these problems, leading to a more specific analysis of the successes, frustrations and joys of students in this task. This article presents a technique that defines a timeline describing the step-by-step of a student when solving a computational problem, creating notes about the instants of time in which feelings were perceived in all these stages, from reading the problem, to coding and sending the judge’s response online. In this context, a contribution of this article is the development of an API for detecting emotions in facial expressions, using Deep Learning techniques and convolutional neural networks. The API is accessible and scalable via HTTP endpoints and allows for easy integration into third-party applications. The first results show that real-time emotion detection can be a valuable tool to improve the user experience on online judge platforms. The technique makes it possible to stratify the diagnosis of problems, analyzing whether there is a presence or predominance of undesirable feelings in any of the stages, be it reading, coding or submission of the solution.

Palavras-chave: neural networks, emotion recognition, machine learning, deep learning

Referências

MAH Akhand, Shuvendu Roy, Nazmul Siddique, Md Abdus Samad Kamal, and Tetsuya Shimamura. 2021. Facial emotion recognition using transfer learning in the deep CNN. Electronics 10, 9 (2021), 1036

Deborah E Allen, Richard S Donham, and Stephen A Bernhardt. 2011. Problem-based learning. New directions for teaching and learning 2011, 128 (2011), 21–29

Kiwook Bae, Heechang Ryu, and Hayong Shin. 2019. Does Adam optimizer keep close to the optimal point?arXiv preprint arXiv:1911.00289 (2019)

Puja Bharati and Ankita Pramanik. 2020. Deep learning techniques—R-CNN to mask R-CNN: a survey. Computational Intelligence in Pattern Recognition: Proceedings of CIPR 2019 (2020), 657–668

Junguk Cho, Shahnam Mirzaei, Jason Oberg, and Ryan Kastner. 2009. Fpga-based face detection system using haar classifiers. In Proceedings of the ACM/SIGDA international symposium on Field programmable gate arrays. 103–112.

DANIEL PAULO DE ASSIS. [n. d.]. Uma Arquitetura de Software Baseada em Web Services. ([n. d.])

Abhinav Dhall, Roland Goecke, Jyoti Joshi, Karan Sikka, and Tom Gedeon. 2014. Emotion recognition in the wild challenge 2014: Baseline, data and protocol. In Proceedings of the 16th international conference on multimodal interaction. 461–466.

Gilse A Morgental Falckembach and Fabrício Viero de Araujo. 2006. Aprendizagem de algoritmos: dificuldades na resolução de problemas. Anais Sulcomp 2 (2006)

Ian J Goodfellow, Dumitru Erhan, Pierre Luc Carrier, Aaron Courville, Mehdi Mirza, Ben Hamner, Will Cukierski, Yichuan Tang, David Thaler, Dong-Hyun Lee, 2013. Challenges in representation learning: A report on three machine learning contests. In Neural Information Processing: 20th International Conference, ICONIP 2013, Daegu, Korea, November 3-7, 2013. Proceedings, Part III 20. Springer, 117–124

H Jabbar and Rafiqul Zaman Khan. 2015. Methods to avoid over-fitting and under-fitting in supervised machine learning (comparative study). Computer Science, Communication and Instrumentation Devices 70 (2015), 163–172

Gerald M Karam. 1994. Visualization using timelines. In Proceedings of the 1994 ACM SIGSOFT international symposium on Software testing and analysis. 125–137.

Gede Putra Kusuma, J Jonathan, and AP Lim. 2020. Emotion recognition on fer-2013 face images using fine-tuned vgg-16. Advances in Science, Technology and Engineering Systems Journal 5, 6 (2020), 315–322

Malhar Lathkar. 2023. Getting Started with FastAPI. In High-Performance Web Apps with FastAPI: The Asynchronous Web Framework Based on Modern Python. Springer, 29–64

S. Li and W. Deng. 2022. Deep Facial Expression Recognition: A Survey. IEEE Transactions on Affective Computing 13, 03 (jul 2022), 1195–1215. https://doi.org/10.1109/TAFFC.2020.2981446

Arilo Neto and Dias Claudio. 2007. Introdução a teste de software. Engenharia de Software Magazine 1 (2007), 22

Rafael Padilla, CFF Costa Filho, and MGF Costa. 2012. Evaluation of haar cascade classifiers designed for face detection. World Academy of Science, Engineering and Technology 64 (2012), 362–365

Henk G Schmidt, Jerome I Rotgans, and Elaine HJ Yew. 2011. The process of problem-based learning: what works and why. Medical education 45, 8 (2011), 792–806

Connor Shorten and Taghi M Khoshgoftaar. 2019. A survey on image data augmentation for deep learning. Journal of big data 6, 1 (2019), 1–48

Karl Weiss, Taghi M Khoshgoftaar, and DingDing Wang. 2016. A survey of transfer learning. Journal of Big data 3, 1 (2016), 1–40

Ramon Zatarain-Cabada, Maria Lucia Barron-Estrada, Francisco Gonzalez-Hernandez, and Hector Rodriguez-Rangel. 2017. Building a face expression recognizer and a face expression database for an intelligent tutoring system. In 2017 IEEE 17th International Conference on Advanced Learning Technologies (ICALT). IEEE, 391–393

Ruizhe Zhao, Xinyu Niu, Yajie Wu, Wayne Luk, and Qiang Liu. 2017. Optimizing CNN-based object detection algorithms on embedded FPGA platforms. In Applied Reconfigurable Computing: 13th International Symposium, ARC 2017, Delft, The Netherlands, April 3-7, 2017, Proceedings 13. Springer, 255–267.