Expanding the Generalization Capability of Deep Learning Methods for Facial Expression Recognition

• Sergio Neres Pereira Junior UFSCar
• Jurandy Almeida UFSCar

DOI: https://doi.org/10.5753/sibgrapi.est.2025.38316

Abstract

Facial expression recognition is a fundamental task in computer vision, yet models often suffer from poor generalization to unseen data. This work aims to enhance the robustness and generalization of the CAFE architecture, a state-of-the-art model designed for FER in real-world scenarios. To this end, we propose a modification to its loss function by introducing a sparsity regularization term. This approach compels the model to focus on a more concise and discriminative subset of facial features, penalizing the activation of irrelevant contextual information. The experimental evaluation was conducted by comparing the modified model with its original version on intradomain and cross-domain benchmarks. The results demonstrate that sparsity regularization yielded a significant performance gain, particularly in cross-domain scenarios, thus confirming the hypothesis that the technique enhances generalization ability. We conclude that inducing sparsity is an effective strategy for developing more reliable and adaptable FER systems, capable of operating with greater accuracy in diverse and uncontrolled environments.

References

R. Jack and P. Schyns, “The human face as a dynamic tool for social communication,” Current Biology, vol. 25, no. 14, pp. R621–R634, 2015. [Online]. Available: [link]

A. R. Khan, “Facial emotion recognition using conventional machine learning and deep learning methods: Current achievements, analysis and remaining challenges,” 6 2022.

L. F. A. Silva, N. Sebe, and J. Almeida, “Tightening classification boundaries in open set domain adaptation through unknown exploitation,” in Conf. Graphics, Patterns and Images – SIBGRAPI, 2023, pp. 157–162.

L. F. A. Silva, S. F. dos Santos, N. Sebe, and J. Almeida, “Beyond the known: Enhancing open set domain adaptation with unknown exploration,” Pattern Recognit. Lett., vol. 189, pp. 265–272, 2025.

M. Karnati, A. Seal, D. Bhattacharjee, A. Yazidi, and O. Krejcar, “Understanding deep learning techniques for recognition of human emotions using facial expressions: A comprehensive survey,” IEEE Transactions on Instrumentation and Measurement, vol. 72, 2023.

Y. Zhang, X. Zheng, C. Liang, J. Hu, and W. Deng, “Generalizable facial expression recognition,” 2024. [Online]. Available: [link]

S. Li and W. Deng, “Deep facial expression recognition: A survey,” IEEE Transactions on Affective Computing, vol. 10, no. 3, pp. 325–345, 2019.

E. Sariyanidi, H. Gunes, and A. Cavallaro, “Automatic analysis of facial affect: A survey of registration, representation, and recognition,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 37, no. 6, pp. 1113–1133, 2015.

A. Mollahosseini, B. Hasani, and M. H. Mahoor, “AffectNet: A database for facial expression, valence, and arousal computing in the wild,” IEEE Transactions on Affective Computing, vol. 8, no. 1, pp. 18–31, 2017.

Z. Zhang, L. Qi, and Y. Liu, “Cross-domain facial expression recognition using domain adaptation,” Neurocomputing, vol. 275, pp. 2163–2172, 2018.

L. F. A. Silva, D. C. G. Pedronette, F. A. Faria, J. P. Papa, and J. Almeida, “Improving transferability of domain adaptation networks through domain alignment layers,” in Conf. Graphics, Patterns and Images – SIBGRAPI, 2021, pp. 168–175.

S. Li and W. Deng, “Reliable crowdsourcing and deep localitypreserving learning for unconstrained facial expression recognition,” IEEE Transactions on Image Processing, vol. 28, no. 1, pp. 356–370, 2019.

E. Barsoum, C. Zhang, C. C. Ferrer, and Z. Zhang, “Training deep networks for facial expression recognition with crowd-sourced label distribution,” 2016. [Online]. Available: [link]

A. Mollahosseini, B. Hasani, and M. H. Mahoor, “Affectnet: A database for facial expression, valence, and arousal computing in the wild,” IEEE Transactions on Affective Computing, vol. 10, no. 1, pp. 18–31, 2019.

D. Tang, S. Cheng, M. Fang, and M. H. Mahoor, “Static facial expressions in the wild (sfew): A database for facial expression recognition under real-world conditions,” in IEEE International Conference on Computer Vision Workshops (ICCVW), 2013, pp. 1324–1330. [Online]. Available: [link]

N. M. Hieu, “Mma facial expression dataset,” 2022, disponível em Kaggle. [Online]. Available: [link]