Image preprocessing techniques for facial expression classification
Abstract
Facial expression classification is an essential aspect of human interaction, and its proper classification is considered fundamental for the future inclusion of multimedia psychology into human-computer interfaces. For this purpose, this article aims to identify a preprocessing pipeline capable of reduce the accuracy variance for facial expression classification. Thereunto, Extended Cohn-Kanade Dataset, Support Vector Machine, and Bag of Features were used in six pipelines. Compared to a pipeline with minimal preprocessing, the best results presented an accuracy variance reduction of 65.63%.
Keywords:
Facial expression, Image processing, Support Vector Machine, Feature computation, Bag of features
References
Keith Anderson and Peter W McOwan. 2006. A real-time automated system for the recognition of human facial expressions. IEEE transactions on systems, man, and cybernetics. Part B, Cybernetics : a publication of the IEEE Systems, Man, and Cybernetics Society 36, 1 (2006), 96–105. https://doi.org/10.1109/TSMCB.2005.854502
Paul Ekman and Wallace V Friesen. 1971. Constants across cultures in the face and emotion. Journal of personality and social psychology 17, 2 (1971), 124.
Paul Ekman and Erika L Rosenberg. 1997. What the face reveals: Basic and applied studies of spontaneous expression using the Facial Action Coding System (FACS). Oxford University Press, USA.
Xiaoyi Feng, Matti Pietikäinen, and Abdenour Hadid. 2007. Facial expression recognition based on local binary patterns. Pattern Recognition and Image Analysis 17, 4 (2007), 592–598.
Yunxin Huang, Fei Chen, Shaohe Lv, and Xiaodong Wang. 2019. Facial expression recognition: A survey. Symmetry 11, 10 (2019), 1189.
James J. Lien, Jeffrey F. Cohn, Takeo Kanade, and Ching Chung Li. 1998. Automated facial expression recognition based on FACS action units. Proceedings - 3rd IEEE International Conference on Automatic Face and Gesture Recognition, FG 1998 (1998), 390–395. https://doi.org/10.1109/AFGR.
Shuming Liu, Xiaopeng Chen, Di Fan, Xu Chen, Fei Meng, and Qiang Huang. 2016. 3D smiling facial expression recognition based on SVM. In 2016 IEEE International Conference on Mechatronics and Automation. IEEE, 1661–1666.
Yu Liu, Jing-dong Wang, and Peng Li. 2011. A feature point tracking method based on the combination of SIFT algorithm and KLT matching algorithm. Journal of Astronautics 32, 7 (2011), 1618–1625.
Patrick Lucey, Jeffrey F Cohn, Takeo Kanade, Jason Saragih, Zara Ambadar, and Iain Matthews. 2010. The extended cohn-kanade dataset (ck+): A complete dataset for action unit and emotion-specified expression. In Computer Vision and Pattern Recognition Workshops (CVPRW), 2010 IEEE Computer Society Conference on. IEEE, 94–101.
Qirong Mao, Qiyu Rao, Yongbin Yu, and Ming Dong. 2016. Hierarchical Bayesian theme models for multipose facial expression recognition. IEEE Transactions on Multimedia 19, 4 (2016), 861–873.
Kenji Mase. 1991. Recognition of facial expression from optical flow. IEICE transactions (E) 74 (1991), 3474–3483.
Albert Mehrabian and James A Russell. 1974. An approach to environmental psychology. the MIT Press.
Stephen O’Hara and Bruce A Draper. 2011. Introduction to the bag of features paradigm for image classification and retrieval. arXiv preprint arXiv:1101.3354 (2011).
Jamal Hussain Shah, Muhammad Sharif, Mussarat Yasmin, and Steven Lawrence Fernandes. 2020. Facial expressions classification and false label reduction using LDA and threefold SVM. Pattern Recognition Letters 139 (2020), 166–173.
Silvan S Tomkins. 1980. Affect as amplification: Some modifications in theory. Emotion: Theory, research, and experience 1 (1980), 141–164.
Paul Viola and Michael Jones. 2001. Rapid object detection using a boosted cascade of simple features. In Computer Vision and Pattern Recognition, 2001. CVPR 2001. Proceedings of the 2001 IEEE Computer Society Conference on, Vol. 1. IEEE, I–I.
Xiao-Hu Wang, An Liu, and Shi-Qing Zhang. 2015. New facial expression recognition based on FSVM and KNN. Optik 126, 21 (2015), 3132–3134.
Song Zhang, Bin Hu, Tiantian Li, and Xiangwei Zheng. 2018. A study on emotion recognition based on hierarchical adaboost multi-class algorithm. In International Conference on Algorithms and Architectures for Parallel Processing. Springer, 105–113.
Paul Ekman and Wallace V Friesen. 1971. Constants across cultures in the face and emotion. Journal of personality and social psychology 17, 2 (1971), 124.
Paul Ekman and Erika L Rosenberg. 1997. What the face reveals: Basic and applied studies of spontaneous expression using the Facial Action Coding System (FACS). Oxford University Press, USA.
Xiaoyi Feng, Matti Pietikäinen, and Abdenour Hadid. 2007. Facial expression recognition based on local binary patterns. Pattern Recognition and Image Analysis 17, 4 (2007), 592–598.
Yunxin Huang, Fei Chen, Shaohe Lv, and Xiaodong Wang. 2019. Facial expression recognition: A survey. Symmetry 11, 10 (2019), 1189.
James J. Lien, Jeffrey F. Cohn, Takeo Kanade, and Ching Chung Li. 1998. Automated facial expression recognition based on FACS action units. Proceedings - 3rd IEEE International Conference on Automatic Face and Gesture Recognition, FG 1998 (1998), 390–395. https://doi.org/10.1109/AFGR.
Shuming Liu, Xiaopeng Chen, Di Fan, Xu Chen, Fei Meng, and Qiang Huang. 2016. 3D smiling facial expression recognition based on SVM. In 2016 IEEE International Conference on Mechatronics and Automation. IEEE, 1661–1666.
Yu Liu, Jing-dong Wang, and Peng Li. 2011. A feature point tracking method based on the combination of SIFT algorithm and KLT matching algorithm. Journal of Astronautics 32, 7 (2011), 1618–1625.
Patrick Lucey, Jeffrey F Cohn, Takeo Kanade, Jason Saragih, Zara Ambadar, and Iain Matthews. 2010. The extended cohn-kanade dataset (ck+): A complete dataset for action unit and emotion-specified expression. In Computer Vision and Pattern Recognition Workshops (CVPRW), 2010 IEEE Computer Society Conference on. IEEE, 94–101.
Qirong Mao, Qiyu Rao, Yongbin Yu, and Ming Dong. 2016. Hierarchical Bayesian theme models for multipose facial expression recognition. IEEE Transactions on Multimedia 19, 4 (2016), 861–873.
Kenji Mase. 1991. Recognition of facial expression from optical flow. IEICE transactions (E) 74 (1991), 3474–3483.
Albert Mehrabian and James A Russell. 1974. An approach to environmental psychology. the MIT Press.
Stephen O’Hara and Bruce A Draper. 2011. Introduction to the bag of features paradigm for image classification and retrieval. arXiv preprint arXiv:1101.3354 (2011).
Jamal Hussain Shah, Muhammad Sharif, Mussarat Yasmin, and Steven Lawrence Fernandes. 2020. Facial expressions classification and false label reduction using LDA and threefold SVM. Pattern Recognition Letters 139 (2020), 166–173.
Silvan S Tomkins. 1980. Affect as amplification: Some modifications in theory. Emotion: Theory, research, and experience 1 (1980), 141–164.
Paul Viola and Michael Jones. 2001. Rapid object detection using a boosted cascade of simple features. In Computer Vision and Pattern Recognition, 2001. CVPR 2001. Proceedings of the 2001 IEEE Computer Society Conference on, Vol. 1. IEEE, I–I.
Xiao-Hu Wang, An Liu, and Shi-Qing Zhang. 2015. New facial expression recognition based on FSVM and KNN. Optik 126, 21 (2015), 3132–3134.
Song Zhang, Bin Hu, Tiantian Li, and Xiangwei Zheng. 2018. A study on emotion recognition based on hierarchical adaboost multi-class algorithm. In International Conference on Algorithms and Architectures for Parallel Processing. Springer, 105–113.
Published
2022-06-22
How to Cite
PEREIRA, Andre Luiz da S.; FERNANDES, Leandro A. F.; CONCI, Aura.
Image preprocessing techniques for facial expression classification. In: LIFE IMPROVEMENT IN QUALITY BY UBIQUITOUS EXPERIENCES WORKSHOP (LIQUE), 2. , 2022, Aveiro.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2022
.
p. 5-8.
DOI: https://doi.org/10.5753/lique.2022.19995.
