Sobel filter and linear classification for deepfake analysis of faces
Abstract
Through artificial intelligence (AI) techniques, deepfakes (deep learning + fakes) are created. Currently, almost six years after this topic's popularization, some people have been using AI algorithms to change, for example, face. In this scenario, this article proposes to identify the most discriminating characteristics, based on the multivariate statistical learning techniques, with and without the Sobel filter. Complementarily, we aim to analyze the areas that humans have visually identified as relevant to detect a deepfake. The results show that the statistical model (PCA plus MLDA) combined with the Sobel filter correctly classified most of the images, highlighting the regions that discriminate a deepfake.
Keywords:
Deepfakes, Sobel, PCA, Eye-tracking
References
Agarwal, S. and Farid, H. (2021). Detecting deep-fake videos from aural and oral dynamics. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 981–989.
Bulat, A. and Tzimiropoulos, G. (2017). How far are we from solving the 2d & 3d face alignment problem?(and a dataset of 230,000 3d facial landmarks). In Proceedings of the IEEE international conference on computer vision, pages 1021–1030.
Caporusso, N., Zhang, K., and Carlson, G. (2020). Using eye-tracking to study the authenticity of images produced by generative adversarial networks. In 2020 International Conference on Electrical, Communication, and Computer Engineering (ICECCE), pages 1–6. IEEE.
Chen, T., Kumar, A., Nagarsheth, P., Sivaraman, G., and Khoury, E. (2020). Generalization of audio deepfake detection. In Proc. Odyssey 2020 The Speaker and Language Recognition Workshop, pages 132–137.
Demir, I. and Ciftci, U. A. (2021). Where do deep fakes look? synthetic face detection via gaze tracking. In ACM Symposium on Eye Tracking Research and Applications, pages 1–11.
Fukunaga, K. and Hummels, D. M. (1989). Leave-one-out procedures for nonparametric error estimates. IEEE Transactions on Pattern Analysis and Machine Intelligence, 11(4):421–423.
Gerstner, C. R. and Farid, H. (2022). Detecting real-time deep-fake videos using active illumination. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 53–60.
Juefei-Xu, F., Wang, R., Huang, Y., Guo, Q., Ma, L., and Liu, Y. (2022). Countering malicious deepfakes: Survey, battleground, and horizon. International Journal of Computer Vision, 130(7):1678–1734.
Kanopoulos, N., Vasanthavada, N., and Baker, R. L. (1988). Design of an image edge detection filter using the sobel operator. IEEE Journal of solid-state circuits, 23(2):358–367.
Kietzmann, J., Lee, L. W., McCarthy, I. P., and Kietzmann, T. C. (2020). Deepfakes: Trick or treat? Business Horizons, 63(2):135–146.
Kohavi, R. et al. (1995). A study of cross-validation and bootstrap for accuracy estimation and model selection. In Ijcai, volume 14, pages 1137–1145. Montreal, Canada.
Li, M., Liu, B., Hu, Y., Zhang, L., and Wang, S. (2021). Deepfake detection using robust spatial and temporal features from facial landmarks. In 2021 IEEE International Workshop on Biometrics and Forensics (IWBF), pages 1–6. IEEE.
Li, Y., Yang, X., Sun, P., Qi, H., and Lyu, S. (2020). Celeb-df: A large-scale challenging dataset for deepfake forensics. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 3207–3216.
Masood, M., Nawaz, M., Malik, K. M., Javed, A., Irtaza, A., and Malik, H. (2022). Deepfakes generation and detection: State-of-the-art, open challenges, countermeasures, and way forward. Applied Intelligence, pages 1–53.
Mitra, A., Mohanty, S. P., Corcoran, P., and Kougianos, E. (2021). A machine learning based approach for deepfake detection in social media through key video frame extraction. SN Computer Science, 2(2):1–18.
Mittal, T., Bhattacharya, U., Chandra, R., Bera, A., and Manocha, D. (2020). Emotions don’t lie: An audio-visual deepfake detection method using affective cues. In Proceedings of the 28th ACM international conference on multimedia, pages 2823–2832.
Mohiuddin, S., Sheikh, K. H., Malakar, S., Velásquez, J. D., and Sarkar, R. (2023). A hierarchical feature selection strategy for deepfake video detection. Neural Computing and Applications, 35(13):9363–9380.
Nguyen, H. M. and Derakhshani, R. (2020). Eyebrow recognition for identifying deepfake videos. In 2020 international conference of the biometrics special interest group (BIOSIG), pages 1–5. IEEE.
Rana, M. S., Nobi, M. N., Murali, B., and Sung, A. H. (2022). Deepfake detection: a systematic literature review. IEEE Access.
Rössler, A., Cozzolino, D., Verdoliva, L., Riess, C., Thies, J., and Nießner, M. (2018). Faceforensics: A large-scale video dataset for forgery detection in human faces. arXiv preprint arXiv:1803.09179.
Rossler, A., Cozzolino, D., Verdoliva, L., Riess, C., Thies, J., and Nießner, M. (2019). Faceforensics++: Learning to detect manipulated facial images. In Proceedings of the IEEE/CVF international conference on computer vision, pages 1–11.
Tariq, S., Jeon, S., and Woo, S. S. (2021). Am i a real or fake celebrity? measuring commercial face recognition web apis under deepfake impersonation attack. arXiv preprint arXiv:2103.00847.
Thomaz, C. E., Kitani, E. C., and Gillies, D. F. (2006). A maximum uncertainty lda-based approach for limited sample size problems—with application to face recognition. Journal of the Brazilian Computer Society, 12(2):7–18.
Tolosana, R., Romero-Tapiador, S., Fierrez, J., and Vera-Rodriguez, R. (2021). Deepfakes evolution: Analysis of facial regions and fake detection performance. In International Conference on Pattern Recognition, pages 442–456. Springer.
Tolosana, R., Vera-Rodriguez, R., Fierrez, J., Morales, A., and Ortega-Garcia, J. (2020). Deepfakes and beyond: A survey of face manipulation and fake detection. Information Fusion, 64:131–148.
Wang, G., Jiang, Q., Jin, X., and Cui, X. (2022). Ffr fd: Effective and fast detection of deepfakes via feature point defects. Information Sciences, 596:472–488.
Wold, S., Esbensen, K., and Geladi, P. (1987). Principal component analysis. Chemometrics and intelligent laboratory systems, 2(1-3):37–52.
Zhang, S., Zhu, X., Lei, Z., Shi, H., Wang, X., and Li, S. Z. (2017). S3fd: Single shot scale-invariant face detector. In Proceedings of the IEEE international conference on computer vision, pages 192–201.
Bulat, A. and Tzimiropoulos, G. (2017). How far are we from solving the 2d & 3d face alignment problem?(and a dataset of 230,000 3d facial landmarks). In Proceedings of the IEEE international conference on computer vision, pages 1021–1030.
Caporusso, N., Zhang, K., and Carlson, G. (2020). Using eye-tracking to study the authenticity of images produced by generative adversarial networks. In 2020 International Conference on Electrical, Communication, and Computer Engineering (ICECCE), pages 1–6. IEEE.
Chen, T., Kumar, A., Nagarsheth, P., Sivaraman, G., and Khoury, E. (2020). Generalization of audio deepfake detection. In Proc. Odyssey 2020 The Speaker and Language Recognition Workshop, pages 132–137.
Demir, I. and Ciftci, U. A. (2021). Where do deep fakes look? synthetic face detection via gaze tracking. In ACM Symposium on Eye Tracking Research and Applications, pages 1–11.
Fukunaga, K. and Hummels, D. M. (1989). Leave-one-out procedures for nonparametric error estimates. IEEE Transactions on Pattern Analysis and Machine Intelligence, 11(4):421–423.
Gerstner, C. R. and Farid, H. (2022). Detecting real-time deep-fake videos using active illumination. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 53–60.
Juefei-Xu, F., Wang, R., Huang, Y., Guo, Q., Ma, L., and Liu, Y. (2022). Countering malicious deepfakes: Survey, battleground, and horizon. International Journal of Computer Vision, 130(7):1678–1734.
Kanopoulos, N., Vasanthavada, N., and Baker, R. L. (1988). Design of an image edge detection filter using the sobel operator. IEEE Journal of solid-state circuits, 23(2):358–367.
Kietzmann, J., Lee, L. W., McCarthy, I. P., and Kietzmann, T. C. (2020). Deepfakes: Trick or treat? Business Horizons, 63(2):135–146.
Kohavi, R. et al. (1995). A study of cross-validation and bootstrap for accuracy estimation and model selection. In Ijcai, volume 14, pages 1137–1145. Montreal, Canada.
Li, M., Liu, B., Hu, Y., Zhang, L., and Wang, S. (2021). Deepfake detection using robust spatial and temporal features from facial landmarks. In 2021 IEEE International Workshop on Biometrics and Forensics (IWBF), pages 1–6. IEEE.
Li, Y., Yang, X., Sun, P., Qi, H., and Lyu, S. (2020). Celeb-df: A large-scale challenging dataset for deepfake forensics. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 3207–3216.
Masood, M., Nawaz, M., Malik, K. M., Javed, A., Irtaza, A., and Malik, H. (2022). Deepfakes generation and detection: State-of-the-art, open challenges, countermeasures, and way forward. Applied Intelligence, pages 1–53.
Mitra, A., Mohanty, S. P., Corcoran, P., and Kougianos, E. (2021). A machine learning based approach for deepfake detection in social media through key video frame extraction. SN Computer Science, 2(2):1–18.
Mittal, T., Bhattacharya, U., Chandra, R., Bera, A., and Manocha, D. (2020). Emotions don’t lie: An audio-visual deepfake detection method using affective cues. In Proceedings of the 28th ACM international conference on multimedia, pages 2823–2832.
Mohiuddin, S., Sheikh, K. H., Malakar, S., Velásquez, J. D., and Sarkar, R. (2023). A hierarchical feature selection strategy for deepfake video detection. Neural Computing and Applications, 35(13):9363–9380.
Nguyen, H. M. and Derakhshani, R. (2020). Eyebrow recognition for identifying deepfake videos. In 2020 international conference of the biometrics special interest group (BIOSIG), pages 1–5. IEEE.
Rana, M. S., Nobi, M. N., Murali, B., and Sung, A. H. (2022). Deepfake detection: a systematic literature review. IEEE Access.
Rössler, A., Cozzolino, D., Verdoliva, L., Riess, C., Thies, J., and Nießner, M. (2018). Faceforensics: A large-scale video dataset for forgery detection in human faces. arXiv preprint arXiv:1803.09179.
Rossler, A., Cozzolino, D., Verdoliva, L., Riess, C., Thies, J., and Nießner, M. (2019). Faceforensics++: Learning to detect manipulated facial images. In Proceedings of the IEEE/CVF international conference on computer vision, pages 1–11.
Tariq, S., Jeon, S., and Woo, S. S. (2021). Am i a real or fake celebrity? measuring commercial face recognition web apis under deepfake impersonation attack. arXiv preprint arXiv:2103.00847.
Thomaz, C. E., Kitani, E. C., and Gillies, D. F. (2006). A maximum uncertainty lda-based approach for limited sample size problems—with application to face recognition. Journal of the Brazilian Computer Society, 12(2):7–18.
Tolosana, R., Romero-Tapiador, S., Fierrez, J., and Vera-Rodriguez, R. (2021). Deepfakes evolution: Analysis of facial regions and fake detection performance. In International Conference on Pattern Recognition, pages 442–456. Springer.
Tolosana, R., Vera-Rodriguez, R., Fierrez, J., Morales, A., and Ortega-Garcia, J. (2020). Deepfakes and beyond: A survey of face manipulation and fake detection. Information Fusion, 64:131–148.
Wang, G., Jiang, Q., Jin, X., and Cui, X. (2022). Ffr fd: Effective and fast detection of deepfakes via feature point defects. Information Sciences, 596:472–488.
Wold, S., Esbensen, K., and Geladi, P. (1987). Principal component analysis. Chemometrics and intelligent laboratory systems, 2(1-3):37–52.
Zhang, S., Zhu, X., Lei, Z., Shi, H., Wang, X., and Li, S. Z. (2017). S3fd: Single shot scale-invariant face detector. In Proceedings of the IEEE international conference on computer vision, pages 192–201.
Published
2023-09-25
How to Cite
TAMANAKA, Fernanda G.; THOMAZ, Carlos E..
Sobel filter and linear classification for deepfake analysis of faces. In: NATIONAL MEETING ON ARTIFICIAL AND COMPUTATIONAL INTELLIGENCE (ENIAC), 20. , 2023, Belo Horizonte/MG.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2023
.
p. 15-27.
ISSN 2763-9061.
DOI: https://doi.org/10.5753/eniac.2023.233505.
