Sobel filter and linear classification for deepfake analysis of faces
Resumo
Por meio de técnicas de inteligência artificial (IA) são criadas deepfakes. Atualmente, quase seis anos após a popularização desse tema, algumas pessoas estão utilizando algoritmos de IA para alterar, por exemplo, o rosto. Neste cenário, este artigo tem como proposta aplicar o filtro Sobel, em conjunto com técnicas multivariadas de aprendizagem estatística, para a identificação das características mais discriminantes. Complementarmente, visamos analisar as áreas que humanos visualmente identificaram como relevantes para detectar uma deepfake. Os resultados mostram que o modelo estatístico (PCA mais MLDA) combinado com o filtro Sobel classificou corretamente a maioria das imagens, realçando as regiões que discriminam uma deepfake.
Palavras-chave:
Deepfakes, Sobel, PCA, Rastreamento ocular
Referências
Agarwal, S. and Farid, H. (2021). Detecting deep-fake videos from aural and oral dynamics. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 981–989.
Bulat, A. and Tzimiropoulos, G. (2017). How far are we from solving the 2d & 3d face alignment problem?(and a dataset of 230,000 3d facial landmarks). In Proceedings of the IEEE international conference on computer vision, pages 1021–1030.
Caporusso, N., Zhang, K., and Carlson, G. (2020). Using eye-tracking to study the authenticity of images produced by generative adversarial networks. In 2020 International Conference on Electrical, Communication, and Computer Engineering (ICECCE), pages 1–6. IEEE.
Chen, T., Kumar, A., Nagarsheth, P., Sivaraman, G., and Khoury, E. (2020). Generalization of audio deepfake detection. In Proc. Odyssey 2020 The Speaker and Language Recognition Workshop, pages 132–137.
Demir, I. and Ciftci, U. A. (2021). Where do deep fakes look? synthetic face detection via gaze tracking. In ACM Symposium on Eye Tracking Research and Applications, pages 1–11.
Fukunaga, K. and Hummels, D. M. (1989). Leave-one-out procedures for nonparametric error estimates. IEEE Transactions on Pattern Analysis and Machine Intelligence, 11(4):421–423.
Gerstner, C. R. and Farid, H. (2022). Detecting real-time deep-fake videos using active illumination. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 53–60.
Juefei-Xu, F., Wang, R., Huang, Y., Guo, Q., Ma, L., and Liu, Y. (2022). Countering malicious deepfakes: Survey, battleground, and horizon. International Journal of Computer Vision, 130(7):1678–1734.
Kanopoulos, N., Vasanthavada, N., and Baker, R. L. (1988). Design of an image edge detection filter using the sobel operator. IEEE Journal of solid-state circuits, 23(2):358–367.
Kietzmann, J., Lee, L. W., McCarthy, I. P., and Kietzmann, T. C. (2020). Deepfakes: Trick or treat? Business Horizons, 63(2):135–146.
Kohavi, R. et al. (1995). A study of cross-validation and bootstrap for accuracy estimation and model selection. In Ijcai, volume 14, pages 1137–1145. Montreal, Canada.
Li, M., Liu, B., Hu, Y., Zhang, L., and Wang, S. (2021). Deepfake detection using robust spatial and temporal features from facial landmarks. In 2021 IEEE International Workshop on Biometrics and Forensics (IWBF), pages 1–6. IEEE.
Li, Y., Yang, X., Sun, P., Qi, H., and Lyu, S. (2020). Celeb-df: A large-scale challenging dataset for deepfake forensics. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 3207–3216.
Masood, M., Nawaz, M., Malik, K. M., Javed, A., Irtaza, A., and Malik, H. (2022). Deepfakes generation and detection: State-of-the-art, open challenges, countermeasures, and way forward. Applied Intelligence, pages 1–53.
Mitra, A., Mohanty, S. P., Corcoran, P., and Kougianos, E. (2021). A machine learning based approach for deepfake detection in social media through key video frame extraction. SN Computer Science, 2(2):1–18.
Mittal, T., Bhattacharya, U., Chandra, R., Bera, A., and Manocha, D. (2020). Emotions don’t lie: An audio-visual deepfake detection method using affective cues. In Proceedings of the 28th ACM international conference on multimedia, pages 2823–2832.
Mohiuddin, S., Sheikh, K. H., Malakar, S., Velásquez, J. D., and Sarkar, R. (2023). A hierarchical feature selection strategy for deepfake video detection. Neural Computing and Applications, 35(13):9363–9380.
Nguyen, H. M. and Derakhshani, R. (2020). Eyebrow recognition for identifying deepfake videos. In 2020 international conference of the biometrics special interest group (BIOSIG), pages 1–5. IEEE.
Rana, M. S., Nobi, M. N., Murali, B., and Sung, A. H. (2022). Deepfake detection: a systematic literature review. IEEE Access.
Rössler, A., Cozzolino, D., Verdoliva, L., Riess, C., Thies, J., and Nießner, M. (2018). Faceforensics: A large-scale video dataset for forgery detection in human faces. arXiv preprint arXiv:1803.09179.
Rossler, A., Cozzolino, D., Verdoliva, L., Riess, C., Thies, J., and Nießner, M. (2019). Faceforensics++: Learning to detect manipulated facial images. In Proceedings of the IEEE/CVF international conference on computer vision, pages 1–11.
Tariq, S., Jeon, S., and Woo, S. S. (2021). Am i a real or fake celebrity? measuring commercial face recognition web apis under deepfake impersonation attack. arXiv preprint arXiv:2103.00847.
Thomaz, C. E., Kitani, E. C., and Gillies, D. F. (2006). A maximum uncertainty lda-based approach for limited sample size problems—with application to face recognition. Journal of the Brazilian Computer Society, 12(2):7–18.
Tolosana, R., Romero-Tapiador, S., Fierrez, J., and Vera-Rodriguez, R. (2021). Deepfakes evolution: Analysis of facial regions and fake detection performance. In International Conference on Pattern Recognition, pages 442–456. Springer.
Tolosana, R., Vera-Rodriguez, R., Fierrez, J., Morales, A., and Ortega-Garcia, J. (2020). Deepfakes and beyond: A survey of face manipulation and fake detection. Information Fusion, 64:131–148.
Wang, G., Jiang, Q., Jin, X., and Cui, X. (2022). Ffr fd: Effective and fast detection of deepfakes via feature point defects. Information Sciences, 596:472–488.
Wold, S., Esbensen, K., and Geladi, P. (1987). Principal component analysis. Chemometrics and intelligent laboratory systems, 2(1-3):37–52.
Zhang, S., Zhu, X., Lei, Z., Shi, H., Wang, X., and Li, S. Z. (2017). S3fd: Single shot scale-invariant face detector. In Proceedings of the IEEE international conference on computer vision, pages 192–201.
Bulat, A. and Tzimiropoulos, G. (2017). How far are we from solving the 2d & 3d face alignment problem?(and a dataset of 230,000 3d facial landmarks). In Proceedings of the IEEE international conference on computer vision, pages 1021–1030.
Caporusso, N., Zhang, K., and Carlson, G. (2020). Using eye-tracking to study the authenticity of images produced by generative adversarial networks. In 2020 International Conference on Electrical, Communication, and Computer Engineering (ICECCE), pages 1–6. IEEE.
Chen, T., Kumar, A., Nagarsheth, P., Sivaraman, G., and Khoury, E. (2020). Generalization of audio deepfake detection. In Proc. Odyssey 2020 The Speaker and Language Recognition Workshop, pages 132–137.
Demir, I. and Ciftci, U. A. (2021). Where do deep fakes look? synthetic face detection via gaze tracking. In ACM Symposium on Eye Tracking Research and Applications, pages 1–11.
Fukunaga, K. and Hummels, D. M. (1989). Leave-one-out procedures for nonparametric error estimates. IEEE Transactions on Pattern Analysis and Machine Intelligence, 11(4):421–423.
Gerstner, C. R. and Farid, H. (2022). Detecting real-time deep-fake videos using active illumination. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 53–60.
Juefei-Xu, F., Wang, R., Huang, Y., Guo, Q., Ma, L., and Liu, Y. (2022). Countering malicious deepfakes: Survey, battleground, and horizon. International Journal of Computer Vision, 130(7):1678–1734.
Kanopoulos, N., Vasanthavada, N., and Baker, R. L. (1988). Design of an image edge detection filter using the sobel operator. IEEE Journal of solid-state circuits, 23(2):358–367.
Kietzmann, J., Lee, L. W., McCarthy, I. P., and Kietzmann, T. C. (2020). Deepfakes: Trick or treat? Business Horizons, 63(2):135–146.
Kohavi, R. et al. (1995). A study of cross-validation and bootstrap for accuracy estimation and model selection. In Ijcai, volume 14, pages 1137–1145. Montreal, Canada.
Li, M., Liu, B., Hu, Y., Zhang, L., and Wang, S. (2021). Deepfake detection using robust spatial and temporal features from facial landmarks. In 2021 IEEE International Workshop on Biometrics and Forensics (IWBF), pages 1–6. IEEE.
Li, Y., Yang, X., Sun, P., Qi, H., and Lyu, S. (2020). Celeb-df: A large-scale challenging dataset for deepfake forensics. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 3207–3216.
Masood, M., Nawaz, M., Malik, K. M., Javed, A., Irtaza, A., and Malik, H. (2022). Deepfakes generation and detection: State-of-the-art, open challenges, countermeasures, and way forward. Applied Intelligence, pages 1–53.
Mitra, A., Mohanty, S. P., Corcoran, P., and Kougianos, E. (2021). A machine learning based approach for deepfake detection in social media through key video frame extraction. SN Computer Science, 2(2):1–18.
Mittal, T., Bhattacharya, U., Chandra, R., Bera, A., and Manocha, D. (2020). Emotions don’t lie: An audio-visual deepfake detection method using affective cues. In Proceedings of the 28th ACM international conference on multimedia, pages 2823–2832.
Mohiuddin, S., Sheikh, K. H., Malakar, S., Velásquez, J. D., and Sarkar, R. (2023). A hierarchical feature selection strategy for deepfake video detection. Neural Computing and Applications, 35(13):9363–9380.
Nguyen, H. M. and Derakhshani, R. (2020). Eyebrow recognition for identifying deepfake videos. In 2020 international conference of the biometrics special interest group (BIOSIG), pages 1–5. IEEE.
Rana, M. S., Nobi, M. N., Murali, B., and Sung, A. H. (2022). Deepfake detection: a systematic literature review. IEEE Access.
Rössler, A., Cozzolino, D., Verdoliva, L., Riess, C., Thies, J., and Nießner, M. (2018). Faceforensics: A large-scale video dataset for forgery detection in human faces. arXiv preprint arXiv:1803.09179.
Rossler, A., Cozzolino, D., Verdoliva, L., Riess, C., Thies, J., and Nießner, M. (2019). Faceforensics++: Learning to detect manipulated facial images. In Proceedings of the IEEE/CVF international conference on computer vision, pages 1–11.
Tariq, S., Jeon, S., and Woo, S. S. (2021). Am i a real or fake celebrity? measuring commercial face recognition web apis under deepfake impersonation attack. arXiv preprint arXiv:2103.00847.
Thomaz, C. E., Kitani, E. C., and Gillies, D. F. (2006). A maximum uncertainty lda-based approach for limited sample size problems—with application to face recognition. Journal of the Brazilian Computer Society, 12(2):7–18.
Tolosana, R., Romero-Tapiador, S., Fierrez, J., and Vera-Rodriguez, R. (2021). Deepfakes evolution: Analysis of facial regions and fake detection performance. In International Conference on Pattern Recognition, pages 442–456. Springer.
Tolosana, R., Vera-Rodriguez, R., Fierrez, J., Morales, A., and Ortega-Garcia, J. (2020). Deepfakes and beyond: A survey of face manipulation and fake detection. Information Fusion, 64:131–148.
Wang, G., Jiang, Q., Jin, X., and Cui, X. (2022). Ffr fd: Effective and fast detection of deepfakes via feature point defects. Information Sciences, 596:472–488.
Wold, S., Esbensen, K., and Geladi, P. (1987). Principal component analysis. Chemometrics and intelligent laboratory systems, 2(1-3):37–52.
Zhang, S., Zhu, X., Lei, Z., Shi, H., Wang, X., and Li, S. Z. (2017). S3fd: Single shot scale-invariant face detector. In Proceedings of the IEEE international conference on computer vision, pages 192–201.
Publicado
25/09/2023
Como Citar
TAMANAKA, Fernanda G.; THOMAZ, Carlos E..
Sobel filter and linear classification for deepfake analysis of faces. In: ENCONTRO NACIONAL DE INTELIGÊNCIA ARTIFICIAL E COMPUTACIONAL (ENIAC), 20. , 2023, Belo Horizonte/MG.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2023
.
p. 15-27.
ISSN 2763-9061.
DOI: https://doi.org/10.5753/eniac.2023.233505.
