A Comparative Study on Synthetic Facial Data Generation Techniques for Face Recognition

• Pedro Vidal UFPR
• Bernardo Biesseck UFPR
• Luiz Coelho Unico – IDTech
• Roger Granada Unico – IDTech
• David Menotti UFPR

DOI: https://doi.org/10.5753/sibgrapi.est.2024.31662

Resumo

Face recognition has become a widely adopted method for user authentication and identification, with applications in various domains such as secure access, law enforcement, and locating missing persons. The success of this technology is largely attributed to deep learning, which leverages large datasets and effective loss functions to achieve highly discriminative features. Despite its advancements, face recognition still faces challenges in areas such as explainability, demographic bias, privacy and robustness against aging, pose variations, illumination changes, occlusions, and expressions. Additionally, the emergence of privacy regulations has led to the discontinuation of several well-established datasets, raising legal, ethical, and privacy concerns. To address these issues, synthetic facial data generation has been proposed as a solution. This technique not only mitigates privacy concerns but also allows for comprehensive experimentation with facial attributes that cause bias, helps alleviate demographic bias, and provides complementary data to enhance models trained with real data. Competitions, such as the FRCSyn and SDFR, have been organized to explore the limitations and potential of face recognition technology trained with synthetic data. This paper compares the effectiveness of established synthetic face datasets with different generation techniques in face recognition tasks. We benchmark the accuracy of seven mainstream datasets, providing a vivid comparison of approaches that are not explicitly contrasted in the literature. Our experiments highlight the diverse techniques used to address the synthetic facial data generation problem and present a comprehensive benchmark of the area. The results demonstrate the effectiveness of various methods in generating synthetic facial data with realistic variations, evidencing the diverse techniques used to deal with the problem.

Referências

Y. Guo, L. Zhang, Y. Hu, X. He, and J. Gao, “Ms-celeb-1m: A dataset and benchmark for large-scale face recognition,” in Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part III 14. Springer, 2016, pp. 87–102.

P. Melzi, C. Rathgeb, R. Tolosana, R. Vera-Rodriguez, A. Morales, D. Lawatsch, F. Domin, and M. Schaubert, “Synthetic data for the mitigation of demographic biases in face recognition,” in 2023 IEEE International Joint Conference on Biometrics (IJCB). IEEE, 2023, pp. 1–9.

G. Balakrishnan, Y. Xiong, W. Xia, and P. Perona, “Towards causal benchmarking of biasin face analysis algorithms,” Deep Learning-Based Face Analytics, pp. 327–359, 2021.

P. Melzi, R. Tolosana, R. Vera-Rodriguez, M. Kim, C. Rathgeb, X. Liu, I. DeAndres-Tame, A. Morales, J. Fierrez, J. Ortega-Garcia et al., “Frcsyn-ongoing: Benchmarking and comprehensive evaluation of real and synthetic data to improve face recognition systems,” Information Fusion, vol. 107, p. 102322, 2024.

I. DeAndres-Tame, R. Tolosana, P. Melzi, R. Vera-Rodriguez, M. Kim, C. Rathgeb, X. Liu, A. Morales, J. Fierrez, J. Ortega-Garcia et al., “Frcsyn challenge at cvpr 2024: Face recognition challenge in the era of synthetic data,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 3173–3183.

H. O. Shahreza, C. Ecabert, A. George, A. Unnervik, S. Marcel, N. Di Domenico, G. Borghi, D. Maltoni, F. Boutros, J. Vogel et al., “Sdfr: Synthetic data for face recognition competition,” arXiv preprint arXiv:2404.04580, 2024.

I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio, “Generative adversarial nets,” Advances in neural information processing systems, vol. 27, 2014.

J. Ho, A. Jain, and P. Abbeel, “Denoising diffusion probabilistic models,” Advances in neural information processing systems, vol. 33, pp. 6840–6851, 2020.

H. Qiu, B. Yu, D. Gong, Z. Li, W. Liu, and D. Tao, “Synface: Face recognition with synthetic data,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021, pp. 10 880–10 890.

J. N. Kolf, T. Rieber, J. Elliesen, F. Boutros, A. Kuijper, and N. Damer, “Identity-driven three-player generative adversarial network for synthetic-based face recognition,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 806–816.

F. Boutros, J. H. Grebe, A. Kuijper, and N. Damer, “Idiff-face: Synthetic-based face recognition through fizzy identity-conditioned diffusion model,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2023, pp. 19 650–19 661.

F. Boutros, M. Huber, P. Siebke, T. Rieber, and N. Damer, “Sface: Privacy-friendly and accurate face recognition using synthetic data,” in 2022 IEEE International Joint Conference on Biometrics (IJCB). IEEE, 2022, pp. 1–11.

M. Kim, F. Liu, A. Jain, and X. Liu, “Dcface: Synthetic face generation with dual condition diffusion model,” in Proceedings of the ieee/cvf conference on computer vision and pattern recognition, 2023, pp. 12 715–12 725.

P. Melzi, C. Rathgeb, R. Tolosana, R. Vera-Rodriguez, D. Lawatsch, F. Domin, and M. Schaubert, “Gandiffface: Controllable generation of synthetic datasets for face recognition with realistic variations,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2023, pp. 3086–3095.

G. Bae, M. de La Gorce, T. Baltrušaitis, C. Hewitt, D. Chen, J. Valentin, R. Cipolla, and J. Shen, “Digiface-1m: 1 million digital face images for face recognition,” in Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, 2023, pp. 3526–3535.

S. Moschoglou, A. Papaioannou, C. Sagonas, J. Deng, I. Kotsia, and S. Zafeiriou, “Agedb: the first manually collected, in-the-wild age database,” in proceedings of the IEEE conference on computer vision and pattern recognition workshops, 2017, pp. 51–59.

M. Wang and W. Deng, “Mitigating bias in face recognition using skewness-aware reinforcement learning,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2020, pp. 9322–9331.

S. Sengupta, J.-C. Chen, C. Castillo, V. M. Patel, R. Chellappa, and D. W. Jacobs, “Frontal to profile face verification in the wild,” in 2016 IEEE winter conference on applications of computer vision (WACV). IEEE, 2016, pp. 1–9.

M. E. Erakιn, U. Demir, and H. K. Ekenel, “On recognizing occluded faces in the wild,” in 2021 International Conference of the Biometrics Special Interest Group (BIOSIG). IEEE, 2021, pp. 1–5.

J. Deng, J. Guo, E. Ververas, I. Kotsia, and S. Zafeiriou, “Retinaface: Single-shot multi-level face localisation in the wild,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2020, pp. 5203–5212.

J. Deng, J. Guo, N. Xue, and S. Zafeiriou, “Arcface: Additive angular margin loss for deep face recognition,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2019, pp. 4690–4699.

Y. Deng, J. Yang, D. C. F. Wen, and X. Tong, “Disentangled and controllable face image generation via 3d imitative-contrastive learning,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020.