Assessing Demographic Bias and Fairness in Facial Recognition Systems: A Framework

• Darian S. R. Rabanni ICTi
• Italo A. D. Oliveira ICTi
• Miguel D. S. Wanderley ICTi
• Cinthya O. Silva ICTi
• Renato A. Almeida ICTi
• João R. Alvim ICTi
• Katia M. Poloni ICTi
• Gustavo C. Bicalho ICTi
• Lucas F. A. O. Pellicer ICTi

Resumo

This paper presents a structured framework for assessing bias in facial recognition (FR) models, with an application to a case study. Bias in FR technologies towards sensitive demographic groups remains a significant challenge and a shared responsibility among researchers and companies in the design and development of fair systems. Despite increasing attention to fairness in FR, most existing studies address isolated stages of bias evaluation, with few offering an end-to-end well-defined solution. The proposed framework was developed in a structured and objective manner: 1) the selection of appropriate datasets; 2) the application of bias evaluation metrics; and 3) the techniques and the comparative analysis of performance across demographic groups. That framework aims not only to assist researchers in assessing bias in FR models but also to support the comparison and establishment of benchmarking tests for bias across different models. As a second contribution, we present a case study that applies the proposed framework to widely used benchmark datasets and relevant state-of-the-art pre-trained FR models, providing a comprehensive evaluation of performance disparities across different demographic groups.