Leveraging Optimal Methods for Diverse Skin Types Classification in Images for Reduced Bias

Luiz Augusto Vieira Manoel; Moacir Antonelli Ponti

doi:10.22456/2175-2745.143537

Authors

Luiz Augusto Vieira Manoel Universidade de São Paulo (USP)
Moacir Antonelli Ponti Universidade de São Paulo (USP)

DOI:

https://doi.org/10.22456/2175-2745.143537

Keywords:

Skin Tone Classification, Image Processing, Representativeness

Abstract

This paper addresses the bias in automatic skin tone classification, highlighting how technologies can mirror societal issues such as inadequate representation of diverse skin tones. The objective is to evaluate methods that ensure balanced performance across skin types despite a scarcity of annotated databases with ethnic-racial details. Using the Fitzpatrick Skin Type classification system, we assess various skin color labeling algorithms, selecting optimal approaches for each type based on f-score. Results show that a single method biases results towards certain skin tones, whereas combining methods enhances accuracy across diverse types. We leverage the LFW and Fitzpatrick17k databases and apply image processing techniques like gamma transformation, CLAHE, histogram equalization, and non-linear order statistics filters. By tailoring processes to specific skin tone ranges, our auto-labeling approach better mirrors manual labeling, aiming for more equitable technology.

Downloads

Download data is not yet available.

References

MOROZOV, E.; MARCONDES, C. Big Tech: a ascensão dos dados e a morte da política. [S.l.]: UBU, 2018. ISBN 9788571260122.

MEHRABI, N. et al. A survey on bias and fairness in machine learning. arXiv preprint arXiv:1908.09635, 2019.

SILVA, J. M. C. da; REZENDE, P. M. B.; PONTI, M. A. Detecting and mitigating issues in image-based COVID-19 diagnosis. In: PMLR. ICML Workshop on Healthcare AI and COVID-19. Baltimore, USA, 2022. p. 127–135.

MELLO, R. F.; PONTI, M. A. Machine Learning: A Practical Approach on the Statistical Learning Theory. Cham: Springer, 2018.

MERLER, M. et al. Diversity in faces. arXiv preprint arXiv:1901.10436, 2019.

FITZPATRICK, T. B. Soleil et peau. J Med Esthet, v. 2, p. 33–34, 1975.

CHARDON, A.; CRETOIS, I.; HOURSEAU, C. Skin colour typology and suntanning pathways. International journal of cosmetic science, Wiley Online Library, v. 13, n. 4, p. 191–208, 1991.

PONTI, M.; NAZARÉ, T. S.; THUMÉ, G. S. Image quantization as a dimensionality reduction procedure in color and texture feature extraction. Neurocomputing, Elsevier, v. 173, p. 385–396, 2016.

PREMA, C.; MANIMEGALAI, D. Survey on skin tone detection using color spaces. International Journal of Applied Information Systems, Citeseer, v. 2, n. 2, p. 18–26, 2012.

HARVILLE, M. et al. Consistent image-based measurement and classification of skin color. In: IEEE Int. Conf. on Image Processing. Genoa, Italy: [s.n.], 2005. p. II–374.

PIZER, S. M. et al. Adaptive histogram equalization and its variations. Computer vision, graphics, and image processing, Elsevier, v. 39, n. 3, p. 355–368, 1987.

KINYANJUI, N. M. et al. Estimating skin tone and effects on classification performance in dermatology datasets. arXiv preprint arXiv:1910.13268, 2019.

D’ORAZIO, J. et al. UV radiation and the skin. International journal of molecular sciences, Multidisciplinary Digital Publishing Institute, v. 14, n. 6, p. 12222–12248, 2013.

HUANG, G. B.; LEARNED-MILLER, E. Labeled Faces in the Wild: Updates and New Reporting Procedures. Amherst, USA, 2014.

GROH, M. et al. Evaluating deep neural networks trained on clinical images in dermatology with the Fitzpatrick 17k dataset. arXiv preprint arXiv:2104.09957, 2021.

VIOLA, P.; JONES, M. Rapid object detection using a boosted cascade of simple features. In: Computer Vision and Pattern Recognition. Kauai, HI, USA: [s.n.], 2001. p. 1063–6919.