Performance Evaluation of Computational Approaches for Automatic Skin Tone Inference

• Igor Joaquim da S. Costa UFMG
• Thais F. Silva UFMG
• Marisa Vasconcelos UFMG
• Julio C. S. Reis UFV
• Jussara M. Almeida UFMG
• Virgílio Almeida UFMG

DOI: https://doi.org/10.5753/laai-ethics.2024.32463

Abstract

Automatic estimates of skin tone are challenging due to racial and gender biases in machine learning approaches. In this work, we explore a labeled dataset of images to evaluate two computational approaches widely explored in the literature, ITA and CASCo, in order to investigate their robustness in performing this task. The results reveal that these approaches still have weaknesses that should be considered before their application in sensitive contexts.

Keywords: Automatic Skin Tone Inference, Machine Learning Biases, Individual Topology Angle (ITA), Classification Algorithm for Skin Color (CASCo)

References

Buolamwini, J. and Gebru, T. (2018). Gender shades: Intersectional accuracy disparities in commercial gender classification. In Proc. of FAT.

Chardon, A., Cretois, I., and Hourseau, C. (1991). Skin colour typology and suntanning pathways. Int. J. of Cosmet. Sci., 13(4):191–208.

Fitzpatrick, T. B. (1988). The validity and practicality of sun-reactive skin types I through VI. Archives of Dermatology, 124(6):869–871.

Khosla, N., Grullon, K., and Rosenblatt, A. (2021). Prevention of racialized medicine in pediatric dermatology: A call to re-examine skin tone typing. Pediatric Dermatology, 38:167–169.

Krishnapriya, K., Pangelinan, G., King, M. C., and Bowyer, K. W. (2022). Analysis of manual and automated skin tone assignments. In Proc. of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 429–438.

Leeb, G., Auchus, I., Law, T., Bickler, P. E., Feiner, J., Hashi, S., Monk, E. P., Igaga, E., Bernstein, M., Chou, Y., Hughes, C., Schornack, D., Lester, J., Moore, K., Okunlola, O., Fernandez, J., Shmuylovich, L., and Lipnick, M. (2024). The performance of 11 fingertip pulse oximeters during hypoxemia in healthy human participants with varied, quantified skin pigment. EBioMedicine.

René Alejandro Rejón Piña, C. M. (2023). Classification algorithm for skin color (CASCo): A new tool to measure skin color in social science research. Social Science Quarterly.

Schumann, C., Olanubi, G., Wright, A., Monk, J., Heldreth, C., and Ricco, S. (2023). Consensus and subjectivity of skin tone annotation for ML fairness. In NeurIPS.