A Model for Skin Tone Detection and Recognition Based on Artificial Intelligence and Image Processing

• Janaina Ferreira Santos UFS
• Gabriel dos Santos Mendonça UFS
• Alcides X. Benicasa UFS

DOI: https://doi.org/10.5753/erbase.2025.13032

Abstract

Computer Vision and Machine Learning have enabled the development of more accurate systems for detecting patterns in images, such as skin tone classification — a task relevant to product personalization and the analysis of racial diversity. However, low accuracy for darker skin tones and lack of diversity in datasets contribute to racial bias. This work proposes a model based on the Monk scale, combining facial detection, rule-based skin segmentation, and color extraction using K-means, comparing the results to a reference palette. Using an experimental and quantitative approach, the model achieved 99% accuracy in controlled environments and 80% in low-light conditions.

References

Ali, H., Alnafaakh, H., Ghazali, R., El abbadi, N., and El Abbadi, N. (2021). A review of human skin detection applications based on image processing. Bulletin of Electrical Engineering and Informatics, 10:129–137.

Borza, D., Borza, D., Nistor, S. C., Nistor, S. C., Dărăbant, A. S., and Darabant, A. S. (2017). Towards automatic skin tone classification in facial images. International Conference on Image Analysis and Processing.

Buolamwini, J. and Gebru, T. (2018). Gender shades: Intersectional accuracy disparities in commercial gender classification. In Conference on fairness, accountability and transparency, pages 77–91. PMLR.

Fitzpatrick, T. B. (1988). The Validity and Practicality of Sun-Reactive Skin Types I Through VI. Archives of Dermatology, 124(6):869–871.

Gonzalez, R. C. and Woods, R. E. (2000). Processamento de imagens digitais. Editora Blucher.

Howard, J. J., Sirotin, Y. B., Tipton, J. L., and Vemury, A. R. (2021). Reliability and validity of image-based and self-reported skin phenotype metrics. IEEE Transactions on Biometrics, Behavior, and Identity Science, 3(4):550–560.

Kolkur, S., Kalbande, D., Shimpi, P., Bapat, C., and Jatakia, J. (2017). Human skin detection using rgb, hsv and ycbcr color models. arXiv preprint arXiv:1708.02694.

Kumar, A., Kaur, A., and Kumar, M. (2019). Face detection techniques: a review. Artificial Intelligence Review, 52:927–948.

Leão, A. C., Araújo, A., and Souza, L. A. C. (2005). Implementação de sistema de gerenciamento de cores para imagens digitais. Teixeira A, Barrére E, Abrão IC. Web e multimídia: desafios e soluções. Poços de Caldas: PUC-Minas.

Manoel, L. A. V. (2022). Reduzindo viés em classificação de tons de pele em bases de dados de imagens. PhD thesis, Universidade de São Paulo.

Marengoni, M. and Stringhini, S. (2010). Tutorial: Introdução à visão computacional usando opencv. Revista de Informática Teórica e Aplicada, 16(1):125–160.

Minaee, S., Luo, P., Lin, Z., and Bowyer, K. W. (2021). Going deeper into face detection: A survey. CoRR, abs/2103.14983.

Monk, E. (2022). Monk skin tone scale. [link].

Ridolf, L. F. G. G. M. (2012). Construo de Espaos de Cor Euclidianos e Perceptualmente Uniformes com base na fórmula CIEDE2000. PhD thesis, Pontifícia Universidade Católica do Rio de Janeiro.

Schumann, C., Olanubi, G. O., Wright, A., Monk Jr, E., Heldreth, C., and Ricco, S. (2023). Consensus and subjectivity of skin tone annotation for ml fairness. arXiv preprint arXiv:2305.09073.

Scuri, A. E. (1999). Fundamentos da imagem digital. Pontifícia Universidade Católica do Rio de Janeiro, page 13.

Zhang, K., Zhang, Z., Li, Z., and Qiao, Y. (2016). Joint face detection and alignment using multitask cascaded convolutional networks. IEEE Signal Processing Letters, 23(10):1499–1503.