Machine Learning Approaches for Sign Recognition in Brazilian Sign Language (Libras)

• Kauã de Melo Alves IFS
• Felipe Jovino dos Santos IFS
• Stephanie Kamarry Alves de Sousa IFS

DOI: https://doi.org/10.5753/erbase.2025.13807

Abstract

The digital inclusion of deaf individuals still faces major challenges due to the lack of accessible technologies for automatic translation of Brazilian Sign Language (Libras). This paper investigates the use of machine learning to recognize static signs in Libras, addressing challenges such as regional variations, lighting conditions, and hand positioning. RNN, Random Forest, and XGBoost models were evaluated using landmarks extracted with MediaPipe from a dataset of 25,000 images. XGBoost achieved the best performance in both accuracy and F1-score. This study contributes by creating a public benchmark for static Libras signs and proposes future work involving dynamic signs and advanced deep learning techniques, aiming to enhance communication accessibility for the deaf community.

References

ABDULHUSSEINA, A. A.; RAHEEM, F. A. Hand Gesture Recognition of Static Letters American Sign Language (ASL) Using Deep Learning. Engineering and Technology Journal, v. 38, n. Part A, No. 06, p. 926-937, 2020. DOI: 10.30684/etj.v38i6A.533. Acesso em: 19 out. 2024.

AL-QURISHI, M.; KHALID, T.; SOUISSI, R. Deep learning for sign language recognition: Current techniques, benchmarks, and open issues. IEEE Access, v. 9, p. 126917-126951, 2021.

ALMEIDA-SILVA, A.; DA CRUZ, R. T. A estrutura do sintagma nominal (SN) em LIBRAS. In: MEDEIROS, A. C. M.; OLIVEIRA Jr, M. (Org.). 30 ANOS DO PROGRAMA DE ESTUDOS LINGUÍSTICOS (PRELIN - PPGLL/UFAL) – Volume II – Estudos em teoria gerativa. Campinas, SP: Pontes Editores, 2022. p. 73-94.

ANDRADE, S. P.; LATINI, L. M. D. Inclusão digital: muito além do mero acesso às tecnologias de informação e comunicação. Revista Jurídica Profissional, v. 1, n. 1, 2022. Disponível em: [link]. Acesso em: 19 out. 2024.

CARVALHO, G.; BRANDÃO, A.; FERREIRA, F. HandArch: A deep learning architecture for LIBRAS hand configuration recognition. In: Anais do XVII Workshop de Visão Computacional. Porto Alegre, RS: SBC, 2021. p. 19-24. Disponível em: [link]. Acesso em: 21 out. 2024.

LOBO-NETO, V. C.; PEDRINI, H. LSWH100: A handshape dataset for Brazilian sign language (Libras) using SignWriting. Data in Brief, v. 56, p. 110780, 2024. DOI: 10.1016/j.dib.2024.110780. Acesso em: 22 out. 2024.

SCHÖNSTRÖM, K. Sign languages and second language acquisition research: An introduction. Journal of the European Second Language Association, v. 5, n. 1, 2021.

SILVA, J. L. S.; VIEIRA, G. S.; FONSECA, A. U.; SOARES, F. Reconhecimento e Tradução de Sinais de Libras para Língua Portuguesa Escrita usando Redes Neurais Profundas. SBA - Sociedade Brasileira de Automática, CBA2022, v. 3, n. 1, 2022. DOI: 10.20906/CBA2022/3720. Acesso em: 22 out. 2024.