Semantic Vector Space Mapping between Brazilian Portuguese and Libras Glosses Using Siamese Models
Resumo
This work presents a study towards constructing a shared semantic vector space between Brazilian Portuguese sentences and sequences of glosses representing utterances in Libras (Brazilian Sign Language). We propose a siamese model based on BERTimbau, trained via contrastive learning, employing mean pooling and controlled generation of negative pairs. We evaluate the model on a retrieval task between Portuguese and glosses. Preliminary results indicate that semantic alignment between textual modalities is feasible even with limited corpus size, constituting a first step towards automatic translation systems and semantic indexing for Libras.Referências
Baltrušaitis, T., Ahuja, C., and Morency, L.-P. (2019). Multimodal machine learning: A survey and taxonomy. IEEE Transactions on Pattern Analysis and Machine Intelligence, 41(2):423–443.
Bezerra, E. T., Oliveira, I. d. S., Fonseca, J. R. M. d., Celestino, E. M., Caitano, T. F., Vieira, A. J. F., Nascimento, R. d. S., Chaves, J. P. d. A., Monteiro, A. L., Pinheiro, J. C. M., et al. (2024). Tecnologias assistivas para o ensino de libras: soluções inovadoras para a educação inclusiva. Revista Foco, 17(11):e6576.
Camgoz, N., Hadfield, S., Koller, O., and Bowden, R. (2020). Sign language transformers: Joint end-to-end sign language recognition and translation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10023–10033.
Costa, V., Tavares, L., Conceição, R., Agostini, L., Santana, B., Lebedeff, T., and Corrêa, G. (2025a). A lightweight i3d-based approach for real-time brazilian sign language recognition. In Proceedings of the 31st Brazilian Symposium on Multimedia and the Web, pages 501–505, Porto Alegre, RS, Brasil. SBC.
Costa, V., Tomaszewski, J. P., Pereira, L., Santana, B. S., and Corrêa, G. (2025b). Ai-based approaches for brazilian sign language recognition: A systematic literature review: Insights on methods, metrics, and resources for libras recognition. In Proceedings of the 31st Brazilian Symposium on Multimedia and the Web (WebMedia 2025), WebMedia 2025, page 585–597. Sociedade Brasileira de Computação - SBC.
De Martino, J. M. et al. (2023). Neural machine translation from text to sign language. Universal Access in the Information Society, pages 1–12.
Grossi, V. S. and Ferreira Filho, B. S. (2024). Aplicação de técnicas de reconhecimento de imagens na classificação de sinais em libras (linguagem brasileira de sinais) para tradução em texto.
Hadsell, R., Chopra, S., and LeCun, Y. (2006). Dimensionality reduction by learning an invariant mapping. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 1735–1742.
Junior, A. K. (2024). K-LIBRAS: SISTEMA DE CONHECIMENTO PARA A TRADUÇÃO DA LÍNGUA BRASILEIRA DE SINAIS (LIBRAS/GLOSA). PhD thesis, UNIVERSIDADE FEDERAL DE SANTA CATARINA.
Khosla, P., Teterwak, P., Wang, C., Sarna, A., Tian, Y., Isola, P., Maschinot, A., Liu, C., and Krishnan, D. (2020). Supervised contrastive learning. In Advances in Neural Information Processing Systems, volume 33, pages 18661–18673.
Koller, O., Ney, H., and Bowden, R. (2019). Weakly supervised learning with multi-stream cnn-lstm-hmms to discover sequential parallelism in sign language videos. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5969–5978.
Krishna, S., P, V. V., and J, D. B. (2021). Signpose: Sign language animation through 3d pose lifting. In 2021 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW), pages 2640–2649.
Maia, W. F., Lopes, A. M., and David, S. A. (2025). Automatic sign language to text translation using MediaPipe and transformer architectures. Neurocomputing, 642:130421.
Martins, A. C. et al. (2026). Corpus libras-ufpel: A parallel dataset of brazilian sign language and portuguese for multimodal research and processing. In Proceedings of the International Conference on Computational Processing of Portuguese (PROPOR 2026), Salvador, BA, Brazil. To appear in ACL Anthology.
Paiva, R. and Costa, O. (2024). Avaliação do uso de modelos de aprendizagem profunda na tradução automática de línguas de sinais.
Rego, R. C., de Morais, L. M., and Almeida, W. M. (2025). Brazilian sign language recognition using deep learning based on fast fourier transform and kinematic features. IEEE Access, 13:202875–202892.
Reimers, N. and Gurevych, I. (2019). Sentence-bert: Sentence embeddings using siamese bert-networks. In Proceedings of EMNLP, pages 3982–3992.
Schroff, F., Kalenichenko, D., and Philbin, J. (2015). Facenet: A unified embedding for face recognition and clustering. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 815–823.
Silva, A., Araújo, S., Farias, M., Santos, I., Pessoa, A., and Junior, G. B. (2025). Reconhecimento inteligente de sinais em libras: Um modelo computacional assistivo para educação inclusiva em ambientes educacionais. RENOTE, 24(2):547–558.
Sincan, O. M., Low, J. H., Asasi, S., and Bowden, R. (2025). Gloss-free sign language translation: An unbiased evaluation of progress in the field. Computer Vision and Image Understanding, 261:104498.
Souza, F., Nogueira, R., and Lotufo, R. (2020). Bertimbau: Pretrained bert models for brazilian portuguese. In Proceedings of the 9th Brazilian Conference on Intelligent Systems (BRACIS), pages 403–417.
Xie, Y., Su, W., Zhong, C., Cai, C., and Yuan, Y. (2025). Gloss-free sign language translation based on fusion attention. Applied Soft Computing, 185:113848.
Bezerra, E. T., Oliveira, I. d. S., Fonseca, J. R. M. d., Celestino, E. M., Caitano, T. F., Vieira, A. J. F., Nascimento, R. d. S., Chaves, J. P. d. A., Monteiro, A. L., Pinheiro, J. C. M., et al. (2024). Tecnologias assistivas para o ensino de libras: soluções inovadoras para a educação inclusiva. Revista Foco, 17(11):e6576.
Camgoz, N., Hadfield, S., Koller, O., and Bowden, R. (2020). Sign language transformers: Joint end-to-end sign language recognition and translation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10023–10033.
Costa, V., Tavares, L., Conceição, R., Agostini, L., Santana, B., Lebedeff, T., and Corrêa, G. (2025a). A lightweight i3d-based approach for real-time brazilian sign language recognition. In Proceedings of the 31st Brazilian Symposium on Multimedia and the Web, pages 501–505, Porto Alegre, RS, Brasil. SBC.
Costa, V., Tomaszewski, J. P., Pereira, L., Santana, B. S., and Corrêa, G. (2025b). Ai-based approaches for brazilian sign language recognition: A systematic literature review: Insights on methods, metrics, and resources for libras recognition. In Proceedings of the 31st Brazilian Symposium on Multimedia and the Web (WebMedia 2025), WebMedia 2025, page 585–597. Sociedade Brasileira de Computação - SBC.
De Martino, J. M. et al. (2023). Neural machine translation from text to sign language. Universal Access in the Information Society, pages 1–12.
Grossi, V. S. and Ferreira Filho, B. S. (2024). Aplicação de técnicas de reconhecimento de imagens na classificação de sinais em libras (linguagem brasileira de sinais) para tradução em texto.
Hadsell, R., Chopra, S., and LeCun, Y. (2006). Dimensionality reduction by learning an invariant mapping. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 1735–1742.
Junior, A. K. (2024). K-LIBRAS: SISTEMA DE CONHECIMENTO PARA A TRADUÇÃO DA LÍNGUA BRASILEIRA DE SINAIS (LIBRAS/GLOSA). PhD thesis, UNIVERSIDADE FEDERAL DE SANTA CATARINA.
Khosla, P., Teterwak, P., Wang, C., Sarna, A., Tian, Y., Isola, P., Maschinot, A., Liu, C., and Krishnan, D. (2020). Supervised contrastive learning. In Advances in Neural Information Processing Systems, volume 33, pages 18661–18673.
Koller, O., Ney, H., and Bowden, R. (2019). Weakly supervised learning with multi-stream cnn-lstm-hmms to discover sequential parallelism in sign language videos. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5969–5978.
Krishna, S., P, V. V., and J, D. B. (2021). Signpose: Sign language animation through 3d pose lifting. In 2021 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW), pages 2640–2649.
Maia, W. F., Lopes, A. M., and David, S. A. (2025). Automatic sign language to text translation using MediaPipe and transformer architectures. Neurocomputing, 642:130421.
Martins, A. C. et al. (2026). Corpus libras-ufpel: A parallel dataset of brazilian sign language and portuguese for multimodal research and processing. In Proceedings of the International Conference on Computational Processing of Portuguese (PROPOR 2026), Salvador, BA, Brazil. To appear in ACL Anthology.
Paiva, R. and Costa, O. (2024). Avaliação do uso de modelos de aprendizagem profunda na tradução automática de línguas de sinais.
Rego, R. C., de Morais, L. M., and Almeida, W. M. (2025). Brazilian sign language recognition using deep learning based on fast fourier transform and kinematic features. IEEE Access, 13:202875–202892.
Reimers, N. and Gurevych, I. (2019). Sentence-bert: Sentence embeddings using siamese bert-networks. In Proceedings of EMNLP, pages 3982–3992.
Schroff, F., Kalenichenko, D., and Philbin, J. (2015). Facenet: A unified embedding for face recognition and clustering. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 815–823.
Silva, A., Araújo, S., Farias, M., Santos, I., Pessoa, A., and Junior, G. B. (2025). Reconhecimento inteligente de sinais em libras: Um modelo computacional assistivo para educação inclusiva em ambientes educacionais. RENOTE, 24(2):547–558.
Sincan, O. M., Low, J. H., Asasi, S., and Bowden, R. (2025). Gloss-free sign language translation: An unbiased evaluation of progress in the field. Computer Vision and Image Understanding, 261:104498.
Souza, F., Nogueira, R., and Lotufo, R. (2020). Bertimbau: Pretrained bert models for brazilian portuguese. In Proceedings of the 9th Brazilian Conference on Intelligent Systems (BRACIS), pages 403–417.
Xie, Y., Su, W., Zhong, C., Cai, C., and Yuan, Y. (2025). Gloss-free sign language translation based on fusion attention. Applied Soft Computing, 185:113848.
Publicado
19/07/2026
Como Citar
PEREIRA, Lucas dos Santos; SANTANA, Brenda S.; MARTINS, Antonielle; CORRÊA, Guilherme.
Semantic Vector Space Mapping between Brazilian Portuguese and Libras Glosses Using Siamese Models. In: WORKSHOP SOBRE AS IMPLICAÇÕES DA COMPUTAÇÃO NA SOCIEDADE (WICS), 7. , 2026, Gramado/RS.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2026
.
p. 303-316.
ISSN 2763-8707.
DOI: https://doi.org/10.5753/wics.2026.23749.
