Leveraging Task-Adaptive Pretraining for Misogynistic Hate Speech Classification in Brazilian Portuguese
Resumo
Este trabalho analisa o impacto do pré-treinamento adaptado à tarefa (TAPT) na identificação de discurso de ódio misógino em português do Brasil. Para isso, foi utilizado o conjunto de dados MINA-BR, cuja porção não rotulada é empregada no TAPT, enquanto a anotada é utilizada para ajuste fino e avaliação. O trabalho compara o desempenho das arquiteturas RoBERTa, ALBERT, BERTimbau e BERTabaporu via validação cruzada. Os resultados indicam que o TAPT melhora o desempenho em todas as arquiteturas, com destaque para o modelo fMINABr-BERTabaporu. Esses achados sugerem que o TAPT contribui para o aprimoramento de modelos de linguagem em cenários com escassez de dados anotados e alta diversidade linguística.Referências
Assis, G., Amorim, A., Carvalho, J., de Oliveira, D., Vianna, D., and Paes, A. (2024). Exploring Portuguese hate speech detection in low-resource settings: Lightly tuning encoder models or in-context learning of large models? In Gamallo, P., Claro, D., Teixeira, A., Real, L., Garcia, M., Oliveira, H. G., and Amaro, R., editors, Proceedings of the 16th International Conference on Computational Processing of Portuguese - Vol. 1, pages 301–311, Santiago de Compostela, Galicia/Spain. Association for Computational Lingustics.
Belay, T. D., Azime, I. A., Ahmad, I. S., Adelani, D. I., Abdulmumin, I., Ayele, A. A., Muhammad, S. H., and Yimam, S. M. (2025). AfroXLMR-social: Adapting pre-trained language models for African languages social media text. In Christodoulopoulos, C., Chakraborty, T., Rose, C., and Peng, V., editors, Findings of the Association for Computational Linguistics: EMNLP 2025, pages 15570–15587, Suzhou, China. Association for Computational Linguistics.
Costa, P. B., Pavan, M. C., Santos, W. R., Silva, S. C., and Paraboni, I. (2023). BERTabaporu: Assessing a genre-specific language model for Portuguese NLP. In Mitkov, R. and Angelova, G., editors, Proceedings of the 14th International Conference on Recent Advances in Natural Language Processing, pages 217–223, Varna, Bulgaria. INCOMA Ltd., Shoumen, Bulgaria.
De Gibert, O., Perez, N., García-Pablos, A., and Cuadros, M. (2018). Hate speech dataset from a white supremacy forum. arXiv preprint arXiv:1809.04444.
de Pelle, R. and Moreira, V. (2017). Offensive comments in the Brazilian web: a dataset and baseline results. In Anais do VI Brazilian Workshop on Social Network Analysis and Mining, pages 510–519, Porto Alegre, RS, Brasil. SBC.
Devlin, J., Chang, M.-W., Lee, K., and Toutanova, K. (2019). Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies, volume 1 (long and short papers), pages 4171–4186.
Grotti, L. and Quick, P. (2024). Berticelli at haspeede 3: Fine-tuning and cross-validating large language models for hate speech detection. In EVALITA Proceedings of the Eighth Evaluation Campaign of Natural Language Processing and Speech Tools for Italian Final Workshop: Parma, Italy, September 7-8th, 2023, volume 2(3), page 188. Accademia University Press.
Gururangan, S., Marasović, A., Swayamdipta, S., Lo, K., Beltagy, I., Downey, D., and Smith, N. A. (2020). Don’t stop pretraining: Adapt language models to domains and tasks. In Jurafsky, D., Chai, J., Schluter, N., and Tetreault, J., editors, Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 8342–8360, Online. Association for Computational Linguistics.
Lan, Z., Chen, M., Goodman, S., Gimpel, K., Sharma, P., and Soricut, R. (2019). Albert: A lite bert for self-supervised learning of language representations. arXiv preprint arXiv:1909.11942.
Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., and Stoyanov, V. (2019). Roberta: A robustly optimized bert pretraining approach. arXiv:cs.CL, page 1907.11692.
Plath, H. O., Paiva, M. E. O., Pinto, D. L., and Costa, P. D. P. (2022). Detecção de discurso de Ódio contra mulheres em textos em português brasileiro: Construção da base mina-br e modelo de classificação. Revista Eletrônica de Iniciação Científica em Computação, 20(3).
Plath, H. O., Paiva, M. E. O., Pinto, D. L., and Costa, P. D. P. (2025). MINA-BR: base de dados para identificação de discurso de ódio contra a mulher na internet. DOI: 10.25824/redu/MQD68Z.
Ramos, S. B. and da Silva Figueiredo, K. (2018). Uma taxonomia dos tipos de preconceito enfrentados por mulheres na Área de tecnologia. In Anais do XII Women in Information Technology, pages 91–94, Porto Alegre, RS, Brasil. SBC.
Ramos de Oliveira, F., Reis, V., and Ebecken, N. (2023). Tupy-e: detecting hate speech in brazilian portuguese social media with a novel dataset and comprehensive analysis of models. arXiv preprint arXiv:2312.17704.
Rawat, A., Kumar, S., and Samant, S. S. (2024). Hate speech detection in social media: Techniques, recent trends, and future challenges. Wiley Interdisciplinary Reviews: Computational Statistics, 16(2):e1648.
Saha, P., Mathew, B., Goyal, P., and Mukherjee, A. (2018). Hateminers: Detecting hate speech against women. arXiv preprint arXiv:1812.06700.
Salles, I., Vargas, F., and Benevenuto, F. (2025). Hatebrxplain: A benchmark dataset with human-annotated rationales for explainable hate speech detection in brazilian portuguese. In Proceedings of the 31st international conference on computational linguistics, pages 6659–6669.
Silva, F. and Freitas, L. (2022). Brazilian portuguese hate speech classification using bertimbau. The International FLAIRS Conference Proceedings, 35.
Souza, F., Nogueira, R., and Lotufo, R. (2020). Bertimbau: pretrained bert models for brazilian portuguese. In Brazilian conference on intelligent systems, pages 403–417, Cham. Springer International Publishing.
Vicente, A. R. R., Andrade, J. O., de Souza Gazolli, K. A., and Komati, K. S. (2025). Comparative analysis of machine learning models for misogynistic hate speech detection in brazilian portuguese: A study using the mina-br dataset. In Proceedings of the IEEE Chilecon 2025 Conference, 2025, Valparaíso, Chile. 2025.
Belay, T. D., Azime, I. A., Ahmad, I. S., Adelani, D. I., Abdulmumin, I., Ayele, A. A., Muhammad, S. H., and Yimam, S. M. (2025). AfroXLMR-social: Adapting pre-trained language models for African languages social media text. In Christodoulopoulos, C., Chakraborty, T., Rose, C., and Peng, V., editors, Findings of the Association for Computational Linguistics: EMNLP 2025, pages 15570–15587, Suzhou, China. Association for Computational Linguistics.
Costa, P. B., Pavan, M. C., Santos, W. R., Silva, S. C., and Paraboni, I. (2023). BERTabaporu: Assessing a genre-specific language model for Portuguese NLP. In Mitkov, R. and Angelova, G., editors, Proceedings of the 14th International Conference on Recent Advances in Natural Language Processing, pages 217–223, Varna, Bulgaria. INCOMA Ltd., Shoumen, Bulgaria.
De Gibert, O., Perez, N., García-Pablos, A., and Cuadros, M. (2018). Hate speech dataset from a white supremacy forum. arXiv preprint arXiv:1809.04444.
de Pelle, R. and Moreira, V. (2017). Offensive comments in the Brazilian web: a dataset and baseline results. In Anais do VI Brazilian Workshop on Social Network Analysis and Mining, pages 510–519, Porto Alegre, RS, Brasil. SBC.
Devlin, J., Chang, M.-W., Lee, K., and Toutanova, K. (2019). Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies, volume 1 (long and short papers), pages 4171–4186.
Grotti, L. and Quick, P. (2024). Berticelli at haspeede 3: Fine-tuning and cross-validating large language models for hate speech detection. In EVALITA Proceedings of the Eighth Evaluation Campaign of Natural Language Processing and Speech Tools for Italian Final Workshop: Parma, Italy, September 7-8th, 2023, volume 2(3), page 188. Accademia University Press.
Gururangan, S., Marasović, A., Swayamdipta, S., Lo, K., Beltagy, I., Downey, D., and Smith, N. A. (2020). Don’t stop pretraining: Adapt language models to domains and tasks. In Jurafsky, D., Chai, J., Schluter, N., and Tetreault, J., editors, Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 8342–8360, Online. Association for Computational Linguistics.
Lan, Z., Chen, M., Goodman, S., Gimpel, K., Sharma, P., and Soricut, R. (2019). Albert: A lite bert for self-supervised learning of language representations. arXiv preprint arXiv:1909.11942.
Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., and Stoyanov, V. (2019). Roberta: A robustly optimized bert pretraining approach. arXiv:cs.CL, page 1907.11692.
Plath, H. O., Paiva, M. E. O., Pinto, D. L., and Costa, P. D. P. (2022). Detecção de discurso de Ódio contra mulheres em textos em português brasileiro: Construção da base mina-br e modelo de classificação. Revista Eletrônica de Iniciação Científica em Computação, 20(3).
Plath, H. O., Paiva, M. E. O., Pinto, D. L., and Costa, P. D. P. (2025). MINA-BR: base de dados para identificação de discurso de ódio contra a mulher na internet. DOI: 10.25824/redu/MQD68Z.
Ramos, S. B. and da Silva Figueiredo, K. (2018). Uma taxonomia dos tipos de preconceito enfrentados por mulheres na Área de tecnologia. In Anais do XII Women in Information Technology, pages 91–94, Porto Alegre, RS, Brasil. SBC.
Ramos de Oliveira, F., Reis, V., and Ebecken, N. (2023). Tupy-e: detecting hate speech in brazilian portuguese social media with a novel dataset and comprehensive analysis of models. arXiv preprint arXiv:2312.17704.
Rawat, A., Kumar, S., and Samant, S. S. (2024). Hate speech detection in social media: Techniques, recent trends, and future challenges. Wiley Interdisciplinary Reviews: Computational Statistics, 16(2):e1648.
Saha, P., Mathew, B., Goyal, P., and Mukherjee, A. (2018). Hateminers: Detecting hate speech against women. arXiv preprint arXiv:1812.06700.
Salles, I., Vargas, F., and Benevenuto, F. (2025). Hatebrxplain: A benchmark dataset with human-annotated rationales for explainable hate speech detection in brazilian portuguese. In Proceedings of the 31st international conference on computational linguistics, pages 6659–6669.
Silva, F. and Freitas, L. (2022). Brazilian portuguese hate speech classification using bertimbau. The International FLAIRS Conference Proceedings, 35.
Souza, F., Nogueira, R., and Lotufo, R. (2020). Bertimbau: pretrained bert models for brazilian portuguese. In Brazilian conference on intelligent systems, pages 403–417, Cham. Springer International Publishing.
Vicente, A. R. R., Andrade, J. O., de Souza Gazolli, K. A., and Komati, K. S. (2025). Comparative analysis of machine learning models for misogynistic hate speech detection in brazilian portuguese: A study using the mina-br dataset. In Proceedings of the IEEE Chilecon 2025 Conference, 2025, Valparaíso, Chile. 2025.
Publicado
19/10/2026
Como Citar
VICENTE, Ana Rubia R.; OLIVEIRA, Hilário T. A. de; GAZOLLI, Kelly A. de S.; KOMATI, Karin S..
Leveraging Task-Adaptive Pretraining for Misogynistic Hate Speech Classification in Brazilian Portuguese. In: ENCONTRO NACIONAL DE INTELIGÊNCIA ARTIFICIAL E COMPUTACIONAL (ENIAC), 23. , 2026, Cuiabá/MT.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2026
.
p. 1340-1351.
ISSN 2763-9061.
DOI: https://doi.org/10.5753/eniac.2026.31079.
