Investigating Contextual Word Embeddings in Semi-Supervised Learning for Toxic Comment Detection - Ricarte Neto, Francisco Assis; Anchiêta, Rafael Torres; Moura, Raimundo Santos; Santos Neto, Pedro de A. dos; Santana, Andre Macedo (Proceedings of the Brazilian Symposium on Multimedia and the Web (WebMedia); 2025: Proceedings of the 31st Brazilian Symposium on Multimedia and the Web, 2025)
The proliferation of toxic messages on the Web has led many social networks to limit or shut down user comments, as these messages are harmful to people and keep them away from online platforms. Most approaches deal with this unacceptable language form, focusing only on identifying if a comment is toxic or not, using a supervised learning strategy, leaving aside the various types of harmful messages, such as LGBT+phobia, racism, xenophobia, and others. In this paper, we investigate three contextual word embedding models: BERTimbau, BERTweet.BR, and LLaMA 3.1 within a semi-supervised approach to distinguish whether a comment is toxic. Also, we explore this strategy to detect six types of toxicity: LGBT+phobia, insult, racism, obscenity, xenophobia, and misogyny. This task is defined as a multi-label classification problem, as a comment may contain several types of abusive language. We evaluated our approach using the ToLD-BR corpus and achieved competitive results for the binary toxicity classification task. In the context of multi-label toxicity detection, our best result outperformed approaches based on supervised learning, using significantly fewer labeled data, and emphasized their efficiency and practicality.