Contextual BERT Model for Toxicity Detection in Messaging Platforms - Buzelin, Arthur; Aquino, Yan; Bento, Pedro; Dayrell, Lucas; Estanislau, Victoria; Malaquias, Samira; Dutenhefner, Pedro; Porfírio, Luisa G.; Rigueira, Pedro B.; Grossi, Caio Souza; Evangelista, Guilherme H. G.; Pappa, Gisele L.; et al. (Anais do Encontro Nacional de Inteligência Artificial e Computacional (ENIAC); 2024: Anais do XXI Encontro Nacional de Inteligência Artificial e ComputacionalDO - 10.5753/eniac.2024.245275, 2024)
The increasing prevalence of messaging platforms has created new challenges in hate speech detection. Traditional classification models designed for social media posts often fall short in these environments due to the lack of contextual information. This paper presents a novel approach to message classification by integrating contextual data from preceding messages, utilizing a fine-tuned BERT model based on PySentimiento. Our results demonstrate that incorporating preceding messages substantially improves the classification task. The average AUC-ROC increased from 0.691 with the PySentimiento base model to 0.784 with standard fine-tuning, and further to an impressive 0.926 with our context-based model.