A Change in Perspective: The Trade-Off Between Perspective API and Custom Models in Classifying Hate Speech in Portuguese - Buzelin, Arthur; Aquino, Yan; Bento, Pedro; Malaquias, Samira; Meira Jr, Wagner; Pappa, Gisele L. (Anais do Simpósio Brasileiro de Tecnologia da Informação e da Linguagem Humana (STIL); 2024: Anais do XV Simpósio Brasileiro de Tecnologia da Informação e da Linguagem Humana, 2024)
This paper examines the performance of the Perspective API, developed by Jigsaw, in detecting hate speech in Portuguese. Although the Perspective API supports multiple languages, its performance metrics are often aggregated, obscuring specific details. Our study reveals that the API’s AUC-ROC score for Portuguese is significantly lower than for English (0.744 vs. 0.942). To address this, we developed a BERT classifier model trained on a Portuguese Twitter hate speech dataset. Our model, with just 100 messages in it’s training set, outperformed the Perspective API. These findings highlight the need for more granular performance metrics and suggest that custom models may offer better solutions for specific languages.