How Good Is ChatGPT For Detecting Hate Speech In Portuguese? - Oliveira, Amanda S.; Cecote, Thiago C.; Silva, Pedro H. L.; Gertrudes, Jadson C.; Freitas, Vander L. S.; Luz, Eduardo J. S. (Anais do Simpósio Brasileiro de Tecnologia da Informação e da Linguagem Humana (STIL); 2023: Anais do XIV Simpósio Brasileiro de Tecnologia da Informação e da Linguagem HumanaDO - 10.5753/stil.2023.233943, 2023)
This study evaluates OpenAI’s ChatGPT, a large language model, for its efficacy in detecting hate speech in Portuguese tweets, comparing it with purpose-trained models. Despite incurring considerable computational costs, ChatGPT as a zero-shot classifier demonstrated commendable performance, even superior to or on par with state-of-the-art methods, with an F1-score of 73.0% on the ToLD-BR. In a cross-dataset evaluation on the HLPHSP dataset, it secured a superior F1-score of 73%. The choice of prompt significantly impacts the outcome, with a wider scope prompt balancing precision and recall metrics. ChatGPT, due to its interpretability and resilience against data distribution shifts, could be a preferred choice for tasks prioritizing these factors.