Comparative Analysis of LLMs for Detecting Racism, Sexism, and Homophobia on Social Media - Bou, Guilherme; Rocha, Adriano Mendonça (Anais do Simpósio Brasileiro de Sistemas de Informação (SBSI), 2026)
Research Context: Social media expansion has increased digital interactions, including hate speech such as racism, sexism, and homophobia. This challenges society and platforms to develop strategies for identifying and moderating harmful content; Scientific and/or Practical Problem: Despite AI advances, automatic detection faces limitations due to linguistic nuances, cultural context, and implementation costs. Scientifically, the challenge is evaluating model effectiveness; practically, it is developing economical, reliable large-scale solutions; Proposed Solution and/or Analysis: We conducted a comparative analysis of LLMs (GPT-3.5-Turbo, GPT-4.0, DeepSeek-V3 and Gemini-2.0-Flash) in detecting offensive social media comments. Tests on raw and preprocessed data using standardized prompts measured precision, cost, and execution time; Related IS Theory: The study draws on Information Systems theories, emphasizing socio-technical, ethical, and cost–benefit aspects (Socio-technical Theory, Actor-Network Theory, Resource-Based View, Dynamic Capabilities); Research Method: Over 2,000 comments were analyzed by LLMs using precision, recall, F1-score, operational cost, and processing time metrics; Summary of Results: GPT-4.0 achieved the highest F1-score (94.19%) but at high cost (US$ 26.99). DeepSeek-V3 balanced performance and cost (F1-score 93.37%, US$ 0.66). Gemini-2.0-Flash was the cheapest (US$ 0.12) but showed inconsistent results; Contributions and Impact to IS area: This work offers a practical framework for selecting LLMs for hate-speech detection based on accuracy, cost, and performance. It advances IS research by evaluating state-of-the-art models in real scenarios and providing guidance for ethical and efficient content moderation.