Automatic Scoring of Elementary School Essays in Brazilian Portuguese with LLMs: Comparing Gemini, GPT-4o, Claude, and Mistral - Lobo, Jamilla; Anthony, Lenon; Falcão, Andreza; Xavier, Cleon; Torrezão, Newarney; Isotani, Seiji; Ibert, Ig; Rodrigues, Luiz; Mello, Rafael (Anais do Simpósio Brasileiro de Informática na Educação (SBIE), 2025)
Writing is an essential skill for the development of students’ critical thinking, communication, and language competencies. However, evaluating written productions efficiently and within an appropriate timeframe remains a challenge, especially in contexts of high teaching demands. This study investigates the use of Large Language Models (LLMs) for the automated analysis of essays with a focus on achieving accurate and consistent assessments. Four advanced models such as Gemini, GPT-4o, Claude 3.7 and Mistral were examined and applied to the evaluation of narrative texts written in Brazilian Portuguese by elementary school students. The results indicate that the Gemini 2.0 Pro model demonstrated greater accuracy in score assignment, while Claude 3.7 stood out for consistency in alignment with human evaluation. The findings highlight the potential of LLMs to support pedagogical practices by providing consistent assessments and contributing to the development of students’ writing skills. The study proposes viable alternatives for the use of artificial intelligence in educational contexts with limited resources.