Assessing DeepSeek-R1’s Performance on Brazil’s Defining National Education Benchmark

• Alexandre Thurow Bender UFPel
• Gabriel Almeida Gomes UFPel
• Ulisses Brisolara Corrêa UFPel
• Ricardo Matsumura Araujo UFPel

Resumo

This paper evaluates DeepSeek-R1’s performance on Brazil’s National High School Exam (ENEM), a culturally-specific educational benchmark that tests reasoning across multiple domains. Using the pass@k metric across three years (2022–2024) of ENEM questions, we found strong reasoning capabilities, particularly in Human Sciences, where 2023–2024 performance exceeded 0.98 pass@1 score. Mathematics showed the most variability, with diverse scores across the three years. The model demonstrated sophisticated self-translation capabilities when handling Brazilian Portuguese language questions without explicit translation instructions. Despite strong overall performance, inconsistencies across subject domains persist. Our findings suggest that recent advances in AI reasoning extend effectively beyond typical AI benchmarks to diverse cultural contexts, with implications for AI deployment in various global settings. This evaluation contributes to understanding how generative AI systems perform when facing multidisciplinary, culturally-situated reasoning challenges.