JABUTI-SQL: Um Benchmark em Português para Avaliação de Abordagens Text-to-SQL em Cenários Reais - Botelho, Matheus; Fonseca, Guilherme; R. Silva, Lucas; Gonçalves, Rodrigo M.; Lobato, Yago; Ribeiro, Gustavo A.; de Camara, Eduardo C.; Zardin, Jaide F. C.; Calais, Pedro; A. Simoes, Emerson; Sene, Allan; C. S. Reis, Julio; et al. (Anais do Dataset Showcase Workshop (DSW); 2026: Anais do VIII Dataset Showcase WorkshopDO - 10.5753/dsw.2026.249611, 2026)
This paper presents JABUTI-SQL (Joint Annotated Benchmark for User-centric Text-to-SQL In Real Health Data), a Brazilian benchmark for Text-to-SQL built from real public healthcare data provided by DATASUS, the Brazilian public health data platform. The benchmark integrates multiple heterogeneous sources related to pharmaceutical assistance, public procurement, hospital infrastructure, and regional healthcare organization into a unified relational schema. Unlike synthetic benchmarks, JABUTI-SQL preserves characteristics commonly found in real governmental databases, including incomplete records, semantic inconsistencies, heterogeneous structures, and complex inter-table relations. The benchmark contains 69 Portuguese question–SQL pairs covering three user personas and multiple difficulty levels, aiming to support the evaluation of Text-to-SQL systems under more realistic conditions.