Enhancing Legal Question Answering in Brazilian Portuguese through Domain-Specific Embedding Models - Novais, Artur M. A.; Ferreira, David O. C.; Silva, Josiel P. C.; Brakes, Matheus F. C.; Presa, João P. C.; Oliveira, Sávio S. T. de (Anais do Encontro Nacional de Inteligência Artificial e Computacional (ENIAC), 2025)
The increasing digitization of legal documents presents significant challenges for information retrieval. Traditional keyword-based search methods often fail to capture the semantic nuances of complex legal queries. RetrievalAugmented Generation (RAG) has emerged as a powerful paradigm for building Question Answering (Q&A) systems, but its effectiveness is highly dependent on the quality of its retrieval component. This paper addresses the problem of improving semantic search over legal texts from the Court of Accounts of the State of Goiás (TCE-GO). We introduce two specialized embedding models created by fine-tuning the state-of-the-art BGE-M3 model on domain-specific corpora of jurisprudence and legislation, respectively. Our experimental results demonstrate that these specialized models significantly outperform general-purpose multilingual and Portuguese models in retrieval tasks, as measured by MRR@10 and Recall@10. Notably, our fine-tuned models, despite their moderate size, surpass much larger models, highlighting that domain specialization is a more parameter-efficient strategy than simply scaling model size for niche domains.