A Comparative Study of BERT Models for Semantic Retrieval of Brazilian Legal Precedents - Justino, Adrielson Ferreira; Jacob Junior, Antônio Fernando Lavareda; Lobato, Fábio Manoel França (Anais do Symposium on Knowledge Discovery, Mining and Learning (KDMiLe); 2025: Anais do XIII Symposium on Knowledge Discovery, Mining and LearningDO - 10.5753/kdmile.2025.247782, 2025)
The growing volume of digital documents in Brazilian courts has intensified challenges related to judicial inefficiency, including the slow identification of established precedents and the risk of inconsistent rulings in similar cases. To address this, this study presents a systematic empirical study on the effectiveness of different classes of BERT-based embedding models for the semantic retrieval of legal documents, focusing on identifying relevant precedents for new complaints. A pipeline employing document chunking to handle long legal texts and using Elasticsearch for large-scale dense vector retrieval was implemented and evaluated. Using a corpus of legal complaints from the Maranhão State Court of Justice, three model categories were compared: (i) a general-purpose Portuguese model, (ii) three domain-specific models trained on Brazilian legal corpora, and (iii) a task-specific Sentence-BERT model fine-tuned for similarity tasks. Performance was assessed using a proxy-based protocol with precedent class labels as the ground truth, measured by Precision@k, MRR@15, and MAP@15. The results indicate that the task-specific SBERT-pt model achieved higher performance within the evaluated setting, consistently outperforming other tested models across the selected metrics. This model showed improvements in both immediate relevance (Precision@1 of 0.787) and overall ranking quality (MAP of 0.806). Although domain adaptation provided marginal benefits over the general-purpose baseline, task-specific finetuning for similarity appeared to be the most influential factor for retrieval quality in this scenario. These findings suggest that optimizing models for retrieval tasks can offer a promising direction for enhancing semantic searches in legal contexts. However, further studies using larger and more diverse datasets are needed to confirm the generalizability of these results and assess their impact on real-world judicial workflows.