Fact, Legal Thesis, and Noise: Comparing Classical ML, BERT, and LLMs for Triage of Initial Petitions in Brazilian Portuguese
Resumo
In Brazilian state courts, manual document triage is slow and error-prone, requiring the separation of factual narrative, legal arguments, and procedural noise. This work compares classical models, fine-tuned BERT encoders, and out-of-the-box large language models (LLMs) for classifying Brazilian Portuguese initial petitions into fact, legal thesis, and noise. To our knowledge, this is the first such comparison for this task in Portuguese. Portuguese-pretrained encoders performed best (macro-F1 about 0.99), followed by a long-context Longformer (0.983) and classical models (0.973); out-of-the-box LLMs reached about 0.85 and were highly sensitive to output format, with formatting failures significantly impacting performance. Errors concentrate on passages mixing factual narration and legal argument, confirmed by three legal professionals (Fleiss’ κ = 0.46). An ablation shows that anonymization markers do not drive the scores. The results help guide model selection for court document triage.
Referências
Adsul, N. and Deshmukh, P. (2023). Automatically classify each sentence from Indian legal judgements to its semantic role. In Proceedings of the 7th International Conference on Computer Applications in Electrical Engineering (CERA 2023). IEEE.
Anthropic (2024). The claude 3 model family: Opus, sonnet, haiku. [link].
Anthropic (2026). System card: Claude sonnet 4.6. [link].
Aragy, R., Fernandes, E. R., and Caceres, E. N. (2021). Rhetorical role identification for Portuguese legal documents. In Intelligent Systems (BRACIS 2021), Lecture Notes in Computer Science. Springer.
Belfathi, A., Hernandez, N., and Monceaux, L. (2023). Enhancing pre-trained language models with sentence position embeddings for rhetorical roles recognition in legal opinions. In CEUR Workshop Proceedings.
Beltagy, I., Peters, M. E., and Cohan, A. (2020). Longformer: The long-document transformer. arXiv preprint arXiv:2004.05150.
Bendová, K., Knap, T., Černý, J., Pour, V., Savelka, J., Kvapilíková, I., and Drápal, J. (2025). What are the facts? automated extraction of court-established facts from criminal-court opinions. In CEUR Workshop Proceedings.
Bhattacharya, P., Paul, S., Ghosh, K., Ghosh, S., and Wyner, A. (2023). DeepRhole: Deep learning for rhetorical role labeling of sentences in legal case documents. Artificial Intelligence and Law.
Bomfim, F., Neto, J., Neves, A., Furtado, V., Silva, S., and Pinheiro, V. (2026). Automating legal reasoning with generative AI: A framework for judicial decision support. In Proceedings of the 20th International Conference on Artificial Intelligence and Law (ICAIL 2025).
Bonfim, F. C. J., Silva, S. P., Neves, A. S., Santos, H., Neto, J. A. M., Araujo, J. L. B., Pires, R. S., Costa, R. A. D. S., Furtado, V., and Pinheiro, V. (2026). SARA: Leveraging LLM agents and jurisprudential ontologies for automated legal reasoning. In Proceedings of the AAAI Conference on Artificial Intelligence.
Canaverde, B., Pires, T. P., Ribeiro, L. M., and Martins, A. F. (2025). Legalbench. pt: a benchmark for portuguese law. arXiv preprint arXiv:2502.16357.
Conneau, A., Khandelwal, K., Goyal, N., Chaudhary, V., Wenzek, G., Guzmán, F., Grave, E., Ott, M., Zettlemoyer, L., and Stoyanov, V. (2019). Unsupervised cross-lingual representation learning at scale. CoRR, abs/1911.02116.
Conselho Nacional de Justiça (Brasil) (2025). Resolução no 615, de 11 de março de 2025. Atos Normativos do Conselho Nacional de Justiça. Access em: 9 dez. 2025.
Csányi, G. M., Üveges, I., Lakatos, D., Ripszám, D., Kozák, K., Nagy, D., and Vadász, J. P. (2025). Sentence-level rhetorical role labeling in judicial decisions. Big Data and Cognitive Computing.
Dal Pont, T. R., Billi, M., Sabo, I. C., Lagioia, F., Hübner, J. F., Sartor, G., and Rover, A. J. (2026). Factor extraction from pretrial detention decisions by Italian and Brazilian supreme courts: A knowledge graph perspective. Computer Law and Security Review.
Darji, H., Heckelmann, M., Kratsch, C., and De Melo, G. (2025). Segmentation and processing of German court decisions from open legal data. In Frontiers in Artificial Intelligence and Applications.
DeepSeek-AI, Liu, A., et al. (2025). Deepseek-v3.2: Pushing the frontier of open large language models. arXiv preprint arXiv:2512.02556. Model used via Amazon Bedrock.
Devlin, J., Chang, M.-W., Lee, K., and Toutanova, K. (2019). BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of NAACL-HLT, pages 4171–4186.
Garcia, E. A. S., Silva, N. F. F., Siqueira, F., Albuquerque, H. O., Gomes, J. R. S., Souza, E., and Lima, E. A. (2024). RoBERTaLexPT: A legal RoBERTa model pretrained with deduplication for Portuguese. In Gamallo, P., Claro, D., Teixeira, A., Real, L., Garcia, M., Oliveira, H. G., and Amaro, R., editors, Proceedings of the 16th International Conference on Computational Processing of Portuguese, pages 374–383, Santiago de Compostela, Galicia/Spain. Association for Computational Lingustics.
Gemma Team, Kamath, A., et al. (2025). Gemma 3 technical report. arXiv preprint arXiv:2503.19786. Reference for Gemma 3 27B. Model used via Amazon Bedrock.
He, P., Gao, J., and Chen, W. (2021a). Debertav3: Improving deberta using electra-style pre-training with gradient-disentangled embedding sharing.
He, P., Liu, X., Gao, J., and Chen, W. (2021b). Deberta: Decoding-enhanced bert with disentangled attention. In International Conference on Learning Representations.
Kalamkar, P., Tiwari, A., Agarwal, A., Karn, S., Gupta, S., Raghavan, V., and Modi, A. (2022). Corpus for automatic structuring of legal documents. In Proceedings of the Language Resources and Evaluation Conference (LREC 2022).
Lai, J., Gan, W., Wu, J., Qi, Z., and Yu, P. S. (2024). Large language models in law: A survey. AI Open, 5:181–196.
Laitz, T., Sales Almeida, T., Abonizio, H., Malaquias Junior, R., Bon’as, G. K., Piau, M., Larcher, C., Pires, R., and Nogueira, R. (2026). Sabi’a-4 technical report. arXiv preprint arXiv:2603.10213. Model accessed via Maritaca API.
Liepin, R., Galloni, F., Lagioia, F., Lippi, M., Musicco, M., Sayin, B., Passerini, A., and Sartor, G. (2025). Legal argument mining: Recent trends and open challenges. In CEUR Workshop Proceedings.
Lou, A., Salaün, O., Westermann, H., and Kosseim, L. (2021). Extracting facts from case rulings through paragraph segmentation of judicial decisions. In Legal Knowledge and Information Systems, Lecture Notes in Computer Science. Springer.
Malik, V., Sanjay, R., Guha, S. K., Hazarika, A., Nigam, S., Bhattacharya, A., and Modi, A. (2022). Semantic segmentation of legal documents via rhetorical roles. In Proceedings of the Natural Legal Language Processing Workshop (NLLP 2022).
Mistral AI (2025). Mistral large 3. [link].
Mistral AI, Rastogi, A., Jiang, A. Q., Lo, A., Berrada, G., Lample, G., et al. (2025). Magistral. arXiv preprint arXiv:2506.10910. Reference for Magistral Small 2509. Model used via Amazon Bedrock.
Mohan, D. and Nair, L. R. (2024). Deep learning-based semantic segmentation for legal texts: Unveiling rhetorical roles in legal case documents. In E3S Web of Conferences.
Mok, J. R., Mok, W. Y., and Mok, R. V. (2021). Sentence classification for contract law cases: A natural language processing approach. In Proceedings of the 18th International Conference on Artificial Intelligence and Law (ICAIL 2021).
Östling, A., Sargeant, H., Xie, H., Bull, L., Terenin, A., Jonsson, L., Magnusson, M., and Steffek, F. (2023). The cambridge law corpus: A dataset for legal ai research. Advances in Neural Information Processing Systems, 36:41355–41385.
Rapôso, C. F. L., de Lima, H. M., de Oliveira Junior, W. F., Silva, P. A. F., and de Souza Barros, E. E. (2019). Lgpd-lei geral de proteção de dados pessoais em tecnologia da informação: Revisão sistemática. RACE-Revista de Administração do Cesmac, 4:58–67.
Santos, W. F., Galvão Filho, A. R., de Oliveira, S. S. T., and Presa, J. P. C. (2025). Comparing prompt-based llms, fine-tuning, and classical models for legal text classification in portuguese. In Encontro Nacional de Inteligência Artificial e Computacional (ENIAC), pages 1138–1149. SBC.
Santos, W. F., Viana, J. G. G., Júnior, A. P. d. C., Trindade, F. R., and Silva, N. F. F. d. (2026). LexIris-pt and LexBert-pt: Specialized sentence embeddings for legal similarity in Brazilian Portuguese. In Souza, M., de Dios-Flores, I., Santos, D., Freitas, L., Souza, J. W. d. C., and Ribeiro, E., editors, Proceedings of the 17th International Conference on Computational Processing of Portuguese (PROPOR 2026) - Vol. 1, pages 540–550, Salvador, Brazil. Association for Computational Linguistics.
Šavelka, J. and Ashley, K. D. (2018). Segmenting U.S. court decisions into functional and issue specific parts. In Frontiers in Artificial Intelligence and Applications (JURIX 2018).
Souza, F., Nogueira, R., and Lotufo, R. (2020). BERTimbau: Pretrained BERT models for Brazilian Portuguese. In Intelligent Systems (BRACIS 2020), volume 12319 of Lecture Notes in Computer Science, pages 403–417. Springer.
Tomasi, S. (2026). Does ChatGPT have law sense? reflections on an experimental study of argument mining in judicial decisions of the Italian court of cassation. BioLaw Journal.
Yang, A. et al. (2025). Qwen3 technical report. arXiv preprint arXiv:2505.09388.
Zaheer, M., Guruganesh, G., Dubey, A., Ainslie, J., Alberti, C., Ontañón, S., Pham, P., Ravula, A., Wang, Q., Yang, L., and Ahmed, A. (2020). Big bird: Transformers for longer sequences. In Advances in Neural Information Processing Systems (NeurIPS), volume 33.
Anthropic (2024). The claude 3 model family: Opus, sonnet, haiku. [link].
Anthropic (2026). System card: Claude sonnet 4.6. [link].
Aragy, R., Fernandes, E. R., and Caceres, E. N. (2021). Rhetorical role identification for Portuguese legal documents. In Intelligent Systems (BRACIS 2021), Lecture Notes in Computer Science. Springer.
Belfathi, A., Hernandez, N., and Monceaux, L. (2023). Enhancing pre-trained language models with sentence position embeddings for rhetorical roles recognition in legal opinions. In CEUR Workshop Proceedings.
Beltagy, I., Peters, M. E., and Cohan, A. (2020). Longformer: The long-document transformer. arXiv preprint arXiv:2004.05150.
Bendová, K., Knap, T., Černý, J., Pour, V., Savelka, J., Kvapilíková, I., and Drápal, J. (2025). What are the facts? automated extraction of court-established facts from criminal-court opinions. In CEUR Workshop Proceedings.
Bhattacharya, P., Paul, S., Ghosh, K., Ghosh, S., and Wyner, A. (2023). DeepRhole: Deep learning for rhetorical role labeling of sentences in legal case documents. Artificial Intelligence and Law.
Bomfim, F., Neto, J., Neves, A., Furtado, V., Silva, S., and Pinheiro, V. (2026). Automating legal reasoning with generative AI: A framework for judicial decision support. In Proceedings of the 20th International Conference on Artificial Intelligence and Law (ICAIL 2025).
Bonfim, F. C. J., Silva, S. P., Neves, A. S., Santos, H., Neto, J. A. M., Araujo, J. L. B., Pires, R. S., Costa, R. A. D. S., Furtado, V., and Pinheiro, V. (2026). SARA: Leveraging LLM agents and jurisprudential ontologies for automated legal reasoning. In Proceedings of the AAAI Conference on Artificial Intelligence.
Canaverde, B., Pires, T. P., Ribeiro, L. M., and Martins, A. F. (2025). Legalbench. pt: a benchmark for portuguese law. arXiv preprint arXiv:2502.16357.
Conneau, A., Khandelwal, K., Goyal, N., Chaudhary, V., Wenzek, G., Guzmán, F., Grave, E., Ott, M., Zettlemoyer, L., and Stoyanov, V. (2019). Unsupervised cross-lingual representation learning at scale. CoRR, abs/1911.02116.
Conselho Nacional de Justiça (Brasil) (2025). Resolução no 615, de 11 de março de 2025. Atos Normativos do Conselho Nacional de Justiça. Access em: 9 dez. 2025.
Csányi, G. M., Üveges, I., Lakatos, D., Ripszám, D., Kozák, K., Nagy, D., and Vadász, J. P. (2025). Sentence-level rhetorical role labeling in judicial decisions. Big Data and Cognitive Computing.
Dal Pont, T. R., Billi, M., Sabo, I. C., Lagioia, F., Hübner, J. F., Sartor, G., and Rover, A. J. (2026). Factor extraction from pretrial detention decisions by Italian and Brazilian supreme courts: A knowledge graph perspective. Computer Law and Security Review.
Darji, H., Heckelmann, M., Kratsch, C., and De Melo, G. (2025). Segmentation and processing of German court decisions from open legal data. In Frontiers in Artificial Intelligence and Applications.
DeepSeek-AI, Liu, A., et al. (2025). Deepseek-v3.2: Pushing the frontier of open large language models. arXiv preprint arXiv:2512.02556. Model used via Amazon Bedrock.
Devlin, J., Chang, M.-W., Lee, K., and Toutanova, K. (2019). BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of NAACL-HLT, pages 4171–4186.
Garcia, E. A. S., Silva, N. F. F., Siqueira, F., Albuquerque, H. O., Gomes, J. R. S., Souza, E., and Lima, E. A. (2024). RoBERTaLexPT: A legal RoBERTa model pretrained with deduplication for Portuguese. In Gamallo, P., Claro, D., Teixeira, A., Real, L., Garcia, M., Oliveira, H. G., and Amaro, R., editors, Proceedings of the 16th International Conference on Computational Processing of Portuguese, pages 374–383, Santiago de Compostela, Galicia/Spain. Association for Computational Lingustics.
Gemma Team, Kamath, A., et al. (2025). Gemma 3 technical report. arXiv preprint arXiv:2503.19786. Reference for Gemma 3 27B. Model used via Amazon Bedrock.
He, P., Gao, J., and Chen, W. (2021a). Debertav3: Improving deberta using electra-style pre-training with gradient-disentangled embedding sharing.
He, P., Liu, X., Gao, J., and Chen, W. (2021b). Deberta: Decoding-enhanced bert with disentangled attention. In International Conference on Learning Representations.
Kalamkar, P., Tiwari, A., Agarwal, A., Karn, S., Gupta, S., Raghavan, V., and Modi, A. (2022). Corpus for automatic structuring of legal documents. In Proceedings of the Language Resources and Evaluation Conference (LREC 2022).
Lai, J., Gan, W., Wu, J., Qi, Z., and Yu, P. S. (2024). Large language models in law: A survey. AI Open, 5:181–196.
Laitz, T., Sales Almeida, T., Abonizio, H., Malaquias Junior, R., Bon’as, G. K., Piau, M., Larcher, C., Pires, R., and Nogueira, R. (2026). Sabi’a-4 technical report. arXiv preprint arXiv:2603.10213. Model accessed via Maritaca API.
Liepin, R., Galloni, F., Lagioia, F., Lippi, M., Musicco, M., Sayin, B., Passerini, A., and Sartor, G. (2025). Legal argument mining: Recent trends and open challenges. In CEUR Workshop Proceedings.
Lou, A., Salaün, O., Westermann, H., and Kosseim, L. (2021). Extracting facts from case rulings through paragraph segmentation of judicial decisions. In Legal Knowledge and Information Systems, Lecture Notes in Computer Science. Springer.
Malik, V., Sanjay, R., Guha, S. K., Hazarika, A., Nigam, S., Bhattacharya, A., and Modi, A. (2022). Semantic segmentation of legal documents via rhetorical roles. In Proceedings of the Natural Legal Language Processing Workshop (NLLP 2022).
Mistral AI (2025). Mistral large 3. [link].
Mistral AI, Rastogi, A., Jiang, A. Q., Lo, A., Berrada, G., Lample, G., et al. (2025). Magistral. arXiv preprint arXiv:2506.10910. Reference for Magistral Small 2509. Model used via Amazon Bedrock.
Mohan, D. and Nair, L. R. (2024). Deep learning-based semantic segmentation for legal texts: Unveiling rhetorical roles in legal case documents. In E3S Web of Conferences.
Mok, J. R., Mok, W. Y., and Mok, R. V. (2021). Sentence classification for contract law cases: A natural language processing approach. In Proceedings of the 18th International Conference on Artificial Intelligence and Law (ICAIL 2021).
Östling, A., Sargeant, H., Xie, H., Bull, L., Terenin, A., Jonsson, L., Magnusson, M., and Steffek, F. (2023). The cambridge law corpus: A dataset for legal ai research. Advances in Neural Information Processing Systems, 36:41355–41385.
Rapôso, C. F. L., de Lima, H. M., de Oliveira Junior, W. F., Silva, P. A. F., and de Souza Barros, E. E. (2019). Lgpd-lei geral de proteção de dados pessoais em tecnologia da informação: Revisão sistemática. RACE-Revista de Administração do Cesmac, 4:58–67.
Santos, W. F., Galvão Filho, A. R., de Oliveira, S. S. T., and Presa, J. P. C. (2025). Comparing prompt-based llms, fine-tuning, and classical models for legal text classification in portuguese. In Encontro Nacional de Inteligência Artificial e Computacional (ENIAC), pages 1138–1149. SBC.
Santos, W. F., Viana, J. G. G., Júnior, A. P. d. C., Trindade, F. R., and Silva, N. F. F. d. (2026). LexIris-pt and LexBert-pt: Specialized sentence embeddings for legal similarity in Brazilian Portuguese. In Souza, M., de Dios-Flores, I., Santos, D., Freitas, L., Souza, J. W. d. C., and Ribeiro, E., editors, Proceedings of the 17th International Conference on Computational Processing of Portuguese (PROPOR 2026) - Vol. 1, pages 540–550, Salvador, Brazil. Association for Computational Linguistics.
Šavelka, J. and Ashley, K. D. (2018). Segmenting U.S. court decisions into functional and issue specific parts. In Frontiers in Artificial Intelligence and Applications (JURIX 2018).
Souza, F., Nogueira, R., and Lotufo, R. (2020). BERTimbau: Pretrained BERT models for Brazilian Portuguese. In Intelligent Systems (BRACIS 2020), volume 12319 of Lecture Notes in Computer Science, pages 403–417. Springer.
Tomasi, S. (2026). Does ChatGPT have law sense? reflections on an experimental study of argument mining in judicial decisions of the Italian court of cassation. BioLaw Journal.
Yang, A. et al. (2025). Qwen3 technical report. arXiv preprint arXiv:2505.09388.
Zaheer, M., Guruganesh, G., Dubey, A., Ainslie, J., Alberti, C., Ontañón, S., Pham, P., Ravula, A., Wang, Q., Yang, L., and Ahmed, A. (2020). Big bird: Transformers for longer sequences. In Advances in Neural Information Processing Systems (NeurIPS), volume 33.
Publicado
19/10/2026
Como Citar
SANTOS, Willgnner F. et al.
Fact, Legal Thesis, and Noise: Comparing Classical ML, BERT, and LLMs for Triage of Initial Petitions in Brazilian Portuguese. In: ENCONTRO NACIONAL DE INTELIGÊNCIA ARTIFICIAL E COMPUTACIONAL (ENIAC), 23. , 2026, Cuiabá/MT.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2026
.
p. 433-444.
ISSN 2763-9061.
DOI: https://doi.org/10.5753/eniac.2026.30008.
