Application of Large Language Models in the Analysis and Synthesis of Legal Documents: A Literature Review

• Matheus Belarmino UFCA
• Rackel Coelho UFCA
• Roberto Lotufo NeuralMind.ai
• Jayr Pereira UFCA

DOI: https://doi.org/10.5753/lasdigov.2025.9088

Abstract

Large Language Models (LLMs) have been increasingly used to optimize the analysis and synthesis of legal documents, enabling the automation of tasks such as summarization, classification, and retrieval of legal information. This study aims to conduct a systematic literature review to identify the state of the art in prompt engineering applied to LLMs in the legal context. The results indicate that models such as GPT-4, BERT, Llama 2, and Legal-Pegasus are widely employed in the legal field, and techniques such as Few-shot Learning, Zero-shot Learning, and Chain-of-Thought prompting have proven effective in improving the interpretation of legal texts. However, challenges such as biases in models and hallucinations still hinder their large-scale implementation. It is concluded that, despite the great potential of LLMs for the legal field, there is a need to improve prompt engineering strategies to ensure greater accuracy and reliability in the generated results.

References

Abonizio, H., Almeida, T. S., Laitz, T., Junior, R. M., Bonás, G. K., Nogueira, R., and Pires, R. (2025). Sabiá-3 technical report.

Chen, K., Chen, B., Gao, D., Dai, H., Jiang, W., Ning, W., Yu, S., Yang, L., and Cai, X. (2024). General2specialized llms translation for e-commerce. In Companion Proceedings of the ACM Web Conference 2024, WWW ’24, page 670–673, New York, NY, USA. Association for Computing Machinery.

Choi, J. H. (2024). How to use large language models for empirical legal research. Journal of Institutional and Theoretical Economics (JITE), 180(2):214–233.

Coelho, G., Celecia, A., de Sousa, J., Lemos, M., Lima, M., Mangeth, A., Frajhof, I., and Casanova, M. (2024). Information extraction in the legal domain: Traditional supervised learning vs. chatgpt. In Proceedings of the 26th International Conference on Enterprise Information Systems, volume 1, pages 579–586.

Deroy, A., Ghosh, K., and Ghosh, S. (2023). How ready are pre-trained abstractive models and llms for legal case judgement summarization?

Deroy, A., Ghosh, K., and Ghosh, S. (2024). Applicability of large language models and generative models for legal case judgement summarization. Artificial Intelligence and Law, pages 1–44.

Ghosh, S., Evuru, C. K., Kumar, S., Ramaneswaran, S., Sakshi, S., Tyagi, U., and Manocha, D. (2023). Dale: Generative data augmentation for low-resource legal nlp.

Hijazi, F., AlHarbi, S., AlHussein, A., Shairah, H. A., AlZahrani, R., AlShamlan, H., Knio, O., and Turkiyyah, G. (2024). Arablegaleval: A multitask benchmark for assessing arabic legal knowledge in large language models. arXiv preprint arXiv:2408.07983.

Kang, X., Qu, L., Soon, L.-K., Trakic, A., Zhuo, T., Emerton, P., and Grant, G. (2023). Can chatgpt perform reasoning using the irac method in analyzing legal scenarios like a lawyer? In Findings of the Association for Computational Linguistics: EMNLP 2023, page 13900–13923. Association for Computational Linguistics.

Moura, A. and Carvalho, A. A. (2023). Literacia de prompts para potenciar o uso da inteligência artificial na educação. RE@ D-Revista de Educação a Distância e Elearning, 6(2):e202308–e202308.

Nunes, R. O., Spritzer, A. S., Freitas, C. M. D. S., and Balreira, D. G. (2024). Out of sesame street: a study of portuguese legal named entity recognition through in-context learning. In Proceedings of the 26th International Conference on Enterprise Information Systems.

Peixoto, F. H. and Bonat, D. (2023). Gpts e direito: impactos prováveis das ias generativas nas atividades jurídicas brasileiras. Sequência (Florianópolis), 44(93):e94238.

Prasad, N., Boughanem, M., and Dkaki, T. (2024). Exploring large language models and hierarchical frameworks for classification of large unstructured legal documents. In Goharian, N., Tonellotto, N., He, Y., Lipani, A., McDonald, G., Macdonald, C., and Ounis, I., editors, Advances in Information Retrieval, pages 221–237, Cham. Springer Nature Switzerland.

SILVA, D. H. S. d. (2023). Análise de sentimentos em decisões e entendimentos do supremo tribunal de justiça utilizando large language models.

Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L. u., and Polosukhin, I. (2017). Attention is all you need. In Guyon, I., Luxburg, U. V., Bengio, S., Wallach, H., Fergus, R., Vishwanathan, S., and Garnett, R., editors, Advances in Neural Information Processing Systems, volume 30. Curran Associates, Inc.

Venkatakrishnan, R., Tanyildizi, E., and Canbaz, M. A. (2024). Semantic interlinking of immigration data using llms for knowledge graph construction. In Companion Proceedings of the ACM Web Conference 2024, WWW ’24, page 605–608, New York, NY, USA. Association for Computing Machinery.

Wei, J., Wang, X., Schuurmans, D., Bosma, M., Xia, F., Chi, E., Le, Q. V., Zhou, D., et al. (2022). Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824–24837.

Wu, Y., Wang, C., Gumusel, E., and Liu, X. (2024). Knowledge-infused legal wisdom: Navigating llm consultation through the lens of diagnostics and positive-unlabeled reinforcement learning.

Wu, Y., Zhou, S., Liu, Y., Lu, W., Liu, X., Zhang, Y., Sun, C., Wu, F., and Kuang, K. (2023). Precedent-enhanced legal judgment prediction with llm and domain-model collaboration.

Zhou, Y., Huang, H., and Wu, Z. (2023). Boosting legal case retrieval by query content selection with large language models. In Proceedings of the Annual International ACM SIGIR Conference on Research and Development in Information Retrieval in the Asia Pacific Region, SIGIR-AP ’23, page 176–184, New York, NY, USA. Association for Computing Machinery.