Abstract
Pre-trained Transformer models have been used to improve the results of several NLP tasks, which includes the Legal Rhetorical Role Labeling (Legal RRL) one. This task assigns semantic functions, such as fact and argument, to sentences from judgment documents. Several Legal RRL works exploit pre-trained Transformers to encode sentences but only a few employ approaches other than fine-tuning to improve the performance of models. In this work, we implement three of such approaches and evaluate them over the same datasets to achieve a better perception of their impacts. In our experiments, approaches based on data augmentation and positional encoders do not provide performance gains to our models. Conversely, the models based on the DFCSC approach overcome the appropriate baselines, and they do remarkably well as the lowest and highest improvements respectively are 5.9% and 10.4%.
This work was supported by NPAD/UFRN, by the Coordenação de Aperfeiçoamento de Pessoal de Nível Superior - Brasil (CAPES, Finance Code 001) and by the LawBot project (ANR-20-CE38-0013), granted by ANR the French Agence Nationale de la Recherche.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
Available at https://github.com/Exploration-Lab/Rhetorical-Roles.
References
Aragy, R., Fernandes, E.R., Caceres, E.N.: Rhetorical role identification for Portuguese legal documents. In: Britto, A., Valdivia Delgado, K. (eds.) BRACIS 2021. LNCS (LNAI), vol. 13074, pp. 557–571. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-91699-2_38
Beltagy, I., Peters, M.E., Cohan, A.: Longformer: the long-document transformer. CoRR abs/2004.05150 (2020). https://arxiv.org/abs/2004.05150
Bhattacharya, P., Paul, S., Ghosh, K., Ghosh, S., Wyner, A.: Deeprhole: deep learning for rhetorical role labeling of sentences in legal case documents. Artificial Intelligence and Law, pp. 1–38 (2021)
Cohan, A., Beltagy, I., King, D., Dalvi, B., Weld, D.: Pretrained language models for sequential sentence classification. In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pp. 3693–3699. Association for Computational Linguistics, Hong Kong, November 2019. https://doi.org/10.18653/v1/D19-1383. https://aclanthology.org/D19-1383
Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: BERT: pre-training of deep bidirectional transformers for language understanding. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pp. 4171–4186. Association for Computational Linguistics, Minneapolis, Minnesota, June 2019. https://doi.org/10.18653/v1/N19-1423. https://aclanthology.org/N19-1423
Dror, R., Shlomov, S., Reichart, R.: Deep dominance - how to properly compare deep neural models. In: Korhonen, A., Traum, D.R., Màrquez, L. (eds.) Proceedings of the 57th Conference of the Association for Computational Linguistics, ACL 2019, Florence, Italy, July 28–August 2, 2019, Volume 1: Long Papers, pp. 2773–2785. Association for Computational Linguistics (2019). https://doi.org/10.18653/v1/p19-1266. https://doi.org/10.18653/v1/p19-1266
Feijo, D., Moreira, V.: Summarizing legal rulings: comparative experiments. In: Proceedings of the International Conference on Recent Advances in Natural Language Processing (RANLP 2019), pp. 313–322. INCOMA Ltd., Varna, Bulgaria, September 2019. https://doi.org/10.26615/978-954-452-056-4_036. http://aclanthology.org/R19-1036
Kalamkar, P., et al.: Corpus for automatic structuring of legal documents. In: Proceedings of the Thirteenth Language Resources and Evaluation Conference, pp. 4420–4429. European Language Resources Association, Marseille, France, June 2022. https://aclanthology.org/2022.lrec-1.470
Li, D., Yang, K., Zhang, L., Yin, D., Peng, D.: Class: a novel method for chinese legal judgments summarization. In: Proceedings of the 5th International Conference on Computer Science and Application Engineering. CSAE 2021. Association for Computing Machinery, New York (2021). https://doi.org/10.1145/3487075.3487161
de Lima, A.G., Boughanem, M., da S. Aranha, E.H., Dkaki, T., Moreno, J.G.: Exploring SBERT and mixup data augmentation in rhetorical role labeling of Indian legal sentences. In: Tamine, L., Amigó, E., Mothe, J. (eds.) Proceedings of the 2nd Joint Conference of the Information Retrieval Communities in Europe (CIRCLE 2022), Samatan, Gers, France, July 4–7, 2022. CEUR Workshop Proceedings, vol. 3178. CEUR-WS.org (2022). https://ceur-ws.org/Vol-3178/CIRCLE_2022_paper_29.pdf
de Lima, A.G., Moreno, J.G., da S. Aranha, E.H.: IRIT_IRIS_A at SemEval-2023 Task 6: legal rhetorical role labeling supported by dynamic-filled contextualized sentence chunks. In: Proceedings of the 17th International Workshop on Semantic Evaluation (SemEval-2023). Association for Computational Linguistics, Toronto, Canada, July 2023
de Lima, A.G., Moreno, J.G., Boughanem, M., Dkaki, T., da S. Aranha, E.H.: Leveraging positional encoding to improve fact identification in legal documents. In: First International Workshop on Legal Information Retrieval (LegalIR) at ECIR 2023, pp. 11–13 (2023). https://tmr.liacs.nl/legalIR/LegalIR2023_proceedings.pdf
Liu, Y., et al.: Roberta: a robustly optimized BERT pretraining approach. CoRR abs/1907.11692 (2019). https://arxiv.org/abs/1907.11692
Ma, L., Zhang, Y., Wang, T., Liu, X., Ye, W., Sun, C., Zhang, S.: Legal judgment prediction with multi-stage case representation learning in the real court setting. In: Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 993–1002. SIGIR ’21. Association for Computing Machinery, New York (2021). https://doi.org/10.1145/3404835.3462945
Malik, V., Sanjay, R., Guha, S.K., Hazarika, A., Nigam, S., Bhattacharya, A., Modi, A.: Semantic segmentation of legal documents via rhetorical roles. In: Proceedings of the Natural Legal Language Processing Workshop 2022, pp. 153–171. Association for Computational Linguistics, Abu Dhabi, United Arab Emirates (Hybrid), December 2022. https://aclanthology.org/2022.nllp-1.13
Paul, S., Mandal, A., Goyal, P., Ghosh, S.: Pre-training transformers on indian legal text. CoRR abs/2209.06049 (2022). https://doi.org/10.48550/arXiv.2209.06049
Reimers, N., Gurevych, I.: Sentence-bert: sentence embeddings using siamese bert-networks. In: Inui, K., Jiang, J., Ng, V., Wan, X. (eds.) Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, EMNLP-IJCNLP 2019, Hong Kong, China, November 3–7, 2019, pp. 3980–3990. Association for Computational Linguistics (2019). https://doi.org/10.18653/v1/D19-1410
Savelka, J., Westermann, H., Benyekhlef, K.: Cross-domain generalization and knowledge transfer in transformers trained on legal data. In: Ashley, K.D., Atkinson, K., Branting, L.K., Francesconi, E., Grabmair, M., Walker, V.R., Waltl, B., Wyner, A.Z. (eds.) Proceedings of the Fourth Workshop on Automated Semantic Analysis of Information in Legal Text held online in conjunction with the 33rd International Conference on Legal Knowledge and Information Systems, ASAIL@JURIX 2020, December 9, 2020. CEUR Workshop Proceedings, vol. 2764. CEUR-WS.org (2020). https://ceur-ws.org/Vol-2764/paper5.pdf
Ulmer, D., Hardmeier, C., Frellsen, J.: Deep-significance-easy and meaningful statistical significance testing in the age of neural networks. arXiv preprint arXiv:2204.06815 (2022)
Vaswani, A., et al.: Attention is all you need. In: Guyon, I., von Luxburg, U., Bengio, S., Wallach, H.M., Fergus, R., Vishwanathan, S.V.N., Garnett, R. (eds.) Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, December 4–9, 2017, Long Beach, CA, USA, pp. 5998–6008 (2017). https://proceedings.neurips.cc/paper/2017/hash/3f5ee243547dee91fbd053c1c4a845aa-Abstract.html
Wang, C., Nulty, P., Lillis, D.: A comparative study on word embeddings in deep learning for text classification. In: Proceedings of the 4th International Conference on Natural Language Processing and Information Retrieval, NLPIR 2020, pp. 37–46. Association for Computing Machinery, New York(2021). https://doi.org/10.1145/3443279.3443304
Yuan, K.H., Hayashi, K.: Bootstrap approach to inference and power analysis based on three test statistics for covariance structure models. Br. J. Math. Stat. Psychol. 56(1), 93–110 (2003). https://doi.org/10.1348/000711003321645368. https://bpspsychub.onlinelibrary.wiley.com/doi/abs/10.1348/000711003321645368
Zhang, H., Cissé, M., Dauphin, Y.N., Lopez-Paz, D.: mixup: beyond empirical risk minimization. In: 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 - May 3, 2018, Conference Track Proceedings. OpenReview.net (2018). https://openreview.net/forum?id=r1Ddp1-Rb
Zhong, H., Xiao, C., Tu, C., Zhang, T., Liu, Z., Sun, M.: How does NLP benefit legal system: a summary of legal artificial intelligence. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. pp. 5218–5230. Association for Computational Linguistics, Online, July 2020. https://doi.org/10.18653/v1/2020.acl-main.466. https://aclanthology.org/2020.acl-main.466
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
de Lima, A.G., Moreno, J.G., Dkaki, T., da S. Aranha, E.H., Boughanem, M. (2023). Evaluating Recent Legal Rhetorical Role Labeling Approaches Supported by Transformer Encoders. In: Naldi, M.C., Bianchi, R.A.C. (eds) Intelligent Systems. BRACIS 2023. Lecture Notes in Computer Science(), vol 14196. Springer, Cham. https://doi.org/10.1007/978-3-031-45389-2_2
Download citation
DOI: https://doi.org/10.1007/978-3-031-45389-2_2
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-45388-5
Online ISBN: 978-3-031-45389-2
eBook Packages: Computer ScienceComputer Science (R0)