Abordagem Hierárquica com BERT para Classificação de Documentos Jurídicos Longos: Uma Análise de Técnicas de Agregação de Características
Resumo
O estudo visa avaliar e implementar técnicas de agregação de características em modelos hierárquicos para lidar com textos longos no domínio jurídico. Modelos baseados em transformers, embora eficazes em vários problemas de Processamento de Linguagem Natural, enfrentam dificuldades devido à sua complexidade quadrática, especialmente com textos que excedem 512 tokens. Para superar esses desafios, o estudo propõe investigar métodos como modelos hierárquicos, estratégias de divisão de texto, atenção local e esparsa, com o objetivo de aprimorar o processamento e a análise desses textos, fornecendo insights valiosos para a compreensão e uso eficaz em contextos jurídicos.Referências
Chalkidis, I., Androutsopoulos, I., and Aletras, N. (2019). Neural legal judgment prediction in English. In Korhonen, A., Traum, D., and Màrquez, L., editors, Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 4317–4323, Florence, Italy. Association for Computational Linguistics.
Chalkidis, I., Fergadiotis, M., Tsarapatsanis, D., Aletras, N., Androutsopoulos, I., and Malakasiotis, P. (2021). Paragraph-level rationale extraction through regularization: A case study on european court of human rights cases. arXiv preprint arXiv:2103.13084.
Chalkidis, I., Jana, A., Hartung, D., Bommarito, M., Androutsopoulos, I., Katz, D., and Aletras, N. (2022). LexGLUE: A benchmark dataset for legal language understanding in English. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 4310–4330, Dublin, Ireland. Association for Computational Linguistics.
Chen, B., Sun, R., Dai, Y., Zheng, H.-T., and Zhang, R. (2022). Lordbert: Embedding long text by segment ordering with bert. In 2022 International Joint Conference on Neural Networks (IJCNN), pages 1–8. IEEE.
Conselho Nacional de Justiça (2023). Justiça em números 2023.
Dai, X., Chalkidis, I., Darkner, S., and Elliott, D. (2022). Revisiting transformer-based models for long document classification. arXiv preprint arXiv:2204.06683.
Devlin, J., Chang, M.-W., Lee, K., and Toutanova, K. (2018). Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv pre-print arXiv:1810.04805.
Jawahar, G., Sagot, B., and Seddah, D. (2019). What does bert learn about the structure of language? In ACL 2019-57th Annual Meeting of the Association for Computational Linguistics.
Spaeth, H. J., Epstein, L., Segal, J. A., Martin, A. D., Ruger, T. J., and Benesh, S. C. (2020). Supreme court database, version 2020 release 01. [link]. Accessed: 2024-06-02.
Tuteja, M. and Juclà, D. G. (2023). Long text classification using transformers with paragraph selection strategies. In Proceedings of the Natural Legal Language Processing Workshop 2023, pages 17–24.
Wagh, V., Khandve, S., Joshi, I., Wani, A., Kale, G., and Joshi, R. (2021). Comparative study of long document classification. In TENCON 2021-2021 IEEE Region 10 Conference (TENCON), pages 732–737. IEEE.
Chalkidis, I., Fergadiotis, M., Tsarapatsanis, D., Aletras, N., Androutsopoulos, I., and Malakasiotis, P. (2021). Paragraph-level rationale extraction through regularization: A case study on european court of human rights cases. arXiv preprint arXiv:2103.13084.
Chalkidis, I., Jana, A., Hartung, D., Bommarito, M., Androutsopoulos, I., Katz, D., and Aletras, N. (2022). LexGLUE: A benchmark dataset for legal language understanding in English. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 4310–4330, Dublin, Ireland. Association for Computational Linguistics.
Chen, B., Sun, R., Dai, Y., Zheng, H.-T., and Zhang, R. (2022). Lordbert: Embedding long text by segment ordering with bert. In 2022 International Joint Conference on Neural Networks (IJCNN), pages 1–8. IEEE.
Conselho Nacional de Justiça (2023). Justiça em números 2023.
Dai, X., Chalkidis, I., Darkner, S., and Elliott, D. (2022). Revisiting transformer-based models for long document classification. arXiv preprint arXiv:2204.06683.
Devlin, J., Chang, M.-W., Lee, K., and Toutanova, K. (2018). Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv pre-print arXiv:1810.04805.
Jawahar, G., Sagot, B., and Seddah, D. (2019). What does bert learn about the structure of language? In ACL 2019-57th Annual Meeting of the Association for Computational Linguistics.
Spaeth, H. J., Epstein, L., Segal, J. A., Martin, A. D., Ruger, T. J., and Benesh, S. C. (2020). Supreme court database, version 2020 release 01. [link]. Accessed: 2024-06-02.
Tuteja, M. and Juclà, D. G. (2023). Long text classification using transformers with paragraph selection strategies. In Proceedings of the Natural Legal Language Processing Workshop 2023, pages 17–24.
Wagh, V., Khandve, S., Joshi, I., Wani, A., Kale, G., and Joshi, R. (2021). Comparative study of long document classification. In TENCON 2021-2021 IEEE Region 10 Conference (TENCON), pages 732–737. IEEE.
Publicado
11/09/2024
Como Citar
SILVA, Jasson Carvalho da; RABELO, Ricardo Andrade Lira; LIMA, Weslley Emmanuel Martins; ALMEIDA, Vitor Augusto Correa Cortez.
Abordagem Hierárquica com BERT para Classificação de Documentos Jurídicos Longos: Uma Análise de Técnicas de Agregação de Características. In: ESCOLA REGIONAL DE COMPUTAÇÃO DO CEARÁ, MARANHÃO E PIAUÍ (ERCEMAPI), 12. , 2024, Parnaíba/PI.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2024
.
p. 80-89.
DOI: https://doi.org/10.5753/ercemapi.2024.243441.
