Information Extraction from Financial Statements based on Visually Rich Document Models

• Elioenai L. G. Alves University of Fortaleza
• Cecília Carvalho University of Fortaleza
• Patrick Martins de Lima University of Fortaleza
• Vládia Pinheiro University of Fortaleza
• Vasco Furtado University of Fortaleza

DOI: https://doi.org/10.5753/eniac.2023.234520

Abstract

This paper presents an Information Extraction system for visually rich financial documents. The system takes pre-trained neural models from the LayoutXLM family and refines them for use in Financial Statements. Two post-processing steps were developed in order to adjust the results generated by the refined model. From empirical evaluations, it is concluded that the proposed system is effective in extracting information from financial documents and offers potential to automate and optimize the process of analysis and validation of financial statements.

Keywords: Visually rich documents, Information extraction, Structuring of information, Hierarchical information, Financial documents, Token classification

References

Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J. D., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020). Language models are few-shot learners. Advances in neural information processing systems, 33:1877–1901.

Cho, S., Moon, J., Bae, J., Kang, J., and Lee, S. (2023). A framework for understanding unstructured financial documents using rpa and multimodal approach. Electronics, 12(4):939.

DAIR.AI (2023). Few-shot prompting. Disponível em: [link]. Acesso em: 29 de Junho 2023.

Déjean, H., Clinchant, S., and Meunier, J.-L. (2022). Layoutxlm vs. gnn: An empirical evaluation of relation extraction for documents. arXiv preprint arXiv:2206.10304.

Hooda, N., Bawa, S., and Rana, P. S. (2018). Fraudulent firm classification: a case study of an external audit. Applied Artificial Intelligence, 32(1):48–64.

Keocheguerian, I. B. and Martins, V. F. (2021). A utilização da inteligência artificial nos trabalhos de auditoria independente. Revista Científica e-Locução, 1(20):21–21.

Sarkhel, R. and Nandi, A. (2019). Visual segmentation for information extraction from heterogeneous visually rich documents. In Proceedings of the 2019 international conference on management of data, pages 247–262.

Stubblebine, T. (2003). Regular expression pocket reference. ”O’Reilly Media, Inc.”.

Wang, J., Jin, L., and Ding, K. (2022). Lilt: A simple yet effective language-independent layout transformer for structured document understanding. arXiv preprint arXiv:2202.13669.

Wu, Y., Schuster, M., Chen, Z., Le, Q. V., Norouzi, M., Macherey, W., Krikun, M., Cao, Y., Gao, Q., Macherey, K., et al. (2016). Google’s neural machine translation system: Bridging the gap between human and machine translation. arXiv preprint arXiv:1609.08144.

Xie, S., Girshick, R., Dollár, P., Tu, Z., and He, K. (2017). Aggregated residual transformations for deep neural networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1492–1500.

Xu, Y., Li, M., Cui, L., Huang, S., Wei, F., and Zhou, M. (2020a). Layoutlm: Pre-training of text and layout for document image understanding. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pages 1192–1200.

Xu, Y., Lv, T., Cui, L., Wang, G., Lu, Y., Florencio, D., Zhang, C., and Wei, F. (2021). Layoutxlm: Multimodal pre-training for multilingual visually-rich document understanding. arXiv preprint arXiv:2104.08836.

Xu, Y., Xu, Y., Lv, T., Cui, L., Wei, F., Wang, G., Lu, Y., Florencio, D., Zhang, C., Che, W., et al. (2020b). Layoutlmv2: Multi-modal pre-training for visually-rich document understanding. arXiv preprint arXiv:2012.14740.

Ylisiurunen, M. et al. (2022). Extracting semi-structured information from receipts.

Yu, W., Lu, N., Qi, X., Gong, P., and Xiao, R. (2021). Pick: processing key information extraction from documents using improved graph learning-convolutional networks. In 2020 25th International Conference on Pattern Recognition (ICPR), pages 4363–4370. IEEE.