Evaluating Table Extraction Strategies for RAG Pipelines over Portuguese PDF Documents

• Danielle Tavares da Silva UFG
• Júlia Souza de Mendonça PUC Goiás
• Vitória Sofia Barbosa Pereira UFG
• Sávio Teles UFG

DOI: https://doi.org/10.5753/eniac.2026.31092

Resumo

Tabular data in PDFs challenges Retrieval-Augmented Generation (RAG) pipelines because standard extraction loses the row-column structure that gives tables their meaning. We study how table representation and retrieval affect end-to-end RAG over Portuguese occupational safety documents, crossing two extraction strategies, rule-based parsing (PDFplumber) and vision-based description by a multimodal LLM, with two retrieval strategies, dense vector search and a dense–sparse hybrid. We evaluate the resulting configurations across nine generator LLMs on a 50-pair manual benchmark and a 221-pair human-validated synthetic benchmark, scoring answer quality with a panel of five LLM judges from distinct provider families and retrieval quality against chunk-level ground truth. Contrary to the intuition that a lexical signal aids exact-match identifiers, dense retrieval outperforms the hybrid scheme across all configurations: in a lexically homogeneous procedural corpus, sparse matching cannot discriminate between candidate chunks and displaces the correct one. Retrieval is the binding constraint on performance, with unrecoverable failures concentrated on atomic operational identifiers; above this floor, generator verbosity governs outcomes, and vision-based descriptions improve quality over rule-based extraction downstream of retrieval.

Referências

Adhikari, N. S. and Agarwal, S. (2024). A comparative study of PDF parsing tools across diverse document categories. arXiv preprint arXiv:2410.09871.

Chen, S.-A., Miculicich, L., Eisenschlos, J. M., Wang, Z., Wang, Z., Chen, Y., Fujii, Y., Lin, H.-T., Lee, C.-Y., and Pfister, T. (2024). TableRAG: Million-token table understanding with language models. Advances in Neural Information Processing Systems, 37:74899–74921.

Cormack, G. V., Clarke, C. L., and Buettcher, S. (2009). Reciprocal rank fusion outperforms condorcet and individual rank learning methods. In Proceedings of the 32nd international ACM SIGIR conference on Research and development in information retrieval, pages 758–759.

Hoang, T. T. U., Rajendran, M., Zhang, K., Wu, Y., and Nguyen, V. A. (2025). PDF retrieval augmented question answering. arXiv preprint arXiv:2506.18027.

José, M. M., Cação, F. N., Ribeiro, M. F., Cheang, R. M., Pirozelli, P., and Cozman, F. G. (2024). Question answering with texts and tables through deep reinforcement learning. In Brazilian Conference on Intelligent Systems, pages 339–353. Springer.

Karpukhin, V., Oguz, B., Min, S., Lewis, P., Wu, L., Edunov, S., Chen, D., and Yih, W.-t. (2020). Dense passage retrieval for open-domain question answering. In Proceedings of the 2020 conference on empirical methods in natural language processing (EMNLP), pages 6769–6781.

Lewis, P., Perez, E., Piktus, A., Petroni, F., Karpukhin, V., Goyal, N., Küttler, H., Lewis, M., Yih, W.-t., Rocktäschel, T., et al. (2020). Retrieval-augmented generation for knowledge-intensive NLP tasks. Advances in neural information processing systems, 33:9459–9474.

Nunes, G., Rolla, V., Pereira, D., Alves, V., Carreiro, A., and Baptista, M. (2025). Benchmarking table extraction: Multimodal LLMs vs traditional OCR. In Proceedings of the 1st Joint Workshop on Large Language Models and Structure Modeling (XLLM 2025), pages 8–15.

Pires, R., Abonizio, H., Almeida, T. S., and Nogueira, R. (2023). Sabiá: Portuguese large language models. In Brazilian Conference on Intelligent Systems, pages 226–240. Springer.

Ribeiro, G. R., Saraiva, P. A., Rosa, L. H., Marques, E. L., Pereira, G. L., Campos, P. M., Pascoal, L. M., and de Oliveira, S. S. (2025). Vision-language models for automated property feature extraction in tax assessment: A case study. In Encontro Nacional de Inteligência Artificial e Computacional (ENIAC), pages 676–687. SBC.

Sobhan, S. and Haque, M. A. (2025). LLM-assisted question-answering on technical documents using structured data-aware retrieval augmented generation. arXiv preprint arXiv:2506.23136.

Souza, F., Nogueira, R., and Lotufo, R. (2020). BERTimbau: pretrained BERT models for Brazilian Portuguese. In Brazilian conference on intelligent systems, pages 403–417. Springer.

Sui, Y., Zhou, M., Zhou, M., Han, S., and Zhang, D. (2024). Table meets LLM: Can large language models understand structured table data? a benchmark and empirical study. In Proceedings of the 17th ACM International Conference on Web Search and Data Mining, pages 645–654.

Zheng, L., Chiang, W.-L., Sheng, Y., Zhuang, S., Wu, Z., Zhuang, Y., Lin, Z., Li, Z., Li, D., Xing, E., et al. (2023). Judging LLM-as-a-judge with MT-Bench and Chatbot Arena. Advances in neural information processing systems, 36:46595–46623.