Chunking Strategies for Retrieval in Enterprise Documents: A Comparative Study and the QI-RAG Proposal

• Maryane de Castro Lima IFCE
• Thomaz Maia de Almeida IFCE

DOI: https://doi.org/10.5753/eniac.2026.31065

Resumo

Chunking is a critical yet underexplored step in enterprise RAG pipelines. We compare four strategies (character-based, recursive, semantic, hierarchical) on 500 queries from financial documents (VDR-IBM REAL-MM-RAG) using BGE-M3 and Nomic embeddings with LLM-as-a-judge metrics. We also propose Question-Indexed RAG (QI-RAG), where a Gemma 3 LLM generates diverse questions per chunk for vector indexing, with threshold-based hybrid retrieval. Baselines show clear progression (53.6%→ 75.5% contextual precision). QI-RAG reaches 83.1% precision and 0.82 accuracy, cutting failures to 2.4% (14.3× below character-based).

Referências

Asai, A., Wu, Z., Wang, Y., Sil, A., and Hajishirzi, H. (2024). Self-RAG: Learning to retrieve, generate, and critique through self-reflection. In International Conference on Learning Representations (ICLR).

Bennani, S. and Moslonka, C. (2026). A systematic analysis of chunking strategies for reliable question answering. arXiv preprint arXiv:2601.14123.

Bruckhaus, T. (2024). RAG does not work for enterprises. arXiv preprint arXiv:2406.04369.

Chen, J., Xiao, S., Zhang, P., Luo, K., Lian, D., and Liu, Z. (2024). BGE M3-Embedding: Multi-lingual, multi-functionality, multi-granularity text embeddings through self-knowledge distillation. In Findings of the Association for Computational Linguistics: ACL 2024.

Confident AI (2024). Deepeval: An open-source evaluation framework for LLM systems. Documentation.

Cunha, M. V. d., Silveira, M. R., Santana, B. S., Freitas, L. A., and Corrêa, U. B. (2025). Optimizing and evaluating a retrieval-augmented generation system for normative document retrieval in hospital settings. In Anais do XXXI Simpósio Brasileiro de Sistemas Multimídia e Web (WebMedia 2025).

Eponon, A. A., Shahiki-Tash, M., Batyrshin, I., Maldonado-Sifuentes, C. E., Sidorov, G., and Gelbukh, A. (2025). Knowledge compression via question generation: Enhancing multihop document retrieval without fine-tuning. arXiv preprint arXiv:2506.13778.

Es, S., James, J., Espinosa-Anke, L., and Schockaert, S. (2024). RAGAS: Automated evaluation of retrieval augmented generation. In Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (EACL 2024): System Demonstrations, pages 150–158.

Gao, L., Ma, X., Lin, J., and Callan, J. (2023). Precise zero-shot dense retrieval without relevance labels. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (ACL).

Gao, Y., Xiong, Y., Gao, X., Jia, K., Pan, J., Bi, Y., Dai, Y., Sun, J., Wang, M., and Wang, H. (2024). Retrieval-augmented generation for large language models: A survey. arXiv preprint arXiv:2312.10997.

Google DeepMind (2025). Gemma 3 technical report. arXiv preprint arXiv:2503.19786.

IBM Research (2024). Docling: An efficient open-source toolkit for AI-ready document conversion. [link].

Karpukhin, V., Oguz, B., Min, S., Lewis, P., Wu, L., Edunov, S., Chen, D., and Yih, W. (2020). Dense passage retrieval for open-domain question answering. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP).

Lee, S. (2025). Transforming questions and documents for semantically aligned retrieval-augmented generation. arXiv preprint arXiv:2508.09755.

Lewis, P., Perez, E., Piktus, A., Petroni, F., Karpukhin, V., Goyal, N., Küttler, H., Lewis, M., Yih, W., Rocktäschel, T., Riedel, S., and Kiela, D. (2020). Retrieval-augmented generation for knowledge-intensive NLP tasks. Advances in Neural Information Processing Systems (NeurIPS).

Nussbaum, Z., Morris, J. X., Duderstadt, B., and Mulyar, A. (2024). Nomic embed: Training a reproducible long context text embedder. arXiv preprint arXiv:2402.01613.

Qu, R., Tu, R., and Bao, F. (2025). Is semantic chunking worth the computational cost? In Findings of the Association for Computational Linguistics: NAACL 2025.

Racine AI (2024). VDR-IBM REAL-MM-RAG: A multimodal retrieval-augmented generation dataset for financial documents. [link].

Raina, V. and Gales, M. (2024). Question-based retrieval using atomic units for enterprise RAG. arXiv preprint arXiv:2405.12363.

Sarmah, B., Hall, B., Rao, R., Patel, S., Pasquali, S., and Mehta, D. (2024). HybridRAG: Integrating knowledge graphs and vector retrieval augmented generation for efficient information extraction. arXiv preprint arXiv:2408.04948.

Sarthi, P., Abdullah, S., Tuli, A., Khanna, S., Goldie, A., and Manning, C. D. (2024). RAPTOR: Recursive abstractive processing for tree-organized retrieval. In International Conference on Learning Representations (ICLR).

Taiwo, S. and Yusoff, M. A. (2026). Evaluating chunking strategies for retrieval-augmented generation in oil and gas enterprise documents. arXiv preprint arXiv:2603.24556.

Toresan, B. R., Moreira, V. P., Paula, F. S. F., and Bencke, L. R. (2025). Advanced chunking techniques: A novel approach for semantic splitters. In Anais do XL Simpósio Brasileiro de Bancos de Dados (SBBD 2025).

Wang, X., Wang, Z., Gao, X., Zhang, F., Wu, Y., Xu, Z., Shi, T., Wang, Z., Li, S., Qian, Q., et al. (2024). Searching for best practices in retrieval-augmented generation. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing (EMNLP).

Wang, Z., Gao, C., Xiao, C., Huang, Y., Si, S., Luo, K., Bai, Y., Li, W., Duan, T., Lv, C., Lu, G., Chen, G., Qi, F., and Sun, M. (2025). Document segmentation matters for retrieval-augmented generation. In Findings of the Association for Computational Linguistics: ACL 2025.

Zheng, L., Chiang, W.-L., Sheng, Y., Zhuang, S., Wu, Z., Zhuang, Y., Lin, Z., Li, Z., Li, D., Xing, E. P., Zhang, H., Gonzalez, J. E., and Stoica, I. (2023). Judging LLM-as-a-judge with MT-Bench and chatbot arena. In Advances in Neural Information Processing Systems (NeurIPS 2023).