Granularity Reciprocal Rank Fusion: Fusing Multiple Chunk Granularities for Robust Retrieval-Augmented Generation

• Jonatas Novais Tomazini UFG
• Salvador Ludovico Paranhos UFG
• Celso Camilo Junior UFG
• Sávio Teles de Oliveira UFG

DOI: https://doi.org/10.5753/eniac.2026.31548

Resumo

Retrieval-Augmented Generation (RAG) systems depend heavily on how textual data is segmented and retrieved. We introduce Granularity Reciprocal Rank Fusion (GRRF), a novel approach for optimizing context selection in RAG pipelines by leveraging multiple levels of text granularity. GRRF indexes the same corpus under different chunking granularities and reranks the retrieved passages using a supervised cross-encoder. The results are then fused through the Reciprocal Rank Fusion (RRF) technique, enabling robust selection of the most informative content. Experiments on legal, news, and audit corpora demonstrate that GRRF’s central, model-agnostic finding is that no single chunking granularity is optimal across domains—smaller chunks win in legal and news, and larger chunks in audit—a result read directly from the baselines and therefore independent of any reranking. GRRF is designed around this finding: instead of committing to one fragile granularity, it fuses evidence from all of them, and in our experiments it consistently surpasses strong single-pass baselines (up to 5.7% relative gain in context quality). These findings highlight the effectiveness of granularity-aware fusion for enhancing both precision and robustness in retrieval for large language models.

Referências

Auer, C., Lysak, M., Nassar, A., Dolfi, M., Livathinos, N., Vagenas, P., Berrospi Ramis, C., Omenetti, M., Lindlbauer, F., Dinkla, K., Mishra, L., Kim, Y., Gupta, S., Teixeira de Lima, R., Weber, V., Morin, L., Meijer, I., Kuropiatnyk, V., and Staar, P. W. J. (2024). Docling technical report.

Bhat, S. R., Rudat, M., Spiekermann, J., and Flores-Herr, N. (2025). Rethinking chunk size for long-document retrieval: A multi-dataset analysis.

Chen, J., Xiao, S., Zhang, P., Luo, K., Lian, D., and Liu, Z. (2024). M3-embedding: Multi-linguality, multi-functionality, multi-granularity text embeddings through self-knowledge distillation. In Findings of the Association for Computational Linguistics: ACL 2024, pages 2318–2335, Bangkok, Thailand. Association for Computational Linguistics.

Chen, W., Bai, T., Su, J., Luan, J., Liu, W., and Shi, C. (2025). Kg-retriever: Efficient knowledge indexing for retrieval-augmented large language models.

Cormack, G. V., Clarke, C. L. A., and Büttcher, S. (2009). Reciprocal rank fusion outperforms condorcet and individual rank learning methods. In Proceedings of the 32nd International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR ’09, pages 758–759, New York, NY, USA. Association for Computing Machinery.

Dong, Z., Ni, J., Bikel, D., Alfonseca, E., Wang, Y., Qu, C., and Zitouni, I. (2022). Exploring dual encoder architectures for question answering. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 9414–9419, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.

Es, S., James, J., Espinosa Anke, L., and Schockaert, S. (2024). RAGAs: Automated evaluation of retrieval augmented generation. In Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics: System Demonstrations, pages 150–158, St. Julians, Malta. Association for Computational Linguistics.

Han, S. (2025). Finding the best chunking strategy for accurate ai responses. NVIDIA Technical Blog. Accessed: 2026-05-04.

Jégou, H., Douze, M., and Schmid, C. (2011). Product quantization for nearest neighbor search. IEEE Transactions on Pattern Analysis and Machine Intelligence, 33(1):117–128.

Kane, A. (2024). pgvector: Open-source vector similarity search for Postgres. Software. Accessed: 2025-08-19.

Karpukhin, V., Oguz, B., Min, S., Lewis, P., Wu, L., Edunov, S., Chen, D., and Yih, W.-t. (2020). Dense passage retrieval for open-domain question answering. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 6769–6781, Online. Association for Computational Linguistics.

LangChain (2024). LangChain: A framework for developing applications powered by language models. Software. Accessed: 2025-08-19.

Lewis, P., Perez, E., Piktus, A., Petroni, F., Karpukhin, V., Goyal, N., Küttler, H., Lewis, M., Yih, W.-t., Rocktäschel, T., Riedel, S., and Kiela, D. (2020). Retrieval-augmented generation for knowledge-intensive NLP tasks. In Advances in Neural Information Processing Systems, volume 33, pages 9459–9474. Curran Associates, Inc.

Lu, W., Chen, K., Qiao, R., and Sun, X. (2025). HiChunk: Evaluating and enhancing retrieval-augmented generation with hierarchical chunking.

Malkov, Y. A. and Yashunin, D. A. (2020). Efficient and robust approximate nearest neighbor search using hierarchical navigable small world graphs. IEEE Transactions on Pattern Analysis and Machine Intelligence, 42(4):824–836.

Nguyen, H. T., Nguyen, T. D., and Nguyen, V. H. (2024). Enhancing retrieval augmented generation with hierarchical text segmentation chunking. In International Symposium on Information and Communication Technology (SOICT 2024), Communications in Computer and Information Science, pages 209–220, Singapore. Springer Nature Singapore.

Rackauckas, Z. (2024). RAG-Fusion: A new take on retrieval-augmented generation. International Journal on Natural Language Computing, 13(1):37–47.

Santana, M. (2017). News of the Brazilian newspaper. Kaggle dataset. Accessed: 2025-08-19.

Sarthi, P., Abdullah, S., Tuli, A., Khanna, S., Goldie, A., and Manning, C. D. (2024). RAPTOR: Recursive abstractive processing for tree-organized retrieval. In The Twelfth International Conference on Learning Representations (ICLR).

Schwaber-Cohen, R. and Patel, A. (2025). Chunking strategies for LLM applications. Pinecone Learn. Accessed: 2026-05-04.

Tao, W., Xing, X., Chen, Y., Huang, L., and Xu, X. (2025). TreeRAG: Unleashing the power of hierarchical storage for enhanced knowledge retrieval in long documents. In Findings of the Association for Computational Linguistics: ACL 2025, pages 356–371, Vienna, Austria. Association for Computational Linguistics.

Toresan, B. R., Moreira, V. P., Paula, F. S. F., and Bencke, L. R. (2025). Advanced chunking techniques: a novel approach for semantic splitters. In Anais do XL Simpósio Brasileiro de Banco de Dados (SBBD), pages 455–465, Fortaleza, Brazil. SBC.

Tribunal de Contas dos Municípios do Estado de Goiás (2024). Portal de auditorias. Accessed: 2025-08-19.

Wang, W., Wei, F., Dong, L., Bao, H., Yang, N., and Zhou, M. (2020). MiniLM: Deep self-attention distillation for task-agnostic compression of pre-trained transformers. In Advances in Neural Information Processing Systems 33 (NeurIPS 2020), Red Hook, NY, USA. Curran Associates Inc.

Zhong, Z., Liu, H., Cui, X., Zhang, X., and Qin, Z. (2025). Mix-of-granularity: Optimize the chunking granularity for retrieval-augmented generation. In Proceedings of the 31st International Conference on Computational Linguistics, pages 5756–5774, Abu Dhabi, UAE. Association for Computational Linguistics.