Enhancing Tool Orchestration and Planning for Agentic RAG Systems: A Case Study

• Renato Miyaji Grupo Visagio https://orcid.org/0000-0002-7279-4546
• Vitor Bedin Grupo Visagio
• Renato Moulin Grupo Visagio
• Leonardo Machado Grupo Visagio

DOI: https://doi.org/10.5753/sbbd.2026.249641

Resumo

Traditional RAG faces bottlenecks in industrial environments. To address these limitations, this paper proposes an Agentic RAG architecture that decouples high-level strategic orchestration from technical execution through an LLM Supervisor and domain-specialized sub-agents. Using a proprietary dataset, we evaluate the efficacy of System 1 and System 2 reasoning paradigms alongside tool integration. Our experimental results demonstrate that ReWOO combined with Few-Shot execution guidance achieves a maximum success rate of 93.33%. Furthermore, we introduce an evaluation methodology which reveals that the ToolVerifier module is essential for complex planning. This study establishes that modular planning and robust verification are critical for reliability in knowledge-intensive industrial applications.

Palavras-chave: Retrieval-Augmented Generation, Large Language Models, Agentic Planning

Referências

Gao, S., Zhao, S., Jiang, X., Duan, L., Chng, Y. X., Chen, Q.-G., Luo, W., Zhang, K., Bian, J.-W., and Gong, M. (2025). Scaling beyond context: A survey of multimodal retrieval-augmented generation for document understanding. arXiv preprint arXiv:2510.15253.

Inaba, T., Kiyomaru, H., Cheng, F., and Kurohashi, S. (2023). Multitool-cot: Gpt-3 can use multiple external tools with chain of thought prompting. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (ACL 2023), pages 1522–1532. Association for Computational Linguistics.

Kahneman, D. (2011). Thinking, Fast and Slow. Farrar, Straus and Giroux.

Lin, Y., Cheng, Z., Zhao, A., Chen, S., Fu, G., Zhang, S., and Wu, D. (2023). Swiftsage: A modular agent with swift and sage components for adaptive and efficient action. In Advances in Neural Information Processing Systems (NeurIPS 2023).

Liu, Y., Iter, D., Xu, Y., Wang, S., Xu, R., and Zhu, C. (2023). G-eval: Nlg evaluation using gpt-4 with better alignment than human annotation. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing (EMNLP 2023), pages 10654–10671. Association for Computational Linguistics.

Madaan, A., Tandon, N., Gupta, P., Hallinan, S., Gao, L., Wiegreffe, W., Alon, U., Dziri, N., Prabhumoye, S., Yang, Y., Gupta, S., Majumder, B. P., Lapania, G., Welleck, S., and Bhadauria, T. (2023). Self-refine: Iterative refinement with self-feedback. In Advances in Neural Information Processing Systems (NeurIPS 2023).

Mekala, D., Weston, J., Lanchantin, J., Raileanu, R., Lomeli, M., Shang, J., and Dwivedi-Yu, J. (2024). Toolverifier: Generalization to new tools via self-verification. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 5026–5041, Miami, Florida, USA. Association for Computational Linguistics.

Shinn, N., Cassano, F., Berman, E., Gopinath, A., Narasimhan, K., and Yao, S. (2023). Reflexion: Language agents with verbal reinforcement learning. In Advances in Neural Information Processing Systems (NeurIPS 2023).

Sun, W., Yan, L., Ma, X., Wang, S., Ren, P., Chen, Z., Yin, D., and Ren, Z. (2023). Is chatgpt good at search? investigating large language models as re-ranking agents. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing (EMNLP 2023).

Wang, X., Wei, J., Schuurmans, D., Le, Q., Chi, E., Narang, S., Chowdhery, A., and Zhou, D. (2023). Self-consistency improves chain of thought reasoning in language models. In International Conference on Learning Representations (ICLR 2023).

Xu, B., Peng, Z., Lei, B., Mukherjee, S., Liu, Y., and Xu, D. (2023). Rewoo: Decoupling reasoning from observations for efficient augmented language models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing (EMNLP 2023).

Zheng, H. S., Mishra, S., Chen, X., Cheng, H.-T., Chi, E. H., Le, Q. V., and Zhou, D. (2024). Take a step back: Evoking reasoning via abstraction in large language models. In International Conference on Learning Representations (ICLR).