Evaluating RAG Strategies in a Modular LLM Architecture - Alboneti, Vinicius Aguiar; Amorim, Leonardo Afonso; Tomazini, Jonatas; Costa, Ricardo; Oliveira, Sávio Salvarino Teles de; Galvão Filho, Arlindo Rodrigues; Soares, Anderson da Silva; Figueiredo, Tales Brumon Medeiros de (Anais da Escola Regional de Informática de Goiás (ERI-GO), 2025)
This paper presents an empirical evaluation of three Retrieval-Augmented Generation (RAG) patterns — Naive RAG, RAG-Fusion, and Self-RAG — applied in a real-world corporate environment in the energy sector. Using a modular and consolidated architecture built on managed services and widely adopted technologies (such as pgvector, Cloud Run, and LLM APIs), we conducted experiments with actual data from an energy company, focusing on regulatory audit processes. We compared RAG strategies using automated evaluation metrics (RAGAS), considering faithfulness, contextual precision, answer relevance, and response time. The results show that the Self-RAG pattern achieves the best balance between response quality and performance, making it the most suitable for enterprise applications that require accuracy, efficiency, and scalability. This practical validation offers guidance on adopting RAG in corporate environments, highlighting the trade-offs associated with selecting different approaches.