Evaluating Domain-Specialized LLMs in Multi-Agent RAG for Enterprise Retrieval - Aguiar, Vinícius A.; Amorim, Leonardo A.; Novais, Artur M. A.; Pereira, Gustavo L. B.; Oliveira, Tales de; Souza, Carlos A. de; G. Filho, Arlindo R.; Soares, Anderson S.; Oliveira, Sávio S. T. de (Anais do Simpósio Brasileiro de Tecnologia da Informação e da Linguagem Humana (STIL), 2025)
This paper evaluates a multi-agent architecture for enterprise knowledge retrieval where a semantic router directs queries to specialized LLM agents covering thematic domains (e.g., legal, regulatory). We benchmarked prominent models, including GPT-4, LLaMA 4, and Gemini, on metrics like answer relevance, faithfulness, and execution time. Our results demonstrate that this specialized approach achieves superior precision and lower latency compared to centralized configurations. We identify GPT-4o, LLaMA 4, and Gemini-2.5-Flash as offering the best balance of accuracy and efficiency. These findings provide a practical guide for designing scalable, high-fidelity retrieval systems for regulated, multi-domain environments.