Quantifying the RAG Advantage: A Multi-Metric Benchmark for LLM-based Code Generation - Baggio, Gabriel Souza; Lunardi, Gabriel Machado; Machado, Guilherme Medeiros; Oliveira, José Palazzo Moreira de (Anais do Simpósio Brasileiro de Banco de Dados (SBBD); 2025: Anais do XL Simpósio Brasileiro de Bancos de Dados, 2025)
The recent advancement of Large Language Models (LLMs) has demonstrated remarkable capabilities in solving programming challenges. However, despite their proficiency, LLMs often suffer from hallucination and limited performance on unfamiliar or complex tasks. Retrieval-Augmented Generation (RAG) has emerged as a promising solution to address these limitations by supplementing prompts with relevant external information. In this paper, we propose a benchmark to assess the efficacy of RAG in solving algorithmic problems by integrating a curated database of 120 LeetCode problems, each paired with corresponding solutions and explanations. An Information Retrieval (IR) system was employed to construct enhanced prompts for solving novel problems.