Towards Secure Retrieval-Augmented Generation: Preventing LLM Data Leaks with RBAC

• Igor Martinelli USP
• Moacir A. Ponti USP

Resumo

Large language models (LLMs) have demonstrated remarkable skills in performing tasks that not so long ago were considered unfeasible. Despite their strong general performance, LLMs often lack in accuracy when dealing with domain-specific tasks, particularly due to susceptibility to hallucinations and reliance on outdated training data. A common solution is to use Retrieval Augmented Generation (RAG), which combines a retrieval step with the generative model to improve responses. However, this mechanism introduces additional risks of sensitive data leakage through both the retrieval and generation components. In this study, we implement a RAG framework and test common failure cases, especially attacks such as prompt injection. We explore different ways to reduce the risk of data leaks, such as choosing better models, prompt design, fine tuning, and filtering the retrieved documents. We show LLMs alone, even state-of-the-art ones, cannot handle security and privacy issues, even with fine-tuning or prompt engineering. The only reliable way to stop leaks is to control what the LLM sees. We propose a simple and effective retrieval filtering layer, based on role-based access control, to prevent sensitive data from ever reaching the model. This approach helps make RAG systems safer by design.