LEME: assistente RAG em português para normas brasileiras e internacionais sobre saúde e IA Responsável
Resumo
Health professionals, auditors, and developers must navigate scattered Brazilian and international regulations, often without legal expertise. We present LEME, a Portuguese-language RAG assistant over a bilingual corpus of Brazilian and international regulations and guidelines, answering by citing its sources or explicitly refusing when the curated corpus does not support an answer. LEME combines a corpus of 63 official sources, multilingual dense retrieval over a vector store, and a LangGraph agent that exposes retrieval as a tool. We also introduce a reproducible evaluation protocol based on a 100-question gold set anchored in the corpus. The evaluation showed comparable Portuguese/English retrieval without query translation, no gain from lexical-dense fusion, fewer severe hallucinations (from 23/100 to ≤4/100 answers), and correct refusals for out-of-scope questions increased from 18/25 to 22/25. The code, corpus manifest, gold set, and evaluation scripts are publicly available.
Palavras-chave:
AG, information retrieval, LLM-as-a-judge, Responsible AI in healthcare, healthcare legislation, CFM resolutions
Referências
Jianlyu Chen et al. 2024. M3-Embedding: Multi-Linguality, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge Distillation. In Proc. of ACL.
Pamela da Silva et al. 2025. Um agente conversacional para suporte emocional a pessoas com estados ansiosos. In Proc. of WFA.
Andrew Maranhão Ventura D’addario. 2025. HealthQA-BR: A System-Wide Benchmark Reveals Critical Knowledge Gaps in Large Language Models. arXiv preprint arXiv:2506.21578 (2025).
Pedro de Carvalho Neto. 2026. LegalBench-BR: A Benchmark for Evaluating Large Language Models on Brazilian Legal Decision Classification. arXiv preprint arXiv:2604.18878 (2026).
Luiz Fernandes et al. 2026. JurisTCU: a Brazilian Portuguese Information Retrieval Dataset with Query Relevance Judgments. Language Resources and Evaluation 60, 1 (2026), 23.
Francisco Junior et al. 2022. NextGISSA, um Sistema Inteligente para Monitoramento e Predição de Hipertensão Arterial em uma Plataforma de Saúde Digital. In Anais Estendidos do XXVIII Simpósio Brasileiro de Sistemas Multimídia e Web (WebMedia) – Workshop de Ferramentas e Aplicações (WFA). SBC, 119–122.
Juvenal Domingos Júnior et al. 2025. BR-TaxQA-R: A Dataset for Question Answering with References for Brazilian Personal Income Tax Law, Including Case Law. In Proc. of BRACIS.
Patrick Lewis et al. 2020. Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks. InAdvances in Neural Information Processing Systems, Vol. 33. Curran Associates, Inc.
Pedro Moreira et al. 2025. Bula Fácil: aplicação de bulário inteligente. In Anais Estendidos do XXXI Simpósio Brasileiro de Sistemas Multimídia e Web (WebMedia) - XXIV Workshop de Ferramentas e Aplicações (WFA). Sociedade Brasileira de Computação (SBC), 95–98.
Navapat Nananukul et al. 2026. ClinicBot: AGuideline-Grounded Clinical Chatbot with Prioritized Evidence RAG and Verifiable Citations. In Proc. of CAIS ’26. ACM.
Matheus Silva et al. 2025. EurecaChat: A Multi-Agent LLM-Based Chatbot for Supporting Students, Faculty, and University Administration. In Proc. of SBC WFA.
Elham Tabassi. 2023. Artificial Intelligence Risk Management Framework (AI RMF 1.0). Technical Report NIST AI 100-1. National Institute of Standards and Technology.
World Health Organization. 2021. Ethics and Governance of Artificial Intelligence for Health: WHO Guidance. Technical Report. WHO, Geneva.
H. Zhang et al. 2025. Shadow RAG Auditing Data Provenance. In Proceedings of the Association for Computational Linguistics (ACL).
Lianmin Zheng et al. 2023. Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena. In Proceedings of the 37th International Conference on Neural Information Processing Systems.
Pamela da Silva et al. 2025. Um agente conversacional para suporte emocional a pessoas com estados ansiosos. In Proc. of WFA.
Andrew Maranhão Ventura D’addario. 2025. HealthQA-BR: A System-Wide Benchmark Reveals Critical Knowledge Gaps in Large Language Models. arXiv preprint arXiv:2506.21578 (2025).
Pedro de Carvalho Neto. 2026. LegalBench-BR: A Benchmark for Evaluating Large Language Models on Brazilian Legal Decision Classification. arXiv preprint arXiv:2604.18878 (2026).
Luiz Fernandes et al. 2026. JurisTCU: a Brazilian Portuguese Information Retrieval Dataset with Query Relevance Judgments. Language Resources and Evaluation 60, 1 (2026), 23.
Francisco Junior et al. 2022. NextGISSA, um Sistema Inteligente para Monitoramento e Predição de Hipertensão Arterial em uma Plataforma de Saúde Digital. In Anais Estendidos do XXVIII Simpósio Brasileiro de Sistemas Multimídia e Web (WebMedia) – Workshop de Ferramentas e Aplicações (WFA). SBC, 119–122.
Juvenal Domingos Júnior et al. 2025. BR-TaxQA-R: A Dataset for Question Answering with References for Brazilian Personal Income Tax Law, Including Case Law. In Proc. of BRACIS.
Patrick Lewis et al. 2020. Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks. InAdvances in Neural Information Processing Systems, Vol. 33. Curran Associates, Inc.
Pedro Moreira et al. 2025. Bula Fácil: aplicação de bulário inteligente. In Anais Estendidos do XXXI Simpósio Brasileiro de Sistemas Multimídia e Web (WebMedia) - XXIV Workshop de Ferramentas e Aplicações (WFA). Sociedade Brasileira de Computação (SBC), 95–98.
Navapat Nananukul et al. 2026. ClinicBot: AGuideline-Grounded Clinical Chatbot with Prioritized Evidence RAG and Verifiable Citations. In Proc. of CAIS ’26. ACM.
Matheus Silva et al. 2025. EurecaChat: A Multi-Agent LLM-Based Chatbot for Supporting Students, Faculty, and University Administration. In Proc. of SBC WFA.
Elham Tabassi. 2023. Artificial Intelligence Risk Management Framework (AI RMF 1.0). Technical Report NIST AI 100-1. National Institute of Standards and Technology.
World Health Organization. 2021. Ethics and Governance of Artificial Intelligence for Health: WHO Guidance. Technical Report. WHO, Geneva.
H. Zhang et al. 2025. Shadow RAG Auditing Data Provenance. In Proceedings of the Association for Computational Linguistics (ACL).
Lianmin Zheng et al. 2023. Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena. In Proceedings of the 37th International Conference on Neural Information Processing Systems.
Publicado
27/10/2026
Como Citar
AZEVEDO, Karolina; AVELAR, Italo; CARVALHO, Matheus; LIMA, Helen Costa; CRISTINA, Isadora; VASCONCELOS, Marisa; BRANDÃO, Michele; MEIRA JR., Wagner.
LEME: assistente RAG em português para normas brasileiras e internacionais sobre saúde e IA Responsável. In: WORKSHOP DE FERRAMENTAS E APLICAÇÕES - SIMPÓSIO BRASILEIRO DE SISTEMAS MULTIMÍDIA E WEB (WEBMEDIA), 32. , 2026, Lavras/MG.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2026
.
p. 138-141.
ISSN 2596-1683.
DOI: https://doi.org/10.5753/webmedia_estendido.2026.34656.
