MulSHEP: MultiStage Health Extraction Pipeline LLM for Brazilian EHR Data
Resumo
This work presents the MultiStage Health Extraction Pipeline (MulSHEP) for extracting structured FHIR clinical data from electronic health records (EHR). MulSHEP is composed of two steps: a Semantic Block Separation and an Interpretation and Data Extraction, which break the EHR data into semantic blocks and a few-shot extraction prompt. We compare MulSHEP’s performance with multiple LLM models. Our results extracting EHR data from multiple health institutions from Ceará, Brazil (107 real EHR and over 47k annotated features), show a promising result, as some LLMs, namely the Gemini-3.1-flash-lite and GPT-OSS-120b, improved their weighted recall by a factor of 1.53 and 1.38, especially for longer documents, compared to a single-step baseline.Referências
Agarwal, S., Ahmad, L., Ai, J., Altman, S., Applebaum, A., Arbus, E., Arora, R. K., Bai, Y., Baker, B., Bao, H., et al. (2025). gpt-oss-120b & gpt-oss-20b model card. arXiv preprint arXiv:2508.10925.
Agrawal, M., Hegselmann, S., Lang, H., Kim, Y., and Sontag, D. (2022). Large language models are few-shot clinical information extractors. In Proceedings of the 2022 conference on empirical methods in natural language processing, pages 1998–2022.
Alkhalaf, M., Yu, P., Yin, M., and Deng, C. (2024). Applying generative ai with retrieval augmented generation to summarize and extract key clinical information from electronic health records. Journal of biomedical informatics, 156:104662.
Almeida, F. and Caminha, C. (2024). Evaluation of entry-level open-source large language models for information extraction from digitized documents. In Anais do XII Symposium on Knowledge Discovery, Mining and Learning, pages 25–32, Porto Alegre, RS, Brasil. SBC.
Almeida, T. S., Abonizio, H., Nogueira, R., and Pires, R. (2024). Sabiá-2: A new generation of portuguese large language models. arXiv preprint arXiv:2403.09887.
Analysis, A. (2026). Artificial analysis intelligence index v4.0. Independent benchmark suite for assessing AI model intelligence. Accessed: 2026-01-07.
Ayaz, M., Pasha, M. F., Alzahrani, M. Y., Budiarto, R., and Stiawan, D. (2021). The fast health interoperability resources (fhir) standard: systematic literature review of implementations, applications, challenges and opportunities. JMIR medical informatics, 9(7):e21929.
Espinosa, L., Kebaili, D., Consoli, S., Kalimeri, K., Mejova, Y., and Salathé, M. (2025). Open-source solution for evaluation and benchmarking of large language models for public health. medRxiv, pages 2025–03.
Garcia-Carmona, A. M., Prieto, M.-L., Puertas, E., and Beunza, J.-J. (2025). Leveraging large language models for accurate retrieval of patient information from medical reports: Systematic evaluation study. JMIR AI, 4(1):e68776.
Guimaraes, N. C., Mello, F. C., Fernandes, T. J., Campelo, L. F., Teodoro, J. G., and Soares, F. A. (2025). An intelligent system based on llms for automated healthcare data collection. In Encontro Nacional de Inteligência Artificial e Computacional (ENIAC), pages 1984–1995. SBC.
Hoerbst, A. and Ammenwerth, E. (2010). Electronic health records. Methods of information in medicine, 49(04):320–336.
Karl, A. L., Fernandes, G. S., Pires, L. A., Serpa, Y. R., and Caminha, C. (2024). Synthetic AI data pipeline for domain-specific speech-to-text solutions. In Simpósio Brasileiro de Tecnologia da Informação e da Linguagem Humana (STIL), pages 37–47. SBC.
Lima, J. V. R. J., Pinheiro, V., and Caminha, C. (2026). Portuguese sentiment analysis with open-source LLMs: Models, prompts, and efficient deployment. In Souza, M., de Dios-Flores, I., Santos, D., Freitas, L., Souza, J. W. d. C., and Ribeiro, E., editors, Proceedings of the 17th International Conference on Computational Processing of Portuguese (PROPOR 2026) - Vol. 1, pages 212–221, Salvador, Brazil. Association for Computational Linguistics.
Liu, N. F., Lin, K., Hewitt, J., Paranjape, A., Bevilacqua, M., Petroni, F., and Liang, P. (2024). Lost in the middle: How language models use long contexts. Transactions of the Association for Computational Linguistics, 12:157–173.
Masayoshi, K., Hashimoto, M., Yokoyama, R., Toda, N., Uwamino, Y., Fukuda, S., Namkoong, H., and Jinzaki, M. (2025). Ehr-mcp: Real-world evaluation of clinical information retrieval by large language models via model context protocol. arXiv preprint arXiv:2509.15957.
Ntinopoulos, V., Biefer, H. R. C., Tudorache, I., Papadopoulos, N., Odavic, D., Risteski, P., Haeussler, A., and Dzemali, O. (2025). Large language models for data extraction from unstructured and semi-structured electronic health records: a multiple model performance evaluation. BMJ health & care informatics, 32(1):e101139.
Paiola, P. H., Garcia, G. L., Manesco, J. R. R., Roder, M., Rodrigues, D., and Papa, J. P. (2024). Adapting llms for the medical domain in portuguese: A study on fine-tuning and model evaluation. arXiv preprint arXiv:2410.00163.
Poser, P. L., Klimas, R., Luerweg, J., Reuter, E., Hanefeld, C., Gold, R., Salmen, A., and Motte, J. (2026). Improving reliability and accuracy of structured data extraction using a consensus large-language model approach–a use case description in multiple sclerosis. Frontiers in Artificial Intelligence, 9:1658575.
Prybutok, V. R. and Prybutok, G. L. (2025). Data-driven insights in healthcare. In Healthcare, volume 13, page 2658. MDPI.
Saberi, M. A., Mcheick, H., and Adda, M. (2025). From data silos to health records without borders: a systematic survey on patient-centered data interoperability. Information, 16(2):106.
Santos, W. C., Juiz, G. T., and Junior, A. C. (2025). Optimizing resume data extraction with small language models: A comparative study on efficiency and privacy. In Encontro Nacional de Inteligência Artificial e Computacional (ENIAC), pages 1128–1137. SBC.
Schmiedmayer, P., Rao, A., Zagar, P., Ravi, V., Zahedivash, A., Fereydooni, A., and Aalami, O. (2024). Llm on fhir–demystifying health records. arXiv preprint arXiv:2402.01711.
Shimaoka, A. M., da Silva Junior, A. C., Duarte, J. M., da Silva Costa, T. B., Pisa, I. T., Lopes, L. R., and Bandiera-Paiva, P. (2026). Large language models in portuguese for healthcare: a systematic review. Research on Biomedical Engineering, 42(1):13.
Sivarajkumar, S. and Wang, Y. (2023). Healthprompt: a zero-shot learning paradigm for clinical natural language processing. In AMIA Annual Symposium Proceedings, volume 2022, page 972.
Sushil, M., Zack, T., Mandair, D., Zheng, Z., Wali, A., Yu, Y.-N., Quan, Y., Lituiev, D., and Butte, A. J. (2024). A comparative study of large language model-based zero-shot inference and task-specific supervised classification of breast cancer pathology reports. Journal of the American Medical Informatics Association, 31(10):2315–2327.
Utino, M. Y. R., Matsushima, E. H., Paes, A., and Mann, P. (2025). Synthetic data for mental health: A comparative analysis of llms, bert, and copy-based augmentation. In Brazilian Conference on Intelligent Systems, pages 393–408. Springer.
Agrawal, M., Hegselmann, S., Lang, H., Kim, Y., and Sontag, D. (2022). Large language models are few-shot clinical information extractors. In Proceedings of the 2022 conference on empirical methods in natural language processing, pages 1998–2022.
Alkhalaf, M., Yu, P., Yin, M., and Deng, C. (2024). Applying generative ai with retrieval augmented generation to summarize and extract key clinical information from electronic health records. Journal of biomedical informatics, 156:104662.
Almeida, F. and Caminha, C. (2024). Evaluation of entry-level open-source large language models for information extraction from digitized documents. In Anais do XII Symposium on Knowledge Discovery, Mining and Learning, pages 25–32, Porto Alegre, RS, Brasil. SBC.
Almeida, T. S., Abonizio, H., Nogueira, R., and Pires, R. (2024). Sabiá-2: A new generation of portuguese large language models. arXiv preprint arXiv:2403.09887.
Analysis, A. (2026). Artificial analysis intelligence index v4.0. Independent benchmark suite for assessing AI model intelligence. Accessed: 2026-01-07.
Ayaz, M., Pasha, M. F., Alzahrani, M. Y., Budiarto, R., and Stiawan, D. (2021). The fast health interoperability resources (fhir) standard: systematic literature review of implementations, applications, challenges and opportunities. JMIR medical informatics, 9(7):e21929.
Espinosa, L., Kebaili, D., Consoli, S., Kalimeri, K., Mejova, Y., and Salathé, M. (2025). Open-source solution for evaluation and benchmarking of large language models for public health. medRxiv, pages 2025–03.
Garcia-Carmona, A. M., Prieto, M.-L., Puertas, E., and Beunza, J.-J. (2025). Leveraging large language models for accurate retrieval of patient information from medical reports: Systematic evaluation study. JMIR AI, 4(1):e68776.
Guimaraes, N. C., Mello, F. C., Fernandes, T. J., Campelo, L. F., Teodoro, J. G., and Soares, F. A. (2025). An intelligent system based on llms for automated healthcare data collection. In Encontro Nacional de Inteligência Artificial e Computacional (ENIAC), pages 1984–1995. SBC.
Hoerbst, A. and Ammenwerth, E. (2010). Electronic health records. Methods of information in medicine, 49(04):320–336.
Karl, A. L., Fernandes, G. S., Pires, L. A., Serpa, Y. R., and Caminha, C. (2024). Synthetic AI data pipeline for domain-specific speech-to-text solutions. In Simpósio Brasileiro de Tecnologia da Informação e da Linguagem Humana (STIL), pages 37–47. SBC.
Lima, J. V. R. J., Pinheiro, V., and Caminha, C. (2026). Portuguese sentiment analysis with open-source LLMs: Models, prompts, and efficient deployment. In Souza, M., de Dios-Flores, I., Santos, D., Freitas, L., Souza, J. W. d. C., and Ribeiro, E., editors, Proceedings of the 17th International Conference on Computational Processing of Portuguese (PROPOR 2026) - Vol. 1, pages 212–221, Salvador, Brazil. Association for Computational Linguistics.
Liu, N. F., Lin, K., Hewitt, J., Paranjape, A., Bevilacqua, M., Petroni, F., and Liang, P. (2024). Lost in the middle: How language models use long contexts. Transactions of the Association for Computational Linguistics, 12:157–173.
Masayoshi, K., Hashimoto, M., Yokoyama, R., Toda, N., Uwamino, Y., Fukuda, S., Namkoong, H., and Jinzaki, M. (2025). Ehr-mcp: Real-world evaluation of clinical information retrieval by large language models via model context protocol. arXiv preprint arXiv:2509.15957.
Ntinopoulos, V., Biefer, H. R. C., Tudorache, I., Papadopoulos, N., Odavic, D., Risteski, P., Haeussler, A., and Dzemali, O. (2025). Large language models for data extraction from unstructured and semi-structured electronic health records: a multiple model performance evaluation. BMJ health & care informatics, 32(1):e101139.
Paiola, P. H., Garcia, G. L., Manesco, J. R. R., Roder, M., Rodrigues, D., and Papa, J. P. (2024). Adapting llms for the medical domain in portuguese: A study on fine-tuning and model evaluation. arXiv preprint arXiv:2410.00163.
Poser, P. L., Klimas, R., Luerweg, J., Reuter, E., Hanefeld, C., Gold, R., Salmen, A., and Motte, J. (2026). Improving reliability and accuracy of structured data extraction using a consensus large-language model approach–a use case description in multiple sclerosis. Frontiers in Artificial Intelligence, 9:1658575.
Prybutok, V. R. and Prybutok, G. L. (2025). Data-driven insights in healthcare. In Healthcare, volume 13, page 2658. MDPI.
Saberi, M. A., Mcheick, H., and Adda, M. (2025). From data silos to health records without borders: a systematic survey on patient-centered data interoperability. Information, 16(2):106.
Santos, W. C., Juiz, G. T., and Junior, A. C. (2025). Optimizing resume data extraction with small language models: A comparative study on efficiency and privacy. In Encontro Nacional de Inteligência Artificial e Computacional (ENIAC), pages 1128–1137. SBC.
Schmiedmayer, P., Rao, A., Zagar, P., Ravi, V., Zahedivash, A., Fereydooni, A., and Aalami, O. (2024). Llm on fhir–demystifying health records. arXiv preprint arXiv:2402.01711.
Shimaoka, A. M., da Silva Junior, A. C., Duarte, J. M., da Silva Costa, T. B., Pisa, I. T., Lopes, L. R., and Bandiera-Paiva, P. (2026). Large language models in portuguese for healthcare: a systematic review. Research on Biomedical Engineering, 42(1):13.
Sivarajkumar, S. and Wang, Y. (2023). Healthprompt: a zero-shot learning paradigm for clinical natural language processing. In AMIA Annual Symposium Proceedings, volume 2022, page 972.
Sushil, M., Zack, T., Mandair, D., Zheng, Z., Wali, A., Yu, Y.-N., Quan, Y., Lituiev, D., and Butte, A. J. (2024). A comparative study of large language model-based zero-shot inference and task-specific supervised classification of breast cancer pathology reports. Journal of the American Medical Informatics Association, 31(10):2315–2327.
Utino, M. Y. R., Matsushima, E. H., Paes, A., and Mann, P. (2025). Synthetic data for mental health: A comparative analysis of llms, bert, and copy-based augmentation. In Brazilian Conference on Intelligent Systems, pages 393–408. Springer.
Publicado
19/10/2026
Como Citar
KARL, Anderson Luiz; SERPA, Yvens R.; PIRES, Leonardo Augusto; CAMINHA, Carlos.
MulSHEP: MultiStage Health Extraction Pipeline LLM for Brazilian EHR Data. In: ENCONTRO NACIONAL DE INTELIGÊNCIA ARTIFICIAL E COMPUTACIONAL (ENIAC), 23. , 2026, Cuiabá/MT.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2026
.
p. 885-896.
ISSN 2763-9061.
DOI: https://doi.org/10.5753/eniac.2026.30608.
