Assessing Large Language Models for Structuring Patient Records

• Eduardo M. Campos Universidade de São Paulo (USP)
• Rafael C. G. Conrado Universidade de São Paulo (USP)
• Caetano Traina Jr. Universidade de São Paulo (USP) https://orcid.org/0000-0002-6625-6047
• Agma J. M. Traina Universidade de São Paulo (USP) https://orcid.org/0000-0003-4929-7258
• Mirela T. Cazzolato Universidade de São Paulo (USP) https://orcid.org/0000-0002-4364-010X

DOI: https://doi.org/10.5753/sbbd_estendido.2025.247576

Resumo

Patient Records are digital documentation of a person’s presence in a health institution. This data has been used to enhance medical experience and has allowed for many breakthroughs in scientific research. Nevertheless, the unstructured and non-standard nature of medical notes constitutes an obstacle to the complete usability of this information in crucial tools such as data analysis, visualization, and machine learning algorithms. In this work, we evaluated the use of Large Language Models to preprocess the unlabeled data into a defined structure, test different model sizes and architectures, and develop prompt strategies and fine-tuning techniques. The proposal was evaluated using the Covid-19 DataSharing dataset. Even low-computation cost models can achieve great performance, which could enable an approach to standardize many kinds of data.

Palavras-chave: LLMs, electronic health records, data mining, medical data

Referências

Barbalho, I. M. P., Fernandes, F., Barros, D. M. S., Paiva, J. C., Henriques, J., Morais, A. H. F., Coutinho, K. D., Coelho Neto, G. C., Chioro, A., and Valentim, R. A. M. (2022). Electronic health records in brazil: Prospects and technological challenges. Frontiers in Public Health, 10. DOI: 10.3389/fpubh.2022.963841.

Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J. D., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020). Language models are few-shot learners. Advances in neural information processing systems, 33:1877–1901. DOI: 10.5555/3495724.3495883.

Costa, L., Figênio, M., Santanchè, A., and Gomes-Jr, L. (2024). Llm-mri python module: a brain scanner for llms. In Anais Estendidos do XXXIX Simpósio Brasileiro de Bancos de Dados, pages 125–130, Porto Alegre, RS, Brasil. SBC. DOI: 10.5753/sbbd_estendido.2024.243136.

FAPESP (2020). FAPESP COVID-19 Data Sharing/BR. Technical report, FAPESP. URL: [link].

Hadi, M. U., Qureshi, R., Shah, A., Irfan, M., Zafar, A., Shaikh, M. B., Akhtar, N., Wu, J., Mirjalili, S., et al. (2023). A survey on large language models: Applications, challenges, limitations, and practical usage. Authorea Preprints.

Jiang, J., Qi, K., Bai, G., and Schulman, K. (2023). Pre-pandemic assessment: a decade of progress in electronic health record adoption among us hospitals. Health Affairs Scholar, 1(5):qxad056.

Liu, P., Yuan, W., Fu, J., Jiang, Z., Hayashi, H., and Neubig, G. (2023). Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing. ACM computing surveys, 55(9):1–35. DOI: 10.1145/3560815.

Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L. u., and Polosukhin, I. (2017). Attention is all you need. In Guyon, I., Luxburg, U. V., Bengio, S., Wallach, H., Fergus, R., Vishwanathan, S., and Garnett, R., editors, Advances in Neural Information Processing Systems, volume 30. Curran Associates, Inc. URL: [link].

Wei, J., Wang, X., Schuurmans, D., Bosma, M., Xia, F., Chi, E., Le, Q. V., Zhou, D., et al. (2022). Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824–24837.