Integração do Contexto Clínico e Imagens de Raios-X do Tórax para Geração Automática de Laudos Radiológicos
Resumo
A geração automática de laudos radiológicos a partir de radiografias de tórax tem se destacado como uma estratégia para apoiar médicos e reduzir a carga de trabalho na interpretação de imagens. Este artigo propõe uma abordagem multimodal baseada em Transformers que integra imagens (frontal e lateral) e histórico clínico do paciente. As características visuais são extraídas com uma ResNet-50 com ajuste fino progressivo, enquanto o contexto clínico é codificado com Bio_ClinicalBERT. As representações são fundidas em uma arquitetura Transformer codificador-decodificador para geração autorregressiva dos laudos. Experimentos no MIMIC-CXR mostram que o modelo produz laudos estruturados e clinicamente coerentes, com desempenho competitivo em métricas de similaridade semântica.Referências
Akhter, Y., Singh, R., and Vatsa, M. (2023). Ai-based radiodiagnosis using chest x-rays: A review. Frontiers in big data, 6:1120989.
Alsentzer, E., Murphy, J. R., Boag, W., Weng, W.-H., Jin, D., Naumann, T., and McDermott, M. (2019). Publicly available clinical bert embeddings. In Proceedings of the 2nd Clinical Natural Language Processing Workshop, pages 72–78. Association for Computational Linguistics.
Bruno, M. A., Walker, E. A., and Abujudeh, H. H. (2015). Understanding and confronting our mistakes: The epidemiology of error in radiology and strategies for error reduction. RadioGraphics, 35(6):1668–1676.
Chen, Z., Shen, Y., Song, Y., and Wan, X. (2021). Cross-modal memory networks for radiology report generation. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 5904–5914.
Chen, Z., Song, Y., Chang, T.-H., and Wan, X. (2020). Generating radiology reports via memory-driven transformer. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1439–1449. arXiv:2010.16056.
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., and Fei-Fei, L. (2009). Imagenet: A large-scale hierarchical image database. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 248–255. IEEE.
Devlin, J., Chang, M.-W., Lee, K., and Toutanova, K. (2019). Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics (NAACL-HLT), pages 4171–4186.
He, K., Zhang, X., Ren, S., and Sun, J. (2016). Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 770–778.
Johnson, A. E. W., Pollard, T. J., Berkowitz, S. J., Greenbaum, N. R., Lungren, M. P., Deng, C.-y., Mark, R. G., and Horng, S. (2019). Mimic-cxr-jpg dataset v2.0.0. Available at: [link].
Litjens, G., Kooi, T., Bejnordi, B. E., Setio, A. A. A., Ciompi, F., Ghafoorian, M., van der Laak, J. A., van Ginneken, B., and Sánchez, C. I. (2017). A survey on deep learning in medical image analysis. Medical Image Analysis, 42:60–88.
Loshchilov, I. and Hutter, F. (2019). Decoupled weight decay regularization. International Conference on Learning Representations (ICLR).
Nicolson, A., Dowling, J., and Koopman, B. (2022). Improving chest x-ray report generation by leveraging warm starting. arXiv preprint, arXiv:2201.09405. Submitted to Elsevier.
Sutskever, I., Vinyals, O., and Le, Q. (2014). Sequence to sequence learning with neural networks. Advances in Neural Information Processing Systems.
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, , and Polosukhin, I. (2017). Attention is all you need. In Advances in Neural Information Processing Systems (NeurIPS), volume 30.
Vinyals, O., Toshev, A., Bengio, S., and Erhan, D. (2015). Show and tell: A neural image caption generator. In Proceedings of the IEEE CVPR, pages 3156–3164.
Yang, S., Wu, X., Ge, S., Zhou, S. K., and Xiao, L. (2022). Knowledge matters: Chest radiology report generation with general and specific knowledge. Medical Image Analysis, xx:xxx–xxx. arXiv:2112.15009.
Alsentzer, E., Murphy, J. R., Boag, W., Weng, W.-H., Jin, D., Naumann, T., and McDermott, M. (2019). Publicly available clinical bert embeddings. In Proceedings of the 2nd Clinical Natural Language Processing Workshop, pages 72–78. Association for Computational Linguistics.
Bruno, M. A., Walker, E. A., and Abujudeh, H. H. (2015). Understanding and confronting our mistakes: The epidemiology of error in radiology and strategies for error reduction. RadioGraphics, 35(6):1668–1676.
Chen, Z., Shen, Y., Song, Y., and Wan, X. (2021). Cross-modal memory networks for radiology report generation. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 5904–5914.
Chen, Z., Song, Y., Chang, T.-H., and Wan, X. (2020). Generating radiology reports via memory-driven transformer. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1439–1449. arXiv:2010.16056.
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., and Fei-Fei, L. (2009). Imagenet: A large-scale hierarchical image database. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 248–255. IEEE.
Devlin, J., Chang, M.-W., Lee, K., and Toutanova, K. (2019). Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics (NAACL-HLT), pages 4171–4186.
He, K., Zhang, X., Ren, S., and Sun, J. (2016). Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 770–778.
Johnson, A. E. W., Pollard, T. J., Berkowitz, S. J., Greenbaum, N. R., Lungren, M. P., Deng, C.-y., Mark, R. G., and Horng, S. (2019). Mimic-cxr-jpg dataset v2.0.0. Available at: [link].
Litjens, G., Kooi, T., Bejnordi, B. E., Setio, A. A. A., Ciompi, F., Ghafoorian, M., van der Laak, J. A., van Ginneken, B., and Sánchez, C. I. (2017). A survey on deep learning in medical image analysis. Medical Image Analysis, 42:60–88.
Loshchilov, I. and Hutter, F. (2019). Decoupled weight decay regularization. International Conference on Learning Representations (ICLR).
Nicolson, A., Dowling, J., and Koopman, B. (2022). Improving chest x-ray report generation by leveraging warm starting. arXiv preprint, arXiv:2201.09405. Submitted to Elsevier.
Sutskever, I., Vinyals, O., and Le, Q. (2014). Sequence to sequence learning with neural networks. Advances in Neural Information Processing Systems.
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, , and Polosukhin, I. (2017). Attention is all you need. In Advances in Neural Information Processing Systems (NeurIPS), volume 30.
Vinyals, O., Toshev, A., Bengio, S., and Erhan, D. (2015). Show and tell: A neural image caption generator. In Proceedings of the IEEE CVPR, pages 3156–3164.
Yang, S., Wu, X., Ge, S., Zhou, S. K., and Xiao, L. (2022). Knowledge matters: Chest radiology report generation with general and specific knowledge. Medical Image Analysis, xx:xxx–xxx. arXiv:2112.15009.
Publicado
01/06/2026
Como Citar
BORGES, Hériclys S.; VIEIRA, Pablo de A.; ARAÚJO, Flávio H. D.; CARVALHO FILHO, Antonio O.; SILVA, Lilian R. G.; V. E SILVA, Romuere R..
Integração do Contexto Clínico e Imagens de Raios-X do Tórax para Geração Automática de Laudos Radiológicos. In: SIMPÓSIO BRASILEIRO DE COMPUTAÇÃO APLICADA À SAÚDE (SBCAS), 26. , 2026, Ouro Preto/MG.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2026
.
p. 966-977.
ISSN 2763-8952.
DOI: https://doi.org/10.5753/sbcas.2026.21594.
