A Transformer-Based CBIR System for Cutaneous Ulcer Analysis with LLM Assistance
Resumo
Chronic dermatological ulcers are a public health concern, with clinical assessment subject to inter-observer variability. Content-Based Image Retrieval (CBIR) systems offer decision support, yet Transformer-based feature extractors remain underexplored in this domain. This work presents a CBIR system augmented with a Large Language Model (LLM) layer that summarizes retrieved cases in natural clinical language. To inform its configuration, we compare five Transformer extractors against handcrafted descriptors and a random reference on a dataset labeled by tissue composition, using Precision@K, MAP, macro-mAP, and NDCG@K with graded Jaccard relevance. The study identifies a configuration with consistent performance across all metrics.
Palavras-chave:
CBIR, LLM assistance, chronic dermatological ulcer, Transformers
Referências
Blanco, G. et al. (2020). A superpixel-driven deep learning approach for the analysis of dermatological wounds. Computer Methods and Programs in Biomedicine, 183:105079. DOI: 10.1016/j.cmpb.2019.105079.
Cazzolato, M. T., Ramos, J. S., Rodrigues, L. S., Scabora, L. C., Chino, D. Y. T., Jorge, A. E. S., de Azevedo-Marques, P. M., Traina Jr., C., and Traina, A. J. M. (2021). The UTrack framework for segmenting and measuring dermatological ulcers through telemedicine. Computer Methods and Programs in Biomedicine. DOI: 10.1016/j.compbiomed.2021.104489.
Chino, D. Y. T. et al. (2018). ICARUS: Retrieving skin ulcer images through bag-of-signatures. In Proc. IEEE 31st International Symposium on Computer-Based Medical Systems (CBMS), pages 82–87. IEEE. DOI: 10.1109/CBMS.2018.00022.
Forti, J. K., Navarro, T. P., and dos Santos, A. L. (2026). Explainable deep learning for etiological classification of vascular ulcers using a hybrid convolutional neural network-transformer model. JVS-Vascular Insights, 4:100419. DOI: 10.1016/j.jvsvi.2026.100419.
GBD 2021 Decubitus Ulcers Collaborators (2025). Global, regional and national burden of decubitus ulcers in 204 countries and territories from 1990 to 2021: a systematic analysis based on the Global Burden of Disease study 2021. Frontiers in Public Health, 13:1494229. DOI: 10.3389/fpubh.2025.1494229.
Johnson, J., Douze, M., and Jégou, H. (2019). Billion-scale similarity search with gpus. IEEE Transactions on Big Data, 7(3):535–547. DOI: 10.1109/TBDATA.2019.2921572.
Lin, W., Zhao, Z., Zhang, X., Wu, C., Zhang, Y., Wang, Y., and Xie, W. (2023). PMC-CLIP: Contrastive language-image pre-training using biomedical documents. In Medical Image Computing and Computer Assisted Intervention – MICCAI 2023, volume 14227 of Lecture Notes in Computer Science, pages 525–536. Springer. DOI: 10.1007/978-3-031-43993-3_51.
Oquab, M., Darcet, T., Moutakanni, T., et al. (2024). DINOv2: Learning robust visual features without supervision. Transactions on Machine Learning Research. [link].
Radford, A. et al. (2021). Learning transferable visual models from natural language supervision. In Proc. 38th International Conference on Machine Learning (ICML). PMLR. [link].
Sen, C. K. (2023). Human wound and its burden: Updated 2022 compendium of estimates. Advances in Wound Care, 12(12):657–670. DOI: 10.1089/wound.2023.0007.
Tschannen, M. et al. (2025). SigLIP 2: Multilingual vision-language encoders with improved semantic understanding, localization, and dense features. arXiv preprint arXiv:2502.14786.
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, Ł., and Polosukhin, I. (2017). Attention is all you need. In Advances in Neural Information Processing Systems (NeurIPS), volume 30. Curran Associates, Inc. [link].
Zhang, S., Xu, Y., Usuyama, N., Xu, H., Bagga, J., Tinn, R., Preston, S., Rao, R., Wei, M., Valluri, N., Wong, C., Tupini, A., Wang, Y., Mazzola, M., Shukla, S., Liden, L., Gao, J., Lungren, M. P., Naumann, T., Wang, S., Poon, H., et al. (2023). BiomedCLIP: a multimodal biomedical foundation model pretrained from fifteen million scientific image-text pairs. arXiv preprint arXiv:2303.00915.
Cazzolato, M. T., Ramos, J. S., Rodrigues, L. S., Scabora, L. C., Chino, D. Y. T., Jorge, A. E. S., de Azevedo-Marques, P. M., Traina Jr., C., and Traina, A. J. M. (2021). The UTrack framework for segmenting and measuring dermatological ulcers through telemedicine. Computer Methods and Programs in Biomedicine. DOI: 10.1016/j.compbiomed.2021.104489.
Chino, D. Y. T. et al. (2018). ICARUS: Retrieving skin ulcer images through bag-of-signatures. In Proc. IEEE 31st International Symposium on Computer-Based Medical Systems (CBMS), pages 82–87. IEEE. DOI: 10.1109/CBMS.2018.00022.
Forti, J. K., Navarro, T. P., and dos Santos, A. L. (2026). Explainable deep learning for etiological classification of vascular ulcers using a hybrid convolutional neural network-transformer model. JVS-Vascular Insights, 4:100419. DOI: 10.1016/j.jvsvi.2026.100419.
GBD 2021 Decubitus Ulcers Collaborators (2025). Global, regional and national burden of decubitus ulcers in 204 countries and territories from 1990 to 2021: a systematic analysis based on the Global Burden of Disease study 2021. Frontiers in Public Health, 13:1494229. DOI: 10.3389/fpubh.2025.1494229.
Johnson, J., Douze, M., and Jégou, H. (2019). Billion-scale similarity search with gpus. IEEE Transactions on Big Data, 7(3):535–547. DOI: 10.1109/TBDATA.2019.2921572.
Lin, W., Zhao, Z., Zhang, X., Wu, C., Zhang, Y., Wang, Y., and Xie, W. (2023). PMC-CLIP: Contrastive language-image pre-training using biomedical documents. In Medical Image Computing and Computer Assisted Intervention – MICCAI 2023, volume 14227 of Lecture Notes in Computer Science, pages 525–536. Springer. DOI: 10.1007/978-3-031-43993-3_51.
Oquab, M., Darcet, T., Moutakanni, T., et al. (2024). DINOv2: Learning robust visual features without supervision. Transactions on Machine Learning Research. [link].
Radford, A. et al. (2021). Learning transferable visual models from natural language supervision. In Proc. 38th International Conference on Machine Learning (ICML). PMLR. [link].
Sen, C. K. (2023). Human wound and its burden: Updated 2022 compendium of estimates. Advances in Wound Care, 12(12):657–670. DOI: 10.1089/wound.2023.0007.
Tschannen, M. et al. (2025). SigLIP 2: Multilingual vision-language encoders with improved semantic understanding, localization, and dense features. arXiv preprint arXiv:2502.14786.
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, Ł., and Polosukhin, I. (2017). Attention is all you need. In Advances in Neural Information Processing Systems (NeurIPS), volume 30. Curran Associates, Inc. [link].
Zhang, S., Xu, Y., Usuyama, N., Xu, H., Bagga, J., Tinn, R., Preston, S., Rao, R., Wei, M., Valluri, N., Wong, C., Tupini, A., Wang, Y., Mazzola, M., Shukla, S., Liden, L., Gao, J., Lungren, M. P., Naumann, T., Wang, S., Poon, H., et al. (2023). BiomedCLIP: a multimodal biomedical foundation model pretrained from fifteen million scientific image-text pairs. arXiv preprint arXiv:2303.00915.
Publicado
08/09/2026
Como Citar
DOS SANTOS, Gabriel B.; JORGE, Ana Elisa; TRAINA, Agma J. M.; CAZZOLATO, Mirela T..
A Transformer-Based CBIR System for Cutaneous Ulcer Analysis with LLM Assistance. In: WORKSHOP DE TRABALHOS DE ALUNOS DA GRADUAÇÃO (WTAG) - SIMPÓSIO BRASILEIRO DE BANCO DE DADOS (SBBD), 41. , 2026, São Carlos/SP.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2026
.
p. 99-105.
DOI: https://doi.org/10.5753/sbbd_estendido.2026.249606.
