Multimodal Artificial Intelligence for Aided Diagnosis of Neglected Tropical Diseases: A Comparative Study of Visual Architectures

Mário de Araújo Carvalho; Allison Oliveira Miranda; Celso Soares Costa; Wesley Nunes Gonçalves

doi:10.5753/sbcas_estendido.2026.26536

Mário de Araújo Carvalho UFMS
Allison Oliveira Miranda Santa Casa - Hospital Vale do Guaporé
Celso Soares Costa IFMS
Wesley Nunes Gonçalves UFMS

DOI: https://doi.org/10.5753/sbcas_estendido.2026.26536

Resumo

As Doenças Tropicais Negligenciadas (DTNs) impõem um ônus desproporcional às populações vulneráveis em todo o mundo e desafiam os sistemas de saúde pública quanto ao diagnóstico oportuno e preciso. A Inteligência Artificial (IA) e a visão computacional surgiram como ferramentas promissoras para a vigilância epidemiológica e o apoio à decisão clínica em ambientes com recursos limitados. Este trabalho compara arquiteturas visuais recentes, abrangendo Redes Neurais Convolucionais (CNNs) tradicionais e modelos de fundação multimodais, para classificar patógenos parasitários em imagens de microscopia. Cinco arquiteturas foram avaliadas: ResNet-50, EfficientNet-B3, ViT-B/16, CLIP e SigLIP. O conjunto de dados contém oito classes: Babesia, Leishmania, Plasmodium, Toxoplasma, Trichomonad, Trypanosome, Leucócitos e Hemácias (RBCs), e foi dividido em subconjuntos de treino, validação e teste segundo uma partição estratificada de 70/15/15. A análise enfatiza as DTNs formalmente reconhecidas: Leishmania e Trypanosome. Os Vision Transformers e os baselines multimodais alcançaram desempenho próximo ao perfeito, com o modelo ViT-B/16 atingindo 99,63% de acurácia e 99,19% de F1-Macro. Os modelos de melhor desempenho também foram disponibilizados em um protótipo funcional de Progressive Web App (PWA), o CADTN Classifier, demonstrando viabilidade multiplataforma em navegadores desktop e mobile. Esses resultados indicam que os modelos computacionais avançados podem fortalecer os sistemas de informação em saúde e apoiar o diagnóstico clínico em ambientes com recursos escassos. O código-fonte está disponível em: https://github.com/MarioCarvalhoBr/multimodal-ntd-classifier.

Referências

Abed, B. A. and Waleed, J. (2024). A developed deep learning-based approach for microorganism classification. In 2024 First International Conference on Software, Systems and Information Technology (SSITCON), pages 1–7. IEEE.

Alrefaei, A. (2023). Parasite dataset: Leishmania, plasmodium babesia.

Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., Uszkoreit, J., and Houlsby, N. (2021). An image is worth 16x16 words: Transformers for image recognition at scale. ICLR.

Goodfellow, I., Bengio, Y., Courville, A., and Bengio, Y. (2016). Deep learning, volume 1. MIT press Cambridge.

He, K., Zhang, X., Ren, S., and Sun, J. (2016). Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778.

Hosen, M. D., Mohiuddin, A. B., Sarker, N., Sakib, M. S., Al Sakib, A., Dip, R. H., Ahmed, M. R., and Haque, R. (2024). Parasitology unveiled: Revolutionizing microorganism classification through deep learning. In 2024 6th International Conference on Electrical Engineering and Information & Communication Technology (ICEEICT), pages 1163–1168. IEEE.

Kornblith, S., Shlens, J., and Le, Q. V. (2019). Do better imagenet models transfer better? In 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 2656–2666.

Kumar, Y., Garg, P., Moudgil, M. R., Singh, R., Woźniak, M., Shafi, J., and Ijaz, M. F. (2024). Enhancing parasitic organism detection in microscopy images through deep learning and fine-tuned optimizer. Scientific Reports, 14(1):5753.

Li, S. (2020). Microscopic Images of Parasites Species.

Litjens, G., Kooi, T., Bejnordi, B. E., Setio, A. A. A., Ciompi, F., Ghafoorian, M., Van Der Laak, J. A., Van Ginneken, B., and Sánchez, C. I. (2017). A survey on deep learning in medical image analysis. Medical image analysis, 42:60–88.

Maturana, C. R., De Oliveira, A. D., Nadal, S., Bilalli, B., Serrat, F. Z., Soley, M. E., Igual, E. S., Bosch, M., Lluch, A. V., Abelló, A., López-Codina, D., Suñé, T. P., Clols, E. S., and Joseph-Munné, J. (2022). Advances and challenges in automated malaria diagnosis using digital microscopy imaging with artificial intelligence tools: A review. Frontiers in Microbiology, 13:1006659.

Organization, W. H. et al. (2022). Ending the neglect to attain the sustainable development goals: a rationale for continued investment in tackling neglected tropical diseases 2021–2030. World Health Organization.

Pamungkas, Y., Triandini, E., Karim, A., and Sangsawang, T. (2026). A vision transformer architecture for automated recognition of parasitic types in microscopic images. International Journal of Robotics and Control Systems, 6(1):383–400.

Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., Antiga, L., et al. (2019). Pytorch: An imperative style, high-performance deep learning library. Advances in neural information processing systems, 32.

Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., Krueger, G., and Sutskever, I. (2021). Learning transferable visual models from natural language supervision. CoRR, abs/2103.00020.

Rajinikanth, V. (2024). Plasmodium detection from blood-cell images using mobilenet and firefly algorithm. In 2024 International Conference on Advances in Technology and Computing (ICATC), pages 1–5. IEEE.

Shafiq, M. and Gu, Z. (2022). Deep residual learning for image recognition: A survey. Applied sciences, 12(18):8972.

Shamshad, F., Khan, S., Zamir, S. W., Khan, M. H., Hayat, M., Khan, F. S., and Fu, H. (2023). Transformers in medical imaging: A survey. Medical image analysis, 88:102802.

Soares, M. Q. and Chiavegatto Filho, A. D. P. (2026). Artificial intelligence in brazilian public health: potential, challenges, and ethical implications for the brazilian unified health system. Revista Brasileira de Epidemiologia, 29:e260006.

Sutton, R. T., Pincock, D., Baumgart, D. C., Sadowski, D. C., Fedorak, R. N., and Kroeker, K. I. (2020). An overview of clinical decision support systems: benefits, risks, and strategies for success. NPJ digital medicine, 3(1):17.

Tan, M. and Le, Q. (2019). Efficientnet: Rethinking model scaling for convolutional neural networks. In International conference on machine learning, pages 6105–6114. PMLR.

Vassalotti, R., Cervantes, J., D’Amario, V., and Colwell, S. (2026). P-1777. ai-powered blood parasite detection: A smartphone diagnostic tool for resource-limited settings. In Open Forum Infectious Diseases, volume 13, pages ofaf695–1947. Oxford University Press US.

Vijayakumar, K., Sivaraman, A., Narayanan, M., Rajinikanth, V., Jebastine, J., and Srivastava, A. (2025). Examination of blood cell images to identify nannal/malaria with efficientnet model and fused features. In 2025 Tenth International Conference on Science Technology Engineering and Mathematics (ICONSTEM), pages 1–6. IEEE.

Wolf, T., Debut, L., Sanh, V., Chaumond, J., Delangue, C., Moi, A., Cistac, P., Rault, T., Louf, R., Funtowicz, M., Davison, J., Shleifer, S., von Platen, P., Ma, C., Jernite, Y., Plu, J., Xu, C., Le Scao, T., Gugger, S., Drame, M., Lhoest, Q., and Rush, A. (2020). Transformers: State-of-the-art natural language processing. In Liu, Q. and Schlangen, D., editors, Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 38–45, Online. Association for Computational Linguistics.

Ying, X. (2019). An overview of overfitting and its solutions. Journal of Physics: Conference Series, 1168(2):022022.

Zhai, X., Mustafa, B., Kolesnikov, A., and Beyer, L. (2023). Sigmoid loss for language image pre-training. In Proceedings of the IEEE/CVF international conference on computer vision, pages 11975–11986.