Ontology-Based Semantic Annotation applied to Medical Images

• Juliana Wolf Pereira UFSCar
• Marcela Xavier Ribeiro UFSCar

DOI: https://doi.org/10.5753/sbcas_estendido.2024.2215

Abstract

Mammograms allow for the early diagnosis of breast cancer, the most common type of cancer among women in Brazil and around the world. In this doctoral research, the MUSA method was developed to classify and semantically annotate mammography images based on the fusion of multimodal information, providing a fuller image annotation than the current state of the art. The approach includes text mining, image mining, and ontology engineering processes. The image mining process’s results for classifying lesions as mass or calcification were more than 92% accurate, reaching better results than the literature. The results also demonstrate that the AnotaMammo ontology adequately performed the semantic enrichment of the classification. In addition, it adequately performed the fusion of multimodal information. Finally, the MUSA method adds information to make the result more semantic and interpretable, thus reducing the semantic gap.

References

Agarwal, V. and Carson, C. (2015). Using deep convolutional neural networks to predict semantic features of lesions in mammograms. C231n Course Project Reports.

Arp, R., Smith, B., and Spear, A. D. (2015). Building ontologies with basic formal ontology. Mit Press.

Khan, H. N., Shahid, A. R., Raza, B., Dar, A. H., and Alquhayz, H. (2019). Multi-view feature fusion based four views model for mammogram classification using convolutional neural network. IEEE Access, 7:165724–165733.

Levy, O., Goldberg, Y., and Dagan, I. (2015). Improving distributional similarity with lessons learned from word embeddings. Transactions of the association for computational linguistics, 3:211–225.

Noy, N. F., McGuinness, D. L., et al. (2001). Ontology development 101: A guide to creating your first ontology.

Pereira, J. W. and Ribeiro, M. X. (2021). Semantic annotation and classification of mammography images using ontologies. In 2021 IEEE 34th International Symposium on Computer-Based Medical Systems (CBMS), pages 378–383. IEEE.

Pereira, J. W. and Ribeiro, M. X. (2022). Hyperparameter for deep learning applied in mammogram image classification. In 2022 IEEE 35th International Symposium on Computer-Based Medical Systems (CBMS), pages 90–95. IEEE.

Wang, J., Yang, X., Cai, H., Tan, W., Jin, C., and Li, L. (2016). Discrimination of breast cancer with microcalcifications on mammography by deep learning. Scientific reports, 6(1):27327.