AI-assisted medical image annotation: a study on skin lesion segmentation by non-experts

• Lorenzo M. Scaramussa UFES
• Andre G. C. Pacheco UFES

DOI: https://doi.org/10.5753/sbcas.2025.6961

Abstract

Image annotation is essential for building databases for training artificial intelligence (AI) algorithms. However, the dependence on specialists makes this process expensive and difficult to scale. This work proposes an interactive framework to assist non-specialists in segmenting skin lesions. Its effectiveness and usability were evaluated in an experiment with 50 volunteers, who segmented skin lesions in two modes: manual and assisted by the tool. The results indicate that the assisted approach improves efficiency without compromising accuracy, especially when combined with crowdsourcing. The proposed tool is available in open-source format and can be used in different medical areas beyond dermatology.

References

Abramson, J., Adler, J., Dunger, et al. (2024). Accurate structure prediction of biomolecular interactions with alphafold 3. Nature, 630:493–500.

Barragán-Montero, A., Javaid, U., Valdés, G., and Others (2021). Artificial intelligence and machine learning for medical imaging: A technology review. Physica Medica, 83.

Benaich, N. (2024). State of ai report. Air Street Press.

Cheplygina, V., Perez-Rovira, et al. (2021). Crowdsourcing airway annotations in chest computed tomography images. PLoS One, 16(4):e0249580.

Clark, J. et al. (2023). Artificial intelligence index report. Stanford University. Último acesso: 9 de fevereiro 2025.

Cocos, A., Qian, T., et al. (2017). Crowd control: Effectively utilizing unscreened crowd workers for biomedical data annotation. JBI, 69:86–92.

Costa, G. S. S., Paiva, et al. (2021). Covid-19 automatic diagnosis with ct images using the novel transformer architecture. pages 293–301. SBC.

Damgaard, C., Eriksen, T. N., Juodelyte, D., Cheplygina, V., and Jiménez-Sánchez, A. (2023). Augmenting chest x-ray datasets with non-expert annotations.

Das, K., Cockerell, C. J., Patil, A., Pietkiewicz, et al. (2021). Machine learning and its application in skin cancer. Int. J. Environ. Res. Public Health, 18(24):13409.

De Angelo, G. G., Pacheco, et al. (2019). Skin lesion segmentation using deep learning for images acquired from smartphones. In 2019 International Joint Conference on Neural Networks (IJCNN), pages 1–8.

De Fauw, J., Ledsam, et al. (2018). Clinically applicable deep learning for diagnosis and referral in retinal disease. Nature medicine, 24(9):1342–1350.

Dosovitskiy, A., Beyer, L., Kolesnikov, A., et al. (2021). An image is worth 16x16 words: Transformers for image recognition at scale.

Kar, K. et al. (2024). Automated intracranial hemorrhage detection using deep learning in medical image analysis. In 2024 International Conference on Data Science and Network Security (ICDSNS), pages 1–6.

Kentley, J., Weber, J., et al. (2023). Agreement between experts and an untrained crowd for identifying dermoscopic features using a gamified app: Reader feasibility study. JMIR Medical Informatics.

Kirillov, A., Mintun, E., Ravi, et al. (2023). Segment anything. arXiv:2304.02643.

Maier-Hein, L., Reinke, A., Godau, et al. (2024). Metrics reloaded: recommendations for image analysis validation. Nature Methods, 21(2):195–212.

Najjar, R. (2023). Redefining radiology: a review of artificial intelligence integration in medical imaging. Diagnostics, 13(17):2760.

Nikolov, S., Blackwell, et al. (2021). Clinically applicable segmentation of head and neck anatomy for radiotherapy: deep learning algorithm development and validation study. Journal of medical Internet research, 23(7):e26151.

Ouyang, D., He, et al. (2020). Video-based ai for beat-to-beat assessment of cardiac function. Nature, 580(7802):252–256.

Pacheco, A. G. C. et al. (2020). Pad-ufes-20: A skin lesion dataset composed of patient data and clinical images collected from smartphones. Data in Brief.

Pacheco, A. G. C. and Krohling, R. A. (2021). An attention-based mechanism to combine images and metadata in deep learning models applied to skin cancer classification. IEEE journal of biomedical and health informatics, 25(9):3554–3563.

Sarwar, N., Irshad, et al. (2024). Skin lesion segmentation using deep learning algorithm with ant colony optimization. BMC Medical Informatics and Decision Making.

Shaik, T., Tao, et al. (2023). Remote patient monitoring using artificial intelligence: Current state, applications, and challenges. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, 13(2):e1485.

Souza Jr, L. A., Pacheco, A. G., Passos, L. A., Santana, et al. (2024). Deepcraftfuse: visual and deeply-learnable features work better together for esophageal cancer detection in patients with barrett’s esophagus. Neural Computing and Applications.

Tao Lei, A. K. N. (2023). Image Segmentation: Principles, Techniques, and Applications. John Wiley & Sons.

Tschandl, P., Rosendahl, et al. (2018). The ham10000 dataset, a large collection of multi-source dermatoscopic images of common pigmented skin lesions. Scientific Data.

Ward, A. et al. (2024). Creating an empirical dermatology dataset through crowdsourcing with web search advertisements. JAMA Network Open, 7(11):e2446615.

Warfield, S. K., Zou, et al. (2004). Simultaneous truth and performance level estimation (STAPLE): an algorithm for the validation of image segmentation. IEEE(T-MI), 23.

Yu, Y., Wang, C., Fu, Q., Kou, R., Huang, F., Yang, B., Yang, T., and Gao, M. (2023). Techniques and challenges of image segmentation: A review. Electronics, 12(5).