Segmentation of Oral Cavity Histological Images: A Study on U-Net Variants and Lightweight Backbones

• Luana R. Borges UFU
• Vitória F. C. Silva UFU
• Davi Soares UFU
• Gustavo C. Miranda UFU
• Luis Felipe G. S. Paim UFU
• Daniel B. Gonçalves UFU
• Adriano B. Silva UFU
• Domingos L. L. de Oliveira UFU / IFSP
• Leandro A. Neves UNESP
• Marcelo Z. do Nascimento UFU

DOI: https://doi.org/10.5753/sbcas.2026.20794

Abstract

Oral cavity cancer presents a high incidence, making early diagnosis essential for improving patient prognosis. In this context, the automatic analysis of histopathological images has gained increasing attention; however, their high structural complexity and morphological variability pose significant challenges. Semantic segmentation is a fundamental step in computer-aided diagnosis systems, as it enables the automatic delineation of regions of interest. This work evaluates different architectures from the U-Net family for histological image segmentation, including conventional models, enhanced variants, and versions incorporating lightweight pre-trained backbones. Experiments were conducted on two public datasets, achieving Dice coefficients of 0.912 and 0.871 for the OCDC and OralEpitheliumDB datasets, respectively. Furthermore, the impact of data augmentation, pre-training, and the trade-off between segmentation performance and computational efficiency were analyzed. The results demonstrate the potential of the evaluated architectures and contribute to a better understanding of more robust and efficient strategies in this domain.

References

Banerji, S. and Mitra, S. (2022). Deep learning in histopathology: A review. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, 12(1):e1439.

Basu, A., Senapati, P., Deb, M., Rai, R., and Dhal, K. G. (2024). A survey on recent trends in deep learning for nucleus segmentation from histopathology images. Evolving Systems, 15(1):203–248.

Cheplygina, V. e. a. (2019). Not-so-supervised: A survey of semi-supervised, multi-instance, and transfer learning in medical image analysis. Medical Image Analysis.

Crepaldi, G. G., de Oliveira, D. L., Tosta, T. A., Neves, L. A., Silva, A. B., Martins, A. S., and do Nascimento, M. Z. (2025). Nuclear segmentation in histological images using multiple attention system mixing. In 2025 IEEE 38th International Symposium on Computer-Based Medical Systems (CBMS), pages 154–159. IEEE.

de Oliveira, D. L., Tosta, T. A., Neves, L. A., Silva, A. B., Martins, A. S., de Fariall, P. R., and do Nascimento, M. Z. (2025). A u-net-based approach for histological tissue segmentation using rcaug data augmentation. In 2025 38th SIBGRAPI Conference on Graphics, Patterns and Images (SIBGRAPI), pages 1–6. IEEE.

Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., and Fei-Fei, L. (2009). Imagenet: A large-scale hierarchical image database. IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

Howard, A. G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., Andreetto, M., and Adam, H. (2017). Mobilenets: Efficient convolutional neural networks for mobile vision applications. arXiv preprint arXiv:1704.04861.

Litjens, G. e. a. (2017). A survey on deep learning in medical image analysis. Medical Image Analysis.

Madabhushi, A. and Lee, G. (2016). Image analysis and machine learning in digital pathology. Medical Image Analysis.

Moscalu, M., Moscalu, R., Dascălu, C. G., T, arcă, V., Cojocaru, E., Costin, I. M., T, arcă, E., and S, erban, I. L. (2023). Histopathological images analysis and predictive modeling implemented in digital pathology—current affairs and perspectives. Diagnostics, 13(14):2379.

Oktay, O., Schlemper, J., Folgoc, L. L., Lee, M., Heinrich, M., Misawa, K., Mori, K., McDonagh, S., Hammerla, N. Y., Kainz, B., et al. (2018). Attention u-net: Learning where to look for the pancreas. arXiv preprint arXiv:1804.03999.

Ronneberger, O., Fischer, P., and Brox, T. (2015). U-net: Convolutional networks for biomedical image segmentation. In Medical image computing and computer-assisted intervention–MICCAI 2015: 18th international conference, Munich, Germany, October 5-9, 2015, proceedings, part III 18, pages 234–241. Springer.

Santos, D. F., de Faria, P. R., Travençolo, B. A., and do Nascimento, M. Z. (2023a). Influence of data augmentation strategies on the segmentation of oral histological images using fully convolutional neural networks. Journal of Digital Imaging, 36(4):1608–1623.

Santos, D. F. D., de Faria, P. R., Loyola, A. M., Cardoso, S. V., Travençolo, B. A. N., and do Nascimento, M. Z. (2023b). Hematoxylin and eosin stained oral squamous cell carcinoma histological images dataset.

Santos, M. d. O. (2023). Estimativa de incidência de câncer no brasil, 2023-2025. Revista Brasileira de Cancerologia, 69(1).

Shorten, C. and Khoshgoftaar, T. M. (2019). A survey on image data augmentation for deep learning. Journal of Big Data.

Silva, A. B., Martins, A. S., Tosta, T. A. A., Loyola, A. M., Cardoso, S. V., Neves, L. A., de Faria, P. R., and do Nascimento, M. Z. (2024a). Oralepitheliumdb: A dataset for oral epithelial dysplasia image segmentation and classification. Journal of Imaging Informatics in Medicine, 37(4):1691–1710.

Silva, A. B., Rozendo, G. B., Tosta, T. A., Martins, A. S., Loyola, A. M., Cardoso, S. V., Lumini, A., Neves, L. A., de Faria, P. R., and do Nascimento, M. Z. (2023). Cnn ensembles for nuclei segmentation on histological images of oed. In 2023 IEEE 36th International Symposium on Computer-Based Medical Systems (CBMS), pages 601–604. IEEE.

Silva, A. B., Tosta, T. A., Neves, L. A., Martins, A. S., De Faria, P. R., and Do Nascimento, M. Z. (2024b). Ensemble of semantic segmentation models for oral epithelial dysplasia images. In 2024 37th SIBGRAPI Conference on Graphics, Patterns and Images (SIBGRAPI), pages 1–6. IEEE.

Springenberg, M., Frommholz, A., Wenzel, M., Weicken, E., Ma, J., and Strodthoff, N. (2023). From modern cnns to vision transformers: Assessing the performance, robustness, and classification strategies of deep learning models in histopathology. Medical Image Analysis, 87:102809.

Srinidhi, C. L., Ciga, O., and Martel, A. L. (2021). Deep neural network models for computational histopathology: A survey. Medical image analysis, 67:101813.

Tan, M. and Le, Q. (2019). Efficientnet: Rethinking model scaling for convolutional neural networks. In International conference on machine learning, pages 6105–6114. PMLR.

Xu, Y., Quan, R., Xu, W., Huang, Y., Chen, X., and Liu, F. (2024). Advances in medical image segmentation: a comprehensive review of traditional, deep learning and hybrid approaches. Bioengineering, 11(10):1034.

Zhou, Z., Rahman Siddiquee, M. M., Tajbakhsh, N., and Liang, J. (2018). Unet++: A nested u-net architecture for medical image segmentation. In Deep learning in medical image analysis and multimodal learning for clinical decision support: 4th international workshop, DLMIA 2018, and 8th international workshop, ML-CDS 2018, held in conjunction with MICCAI 2018, Granada, Spain, September 20, 2018, proceedings 4, pages 3–11. Springer.

Zunair, H. and Hamza, A. B. (2021). Sharp u-net: Depthwise convolutional network for biomedical image segmentation. Computers in biology and medicine, 136:104699.