User-Assisted Design of a Neural Network for Brain Tumor Segmentation

• Matheus A. Cerqueira UNICAMP
• Alexandre X. Falcão UNICAMP

DOI: https://doi.org/10.5753/sibgrapi.est.2023.27455

Resumo

Brain tumor segmentation is a complicated task, with deep learning (DL) presenting the best results. However, DL segmentation models have been increasing in complexity over the last few years, requiring a high volume of fully-annotated images, which is aggravated by the fact that those models are trained to optimize a loss function with no or less understanding of how the features are learned. In contrast, a recent methodology, Feature Learning from Image Markers (FLIM), has involved an expert in the learning loop while reducing human effort in data annotation without using the backpropagation algorithm. In this work, We employ a method for estimating and selecting filters that explore user knowledge, ensuring that the first convolutional layer has features that activate the different lesions and healthy tissue patterns. We used a small U-shaped network (sU-Net) where the encoder is trained with two FLIM modifications, first with multiple training steps (MS-FLIM) and the second by using distinct configurations of markers and images using a biased FLIM (B-FLIM). The results show that the sU-Net based on MS-FLIM and B-FLIM outperforms the standard FLIM and the backpropagation algorithm. Also, We showed that our methodology achieves effectiveness within the standard deviations of the SOTA models while using a small number of layers.

Referências

Q. T. Ostrom, H. Gittleman, J. Fulop, M. Liu, R. Blanda, C. Kromer, Y. Wolinsky, C. Kruchko, and J. S. Barnholtz-Sloan, “Cbtrus statistical report: primary brain and central nervous system tumors diagnosed in the united states in 2008-2012,” Neuro-oncology, vol. 17, no. suppl 4, pp. iv1–iv62, 2015.

Q. T. Ostrom, M. Price, C. Neff, G. Cioffi, K. A. Waite, C. Kruchko, and J. S. Barnholtz-Sloan, “Cbtrus statistical report: Primary brain and other central nervous system tumors diagnosed in the united states in 2015–2019,” Neuro-oncology, vol. 24, no. Supplement 5, pp. v1–v95, 2022.

V. Simi and J. Joseph, “Segmentation of glioblastoma multiforme from mr images–a comprehensive review,” The Egyptian Journal of Radiology and Nuclear Medicine, vol. 46, no. 4, pp. 1105–1110, 2015.

C. Dupont, N. Betrouni, N. Reyns, and M. Vermandel, “On image segmentation methods applied to glioblastoma: state of art and new trends,” IRBM, vol. 37, no. 3, pp. 131–143, 2016.

A. Myronenko, “3d mri brain tumor segmentation using autoencoder regularization,” in International MICCAI Brainlesion Workshop. Springer, 2018, pp. 311–320.

Z. Jiang, C. Ding, M. Liu, and D. Tao, “Two-stage cascaded unet: 1st place solution to brats challenge 2019 segmentation task,” in International MICCAI Brainlesion Workshop. Springer, 2019, pp. 231–241.

F. Isensee, P. F. Jäger, P. M. Full, P. Vollmuth, and K. H. Maier-Hein, “nnu-net for brain tumor segmentation,” in Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries: 6th International Workshop, BrainLes 2020, Held in Conjunction with MICCAI 2020, Lima, Peru, October 4, 2020, Revised Selected Papers, Part II 6. Springer, 2021, pp. 118–132.

H. M. Luu and S.-H. Park, “Extending nn-unet for brain tumor segmentation,” in Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries: 7th International Workshop, BrainLes 2021, Held in Conjunction with MICCAI 2021, Virtual Event, September 27, 2021, Revised Selected Papers, Part II. Springer, 2022, pp. 173–186.

Z. Zhao, P. Xu, C. Scheidegger, and L. Ren, “Human-in-the-loop extraction of interpretable concepts in deep learning models,” IEEE Transactions on Visualization and Computer Graphics, vol. 28, no. 1, pp. 780–790, 2021.

A. Singh, S. Sengupta, and V. Lakshminarayanan, “Explainable deep learning models in medical image analysis,” Journal of Imaging, vol. 6, no. 6, p. 52, 2020.

I. E. De Souza and A. X. Falcão, “Learning cnn filters from user-drawn image markers for coconut-tree image classification,” IEEE Geoscience and Remote Sensing Letters, 2020.

I. E. de Souza, B. C. Benato, and A. X. Falcão, “Feature learning from image markers for object delineation,” in 2020 33rd SIBGRAPI Conference on Graphics, Patterns and Images (SIBGRAPI). IEEE, 2020, pp. 116–123.

D. Lin, J. Dai, J. Jia, K. He, and J. Sun, “Scribblesup: Scribblesupervised convolutional networks for semantic segmentation,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 3159–3167.

Y. B. Can, K. Chaitanya, B. Mustafa, L. M. Koch, E. Konukoglu, and C. F. Baumgartner, “Learning to segment medical images with scribblesupervision alone,” in Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support. Springer, 2018, pp. 236–244.

M. Tang, F. Perazzi, A. Djelouah, I. Ben Ayed, C. Schroers, and Y. Boykov, “On regularized losses for weakly-supervised cnn segmentation,” in Proceedings of the European Conference on Computer Vision (ECCV), 2018, pp. 507–522.

R. Dorent, S. Joutard, J. Shapey, S. Bisdas, N. Kitchen, R. Bradford, S. Saeed, M. Modat, S. Ourselin, and T. Vercauteren, “Scribble-based domain adaptation via co-segmentation,” in Medical Image Computing and Computer Assisted Intervention–MICCAI 2020: 23rd International Conference, Lima, Peru, October 4–8, 2020, Proceedings, Part I 23. Springer, 2020, pp. 479–489.

B. C. Benato, I. E. de Souza, F. L. Galvão, and A. X. Falcão, “Convolutional neural networks from image markers,” arXiv preprint arXiv:2012.12108, 2020.

A. M. Sousa, F. Reis, R. Zerbini, J. L. Comba, and A. X. Falcão, “Cnn filter learning from drawn markers for the detection of suggestive signs of covid-19 in ct images,” in 2021 43rd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC). IEEE, 2021, pp. 3169–3172.

I. E. de Souza, C. L. Cazarin, M. R. Veronez, L. Gonzaga, and A. X. Falcão, “User-guided data expansion modeling to train deep neural networks with little supervision,” IEEE Geoscience and Remote Sensing Letters, vol. 19, pp. 1–5, 2022.

M. A. Cerqueira, “User-assisted design of a neural network for brain tumor segmentation,” Master’s thesis, Universidade Estadual de Campinas, Instituto de Computação, 2023.

S. Bakas, H. Akbari, A. Sotiras, M. Bilello, M. Rozycki, J. S. Kirby, J. B. Freymann, K. Farahani, and C. Davatzikos, “Advancing the cancer genome atlas glioma mri collections with expert segmentation labels and radiomic features,” Scientific data, vol. 4, no. 1, pp. 1–13, 2017.

G. C. Ruppert, L. Teverovskiy, C.-P. Yu, A. X. Falcao, and Y. Liu, “A new symmetry-based method for mid-sagittal plane extraction in neuroimages,” in 2011 IEEE international symposium on biomedical imaging: from nano to macro. IEEE, 2011, pp. 285–288.

S. B. Martins, J. Bragantini, A. X. Falcão, and C. L. Yasuda, “An adaptive probabilistic atlas for anomalous brain segmentation in mr images,” Medical physics, vol. 46, no. 11, pp. 4940–4950, 2019.

V. S. Fonov, A. C. Evans, R. C. McKinstry, C. Almli, and D. Collins, “Unbiased nonlinear average age-appropriate brain templates from birth to adulthood,” NeuroImage, vol. 47, p. S102, 2009.

B. H. Menze, A. Jakab, S. Bauer, J. Kalpathy-Cramer, K. Farahani, J. Kirby, Y. Burren, N. Porz, J. Slotboom, R. Wiest et al., “The multimodal brain tumor image segmentation benchmark (brats),” IEEE transactions on medical imaging, vol. 34, no. 10, pp. 1993–2024, 2014.

S. Bakas, M. Reyes, A. Jakab, S. Bauer, M. Rempfler, A. Crimi, R. T. Shinohara, C. Berger, S. M. Ha, M. Rozycki et al., “Identifying the best machine learning algorithms for brain tumor segmentation, progression assessment, and overall survival prediction in the brats challenge,” arXiv preprint arXiv:1811.02629, 2018.

K. Kamnitsas, C. Ledig, V. F. Newcombe, J. P. Simpson, A. D. Kane, D. K. Menon, D. Rueckert, and B. Glocker, “Efficient multi-scale 3d cnn with fully connected crf for accurate brain lesion segmentation,” Medical image analysis, vol. 36, pp. 61–78, 2017.

M. A. Cerqueira, F. Sprenger, B. C. Teixeira, and A. X. Falcão, “Building brain tumor segmentation networks with user-assisted filter estimation and selection,” in 18th International Symposium on Medical Information Processing and Analysis, vol. 12567. SPIE, 2023, pp. 202–211.