Long-Range Decoder Skip Connections: Exploiting Multi-Context Information for Cardiac Image Segmentation
Resumo
Heart is one of the most important organs inour body and many critical diseases are associated with itsmalfunctioning. To assess heart conditions, Magnetic ResonanceImaging (MRI) has become the golden standard imaging technique, as it provides to the clinicians stacks of 2D images of theheart. The problem is that examination of these stacks of 2D data,often based on the delineation of heart structures, is tedious anderror prone due to inter and intra variability. For this reason,the investigation of fully automated methods to support heartsegmentation is of paramount importance. Most of the successfulproposed methods to this problem are based on deep-learningsolutions. Especially, the U-net has been demonstrated to be avery effective architecture. In this paper, we propose the usage oflong-range skip connections on the decoder in order to improvethe generalization of the models and also to add multi-contextinformation to refine segmentation results. We evaluate ourapproach in the ACDC and LVSC heart segmentation challenges.Performed experiments on both datasets demonstrate that ourapproach leads to an improvement between 2.0% − 3.3% on thetotal Dice for the segmentation of the heart, when combined withtwo state-of-the-art U-net-based approaches.
Palavras-chave:
semantic image segmentation, deep learning, cardiac image analysis, biomedical imaging
Referências
O. Ronneberger P. Fischer T. Brox "U-net: Convolutional networks for biomedical image segmentation" International Conference on Medical image computing and computer-assisted intervention pp. 234-2015.
O. Bernard A. Lalande C. Zotti F. Cervenansky X. Yang P.-A. Heng I. Cetin K. Lekadir O. Camara M. A. G. Ballester et al. "Deep learning techniques for automatic mri cardiac multi-structures segmentation and diagnosis: Is the problem solved?" IEEE transactions on medical imaging vol. 37 no. 11 pp. 2514-22018.
P. Radau Y. Lu K. Connelly G. Paul A. Dick G. Wright "Evaluation framework for algorithms segmenting short axis cardiac mri" The MIDAS Journal-Cardiac MR Left Ventricle Segmentation Challenge vol. 49 2009.
E. J. Benjamin S. S. Virani C. W. Callaway A. M. Chamberlain A. R. Chang S. Cheng S. E. Chiuve M. Cushman F. N. Delling R. Deo et al. "Heart disease and stroke statistics-2018 update: a report from the american heart association" Circulation vol. no. 12 pp. e67 2018.
S. S. Islam S. Rahman M. M. Rahman E. K. Dey M. Shoyaib "Application of deep learning to computer vision: A comprehensive study" 2016 5th International Conference on Informatics Electronics and Vision (ICIEV) pp. 592-2016.
K. He G. Gkioxari P. Dollár R. Girshick "Mask r-cnn" Proceedings of the IEEE international conference on computer vision pp. 2961-2969 2017.
Z. Wojna V. Ferrari S. Guadarrama N. Silberman L.-C. Chen A. Fathi J. Uijlings "The devil is in the decoder: Classification regression and gans" International Journal of Computer Vision pp. 1-13 2019.
T.-C. Wang M.-Y. Liu J.-Y. Zhu A. Tao J. Kautz B. Catanzaro "High-resolution image synthesis and semantic manipulation with conditional gans" Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition pp. 8798-82018.
F. Isensee P. Kickingereder W. Wick M. Bendszus K. H. Maier-Hein "No new-net" International MICCAI Brainlesion Workshop pp. 234-2018.
P. V. Tran A fully convolutional neural network for cardiac segmentation in short-axis mri 2016.
R. P. Poudel P. Lamata G. Montana "Recurrent fully convolutional neural networks for multi-slice mri cardiac segmentation" Reconstruction segmentation and analysis of medical images. Springer pp. 83-94 2016.
C. F. Baumgartner L. M. Koch M. Pollefeys E. Konukoglu "An exploration of 2d and 3d deep learning techniques for cardiac mr image segmentation" International Workshop on Statistical Atlases and Computational Models of the Heart pp. 111-2017.
F. Isensee P. F. Jaeger P. M. Full I. Wolf S. Engelhardt K. H. Maier-Hein "Automatic cardiac disease assessment on cine-mri via time-series segmentation and domain specific features" International workshop on statistical atlases and computational models of the heart. Springer pp. 120-2017.
G. Huang Z. Liu L. van der Maaten K. Q. Weinberger "Densely connected convolutional networks" Proceedings of the IEEE conference on computer vision and pattern recognition pp. 4700-42017.
H. Li Z. Xu G. Taylor C. Studer T. Goldstein "Visualizing the loss landscape of neural nets" Advances in Neural Information Processing Systems pp. 6389-62018.
A. E. Orhan X. Pitkow Skip connections eliminate singularities 2017.
K. He X. Zhang S. Ren J. Sun "Deep residual learning for image recognition" Proceedings of the IEEE conference on computer vision and pattern recognition pp. 770-2016.
H. Liu H. Hu X. Xu E. Song "Automatic left ventricle segmentation in cardiac mri using topological stable-state thresholding and region restricted dynamic programming" Academic radiology vol. 19 no. 6 pp. 723-2012.
J. Ulén P. Strandmark F. Kahl "An efficient optimization framework for multi-region segmentation based on lagrangian duality" IEEE transactions on medical imaging vol. 32 no. 2 pp. 178-2013.
T. Chen J. Babb P. Kellman L. Axel D. Kim "Semiautomated segmentation of myocardial contours for fast strain analysis in cine displacement-encoded mri" IEEE Transactions on Medical Imaging vol. 27 no. 8 pp. 1084-1094 2008.
I. B. Ayed H.-m. Chen K. Punithakumar I. Ross S. Li "Max-flow segmentation of the left ventricle by recovering subject-specific distributions via a bound of the bhattacharyya measure" Medical image analysis vol. 16 no. 1 pp. 87-2012.
S. Queirós D. Barbosa B. Heyde P. Morais J. L. Vilaça D. Friboulet O. Bernard J. Dhooge "Fast automatic myocardial segmentation in 4d cine cmr datasets" Medical image analysis vol. 18 no. 7 pp. 1115-12014.
S. C. Mitchell J. G. Bosch B. P. Lelieveldt R. J. van der Geest J. H. Reiber M. Sonka "3-d active appearance models: segmentation of cardiac mr and ultrasound images" IEEE transactions on medical imaging vol. 21 no. 9 pp. 1167-12002.
W. Bai W. Shi C. Ledig D. Rueckert "Multi-atlas segmentation with augmented features for cardiac mr images" Medical image analysis vol. 19 no. 1 pp. 98-2015.
J. Long E. Shelhamer T. Darrell "Fully convolutional networks for semantic segmentation" Proceedings of the IEEE conference on computer vision and pattern recognition pp. 3431-32015.
J. Patravali S. Jain S. Chilamkurthy "2d-3d fully convolutional neural networks for cardiac mr segmentation" International Workshop on Statistical Atlases and Computational Models of the Heart pp. 130-2017.
X. Yang C. Bian L. Yu D. Ni P-A. Heng "Class-balanced deep neural network for automatic ventricular structure segmentation" International Workshop on Statistical Atlases and Computational Models of the Heart pp. 152-2017.
B. Kayalibay G. Jensen P. van der Smagt Cnn-based segmentation of medical imaging data 2017.
L. R. Dice "Measures of the amount of ecologic association between species" Ecology vol. 26 no. 3 pp. 297-1945.
D. P. Huttenlocher G. A. Klanderman W. J. Rucklidge "Comparing images using the hausdorff distance" IEEE Transactions on pattern analysis and machine intelligence vol. 15 no. 9 pp. 850-1993.
P. Peng K. Lekadir A. Gooya L. Shao S. E. Petersen A. F. Frangi "A review of heart chamber segmentation for structural and functional analysis using cardiac magnetic resonance imaging" Magnetic Resonance Materials in Physics Biology and Medicine vol. 29 no. 2 pp. 155-2016.
K. He X. Zhang S. Ren J. Sun "Delving deep into rectifiers: Surpassing human-level performance on imagenet classification" Proceedings of the IEEE international conference on computer vision pp. 1026-1034 2015.
D. P. Kingma J. Ba Adam: A method for stochastic optimization 2014.
O. Bernard A. Lalande C. Zotti F. Cervenansky X. Yang P.-A. Heng I. Cetin K. Lekadir O. Camara M. A. G. Ballester et al. "Deep learning techniques for automatic mri cardiac multi-structures segmentation and diagnosis: Is the problem solved?" IEEE transactions on medical imaging vol. 37 no. 11 pp. 2514-22018.
P. Radau Y. Lu K. Connelly G. Paul A. Dick G. Wright "Evaluation framework for algorithms segmenting short axis cardiac mri" The MIDAS Journal-Cardiac MR Left Ventricle Segmentation Challenge vol. 49 2009.
E. J. Benjamin S. S. Virani C. W. Callaway A. M. Chamberlain A. R. Chang S. Cheng S. E. Chiuve M. Cushman F. N. Delling R. Deo et al. "Heart disease and stroke statistics-2018 update: a report from the american heart association" Circulation vol. no. 12 pp. e67 2018.
S. S. Islam S. Rahman M. M. Rahman E. K. Dey M. Shoyaib "Application of deep learning to computer vision: A comprehensive study" 2016 5th International Conference on Informatics Electronics and Vision (ICIEV) pp. 592-2016.
K. He G. Gkioxari P. Dollár R. Girshick "Mask r-cnn" Proceedings of the IEEE international conference on computer vision pp. 2961-2969 2017.
Z. Wojna V. Ferrari S. Guadarrama N. Silberman L.-C. Chen A. Fathi J. Uijlings "The devil is in the decoder: Classification regression and gans" International Journal of Computer Vision pp. 1-13 2019.
T.-C. Wang M.-Y. Liu J.-Y. Zhu A. Tao J. Kautz B. Catanzaro "High-resolution image synthesis and semantic manipulation with conditional gans" Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition pp. 8798-82018.
F. Isensee P. Kickingereder W. Wick M. Bendszus K. H. Maier-Hein "No new-net" International MICCAI Brainlesion Workshop pp. 234-2018.
P. V. Tran A fully convolutional neural network for cardiac segmentation in short-axis mri 2016.
R. P. Poudel P. Lamata G. Montana "Recurrent fully convolutional neural networks for multi-slice mri cardiac segmentation" Reconstruction segmentation and analysis of medical images. Springer pp. 83-94 2016.
C. F. Baumgartner L. M. Koch M. Pollefeys E. Konukoglu "An exploration of 2d and 3d deep learning techniques for cardiac mr image segmentation" International Workshop on Statistical Atlases and Computational Models of the Heart pp. 111-2017.
F. Isensee P. F. Jaeger P. M. Full I. Wolf S. Engelhardt K. H. Maier-Hein "Automatic cardiac disease assessment on cine-mri via time-series segmentation and domain specific features" International workshop on statistical atlases and computational models of the heart. Springer pp. 120-2017.
G. Huang Z. Liu L. van der Maaten K. Q. Weinberger "Densely connected convolutional networks" Proceedings of the IEEE conference on computer vision and pattern recognition pp. 4700-42017.
H. Li Z. Xu G. Taylor C. Studer T. Goldstein "Visualizing the loss landscape of neural nets" Advances in Neural Information Processing Systems pp. 6389-62018.
A. E. Orhan X. Pitkow Skip connections eliminate singularities 2017.
K. He X. Zhang S. Ren J. Sun "Deep residual learning for image recognition" Proceedings of the IEEE conference on computer vision and pattern recognition pp. 770-2016.
H. Liu H. Hu X. Xu E. Song "Automatic left ventricle segmentation in cardiac mri using topological stable-state thresholding and region restricted dynamic programming" Academic radiology vol. 19 no. 6 pp. 723-2012.
J. Ulén P. Strandmark F. Kahl "An efficient optimization framework for multi-region segmentation based on lagrangian duality" IEEE transactions on medical imaging vol. 32 no. 2 pp. 178-2013.
T. Chen J. Babb P. Kellman L. Axel D. Kim "Semiautomated segmentation of myocardial contours for fast strain analysis in cine displacement-encoded mri" IEEE Transactions on Medical Imaging vol. 27 no. 8 pp. 1084-1094 2008.
I. B. Ayed H.-m. Chen K. Punithakumar I. Ross S. Li "Max-flow segmentation of the left ventricle by recovering subject-specific distributions via a bound of the bhattacharyya measure" Medical image analysis vol. 16 no. 1 pp. 87-2012.
S. Queirós D. Barbosa B. Heyde P. Morais J. L. Vilaça D. Friboulet O. Bernard J. Dhooge "Fast automatic myocardial segmentation in 4d cine cmr datasets" Medical image analysis vol. 18 no. 7 pp. 1115-12014.
S. C. Mitchell J. G. Bosch B. P. Lelieveldt R. J. van der Geest J. H. Reiber M. Sonka "3-d active appearance models: segmentation of cardiac mr and ultrasound images" IEEE transactions on medical imaging vol. 21 no. 9 pp. 1167-12002.
W. Bai W. Shi C. Ledig D. Rueckert "Multi-atlas segmentation with augmented features for cardiac mr images" Medical image analysis vol. 19 no. 1 pp. 98-2015.
J. Long E. Shelhamer T. Darrell "Fully convolutional networks for semantic segmentation" Proceedings of the IEEE conference on computer vision and pattern recognition pp. 3431-32015.
J. Patravali S. Jain S. Chilamkurthy "2d-3d fully convolutional neural networks for cardiac mr segmentation" International Workshop on Statistical Atlases and Computational Models of the Heart pp. 130-2017.
X. Yang C. Bian L. Yu D. Ni P-A. Heng "Class-balanced deep neural network for automatic ventricular structure segmentation" International Workshop on Statistical Atlases and Computational Models of the Heart pp. 152-2017.
B. Kayalibay G. Jensen P. van der Smagt Cnn-based segmentation of medical imaging data 2017.
L. R. Dice "Measures of the amount of ecologic association between species" Ecology vol. 26 no. 3 pp. 297-1945.
D. P. Huttenlocher G. A. Klanderman W. J. Rucklidge "Comparing images using the hausdorff distance" IEEE Transactions on pattern analysis and machine intelligence vol. 15 no. 9 pp. 850-1993.
P. Peng K. Lekadir A. Gooya L. Shao S. E. Petersen A. F. Frangi "A review of heart chamber segmentation for structural and functional analysis using cardiac magnetic resonance imaging" Magnetic Resonance Materials in Physics Biology and Medicine vol. 29 no. 2 pp. 155-2016.
K. He X. Zhang S. Ren J. Sun "Delving deep into rectifiers: Surpassing human-level performance on imagenet classification" Proceedings of the IEEE international conference on computer vision pp. 1026-1034 2015.
D. P. Kingma J. Ba Adam: A method for stochastic optimization 2014.
Publicado
28/10/2019
Como Citar
GUTIERREZ, Nicolas; TORRES, Ricardo da Silva ; FALCAO, Alexandre Xavier; KOZERKE, Sebastian; SCHWITTER, Jürg; MASCI, Pier-Giorgio; MONTOYA, Javier.
Long-Range Decoder Skip Connections: Exploiting Multi-Context Information for Cardiac Image Segmentation. In: CONFERENCE ON GRAPHICS, PATTERNS AND IMAGES (SIBGRAPI), 32. , 2019, Rio de Janeiro.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2019
.
DOI: https://doi.org/10.5753/sibgrapi.2019.9808.
