A Study on Class Activation Map Methods to Detect Masses in Mammography Images using Weakly Supervised Learning
Abstract
In the last years, weakly supervised models have aided in the mass detection using mammography images, decreasing the need for pixel-level annotations. Most of the models in literature are based on CAM activation maps, not exploring the use of other activation methods for detection in mammography images. This works presents a study of different state-of-the-art activation maps methods, applied to weakly supervised training in mammography images. In this study, we compare the methods CAM, GradCAM, GradCAM++, XGradCAM and LayerCAM, using the GMIC model to detect the presence of mass in mammography images. The evaluation is done using the dataset VinDr-Mammo, using the metrics Accuracy, TPR, FNR and FPPI. Results show that the use of different strategies of activation maps during training and test stages lead to an improvement of the model. With this strategy, we improve the results of the GMIC method, decreasing the FPPI value and increasing TPR.
References
World Health Organization, "Breast cancer screening," 2017, disponivel em: http://www.who.int/cancer/prevention/diagnosis-screening/breast-cancer/en/, acessado em 08 dez. 2017.
P. Autier, M. Boniol, R. Middleton, J.-F. Doré, C. Héry, T. Zheng, and A. Gavin, "Advanced breast cancer incidence following population-based mammographic screening," Annals of Oncology, pp. 600-633, 2011.
D. S. Deshpande, A. M. Rajurkar, and R. M. Manthalkar, "Medical image analysis an attempt for mammogram classification using texture based association rule mining," in Computer Vision, Pattern Recognition, Image Processing and Graphics (NCV-PRIPG), 2013 Fourth National Conference on. IEEE, 2013, pp. 1-5.
M. H. Hesamian, W. Jia, X. He, and P. Kennedy, "Deep learning techniques for medical image segmentation: achievements and challenges," Journal of digital imaging, vol. 32, no. 4, pp. 582-596, 2019.
S. S. Yadav and S. M. Jadhav, "Deep convolutional neural network based medical image classification for disease diagnosis," Journal of Big Data, vol. 6, no. 1, pp. 1-18, 2019.
Y. Shen, N. Wu, J. Phang, J. Park, K. Liu, S. Tyagi, L. Heacock, S. G. Kim, L. Moy, K. Cho et al., "An interpretable classifier for high-resolution breast cancer screening images utilizing weakly supervised localization," Medical image analysis, vol. 68, p. 101908, 2021.
S.-T. Tran, C.-H. Cheng, T.-T. Nguyen, M.-H. Le, and D.-G. Liu, "Tmd-unet: Triple-unet with multi-scale input features and dense skip connection for medical image segmentation," in Healthcare, vol. 9, no. 1. Multidisciplinary Digital Publishing Institute, 2021, p. 54.
X. Xie, J. Chen, Y. Li, L. Shen, K. Ma, and Y. Zheng, "Instance-aware self-supervised learning for nuclei segmentation," in International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, 2020, pp. 341-350.
A. Diba, V. Sharma, A. Pazandeh, H. Pirsiavash, and L. Van Gool, "Weakly supervised cascaded convolutional networks," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), July 2017.
X. Zhang, Y. Wei, J. Feng, Y. Yang, and T. S. Huang, "Adversarial complementary learning for weakly supervised object localization," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp. 1325-1334.
X. Ouyang, Z. Xue, Y. Zhan, X. S. Zhou, Q. Wang, Y. Zhou, Q. Wang, and J.-Z. Cheng, "Weakly supervised segmentation framework with uncertainty: A study on pneumothorax segmentation in chest x-ray," in International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, 2019, pp. 613-621.
B. Zhou, A. Khosla, A. Lapedriza, A. Oliva, and A. Torralba, "Learning deep features for discriminative localization," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 2921-2929.
R. R. Selvaraju, M. Cogswell, A. Das, R. Vedantam, D. Parikh, and D. Batra, "Gradcam: Visual explanations from deep networks via gradient-based localization," in Proceedings of the IEEE international conference on computer vision, 2017, pp. 618-626.
A. Chattopadhay, A. Sarkar, P. Howlader, and V. N. Balasubramanian, "Grad-cam++: Generalized gradient-based visual explanations for deep convolutional networks," in 2018 IEEE winter conference on applications of computer vision (WACV). IEEE, 2018, pp. 839-847.
R. Fu, Q. Hu, X. Dong, Y. Guo, Y. Gao, and B. Li, "Axiom-based grad-cam: Towards accurate visualization and explanation of cnns," 2020.
P.-T. Jiang, C.-B. Zhang, Q. Hou, M.-M. Cheng, and Y. Wei, "Layercam: Exploring hierarchical class activation maps for localization," IEEE Transactions on Image Processing, vol. 30, pp. 5875-5888, 2021.
Y. Shen, N. Wu, J. Phang, J. Park, K. Liu, S. Tyagi, L. Heacock, S. G. Kim, L. Moy, K. Cho et al., "An interpretable classifier for high-resolution breast cancer screening images utilizing weakly supervised localization," Medical image analysis, vol. 68, p. 101908, 2021.
H. T. Nguyen, H. Q. Nguyen, H. H. Pham, K. Lam, L. T. Le, M. Dao, and V. Vu, "Vindr-mammo: A large-scale benchmark dataset for computer-aided diagnosis in full-field digital mammography," medRxiv, 2022. [Online]. Available: https://www.medrxiv.org/content/early/2022/03/10/2022.03.07.22272009
G. Liang, X. Wang, Y. Zhang, and N. Jacobs, "Weakly-supervised self-training for breast cancer localization," in 2020 42nd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC). IEEE, 2020, pp. 1124-1127.
R. Bakalo, R. Ben-Ari, and J. Goldberger, "Classification and detection in mammograms with weak supervision via dual branch deep neural net," in 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019). IEEE, 2019, pp. 1905- 1909.
W. Zhu, Q. Lou, Y. S. Vang, and X. Xie, "Deep multi-instance networks with sparse label assignment for whole mammogram classification," in International conference on medical image computing and computer-assisted intervention. Springer, 2017, pp. 603-611.
M. Dundar, B. Krishnapuram, R. Rao, and G. Fung, "Multiple instance learning for computer aided diagnosis," Advances in neural information processing systems, vol. 19, 2006.
Z. Qin, D. Kim, and T. Gedeon, "Neural network classifier as mutual information estimator," https://github.com/ZhenyueQin/Research-Softmax-with-Mutual-Information, 2021.
X. Wang, Y. Peng, L. Lu, Z. Lu, M. Bagheri, and R. M. Summers, "Chestx-ray8: Hospital-scale chest x-ray database and benchmarks on weakly-supervised classification and localization of common thorax diseases," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2017, pp. 2097-2106.
P. Rajpurkar, J. Irvin, R. L. Ball, K. Zhu, B. Yang, H. Mehta, T. Duan, D. Ding, A. Bagul, C. P. Langlotz et al., "Deep learning for chest radiograph diagnosis: A retrospective comparison of the chexnext algorithm to practicing radiologists," PLoS medicine, vol. 15, no. 11, p. e1002686, 2018.
S. Poppi, M. Cornia, L. Baraldi, and R. Cucchiara, "Revisiting the evaluation of class activation mapping for explainability: A novel metric and experimental analysis," in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021, pp. 2299-2304.
R. R. Selvaraju, M. Cogswell, A. Das, R. Vedantam, D. Parikh, and D. Batra, "Gradcam: Visual explanations from deep networks via gradient-based localization," in Proceedings of the IEEE international conference on computer vision, 2017, pp. 618-626.
N. Wu, J. Phang, J. Park, Y. Shen, S. G. Kim, L. Heacock, L. Moy, K. Cho, and K. J. Geras, "The nyu breast cancer screening dataset v1. 0," New York Univ., New York, NY, USA, Tech. Rep, 2019.
K. He, X. Zhang, S. Ren, and J. Sun, "Deep residual learning for image recognition," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 770-778.
J. Gildenblat and contributors, "Pytorch library for cam methods," https://github.com/jacobgil/pytorch-grad-cam, 2021.
D. Ribli, A. Horváth, Z. Unger, P. Pollner, and I. Csabai, "Detecting and classifying lesions in mammograms with deep learning," Scientific reports, vol. 8, no. 1, pp. 1-7, 2018.
L. Shen, L. R. Margolies, J. H. Rothstein, E. Fluder, R. McBride, and W. Sieh, "Deep learning to improve breast cancer detection on screening mammography," Scientific reports, vol. 9, no. 1, pp. 1-12, 2019.
H. Jung, B. Kim, I. Lee, M. Yoo, J. Lee, S. Ham, O. Woo, and J. Kang, "Detection of masses in mammograms using a one-stage object detector based on a deep convolutional neural network," PloS one, vol. 13, no. 9, p. e0203355, 2018.
R. Agarwal, O. Díaz, M. H. Yap, X. Lladó, and R. Martí, "Deep learning for mass detection in full field digital mammograms," Computers in biology and medicine, vol. 121, p. 103774, 2020.
P. Autier, M. Boniol, R. Middleton, J.-F. Doré, C. Héry, T. Zheng, and A. Gavin, "Advanced breast cancer incidence following population-based mammographic screening," Annals of Oncology, pp. 600-633, 2011.
D. S. Deshpande, A. M. Rajurkar, and R. M. Manthalkar, "Medical image analysis an attempt for mammogram classification using texture based association rule mining," in Computer Vision, Pattern Recognition, Image Processing and Graphics (NCV-PRIPG), 2013 Fourth National Conference on. IEEE, 2013, pp. 1-5.
M. H. Hesamian, W. Jia, X. He, and P. Kennedy, "Deep learning techniques for medical image segmentation: achievements and challenges," Journal of digital imaging, vol. 32, no. 4, pp. 582-596, 2019.
S. S. Yadav and S. M. Jadhav, "Deep convolutional neural network based medical image classification for disease diagnosis," Journal of Big Data, vol. 6, no. 1, pp. 1-18, 2019.
Y. Shen, N. Wu, J. Phang, J. Park, K. Liu, S. Tyagi, L. Heacock, S. G. Kim, L. Moy, K. Cho et al., "An interpretable classifier for high-resolution breast cancer screening images utilizing weakly supervised localization," Medical image analysis, vol. 68, p. 101908, 2021.
S.-T. Tran, C.-H. Cheng, T.-T. Nguyen, M.-H. Le, and D.-G. Liu, "Tmd-unet: Triple-unet with multi-scale input features and dense skip connection for medical image segmentation," in Healthcare, vol. 9, no. 1. Multidisciplinary Digital Publishing Institute, 2021, p. 54.
X. Xie, J. Chen, Y. Li, L. Shen, K. Ma, and Y. Zheng, "Instance-aware self-supervised learning for nuclei segmentation," in International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, 2020, pp. 341-350.
A. Diba, V. Sharma, A. Pazandeh, H. Pirsiavash, and L. Van Gool, "Weakly supervised cascaded convolutional networks," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), July 2017.
X. Zhang, Y. Wei, J. Feng, Y. Yang, and T. S. Huang, "Adversarial complementary learning for weakly supervised object localization," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp. 1325-1334.
X. Ouyang, Z. Xue, Y. Zhan, X. S. Zhou, Q. Wang, Y. Zhou, Q. Wang, and J.-Z. Cheng, "Weakly supervised segmentation framework with uncertainty: A study on pneumothorax segmentation in chest x-ray," in International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, 2019, pp. 613-621.
B. Zhou, A. Khosla, A. Lapedriza, A. Oliva, and A. Torralba, "Learning deep features for discriminative localization," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 2921-2929.
R. R. Selvaraju, M. Cogswell, A. Das, R. Vedantam, D. Parikh, and D. Batra, "Gradcam: Visual explanations from deep networks via gradient-based localization," in Proceedings of the IEEE international conference on computer vision, 2017, pp. 618-626.
A. Chattopadhay, A. Sarkar, P. Howlader, and V. N. Balasubramanian, "Grad-cam++: Generalized gradient-based visual explanations for deep convolutional networks," in 2018 IEEE winter conference on applications of computer vision (WACV). IEEE, 2018, pp. 839-847.
R. Fu, Q. Hu, X. Dong, Y. Guo, Y. Gao, and B. Li, "Axiom-based grad-cam: Towards accurate visualization and explanation of cnns," 2020.
P.-T. Jiang, C.-B. Zhang, Q. Hou, M.-M. Cheng, and Y. Wei, "Layercam: Exploring hierarchical class activation maps for localization," IEEE Transactions on Image Processing, vol. 30, pp. 5875-5888, 2021.
Y. Shen, N. Wu, J. Phang, J. Park, K. Liu, S. Tyagi, L. Heacock, S. G. Kim, L. Moy, K. Cho et al., "An interpretable classifier for high-resolution breast cancer screening images utilizing weakly supervised localization," Medical image analysis, vol. 68, p. 101908, 2021.
H. T. Nguyen, H. Q. Nguyen, H. H. Pham, K. Lam, L. T. Le, M. Dao, and V. Vu, "Vindr-mammo: A large-scale benchmark dataset for computer-aided diagnosis in full-field digital mammography," medRxiv, 2022. [Online]. Available: https://www.medrxiv.org/content/early/2022/03/10/2022.03.07.22272009
G. Liang, X. Wang, Y. Zhang, and N. Jacobs, "Weakly-supervised self-training for breast cancer localization," in 2020 42nd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC). IEEE, 2020, pp. 1124-1127.
R. Bakalo, R. Ben-Ari, and J. Goldberger, "Classification and detection in mammograms with weak supervision via dual branch deep neural net," in 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019). IEEE, 2019, pp. 1905- 1909.
W. Zhu, Q. Lou, Y. S. Vang, and X. Xie, "Deep multi-instance networks with sparse label assignment for whole mammogram classification," in International conference on medical image computing and computer-assisted intervention. Springer, 2017, pp. 603-611.
M. Dundar, B. Krishnapuram, R. Rao, and G. Fung, "Multiple instance learning for computer aided diagnosis," Advances in neural information processing systems, vol. 19, 2006.
Z. Qin, D. Kim, and T. Gedeon, "Neural network classifier as mutual information estimator," https://github.com/ZhenyueQin/Research-Softmax-with-Mutual-Information, 2021.
X. Wang, Y. Peng, L. Lu, Z. Lu, M. Bagheri, and R. M. Summers, "Chestx-ray8: Hospital-scale chest x-ray database and benchmarks on weakly-supervised classification and localization of common thorax diseases," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2017, pp. 2097-2106.
P. Rajpurkar, J. Irvin, R. L. Ball, K. Zhu, B. Yang, H. Mehta, T. Duan, D. Ding, A. Bagul, C. P. Langlotz et al., "Deep learning for chest radiograph diagnosis: A retrospective comparison of the chexnext algorithm to practicing radiologists," PLoS medicine, vol. 15, no. 11, p. e1002686, 2018.
S. Poppi, M. Cornia, L. Baraldi, and R. Cucchiara, "Revisiting the evaluation of class activation mapping for explainability: A novel metric and experimental analysis," in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021, pp. 2299-2304.
R. R. Selvaraju, M. Cogswell, A. Das, R. Vedantam, D. Parikh, and D. Batra, "Gradcam: Visual explanations from deep networks via gradient-based localization," in Proceedings of the IEEE international conference on computer vision, 2017, pp. 618-626.
N. Wu, J. Phang, J. Park, Y. Shen, S. G. Kim, L. Heacock, L. Moy, K. Cho, and K. J. Geras, "The nyu breast cancer screening dataset v1. 0," New York Univ., New York, NY, USA, Tech. Rep, 2019.
K. He, X. Zhang, S. Ren, and J. Sun, "Deep residual learning for image recognition," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 770-778.
J. Gildenblat and contributors, "Pytorch library for cam methods," https://github.com/jacobgil/pytorch-grad-cam, 2021.
D. Ribli, A. Horváth, Z. Unger, P. Pollner, and I. Csabai, "Detecting and classifying lesions in mammograms with deep learning," Scientific reports, vol. 8, no. 1, pp. 1-7, 2018.
L. Shen, L. R. Margolies, J. H. Rothstein, E. Fluder, R. McBride, and W. Sieh, "Deep learning to improve breast cancer detection on screening mammography," Scientific reports, vol. 9, no. 1, pp. 1-12, 2019.
H. Jung, B. Kim, I. Lee, M. Yoo, J. Lee, S. Ham, O. Woo, and J. Kang, "Detection of masses in mammograms using a one-stage object detector based on a deep convolutional neural network," PloS one, vol. 13, no. 9, p. e0203355, 2018.
R. Agarwal, O. Díaz, M. H. Yap, X. Lladó, and R. Martí, "Deep learning for mass detection in full field digital mammograms," Computers in biology and medicine, vol. 121, p. 103774, 2020.
Published
2022-11-28
How to Cite
SAMPAIO, Vicente; CORDEIRO, Filipe R..
A Study on Class Activation Map Methods to Detect Masses in Mammography Images using Weakly Supervised Learning. In: NATIONAL MEETING ON ARTIFICIAL AND COMPUTATIONAL INTELLIGENCE (ENIAC), 19. , 2022, Campinas/SP.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2022
.
p. 437-448.
ISSN 2763-9061.
DOI: https://doi.org/10.5753/eniac.2022.227318.
