A Study on Class Activation Map Methods to Detect Masses in Mammography Images using Weakly Supervised Learning
Resumo
Nos últimos anos, modelos de aprendizado fracamente supervisionado têm auxiliado na detecção de lesões em imagens de mamografia, diminuindo a necessidade de anotações de pixels na imagem. A maioria dos modelos na literatura se baseia no uso de mapas de ativação CAM, não sendo explorado o uso de outros modelos de ativação para detecção em imagens de mamografia. Este trabalho apresenta um estudo do uso de diferentes métodos de mapas de ativação do estado da arte, aplicados para treinamento fracamente supervisionado em imagens de mamografia. Neste estudo, comparamos os métodos CAM, GradCAM, GradCAM++, XGradCAM e LayerCAM, utilizando a rede GMIC para detectar a presença de lesões em imagens de mamografia. A avaliação é feita utilizando a base VinDr-Mammo, utilizando as métricas de Acurácia, TPR, FNR e FPPI. Resultados mostram que o uso de diferentes estratégias de mapas de ativação nas etapas de treino e teste melhoram o resultado do modelo. Com isso, melhoramos os resultados do método GMIC, reduzindo o valor de FPPI e aumentando o valor de TPR.
Referências
World Health Organization, "Breast cancer screening," 2017, disponivel em: http://www.who.int/cancer/prevention/diagnosis-screening/breast-cancer/en/, acessado em 08 dez. 2017.
P. Autier, M. Boniol, R. Middleton, J.-F. Doré, C. Héry, T. Zheng, and A. Gavin, "Advanced breast cancer incidence following population-based mammographic screening," Annals of Oncology, pp. 600-633, 2011.
D. S. Deshpande, A. M. Rajurkar, and R. M. Manthalkar, "Medical image analysis an attempt for mammogram classification using texture based association rule mining," in Computer Vision, Pattern Recognition, Image Processing and Graphics (NCV-PRIPG), 2013 Fourth National Conference on. IEEE, 2013, pp. 1-5.
M. H. Hesamian, W. Jia, X. He, and P. Kennedy, "Deep learning techniques for medical image segmentation: achievements and challenges," Journal of digital imaging, vol. 32, no. 4, pp. 582-596, 2019.
S. S. Yadav and S. M. Jadhav, "Deep convolutional neural network based medical image classification for disease diagnosis," Journal of Big Data, vol. 6, no. 1, pp. 1-18, 2019.
Y. Shen, N. Wu, J. Phang, J. Park, K. Liu, S. Tyagi, L. Heacock, S. G. Kim, L. Moy, K. Cho et al., "An interpretable classifier for high-resolution breast cancer screening images utilizing weakly supervised localization," Medical image analysis, vol. 68, p. 101908, 2021.
S.-T. Tran, C.-H. Cheng, T.-T. Nguyen, M.-H. Le, and D.-G. Liu, "Tmd-unet: Triple-unet with multi-scale input features and dense skip connection for medical image segmentation," in Healthcare, vol. 9, no. 1. Multidisciplinary Digital Publishing Institute, 2021, p. 54.
X. Xie, J. Chen, Y. Li, L. Shen, K. Ma, and Y. Zheng, "Instance-aware self-supervised learning for nuclei segmentation," in International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, 2020, pp. 341-350.
A. Diba, V. Sharma, A. Pazandeh, H. Pirsiavash, and L. Van Gool, "Weakly supervised cascaded convolutional networks," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), July 2017.
X. Zhang, Y. Wei, J. Feng, Y. Yang, and T. S. Huang, "Adversarial complementary learning for weakly supervised object localization," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp. 1325-1334.
X. Ouyang, Z. Xue, Y. Zhan, X. S. Zhou, Q. Wang, Y. Zhou, Q. Wang, and J.-Z. Cheng, "Weakly supervised segmentation framework with uncertainty: A study on pneumothorax segmentation in chest x-ray," in International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, 2019, pp. 613-621.
B. Zhou, A. Khosla, A. Lapedriza, A. Oliva, and A. Torralba, "Learning deep features for discriminative localization," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 2921-2929.
R. R. Selvaraju, M. Cogswell, A. Das, R. Vedantam, D. Parikh, and D. Batra, "Gradcam: Visual explanations from deep networks via gradient-based localization," in Proceedings of the IEEE international conference on computer vision, 2017, pp. 618-626.
A. Chattopadhay, A. Sarkar, P. Howlader, and V. N. Balasubramanian, "Grad-cam++: Generalized gradient-based visual explanations for deep convolutional networks," in 2018 IEEE winter conference on applications of computer vision (WACV). IEEE, 2018, pp. 839-847.
R. Fu, Q. Hu, X. Dong, Y. Guo, Y. Gao, and B. Li, "Axiom-based grad-cam: Towards accurate visualization and explanation of cnns," 2020.
P.-T. Jiang, C.-B. Zhang, Q. Hou, M.-M. Cheng, and Y. Wei, "Layercam: Exploring hierarchical class activation maps for localization," IEEE Transactions on Image Processing, vol. 30, pp. 5875-5888, 2021.
Y. Shen, N. Wu, J. Phang, J. Park, K. Liu, S. Tyagi, L. Heacock, S. G. Kim, L. Moy, K. Cho et al., "An interpretable classifier for high-resolution breast cancer screening images utilizing weakly supervised localization," Medical image analysis, vol. 68, p. 101908, 2021.
H. T. Nguyen, H. Q. Nguyen, H. H. Pham, K. Lam, L. T. Le, M. Dao, and V. Vu, "Vindr-mammo: A large-scale benchmark dataset for computer-aided diagnosis in full-field digital mammography," medRxiv, 2022. [Online]. Available: https://www.medrxiv.org/content/early/2022/03/10/2022.03.07.22272009
G. Liang, X. Wang, Y. Zhang, and N. Jacobs, "Weakly-supervised self-training for breast cancer localization," in 2020 42nd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC). IEEE, 2020, pp. 1124-1127.
R. Bakalo, R. Ben-Ari, and J. Goldberger, "Classification and detection in mammograms with weak supervision via dual branch deep neural net," in 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019). IEEE, 2019, pp. 1905- 1909.
W. Zhu, Q. Lou, Y. S. Vang, and X. Xie, "Deep multi-instance networks with sparse label assignment for whole mammogram classification," in International conference on medical image computing and computer-assisted intervention. Springer, 2017, pp. 603-611.
M. Dundar, B. Krishnapuram, R. Rao, and G. Fung, "Multiple instance learning for computer aided diagnosis," Advances in neural information processing systems, vol. 19, 2006.
Z. Qin, D. Kim, and T. Gedeon, "Neural network classifier as mutual information estimator," https://github.com/ZhenyueQin/Research-Softmax-with-Mutual-Information, 2021.
X. Wang, Y. Peng, L. Lu, Z. Lu, M. Bagheri, and R. M. Summers, "Chestx-ray8: Hospital-scale chest x-ray database and benchmarks on weakly-supervised classification and localization of common thorax diseases," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2017, pp. 2097-2106.
P. Rajpurkar, J. Irvin, R. L. Ball, K. Zhu, B. Yang, H. Mehta, T. Duan, D. Ding, A. Bagul, C. P. Langlotz et al., "Deep learning for chest radiograph diagnosis: A retrospective comparison of the chexnext algorithm to practicing radiologists," PLoS medicine, vol. 15, no. 11, p. e1002686, 2018.
S. Poppi, M. Cornia, L. Baraldi, and R. Cucchiara, "Revisiting the evaluation of class activation mapping for explainability: A novel metric and experimental analysis," in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021, pp. 2299-2304.
R. R. Selvaraju, M. Cogswell, A. Das, R. Vedantam, D. Parikh, and D. Batra, "Gradcam: Visual explanations from deep networks via gradient-based localization," in Proceedings of the IEEE international conference on computer vision, 2017, pp. 618-626.
N. Wu, J. Phang, J. Park, Y. Shen, S. G. Kim, L. Heacock, L. Moy, K. Cho, and K. J. Geras, "The nyu breast cancer screening dataset v1. 0," New York Univ., New York, NY, USA, Tech. Rep, 2019.
K. He, X. Zhang, S. Ren, and J. Sun, "Deep residual learning for image recognition," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 770-778.
J. Gildenblat and contributors, "Pytorch library for cam methods," https://github.com/jacobgil/pytorch-grad-cam, 2021.
D. Ribli, A. Horváth, Z. Unger, P. Pollner, and I. Csabai, "Detecting and classifying lesions in mammograms with deep learning," Scientific reports, vol. 8, no. 1, pp. 1-7, 2018.
L. Shen, L. R. Margolies, J. H. Rothstein, E. Fluder, R. McBride, and W. Sieh, "Deep learning to improve breast cancer detection on screening mammography," Scientific reports, vol. 9, no. 1, pp. 1-12, 2019.
H. Jung, B. Kim, I. Lee, M. Yoo, J. Lee, S. Ham, O. Woo, and J. Kang, "Detection of masses in mammograms using a one-stage object detector based on a deep convolutional neural network," PloS one, vol. 13, no. 9, p. e0203355, 2018.
R. Agarwal, O. Díaz, M. H. Yap, X. Lladó, and R. Martí, "Deep learning for mass detection in full field digital mammograms," Computers in biology and medicine, vol. 121, p. 103774, 2020.
P. Autier, M. Boniol, R. Middleton, J.-F. Doré, C. Héry, T. Zheng, and A. Gavin, "Advanced breast cancer incidence following population-based mammographic screening," Annals of Oncology, pp. 600-633, 2011.
D. S. Deshpande, A. M. Rajurkar, and R. M. Manthalkar, "Medical image analysis an attempt for mammogram classification using texture based association rule mining," in Computer Vision, Pattern Recognition, Image Processing and Graphics (NCV-PRIPG), 2013 Fourth National Conference on. IEEE, 2013, pp. 1-5.
M. H. Hesamian, W. Jia, X. He, and P. Kennedy, "Deep learning techniques for medical image segmentation: achievements and challenges," Journal of digital imaging, vol. 32, no. 4, pp. 582-596, 2019.
S. S. Yadav and S. M. Jadhav, "Deep convolutional neural network based medical image classification for disease diagnosis," Journal of Big Data, vol. 6, no. 1, pp. 1-18, 2019.
Y. Shen, N. Wu, J. Phang, J. Park, K. Liu, S. Tyagi, L. Heacock, S. G. Kim, L. Moy, K. Cho et al., "An interpretable classifier for high-resolution breast cancer screening images utilizing weakly supervised localization," Medical image analysis, vol. 68, p. 101908, 2021.
S.-T. Tran, C.-H. Cheng, T.-T. Nguyen, M.-H. Le, and D.-G. Liu, "Tmd-unet: Triple-unet with multi-scale input features and dense skip connection for medical image segmentation," in Healthcare, vol. 9, no. 1. Multidisciplinary Digital Publishing Institute, 2021, p. 54.
X. Xie, J. Chen, Y. Li, L. Shen, K. Ma, and Y. Zheng, "Instance-aware self-supervised learning for nuclei segmentation," in International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, 2020, pp. 341-350.
A. Diba, V. Sharma, A. Pazandeh, H. Pirsiavash, and L. Van Gool, "Weakly supervised cascaded convolutional networks," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), July 2017.
X. Zhang, Y. Wei, J. Feng, Y. Yang, and T. S. Huang, "Adversarial complementary learning for weakly supervised object localization," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp. 1325-1334.
X. Ouyang, Z. Xue, Y. Zhan, X. S. Zhou, Q. Wang, Y. Zhou, Q. Wang, and J.-Z. Cheng, "Weakly supervised segmentation framework with uncertainty: A study on pneumothorax segmentation in chest x-ray," in International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, 2019, pp. 613-621.
B. Zhou, A. Khosla, A. Lapedriza, A. Oliva, and A. Torralba, "Learning deep features for discriminative localization," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 2921-2929.
R. R. Selvaraju, M. Cogswell, A. Das, R. Vedantam, D. Parikh, and D. Batra, "Gradcam: Visual explanations from deep networks via gradient-based localization," in Proceedings of the IEEE international conference on computer vision, 2017, pp. 618-626.
A. Chattopadhay, A. Sarkar, P. Howlader, and V. N. Balasubramanian, "Grad-cam++: Generalized gradient-based visual explanations for deep convolutional networks," in 2018 IEEE winter conference on applications of computer vision (WACV). IEEE, 2018, pp. 839-847.
R. Fu, Q. Hu, X. Dong, Y. Guo, Y. Gao, and B. Li, "Axiom-based grad-cam: Towards accurate visualization and explanation of cnns," 2020.
P.-T. Jiang, C.-B. Zhang, Q. Hou, M.-M. Cheng, and Y. Wei, "Layercam: Exploring hierarchical class activation maps for localization," IEEE Transactions on Image Processing, vol. 30, pp. 5875-5888, 2021.
Y. Shen, N. Wu, J. Phang, J. Park, K. Liu, S. Tyagi, L. Heacock, S. G. Kim, L. Moy, K. Cho et al., "An interpretable classifier for high-resolution breast cancer screening images utilizing weakly supervised localization," Medical image analysis, vol. 68, p. 101908, 2021.
H. T. Nguyen, H. Q. Nguyen, H. H. Pham, K. Lam, L. T. Le, M. Dao, and V. Vu, "Vindr-mammo: A large-scale benchmark dataset for computer-aided diagnosis in full-field digital mammography," medRxiv, 2022. [Online]. Available: https://www.medrxiv.org/content/early/2022/03/10/2022.03.07.22272009
G. Liang, X. Wang, Y. Zhang, and N. Jacobs, "Weakly-supervised self-training for breast cancer localization," in 2020 42nd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC). IEEE, 2020, pp. 1124-1127.
R. Bakalo, R. Ben-Ari, and J. Goldberger, "Classification and detection in mammograms with weak supervision via dual branch deep neural net," in 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019). IEEE, 2019, pp. 1905- 1909.
W. Zhu, Q. Lou, Y. S. Vang, and X. Xie, "Deep multi-instance networks with sparse label assignment for whole mammogram classification," in International conference on medical image computing and computer-assisted intervention. Springer, 2017, pp. 603-611.
M. Dundar, B. Krishnapuram, R. Rao, and G. Fung, "Multiple instance learning for computer aided diagnosis," Advances in neural information processing systems, vol. 19, 2006.
Z. Qin, D. Kim, and T. Gedeon, "Neural network classifier as mutual information estimator," https://github.com/ZhenyueQin/Research-Softmax-with-Mutual-Information, 2021.
X. Wang, Y. Peng, L. Lu, Z. Lu, M. Bagheri, and R. M. Summers, "Chestx-ray8: Hospital-scale chest x-ray database and benchmarks on weakly-supervised classification and localization of common thorax diseases," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2017, pp. 2097-2106.
P. Rajpurkar, J. Irvin, R. L. Ball, K. Zhu, B. Yang, H. Mehta, T. Duan, D. Ding, A. Bagul, C. P. Langlotz et al., "Deep learning for chest radiograph diagnosis: A retrospective comparison of the chexnext algorithm to practicing radiologists," PLoS medicine, vol. 15, no. 11, p. e1002686, 2018.
S. Poppi, M. Cornia, L. Baraldi, and R. Cucchiara, "Revisiting the evaluation of class activation mapping for explainability: A novel metric and experimental analysis," in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021, pp. 2299-2304.
R. R. Selvaraju, M. Cogswell, A. Das, R. Vedantam, D. Parikh, and D. Batra, "Gradcam: Visual explanations from deep networks via gradient-based localization," in Proceedings of the IEEE international conference on computer vision, 2017, pp. 618-626.
N. Wu, J. Phang, J. Park, Y. Shen, S. G. Kim, L. Heacock, L. Moy, K. Cho, and K. J. Geras, "The nyu breast cancer screening dataset v1. 0," New York Univ., New York, NY, USA, Tech. Rep, 2019.
K. He, X. Zhang, S. Ren, and J. Sun, "Deep residual learning for image recognition," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 770-778.
J. Gildenblat and contributors, "Pytorch library for cam methods," https://github.com/jacobgil/pytorch-grad-cam, 2021.
D. Ribli, A. Horváth, Z. Unger, P. Pollner, and I. Csabai, "Detecting and classifying lesions in mammograms with deep learning," Scientific reports, vol. 8, no. 1, pp. 1-7, 2018.
L. Shen, L. R. Margolies, J. H. Rothstein, E. Fluder, R. McBride, and W. Sieh, "Deep learning to improve breast cancer detection on screening mammography," Scientific reports, vol. 9, no. 1, pp. 1-12, 2019.
H. Jung, B. Kim, I. Lee, M. Yoo, J. Lee, S. Ham, O. Woo, and J. Kang, "Detection of masses in mammograms using a one-stage object detector based on a deep convolutional neural network," PloS one, vol. 13, no. 9, p. e0203355, 2018.
R. Agarwal, O. Díaz, M. H. Yap, X. Lladó, and R. Martí, "Deep learning for mass detection in full field digital mammograms," Computers in biology and medicine, vol. 121, p. 103774, 2020.
Publicado
28/11/2022
Como Citar
SAMPAIO, Vicente; CORDEIRO, Filipe R..
A Study on Class Activation Map Methods to Detect Masses in Mammography Images using Weakly Supervised Learning. In: ENCONTRO NACIONAL DE INTELIGÊNCIA ARTIFICIAL E COMPUTACIONAL (ENIAC), 19. , 2022, Campinas/SP.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2022
.
p. 437-448.
ISSN 2763-9061.
DOI: https://doi.org/10.5753/eniac.2022.227318.
