CNN Hyperparameter Optimization for Pulmonary Nodule Classification
Resumo
Redes Neurais Convolucionais (RNCs) são uma técnica poderosa para sistemas de diagnóstico auxiliado por computador, mas a configuração manual de redes complexas é inviável. A otimização automática de hiper-parâmetros é uma abordagem promissora, mas não há consenso sobre a técnica mais adequada. Neste trabalho, comparamos busca direta, probabilística e otimização bayesiana na otimização de RNCs 2D e 3D para classificação de nódulos pulmonares. Foram obtidas AUC de 0,88, sensibilidade de 87,03% e especificidade de 78,66%. Nossos experimentos demonstram o fraco desempenho da busca em grade, enquanto mostram que técnicas simples, como a busca aleatória, pode ter desempenho comparável a abordagens probabilísticas
Referências
Abadi, M., Barham, P., Chen, J., Chen, Z., Davis, A., Dean, J., Devin, M., Ghemawat, S., Irving, G., Isard, M., et al. (2016). Tensorflow: A system for large-scale machine learning. In 12th {USENIX} Symposium on Operating Systems Design and Implementation ({OSDI} 16), pages 265–283.
Armato, S. G., McLennan, G., Bidaut, L., McNitt-Gray, M. F., Meyer, C. R., Reeves, A. P., Zhao, B., Aberle, D. R., Henschke, C. I., Hoffman, E. A., et al. (2011). The lung image database consortium (lidc) and image database resource initiative (idri): a completed reference database of lung nodules on ct scans. Medical physics, 38(2):915– 931.
Bergstra, J. and Bengio, Y. (2012). Random search for hyper-parameter optimization. Journal of Machine Learning Research, 13(Feb):281–305.
Bergstra, J., Yamins, D., and Cox, D. D. (2013). Hyperopt: A python library for optimizing the hyperparameters of machine learning algorithms. In Proceedings of the 12th Python in science conference, pages 13–20. Citeseer.
Bergstra, J. S., Bardenet, R., Bengio, Y., and Kegl, B. (2011). Algorithms for hyper- parameter optimization. In Advances in neural information processing systems, pages 2546–2554.
Blandin Knight, S., Crosbie, P. A., Balata, H., Chudziak, J., Hussell, T., and Dive, C. (2017). Progress and prospects of early detection in lung cancer. Open biology, 7(9):170070.
Bray, F., Ferlay, J., Soerjomataram, I., Siegel, R. L., Torre, L. A., and Jemal, A. (2018). Global cancer statistics 2018: Globocan estimates of incidence and mortality worldwide for 36 cancers in 185 countries. CA: a cancer journal for clinicians, 68(6):394– 424.
Chollet, F. et al. (2015). Keras. https://github.com/fchollet/keras.
Chuquicusma, M. J., Hussein, S., Burt, J., and Bagci, U. (2018). How to fool radiologists with generative adversarial networks? a visual turing test for lung cancer diagnosis. In 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018), pages 240–244. IEEE.
Claesen, M. and De Moor, B. (2015). Hyperparameter search in machine learning. arXiv preprint arXiv:1502.02127.
da Silva, G. L., da Silva Neto, O. P., Silva, A. C., de Paiva, A. C., and Gattass, M. (2017). Lung nodules diagnosis based on evolutionary convolutional neural network. Multimedia Tools and Applications, 76(18):19039–19055.
Dey, R., Lu, Z., and Hong, Y. (2018). Diagnostic classification of lung nodules using 3d neural networks. In 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018), pages 774–778. IEEE.
Fawcett, T. (2006). An introduction to roc analysis. Pattern recognition letters, 27(8):861–874.
Ferreira, J. R., Oliveira, M. C., and de Azevedo-Marques, P. M. (2018). Characterization of pulmonary nodules based on features of margin sharpness and texture. Journal of digital imaging, 31(4):451–463.
Goodfellow, I., Bengio, Y., and Courville, A. (2016). Deep Learning. MIT Press. http:
//www.deeplearningbook.org.
Hua, K.-L., Hsu, C.-H., Hidayati, S. C., Cheng, W.-H., and Chen, Y.-J. (2015). Computeraided classification of lung nodules on computed tomography images via deep learning technique. OncoTargets and therapy, 8.
Kang, G., Liu, K., Hou, B., and Zhang, N. (2017). 3d multi-view convolutional neural networks for lung nodule classification. PloS one, 12(11):e0188290.
Kumar, D., Wong, A., and Clausi, D. A. (2015). Lung nodule classification using deep features in ct images. In 2015 12th Conference on Computer and Robot Vision, pages 133–138. IEEE.
Litjens, G., Kooi, T., Bejnordi, B. E., Setio, A. A. A., Ciompi, F., Ghafoorian, M., Van Der Laak, J. A., Van Ginneken, B., and Sanchez, C. I. (2017).á survey on deep learning in medical image analysis. Medical image analysis, 42:60–88.
Miikkulainen, R., Liang, J., Meyerson, E., Rawal, A., Fink, D., Francon, O., Raju, B., Shahrzad, H., Navruzyan, A., Duffy, N., et al. (2019). Evolving deep neural networks. In Artificial Intelligence in the Age of Neural Networks and Brain Computing, pages 293–312. Elsevier.
Montavon, G., Orr, G., and Muller, K.-R. (2012).¨ Neural networks: tricks of the trade, volume 7700. springer.
Onishi, Y., Teramoto, A., Tsujimoto, M., Tsukamoto, T., Saito, K., Toyama, H., Imaizumi, K., and Fujita, H. (2019). Automated pulmonary nodule classification in computed tomography images using a deep convolutional neural network trained by generative adversarial networks. BioMed research international, 2019.
Pumperla, M. (2019). Hyperas.
Shen, W., Zhou, M., Yang, F., Yang, C., and Tian, J. (2015). Multi-scale convolutional neural networks for lung nodule classification. In International Conference on Information Processing in Medical Imaging, pages 588–599. Springer.
Sun, W., Zheng, B., and Qian, W. (2016). Computer aided lung cancer diagnosis with deep learning algorithms. Medical Imaging 2016: Computer-Aided Diagnosis, 9785(March):97850Z.
World Health Organisation (2019). Vision impairment and blindness, Fact Sheet No282. http://www.who.int/mediacentre/factsheets/fs282/fr/, Last accessed on 2019-02-14.
Yang, Y., Feng, X., Chi, W., Li, Z., Duan, W., Liu, H., Liang, W., Wang, W., Chen, P., He, J., et al. (2018). Deep learning aided decision support for pulmonary nodules diagnosing: a review. Journal of thoracic disease, 10(Suppl 7):S867.
Zhu, W., Liu, C., Fan, W., and Xie, X. (2018). Deeplung: Deep 3d dual path nets for automated pulmonary nodule detection and classification. In 2018 IEEE Winter Conference on Applications of Computer Vision (WACV), pages 673–681. IEEE.
Armato, S. G., McLennan, G., Bidaut, L., McNitt-Gray, M. F., Meyer, C. R., Reeves, A. P., Zhao, B., Aberle, D. R., Henschke, C. I., Hoffman, E. A., et al. (2011). The lung image database consortium (lidc) and image database resource initiative (idri): a completed reference database of lung nodules on ct scans. Medical physics, 38(2):915– 931.
Bergstra, J. and Bengio, Y. (2012). Random search for hyper-parameter optimization. Journal of Machine Learning Research, 13(Feb):281–305.
Bergstra, J., Yamins, D., and Cox, D. D. (2013). Hyperopt: A python library for optimizing the hyperparameters of machine learning algorithms. In Proceedings of the 12th Python in science conference, pages 13–20. Citeseer.
Bergstra, J. S., Bardenet, R., Bengio, Y., and Kegl, B. (2011). Algorithms for hyper- parameter optimization. In Advances in neural information processing systems, pages 2546–2554.
Blandin Knight, S., Crosbie, P. A., Balata, H., Chudziak, J., Hussell, T., and Dive, C. (2017). Progress and prospects of early detection in lung cancer. Open biology, 7(9):170070.
Bray, F., Ferlay, J., Soerjomataram, I., Siegel, R. L., Torre, L. A., and Jemal, A. (2018). Global cancer statistics 2018: Globocan estimates of incidence and mortality worldwide for 36 cancers in 185 countries. CA: a cancer journal for clinicians, 68(6):394– 424.
Chollet, F. et al. (2015). Keras. https://github.com/fchollet/keras.
Chuquicusma, M. J., Hussein, S., Burt, J., and Bagci, U. (2018). How to fool radiologists with generative adversarial networks? a visual turing test for lung cancer diagnosis. In 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018), pages 240–244. IEEE.
Claesen, M. and De Moor, B. (2015). Hyperparameter search in machine learning. arXiv preprint arXiv:1502.02127.
da Silva, G. L., da Silva Neto, O. P., Silva, A. C., de Paiva, A. C., and Gattass, M. (2017). Lung nodules diagnosis based on evolutionary convolutional neural network. Multimedia Tools and Applications, 76(18):19039–19055.
Dey, R., Lu, Z., and Hong, Y. (2018). Diagnostic classification of lung nodules using 3d neural networks. In 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018), pages 774–778. IEEE.
Fawcett, T. (2006). An introduction to roc analysis. Pattern recognition letters, 27(8):861–874.
Ferreira, J. R., Oliveira, M. C., and de Azevedo-Marques, P. M. (2018). Characterization of pulmonary nodules based on features of margin sharpness and texture. Journal of digital imaging, 31(4):451–463.
Goodfellow, I., Bengio, Y., and Courville, A. (2016). Deep Learning. MIT Press. http:
//www.deeplearningbook.org.
Hua, K.-L., Hsu, C.-H., Hidayati, S. C., Cheng, W.-H., and Chen, Y.-J. (2015). Computeraided classification of lung nodules on computed tomography images via deep learning technique. OncoTargets and therapy, 8.
Kang, G., Liu, K., Hou, B., and Zhang, N. (2017). 3d multi-view convolutional neural networks for lung nodule classification. PloS one, 12(11):e0188290.
Kumar, D., Wong, A., and Clausi, D. A. (2015). Lung nodule classification using deep features in ct images. In 2015 12th Conference on Computer and Robot Vision, pages 133–138. IEEE.
Litjens, G., Kooi, T., Bejnordi, B. E., Setio, A. A. A., Ciompi, F., Ghafoorian, M., Van Der Laak, J. A., Van Ginneken, B., and Sanchez, C. I. (2017).á survey on deep learning in medical image analysis. Medical image analysis, 42:60–88.
Miikkulainen, R., Liang, J., Meyerson, E., Rawal, A., Fink, D., Francon, O., Raju, B., Shahrzad, H., Navruzyan, A., Duffy, N., et al. (2019). Evolving deep neural networks. In Artificial Intelligence in the Age of Neural Networks and Brain Computing, pages 293–312. Elsevier.
Montavon, G., Orr, G., and Muller, K.-R. (2012).¨ Neural networks: tricks of the trade, volume 7700. springer.
Onishi, Y., Teramoto, A., Tsujimoto, M., Tsukamoto, T., Saito, K., Toyama, H., Imaizumi, K., and Fujita, H. (2019). Automated pulmonary nodule classification in computed tomography images using a deep convolutional neural network trained by generative adversarial networks. BioMed research international, 2019.
Pumperla, M. (2019). Hyperas.
Shen, W., Zhou, M., Yang, F., Yang, C., and Tian, J. (2015). Multi-scale convolutional neural networks for lung nodule classification. In International Conference on Information Processing in Medical Imaging, pages 588–599. Springer.
Sun, W., Zheng, B., and Qian, W. (2016). Computer aided lung cancer diagnosis with deep learning algorithms. Medical Imaging 2016: Computer-Aided Diagnosis, 9785(March):97850Z.
World Health Organisation (2019). Vision impairment and blindness, Fact Sheet No282. http://www.who.int/mediacentre/factsheets/fs282/fr/, Last accessed on 2019-02-14.
Yang, Y., Feng, X., Chi, W., Li, Z., Duan, W., Liu, H., Liang, W., Wang, W., Chen, P., He, J., et al. (2018). Deep learning aided decision support for pulmonary nodules diagnosing: a review. Journal of thoracic disease, 10(Suppl 7):S867.
Zhu, W., Liu, C., Fan, W., and Xie, X. (2018). Deeplung: Deep 3d dual path nets for automated pulmonary nodule detection and classification. In 2018 IEEE Winter Conference on Applications of Computer Vision (WACV), pages 673–681. IEEE.
Publicado
15/09/2020
Como Citar
JATOBÁ, Anthony; LIMA, Lucas; AMORIM, Lucas; OLIVEIRA, Marcelo.
CNN Hyperparameter Optimization for Pulmonary Nodule Classification. In: SIMPÓSIO BRASILEIRO DE COMPUTAÇÃO APLICADA À SAÚDE (SBCAS), 20. , 2020, Evento Online.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2020
.
p. 25-36.
ISSN 2763-8952.
DOI: https://doi.org/10.5753/sbcas.2020.11499.
