Convolutional architectures with LSTM and TCN to embolism classification: exploring dependency between data

• Luiz G. K. Zanini USP
• Aldomar P. S. Silva USP
• Felipe V. de Almeida USP
• Fátima L. S. N. Marques USP
• Anna H. R. Costa USP

DOI: https://doi.org/10.5753/eniac.2022.227585

Resumo

Pulmonary Embolism is an affection caused by obstruction of the pulmonary artery or one of its branches. This condition imposes a high mortality incidence, in the United States approximately 100.000 deaths per year. Computed Tomography Pulmonary Angiography is a radiologic modality and an essential technology for diagnosing this disease, providing a series of axial images. We trained two Convolutional Neural Networks (Efficient Net B0 and Resnet 3D 18) in the RSNA-STR Computed Tomography Pulmonary Angiography Dataset to identify this affection. After training these Convolutional Neural Networks, we added a new layer to the architecture by exploring the dependency between the images along the exam using Long Short-Term Memory or Temporal Convolutional Networks. With the models trained and tested, we compared these different approaches using different metrics. As a result, the Temporal Convolutional Network approach with Resnet 3D 18 improved significantly compared to the results found in the other methods. The main contribution of this work was to observe how different combinations of architectures can help classify Computed Tomography Pulmonary Angiography.

Referências

Aurelio, Y. S., de Almeida, G. M., de Castro, C. L., and Braga, A. P. (2019). Learning from Imbalanced Data Sets with Weighted Cross-Entropy Function. Neural Processing Letters, 50(2):1937-1949.

Bai, S., Kolter, J. Z., and Koltun, V. (2018). An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling. arXiv preprint arXiv:1803.01271.

Colak, E., Kitamura, F. C., Hobbs, S. B., Wu, C. C., Lungren, M. P., Prevedello, L. M., Kalpathy-Cramer, J., Ball, R. L., Shih, G., Stein, A., Halabi, S. S., Altinmakas, E., Law, M., Kumar, P., Manzalawi, K. A., Nelson Rubio, D. C., Sechrist, J. W., Germaine, P., Lopez, E. C., Amerio, T., Gupta, P., Jain, M., Kay, F. U., Lin, C. T., Sen, S., Revels, J. W., Brussaard, C. C., Mongan, J., Abdala, N., Bearce, B., Carrete, H., Dogan, H., Huang, S.-C., Crivellaro, P., Dincler, S., Kavnoudias, H., Lee, R., Lin, H.-M., Salehinejad, H., Samorodova, O., Rodrigues dos Santos, E., Seah, J., Zia, A., Arteaga, V. A., Batra, K., Castelli von Atzingen, A., Chacko, A., DiDomenico, P. B., Gill, R. R., Hafez, M. A., John, S., Karl, R. L., Kanne, J. P., Mathilakath Nair, R. V., McDermott, S., Mittal, P. K., Mumbower, A., Lee, C., Orausclio, P. J., Palacio, D., Pozzessere, C., Rajiah, P., Ramos, O. A., Rodriguez, S., Shaaban, M. N., Shah, P. N., Son, H., Sonavane, S. K., Spieler, B., Tsai, E., Vásquez, A., Vijayakumar, D., Wali, P. P., Wand, A., and Zamora Endara, G. E. (2021). The RSNA Pulmonary Embolism CT Dataset. Radiology: Artificial Intelligence, 3(2):e200254. Publisher: Radiological Society of North America.

Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., and Fei-Fei, L. (2009). ImageNet: A large-scale hierarchical image database. In 2009 IEEE Conference on Computer Vision and Pattern Recognition, pages 248-255. ISSN: 1063-6919.

DenOtter, T. D. and Schubert, J. (2022). Hounsfield Unit. In StatPearls. StatPearls Publishing.

Huang, Z., Xu, W., and Yu, K. (2015). Bidirectional LSTM-CRF models for Sequence Tagging. arXiv preprint arXiv:1508.01991.

Huhtanen, H., Nyman, M., Mohsen, T., Virkki, A., Karlsson, A., and Hirvonen, J. (2022). Automated detection of pulmonary embolism from CT-angiograms using deep learning. BMC Medical Imaging, 22(1):43.

Kwok, C. S., Wong, C. W., Lovatt, S., Myint, P. K., and Loke, Y. K. (2022). Misdiagnosis of pulmonary embolism and missed pulmonary embolism: A systematic review of the literature. Health Sciences Review, 3:100022.

Latha, R., R. Sreekanth, G. R., Suganthe, R., and Selvaraj, R. E. (2021). A survey on the applications of Deep Neural Networks. In 2021 International Conference on Computer Communication and Informatics (ICCCI), pages 1-3. ISSN: 2329-7190.

Ma, X., Ferguson, E. C., Jiang, X., Savitz, S. I., and Shams, S. (2022). A multitask deep learning approach for pulmonary embolism detection and identification. Scientific Reports, 12(1):13087.

MetaAI (2022). PyTorch. https://pytorch.org/. Acessed on 10-Aug-2022.

Pang, G., Aggarwal, C., Shen, C., and Sebe, N. (2022). Editorial Deep Learning for Anomaly Detection. IEEE Transactions on Neural Networks and Learning Systems, 33(6):2282-2286.

Peng, Y., Li, C., Wang, K., Gao, Z., and Yu, R. (2020). Examining imbalanced classification algorithms in predicting real-time traffic crash risk. Accident Analysis & Prevention, 144:105610.

Righini, M., Robert-Ebadi, H., and Le Gal, G. (2017). Diagnosis of acute pulmonary embolism. Journal of Thrombosis and Haemostasis, 15(7):1251-1261.

RSNA (2020). RSNA STR Pulmonary Embolism Detection. https://kaggle.com/competitions/rsna-str-pulmonary-embolism-detection. Acessed on 10-Aug-2022.

Ruopp, M. D., Perkins, N. J., Whitcomb, B. W., and Schisterman, E. F. (2008). Youden Index and Optimal Cut-Point Estimated from Observations Affected by a Lower Limit of Detection. Biometrical journal. Biometrische Zeitschrift, 50(3):419-430.

Shorten, C. and Khoshgoftaar, T. M. (2019). A survey on Image Data Augmentation for Deep Learning. Journal of Big Data, 6(1):60.

Tan, M. and Le, Q. (2019). Efficientnet: Rethinking model scaling for convolutional neural networks. In International conference on machine learning, pages 6105-6114. PMLR.

Tran, D., Wang, H., Torresani, L., Ray, J., LeCun, Y., and Paluri, M. (2018). A closer look at spatiotemporal convolutions for action recognition. In Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, pages 6450-6459.

Wang, F. and Tax, D. M. (2016). Survey on the attention based RNN model and its applications in computer vision. arXiv preprint arXiv:1601.06823.

Wittram, C., Maher, M. M., Yoo, A. J., Kalra, M. K., Shepard, J.-A. O., and McLoud, T. C. (2004). CT Angiography of Pulmonary Embolism: Diagnostic Criteria and Causes of Misdiagnosis. RadioGraphics, 24(5):1219-1238. Publisher: Radiological Society of North America.