Semantic Segmentation with Multi-Source Domain Adaptation for Radiological Images

• Hugo Oliveira UFMG
• Arnaldo de A. Araújo UFMG
• Jefersson A. dos Santos UFMG

DOI: https://doi.org/10.5753/sibgrapi.est.2021.20020

Resumo

Differences in digitization equipment and techniques in radiology may hamper the use of data-driven deep learning approaches. In order to mitigate this limitation, in this work we merge generative image translation networks with supervised semantic segmentation architectures, yielding two semisupervised methods for domain adaptation in medical images. We compare our methods with traditional baselines in the literature using 3 image domains, 16 datasets and 8 segmentation tasks organized into three sets of experiments. Analysis of the results showed that the proposed methods for Domain Adaptation often reached Jaccard scores of 0.9 or higher in unsupervised or semi-supervised settings. We observe that unsupervised domain adaptation performance is close to the performance of fully supervised adaptation in most cases, bridging an important gap in the efficacy of neural networks between labeled and unlabeled datasets.

Referências

G. Litjens, T. Kooi, B. E. Bejnordi, A. A. A. Setio, F. Ciompi, M. Ghafoorian, J. A. van der Laak, B. van Ginneken, and C. I. Sánchez, “A Survey on Deep Learning in Medical Image Analysis,” Medical Image Analysis, vol. 42, pp. 60–88, 2017.

T. Zhou, S. Ruan, and S. Canu, “A Review: Deep Learning for Medical Image Segmentation Using Multi-Modality Fusion,” Array, vol. 3, p. 100004, 2019.

G. Haskins, U. Kruger, and P. Yan, “Deep Learning in Medical Image Registration: A Survey,” Machine Vision and Applications, vol. 31, no. 1, p. 8, 2020.

J. Zhang, W. Li, and P. Ogunbona, “Transfer Learning For Cross-Dataset Recognition: A Survey,” 2017.

Z. Cao, L. Ma, M. Long, and J. Wang, “Partial Adversarial Domain Adaptation,” in ECCV, 2018, pp. 135–150.

W. Zhang, W. Ouyang, W. Li, and D. Xu, “Collaborative and Adversarial Network for Unsupervised Domain Adaptation,” in CVPR, 2018, pp. 3801–3809.

M. Yamada, L. Sigal, and Y. Chang, “Domain Adaptation for Structured Regression,” International Journal of Computer Vision, vol. 109, no. 1-2, pp. 126–145, 2014.

Y. Wu and Q. Ji, “Constrained Deep Transfer Feature Learning and its Applications,” in CVPR, 2016, pp. 5101–5109.

P. Koniusz, Y. Tas, and F. Porikli, “Domain Adaptation by Mixture of Alignments of Second-or Higher-Order Scatter Tensors,” in CVPR, vol. 2, 2017.

P. Isola, J.-Y. Zhu, T. Zhou, and A. A. Efros, “Image-to-Image Translation with Conditional Adversarial Networks,” in CVPR. IEEE, 2017, pp. 5967–5976.

J.-Y. Zhu, T. Park, P. Isola, and A. A. Efros, “Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks,” in ICCV, 2017.

I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio, “Generative Adversarial Nets,” in NIPS, 2014, pp. 2672–2680.

H. Oliveira, “Semantic segmentation with multi-source domain adaptation for radiological images,” Ph.D. dissertation, Universidade Federal de Minas Gerais, 2020.

H. N. Oliveira and J. A. dos Santos, “Deep Transfer Learning for Segmentation of Anatomical Structures in Chest Radiographs,” in SIBGRAPI. IEEE, 2018.

H. N. Oliveira, E. Ferreira, and J. A. Dos Santos, “Truly Generalizable Radiograph Segmentation With Conditional Domain Adaptation,” IEEE Access, vol. 8, pp. 84 037–84 062, 2020.

H. Oliveira, V. Mota, A. M. Machado, and J. A. dos Santos, “From 3d to 2d: Transferring knowledge for rib segmentation in chest x-rays,” PRL, vol. 140, pp. 10–17, 2020.

M.-Y. Liu, T. Breuel, and J. Kautz, “Unsupervised Image-to-Image Translation Networks,” in NIPS, 2017, pp. 700–708.

X. Huang, M.-Y. Liu, S. Belongie, and J. Kautz, “Multimodal Unsupervised Image-to-Image Translation,” in ECCV, 2018, pp. 172–189.

J. Hoffman, E. Tzeng, T. Park, J.-Y. Zhu, P. Isola, K. Saenko, A. Efros, and T. Darrell, “CyCADA: Cycle-Consistent Adversarial Domain Adaptation,” in ICML, 2018, pp. 1994–2003.

Z. Murez, S. Kolouri, D. Kriegman, R. Ramamoorthi, and K. Kim, “Image to Image Translation for Domain Adaptation,” in CVPR, 2018, pp. 4500–4509.

Z. Wu, X. Han, Y.-L. Lin, M. Gokhan Uzunbas, T. Goldstein, S. Nam Lim, and L. S. Davis, “DCAN: Dual Channel-wise Alignment Networks for Unsupervised Scene Adaptation,” in ECCV, 2018, pp. 518– 534.

Y. Zou, Z. Yu, B. Vijaya Kumar, and J. Wang, “Unsupervised Domain Adaptation for Semantic Segmentation via Class-Balanced Self- Training,” in ECCV, 2018, pp. 289–305.

S. Candemir, S. Jaeger, S. Antani, U. Bagci, L. R. Folio, Z. Xu, and G. Thoma, “Atlas-based Rib-Bone Detection in Chest X-Rays,” Computerized Medical Imaging and Graphics, vol. 51, pp. 32–39, 2016.

Y. Zhang, S. Miao, T. Mansi, and R. Liao, “Task Driven Generative Modeling for Unsupervised Domain Adaptation: Application to X-Ray Image Segmentation,” in MICCAI. Springer, 2018, pp. 599–607.

J. Shiraishi, S. Katsuragawa, J. Ikezoe, T. Matsumoto, T. Kobayashi, K.-i. Komatsu, M. Matsui, H. Fujita, Y. Kodera, and K. Doi, “Development of a Digital Image Database for Chest Radiographs with and without a Lung Nodule: Receiver Operating Characteristic Analysis of Radiologists’ Detection of Pulmonary Nodules,” American Journal of Roentgenology, vol. 174, no. 1, pp. 71–74, 2000.

S. Jaeger, S. Candemir, S. Antani, Y.-X. J. Wáng, P.-X. Lu, and G. Thoma, “Two Public Chest X-Ray Datasets for Computer-Aided Screening of Pulmonary Diseases,” Quantitative Imaging in Medicine and Surgery, vol. 4, no. 6, p. 475, 2014.

H. N. Oliveira, J. A. Dos Santos, M. C. de Melo, T. G. do Rêgo, and L. V. Batista, “Information theory-based detection of noisy bit planes in medical images,” in SIBGRAPI. IEEE, 2016, pp. 32–39.

E. Ferreira, H. Oliveira, M. S. Alvim, and J. A. dos Santos, “A Comparative Study on Unsupervised Domain Adaptation for Coffee Crop Mapping,” in CIARP. Springer, 2018, pp. 72–80.

H. N. Oliveira, C. S. Avelar, A. M. C. Machado, A. A. Araujo, and J. A. dos Santos, “Exploring Deep-Based Approaches for Semantic Segmentation of Mammographic Images,” in CIARP. Springer, 2018.

C. C. da Silva, K. Nogueira, H. N. Oliveira, and J. A. dos Santos, “Towards open-set semantic segmentation of aerial images,” in LAGIRS. IEEE, 2020, pp. 16–21.

V. F. Mota, H. N. de Oliveira, S. Scalzo, D. Dittz, R. J. Santos, J. A. dos Santos, and A. d. A. Araújo, “From Video Pornography to Cancer Cells: A Tensor Framework for Spatiotemporal Description,” MTAP, pp. 1–31, 2020.