Self-Supervised Image Denoising Methods: an Application in Fetal MRI
Resumo
The process of image denoising in magnetic resonance imaging (MRI) is more and more common and important in the medical area. However, it is usual that state-of-the-art deep learning methods require pair images (clean and noisy ones) to train the models which poses limitations in practice. In this sense, this work applied two recent techniques that do not need a clean image to train the models and reached good results for denoising tasks. We applied the NOISE2NOISE (N2N) and the NOISE2VOID (N2V) learning approaches and compared the results for denoising tasks using a fetal MRI dataset. The results showed that the N2N method outperformed the N2V one, considering the Peak Signal-to-Noise Ratio (PSNR), Root Mean Squared Error (RMSE) evaluation metrics, and visual analysis.
Palavras-chave:
image denoising, medical area, NOISE2NOISE, NOISE2VOID, fetal MRI
Referências
S. Kollem, K. R. Reddy, and D. S. Rao, ‘A novel diffusivity function-based image denoising for MRI medical images’, Multimed. Tools Appl., Mar. 2023, doi: 10.1007/s11042-023-14457-3.
A. Lim, J. Lo, M. W. Wagner, B. Ertl-Wagner, and D. Sussman, ‘Motion artifact correction in fetal MRI based on a Generative Adversarial network method’, Biomed. Signal Process. Control, vol. 81, p. 104484, Mar. 2023, doi: 10.1016/j.bspc.2022.104484.
J.-J. Huang and P. L. Dragotti, ‘WINNet: Wavelet-Inspired Invertible Network for Image Denoising’, IEEE Trans. Image Process., vol. 31, pp. 4377–4392, 2022, doi: 10.1109/TIP.2022.3184845.
K. Prabakaran, N. Lalithamani, and P. Kiruthika, ‘Speckle Reduction Algorithm for Medical Ultrasound Imaging’, Asian J. Res. Soc. Sci. Humanit., vol. 6, no. 6, p. 148, 2016, doi: 10.5958/2249-7315.2016.00202.1.
R. S. Thakur, S. Chatterjee, R. N. Yadav, and L. Gupta, ‘Chapter 5 - Medical image denoising using convolutional neural networks’, in Digital Image Enhancement and Reconstruction, S. S. Rajput, N. U. Khan, A. K. Singh, and K. V. Arya, Eds., in Hybrid Computational Intelligence for Pattern Analysis. , Academic Press, 2023, pp. 115–138. doi: 10.1016/B978-0-32-398370-9.00012-3.
J. Lehtinen, J. Munkberg, J. Hasselgren, S. Laine, T. Karras, M. Aittala,and T. Aila, “Noise2noise: Learning image restoration without cleandata,” in ICML, 2018.
A. Krull, T.-O. Buchholz, and F. Jug, Noise2void-learning denoising from single noisy images. Proceedingsof the IEEE Computer Society Conference on Computer Vision and Pattern Recognition 2019–June:2124–2132, 2019, doi:10.1109/CVPR.2019.00223.
Y. Zharov, E. Ametova, R. Spiecker, T. Baumbach, G. Burca, and V. Heuveline, ‘Shot noise reduction in radiographic and tomographic multi-channel imaging with self-supervised deep learning’, Opt. Express, vol. 31, no. 16, pp. 26226–26244, Jul. 2023, doi: 10.1364/OE.492221.
R. Houhou et al., ‘Comparison of denoising tools for the reconstruction of nonlinear multimodal images’, Biomed. Opt. Express, vol. 14, no. 7, pp. 3259–3278, Jul. 2023, doi: 10.1364/BOE.477384.
W. Jung et al., ‘MR-self Noise2Noise: self-supervised deep learning–based image quality improvement of submillimeter resolution 3D MR images’, Eur. Radiol., vol. 33, no. 4, pp. 2686–2698, Apr. 2023, doi: 10.1007/s00330-022-09243-y.
J. Jurek, A. Materka, K. Ludwisiak, A. Majos, and F. Szczepankiewicz, ‘Phase Correction and Noise-to-Noise Denoising of Diffusion Magnetic Resonance Images Using Neural Networks’, in Computational Science – ICCS 2023, J. Mikyška, C. de Mulatier, M. Paszynski, V. V. Krzhizhanovskaya, J. J. Dongarra, and P. M. A. Sloot, Eds., in Lecture Notes in Computer Science. Cham: Springer Nature Switzerland, 2023, pp. 638–652. doi: 10.1007/978-3-031-36021-3_61.
J. Batson and L. Royer, ‘Noise2Self: Blind Denoising by Self-Supervision’, in Proceedings of the 36th International Conference on Machine Learning, PMLR, May 2019, pp. 524–533. Accessed: May 16, 2023. [Online]. Available: [link]
G. Ashwini and T. Ramashri, ‘Denoising of COVID-19 CT and chest X-ray images using deep learning techniques for various noises using single image’, in 2023 International Conference on Signal Processing, Computation, Electronics, Power and Telecommunication (IConSCEPT), May 2023, pp. 1–6. doi: 10.1109/IConSCEPT57958.2023.10170038.
M. Lesage et al., ‘An end-to-end pipeline based on open source deep learning tools for reliable analysis of complex 3D images of ovaries’, Development, vol. 150, no. 7, p. dev201185, Apr. 2023, doi: 10.1242/dev.201185.
S. Yun et al., ‘Penalty-driven enhanced self-supervised learning (Noise2Void) for CBCT denoising’, in Medical Imaging 2023: Physics of Medical Imaging, SPIE, Apr. 2023, pp. 464–469. doi: 10.1117/12.2652826.
S. Kojima, T. Ito, and T. Hayashi, ‘Denoising Using Noise2Void for Low-Field Magnetic Resonance Imaging: A Phantom Study’, J. Med. Phys., vol. 47, no. 4, p. 387, Dec. 2022, doi: 10.4103/jmp.jmp_71_22.
Z. Wu, X, Chen, S. Xie and Y. Zeng, ‘Super-resolution of brain MRI images based on denoising diffusion probabilistic model’, Biomed. Signal Process. Control, vol. 85, p. 104901, Aug. 2023, doi: 10.1016/j.bspc.2023.104901.
A. Khmag, ‘Additive Gaussian noise removal based on generative adversarial network model and semi-soft thresholding approach’, Multimed. Tools Appl., vol. 82, no. 5, pp. 7757–7777, Feb. 2023, doi: 10.1007/s11042-022-13569-6.
O. Ronneberger, P. Fischer, and T. Brox, U-Net: Convolutional Networks for Biomedical Image Segmentation, vol. 9351. 2015, p. 241. doi: 10.1007/978-3-319-24574-4_28.
S. Ioffe and C. Szegedy, ‘Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift’. arXiv, Mar. 02, 2015. doi: 10.48550/arXiv.1502.03167.
A. Paszke et al., ‘PyTorch: An Imperative Style, High-Performance Deep Learning Library’. arXiv, Dec. 03, 2019. doi: 10.48550/arXiv.1912.01703.
M. Abadi et al., ‘TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems’, ArXiv160304467 Cs, Mar. 2016, Accessed: Jul. 03, 2021. [Online]. Available: [link]
N. Ketkar, ‘Introduction to Keras’, in Deep Learning with Python: A Hands-on Introduction, N. Ketkar, Ed., Berkeley, CA: Apress, 2017, pp. 97–111. doi: 10.1007/978-1-4842-2766-4_7.
K. He, X. Zhang, S. Ren, and J. Sun, ‘Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification’, IEEE Int. Conf. Comput. Vis. ICCV 2015, vol. 1502, Feb. 2015, doi: 10.1109/ICCV.2015.123.
D. P. Kingma and J. Ba, ‘Adam: A Method for Stochastic Optimization’. arXiv, Jan. 29, 2017. doi: 10.48550/arXiv.1412.6980.
A. Shah et al., ‘Comparative analysis of median filter and its variants for removal of impulse noise from gray scale images’, J. King Saud Univ. - Comput. Inf. Sci., vol. 34, no. 3, pp. 505–519, Mar. 2022, doi: 10.1016/j.jksuci.2020.03.007.
D. Ulyanov, A. Vedaldi, and V. Lempitsky, ‘Deep Image Prior’, presented at the Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018, pp. 9446–9454. Accessed: May 16, 2023. [Online]. Available: [link].
A. Lim, J. Lo, M. W. Wagner, B. Ertl-Wagner, and D. Sussman, ‘Motion artifact correction in fetal MRI based on a Generative Adversarial network method’, Biomed. Signal Process. Control, vol. 81, p. 104484, Mar. 2023, doi: 10.1016/j.bspc.2022.104484.
J.-J. Huang and P. L. Dragotti, ‘WINNet: Wavelet-Inspired Invertible Network for Image Denoising’, IEEE Trans. Image Process., vol. 31, pp. 4377–4392, 2022, doi: 10.1109/TIP.2022.3184845.
K. Prabakaran, N. Lalithamani, and P. Kiruthika, ‘Speckle Reduction Algorithm for Medical Ultrasound Imaging’, Asian J. Res. Soc. Sci. Humanit., vol. 6, no. 6, p. 148, 2016, doi: 10.5958/2249-7315.2016.00202.1.
R. S. Thakur, S. Chatterjee, R. N. Yadav, and L. Gupta, ‘Chapter 5 - Medical image denoising using convolutional neural networks’, in Digital Image Enhancement and Reconstruction, S. S. Rajput, N. U. Khan, A. K. Singh, and K. V. Arya, Eds., in Hybrid Computational Intelligence for Pattern Analysis. , Academic Press, 2023, pp. 115–138. doi: 10.1016/B978-0-32-398370-9.00012-3.
J. Lehtinen, J. Munkberg, J. Hasselgren, S. Laine, T. Karras, M. Aittala,and T. Aila, “Noise2noise: Learning image restoration without cleandata,” in ICML, 2018.
A. Krull, T.-O. Buchholz, and F. Jug, Noise2void-learning denoising from single noisy images. Proceedingsof the IEEE Computer Society Conference on Computer Vision and Pattern Recognition 2019–June:2124–2132, 2019, doi:10.1109/CVPR.2019.00223.
Y. Zharov, E. Ametova, R. Spiecker, T. Baumbach, G. Burca, and V. Heuveline, ‘Shot noise reduction in radiographic and tomographic multi-channel imaging with self-supervised deep learning’, Opt. Express, vol. 31, no. 16, pp. 26226–26244, Jul. 2023, doi: 10.1364/OE.492221.
R. Houhou et al., ‘Comparison of denoising tools for the reconstruction of nonlinear multimodal images’, Biomed. Opt. Express, vol. 14, no. 7, pp. 3259–3278, Jul. 2023, doi: 10.1364/BOE.477384.
W. Jung et al., ‘MR-self Noise2Noise: self-supervised deep learning–based image quality improvement of submillimeter resolution 3D MR images’, Eur. Radiol., vol. 33, no. 4, pp. 2686–2698, Apr. 2023, doi: 10.1007/s00330-022-09243-y.
J. Jurek, A. Materka, K. Ludwisiak, A. Majos, and F. Szczepankiewicz, ‘Phase Correction and Noise-to-Noise Denoising of Diffusion Magnetic Resonance Images Using Neural Networks’, in Computational Science – ICCS 2023, J. Mikyška, C. de Mulatier, M. Paszynski, V. V. Krzhizhanovskaya, J. J. Dongarra, and P. M. A. Sloot, Eds., in Lecture Notes in Computer Science. Cham: Springer Nature Switzerland, 2023, pp. 638–652. doi: 10.1007/978-3-031-36021-3_61.
J. Batson and L. Royer, ‘Noise2Self: Blind Denoising by Self-Supervision’, in Proceedings of the 36th International Conference on Machine Learning, PMLR, May 2019, pp. 524–533. Accessed: May 16, 2023. [Online]. Available: [link]
G. Ashwini and T. Ramashri, ‘Denoising of COVID-19 CT and chest X-ray images using deep learning techniques for various noises using single image’, in 2023 International Conference on Signal Processing, Computation, Electronics, Power and Telecommunication (IConSCEPT), May 2023, pp. 1–6. doi: 10.1109/IConSCEPT57958.2023.10170038.
M. Lesage et al., ‘An end-to-end pipeline based on open source deep learning tools for reliable analysis of complex 3D images of ovaries’, Development, vol. 150, no. 7, p. dev201185, Apr. 2023, doi: 10.1242/dev.201185.
S. Yun et al., ‘Penalty-driven enhanced self-supervised learning (Noise2Void) for CBCT denoising’, in Medical Imaging 2023: Physics of Medical Imaging, SPIE, Apr. 2023, pp. 464–469. doi: 10.1117/12.2652826.
S. Kojima, T. Ito, and T. Hayashi, ‘Denoising Using Noise2Void for Low-Field Magnetic Resonance Imaging: A Phantom Study’, J. Med. Phys., vol. 47, no. 4, p. 387, Dec. 2022, doi: 10.4103/jmp.jmp_71_22.
Z. Wu, X, Chen, S. Xie and Y. Zeng, ‘Super-resolution of brain MRI images based on denoising diffusion probabilistic model’, Biomed. Signal Process. Control, vol. 85, p. 104901, Aug. 2023, doi: 10.1016/j.bspc.2023.104901.
A. Khmag, ‘Additive Gaussian noise removal based on generative adversarial network model and semi-soft thresholding approach’, Multimed. Tools Appl., vol. 82, no. 5, pp. 7757–7777, Feb. 2023, doi: 10.1007/s11042-022-13569-6.
O. Ronneberger, P. Fischer, and T. Brox, U-Net: Convolutional Networks for Biomedical Image Segmentation, vol. 9351. 2015, p. 241. doi: 10.1007/978-3-319-24574-4_28.
S. Ioffe and C. Szegedy, ‘Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift’. arXiv, Mar. 02, 2015. doi: 10.48550/arXiv.1502.03167.
A. Paszke et al., ‘PyTorch: An Imperative Style, High-Performance Deep Learning Library’. arXiv, Dec. 03, 2019. doi: 10.48550/arXiv.1912.01703.
M. Abadi et al., ‘TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems’, ArXiv160304467 Cs, Mar. 2016, Accessed: Jul. 03, 2021. [Online]. Available: [link]
N. Ketkar, ‘Introduction to Keras’, in Deep Learning with Python: A Hands-on Introduction, N. Ketkar, Ed., Berkeley, CA: Apress, 2017, pp. 97–111. doi: 10.1007/978-1-4842-2766-4_7.
K. He, X. Zhang, S. Ren, and J. Sun, ‘Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification’, IEEE Int. Conf. Comput. Vis. ICCV 2015, vol. 1502, Feb. 2015, doi: 10.1109/ICCV.2015.123.
D. P. Kingma and J. Ba, ‘Adam: A Method for Stochastic Optimization’. arXiv, Jan. 29, 2017. doi: 10.48550/arXiv.1412.6980.
A. Shah et al., ‘Comparative analysis of median filter and its variants for removal of impulse noise from gray scale images’, J. King Saud Univ. - Comput. Inf. Sci., vol. 34, no. 3, pp. 505–519, Mar. 2022, doi: 10.1016/j.jksuci.2020.03.007.
D. Ulyanov, A. Vedaldi, and V. Lempitsky, ‘Deep Image Prior’, presented at the Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018, pp. 9446–9454. Accessed: May 16, 2023. [Online]. Available: [link].
Publicado
13/11/2023
Como Citar
NEGREIROS, Ana Cláudia Souza Vidal de; GIRALDI, Gilson; WERNER, Heron; SANTOS, Ítalo Messias Feliz.
Self-Supervised Image Denoising Methods: an Application in Fetal MRI. In: WORKSHOP DE VISÃO COMPUTACIONAL (WVC), 18. , 2023, São Bernardo do Campo/SP.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2023
.
p. 137-141.
DOI: https://doi.org/10.5753/wvc.2023.27546.
