Knowledge Distillation and Composite Loss as a Low-Resource Strategy for LDCT Image Denoising

• Raíssa X. Contassot PUCRS
• Augusto O. Ughini PUCRS
• Jerusa D. Finatto PUCRS
• Kristen K. Arguello PUCRS
• Rafaela C. Ravazio PUCRS
• Otávio Parraga PUCRS
• Christian Mattjie PUCRS
• Lucas S. Kupssinskü PUCRS
• Rodrigo C. Barros PUCRS / Kunumi Institute

DOI: https://doi.org/10.5753/eniac.2026.31186

Resumo

Low-dose CT (LDCT) lowers radiation exposure at the cost of noisier images that can hinder diagnosis. Deep learning restores quality, but state-of-the-art models often need tens of millions of parameters, out of reach for institutions without dedicated GPU infrastructure. We propose knowledge distillation (KD) as a low-resource strategy: a compact 1.11M-parameter RED-CNN-SE student distilled from a 12M-parameter AttentionUNet-Pro teacher, trained with a composite Charbonnier, Sobel, and Stationary Wavelet Transform loss. In a single forward pass the student reaches 33.72 dB PSNR, within 0.02 dB of the teacher and at 99.6% of the PSNR reported by the state of the art, with roughly 10× fewer parameters. Optional test-time augmentation adds a further 0.04 dB at eight times the inference cost and is reported separately, since it conflicts with the low-resource premise of this work.

Referências

American Association of Physicists in Medicine (2016). Low Dose CT Grand Challenge. [link]. Accessed: Apr. 24, 2026.

Chen, H. et al. (2017). Low-dose ct with a residual encoder-decoder convolutional neural network. IEEE transactions on medical imaging, 36(12):2524–2535.

Chen, H. et al. (2024). Deep learning-based algorithms for low-dose ct imaging: A review. European Journal of Radiology, 172:111355.

Demir, A. et al. (2023). Low-dose ct image enhancement using deep learning. arXiv preprint arXiv:2310.20265.

dos Santos Junior, J. A., JansonNey, M. S. C., and de Souza Fonseca, G. V. (2020). Dose efetiva de radiação nos exames de tomografia computadorizada: um estudo retrospectivo e descritivo. Diagnóstico e Tratamento, 25(2):46–51.

Eulig, E., Ommer, B., and Kachelrieß, M. (2024). Benchmarking deep learning-based low-dose ct image denoising algorithms. Medical physics, 51(12):8776–8788.

Gajera, B. et al. (2021). Ct-scan denoising using a charbonnier loss generative adversarial network. IEEE Access, 9:84093–84109.

Gao, Q. et al. (2023). Corediff: Contextual error-modulated generalized diffusion model for low-dose ct denoising and generalization. IEEE Transactions on Medical Imaging, 43(2):745–759.

He, Z. et al. (2020). FAKD: Feature-affinity based knowledge distillation for efficient image super-resolution. In IEEE ICIP.

Hinton, G., Vinyals, O., and Dean, J. (2015). Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531.

Hu, J., Shen, L., and Sun, G. (2018). Squeeze-and-excitation networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 7132–7141.

Hu, Q.-j. et al. (2021). Prospective study of low-and standard-dose chest ct for pulmonary nodule detection: a comparison of image quality, size measurements and radiation exposure. Current Medical Science, 41:966–973.

Huang, J., Zhong, A., and Liu, Y. (2024). Multi-scale perceptual modulation network for low-dose computed tomography denoising. Quantitative Imaging in Medicine and Surgery, 14(12):9290–9305.

Jo, G. D. et al. (2023). 75% radiation dose reduction using deep learning reconstruction on low-dose chest ct. BMC Medical Imaging, 23(1):121.

Junda, M., Muller, H., and Friedrich-Nel, H. (2021). Local diagnostic reference levels for routine chest x-ray examinations at a public sector hospital in central south africa. Health SA Gesondheid (Online), 26:1–8.

Kang, E., Min, J., and Ye, J. C. (2017). A deep convolutional neural network using directional wavelets for low-dose X-ray CT reconstruction. Medical Physics, 44(10):e360–e375.

Korkmaz, C. and Tekalp, A. M. (2024). Training transformer models by wavelet losses improves quantitative and visual performance in single image super-resolution. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 6661–6670.

Liang, T. et al. (2020). EDCNN: Edge enhancement-based densely connected network with compound loss for low-dose CT denoising. arXiv preprint arXiv:2011.00139.

Romero, A. et al. (2015). Fitnets: Hints for thin deep nets.

Wang, D., Fan, F., Wu, Z., Liu, R., Wang, F., and Yu, H. (2023a). Ctformer: convolution-free token2token dilated vision transformer for low-dose ct denoising. Physics in Medicine & Biology, 68(6):065012.

Wang, J. et al. (2023b). A self-supervised guided knowledge distillation framework for unpaired low-dose CT image denoising. Computerized Medical Imaging and Graphics, 107:102237.

Wang, X. and Pang, B. (2026). Low-dose CT image denoising based on a progressive fusion distillation network with pixel attention. Applied Sciences, 16(7):3292.

Yang, e. a. (2025). Knowledge distillation for image restoration: Simultaneous learning from degraded and clean images. arXiv preprint arXiv:2501.09268.

Yang, Q. et al. (2018). Low-dose CT image denoising using a generative adversarial network with Wasserstein distance and perceptual loss. IEEE Transactions on Medical Imaging, 37(6):1348–1357.

Zhang, Z. et al. (2022). Mkanet: An efficient network with sobel boundary loss for landcover classification of satellite remote sensing imagery. Remote Sensing, 14(18):4514.

Zhao, e. a. (2025). Distillation guided deep unfolding network with frequency hierarchical regularization for low-dose CT image denoising. Knowledge-Based Systems.

Zhou, Z. et al. (2023). Patient-specific uncertainty and bias quantification of nontransparent convolutional neural network model through knowledge distillation and bayesian deep learning. Proceedings of SPIE Medical Imaging, 12463.

Zou, B. et al. (2022). Toward efficient image denoising: A lightweight network with retargeting supervision driven knowledge distillation. In ACCV.