A multi-stream dense network with different receptive fields to assess visual quality

  • Luan A. Gonçalves Programa de Pós-Graduação em Engenharia Elétrica, Universidade Federal do Pará
  • Ronaldo F. Zampolo Faculdade de Engenharia da Computação e Telecomunicações, Universidade Federal do Pará
  • Fabrício B. Barros Programa de Pós-Graduação em Engenharia Elétrica, Universidade Federal do Pará

Resumo


The prediction of visual quality is crucial in image and video systems. Image quality metrics based on the mean square error prevail in the field, due to their mathematical straightforwardness, even though they do not correlate well with the visual human perception. Latest achievements in the area support that the use of convolutional neural networks (CNN) to assess perceptual visual quality is a clear trend. Results in other applications, like blur detection and de-raining, indicate the combination of different receptive fields (i.e., convolutional kernels with different dimensions) improves a CNN performance. However, to the best of our knowledge, the role of different receptive fields in visual quality characterization is still an open issue. Thus, in this paper, we investigate the influence of using different receptive fields to predict image distortion. Specifically, we propose a multi-stream dense network that estimates a spatially-varying quality metric parameter from either reference or distorted images. The performance of the proposed method is compared with a competing state-of-the-art approach by using a public image database. Results show the proposed strategy outperforms the competing technique when the quality metric parameter is estimated from degraded images.

Palavras-chave: Convolution neural network, different receptive fields, differential mean opinion score, multi-stream dense network, peak signal-to-noise ratio, visual quality assessment

Referências

Bell, A. J. and Sejnowski, T. J. The “independent components” of natural scenes are edge filters. Vision Research
37 (23): 3327 – 3338, 1997.

Bosse, S., Becker, S., Fisches, Z. V., Samek, W., and Wiegand, T. Neural Network-Based Estimation of
Distortion Sensitivity for Image Quality Prediction. In 2018 25th IEEE International Conference on Image Processing (ICIP). IEEE, Athens, Greece, pp. 629–633, 2018.

Bosse, S., Becker, S., Müller, K.-R., Samek, W., and Wiegand, T. Estimation of distortion sensitivity for visual
quality prediction using a convolutional neural network. Digital Signal Processing vol. 91, pp. 54–64, dec, 2019.

Bosse, S., Maniry, D., Müller, K.-R., Wiegand, T., and Samek, W. Deep Neural Networks for No-Reference
and Full-Reference Image Quality Assessment. IEEE Transactions on Image Processing vol. 17, pp. 2016–219, dec,
2016.

Gillibert, L., Chabardès, T., and Marcotegui, B. Local multiscale blur estimation based on toggle mapping for
sharp region extraction. IET Image Processing 12 (12): 2138–2146, dec, 2018.

Girod, B. Whats Wrong with Mean-squared Error? In A. B. Wattson (Ed.), Digital Images and Human Visions.
MIT press, Cambridge,MA, pp. 207–220, 1993.

Huang, R., Feng, W., Fan, M., Wan, L., and Sun, J. Multiscale blur detection by learning discriminative deep
features. Neurocomputing vol. 285, pp. 154–166, apr, 2018.

Kang, L., Ye, P., Li, Y., and Doermann, D. Convolutional Neural Networks for No-Reference Image Quality
Assessment. In 2014 IEEE Conference on Computer Vision and Pattern Recognition. IEEE, Columbus, OH, USA,
pp. 1733–1740, 2014.

Lin Zhang, Lei Zhang, Xuanqin Mou, and Zhang, D. FSIM: A Feature Similarity Index for Image Quality
Assessment. IEEE Transactions on Image Processing 20 (8): 2378–2386, aug, 2011.

Paszke, A., Gross, S., Chintala, S., Chanan, G., Yang, E., DeVito, Z., Lin, Z., Desmaison, A., Antiga, L.,
and Lerer, A. Automatic differentiation in pytorch. In 31st Conference on Neural Information Processing Systems
(NIPS 2017). NIPS, Long Beach, CA, USA., 2017.

Reisenhofer, R., Bosse, S., Kutyniok, G., and Wiegand, T. A Haar wavelet-based perceptual similarity index
for image quality assessment. Signal Processing: Image Communication vol. 61, pp. 33–43, feb, 2018.

Ruderman, D. L. The statistics of natural images. Network: Computation in Neural Systems 5 (4): 517–548, 1994.

Sheikh, H. R., Sabir, M. F., and Bovik, A. C. A statistical evaluation of recent full reference image quality
assessment algorithms. Trans. Img. Proc. 15 (11): 3440–3451, Nov., 2006.

Wang, Z., B. A. S. H. S. E. Modern image quality assessment. Synthesis Lectures on Image, Video, and Multimedia
Processing 2 (1): 1–156, jan, 2006.

Wang, Z., Bovik, A., Sheikh, H., and Simoncelli, E. Image Quality Assessment: From Error Visibility to Structural
Similarity. IEEE Transactions on Image Processing 13 (4): 600–612, apr, 2004.

Wang, Z., Simoncelli, E., and Bovik, A. Multiscale structural similarity for image quality assessment. In The
Thrity-Seventh Asilomar Conference on Signals, Systems & Computers, 2003. IEEE, Pacific Grove, CA, USA, pp.
1398–1402, 2003.

Yang, W., Tan, R. T., Feng, J., Liu, J., Guo, Z., and Yan, S. Deep Joint Rain Detection and Removal from a
Single Image. In 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, Honolulu,
USA, 2016.

Zhang, H. and Patel, V. M. Density-aware Single Image De-raining using a Multi-stream Dense Network. In 2018
IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE, Salt Lake City, USA, 2018.
Publicado
18/11/2019
Como Citar

Selecione um Formato
GONÇALVES, Luan A.; ZAMPOLO, Ronaldo F.; BARROS, Fabrício B.. A multi-stream dense network with different receptive fields to assess visual quality. In: SYMPOSIUM ON KNOWLEDGE DISCOVERY, MINING AND LEARNING (KDMILE) , 2019, Fortaleza. Anais do VII Symposium on Knowledge Discovery, Mining and Learning. Porto Alegre: Sociedade Brasileira de Computação, nov. 2019 . p. 65-72. DOI: https://doi.org/10.5753/kdmile.2019.8790.