Using CNNs for Quality Assessment of No-Reference and Full-Reference Compressed-Video Frames

  • Renato da Silva UFU
  • Luiz Brito UFU
  • Marcelo Albertini UFU
  • Marcelo do Nascimento UFU
  • André Backes UFU

Resumo


For videos to be streamed, they have to be coded and sent to users as signals that are decoded back to be reproduced. This coding-decoding process may result in distortion that can bring differences in the quality perception of the content, consequently, influencing user experience. The approach proposed by Bosse et al. [1] suggests an Image Quality Assessment (IQA) method using an automated process. They use image datasets prelabeled with quality scores to perform a Convolutional Neural Network (CNN) training. Then, based on the CNN models, they are able to perform predictions of image quality using both Full- Reference (FR) and No-Reference (NR) evaluation. In this paper, we explore these methods exposing the CNN quality prediction to images extracted from actual videos. Various quality compression levels were applied to them as well as two different video codecs. We also evaluated how their models perform while predicting human visual perception of quality in scenarios where there is no human pre-evaluation, observing its behavior along with metrics such as SSIM and PSNR. We observe that FR model is able to better infer human perception of quality for compressed videos. Differently, NR model does not show the same behaviour for most of the evaluated videos.

Palavras-chave: Convolutional Neural Network, Digital Video Streaming, Quality Analysis

Referências

S. Bosse, D. Maniry, K.-R. Müller, T. Wiegand, and W. Samek, "Deep neural networks for no-reference and full-reference image quality as- sessment," IEEE Trans. Image Processing, vol. 27, no. 1, pp. 206–219, 2018.

Z.-N. Li, M. S. Drew, and J. Liu, Fundamentals of Multimedia, ser. Texts in Computer Science. Springer, 2014.

Z. Wang, A. C. Bovik, and L. Lu, "Why is image quality assessment so IEEE, 2002, pp. 3313–3316. [Online]. Available: difcult?" in ICASSP. http://ieeexplore.ieee.org/xpl/mostRecentIssue.jsp?punumber=7874

T. K. Tan, R. Weerakkody, M. Mrak, N. Ramzan, V. Baroncini, J.- R. Ohm, and G. J. Sullivan, "Video quality evaluation methodology and verication testing of hevc compression performance," IEEE Trans. Circuits Syst. Video Techn, vol. 26, no. 1, pp. 76–90, 2016.

K. Simonyan and A. Zisserman, "Very deep convolutional networks for large-scale image recognition," Apr. 10 2014. [Online]. Available: http://arxiv.org/abs/1409.1556

L. Kang, P. Ye, Y. Li, and D. S. Doermann, "Convolutional neural for no-reference image quality assessment," in CVPR. networks IEEE Computer Society, 2014, pp. 1733–1740. [Online]. Available: http://ieeexplore.ieee.org/xpl/mostRecentIssue.jsp?punumber=6909096

N. Ponomarenko, L. Jin, O. Ieremeiev, V. Lukin, K. Egiazarian, J. Astola, B. Vozel, K. Chehdi, M. Carli, F. Battisti, and C.-C. J. Kuo, "Image database tid2013: Peculiarities, results and perspectives," Image Communication, vol. 30, pp. 57 – 77, Signal Processing: 2015. [Online]. Available: http://www.sciencedirect.com/science/article/pii/S0923596514001490

H. R. Sheikh, M. F. Sabir, and A. C. Bovik, "A statistical evaluation of recent full reference image quality assessment algorithms," IEEE Transactions on Image Processing, vol. 15, no. 11, pp. 3440–3451, Nov 2006.

Xiph.Org Foundation, "Non-prot corporation dedicated to protecting the foundations of internet multimedia from control by private interests," https://www.xiph.org/, 1994–2019.

FFmpeg Software, "Complete, cross-platform solution to record, convert and stream audio and video," https://ffmpeg.org/, 2000–2019.

A. Mittal, A. K. Moorthy, and A. C. Bovik, "Visually Lossless H.264 Compression of Natural Videos," The Computer Journal, vol. 56, no. 5, pp. 617–627, 07 2012. [Online]. Available: https://doi.org/10.1093/comjnl/bxs105

V. Sze and M. Budagavi, "High throughput cabac entropy coding in hevc," IEEE Transactions on Circuits and Systems for Video Technology, vol. 22, no. 12, pp. 1778–1791, Dec 2012.

W. B. Pennebaker and J. L. Mitchell, JPEG: Still image data compres- sion standard. Springer Science & Business Media, 1992.

D. Taubman and M. Marcellin, JPEG2000 image compression fun- damentals, standards and practice: image compression fundamentals, standards and practice. Springer Science & Business Media, 2012, vol. 642.

ImageMagick Software, "Free software delivered as a ready-to-run binary distribution or as source code that you may use, copy, mod- ify, and distribute in both open and proprietary applications," https: //imagemagick.org/index.php, 1987–2019.
Publicado
07/10/2020
DA SILVA, Renato; BRITO, Luiz; ALBERTINI, Marcelo; DO NASCIMENTO, Marcelo; BACKES, André. Using CNNs for Quality Assessment of No-Reference and Full-Reference Compressed-Video Frames. In: WORKSHOP DE VISÃO COMPUTACIONAL (WVC), 16. , 2020, Evento Online. Anais [...]. Porto Alegre: Sociedade Brasileira de Computação, 2020 . p. 75-80. DOI: https://doi.org/10.5753/wvc.2020.13484.

Artigos mais lidos do(s) mesmo(s) autor(es)

Obs.: Esse plugin requer que pelo menos um plugin de estatísticas/relatórios esteja habilitado. Se o seu plugins de estatísticas oferece mais que uma métrica, então, por favor, também selecione uma métrica principal na página de configurações administrativas do site e/ou da revista.