Using CNNs for Quality Assessment of No-Reference and Full-Reference Compressed-Video Frames
Resumo
For videos to be streamed, they have to be coded and sent to users as signals that are decoded back to be reproduced. This coding-decoding process may result in distortion that can bring differences in the quality perception of the content, consequently, influencing user experience. The approach proposed by Bosse et al. [1] suggests an Image Quality Assessment (IQA) method using an automated process. They use image datasets prelabeled with quality scores to perform a Convolutional Neural Network (CNN) training. Then, based on the CNN models, they are able to perform predictions of image quality using both Full- Reference (FR) and No-Reference (NR) evaluation. In this paper, we explore these methods exposing the CNN quality prediction to images extracted from actual videos. Various quality compression levels were applied to them as well as two different video codecs. We also evaluated how their models perform while predicting human visual perception of quality in scenarios where there is no human pre-evaluation, observing its behavior along with metrics such as SSIM and PSNR. We observe that FR model is able to better infer human perception of quality for compressed videos. Differently, NR model does not show the same behaviour for most of the evaluated videos.
Referências
Z.-N. Li, M. S. Drew, and J. Liu, Fundamentals of Multimedia, ser. Texts in Computer Science. Springer, 2014.
Z. Wang, A. C. Bovik, and L. Lu, "Why is image quality assessment so IEEE, 2002, pp. 3313–3316. [Online]. Available: difcult?" in ICASSP. http://ieeexplore.ieee.org/xpl/mostRecentIssue.jsp?punumber=7874
T. K. Tan, R. Weerakkody, M. Mrak, N. Ramzan, V. Baroncini, J.- R. Ohm, and G. J. Sullivan, "Video quality evaluation methodology and verication testing of hevc compression performance," IEEE Trans. Circuits Syst. Video Techn, vol. 26, no. 1, pp. 76–90, 2016.
K. Simonyan and A. Zisserman, "Very deep convolutional networks for large-scale image recognition," Apr. 10 2014. [Online]. Available: http://arxiv.org/abs/1409.1556
L. Kang, P. Ye, Y. Li, and D. S. Doermann, "Convolutional neural for no-reference image quality assessment," in CVPR. networks IEEE Computer Society, 2014, pp. 1733–1740. [Online]. Available: http://ieeexplore.ieee.org/xpl/mostRecentIssue.jsp?punumber=6909096
N. Ponomarenko, L. Jin, O. Ieremeiev, V. Lukin, K. Egiazarian, J. Astola, B. Vozel, K. Chehdi, M. Carli, F. Battisti, and C.-C. J. Kuo, "Image database tid2013: Peculiarities, results and perspectives," Image Communication, vol. 30, pp. 57 – 77, Signal Processing: 2015. [Online]. Available: http://www.sciencedirect.com/science/article/pii/S0923596514001490
H. R. Sheikh, M. F. Sabir, and A. C. Bovik, "A statistical evaluation of recent full reference image quality assessment algorithms," IEEE Transactions on Image Processing, vol. 15, no. 11, pp. 3440–3451, Nov 2006.
Xiph.Org Foundation, "Non-prot corporation dedicated to protecting the foundations of internet multimedia from control by private interests," https://www.xiph.org/, 1994–2019.
FFmpeg Software, "Complete, cross-platform solution to record, convert and stream audio and video," https://ffmpeg.org/, 2000–2019.
A. Mittal, A. K. Moorthy, and A. C. Bovik, "Visually Lossless H.264 Compression of Natural Videos," The Computer Journal, vol. 56, no. 5, pp. 617–627, 07 2012. [Online]. Available: https://doi.org/10.1093/comjnl/bxs105
V. Sze and M. Budagavi, "High throughput cabac entropy coding in hevc," IEEE Transactions on Circuits and Systems for Video Technology, vol. 22, no. 12, pp. 1778–1791, Dec 2012.
W. B. Pennebaker and J. L. Mitchell, JPEG: Still image data compres- sion standard. Springer Science & Business Media, 1992.
D. Taubman and M. Marcellin, JPEG2000 image compression fun- damentals, standards and practice: image compression fundamentals, standards and practice. Springer Science & Business Media, 2012, vol. 642.
ImageMagick Software, "Free software delivered as a ready-to-run binary distribution or as source code that you may use, copy, mod- ify, and distribute in both open and proprietary applications," https: //imagemagick.org/index.php, 1987–2019.
