Full Reference Stereoscopic Objective Quality Assessment using Lightweight Machine Learning

• Narúsci Dos Santos Bastos UFPel
• Lucas Seidy Ribeiro Dos Santos Ikenoue UFPel
• Daniel Palomino UFPel
• Guilherme Corrêa UFPel
• Tatiana Tavares UFPel
• Bruno Zatt UFPel

Resumo

Decades of research on Image Quality Assessment (IQA) have pro moted the creation of a variety of objective quality metrics that strongly correlate to subjective image quality. However, challenges remain when considering quality assessment of 3D/stereo images. Multiple objective quality metrics for 3D images were designed by extending the well-known 2D metrics. As a result, these so lutions tend to present weaknesses under 3D-specific artifacts. Recent works demonstrate the effectiveness of machine-learning techniques in the design of 3D quality metrics. Although effective, some machine learning-based solutions may lead to high compu tational effort and restrict its adoption in low-latency lightweight systems/applications. This paper presents a study on full-reference stereoscopic objective quality assessment considering lightweight machine learning. We evaluated four different decision tree-based algorithms considering eight distinct sets of image features. The classifiers were trained using data from the Waterloo IVC 3D Image Quality Database to determine the subjective quality score mea sured using Mean Opinion Score (MOS). The results show that Ran domForest generally obtains the best accuracy. Our study demon strates the feasibility of decision tree-based solutions as an accurate and lightweight approach for 3D image quality assessment.

Palavras-chave: 3D IQA, Machine Learning, Image Quality Assessment

Referências

Md Nasim Adnan and Md Zahidul Islam. 2017. Forest PA: Constructing a decision forest by penalizing attributes used in previous trees. Expert Systems with Applications 89 (2017), 389–403. https://doi.org/10.1016/j.eswa.2017.08.002

Jehad Ali, Rehanullah Khan, Nasir Ahmad, and Imran Maqsood. 2012. Random forests and decision trees. International Journal of Computer Science Issues (IJCSI) 9, 5 (2012), 272. http://ijcsi.org/articles/Random-forests-and-decision-trees.php

Miguel Thiago Alvarenga. 2014. Utilização da ferramenta j48 para descoberta do conhecimento em bases de dados fitossanitários, climáticos e espectrais. Ph. D. Dissertation. Master thesis, Universidade Federal de Lavras, Minas Gerais, Brazil. http://repositorio.ufla.br/jspui/handle/1/31295

Amin Banitalebi-Dehkordi, Mahsa T. Pourazad, and Panos Nasiopoulos. 2016. An efficient human visual system based quality metric for 3D video. Multimedia Tools and Applications 75, 8 (2016), 4187–4215. https://doi.org/10.48550/arXiv.1803.04832

Thiago Bubolz, Bruno Zatt, Mateus Grellert, and Guilherme Corrêa. 2019. Evaluation of Machine Learning Algorithms for Fast Video Transcoding in Streaming Services. In Anais do XXV Simpósio Brasileiro de Sistemas Multimídia e Web (Riode Janeiro). SBC, Porto Alegre, RS, Brasil, 181–184. https://sol.sbc.org.br/index.php/webmedia/article/view/8019

Cássio O. Camilo and João Carlos da Silva. 2009. Mineração de dados: Conceitos, tarefas, métodos e ferramentas. Universidade Federal de Goiás (UFC) (2009), 1–29.

Christophe Charrier, Olivier Lézoray, and Gilles Lebrun. 2012. Machine learning to design full-reference image quality assessment algorithm. Signal processing: Image communication 27, 3 (2012), 209–219. https://doi.org/10.1016/j.image.2012.01.002

Roberto Nery da Fonseca. 2008. Algoritmos para avaliação da qualidade de vídeo em sistemas de televisão digital. Ph. D. Dissertation. Universidade de São Paulo. https://doi.org/10.11606/D.3.2008.tde-28042009-170527

Bezerra da S. Wyllian. 2013. Métodos sem referência baseados em características espaço-temporais para avaliação objetiva de qualidade de vídeo digital. Ph. D. Dissertation. Universidade Tecnológica Federal do Paraná. http://repositorio.utfpr.edu.br/jspui/handle/1/525

Tamires Tessarolli de Souza Barbieri and Rudinei Goularte. 2020. Investigating Subjectivity Criterion for Multi-video Summarization. In Proceedings of the Brazilian Symposium on Multimedia and the Web (New York, NY, United States). Association for Computing Machinery, 137–144. https://doi.org/10.1145/3428658.3430964

Yuming Fang, Xiangjie Sui, and Jiheng Wang. 2019. A Spatial-Temporal Weighted Method for Asymmetrically Distorted Stereo Video Quality Assessment. In 2019 IEEE International Symposium on Circuits and Systems (ISCAS) (Sapporo, Japan). IEEE, 1–5. https://doi.org/10.1109/ISCAS.2019.8702796

Yuming Fang, Xiangjie Sui, Jiheng Wang, Jiebin Yan, Jianjun Lei, and Patrick Le Callet. 2020. Perceptual quality assessment for asymmetrically distorted stereoscopic video by temporal binocular rivalry. IEEE Transactions on Circuits and Systems for Video Technology 31, 8 (2020), 3010–3024. https://doi.org/10.1109/TCSVT.2020.3035679

Chathura Galkandage, Janko Calic, Safak Dogan, and Jean-Yves Guillemaut. 2017. Stereoscopic video quality assessment using binocular energy. IEEE Journal of Selected Topics in Signal Processing 11, 1 (2017), 102–112. https://doi.org/10.1109/JSTSP.2016.2632045

Simone C Garcia. 2003. O uso de árvores de decisão na descoberta de conhecimento na área da saúde. (2003).

Paolo Gastaldo and Judith A. Redi. 2012. Machine learning solutions for objective visual quality assessment. 12 (2012).

Armando M. Gutiérrez, Patricia A. Pacheco, José C. Gutiérrez, and Graça Bressan. 2019. Development of a Naive Bayes classifier for image quality assessment in biometric face images. In Anais do XXV Simpósio Brasileiro de Sistemas Multimídia e Web (Rio de Janeiro). SBC, Porto Alegre, RS, Brasil, 177–180. https://sol.sbc.org.br/index.php/webmedia/article/view/8018

Quan Huynh-Thu and Mohammed Ghanbari. 2008. Scope of validity of PSNR in image/video quality assessment. Electronics letters 44, 13 (2008), 800–801. https://doi.org/10.1049/el:20080522

Recommendation ITU-R BT.500. 2002. 500-11,“Methodology for the Subjective Assessment of the Quality of Television Pictures,” Recommendation ITU-R BT. 500-11. ITU Telecom. Standardization Sector of ITU 7 (2002).

Recommandation ITU-T P.910. 2008. P910. Subjective video quality assessment methods for multimedia applications (2008).

Martin Jansche. 2005. Maximum expected F-measure training of logistic regression models. In Proceedings of Human Language Technology Conference and Conference on Empirical Methods in Natural Language Processing (United States). Association for Computational Linguistics, 692–699. https://doi.org/10.3115/1220575.1220662

Sushilkumar Kalmegh. 2015. Analysis of weka data mining algorithm reptree, simple cart and randomtree for classification of indian news. International Journal of Innovative Science, Engineering & Technology 2, 2 (2015), 438–446.

Antonio Liotta, Decebal Constantin Mocanu, Vlado Menkovski, Luciana Cagnetta, and Georgios Exarchakos. 2013. Instantaneous video quality assessment for lightweight devices. In Proceedings of International Conference on Advances in Mobile Computing & Multimedia (New York, NY, United States). Association for Computing Machinery, 525–531. https://doi.org/10.1145/2536853.2536903

Manish Narwaria and Weisi Lin. 2011. SVD-based quality metric for image and video using machine learning. IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics) 42, 2 (2011), 347–364. https://doi.org/10.1109/TSMCB.2011.2163391

MSA Carlos DM Regis. 2013. Métrica de Avaliação Objetiva de Vídeo Usando a Informação Espacial, a Temporal ea Disparidade. Ph. D. Dissertation. Ph. D. dissertation, Federal University of Campina Grande–UFCG.

Alim Samat, Sicong Liu, Claudio Persello, Erzhu Li, Zelang Miao, and Jilili Abuduwaili. 2019. Evaluation of ForestPA for VHR RS image classification using spectral and superpixel-guided morphological profiles. European journal of remote sensing 52, 1 (2019), 107–121. https://doi.org/10.1080/22797254.2019.1565418

Varuna De Silva, Hemantha K. Arachchi, Erhan Ekmekcioglu, and Ahmet Kondoz. 2013. Toward an impairment metric for stereoscopic video: A full-reference video quality metric to assess compressed stereoscopic video. IEEE transactions on image processing 22, 9 (2013), 3392–3404. https://doi.org/10.1109/TIP.2013.2268422

José A. Sáez, Mikel Galar, Julián Luengo, and Francisco Herrera. 2016. INFFC: An iterative class noise filter based on the fusion of classifiers with noise sensitivity control. Information Fusion 27 (2016), 19–32. https://doi.org/10.1016/j.inffus.2015.04.002

Jiheng Wang, Abdul Rehman, Kai Zeng, Shiqi Wang, and Zhou Wang. 2015. Quality prediction of asymmetrically distorted stereoscopic 3D images. IEEE Transactions on Image Processing 24, 11 (2015), 3400–3414.

Jiheng Wang, Qingbo Wu, Abdul Rehman, Shiqi Wang, and Zhou Wang. 2017. Blind quality prediction of stereoscopic 3D images. IS and T International Symposium on Electronic Imaging Science and Technology 2017, 14 (2017), 70–76. https://doi.org/10.2352/ISSN.2470-1173.2017.14.HVEI-379

Zhou Wang and Alan C Bovik. 2009. Mean squared error: Love it or leave it? A new look at signal fidelity measures. IEEE signal processing magazine 26, 1 (2009), 98–117. https://doi.org/10.1109/MSP.2008.930649

Zhou Wang, Alan C. Bovik, Hamid R. Sheikh, and Eero P. Simoncelli. 2004. Image quality assessment: from error visibility to structural similarity. IEEE transactions on image processing 13, 4 (2004), 600–612. https://doi.org/10.1109/TIP.2003.819861

Zhou Wang, Ligang Lu, and Alan C. Bovik. 2004. Video quality assessment based on structural distortion measurement. Signal processing: Image communication 19, 2 (2004), 121–132. https://doi.org/10.1016/S0923-5965(03)00076-6

Zhou Wang, Hamid R. Sheikh, Alan C. Bovik, et al. 2003. Objective video quality assessment. The handbook of video databases: design and applications 41 (2003), 1041–1078.

Demóstenes R. Zegarra. 2014. Proposta da métrica eVSQM para avaliação de QoE no serviço de streaming de vídeo sobre TCP. Ph. D. Dissertation. Universidade de São Paulo.

Ce Zhu and Bing Xiong. 2009. Transform-exempted calculation of sum of absolute Hadamard transformed differences. ieee transactions on circuits and systems for video technology 19, 8 (2009), 1183–1188. https://doi.org/10.1109/TCSVT.2009. 2020264