Beyond the Score: A Zero-Shot Vision–Language Judge for Interpretable, Artifact-Aware Quality Assessment of Traditional and Neural Video Codecs

• Breno José Ribeiro de Vasconcelos UFPE
• Carlos André Guimarães Ferraz UFPE

DOI: https://doi.org/10.5753/webmedia.2026.30811

Resumo

Standard video-quality metrics quantify how much a compressed frame degrades — not why. Blocking, texture blur, ringing: the artifact type dictates the remedy, but every metric collapses them into a single number and remains silent about the cause. The problem deepens with neural codecs, which fail differently from traditional ones — smearing fine texture rather than introducing blocking — but no metric distinguishes the two.We asked whether off-the-shelf vision–language models, with no quality-specific training, can act as perceptual judges that name the artifact and justify their verdict. Querying Claude Opus 4.8, GPT-5.5, and Gemini 3.5 Flash zero-shot on 216 compressed sequences (traditional AV1/VVC and neural DCVC-FM/RT), we found they outperform every classical frame-level metric — PSNR (0.63), SSIM (0.59), LPIPS (0.62) — tracking human opinion at SROCC 0.71–0.74 on identical perframe inputs, without any training. Beyond the score, they name the artifact: every blocking label was assigned to a traditional codec; not one was wrongly attributed to a neural one. Three independent AI vendors agreed on this, with inter-model correlation above 0.88. The one metric they do not surpass is VMAF — which, unlike the judges, integrates the entire video sequence. That gap is real, and we report it explicitly. Within the per-frame regime, however, these zero-shot judges surpass every classical frame-level metric and return something none of them can: a reason for the score.

Palavras-chave: video quality assessment, neural video codecs, vision–language models, LLM-as-judge, compression artifacts, perceptual metrics, VMAF

Referências

Jinze Bai, Shuai Bai, Shusheng Yang, ShijieWang, Sinan Tan, PengWang, Junyang Lin, Chang Zhou, and Jingren Zhou. 2023. Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyond. arXiv preprint arXiv:2308.12966.

Benjamin Bross, Ye-Kui Wang, Yan Ye, Shan Liu, Jianle Chen, Gary J. Sullivan, and Jens-Rainer Ohm. 2021. Overview of the Versatile Video Coding (VVC) Standard and its Applications. IEEE Transactions on Circuits and Systems for Video Technology 31, 10 (2021), 3736–3764. DOI: 10.1109/TCSVT.2021.3101953

Zhe Chen, Jiannan Wu, Wenhai Wang, Weijie Su, Guo Chen, Sen Xing, Muyan Zhong, Qinglong Zhang, Xizhou Zhu, Lewei Lu, Bin Li, Ping Luo, Tong Lu, Yu Qiao, and Jifeng Dai. 2024. InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Tasks. In Proc. IEEE/CVF Conf. on Computer Vision and Pattern Recognition (CVPR). 24185–24198. DOI: 10.1109/CVPR52733.2024.02283

Keyan Ding, Kede Ma, Shiqi Wang, and Eero P. Simoncelli. 2022. Image Quality Assessment: Unifying Structure and Texture Similarity. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI) 44, 5 (2022), 2567–2581. DOI: 10.1109/TPAMI.2020.3045810

Qihang Ge,Wei Sun, Yu Zhang, Yunhao Li, Zhongpeng Ji, Fengyu Sun, Shangling Jui, Xiongkuo Min, and Guangtao Zhai. 2025. LMM-VQA: Advancing Video Quality Assessment with Large Multimodal Models. IEEE Transactions on Circuits and Systems for Video Technology 35, 11 (2025), 11083–11096. DOI: 10.1109/tcsvt.2025.3571788

Aleksandr Gushchin et al. 2025. LEHA-CVQAD: Dataset to Enable Generalized Video Quality Assessment of Compression Artifacts. In Proc. 33rd ACM Int. Conf. on Multimedia (ACM MM). 13405–13412. DOI: 10.1145/3746027.3758303

Xuan He, Dongfu Jiang, Ge Zhang, Max Ku, Achint Soni, Sherman Siu, Haonan Chen, Abhranil Chandra, Ziyan Jiang, Aaran Arulraj, Kai Wang, Quy Duc Do, Yuansheng Ni, Bohan Lyu, Yaswanth Narsupalli, Rongqi Fan, Zhiheng Lyu, Yuchen Lin, and Wenhu Chen. 2024. VideoScore: Building Automatic Metrics to Simulate Fine-grained Human Feedback for Video Generation. In Proc. Conf. on Empirical Methods in Natural Language Processing (EMNLP). 2105–2123. DOI: 10.18653/v1/2024.emnlp-main.127

ITU-R. 2023. Recommendation BT.500-15: Methodologies for the Subjective Assessment of the Quality of Television Images. Technical Report. International Telecommunication Union. [link]

ITU-T. 2020. Recommendation P.1401: Methods, Metrics and Procedures for Statistical Evaluation, Qualification and Comparison of Objective Quality Prediction Models. Technical Report. International Telecommunication Union. [link]

ITU-T. 2023. Recommendation P.910: Subjective Video Quality Assessment Methods for Multimedia Applications. Technical Report. International Telecommunication Union. [link]

Zhaoyang Jia, Bin Qi, Jiahao Li, Bin Li, Ruiqi Xie, Houqiang Li, and Yan Lu. 2025. Towards Practical Real-Time Neural Video Compression (DCVC-RT). In Proc. IEEE/CVF Conf. on Computer Vision and Pattern Recognition (CVPR). 12543–12552. DOI: 10.1109/cvpr52734.2025.01170

Jiahao Li, Bin Li, and Yan Lu. 2024. Neural Video Compression with Feature Modulation (DCVC-FM). In Proc. IEEE/CVF Conf. on Computer Vision and Pattern Recognition (CVPR). 26099–26108. DOI: 10.1109/cvpr52733.2024.02466

Zhi Li, Anne Aaron, Ioannis Katsavounidis, Anush Moorthy, and Megha Manohara. 2016. Toward a Practical Perceptual Video Quality Metric (VMAF). Netflix Technology Blog. [link]

Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. 2023. Visual Instruction Tuning. In Advances in Neural Information Processing Systems (NeurIPS), Vol. 36. 34892–34916. [link]

Rakesh Rao Ramachandra Rao et al. 2025. Evaluating Video Quality Metrics for Neural and Traditional Codecs using 4K/UHD-1 Videos (AVT-VQDB-UHD-1-NVC). arXiv preprint arXiv:2511.00969. Dataset used in this work; preprint from TU Ilmenau.

Rakesh Rao Ramachandra Rao, Steve Göring, Patrick Vogel, Nicolas Pachatz, Juan Villamar Villarreal,Werner Robitza, Peter List, Bernhard Feiten, and Alexander Raake Müller. 2019. Adaptive Video Streaming with Current Codecs and Formats: Extensions to Parametric Video Quality Models (AVT-VQDB-UHD-1). In Proc. IS&T Electronic Imaging: Image Quality and System Performance XVI. DOI: 10.2352/issn.2470-1173.2019.10.iqsp-314

Reza Rassool. 2017. VMAF Reproducibility: Validating a Perceptual Practical Video Quality Metric. In Proc. IEEE Int. Symp. on Broadband Multimedia Systems and Broadcasting (BMSB). DOI: 10.1109/BMSB.2017.7986143

Maksim Smirnov et al. 2024. AIM 2024 Challenge on Compressed Video Quality Assessment: Methods and Results. In Computer Vision – ECCV 2024 Workshops (Lecture Notes in Computer Science, Vol. 15632). 212–228. DOI: 10.1007/978-3-031-91856-8_13

Leandro Tavares, Ruhan Conceição, Victor Costa, Luciano Agostini, Marcelo Porto, and Guilherme Corrêa. 2025. Traditional vs. Neural Video Codecs: Compression Efficiency, Visual Artifacts, and Quality Analysis Beyond PSNR. In Proc. Brazilian Symp. on Multimedia and the Web (WebMedia). SBC, 446–454. DOI: 10.5753/webmedia.2025.16011

XinyiWang, Angeliki Katsenou, Junxiao Shen, and David Bull. 2026. CAMP-VQA: Caption-Embedded Multimodal Perception for No-Reference Quality Assessment of Compressed Video. In Proc. IEEE/CVF Winter Conf. on Applications of Computer Vision (WACV). 2042–2051. [link]

Zhou Wang, Alan C. Bovik, Hamid R. Sheikh, and Eero P. Simoncelli. 2004. Image Quality Assessment: From Error Visibility to Structural Similarity. IEEE Transactions on Image Processing 13, 4 (2004), 600–612. DOI: 10.1109/TIP.2003.819861

Haoning Wu, Zicheng Zhang, Erli Zhang, Chaofeng Chen, Liang Liao, Annan Wang, Chunyi Li, Wenxiu Sun, Qiong Yan, Guangtao Zhai, and Weisi Lin. 2024. Q-Bench: A Benchmark for General-Purpose Foundation Models on Low-Level Vision. In Proc. Int. Conf. on Learning Representations (ICLR), Spotlight. [link]

HaoningWu, Zicheng Zhang,Weixia Zhang, Chaofeng Chen, Liang Liao, Chunyi Li, Yixuan Gao, AnnanWang,Wei Sun, Qiong Yan, Xiongkuo Min, Guangtao Zhai, and Weisi Lin. 2024. Q-Align: Teaching LMMs for Visual Scoring via Discrete Text-Defined Levels. In Proc. Int. Conf. on Machine Learning (ICML), PMLR vol. 235. [link]

Tianhe Wu, Kede Ma, Jie Liang, Yujiu Yang, and Lei Zhang. 2024. A Comprehensive Study of Multimodal Large Language Models for Image Quality Assessment. In Computer Vision – ECCV 2024 (Lecture Notes in Computer Science, Vol. 15132). Springer Nature Switzerland, 143–160. DOI: 10.1007/978-3-031-72904-1_9

Richard Zhang, Phillip Isola, Alexei A. Efros, Eli Shechtman, and Oliver Wang. 2018. The Unreasonable Effectiveness of Deep Features as a Perceptual Metric. In Proc. IEEE/CVF Conf. on Computer Vision and Pattern Recognition (CVPR). 586–595. DOI: 10.1109/CVPR.2018.00068

Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric P. Xing, Hao Zhang, Joseph E. Gonzalez, and Ion Stoica. 2023. Judging LLM-as-a-Judge with MTBench and Chatbot Arena. In Advances in Neural Information Processing Systems (NeurIPS), Datasets and Benchmarks Track. [link]

Hanwei Zhu, Xiangjie Sui, Baoliang Chen, Xuelin Liu, Peilin Chen, Yuming Fang, and Shiqi Wang. 2024. 2AFC Prompting of Large Multimodal Models for Image Quality Assessment. IEEE Transactions on Circuits and Systems for Video Technology 34, 12 (2024), 12873–12878. DOI: 10.1109/tcsvt.2024.3434999,