A Coding-Efficiency-Aware Fast Heuristic for VVC Intra-Frame Prediction Targeting 360° Videos
Resumo
360° videos have gained prominence among immersive media, providing users with an impressive immersive scene experience. But the computational effort necessary to encode these videos is very high since the 360º videos demand more data to be represented. This paper presents a heuristic-based decision solution for the intraframe prediction in Versatile Video Coding (VVC) focused on 360º videos. The main goal is to reduce the computational effort of the intra-frame prediction process while ensuring minimal impact on coding efficiency. This is achieved exploring the distortions introduced by the equirectangular projection used to encode 360º videos. The results of the proposed heuristic present time savings of 8.34 % with a minor coding efficiency loss of only 0.16% BD-BR. This loss is the smallest among related works, allowing the proposed solution to achieve the best balance between time savings and coding efficiency losses among works in the literature.
Palavras-chave:
360° videos, VVC, fast intra-frame prediction, coding-efficiency-aware
Referências
Bernardo Beling, Iago Storch, Luciano Agostini, Bruno Zatt, Sergio Bampi, and Daniel Palomino. 2020. ERP-Based CTU Splitting Early Termination for Intra Prediction of 360 videos. In 2020 IEEE International Conference on Visual Communications and Image Processing (VCIP). 359–362. DOI: 10.1109/VCIP49819.2020.9301879
Gisle Bjontegaard. 2001. Calculation of average PSNR differences between RD-curves. Technical Report VCEG-M33, ITU-TSG16/Q6, Austin, Texas, USA.
Benjamin Bross, Ye-Kui Wang, Yan Ye, Shan Liu, Jianle Chen, Gary J. Sullivan, and Jens-Rainer Ohm. 2021. Overview of the Versatile Video Coding (VVC) Standard and its Applications. IEEE Transactions on Circuits and Systems for Video Technology 31, 10 (2021), 3736–3764. DOI: 10.1109/TCSVT.2021.3101953
Jose Filipe, João Carreira, Luis Tavora, Sergio de Faria, Antonio Navarro, and Pedro Assuncao. 2021. Tree-Based Ensemble Methods for Complexity Reduction of VVC Intra Coding. In 2021 Telecoms Conference (ConfTELE). 1–6. DOI: 10.1109/ConfTELE50222.2021.9435476
Wassim Hamidouche, Thibaud Biatek, Mohsen Abdoli, Edouard François, Fernando Pescador, Miloš Radosavljević, Daniel Menard, and Mickael Raulet. 2022. Versatile Video Coding Standard: A Review From Coding Tools to Consumers Deployment. IEEE Consumer Electronics Magazine 11, 5 (2022), 10–24. DOI: 10.1109/MCE.2022.3144545
Philippe Hanhart, Jill Boyce, Kiho Choi, and Jian-Liang Lin. 2018. JVET common test conditions and evaluation procedures for 360° video. JVET-L1012-v1.
Yu-Wen Huang, Jicheng An, Han Huang, Xiang Li, Shih-Ta Hsiang, Kai Zhang, Han Gao, Jackie Ma, and Olena Chubach. 2021. Block Partitioning Structure in the VVC Standard. IEEE Transactions on Circuits and Systems for Video Technology 31, 10 (2021), 3818–3833. DOI: 10.1109/TCSVT.2021.3088134
Zhi Liu, Cai Xu, Mengmeng Zhang, and Xiaohan Guan. 2019. Fast Intra Prediction Algorithm for Virtual Reality 360 Degree Video Based on Improved RMD. In 2019 Data Compression Conference (DCC). 593–593. DOI: 10.1109/DCC.2019.00105
Mário Saldanha, Gustavo Sanchez, César Marcon, and Luciano Agostini. 2021. Performance analysis of VVC intra coding. Journal of Visual Communication and Image Representation 79 (2021), 103202. DOI: 10.1016/j.jvcir.2021.103202
Iago Storch, Bruno Zatt, Luciano Agostini, Guilherme Correa, Luis A. da Silva Cruz, and Daniel Palomino. 2020. Spatially Adaptive Intra Mode Pre-Selection for ERP 360 Video Coding. In ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). 2178–2182. DOI: 10.1109/ICASSP40776.2020.9053374
Karsten Suehring. 2022. VTM-19.0. [link] Acessado: 28-06-2024.
Zhewen Sun, Li Yu, and Wei Peng. 2023. QTMT-LNN: A fast intra CU partition using lightweight neural network for 360-degree video coding on VVC. IET Image Processing 17, 2 (2023), 597–612. DOI: 10.1049/ipr2.12658
International Telecommunication Union. 2023. Recommendation ITU-T H.266 (V3) Versatile video coding. [link]. Acessado: 28-06-2024.
Mai Xu, Chen Li, Shanyi Zhang, and Patrick Le Callet. 2020. State-of-the-Art in 360° Video/Image Processing: Perception, Assessment and Compression. IEEE Journal of Selected Topics in Signal Processing 14, 1 (2020), 5–26. DOI: 10.1109/JSTSP.2020.2966864
Yan Ye, Jill M. Boyce, and Philippe Hanhart. 2020. Omnidirectional 360° Video Coding Technology in Responses to the Joint Call for Proposals on Video Compression With Capability Beyond HEVC. IEEE Transactions on Circuits and Systems for Video Technology 30, 5 (2020), 1241–1252. DOI: 10.1109/TCSVT.2019.2953827
Matt Yu, Haricharan Lakshman, and Bernd Girod. 2015. A Framework to Evaluate Omnidirectional Video Coding Schemes. In 2015 IEEE International Symposium on Mixed and Augmented Reality. 31–36. DOI: 10.1109/ISMAR.2015.12
Mengmeng Zhang, Yan Hou, and Zhi Liu. 2023. An early CU partition mode decision algorithm in VVC based on variogram for virtual reality 360 degree videos. EURASIP Journal on Image and Video Processing 2023 (2023), 9. DOI: 10.1186/s13640-023-00605-0
Shu Zheng-Jie, Peng Zong-Ju, Jiang Gang-Yi, Chen Fen, and Yuan Bo-Sen. 2023. Fast intra partition and mode prediction for equirectangular projection 360-degree video coding. IET Image Processing 17, 2 (2023), 558–569. DOI: 10.1049/ipr2.12655
Gisle Bjontegaard. 2001. Calculation of average PSNR differences between RD-curves. Technical Report VCEG-M33, ITU-TSG16/Q6, Austin, Texas, USA.
Benjamin Bross, Ye-Kui Wang, Yan Ye, Shan Liu, Jianle Chen, Gary J. Sullivan, and Jens-Rainer Ohm. 2021. Overview of the Versatile Video Coding (VVC) Standard and its Applications. IEEE Transactions on Circuits and Systems for Video Technology 31, 10 (2021), 3736–3764. DOI: 10.1109/TCSVT.2021.3101953
Jose Filipe, João Carreira, Luis Tavora, Sergio de Faria, Antonio Navarro, and Pedro Assuncao. 2021. Tree-Based Ensemble Methods for Complexity Reduction of VVC Intra Coding. In 2021 Telecoms Conference (ConfTELE). 1–6. DOI: 10.1109/ConfTELE50222.2021.9435476
Wassim Hamidouche, Thibaud Biatek, Mohsen Abdoli, Edouard François, Fernando Pescador, Miloš Radosavljević, Daniel Menard, and Mickael Raulet. 2022. Versatile Video Coding Standard: A Review From Coding Tools to Consumers Deployment. IEEE Consumer Electronics Magazine 11, 5 (2022), 10–24. DOI: 10.1109/MCE.2022.3144545
Philippe Hanhart, Jill Boyce, Kiho Choi, and Jian-Liang Lin. 2018. JVET common test conditions and evaluation procedures for 360° video. JVET-L1012-v1.
Yu-Wen Huang, Jicheng An, Han Huang, Xiang Li, Shih-Ta Hsiang, Kai Zhang, Han Gao, Jackie Ma, and Olena Chubach. 2021. Block Partitioning Structure in the VVC Standard. IEEE Transactions on Circuits and Systems for Video Technology 31, 10 (2021), 3818–3833. DOI: 10.1109/TCSVT.2021.3088134
Zhi Liu, Cai Xu, Mengmeng Zhang, and Xiaohan Guan. 2019. Fast Intra Prediction Algorithm for Virtual Reality 360 Degree Video Based on Improved RMD. In 2019 Data Compression Conference (DCC). 593–593. DOI: 10.1109/DCC.2019.00105
Mário Saldanha, Gustavo Sanchez, César Marcon, and Luciano Agostini. 2021. Performance analysis of VVC intra coding. Journal of Visual Communication and Image Representation 79 (2021), 103202. DOI: 10.1016/j.jvcir.2021.103202
Iago Storch, Bruno Zatt, Luciano Agostini, Guilherme Correa, Luis A. da Silva Cruz, and Daniel Palomino. 2020. Spatially Adaptive Intra Mode Pre-Selection for ERP 360 Video Coding. In ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). 2178–2182. DOI: 10.1109/ICASSP40776.2020.9053374
Karsten Suehring. 2022. VTM-19.0. [link] Acessado: 28-06-2024.
Zhewen Sun, Li Yu, and Wei Peng. 2023. QTMT-LNN: A fast intra CU partition using lightweight neural network for 360-degree video coding on VVC. IET Image Processing 17, 2 (2023), 597–612. DOI: 10.1049/ipr2.12658
International Telecommunication Union. 2023. Recommendation ITU-T H.266 (V3) Versatile video coding. [link]. Acessado: 28-06-2024.
Mai Xu, Chen Li, Shanyi Zhang, and Patrick Le Callet. 2020. State-of-the-Art in 360° Video/Image Processing: Perception, Assessment and Compression. IEEE Journal of Selected Topics in Signal Processing 14, 1 (2020), 5–26. DOI: 10.1109/JSTSP.2020.2966864
Yan Ye, Jill M. Boyce, and Philippe Hanhart. 2020. Omnidirectional 360° Video Coding Technology in Responses to the Joint Call for Proposals on Video Compression With Capability Beyond HEVC. IEEE Transactions on Circuits and Systems for Video Technology 30, 5 (2020), 1241–1252. DOI: 10.1109/TCSVT.2019.2953827
Matt Yu, Haricharan Lakshman, and Bernd Girod. 2015. A Framework to Evaluate Omnidirectional Video Coding Schemes. In 2015 IEEE International Symposium on Mixed and Augmented Reality. 31–36. DOI: 10.1109/ISMAR.2015.12
Mengmeng Zhang, Yan Hou, and Zhi Liu. 2023. An early CU partition mode decision algorithm in VVC based on variogram for virtual reality 360 degree videos. EURASIP Journal on Image and Video Processing 2023 (2023), 9. DOI: 10.1186/s13640-023-00605-0
Shu Zheng-Jie, Peng Zong-Ju, Jiang Gang-Yi, Chen Fen, and Yuan Bo-Sen. 2023. Fast intra partition and mode prediction for equirectangular projection 360-degree video coding. IET Image Processing 17, 2 (2023), 558–569. DOI: 10.1049/ipr2.12655
Publicado
14/10/2024
Como Citar
ROSADO, Rogério; SANTOS, Otávio; OLIVEIRA, Franklin; SILVA, Lucas; SANCHEZ, Gustavo; STORCH, Iago; PALOMINO, Daniel; AGOSTINI, Luciano.
A Coding-Efficiency-Aware Fast Heuristic for VVC Intra-Frame Prediction Targeting 360° Videos. In: BRAZILIAN SYMPOSIUM ON MULTIMEDIA AND THE WEB (WEBMEDIA), 30. , 2024, Juiz de Fora/MG.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2024
.
p. 355-359.
DOI: https://doi.org/10.5753/webmedia.2024.243128.
