Adson Duarte
ABSTRACT
The VVC video coding standard achieves high compression rates due to innovative tools that were introduced mainly in the intra prediction. However, the high computational effort associated with the intra mode decision poses a challenge for real-time video coding applications. In this paper, we propose a machine learning-based solution to accelerate the intra mode decision of VVC. The intra modes are organized in three classes (Planar/DC, Angular and MIP) and a Decision Tree model is developed to predict the class of modes more likely to be chosen, avoiding the evaluation of the classes of modes with less chance to be the optimal. As a result, the proposed solution can reduce the total encoding time in 15.67% on average with only 0.80% of coding efficiency loss. When compared with related works, our solution presents good results.
- James Bergstra and Yoshua Bengio. 2012. Random search for hyper-parameter optimization.Journal of machine learning research 13, 2 (2012), 281–305. https://www.jmlr.org/papers/volume13/bergstra12a/bergstra12a.pdfGoogle Scholar
- Gisle Bjontegaard. 2001. Calculation of average PSNR differences between RD-curves. https://www.itu.int/wftp3/av-arch/video-site/0104_Aus/VCEG-M33.doc. VCEG Meeting.Google Scholar
- Frank Bossen, Jill Boyce, Karsten Sühring, Xiang Li, and Vadim Seregin. 2020. VTM common test conditions and software reference configurations for SDR video. https://jvet-experts.org/doc_end_user/current_document.php?id=10545. JVET-T2010-v1.Google Scholar
- Benjamin Bross, Jianle Chen, Shan Liu, and Ye-Kui Wang. 2020. Versatile Video Coding Editorial Refinements on Draft 10. https://jvet-experts.org/doc_end_user/current_document.php?id=10540. JVET-T2001-v2.Google Scholar
- Yao-Jen Chang, Hong-Jheng Jhu, Hui-Yu Jiang, Liang Zhao, Xin Zhao, Xiang Li, Shan Liu, Benjamin Bross, Paul Keydel, Heiko Schwarz, Detlev Marpe, and Thomas Wiegand. 2019. Multiple Reference Line Coding for Most Probable Modes in Intra Prediction. In 2019 Data Compression Conference (DCC). IEEE, Snowbird, UT, USA, 559–559. https://doi.org/10.1109/DCC.2019.00071Google ScholarCross Ref
- Thomas Daede, Andrey Norkin, and Ilya Brailovkkiy. 2018. Video Codec Testing and Quality Measurement. https://tools.ietf.org/html/draft-ietf-netvc-testing-07.Google Scholar
- Santiago De-Luxán-Hernández, Valeri George, Jackie Ma, Tung Nguyen, Heiko Schwarz, Detlev Marpe, and Thomas Wiegand. 2019. An Intra Subpartition Coding Mode for VVC. In 2019 IEEE International Conference on Image Processing (ICIP). IEEE, Taipei, Taiwan, 1203–1207. https://doi.org/10.1109/ICIP.2019.8803777Google ScholarCross Ref
- Adson Duarte, Paulo Gonçalves, Luciano Agostini, Bruno Zatt, Guilherme Correa, Marcelo Porto, and Daniel Palomino. 2022. Fast Affine Motion Estimation for VVC using Machine-Learning-Based Early Search Termination. In 2022 IEEE International Symposium on Circuits and Systems (ISCAS). IEEE, Austin, TX, USA, 1–5. https://doi.org/10.1109/ISCAS48785.2022.9937973Google ScholarCross Ref
- ITU-T. 2022. Subjective video quality assessment methods for multimedia applications. https://www.itu.int/rec/T-REC-P.910. P.910.Google Scholar
- Zheng Liu, Tianyi Li, Ying Chen, Kaijin Wei, Mai Xu, and Honggang Qi. 2023. Deep Multi-task Learning based Fast Intra-mode Decision for Versatile Video Coding. IEEE Transactions on Circuits and Systems for Video Technology (2023), 1–1. https://doi.org/10.1109/TCSVT.2023.3262733Google ScholarDigital Library
- Alexandre Mercat, Marko Viitanen, and Jarno Vanne. 2020. UVG Dataset: 50/120fps 4K Sequences for Video Codec Analysis and Development. In Proceedings of the 11th ACM Multimedia Systems Conference (Istanbul, Turkey) (MMSys ’20). Association for Computing Machinery, New York, NY, USA, 297–302. https://doi.org/10.1145/3339825.3394937Google ScholarDigital Library
- Chi-Ting Ni, Shih-Hsiang Lin, Pei-Yin Chen, and Yu-Ting Chu. 2022. High Efficiency Intra CU Partition and Mode Decision Method for VVC. IEEE Access 10 (2022), 77759–77771. https://doi.org/10.1109/ACCESS.2022.3193401Google ScholarCross Ref
- Jeeyoon Park, Bumyoon Kim, and Byeungwoo Jeon. 2022. Fast VVC Intra Subpartition based on Position of Reference Pixels. In 2022 International Conference on Electronics, Information, and Communication (ICEIC). IEEE, Jeju, Korea, Republic of, 1–2. https://doi.org/10.1109/ICEIC54506.2022.9748554Google ScholarCross Ref
- Jeeyoon Park, Bumyoon Kim, Jeehwan Lee, and Byeungwoo Jeon. 2022. Machine Learning-Based Early Skip Decision for Intra Subpartition Prediction in VVC. IEEE Access 10 (2022), 111052–111065. https://doi.org/10.1109/ACCESS.2022.3215163Google ScholarCross Ref
- Fabian Pedregosa, Gaël Varoquaux, Alexandre Gramfort, Vincent Michel, Bertrand Thirion, Olivier Grisel, Mathieu Blondel, Peter Prettenhofer, Ron Weiss, Vincent Dubourg, Jake Vanderplas, Alexandre Passos, David Cournapeau, Matthieu Brucher, Matthieu Perrot, and Édouard Duchesnay. 2011. Scikit-learn: Machine Learning in Python. Journal of Machine Learning Research 12, 85 (2011), 2825–2830. http://jmlr.org/papers/v12/pedregosa11a.htmlGoogle ScholarDigital Library
- Jonathan Pfaff, Alexey Filippov, Shan Liu, Xin Zhao, Jianle Chen, Santiago De-Luxán-Hernández, Thomas Wiegand, Vasily Rufitskiy, Adarsh Krishnan Ramasubramonian, and Geert Van der Auwera. 2021. Intra Prediction and Mode Coding in VVC. IEEE Transactions on Circuits and Systems for Video Technology 31, 10 (2021), 3834–3847. https://doi.org/10.1109/TCSVT.2021.3072430Google ScholarCross Ref
- Mário Saldanha, Gustavo Sanchez, César Marcon, and Luciano Agostini. 2020. Complexity Analysis Of VVC Intra Coding. In 2020 IEEE International Conference on Image Processing (ICIP). IEEE, Abu Dhabi, United Arab Emirates, 3119–3123. https://doi.org/10.1109/ICIP40778.2020.9190970Google ScholarCross Ref
- Mário Saldanha, Gustavo Sanchez, César Marcon, and Luciano Agostini. 2021. Learning-Based Complexity Reduction Scheme for VVC Intra-Frame Prediction. In 2021 International Conference on Visual Communications and Image Processing (VCIP). IEEE, Munich, Germany, 1–5. https://doi.org/10.1109/VCIP53242.2021.9675394Google ScholarCross Ref
- Mário Saldanha, Gustavo Sanchez, César Marcon, and Luciano Agostini. 2022. Fast Transform Decision Scheme for VVC Intra-Frame Prediction Using Decision Trees. In 2022 IEEE International Symposium on Circuits and Systems (ISCAS). IEEE, Austin, TX, USA, 1948–1952. https://doi.org/10.1109/ISCAS48785.2022.9938000Google ScholarCross Ref
- Michael Schäfer, Björn Stallenberger, Jonathan Pfaff, Philipp Helle, Heiko Schwarz, Detlev Marpe, and Thomas Wiegand. 2019. An Affine-Linear Intra Prediction With Complexity Constraints. In 2019 IEEE International Conference on Image Processing (ICIP). IEEE, Taipei, Taiwan, 1089–1093. https://doi.org/10.1109/ICIP.2019.8803724Google ScholarCross Ref
- Julia Stoll. 2023. Number of digital video viewers worldwide from 2019 to 2023. Statista. Retrieved Jun 06, 2023 from https://www.statista.com/statistics/1061017/digital-video-viewers-number-worldwide/Google Scholar
- Gary Sullivan and Thomas Wiegand. 1998. Rate-distortion optimization for video compression. IEEE Signal Processing Magazine 15, 6 (1998), 74–90. https://doi.org/10.1109/79.733497Google ScholarCross Ref
- Liang Zhao, Li Zhang, Siwei Ma, and Debin Zhao. 2011. Fast mode decision algorithm for intra prediction in HEVC. In 2011 Visual Communications and Image Processing (VCIP). IEEE, Tainan, Taiwan, 1–4. https://doi.org/10.1109/VCIP.2011.6115979Google ScholarCross Ref
- Naima Zouidi, Amina Kessentini, Wassim Hamidouche, Nouri Masmoudi, and Daniel Menard. 2023. Complexity assessment of the intra prediction in Versatile Video Coding. Multimedia Tools and Applications (2023), 1–20.Google Scholar
Index Terms
- A Machine Learning-Based Solution to Accelerate the Intra Mode Decision for the VVC Standard
Recommendations
Cascaded mode decision for MPEG-2 to H.264/AVC intra frame transcoding
The newest international video-coding standard H.264/AVC significantly achieves better coding performance compared to prior video coding standards such as MPEG-2, which has been widely used in today's digital video applications. To provide the ...
Texture-Based Fast CU Size Decision and Intra Mode Decision Algorithm for VVC
MultiMedia ModelingAbstractVersatile Video Coding (VVC) is the next generation video coding standard. Compared with HEVC/H.265, in order to improve coding efficiency, its complexity of intra coding increases significantly. Too much encoding time makes it difficult for real-...
AVS2 Fast Intra Mode Decision Hardware Design
EITCE '20: Proceedings of the 2020 4th International Conference on Electronic Information Technology and Computer EngineeringAVS2 is a kind of advanced video coding standard developed by China, which achieves the same compression efficiency as the international standard HEVC/H.265. In intra coding, there are 33 intra prediction modes, and the optimal intra prediction mode is ...
Comments