A Machine-Learning-Driven Fast Video-based Point Cloud Compression (V-PCC)
Resumo
In recent years, 3D point cloud content has gained attention due to its application possibilities, such as multimedia systems, virtual, augmented, and mixed reality, through the mapping and visualization of environments and/or 3D objects, real-time immersive communications, and autonomous driving systems. However, raw point clouds demand a large amount of data for their representation, and compression is mandatory to allow efficient transmission and storage. The MPEG group proposed the Video-based Point Cloud Compression (V-PCC) standard, which is a dynamic point cloud encoder based on the use of video encoders through projections into 2D space. However, V-PCC demands a high computational cost, demanding fast implementations for real-time processing and, especially, for mobile device applications. In this paper, a machine-learning-based fast implementation of V-PCC is proposed, where the main approach is the use of trained decision trees to speed up the block partitioning process during the point cloud compression. The results show that the proposed fast V-PCC solution is able to achieve an encoding time reduction of 42.73% for the geometry video sub-stream and 55.3% for the attribute video sub-stream, with a minimal impact on bitrate and objective quality.
Referências
Benjamin Bross, Ye-Kui Wang, Yan Ye, Shan Liu, Jianle Chen, Gary J Sullivan, and Jens-Rainer Ohm. 2021. Overview of the versatile video coding (VVC) standard and its applications. IEEE Transactions on Circuits and Systems for Video Technology 31, 10 (2021), 3736–3764.
Guilherme Correa, Pedro Assuncao, Luis A. da Silva Cruz, and Luciano Agostini. 2014. Classification-based early termination for coding tree structure decision in HEVC. In 2014 21st IEEE International Conference on Electronics, Circuits and Systems (ICECS). 239–242.
Guilherme Correa, Pedro A. Assuncao, Luciano Volcan Agostini, and Luis A. da Silva Cruz. 2015. Fast HEVC Encoding Decisions Using Data Mining. IEEE Transactions on Circuits and Systems for Video Technology 25, 4 (2015), 660–673.
Tianyu Dong, Kyutae Kim, and Euee S. Jang. 2021. Performance Evaluation of the Codec Agnostic Approach in MPEG-I Video-Based Point Cloud Compression. IEEE Acessado: 01-09-2021.
Jay Fenlason. 2024. Gprof. [link]. Acessado: 15-04-2024.
Wei Gao, Hang Yuan, Ge Li, Zhu Li, and Hui Yuan. 2023. Low Complexity Coding Unit Decision for Video-Based Point Cloud Compression. IEEE Transactions on Image Processing 33 (2023), 149–162.
Danillo Graziosi, Ohji Nakagami, Satoru Kuma, Alexandre Zaghetto, Teruhiko Suzuki, and Ali Tabatabai. 2020. An overview of ongoing point cloud compression standardization activities: Video-based (V-PCC) and geometry-based (G-PCC). APSIPA Transactions on Signal and Information Processing 9 (2020), e13.
Dan Grois, Detlev Marpe, Amit Mulayoff, Benaya Itzhaky, and Ofer Hadar. 2013. Performance comparison of H.265/MPEG-HEVC, VP9, and H.264/MPEG-AVC encoders. In 2013 Picture Coding Symposium (PCS). 394–397.
Il-Koo Kim, Junghye Min, Tammy Lee, Woo-Jin Han, and JeongHoon Park. 2012. Block partitioning structure in the HEVC standard. IEEE transactions on circuits and systems for video technology 22, 12 (2012), 1697–1706.
Tianyi Li, Mai Xu, and Xin Deng. 2017. A deep convolutional neural network approach for complexity reduction on intra-mode HEVC. In 2017 IEEE International Conference on Multimedia and Expo (ICME). IEEE, 1255–1260.
Yue Li, Jun Huang, Chaofeng Wang, and Hongyue Huang. 2024. Unsupervised learning-based fast CU size decision for geometry videos in V-PCC. Journal of Real-Time Image Processing 21, 1 (2024), 11.
MPEG. 2020. Common Test Conditions for V3C and V-PCC. ISO/IEC JTC 1/SC 29/WG 11 (2020).
MPEG. 2024. Video Point Cloud Compression - VPCC - mpeg-pcc-tmc2 test model candidate software. [link].
Sang-hyo Park and Je-Won Kang. 2020. Fast multi-type tree partitioning for versatile video coding using a lightweight neural network. IEEE Transactions on Multimedia 23 (2020), 4388–4399.
Marius Preda. 2020. V-PCC codec description. ISO/IEC JTC 1/SC 29/WG 7, Virtual (2020).
R Schaefer. 2017. Call for proposals for point cloud compression V2. In ISO/IEC JTC1 SC29/WG11 MPEG, 117th Meeting. Hobart, TAS.
Yun Song, Biao Zeng, Miaohui Wang, and Zelin Deng. 2022. An efficient low-complexity block partition scheme for VVC intra coding. Journal of Real-Time Image Processing (2022), 1–12.
Gary J Sullivan, Jens-Rainer Ohm, Woo-Jin Han, and Thomas Wiegand. 2012. Overview of the high efficiency video coding (HEVC) standard. IEEE Transactions on circuits and systems for video technology 22, 12 (2012), 1649–1668.
Vivienne Sze, Madhukar Budagavi, and Gary J Sullivan. 2014. High efficiency video coding (HEVC). In Integrated circuit and systems, algorithms and architectures. Vol. 39. Springer, 40.
Alexandre Tissier, Wassim Hamidouche, Jarno Vanne, F Galpin, and Daniel Menard. 2020. CNN oriented complexity reduction of VVC intra encoder. In 2020 IEEE International Conference on Image Processing (ICIP). IEEE, 3139–3143.
Yihan Wang, Yongfang Wang, Tengyao Cui, and Zhijun Fang. 2024. Fast Video-Based Point Cloud Compression Based on Early Termination and Transformer Model. IEEE Transactions on Emerging Topics in Computational Intelligence (2024).
Natasha Westland, André Seixas Dias, and Marta Mrak. 2019. Decision trees for complexity reduction in video compression. In 2019 IEEE International Conference on Image Processing (ICIP). IEEE, 2666–2670.
Yun Zhang, Sam Kwong, and Shiqi Wang. 2020. Machine learning based video coding optimizations: A survey. Information Sciences 506 (2020), 395–423.
