3D Point-Cloud Quality Assessment Using Color and Geometry Texture Descriptors
Resumo
Since the mid-20th century, the use of digital formats for visual content allowed a great evolution in how society communicates. The Internet and digital broadcast systems introduced in the decade 90 to the wider public allowed an incredible expansion of multimedia consumption by the people, while the telecommunication networks and providers were pushed to their limits to address the growing multimedia content demand. Older electronic imaging systems, notably TV broadcasting systems, were designed after long subjective quality analysis for the definition of parameters like the number of lines of the video. But recent digital visual content services need faster and more affordable ways of evaluating the human perceived quality of the always-evolving multimedia systems. To address the need for automatic quality assessment, in the past decades many visual quality models based on algorithms that run on digital computers have been proposed. While the existing models are remarkably advanced for 2D digital imagery, a new set of immersive media is dawning, with different data structures, to which the 2D methods are not applicable, and need novel quality assessment metrics. These novel dawning immersive media formats provide a 3D visual representation of real objects and scenes. In this new visual format, objects can be captured, compressed, transmitted, and visualized in real-time not anymore as a flat 2D image, but as 3D content, allowing free-viewpoint selection by a consumer of such media. One of the most popular formats for immersive media is Point Cloud (PC), which is composed of points with 3 geometry coordinates plus color information, and sometimes, other information like reflectance and transparency. This work presents a research on the quality assessment of 3D PC based on novel color and geometric texture statistics. Considering that distortions to both color and geometry attributes of 3D visual content affect the perceived visual quality, it is proposed in this work to use both color-based and geometry-based texture descriptors for PC to obtain the visual degradation through their statistics. This work introduces 4 novels PC texture descriptors, 3 of them color-based, while 1 is geometry-based. Also, a new voxelization method is proposed, which converts points to voxels (volume elements), and improves the performance of the color-based texture descriptors. The performance of the proposed PC quality assessment method is among the best of the state-of-the-art PC quality assessment methods while being flexible and extensible to adapt to different types of distortions.
Referências
P. Astola, L. A. da Silva Cruz, E. A. da Silva, T. Ebrahimi, P. G. Freitas, A. Gilles, K.-J. Oh, C. Pagliari, F. Pereira, C. Perra et al., "Jpeg pleno: Standardizing a coding framework and tools for plenoptic imaging modalities," ITU Journal: ICT Discoveries, 2020.
S. Schwarz, M. Preda, V. Baroncini, M. Budagavi, P. Cesar, P. A. Chou, R. A. Cohen, M. Krivokuća, S. Lasserre, Z. Li et al., "Emerging mpeg standards for point cloud compression," IEEE Journal on Emerging and Selected Topics in Circuits and Systems, vol. 9, no. 1, pp. 133-148, 2018.
M. 3DG, "ISO/IEC JTC 1/SC29/WG11: ""call for proposals for point cloud compression v2" MPEG Meeting, Hobart, Australia," International Organization for Standardization, Tech. Rep., April, 2017.
R. Mekuria, C. Tulvan, and Z. Li, "ISO/IEC JTC 1/SC29/WG11 input: "requirements for point cloud compression" MPEG Meeting, Geneva, Switzerland," International Organization for Standardization, Tech. Rep., June, 2016.
D. Graziosi, O. Nakagami, S. Kuma, A. Zaghetto, T. Suzuki, and A. Tabatabai, "An overview of ongoing point cloud compression standardization activities: Video-based (v-pcc) and geometry-based (g-pcc)," APSIPA Transactions on Signal and Information Processing, vol. 9, 2020.
M. Schütz and M. Wimmer, "High-quality point-based rendering using fast single-pass interpolation," in 2015 Digital Heritage, vol. 1. IEEE, 2015, pp. 369-372.
P. Rosenthal and L. Linsen, "Image-space point cloud rendering," in Proceedings of Computer Graphics International. Citeseer, 2008, pp. 136-143.
M. Schütz, B. Kerbl, and M. Wimmer, "Rendering point clouds with compute shaders and vertex order optimization," in Computer Graphics Forum, vol. 40, no. 4. Wiley Online Library, 2021, pp. 115-126.
E. Alexiou and T. Ebrahimi, "On the performance of metrics to predict quality in point cloud representations," in Applications of Digital Image Processing XL, vol. 10396. International Society for Optics and Photonics, 2017, p. 103961H.
——, "On subjective and objective quality evaluation of point cloud geometry," in 2017 Ninth International Conference on Quality of Multimedia Experience (QoMEX). IEEE, 2017, pp. 1-3.
H. Su, Z. Duanmu, W. Liu, Q. Liu, and Z. Wang, "Perceptual quality assessment of 3d point clouds," in 2019 IEEE International Conference on Image Processing (ICIP). IEEE, 2019, pp. 3182-3186.
E. Alexiou and T. Ebrahimi, "Impact of visualisation strategy for subjective quality assessment of point clouds," in 2018 IEEE International Conference on Multimedia & Expo Workshops (ICMEW). IEEE, 2018, pp. 1-6.
E. Alexious, A. M. Pinheiro, C. Duarte, D. Matković, E. Dumić, L. A. da Silva Cruz, L. G. Dmitrović, M. V. Bernardo, M. Pereira, and T. Ebrahimi, "Point cloud subjective evaluation methodology based on reconstructed surfaces," in Applications of Digital Image Processing XLI, vol. 10752. International Society for Optics and Photonics, 2018, p. 107520H.
E. Alexiou and T. Ebrahimi, "Benchmarking of objective quality metrics for colorless point clouds," in 2018 Picture Coding Symposium (PCS). IEEE, 2018, pp. 51-55.
L. A. da Silva Cruz, E. Dumić, E. Alexiou, J. Prazeres, R. Duarte, M. Pereira, A. Pinheiro, and T. Ebrahimi, "Point cloud quality evaluation: Towards a definition for test conditions," in 2019 Eleventh International Conference on Quality of Multimedia Experience (QoMEX). IEEE, 2019, pp. 1-6.
A. Javaheri, C. Brites, F. Pereira, and J. Ascenso, "Subjective and objective quality evaluation of 3d point cloud denoising algorithms," in 2017 IEEE International Conference on Multimedia & Expo Workshops (ICMEW). IEEE, 2017, pp. 1-6.
E. Alexiou, E. Upenik, and T. Ebrahimi, "Towards subjective quality assessment of point cloud imaging in augmented reality," in 2017 IEEE 19th International Workshop on Multimedia Signal Processing (MMSP), 2017, pp. 1-6.
R. Diniz, P. G. Freitas, and M. C. Farias, "Towards a point cloud quality assessment model using local binary patterns," in 2020 Twelfth International Conference on Quality of Multimedia Experience (QoMEX). IEEE, 2020, pp. 1-6.
——, "Local luminance patterns for point cloud quality assessment," in 2020 IEEE 22nd International Workshop on Multimedia Signal Processing (MMSP). IEEE, 2020, pp. 1-6.
——, "Multi-distance point cloud quality assessment," in 2020 IEEE International Conference on Image Processing (ICIP). IEEE, 2020, pp. 3443-3447.
R. Diniz, P. G. Freitas, and M. Farias, "A novel point cloud quality assessment metric based on perceptual color distance patterns," Electronic Imaging, vol. 2021, no. 9, pp. 256-1, 2021.
R. Diniz, P. G. Freitas, and M. C. Farias, "Color and geometry texture descriptors for point-cloud quality assessment," IEEE Signal Processing Letters, vol. 28, pp. 1150-1154, 2021.
R. Diniz, P. Garcia Freitas, and M. C. Farias, "Point cloud quality assessment based on geometry-aware texture descriptors," Computers & Graphics, vol. 103, pp. 31-44, 2022. [Online]. Available: [link]
E. M. Torlig, E. Alexiou, T. A. Fonseca, R. L. de Queiroz, and T. Ebrahimi, "A novel methodology for quality assessment of voxelized point clouds," in Applications of Digital Image Processing XLI, vol. 10752. International Society for Optics and Photonics, 2018, p. 107520I.
E. Alexiou, I. Viola, T. M. Borges, T. A. Fonseca, R. L. De Queiroz, and T. Ebrahimi, "A comprehensive study of the rate-distortion performance in mpeg point cloud compression," APSIPA Transactions on Signal and Information Processing, vol. 8, 2019.
S. Perry, H. P. Cong, L. A. da Silva Cruz, J. Prazeres, M. Pereira, A. Pinheiro, E. Dumic, E. Alexiou, and T. Ebrahimi, "Quality evaluation of static point clouds encoded using mpeg codecs," in 2020 IEEE International Conference on Image Processing (ICIP). IEEE, 2020, pp. 3428-3432.
Q. Yang, H. Chen, Z. Ma, Y. Xu, R. Tang, and J. Sun, "Predicting the perceptual quality of point cloud: A 3d-to-2d projection-based exploration," IEEE Transactions on Multimedia, vol. 23, pp. 3877-3891, 2020.
K. Brunnstrom, D. Hands, F. Speranza, and A.Webster, "Vqeg validation and itu standardization of objective perceptual video quality metrics [standards in a nutshell]," IEEE Signal processing magazine, vol. 26, no. 3, pp. 96-101, 2009.
