Exploring ERP Distortions to Reduce the Encoding Time of 360 Videos

• Iago Storch UFPel
• Luciano Agostini UFPel
• Bruno Zatt UFPel
• Daniel Palomino UFPel

DOI: https://doi.org/10.5753/sibgrapi.est.2021.20026

Resumo

The encoding of 360 videos presents many challenges. These videos require a planar projection stage before encoding, and there are no studies on how projection interferes with the encoder decisions. Also, due to the increased resolution, encoding 360 videos is computationally costlier than conventional videos. In this work, we perform an extensive evaluation of how the texture distortions caused by the projection interfere with the encoder's behavior and exploit these interferences to propose multiple fast algorithms to accelerate the encoding. Experimental results show that the proposed algorithms reduce encoding time significantly with minor coding efficiency penalties, while the evaluations expose novel insights into the encoder behavior.

Referências

I. C. Storch, “Exploração das distorções da projeção erp para redução de complexidade da codificação de vídeos omnidirecionais,” Master’s thesis, Universidade Federal de Pelotas, 2020. [Online]. Available: http://guaiaca.ufpel.edu.br:8080/handle/prefix/6086

V. Cisco, “Cisco visual networking index: Forecast and methodology, 2017–2022,” CISCO White paper, 2019.

B. Gleb, “Five innovative ways you can use virtual reality in the real estate business,” https://rubygarage.org/blog/virtual-reality-in-real-estate, 2020, [Online; accessed 9-July-2021].

K. DeTrempe, “Hospital-wide access to virtual reality alleviates pain and anxiety for pediatric patients,” [link], 2017, [Online; accessed 9-July-2021].

A. Wexelblat, Virtual reality: applications and explorations. Academic Press, 2014.

A. Torii, A. Imiya, and N. Ohnishi, “Two-and three-view geometry for spherical cameras,” in Proceedings of the sixth workshop on omnidirectional vision, camera networks and non-classical cameras. Citeseer, 2005, pp. 81–88.

Y. Ye, E. Alshima, and J. Boyce, “Jvet-e1003: Algorithm descriptions of projection format conversion and video quality metrics in 360lib,” Joint Video Exploration Team (JVET) of ITU-T SG, vol. 16, 2017.

JCT-VC, “Recommendation itu-t h. 265,” International Standard ISO/IEC, pp. 23 008–2, 2013.

J. Vanne, M. Viitanen, T. D. Hamalainen, and A. Hallapuro, “Comparative rate-distortion-complexity analysis of hevc and avc video codecs,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 22, no. 12, pp. 1885–1898, 2012.

Y. Wang, Y. Li, D. Yang, and Z. Chen, “A fast intra prediction algorithm for 360-degree equirectangular panoramic video,” in 2017 IEEE Visual Communications and Image Processing (VCIP). IEEE, 2017, pp. 1–4.

M. Zhang, X. Dong, Z. Liu, F. Mao, and W. Yue, “Fast intra algorithm based on texture characteristics for 360 videos,” EURASIP Journal on Image and Video Processing, vol. 2019, no. 1, pp. 1–16, 2019.

B. Ray, J. Jung, and M.-C. Larabi, “A low-complexity video encoder for equirectangular projected 360 video content,” in 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2018, pp. 1723–1727.

Z. Liu, P. Song, and M. Zhang, “A cu split early termination algorithm based knn for 360-degree video,” in 2019 Data Compression Conference (DCC). IEEE, 2019, pp. 594–594.

F. Bossen et al., “Common test conditions and software reference configurations,” JCTVC-L1100, vol. 12, no. 7, 2013.

J. Boyce, E. Alshina, A. Abbas, and Y. Ye, “Jvet-h1030: Jvet common test conditions and evaluation procedures for 360° video,” Joint Video Explor. Team, Macau, China, Rep. JVET-H1030, 2017.

JCT-VC. (2017, Aug.) Hevc test model version 16.16. [Online]. Available: https://hevc.hhi.fraunhofer.de/svn/svn_HEVCSoftware/tags/HM-16.16/

B. Zatt, M. Shafique, S. Bampi, and J. Henkel, “An adaptive early skip mode decision scheme for multiview video coding,” in 28th Picture Coding Symposium. IEEE, 2010, pp. 42–45.

P. Gonçalves, G. Correa, M. Porto, B. Zatt, and L. Agostini, “Multiple early-termination scheme for tz search algorithm based on data mining and decision trees,” in 2017 IEEE 19th International Workshop on Multimedia Signal Processing (MMSP). IEEE, 2017, pp. 1–6.

G. Bjontegaard, “Calculation of average psnr differences between rdcurves,” VCEG-M33, 2001.

J. Chen, E. Alshina, G. Sullivan, and J. R. Ohm, “Jvet-c1001: Algorithm description of joint exploration test model 3,” Joint Video Exploration Team (JVET) of ITU-T SG, 2016.

I. Storch, B. Zatt, L. Agostini, G. Correa, L. A. da Silva Cruz, and D. Palomino, “Spatially adaptive intra mode pre-selection for erp 360 video coding,” in ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2020, pp. 2178–2182.

I. Storch, D. Palomino, L. Agostini, B. Zatt, and L. Cruz, “An assessment of hevc intra-frame prediction over 360-degrees videos,” in 18th Microelectronics Students Forum. SBMicro, 2018, pp. 1–4.

I. Storch, L. A. da Silva Cruz, L. Agostini, B. Zatt, and D. Palomino, “The impacts of equirectangular 360-degrees videos in the intra-frame prediction of hevc,” Journal of Integrated Circuits and Systems, vol. 14, no. 1, pp. 1–10, 2019.

I. Storch, G. Correa, B. Zatt, L. Agostini, and D. Palomino, “Esa360- early skip mode decision algorithm for fast erp 360 video coding,” in 2020 28th European Signal Processing Conference (EUSIPCO). IEEE, 2021, pp. 535–539.

I. Storch, L. Agostini, B. Zatt, S. Bampi, and D. Palomino, “Fastinter360: A fast inter mode decision for hevc 360 video coding,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 31, no. 8, p. to appear, 2021.

B. Beling, I. Storch, L. Agostini, B. Zatt, S. Bampi, and D. Palomino, “Erp-based ctu splitting early termination for intra prediction of 360 videos,” in 2020 IEEE International Conference on Visual Communications and Image Processing (VCIP). IEEE, 2020, pp. 359–362.