Learning-based End-to-End Video Compression Using Predictive Coding

• Matheus C. de Oliveira UnB
• Luiz G. R. Martins UnB
• Henrique Costa Jung UnB
• Nilson Donizete Guerin UnB
• Renam Castro da Silva Samsung R&D Brazil
• Eduardo Peixoto UnB
• Bruno Macchiavello UnB
• Edson M. Hung UnB
• Vanessa Testoni Samsung R&D Brazil
• Pedro Garcia Freitas Samsung R&D Brazil

Resumo

Driven by the growing demand for video applications, deep learning techniques have become alternatives for implementing end-to-end encoders to achieve applicable compression rates. Conventional video codecs exploit both spatial and temporal correlation. However, due to some restrictions (e.g. computational complexity), they are commonly limited to linear transformations and translational motion estimation. Autoencoder models open up the way for exploiting predictive end-to-end video codecs without such limitations. This paper presents an entire learning-based video codec that exploits spatial and temporal correlations. The presented codec extends the idea of P-frame prediction presented in our previous work. The architecture adopted for I-frame coding is defined by a variational autoencoder with non-parametric entropy modeling. Besides an entropy model parameterized by a hyperprior, the inter-frame encoder architecture has two other independent networks, responsible for motion estimation and residue prediction. Experimental results indicate that some improvements still have to be incorporated into our codec to overcome the all-intra coding set up regarding the traditional algorithms High Efficiency Video Coding (HEVC) and Versatile Video Coding (VVC).

Palavras-chave: Correlation, Motion estimation, Computer architecture, Predictive models, Video compression, Predictive coding, Entropy, learning based coding, video compression, deep learning, predictive coding