Fast AV1 LocalWarped Motion Compensation Using Machine Learning
Resumo
The growing demand for high-efficiency video compression has driven the development of advanced codecs like AV1, which achieve superior compression rates but face challenges related to computational complexity. This paper addresses these challenges by proposing a machine learning-based optimization for the AV1 Local Warped Motion Compensation (LWMC) tool. This solution uses a Decision Tree model to skip unnecessary LWMC executions, reducing its processing time by 52% while maintaining a low impact on coding efficiency of only 0.21% in BD-BR. Compared to complete LWMC deactivation, our method demonstrates significantly better performance, particularly for content with complex motion patterns. To the best of the author’s knowledge, this is the first work in the literature to explore machine learning-based solutions applied to the AV1 LWMC tool.
Palavras-chave:
Video Coding, AV1, LocalWarped Motion Compensation, Complexity Reduction, Machine Learning
Referências
Codec Working Group Alliance for Open Media. 2021. AOM Common Test Conditions v2.0. [link]
AOM. 2019. AV1 Video Codec. [link]
Isis Bender, Alex Borges, Luciano Agostini, Bruno Zatt, Guilherme Correa, and Marcelo Porto. 2023. Complexity and compression efficiency analysis of libaom AV1 video codec. J. Real-Time Image Process. 20, 3 (may 2023), 14 pages. DOI: 10.1007/s11554-023-01308-5
Gisle Bjontegaard. 2008. Improvements of the BD-PSNR model. ITU-T SC16/Q6. In 35th VCEG Meeting, Berlin, Germany, Doc. VCEG-AI11.
L. Breiman, J. Friedman, C.J. Stone, and R.A. Olshen. 1984. Classification and Regression Trees. Taylor & Francis. [link]
Yue Chen, Debargha Mukherjee, Jingning Han, Adrian Grange, Yaowu Xu, Zoe Liu, Sarah Parker, Cheng Chen, Hui Su, Urvang Joshi, Ching-Han Chiang, Yunqing Wang, Paul Wilkins, Jim Bankoski, Luc Trudeau, Nathan Egge, Jean-Marc Valin, Thomas Davies, Steinar Midtskogen, and Peter Rivaz. 2018. An Overview of Core Coding Tools in the AV1 Video Codec. 41–45. DOI: 10.1109/PCS.2018.8456249
Robson Domanski, William Kolodziejski, Wagner Penny, Marcelo Porto, Bruno Zatt, and Luciano Agostini. 2023. High-Throughput and Multiplierless Hardware Design for the AV1 Local Warped MC Interpolation. In 2023 IEEE International Conference on Image Processing (ICIP). 2680–2684. DOI: 10.1109/ICIP49359.2023.10222657
Anna Veronika Dorogush, Vasily Ershov, and Andrey Gulin. 2018. CatBoost: gradient boosting with categorical features support. arXiv:1810.11363 [cs.LG] [link]
Adson Duarte, Paulo Gonçalves, Luciano Agostini, Bruno Zatt, Guilherme Correa, Marcelo Porto, and Daniel Palomino. 2022. Fast Affine Motion Estimation for VVC using Machine-Learning-Based Early Search Termination. In 2022 IEEE International Symposium on Circuits and Systems (ISCAS). 1–5. DOI: 10.1109/ISCAS48785.2022.9937973
Alliance for Open Media. 2022. Local Warped Motion Compensation. [link]
Alliance for Open Media. 2022. Transform Type and Transform Size Search. [link]
Jerome Friedman, Trevor Hastie, and Robert Tibshirani. 2000. Special Invited Paper. Additive Logistic Regression: A Statistical View of Boosting. The Annals of Statistics 28, 2 (2000), 337–374. [link]
Jingning Han, Bohan Li, Debargha Mukherjee, Ching-Han Chiang,Adrian Grange, Cheng Chen, Hui Su, Sarah Parker, Sai Deng, Urvang Joshi, Yue Chen, Yunqing Wang, Paul Wilkins, Yaowu Xu, and James Bankoski. 2021. A Technical Overview of AV1. Proc. IEEE 109, 9 (2021), 1435–1462. DOI: 10.1109/JPROC.2021.3058584
W. Kolodziejski. 2025. List of Features. [link]
W. Kolodziejski. 2025. List of Sequences. [link]
W. Kolodziejski. 2025. List of Used Features. [link]
William Kolodziejski, Robson Domanski, and Luciano Agostini. 2024. Speeding Up the AV1 GlobalWarped Motion Compensation. In 2024 IEEE 15th Latin America Symposium on Circuits and Systems (LASCAS). 1–5. DOI: 10.1109/LASCAS60203.2024.10506160
William Kolodziejski, Robson Domanski, and Luciano Agostini. 2025. FastGW: A Machine Learning-Based Early Skip for the AV1 Global Warped Motion Compensation. IEEE Transactions on Circuits and Systems I: Regular Papers 72, 3 (2025), 977–988. DOI: 10.1109/TCSI.2024.3486243
Xiph.org Test Media. 2004. Collection of test sequences and clips for evaluating compression technology. [link]
Tung Nguyen and Detlev Marpe. 2018. Future Video Coding Technologies: A Performance Evaluation of AV1, JEM, VP9, and HM. In 2018 Picture Coding Symposium (PCS). 31–35. DOI: 10.1109/PCS.2018.8456289
Sarah Parker, Yue Chen, David Barker, Peter Rivaz, and Debargha Mukherjee. 2017. Global and locally adaptive warped motion compensation in video compression. 275–279. DOI: 10.1109/ICIP.2017.8296286
Liudmila Prokhorenkova, Gleb Gusev, Aleksandr Vorobev, Anna Veronika Dorogush, and Andrey Gulin. 2019. CatBoost: unbiased boosting with categorical features. arXiv:1706.09516 [cs.LG] [link]
Payam Refaeilzadeh, Lei Tang, and Huan Liu. 2009. Cross-Validation. Springer US, Boston, MA, 532–538. DOI: 10.1007/978-0-387-39940-9_565
Fernando Sagrilo, Marta Loose, Ramiro Viana, Gustavo Sanchez, Guilherme Corrêa, and Luciano Agostini. 2023. Learning-Based Fast VVC Affine Motion Estimation. In 2023 IEEE International Symposium on Circuits and Systems (ISCAS). 1–5. DOI: 10.1109/ISCAS46773.2023.10181659
Ramiro Viana, Marta Loose, Rafael Ferreira, Marcelo Porto, Guilherme Corrêa, and Luciano Agostini. 2024. A Hardware-Friendly Acceleration of VVC Affine Motion Estimation Using Decision Trees. In 2024 37th SBC/SBMicro/IEEE Symposium on Integrated Circuits and Systems Design (SBCCI). 1–5. DOI: 10.1109/SBCCI62366.2024.10703987
WebM. 2013. VP9 Video Codec. [link]
AOM. 2019. AV1 Video Codec. [link]
Isis Bender, Alex Borges, Luciano Agostini, Bruno Zatt, Guilherme Correa, and Marcelo Porto. 2023. Complexity and compression efficiency analysis of libaom AV1 video codec. J. Real-Time Image Process. 20, 3 (may 2023), 14 pages. DOI: 10.1007/s11554-023-01308-5
Gisle Bjontegaard. 2008. Improvements of the BD-PSNR model. ITU-T SC16/Q6. In 35th VCEG Meeting, Berlin, Germany, Doc. VCEG-AI11.
L. Breiman, J. Friedman, C.J. Stone, and R.A. Olshen. 1984. Classification and Regression Trees. Taylor & Francis. [link]
Yue Chen, Debargha Mukherjee, Jingning Han, Adrian Grange, Yaowu Xu, Zoe Liu, Sarah Parker, Cheng Chen, Hui Su, Urvang Joshi, Ching-Han Chiang, Yunqing Wang, Paul Wilkins, Jim Bankoski, Luc Trudeau, Nathan Egge, Jean-Marc Valin, Thomas Davies, Steinar Midtskogen, and Peter Rivaz. 2018. An Overview of Core Coding Tools in the AV1 Video Codec. 41–45. DOI: 10.1109/PCS.2018.8456249
Robson Domanski, William Kolodziejski, Wagner Penny, Marcelo Porto, Bruno Zatt, and Luciano Agostini. 2023. High-Throughput and Multiplierless Hardware Design for the AV1 Local Warped MC Interpolation. In 2023 IEEE International Conference on Image Processing (ICIP). 2680–2684. DOI: 10.1109/ICIP49359.2023.10222657
Anna Veronika Dorogush, Vasily Ershov, and Andrey Gulin. 2018. CatBoost: gradient boosting with categorical features support. arXiv:1810.11363 [cs.LG] [link]
Adson Duarte, Paulo Gonçalves, Luciano Agostini, Bruno Zatt, Guilherme Correa, Marcelo Porto, and Daniel Palomino. 2022. Fast Affine Motion Estimation for VVC using Machine-Learning-Based Early Search Termination. In 2022 IEEE International Symposium on Circuits and Systems (ISCAS). 1–5. DOI: 10.1109/ISCAS48785.2022.9937973
Alliance for Open Media. 2022. Local Warped Motion Compensation. [link]
Alliance for Open Media. 2022. Transform Type and Transform Size Search. [link]
Jerome Friedman, Trevor Hastie, and Robert Tibshirani. 2000. Special Invited Paper. Additive Logistic Regression: A Statistical View of Boosting. The Annals of Statistics 28, 2 (2000), 337–374. [link]
Jingning Han, Bohan Li, Debargha Mukherjee, Ching-Han Chiang,Adrian Grange, Cheng Chen, Hui Su, Sarah Parker, Sai Deng, Urvang Joshi, Yue Chen, Yunqing Wang, Paul Wilkins, Yaowu Xu, and James Bankoski. 2021. A Technical Overview of AV1. Proc. IEEE 109, 9 (2021), 1435–1462. DOI: 10.1109/JPROC.2021.3058584
W. Kolodziejski. 2025. List of Features. [link]
W. Kolodziejski. 2025. List of Sequences. [link]
W. Kolodziejski. 2025. List of Used Features. [link]
William Kolodziejski, Robson Domanski, and Luciano Agostini. 2024. Speeding Up the AV1 GlobalWarped Motion Compensation. In 2024 IEEE 15th Latin America Symposium on Circuits and Systems (LASCAS). 1–5. DOI: 10.1109/LASCAS60203.2024.10506160
William Kolodziejski, Robson Domanski, and Luciano Agostini. 2025. FastGW: A Machine Learning-Based Early Skip for the AV1 Global Warped Motion Compensation. IEEE Transactions on Circuits and Systems I: Regular Papers 72, 3 (2025), 977–988. DOI: 10.1109/TCSI.2024.3486243
Xiph.org Test Media. 2004. Collection of test sequences and clips for evaluating compression technology. [link]
Tung Nguyen and Detlev Marpe. 2018. Future Video Coding Technologies: A Performance Evaluation of AV1, JEM, VP9, and HM. In 2018 Picture Coding Symposium (PCS). 31–35. DOI: 10.1109/PCS.2018.8456289
Sarah Parker, Yue Chen, David Barker, Peter Rivaz, and Debargha Mukherjee. 2017. Global and locally adaptive warped motion compensation in video compression. 275–279. DOI: 10.1109/ICIP.2017.8296286
Liudmila Prokhorenkova, Gleb Gusev, Aleksandr Vorobev, Anna Veronika Dorogush, and Andrey Gulin. 2019. CatBoost: unbiased boosting with categorical features. arXiv:1706.09516 [cs.LG] [link]
Payam Refaeilzadeh, Lei Tang, and Huan Liu. 2009. Cross-Validation. Springer US, Boston, MA, 532–538. DOI: 10.1007/978-0-387-39940-9_565
Fernando Sagrilo, Marta Loose, Ramiro Viana, Gustavo Sanchez, Guilherme Corrêa, and Luciano Agostini. 2023. Learning-Based Fast VVC Affine Motion Estimation. In 2023 IEEE International Symposium on Circuits and Systems (ISCAS). 1–5. DOI: 10.1109/ISCAS46773.2023.10181659
Ramiro Viana, Marta Loose, Rafael Ferreira, Marcelo Porto, Guilherme Corrêa, and Luciano Agostini. 2024. A Hardware-Friendly Acceleration of VVC Affine Motion Estimation Using Decision Trees. In 2024 37th SBC/SBMicro/IEEE Symposium on Integrated Circuits and Systems Design (SBCCI). 1–5. DOI: 10.1109/SBCCI62366.2024.10703987
WebM. 2013. VP9 Video Codec. [link]
Publicado
10/11/2025
Como Citar
KOLODZIEJSKI, William; BRAGA, Leonardo; REZENDE, Marcelo; PORTO, Marcelo; AGOSTINI, Luciano.
Fast AV1 LocalWarped Motion Compensation Using Machine Learning. In: BRAZILIAN SYMPOSIUM ON MULTIMEDIA AND THE WEB (WEBMEDIA), 31. , 2025, Rio de Janeiro/RJ.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2025
.
p. 545-549.
DOI: https://doi.org/10.5753/webmedia.2025.14161.
