Multi-Resolution Denoising for Foveated Rendering Path Tracing
Resumo
Path tracing rendering algorithms present a significant challenge for interactive applications due to the high sample counts required per pixel to achieve low-variance images. To mitigate this issue, denoising techniques have been developed to reduce computational costs by predicting the fully converged render from noisy images generated with fewer samples per pixel. This challenge is exacerbated when rendering for head-mounted displays, where the proximity of the user’s eyes to the screen increases computational demands. However, this proximity can be leveraged to reduce the time required to generate the final image. In this study, we propose a method for accelerating path tracing rendering specifically designed for extended reality (XR) devices. Our approach incorporates multi-resolution foveated rendering techniques combined with multi-pass nonhomogenous denoising strategies for different image regions. Experimental results in testing scenarios with high triangle counts demonstrate significant improvements in both image quality and acceleration rates, rendering frames three times faster than standard path tracing strategies.
Palavras-chave:
Denoising, Foveated Rendering, Path Tracing, Extended Reality
Referências
Attila T. Afra. Intel Open Image Denoise, 2024.
Steve Bako, Mark Meyer, Tony DeRose, and Pradeep Sen. Offline deep importance sampling for monte carlo path tracing. In Computer Graphics Forum, volume 38, pages 527–542. Wiley Online Library, 2019.
Brian Caulfield. What Is Path Tracing?, March 2022.
Chakravarty R. Alla Chaitanya, Anton S. Kaplanyan, Christoph Schied, Marco Salvi, Aaron Lefohn, Derek Nowrouzezahrai, and Timo Aila. Interactive reconstruction of monte carlo image sequences using a recurrent denoising autoencoder. ACM Trans. Graph. 36(4), Jul 2017.
David Cline, Justin Talbot, and Parris Egbert. Energy redistribution path tracing. ACM Transactions on Graphics (TOG), 24(3):1186–1195, 2005.
Robert L. Cook, Thomas Porter, and Loren Carpenter. Distributed ray tracing. In Proceedings of the 11th Annual Conference on Computer Graphics and Interactive Techniques, SIGGRAPH ’84, page 137–145, New York, NY, USA, 1984. Association for Computing Machinery.
Holger Dammertz, Daniel Sewtz, Johannes Hanika, and Hendrik PA Lensch. Edge-avoiding a-trous wavelet transform for fast global illumination filtering. In Proceedings of the Conference on High Performance Graphics, pages 67–75, 2010.
Ethan Einhorn and Jason Mawdsley. Generate Groundbreaking Ray-Traced Images with Next-Generation NVIDIA DLSS, August 2023.
Brian Guenter, Mark Finch, Steven Drucker, Desney Tan, and John Snyder. Foveated 3D graphics. ACM Trans. Graph. 31(6), Nov 2012.
Eric Haines and Tomas Akenine-Moller, editors. Ray Tracing Gems. Apress, 2019. [link].
Henrik Wann Jensen. Importance driven path tracing using the photon map. In Rendering Techniques’ 95: Proceedings of the Eurographics Workshop in Dublin, Ireland, June 12–14, 1995 6, pages 326–335. Springer, 1995.
James T. Kajiya. The rendering equation. In Proceedings of the 13th Annual Conference on Computer Graphics and Interactive Techniques, SIGGRAPH ’86, page 143–150, New York, NY, USA, 1986. Association for Computing Machinery.
Markus Kettunen, Marco Manzi, Miika Aittala, Jaakko Lehtinen, Fredo Durand, and Matthias Zwicker. Gradient-domain path tracing. ACM Transactions on Graphics (TOG), 34(4):1–13, 2015.
Matias Koskela, Atro Lotvonen, Markku Makitalo, Petrus Kivi, Timo Viitanen, and Pekka Jääskeläinen. Foveated Real-Time Path Tracing in Visual-Polar Space. In Tamy Boubekeur and Pradeep Sen, editors, Eurographics Symposium on Rendering - DL-only and Industry Track. The Eurographics Association, 2019.
Matias Koskela, Timo Viitanen, Pekka Jääskeläinen, and Jarmo Takala. Foveated path tracing. In George Bebis, Richard Boyle, Bahram Parvin, Darko Koracin, Fatih Porikli, Sandra Skaff, Alireza Entezari, Jianyuan Min, Daisuke Iwai, Amela Sadagic, Carlos Scheidegger, and Tobias Isenberg, editors, Advances in Visual Computing, pages 723–732, Cham, 2016. Springer International Publishing.
Marc Levoy, J Gerth, Brian Curless, and K Pull. The Stanford 3D Scanning Repository. 5(10), 2023.
Marc Levoy and Ross Whitaker. Gaze-directed volume rendering. In Proceedings of the 1990 Symposium on Interactive 3D Graphics, I3D ’90, page 217–223, New York, NY, USA, 1990. Association for Computing Machinery.
Rafal K. Mantiuk, Gyorgy Denes, Alexandre Chapiro, Anton Kaplanyan, Gizem Rufo, Romain Bachy, Trisha Lian, and Anjul Patney. Fovvideovdp A visible difference predictor for wide field-of-view video. ACM Transactions on Graphics (TOG), 40(4):1–19, 2021.
NVIDIA. NVIDIA Nsight Systems, 2024.
NVIDIA. NVIDIA OptiX AI-Accelerated Denoiser, 2024.
NVIDIA. OptiX 8.0 Programming Guide, 2024.
NVIDIA. vk denoise, 2024.
Victor Peres, Esteban Clua, Thiago Porcino, and Anselmo Montenegro. Non-homogeneous denoising for virtual reality in real-time path tracing rendering. Graphical Models, 129:101184, 2023.
Maria Perez-Ortiz, Aliaksei Mikhailiuk, Emin Zerman, Vedad Hulusic, Giuseppe Valenzise, and Rafal K. Mantiuk. From pairwise comparisons and rating to a unified quality scale. IEEE Transactions on Image Processing, 29:1139–1151, 2019.
Steve Hollasch, Peter Shirley, Trevor David Black. Ray tracing in one weekend, August 2024. [link].
Christoph Schied, Anton Kaplanyan, Chris Wyman, Anjul Patney, Chakravarty R. Alla Chaitanya, John Burgess, Shiqiu Liu, Carsten Dachsbacher, Aaron Lefohn, and Marco Salvi. Spatiotemporal variance-guided filtering real-time reconstruction for path-traced global illumination. In Proceedings of High Performance Graphics, HPG ’17, New York, NY, USA, 2017. Association for Computing Machinery.
R. Szeliski. Computer Vision Algorithms and Applications. Texts in Computer Science. Springer International Publishing, 2022.
The Khronos Group Inc. Vulkan cross platform 3D graphics, 2024.
Thijs Vogels, Fabrice Rousselle, Brian McWilliams, Gerhard Rothlin, Alex Harvill, David Adler, Mark Meyer, and Jan Novak. Denoising with kernel prediction and asymmetric loss functions. ACM Transactions on Graphics (TOG), 37(4):1–15, 2018.
Lili Wang, Xuehuai Shi, and Yi Liu. Foveated rendering A state-of-the-art survey. Computational Visual Media, 9(2):195–228, Jun 2023.
Zhou Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli. Image quality assessment from error visibility to structural similarity. IEEE Transactions on Image Processing, 13(4):600–612, 2004.
Martin Weier, Thorsten Roth, Ernst Kruijff, Andre Hinkenjann, Arsene Perard-Gayot, Philipp Slusallek, and Yongmin Li. Foveated real-time ray tracing for head-mounted displays. Comput. Graph. Forum, 35(7):289–298, Oct 2016.
Turner Whitted. An improved illumination model for shaded display. Commun. ACM, 23(6):343–349, Jun 1980.
Richard Zhang, Phillip Isola, Alexei A. Efros, Eli Shechtman, and Oliver Wang. The unreasonable effectiveness of deep features as a perceptual metric. In CVPR, 2018.
Steve Bako, Mark Meyer, Tony DeRose, and Pradeep Sen. Offline deep importance sampling for monte carlo path tracing. In Computer Graphics Forum, volume 38, pages 527–542. Wiley Online Library, 2019.
Brian Caulfield. What Is Path Tracing?, March 2022.
Chakravarty R. Alla Chaitanya, Anton S. Kaplanyan, Christoph Schied, Marco Salvi, Aaron Lefohn, Derek Nowrouzezahrai, and Timo Aila. Interactive reconstruction of monte carlo image sequences using a recurrent denoising autoencoder. ACM Trans. Graph. 36(4), Jul 2017.
David Cline, Justin Talbot, and Parris Egbert. Energy redistribution path tracing. ACM Transactions on Graphics (TOG), 24(3):1186–1195, 2005.
Robert L. Cook, Thomas Porter, and Loren Carpenter. Distributed ray tracing. In Proceedings of the 11th Annual Conference on Computer Graphics and Interactive Techniques, SIGGRAPH ’84, page 137–145, New York, NY, USA, 1984. Association for Computing Machinery.
Holger Dammertz, Daniel Sewtz, Johannes Hanika, and Hendrik PA Lensch. Edge-avoiding a-trous wavelet transform for fast global illumination filtering. In Proceedings of the Conference on High Performance Graphics, pages 67–75, 2010.
Ethan Einhorn and Jason Mawdsley. Generate Groundbreaking Ray-Traced Images with Next-Generation NVIDIA DLSS, August 2023.
Brian Guenter, Mark Finch, Steven Drucker, Desney Tan, and John Snyder. Foveated 3D graphics. ACM Trans. Graph. 31(6), Nov 2012.
Eric Haines and Tomas Akenine-Moller, editors. Ray Tracing Gems. Apress, 2019. [link].
Henrik Wann Jensen. Importance driven path tracing using the photon map. In Rendering Techniques’ 95: Proceedings of the Eurographics Workshop in Dublin, Ireland, June 12–14, 1995 6, pages 326–335. Springer, 1995.
James T. Kajiya. The rendering equation. In Proceedings of the 13th Annual Conference on Computer Graphics and Interactive Techniques, SIGGRAPH ’86, page 143–150, New York, NY, USA, 1986. Association for Computing Machinery.
Markus Kettunen, Marco Manzi, Miika Aittala, Jaakko Lehtinen, Fredo Durand, and Matthias Zwicker. Gradient-domain path tracing. ACM Transactions on Graphics (TOG), 34(4):1–13, 2015.
Matias Koskela, Atro Lotvonen, Markku Makitalo, Petrus Kivi, Timo Viitanen, and Pekka Jääskeläinen. Foveated Real-Time Path Tracing in Visual-Polar Space. In Tamy Boubekeur and Pradeep Sen, editors, Eurographics Symposium on Rendering - DL-only and Industry Track. The Eurographics Association, 2019.
Matias Koskela, Timo Viitanen, Pekka Jääskeläinen, and Jarmo Takala. Foveated path tracing. In George Bebis, Richard Boyle, Bahram Parvin, Darko Koracin, Fatih Porikli, Sandra Skaff, Alireza Entezari, Jianyuan Min, Daisuke Iwai, Amela Sadagic, Carlos Scheidegger, and Tobias Isenberg, editors, Advances in Visual Computing, pages 723–732, Cham, 2016. Springer International Publishing.
Marc Levoy, J Gerth, Brian Curless, and K Pull. The Stanford 3D Scanning Repository. 5(10), 2023.
Marc Levoy and Ross Whitaker. Gaze-directed volume rendering. In Proceedings of the 1990 Symposium on Interactive 3D Graphics, I3D ’90, page 217–223, New York, NY, USA, 1990. Association for Computing Machinery.
Rafal K. Mantiuk, Gyorgy Denes, Alexandre Chapiro, Anton Kaplanyan, Gizem Rufo, Romain Bachy, Trisha Lian, and Anjul Patney. Fovvideovdp A visible difference predictor for wide field-of-view video. ACM Transactions on Graphics (TOG), 40(4):1–19, 2021.
NVIDIA. NVIDIA Nsight Systems, 2024.
NVIDIA. NVIDIA OptiX AI-Accelerated Denoiser, 2024.
NVIDIA. OptiX 8.0 Programming Guide, 2024.
NVIDIA. vk denoise, 2024.
Victor Peres, Esteban Clua, Thiago Porcino, and Anselmo Montenegro. Non-homogeneous denoising for virtual reality in real-time path tracing rendering. Graphical Models, 129:101184, 2023.
Maria Perez-Ortiz, Aliaksei Mikhailiuk, Emin Zerman, Vedad Hulusic, Giuseppe Valenzise, and Rafal K. Mantiuk. From pairwise comparisons and rating to a unified quality scale. IEEE Transactions on Image Processing, 29:1139–1151, 2019.
Steve Hollasch, Peter Shirley, Trevor David Black. Ray tracing in one weekend, August 2024. [link].
Christoph Schied, Anton Kaplanyan, Chris Wyman, Anjul Patney, Chakravarty R. Alla Chaitanya, John Burgess, Shiqiu Liu, Carsten Dachsbacher, Aaron Lefohn, and Marco Salvi. Spatiotemporal variance-guided filtering real-time reconstruction for path-traced global illumination. In Proceedings of High Performance Graphics, HPG ’17, New York, NY, USA, 2017. Association for Computing Machinery.
R. Szeliski. Computer Vision Algorithms and Applications. Texts in Computer Science. Springer International Publishing, 2022.
The Khronos Group Inc. Vulkan cross platform 3D graphics, 2024.
Thijs Vogels, Fabrice Rousselle, Brian McWilliams, Gerhard Rothlin, Alex Harvill, David Adler, Mark Meyer, and Jan Novak. Denoising with kernel prediction and asymmetric loss functions. ACM Transactions on Graphics (TOG), 37(4):1–15, 2018.
Lili Wang, Xuehuai Shi, and Yi Liu. Foveated rendering A state-of-the-art survey. Computational Visual Media, 9(2):195–228, Jun 2023.
Zhou Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli. Image quality assessment from error visibility to structural similarity. IEEE Transactions on Image Processing, 13(4):600–612, 2004.
Martin Weier, Thorsten Roth, Ernst Kruijff, Andre Hinkenjann, Arsene Perard-Gayot, Philipp Slusallek, and Yongmin Li. Foveated real-time ray tracing for head-mounted displays. Comput. Graph. Forum, 35(7):289–298, Oct 2016.
Turner Whitted. An improved illumination model for shaded display. Commun. ACM, 23(6):343–349, Jun 1980.
Richard Zhang, Phillip Isola, Alexei A. Efros, Eli Shechtman, and Oliver Wang. The unreasonable effectiveness of deep features as a perceptual metric. In CVPR, 2018.
Publicado
30/09/2025
Como Citar
SILVA, Leonardo Noia Nanci de Araujo; HENRIQUES, Horácio Brescia Macedo; PORCINO, Thiago; MONTENEGRO, Anselmo Antunes; CLUA, Esteban W..
Multi-Resolution Denoising for Foveated Rendering Path Tracing. In: SIMPÓSIO DE REALIDADE VIRTUAL E AUMENTADA (SVR), 27. , 2025, Salvador/BA.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2025
.
p. 199-205.
