Visual Effects for 3D Gaussian Splatting in Extended Reality

• Andre Corrêa Santos Insper
• Luciano Pereira Soares Insper

Resumo

Real-time immersive applications increasingly require 3D content that balances photorealistic rendering with creative visual control, demands that often exceed the capabilities of traditional techniques. This paper introduces a framework for applying selective visual effects to 3D Gaussian Splatting models, designed for augmented and virtual reality environments. The system integrates with both game engines and web-based platforms, leveraging Signed Distance Function based spatial selection to enable real-time manipulation of displacement, relighting, and stylistic effects within localized regions of a scene. The proposed method was validated through an interactive augmented reality application deployed on a consumer-grade headset, demonstrating sub-frame latency and user-driven region control. Additional visual effects, such as geometric transformations, dynamic lighting, and non-photorealistic rendering, showcase the system’s flexibility for expressive content creation. Integration with implicit neural SDFs was also explored to support more complex and artist-defined spatial regions. Together, these capabilities establish the framework as a scalable and performant solution for high-fidelity visual augmentation for 3D Gaussian Splatting in immersive applications.

Palavras-chave: Gaussian Splatting, 3D Rendering, Real-Time Graphics, Game Engines, Signed Distance Functions, Non-Photorealistic Rendering, Unity

Referências

B. Mildenhall, P. P. S. nad Matthew Tancik, J. T. Barron, R. Ramamoorthi, and R. Ng. Nerf Representing scenes as neural radiance fields for view synthesis. arXiv, 1(2003.08934):0–19, 2020. Available: [link]

B. Kerbl, G. Kopanas, T. Leimkuhler, and G. Drettakis. 3D Gaussian splatting for real-time radiance field rendering. ACM Transactions on Graphics, 42(4), July 2023. Available: [link]

G. Feng, S. Chen, R. Fu, Z. Liao, Y. Wang, T. Liu, Z. Pei, H. Li, X. Zhang, and B. Dai. FlashGS Efficient 3D Gaussian splatting for large-scale and high-resolution rendering. 2024. Available: [link]

N. Moenne-Loccoz, A. Mirzaei, O. Perel, R. de Lutio, J. M. Esturo, G. State, S. Fidler, N. Sharp, and Z. Gojcic. 3D Gaussian ray tracing Fast tracing of particle scenes. 2024. Available: [link]

H. Henriques, E. Oliveira, E. Clua, and D. Trevisan. A mixed path tracing and NeRF approach for optimizing rendering in XR displays. Proceedings of the 25th Symposium on Virtual and Augmented Reality, SVR ’23. ACM, 2024, 123–130. DOI: 10.1145/3625008.3625027

H. Henriques, E. Oliveira, E. Clua, and D. Trevisan. Analysing hybrid neural and ray tracing perception for foveated rendering. Proceedings of the 26th Symposium on Virtual and Augmented Reality, SVR ’24. ACM, 2024, 21–30. DOI: 10.1145/3691573.3691580

Z. Zhu, B. Wang, and J. Yang. GS-ROR2 Bidirectional-guided 3DGS and SDF for reflective object relighting and reconstruction. ACM Transactions on Graphics, Aug. 2025. DOI: 10.1145/3759248

S. McAuley, S. Hill, N. Hoffman, Y. Gotanda, B. Smits, B. Burley, and A. Martinez. Practical physically-based shading in film and game production. ACM SIGGRAPH 2012 Courses, SIGGRAPH ’12. ACM, 2012. DOI: 10.1145/2343483.2343493

S. R. Bulo, N. Bartolovic, L. Porzi, and P. Kontschieder. Hardware-rasterized ray-based Gaussian splatting. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2025, 485–494.

O. Wegen, W. Scheibel, M. Trapp, R. Richter, and J. Dollner. A survey on non-photorealistic rendering approaches for point cloud visualization. IEEE Transactions on Visualization and Computer Graphics, PP:1–20, May 2024.

J. Liao, K. Van, Z. Xia, and R. Suzuki. RealityEffects Augmenting 3D volumetric videos with object-centric annotation and dynamic visual effects. Proceedings of the 2024 ACM Designing Interactive Systems Conference, DIS ’24. ACM, 2024, 1248–1261. DOI: 10.1145/3643834.3661631

V. Sitzmann, J. N. P. Martel, A. W. Bergman, D. B. Lindell, and G. Wetzstein. Implicit neural representations with periodic activation functions. CoRR, abs/2006.09661, 2020. Available: [link]

J. Grossman. Point sample rendering. Master’s thesis, Massachusetts Institute of Technology, Cambridge, MA, August 1998.

L. Radl, M. Steiner, M. Parger, A. Weinrauch, B. Kerbl, and M. Steinberger. StopThePop Sorted Gaussian Splatting for view-consistent real-time rendering. ACM Transactions on Graphics, 4(43), 2024.

K. Yurkova. A comprehensive overview of Gaussian Splatting. Towards Data Science, Dec 2023. Available: [link]

Unity Technologies. Unity. Game development platform, 2023. Available: [link]

A. Pranckevicius. UnityGaussianSplatting. GitHub repository, 2024. Available: [link]

F. Wu and Y. Chen. FruitNinja 3D object interior texture generation with Gaussian Splatting. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2025, 11051–11060.

Mozilla. A-Frame A web framework for building virtual reality experiences. 2015. Available: [link]

quadjr, hybridherbst, and kfarr. AFrame-Gaussian-Splatting A-Frame implementation of real-time 3D Gaussian Splatting. GitHub repository, 2023. Available: [link]

D.-Y. Zhong, L.-G. Wang, and L. Bi. Implicit surface reconstruction based on generalized radial basis functions interpolant with distinct constraints. Applied Mathematical Modelling, 71:408–420, 2019. Available: [link]