Real-Time Volumetric Telepresence: A Point-Cloud Overlap System with Integrated Spatial Audio
Resumo
The paper introduces a real-time volumetric telepresence system that creates a spatial overlap between a physical space and its virtual counterpart. Using an Orbbec scanner, the system captures RGB point clouds via C++ and the scanner’s SDK, transmitting data through TCP to a Node.js server. Clients visualize these point clouds using A-Frame and WebSocket connections, enabling immersive 3D interactions complemented by synchronized real-time audio. Our main contribution is a low-cost, web-based pipeline enabling precise physical–virtual overlap of live point clouds with integrated spatial audio for remote collaboration.Referências
Alkhalili, Y., Meuser, T., and Steinmetz, R. (2020). A survey of volumetric content streaming approaches. In 2020 IEEE Sixth International Conference on Multimedia Big Data (BigMM), pages 191–199.
Hudák, M., Korečko, , and Sobota, B. (2020). Advanced user interaction for web-based collaborative virtual reality. In 2020 11th IEEE International Conference on Cognitive Infocommunications (CogInfoCom), pages 000343–000348.
Kim, H. S., Hong, S. J., Yu, C. R., and Gil, Y. H. (2024). Rgb-d surface reconstruction using 3d gaussian splatting. In 2024 15th International Conference on Information and Communication Technology Convergence (ICTC), pages 2217–2218.
Kodama, R., Koge, M., Taguchi, S., and Kajimoto, H. (2017). Coms-vr: Mobile virtual reality entertainment system using electric car and head-mounted display. In 2017 IEEE Symposium on 3D User Interfaces (3DUI), pages 130–133.
Li, C., Zong, T., Hu, Y., Wang, Y., and Liu, Y. (2025). Spatial visibility and temporal dynamics: Rethinking field of view prediction in adaptive point cloud video streaming. In Proceedings of the 16th ACM Multimedia Systems Conference, MMSys ’25, page 24–34, New York, NY, USA. Association for Computing Machinery.
Liu, Z., Li, Q., Chen, X., Wu, C., Ishihara, S., Li, J., and Ji, Y. (2021). Point cloud video streaming: Challenges and solutions. IEEE Network, 35(5):202–209.
Ostrowski, A. and Gaczkowski, P. (2021). Software Architecture with C++: Design modern systems using effective architecture concepts, design patterns, and techniques with C++20. Packt Publishing.
Santos, S. G. and Cardoso, J. C. S. (2019). Web-based virtual reality with a-frame. In 2019 14th Iberian Conference on Information Systems and Technologies (CISTI), pages 1–2.
Zhang, X. (2019). The college english teaching reform supported by multimedia teaching technology and immersive virtual reality technology. In 2019 International Conference on Virtual Reality and Intelligent Systems (ICVRIS), pages 77–80.
Zhang, Z., Wang, C., Weng, D., Liu, Y., and Wang, Y. (2019). Symmetrical reality: Toward a unified framework for physical and virtual reality. In 2019 IEEE Conference on Virtual Reality and 3D User Interfaces (VR), pages 1275–1276.
Zhao, Y., Baghaei, N., Schnack, A., and Stemmet, L. (2021). Assessing telepresence, social presence and stress response in a virtual reality store. In 2021 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct), pages 52–56.
Hudák, M., Korečko, , and Sobota, B. (2020). Advanced user interaction for web-based collaborative virtual reality. In 2020 11th IEEE International Conference on Cognitive Infocommunications (CogInfoCom), pages 000343–000348.
Kim, H. S., Hong, S. J., Yu, C. R., and Gil, Y. H. (2024). Rgb-d surface reconstruction using 3d gaussian splatting. In 2024 15th International Conference on Information and Communication Technology Convergence (ICTC), pages 2217–2218.
Kodama, R., Koge, M., Taguchi, S., and Kajimoto, H. (2017). Coms-vr: Mobile virtual reality entertainment system using electric car and head-mounted display. In 2017 IEEE Symposium on 3D User Interfaces (3DUI), pages 130–133.
Li, C., Zong, T., Hu, Y., Wang, Y., and Liu, Y. (2025). Spatial visibility and temporal dynamics: Rethinking field of view prediction in adaptive point cloud video streaming. In Proceedings of the 16th ACM Multimedia Systems Conference, MMSys ’25, page 24–34, New York, NY, USA. Association for Computing Machinery.
Liu, Z., Li, Q., Chen, X., Wu, C., Ishihara, S., Li, J., and Ji, Y. (2021). Point cloud video streaming: Challenges and solutions. IEEE Network, 35(5):202–209.
Ostrowski, A. and Gaczkowski, P. (2021). Software Architecture with C++: Design modern systems using effective architecture concepts, design patterns, and techniques with C++20. Packt Publishing.
Santos, S. G. and Cardoso, J. C. S. (2019). Web-based virtual reality with a-frame. In 2019 14th Iberian Conference on Information Systems and Technologies (CISTI), pages 1–2.
Zhang, X. (2019). The college english teaching reform supported by multimedia teaching technology and immersive virtual reality technology. In 2019 International Conference on Virtual Reality and Intelligent Systems (ICVRIS), pages 77–80.
Zhang, Z., Wang, C., Weng, D., Liu, Y., and Wang, Y. (2019). Symmetrical reality: Toward a unified framework for physical and virtual reality. In 2019 IEEE Conference on Virtual Reality and 3D User Interfaces (VR), pages 1275–1276.
Zhao, Y., Baghaei, N., Schnack, A., and Stemmet, L. (2021). Assessing telepresence, social presence and stress response in a virtual reality store. In 2021 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct), pages 52–56.
Publicado
30/09/2025
Como Citar
MOREIRA, Vitor S.; SOARES, Marcos F. dos Santos; NASCIMENTO, Hugo A. D. do; SALLES, Laurita R. de.
Real-Time Volumetric Telepresence: A Point-Cloud Overlap System with Integrated Spatial Audio. In: WORKSHOP DE TRABALHOS DE GRADUAÇÃO - SIMPÓSIO DE REALIDADE VIRTUAL E AUMENTADA (SVR), 27. , 2025, Salvador/BA.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2025
.
p. 109-114.
DOI: https://doi.org/10.5753/svr_estendido.2025.15786.
