R3-D3: (Photo-)Realistic Real-time Representations of Dynamic 3D-Driven Industrial Trainings
Abstract
Extended Reality (XR) technologies offer transformative potential to enhance remote communication in industrial settings, particularly by facilitating immersive and interactive experiences that bridge physical distances. In this paper, we outline some of the main challenges based on the shortage of experts and deficiencies of the quality of technical skills, of personnel. We also outline how Extended Reality can help with technical training and communication in the industry to mitigate these issues. We outline three different representation formats for (photo-)realistic real-time representations and give concrete examples on how these representation formats can be utilized in dynamic 3D-driven industrial XR applications to offer remote training, remote expertise and quality control, and general remote communication. Finally, the paper proposes a pipeline and generic tool that can significantly simplify testing and experimentation in different use cases of industrial training.
Keywords:
Immersive and interactive multimedia systems, 3D virtual environments, Social XR, RGBD, Point Cloud, Mesh, Stereo 3D, Education
References
Frank Ansorge, Simon N.B. Meltzer, Elias; Gunkel, Fangtian Li, Rui; Deng, Wenfeng Zhu, Rulu Liao, Bao Trung Duong, and Amelie Hagelauer. 2023. "Plug and Learn: eXpeRtise at a Distance. In Proceedings of the 20th EuroXR International Conference (EuroXR 2023). 132–135. DOI: 10.32040/2242-122X.2022.T408
Ying Cao, Giap-Weng Ng, and Sha-Sha Ye. 2023. Design and Evaluation for Immersive Virtual Reality Learning Environment: A Systematic Literature Review. Sustainability 15, 3 (2023). DOI: 10.3390/su15031964
Helen Colman. 2020. 5 Remote Learning Challenges and How to Overcome Them. Technical Report. Training Industry. [link]
Sylvie Dijkstra-Soudarissanane, Simon N. B. Gunkel, and Véronne Reinders. 2022. Virtual visits: life-size immersive communication. In Proceedings of the 13th ACM Multimedia Systems Conference (Athlone, Ireland) (MMSys ’22). Association for Computing Machinery, New York, NY, USA, 310–314. DOI: 10.1145/3524273. 3532903
Sanika Doolani, Callen Wessels, Varun Kanal, Christos Sevastopoulos, Ashish Jaiswal, Harish Nambiappan, and Fillia Makedon. 2020. A Review of Extended Reality (XR) Technologies for Manufacturing Training. Technologies 8, 4 (2020). DOI: 10.3390/technologies8040077
US EPA. 2025. Training Through Immersive Technology. (2025). [link]
Simon NB Gunkel, Sylvie Dijkstra-Soudarissanane, and Omar Niamut. 2023. " You AR’right in front of me": RGBD-based capture and rendering for remote training. In Proceedings of the 14th Conference on ACM Multimedia Systems. 307–311.
Simon N.B. Gunkel, Sylvie Dijkstra-Soudarissanane, Hans M. Stokking, and Omar Niamut. 2023. From 2D to 3D video conferencing: Modular RGB-D capture and reconstruction for interactive natural user representations in immersive Extended Reality (XR) communication. Frontiers in Signal Processing 3 (2023). DOI: 10.3389/frsip.2023.1139897
Simon NB Gunkel, Rick Hindriks, Karim M El Assal, Hans M Stokking, Sylvie Dijkstra-Soudarissanane, Frank ter Haar, and Omar Niamut. 2021. VRComm: an end-to-end web system for real-time photorealistic social VR communication. In Proceedings of the 12th ACM Multimedia Systems Conference. 65–79.
Simon N.B. Gunkel, Tessa Klunder, Frank Ansorge, and Piotr Zuraniewski. 2025. The Stereo Microscope: Stereo 3D images for complex remote soldering teaching. In Proceedings of the 17th International Workshop on IMmersive Mixed and Virtual Environment Systems (MMVE ’25).
Simon N. B. Gunkel, Tessa Klunder, and Gianluca Cernigliaro. 2025. Ready Expert One: Universal 3D Workbench for Remote Industrial Training. (2025). DOI: 10.1145/3706370.373170
InnovateEnergy. 2025. Industrial Immersive Case Study Roundup: The Best of 2025 So Far. (2025). [link]
Md Tariqul Islam, Kamelia Sepanloo, SeonhoWoo, Seung HoWoo, and Young-Jun Son. 2025. A review of the industry 4.0 to 5.0 transition: exploring the intersection, challenges, and opportunities of technology and Human–Machine collaboration. Machines 13, 4 (March 2025), 267. DOI: 10.3390/machines13040267
Jianmin Liu, Yifeng Shen,Wenye Fan, and XiyaWu. 2024. Industrial technological advance and the employment demand for China’s labour force—Micro evidence from labour hiring in companies. PLoS ONE 19, 11 (Nov. 2024), e0307564. DOI: 10.1371/journal.pone.0307564
Pauliina Rikala, Greta Braun, Miitta Järvinen, Johan Stahre, and Raija Hämäläinen. 2024. Understanding and measuring skill gaps in Industry 4.0 — A review. Technological Forecasting and Social Change 201 (Jan. 2024), 123206. DOI: 10.1016/j.techfore.2024.123206
Simardeep Singh, Sylvie Dijkstra-Soudarissanane, and Simon Gunkel. 2022. Engagement and quality of experience in remote business meetings: A social vr study. In Proceedings of the 1st Workshop on Interactive eXtended Reality. 77–82.
Frank ter Haar, Sylvie Dijkstra-Soudarissanane, Piotr Zuraniewski, Rick Hindriks, Karim El Assal, Simon Gunkel, Galit Rahim, and Omar Niamut. 2023. Remote Expert Assistance System for Mixed-HMD Clients over 5G Infrastructure. In Proceedings of the 14th Conference on ACM Multimedia Systems. 318–322.
Konstantinos Koumaditis Unnikrishnan Radhakrishnan and Francesco Chinello. 2021. A systematic review of immersive virtual reality for industrial skills training. Behaviour & Information Technology 40, 12 (2021), 1310–1339. DOI: 10.1080/0144929X.2021.1954693
Ying Cao, Giap-Weng Ng, and Sha-Sha Ye. 2023. Design and Evaluation for Immersive Virtual Reality Learning Environment: A Systematic Literature Review. Sustainability 15, 3 (2023). DOI: 10.3390/su15031964
Helen Colman. 2020. 5 Remote Learning Challenges and How to Overcome Them. Technical Report. Training Industry. [link]
Sylvie Dijkstra-Soudarissanane, Simon N. B. Gunkel, and Véronne Reinders. 2022. Virtual visits: life-size immersive communication. In Proceedings of the 13th ACM Multimedia Systems Conference (Athlone, Ireland) (MMSys ’22). Association for Computing Machinery, New York, NY, USA, 310–314. DOI: 10.1145/3524273. 3532903
Sanika Doolani, Callen Wessels, Varun Kanal, Christos Sevastopoulos, Ashish Jaiswal, Harish Nambiappan, and Fillia Makedon. 2020. A Review of Extended Reality (XR) Technologies for Manufacturing Training. Technologies 8, 4 (2020). DOI: 10.3390/technologies8040077
US EPA. 2025. Training Through Immersive Technology. (2025). [link]
Simon NB Gunkel, Sylvie Dijkstra-Soudarissanane, and Omar Niamut. 2023. " You AR’right in front of me": RGBD-based capture and rendering for remote training. In Proceedings of the 14th Conference on ACM Multimedia Systems. 307–311.
Simon N.B. Gunkel, Sylvie Dijkstra-Soudarissanane, Hans M. Stokking, and Omar Niamut. 2023. From 2D to 3D video conferencing: Modular RGB-D capture and reconstruction for interactive natural user representations in immersive Extended Reality (XR) communication. Frontiers in Signal Processing 3 (2023). DOI: 10.3389/frsip.2023.1139897
Simon NB Gunkel, Rick Hindriks, Karim M El Assal, Hans M Stokking, Sylvie Dijkstra-Soudarissanane, Frank ter Haar, and Omar Niamut. 2021. VRComm: an end-to-end web system for real-time photorealistic social VR communication. In Proceedings of the 12th ACM Multimedia Systems Conference. 65–79.
Simon N.B. Gunkel, Tessa Klunder, Frank Ansorge, and Piotr Zuraniewski. 2025. The Stereo Microscope: Stereo 3D images for complex remote soldering teaching. In Proceedings of the 17th International Workshop on IMmersive Mixed and Virtual Environment Systems (MMVE ’25).
Simon N. B. Gunkel, Tessa Klunder, and Gianluca Cernigliaro. 2025. Ready Expert One: Universal 3D Workbench for Remote Industrial Training. (2025). DOI: 10.1145/3706370.373170
InnovateEnergy. 2025. Industrial Immersive Case Study Roundup: The Best of 2025 So Far. (2025). [link]
Md Tariqul Islam, Kamelia Sepanloo, SeonhoWoo, Seung HoWoo, and Young-Jun Son. 2025. A review of the industry 4.0 to 5.0 transition: exploring the intersection, challenges, and opportunities of technology and Human–Machine collaboration. Machines 13, 4 (March 2025), 267. DOI: 10.3390/machines13040267
Jianmin Liu, Yifeng Shen,Wenye Fan, and XiyaWu. 2024. Industrial technological advance and the employment demand for China’s labour force—Micro evidence from labour hiring in companies. PLoS ONE 19, 11 (Nov. 2024), e0307564. DOI: 10.1371/journal.pone.0307564
Pauliina Rikala, Greta Braun, Miitta Järvinen, Johan Stahre, and Raija Hämäläinen. 2024. Understanding and measuring skill gaps in Industry 4.0 — A review. Technological Forecasting and Social Change 201 (Jan. 2024), 123206. DOI: 10.1016/j.techfore.2024.123206
Simardeep Singh, Sylvie Dijkstra-Soudarissanane, and Simon Gunkel. 2022. Engagement and quality of experience in remote business meetings: A social vr study. In Proceedings of the 1st Workshop on Interactive eXtended Reality. 77–82.
Frank ter Haar, Sylvie Dijkstra-Soudarissanane, Piotr Zuraniewski, Rick Hindriks, Karim El Assal, Simon Gunkel, Galit Rahim, and Omar Niamut. 2023. Remote Expert Assistance System for Mixed-HMD Clients over 5G Infrastructure. In Proceedings of the 14th Conference on ACM Multimedia Systems. 318–322.
Konstantinos Koumaditis Unnikrishnan Radhakrishnan and Francesco Chinello. 2021. A systematic review of immersive virtual reality for industrial skills training. Behaviour & Information Technology 40, 12 (2021), 1310–1339. DOI: 10.1080/0144929X.2021.1954693
Published
2025-06-03
How to Cite
GUNKEL, Simon N. B.; KLUNDER, Tessa; CERNIGLIARO, Gianluca.
R3-D3: (Photo-)Realistic Real-time Representations of Dynamic 3D-Driven Industrial Trainings. In: ACM INTERNATIONAL CONFERENCE ON INTERACTIVE MEDIA EXPERIENCES WORKSHOPS (IMXW), 25. , 2025, Niterói/RJ.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2025
.
p. 140-144.
DOI: https://doi.org/10.5753/imxw.2025.6044.
