Mixed Reality for Remote Supervision and Risk Assessment of Unstable Terrains

• Lidia Rocha UFSCar
• Marcos M. Futai USP
• Kelen C. T. Vivaldini UFSCar

DOI: https://doi.org/10.5753/sbgames_estendido.2025.8820

Resumo

Introduction: Operating drones in unstable terrains poses safety and logistical challenges. This Serious Game leverages Mixed Reality (MR) to simulate hazardous environments using Virtual Reality (VR) for training and Augmented Reality (AR) for collaborative analysis. Objective: This work develops an MR-based training simulation where players explore unstable environments, evaluate pre-planned drone trajectories, and predict terrain changes to improve flight strategy and risk assessment. Methodology: Players use VR to safely explore hazardous terrains, analyze drone data, and rehearse flight paths before real missions. The system gradually trains operators to transition to real drone use. A future AI prediction module is intended to enhance terrain analysis by anticipating environmental changes. AR supports strategic collaboration by enabling shared visualization and discussion of unstable environments. Expected Results: The game aims to enhance spatial awareness, improve flight planning, and support safer decision-making. It offers a risk-free environment to evaluate terrain evolution and drone strategies without field exposure.

Palavras-chave: Mixed Reality, Training, Prediction, Visualization, Drone

Referências

AirSkill (2024). Airskill’s vr - virtual reality flight training system. A virtual reality flight training platform designed to improve pilot skills through immersive and interactive simulations.

Butler, A., Hilliges, O., Izadi, S., Hodges, S., Molyneaux, D., Kim, D., e Kong, D. (2011). Vermeer: direct interaction with a 360 viewable 3d display. In Proceedings of the 24th annual ACM symposium on User interface software and technology, pages 569–576.

Khan, A., Gupta, S., e Gupta, S. K. (2022). Emerging uav technology for disaster detection, mitigation, response, and preparedness. Journal of Field Robotics, 39.

Marron MS, T., Captain, N., Mac Namee PhD, B., O’Hagan PhD, A. D., et al. (2024). Virtual reality & pilot training: Existing technologies, challenges & opportunities. Journal of Aviation/Aerospace Education & Research, 33(1):1.

Mutanga, O. e Kumar, L. (2019). Google earth engine applications.

Nalamothu, R., Sontha, P., Karravula, J., e Agrawal, A. (2024). Leveraging augmented reality for improved situational awareness during uav-driven search and rescue missions. In 2024 IEEE International Symposium on Safety Security Rescue Robotics (SSRR), pages 221–228. IEEE.

Perera, H., Wickramasinghe, P., Wijayawardana, P., Chathuranga, U., Akalanka, H., e Jayathilake, D. (2024). Application of vr technology and 3d reality models in enhancing community drr for landslides. 14th Annual Research Symposium.

Sivanesan, V., Ng, Z. L., Lim, T. X., Tan, H. K., Yew, K. S., e Goh, W. W. (2021). The use of augmented reality in collaboration within the construction industry. In Journal of Physics: Conference Series, volume 2120, page 012032. IOP Publishing.

Sousa, P. D., Silva, D. C., e Reis, L. P. (2010). Air traffic control with microsoft flight simulator x. In 5th Iberian Conf. on Inf. Systems and Technologies. IEEE.

Tegel, R. e Mengotti, T. (2005). Gpu, framework extensions for the distributed search engine and the terragen landscape generator.

Thomas Ph D, R. L., Albelo Ph D, J. L., Wiggins Ed D, M., et al. (2023). Enhancing pilot training through virtual reality: Recognizing and mitigating aviation visual and vestibular illusions. International Journal of Aviation, Aeronautics, and Aerospace, 10(3):10.

Zintl, M., Kimura, S., e Holzapfel, F. (2024). A mixed reality research flight simulator for advanced air mobility vehicles. In AIAA AVIATION FORUM AND ASCEND 2024, page 4651.