Helicopter visual signaling simulation: Integrating VR and ML into a low-cost solution to optimize Brazilian Navy training

• Antônio Doneda IME
• Jauvane Cavalcante de Oliveira LNCC

Resumo

The Landing Signalman (LS) is the military in charge of visually signaling the helicopter pilot when landing and taking off from a moving ship, ensuring general safety conditions of the flight deck area. It requires self-confidence, knowledge, skills, team coordination, and appropriate reaction capacity only achieved with intensive training. After studying the theory, Brazilian Navy LS trainees go directly to the practice stage at sea. This hands-on training phase straight after classroom reveals a series of limitations on LS trainee's performance. To better prepare the LS trainee, reduce training costs and provide a safe environment to perform unlimited training opportunities, we developed a lightweight, portable, and low-cost VR simulator for training optimization, denominated Helicopter Visual Signal Simulator (HVSS). Due to its specificities, a Machine Learning gesture recognition method was integrated into the virtual environment, and two datasets were created. This study details the process of achieving the design goals, developing the VR simulator prototype, and analyzing the tests we performed with 15 experienced instructors. The results indicate that HVSS meets all requirements to provide a reliable training solution.

Palavras-chave: Simulation, Virtual Reality, Machine Learning, Gesture Recognition