Mapless Navigation of a Hybrid Aerial Underwater Vehicle with Deep Reinforcement Learning Through Environmental Generalization

• Ricardo B. Grando Technological University of Uruguay
• Junior C. de Jesus FURG
• Victor A. Kich UFSM
• Alisson H. Kolling UFSM
• Pedro M. Pinheiro FURG
• Rodrigo S. Guerra FURG
• Paulo L. J. Drews FURG

Resumo

Previous works showed that Deep-RL can be applied to perform mapless navigation, including the medium transition of Hybrid Unmanned Aerial Underwater Vehicles (HUAUVs). This paper presents new approaches based on the state-of-the-art actor-critic algorithms to address the navigation and medium transition problems for a HUAUV. We show that a double critic Deep-RL with Recurrent Neural Networks improves the navigation performance of HUAUVs using solely range data and relative localization. Our Deep-RL approaches achieved better navigation and transitioning capabilities with a solid generalization of learning through distinct simulated scenarios, outperforming previous approaches.

Palavras-chave: Location awareness, Recurrent neural networks, Navigation, Education, Reinforcement learning, Robot sensing systems, Solids