Aprendizado por Reforço Profundo para Navegação sem Mapa de um Veículo Híbrido Aéreo-Aquático
Resumo
Realizar a navegação entre meios é uma tarefa desafiadora para robôs móveis híbridos, especialmente em cenários com obstáculos. Este trabalho apresenta uma abordagem baseada em aprendizado por reforço profundo (Deep-RL) para navegação autônoma de um tipo específico de robô móvel híbrido: um Veículo Híbrido Tipo Ar-Água (HUAUV). A abordagem proposta utiliza somente informação de sensores de distância e de informações relativas à localização do veículo para realizar a navegação. Resultados obtidos mostram que é possível realizar navegação sem mapa do início ao fim sem usar nenhum tipo de operação manual, somente utilizando os agentes baseados em Deep-RL. A navegação dos agentes treinados é comparada com a navegação sem mapa realizada por um algoritmo clássico - BUG2. A abordagem é baseada em dois métodos do estado da arte para navegação sem mapa de robôs terrestres: Política de Gradiente Determinístico Profundo (DDPG) e Soft Actor-Critic (SAC).
Palavras-chave:
Técnicas de IA robustas e sensoriamento, Veículos aéreos, Robôs aquáticos
Referências
Carlucho, I., De Paula, M., Wang, S., Menna, B. V., Petillot, Y. R., and Acosta, G. G. (2018). Auv position tracking control using end-to-end deep reinforcement learning. In OCEANS 2018 MTS/IEEE Charleston, pages 1–8. IEEE.
Cerqueira, R., Trocoli, T., Neves, G., Oliveira, L., Joyeux, S., Albiez, J., and Center, R. I. (2016). Custom shader and 3d rendering for computationally efficient sonar simulation. In 29th Conference on Graphics, Patterns and Images-SIBGRAPI.
Drews, P. L., Neto, A. A., and Campos, M. F. (2014). Hybrid unmanned aerial underwater vehicle: Modeling and simulation. In 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems, pages 4637–4642. IEEE.
Grando, R. B., de Jesus, J. C., and Drews-Jr, P. L. (2020). Deep reinforcement learning for mapless navigation of unmanned aerial vehicles. In 2020 Latin American Robotics Symposium (LARS), 2020 Brazilian Symposium on Robotics (SBR) and 2020 Workshop on Robotics in Education (WRE), pages 1–6. IEEE.
Grando, R. B., de Jesus, J. C., Kich, V. A., Kolling, A. H., Bortoluzzi, N. P., Pinheiro, P. M., Neto, A. A., and Drews Jr, P. L. J. (2021). Deep reinforcement learning for mapless navigation of a hybrid aerial underwater vehicle with medium transition. arXiv preprint arXiv:2103.12883.
Horn, A. C., Pinheiro, P. M., Silva, C. B., Neto, A. A., and Drews-Jr, P. L. (2019). A study on configuration of propellers for multirotor-like hybrid aerial-aquatic vehicles. In 2019 19th International Conference on Advanced Robotics (ICAR), pages 173–178. IEEE.
Li, Y. (2017). Deep reinforcement learning: An overview. arXiv preprint arXiv:1701.07274.
Ma, Z., Feng, J., and Yang, J. (2018). Research on vertical air–water trans-media control of hybrid unmanned aerial underwater vehicles based on adaptive sliding mode dynamical surface control. International Journal of Advanced Robotic Systems, 15(2):1729881418770531.
Maia, M. M., Mercado, D. A., and Diez, F. J. (2017). Design and implementation of multirotor aerial-underwater vehicles with experimental results. In 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 961–966. IEEE.
Maia, M. M., Soni, P., and Diez, F. J. (2015). Demonstration of an aerial and submersible vehicle capable of flight and underwater navigation with seamless air-water transition. arXiv preprint arXiv:1507.01932.
Mercado, D., Maia, M., and Diez, F. J. (2019). Aerial-underwater systems, a new paradigm in unmanned vehicles. Journal of Intelligent & Robotic Systems, 95(1):229–238.
Ravell, D. A. M., Maia, M. M., and Diez, F. J. (2018). Modeling and control of unmanned aerial/underwater vehicles using hybrid control. Control Engineering Practice, 76:112–122.
Rodriguez-Ramos, A., Sampedro, C., Bavle, H., Moreno, I. G., and Campoy, P. (2018). A deep reinforcement learning technique for vision-based autonomous multirotor landing on a moving platform. In 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 1010–1017. IEEE.
Srinivas, A., Laskin, M., and Abbeel, P. (2020). Curl: Contrastive unsupervised representations for reinforcement learning. arXiv preprint arXiv:2004.04136.
Tai, L., Paolo, G., and Liu, M. (2017). Virtual-to-real deep reinforcement learning: Continuous control of mobile robots for mapless navigation. In 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 31–36. IEEE
Cerqueira, R., Trocoli, T., Neves, G., Oliveira, L., Joyeux, S., Albiez, J., and Center, R. I. (2016). Custom shader and 3d rendering for computationally efficient sonar simulation. In 29th Conference on Graphics, Patterns and Images-SIBGRAPI.
Drews, P. L., Neto, A. A., and Campos, M. F. (2014). Hybrid unmanned aerial underwater vehicle: Modeling and simulation. In 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems, pages 4637–4642. IEEE.
Grando, R. B., de Jesus, J. C., and Drews-Jr, P. L. (2020). Deep reinforcement learning for mapless navigation of unmanned aerial vehicles. In 2020 Latin American Robotics Symposium (LARS), 2020 Brazilian Symposium on Robotics (SBR) and 2020 Workshop on Robotics in Education (WRE), pages 1–6. IEEE.
Grando, R. B., de Jesus, J. C., Kich, V. A., Kolling, A. H., Bortoluzzi, N. P., Pinheiro, P. M., Neto, A. A., and Drews Jr, P. L. J. (2021). Deep reinforcement learning for mapless navigation of a hybrid aerial underwater vehicle with medium transition. arXiv preprint arXiv:2103.12883.
Horn, A. C., Pinheiro, P. M., Silva, C. B., Neto, A. A., and Drews-Jr, P. L. (2019). A study on configuration of propellers for multirotor-like hybrid aerial-aquatic vehicles. In 2019 19th International Conference on Advanced Robotics (ICAR), pages 173–178. IEEE.
Li, Y. (2017). Deep reinforcement learning: An overview. arXiv preprint arXiv:1701.07274.
Ma, Z., Feng, J., and Yang, J. (2018). Research on vertical air–water trans-media control of hybrid unmanned aerial underwater vehicles based on adaptive sliding mode dynamical surface control. International Journal of Advanced Robotic Systems, 15(2):1729881418770531.
Maia, M. M., Mercado, D. A., and Diez, F. J. (2017). Design and implementation of multirotor aerial-underwater vehicles with experimental results. In 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 961–966. IEEE.
Maia, M. M., Soni, P., and Diez, F. J. (2015). Demonstration of an aerial and submersible vehicle capable of flight and underwater navigation with seamless air-water transition. arXiv preprint arXiv:1507.01932.
Mercado, D., Maia, M., and Diez, F. J. (2019). Aerial-underwater systems, a new paradigm in unmanned vehicles. Journal of Intelligent & Robotic Systems, 95(1):229–238.
Ravell, D. A. M., Maia, M. M., and Diez, F. J. (2018). Modeling and control of unmanned aerial/underwater vehicles using hybrid control. Control Engineering Practice, 76:112–122.
Rodriguez-Ramos, A., Sampedro, C., Bavle, H., Moreno, I. G., and Campoy, P. (2018). A deep reinforcement learning technique for vision-based autonomous multirotor landing on a moving platform. In 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 1010–1017. IEEE.
Srinivas, A., Laskin, M., and Abbeel, P. (2020). Curl: Contrastive unsupervised representations for reinforcement learning. arXiv preprint arXiv:2004.04136.
Tai, L., Paolo, G., and Liu, M. (2017). Virtual-to-real deep reinforcement learning: Continuous control of mobile robots for mapless navigation. In 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 31–36. IEEE
Publicado
14/10/2021
Como Citar
GRANDO, Ricardo B.; DREWS-JR, Paulo L. J..
Aprendizado por Reforço Profundo para Navegação sem Mapa de um Veículo Híbrido Aéreo-Aquático. In: CONCURSO DE TESES E DISSERTAÇÕES EM ROBÓTICA - CTDR (MESTRADO) - SIMPÓSIO BRASILEIRO DE ROBÓTICA E SIMPÓSIO LATINO-AMERICANO DE ROBÓTICA (SBR/LARS), 9. , 2021, Online.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2021
.
p. 22-33.
DOI: https://doi.org/10.5753/wtdr_ctdr.2021.18682.
