Q-Learning-Based Multi-Objective Global Path Planning for UGV Navigation - Batista, Hiago O. B.; Hudson, Thayron M.; Carvalho, Kevin B. de; Brandão, Alexandre S. (Anais do Simpósio Brasileiro de Robótica e Simpósio Latino Americano de Robótica (SBR/LARS); 2024: Anais do XXI Simpósio Latino Americano de Robótica, 2024)
This paper presents an offline path planning strategy for unmanned ground vehicles (UGVs) using Q-learning. The proposed method addresses path optimization in warehouse-like environments, where tasks involve item pickup and delivery to specific locations. The Q-learning algorithm trains an agent to determine the most efficient routes, with validation conducted in an 8×5 meter workspace equipped with an Optitrack motion capture system. The workspace was discretized into a 16×10 grid, allowing the Q-learning to effectively navigate through complex obstacle-laden scenarios. Experimental results indicate that the Q-learning approach outperforms traditional methods such as Dijkstra, A-star, and Breadth-First Search in terms of path length, number of turns, planning time, and overall success rate; being up to 7 times faster to plan a path and reducing the number of bends by up to 41%. The Q-learning based paths feature more linear segments, which contribute to energy savings and improved navigational efficiency. Future work will explore applications in heterogeneous multi-agent systems and enhancements in training time and agent collaboration.