Hybrid Reinforcement Learning Architecture with Geometric Curriculum Learning for Non-Holonomic Leader-Follower Navigation

• Natasha Araújo Caxias UFAM
• Abel Severo Rocha UFAM
• José Reginaldo Hughes Carvalho UFAM

DOI: https://doi.org/10.5753/semish.2026.21882

Resumo

This paper proposes a hybrid architecture for leader-follower non-holonomic navigation, combining a Dueling Deep Q-Network (DDQN), Prioritized Experience Replay (PER), Frame Stacking, and Golden Ratio action discretization. A three-phase Geometric Curriculum Learning strategy addresses sparse rewards and catastrophic forgetting, progressing from fixed geometries to full Domain Randomization. Ablation studies confirm that Frame Stacking and Golden Ratio discretization are strictly necessary, their removal collapses success to 0%, while PER acts as a convergence accelerator rather than a behavior enabler. The full system achieves 100% success on fixed configurations and 74% on procedural generalization.

Referências

Bengio, Y., Louradour, J., Collobert, R., and Weston, J. Curriculum learning. In Proceedings of the 26th International Conference on Machine Learning (ICML).

Hasselt, H. V., Guez, A., and Silver, D. Deep reinforcement learning with double q-learning. In Proceedings of the 13th AAAI Conference on Artificial Intelligence.

Khatib, O. (1986). Real-time obstacle avoidance for manipulators and mobile robots. International Journal of Robotics Research, 5(1):90–98.

Mnih, V., Kavukcuoglu, K., Silver, D., et al. (2015). Human-level control through deep reinforcement learning. Nature, 518(7540):529–533.

Narvekar, S., Peng, B., Leonetti, M., Sinapov, J., Taylor, M. E., and Stone, P. (2020). Curriculum learning for reinforcement learning domains: A framework and survey. Journal of Machine Learning Research, 21(181):1–50.

Oh, K.-K., Park, M.-C., and Ahn, H.-S. (2015). A survey of multi-agent formation control. Automatica, 53:424–440.

Schaul, T., Quan, J., Antonoglou, I., and Silver, D. (2016). Prioritized experience replay. arXiv preprint arXiv:1511.05952.

Siegwart, R., Nourbakhsh, I. R., and Scaramuzza, D. (2011). Introduction to Autonomous Mobile Robots. MIT Press, Cambridge, MA, 2 edition.

Tobin, J., Fong, R., Ray, A., Schneider, J., Zaremba, W., and Abbeel, P. (2017). Domain randomization for transferring deep neural networks from simulation to the real world. In Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 23–30.

Wang, X. et al. (2022). Leader-follower formation control of multiple nonholonomic mobile robots with deep reinforcement learning. IEEE Transactions on Industrial Informatics, 18(10):6683–6692.

Wang, Z., Schaul, T., Hessel, M., van Hasselt, H., Lanctot, M., and de Freitas, N. (2016). Dueling network architectures for deep reinforcement learning. In Proceedings of the International Conference on Machine Learning (ICML), pages 1995–2003.

Zhang, K., Yang, Z., and Basar, T. (2024). Multi-agent reinforcement learning: A selective overview of theories and algorithms. Handbook of Reinforcement Learning and Control, pages 321–384.