Agentes BDI e Aprendizagem: um mapeamento sistemático e utilização com a biblioteca MASPY
Resumo
Os algoritmos de Aprendizagem por Reforço são capazes de resolver processos de decisão sequenciais por meio de interações repetidas com um ambiente. Essa abordagem permite a solução de desafios complexos e possibilita inovações tecnológicas, como os Veículos Autônomos (VAs). Com isso em mente, este artigo apresenta o planejamento, execução e conclusões de um mapeamento sistemático da literatura sobre algoritmos de aprendizagem para VAs. Uma lacuna identificada é a integração de arquitetura de Agentes Inteligentes BDI com Aprendizagem por Reforço. Para abordar isso, é apresentado um exemplo usando a biblioteca MASPY em Python, em que é programado um agente BDI que utiliza componentes de aprendizagem.Referências
Arvind, C. S. and Senthilnath, J. (2019). Autonomous RL: Autonomous Vehicle Obstacle Avoidance in a Dynamic Environment using MLP-SARSA Reinforcement Learning. In 2019 IEEE 5th International Conference on Mechatronics System and Robots (ICMSR), pages 120–124, Singapore. IEEE.
Bellman, R. (1957). Dynamic Programming. Princeton University Press, Princeton, NJ, USA, 1 edition.
Bin Issa, R., Das, M., Rahman, M. S., Barua, M., Rhaman, M. K., Ripon, K. S. N., and Alam, M. G. R. (2021). Double Deep Q-Learning and Faster R-CNN-Based Autonomous Vehicle Navigation and Obstacle Avoidance in Dynamic Environment. Sensors, 21(4):1468. Number: 4 Publisher: Multidisciplinary Digital Publishing Institute.
Bosello, M. (2020). Integrating bdi and reinforcement learning: the case study of autonomous driving.
Bratman, M. (1987). Intention, Plans, and Practical Reason. Cambridge: Cambridge, MA: Harvard University Press.
Cabezas-Olivenza, M., Zulueta, E., Sanchez-Chica, A., Fernandez-Gamiz, U., and Teso-Fz-Betoño, A. (2023). Stability Analysis for Autonomous Vehicle Navigation Trained over Deep Deterministic Policy Gradient. Mathematics, 11(1):132. Number: 1 Publisher: Multidisciplinary Digital Publishing Institute.
Dar, S. A., Palanivel, S., and Geetha, M. K. Autonomous Taxi Driving Environment Using Reinforcement Learning Algorithms. International Journal of Modern Education and Computer Science, 14(3):88.
Dhinakaran, M., Rajasekaran, R., Balaji, V., Aarthi, V., and Ambika, S. (2024). Advanced Deep Reinforcement Learning Strategies for Enhanced Autonomous Vehicle Navigation Systems. In 2024 2nd International Conference on Computer, Communication and Control (IC4), pages 1–4.
Freitas, V. and Segatto, W. (2021). Perform Systematic Literature Reviews.
Gao, L., Wu, Y., Wang, L., Wang, L., Zhang, J., and Li, K. (2023). End-to-end autonomous vehicle navigation control method guided by the dynamic window approach. In 2023 IEEE 6th International Electrical and Energy Conference (CIEEC), pages 4472–4476.
González-Miranda, O., Miranda, L. A. L., and Ibarra-Zannatha, J. M. (2023). Q-Learning for Autonomous Vehicle Navigation. In 2023 XXV Robotics Mexican Congress (COM-Rob), pages 138–142.
Gronauer, S. and Diepold, K. (2022). Multi-agent deep reinforcement learning: a survey. Artificial Intelligence Review, pages 1–49.
Hartmann, G., Shiller, Z., and Azaria, A. (2020). Model-Based Reinforcement Learning for Time-Optimal Velocity Control. IEEE Robotics and Automation Letters, 5(4):6185–6192. Conference Name: IEEE Robotics and Automation Letters.
Hook, J., El-Sedky, S., De Silva, V., and Kondoz, A. (2021). Learning data-driven decision-making policies in multi-agent environments for autonomous systems. Cognitive Systems Research, 65:40–49.
Jacinto, E., Martinez, F., and Martinez, F. (2023). Navigation of Autonomous Vehicles using Reinforcement Learning with Generalized Advantage Estimation. International Journal of Advanced Computer Science and Applications (IJACSA), 14(1). Number: 1 Publisher: The Science and Information (SAI) Organization Limited.
Josef, S. and Degani, A. (2020). Deep Reinforcement Learning for Safe Local Planning of a Ground Vehicle in Unknown Rough Terrain. IEEE Robotics and Automation Letters, 5(4):6748–6755. Conference Name: IEEE Robotics and Automation Letters.
Kitchenham, B. A. and Charters, S. (2007). Guidelines for performing systematic literature reviews in software engineering. Technical Report EBSE 2007-001, Keele University and Durham University Joint Report.
Lambert, R., Li, J., Wu, L.-F., and Mahmoudian, N. (2021). Robust ASV Navigation Through Ground to Water Cross-Domain Deep Reinforcement Learning. Frontiers in Robotics and AI, 8. Publisher: Frontiers.
Liang, J., Weerakoon, K., Guan, T., Karapetyan, N., and Manocha, D. (2023). AdaptiveON: Adaptive Outdoor Local Navigation Method for Stable and Reliable Actions. IEEE Robotics and Automation Letters, 8(2):648–655.
Mackay, A. K., Riazuelo, L., and Montano, L. (2022). RL-DOVS: Reinforcement Learning for Autonomous Robot Navigation in Dynamic Environments. Sensors, 22(10):3847. Number: 10 Publisher: Multidisciplinary Digital Publishing Institute.
Mellado, A. L. L., G., F. I., Alves, G. V., and Borges, A. P. (2023). Maspy: Towards the creation of bdi multi-agent systems. In Proceedings of the 17th Workshop-School on Agents, Environments, and Applications (WESAAC 2023), pages 106–117.
Qin, J., Qin, J., Qiu, J., Liu, Q., Li, M., and Ma, Q. (2024). SRL-ORCA: A Socially Aware Multi-Agent Mapless Navigation Algorithm in Complex Dynamic Scenes. IEEE Robotics and Automation Letters, 9(1):143–150. Conference Name: IEEE Robotics and Automation Letters.
Shakya, A. K., Pillai, G., and Chakrabarty, S. (2023). Reinforcement learning algorithms: A brief survey. Expert Systems with Applications, page 120495.
Sun, Q., Zhang, L., Yu, H., Zhang, W., Mei, Y., and Xiong, H. (2023). Hierarchical Reinforcement Learning for Dynamic Autonomous Vehicle Navigation at Intelligent Intersections. In Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, KDD ’23, pages 4852–4861, New York, NY, USA. Association for Computing Machinery.
Sutton, R. S. and Barto, A. G. (2018). Reinforcement Learning: An Introduction. The MIT Press, second edition.
Taghavifar, H., Wei, C., and Taghavifar, L. (2024). Socially Intelligent Reinforcement Learning for Optimal Automated Vehicle Control in Traffic Scenarios. IEEE Transactions on Automation Science and Engineering, pages 1–12. Conference Name: IEEE Transactions on Automation Science and Engineering.
Tiong, T., Saad, I., Teo, K. T. K., and Lago, H. B. (2023). Autonomous Vehicle Driving Path Control with Deep Reinforcement Learning. In 2023 IEEE 13th Annual Computing and Communication Workshop and Conference (CCWC), pages 0084–0092, Las Vegas, NV, USA. IEEE.
Yang, H., Yao, C., Liu, C., and Chen, Q. (2023). RMRL: Robot Navigation in Crowd Environments With Risk Map-Based Deep Reinforcement Learning. IEEE Robotics and Automation Letters, 8(12):7930–7937. Conference Name: IEEE Robotics and Automation Letters.
Zhai, Y., Ding, B., Liu, X., Jia, H., Zhao, Y., and Luo, J. (2021). Decentralized Multi-Robot Collision Avoidance in Complex Scenarios With Selective Communication. IEEE Robotics and Automation Letters, 6(4):8379–8386. Conference Name: IEEE Robotics and Automation Letters.
Zhang, W., Gai, J., Zhang, Z., Tang, L., Liao, Q., and Ding, Y. (2019). Double-DQN based path smoothing and tracking control method for robotic vehicle navigation. Computers and Electronics in Agriculture, 166:104985.
Zhu, W. and Hayashibe, M. (2023). Autonomous Navigation System in Pedestrian Scenarios Using a Dreamer-Based Motion Planner. IEEE Robotics and Automation Letters, 8(6):3836–3843.
Bellman, R. (1957). Dynamic Programming. Princeton University Press, Princeton, NJ, USA, 1 edition.
Bin Issa, R., Das, M., Rahman, M. S., Barua, M., Rhaman, M. K., Ripon, K. S. N., and Alam, M. G. R. (2021). Double Deep Q-Learning and Faster R-CNN-Based Autonomous Vehicle Navigation and Obstacle Avoidance in Dynamic Environment. Sensors, 21(4):1468. Number: 4 Publisher: Multidisciplinary Digital Publishing Institute.
Bosello, M. (2020). Integrating bdi and reinforcement learning: the case study of autonomous driving.
Bratman, M. (1987). Intention, Plans, and Practical Reason. Cambridge: Cambridge, MA: Harvard University Press.
Cabezas-Olivenza, M., Zulueta, E., Sanchez-Chica, A., Fernandez-Gamiz, U., and Teso-Fz-Betoño, A. (2023). Stability Analysis for Autonomous Vehicle Navigation Trained over Deep Deterministic Policy Gradient. Mathematics, 11(1):132. Number: 1 Publisher: Multidisciplinary Digital Publishing Institute.
Dar, S. A., Palanivel, S., and Geetha, M. K. Autonomous Taxi Driving Environment Using Reinforcement Learning Algorithms. International Journal of Modern Education and Computer Science, 14(3):88.
Dhinakaran, M., Rajasekaran, R., Balaji, V., Aarthi, V., and Ambika, S. (2024). Advanced Deep Reinforcement Learning Strategies for Enhanced Autonomous Vehicle Navigation Systems. In 2024 2nd International Conference on Computer, Communication and Control (IC4), pages 1–4.
Freitas, V. and Segatto, W. (2021). Perform Systematic Literature Reviews.
Gao, L., Wu, Y., Wang, L., Wang, L., Zhang, J., and Li, K. (2023). End-to-end autonomous vehicle navigation control method guided by the dynamic window approach. In 2023 IEEE 6th International Electrical and Energy Conference (CIEEC), pages 4472–4476.
González-Miranda, O., Miranda, L. A. L., and Ibarra-Zannatha, J. M. (2023). Q-Learning for Autonomous Vehicle Navigation. In 2023 XXV Robotics Mexican Congress (COM-Rob), pages 138–142.
Gronauer, S. and Diepold, K. (2022). Multi-agent deep reinforcement learning: a survey. Artificial Intelligence Review, pages 1–49.
Hartmann, G., Shiller, Z., and Azaria, A. (2020). Model-Based Reinforcement Learning for Time-Optimal Velocity Control. IEEE Robotics and Automation Letters, 5(4):6185–6192. Conference Name: IEEE Robotics and Automation Letters.
Hook, J., El-Sedky, S., De Silva, V., and Kondoz, A. (2021). Learning data-driven decision-making policies in multi-agent environments for autonomous systems. Cognitive Systems Research, 65:40–49.
Jacinto, E., Martinez, F., and Martinez, F. (2023). Navigation of Autonomous Vehicles using Reinforcement Learning with Generalized Advantage Estimation. International Journal of Advanced Computer Science and Applications (IJACSA), 14(1). Number: 1 Publisher: The Science and Information (SAI) Organization Limited.
Josef, S. and Degani, A. (2020). Deep Reinforcement Learning for Safe Local Planning of a Ground Vehicle in Unknown Rough Terrain. IEEE Robotics and Automation Letters, 5(4):6748–6755. Conference Name: IEEE Robotics and Automation Letters.
Kitchenham, B. A. and Charters, S. (2007). Guidelines for performing systematic literature reviews in software engineering. Technical Report EBSE 2007-001, Keele University and Durham University Joint Report.
Lambert, R., Li, J., Wu, L.-F., and Mahmoudian, N. (2021). Robust ASV Navigation Through Ground to Water Cross-Domain Deep Reinforcement Learning. Frontiers in Robotics and AI, 8. Publisher: Frontiers.
Liang, J., Weerakoon, K., Guan, T., Karapetyan, N., and Manocha, D. (2023). AdaptiveON: Adaptive Outdoor Local Navigation Method for Stable and Reliable Actions. IEEE Robotics and Automation Letters, 8(2):648–655.
Mackay, A. K., Riazuelo, L., and Montano, L. (2022). RL-DOVS: Reinforcement Learning for Autonomous Robot Navigation in Dynamic Environments. Sensors, 22(10):3847. Number: 10 Publisher: Multidisciplinary Digital Publishing Institute.
Mellado, A. L. L., G., F. I., Alves, G. V., and Borges, A. P. (2023). Maspy: Towards the creation of bdi multi-agent systems. In Proceedings of the 17th Workshop-School on Agents, Environments, and Applications (WESAAC 2023), pages 106–117.
Qin, J., Qin, J., Qiu, J., Liu, Q., Li, M., and Ma, Q. (2024). SRL-ORCA: A Socially Aware Multi-Agent Mapless Navigation Algorithm in Complex Dynamic Scenes. IEEE Robotics and Automation Letters, 9(1):143–150. Conference Name: IEEE Robotics and Automation Letters.
Shakya, A. K., Pillai, G., and Chakrabarty, S. (2023). Reinforcement learning algorithms: A brief survey. Expert Systems with Applications, page 120495.
Sun, Q., Zhang, L., Yu, H., Zhang, W., Mei, Y., and Xiong, H. (2023). Hierarchical Reinforcement Learning for Dynamic Autonomous Vehicle Navigation at Intelligent Intersections. In Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, KDD ’23, pages 4852–4861, New York, NY, USA. Association for Computing Machinery.
Sutton, R. S. and Barto, A. G. (2018). Reinforcement Learning: An Introduction. The MIT Press, second edition.
Taghavifar, H., Wei, C., and Taghavifar, L. (2024). Socially Intelligent Reinforcement Learning for Optimal Automated Vehicle Control in Traffic Scenarios. IEEE Transactions on Automation Science and Engineering, pages 1–12. Conference Name: IEEE Transactions on Automation Science and Engineering.
Tiong, T., Saad, I., Teo, K. T. K., and Lago, H. B. (2023). Autonomous Vehicle Driving Path Control with Deep Reinforcement Learning. In 2023 IEEE 13th Annual Computing and Communication Workshop and Conference (CCWC), pages 0084–0092, Las Vegas, NV, USA. IEEE.
Yang, H., Yao, C., Liu, C., and Chen, Q. (2023). RMRL: Robot Navigation in Crowd Environments With Risk Map-Based Deep Reinforcement Learning. IEEE Robotics and Automation Letters, 8(12):7930–7937. Conference Name: IEEE Robotics and Automation Letters.
Zhai, Y., Ding, B., Liu, X., Jia, H., Zhao, Y., and Luo, J. (2021). Decentralized Multi-Robot Collision Avoidance in Complex Scenarios With Selective Communication. IEEE Robotics and Automation Letters, 6(4):8379–8386. Conference Name: IEEE Robotics and Automation Letters.
Zhang, W., Gai, J., Zhang, Z., Tang, L., Liao, Q., and Ding, Y. (2019). Double-DQN based path smoothing and tracking control method for robotic vehicle navigation. Computers and Electronics in Agriculture, 166:104985.
Zhu, W. and Hayashibe, M. (2023). Autonomous Navigation System in Pedestrian Scenarios Using a Dreamer-Based Motion Planner. IEEE Robotics and Automation Letters, 8(6):3836–3843.
Publicado
14/08/2024
Como Citar
IZIDORIO, Felipe Merenda; MELLADO, Alexandre L. L.; BORGES, André Pinz; ALVES, Gleifer Vaz.
Agentes BDI e Aprendizagem: um mapeamento sistemático e utilização com a biblioteca MASPY. In: WORKSHOP-ESCOLA DE SISTEMAS DE AGENTES, SEUS AMBIENTES E APLICAÇÕES (WESAAC), 18. , 2024, Brasília/DF.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2024
.
p. 108-119.
ISSN 2326-5434.
