Accelerating Learning of Route Choices With C2I: A Preliminary Investigation
Resumo
How to choose a route that takes you from A to B? This is an issue that is turning more and more important in modern societies. One way to address this agenda is through the use of communication between the infrastructure (network), and the demand (vehicles). In this paper, we use car-to-infrastructure (C2I) communication to investigate whether the road users (agents) can accelerate their learning process regarding route choice problem, via reinforcement learning (RL). We employ a microscopic simulator in order to compare our method with two others: RL without communication and an iterative method. Experimental results show that our method outperforms both methods in terms of effectiveness and efficiency.
Palavras-chave:
Multi-Agent Reinforcement Learning, Route Choice, Car-to-Infrastructure Communication
Referências
Auld, J., Verbas, O., and Stinson, M. Agent-based dynamic traffic assignment with information mixing. Procedia Computer Science vol. 151, pp. 864 – 869, 2019.
Bazzan, A. L. C., Fehler, M., and Klügl, F. Learning to coordinate in a network of social drivers: The role of information. In Proceedings of the International Workshop on Learning and Adaptation in MAS (LAMAS 2005), K. Tuyls, P. J. Hoen, K. Verbeeck, and S. Sen (Eds.). Number 3898 in Lecture Notes in Artificial Intelligence. Utrecht, pp. 115–128, 2006.
Bazzan, A. L. C. and Grunitzki, R. A multiagent reinforcement learning approach to en-route trip building. In 2016 International Joint Conference on Neural Networks (IJCNN). pp. 5288–5295, 2016.
Grunitzki, R. and Bazzan, A. L. C. Combining car-to-infrastructure communication and multi-agent reinforcement learning in route choice. In Proceedings of the Ninth Workshop on Agents in Traffic and Transportation (ATT-2016), A. L. C. Bazzan, F. Klügl, S. Ossowski, and G. Vizzari (Eds.). CEUR-WS.org, New York, 2016.
Grunitzki, R. and Bazzan, A. L. C. Comparing two multiagent reinforcement learning approaches for the traffic assignment problem. In Intelligent Systems (BRACIS), 2017 Brazilian Conference on, 2017.
Koster, A., Tettamanzi, A., Bazzan, A. L. C., and Pereira, C. d. C. Using trust and possibilistic reasoning to deal with untrustworthy communication in VANETs. In Proceedings of the 16th IEEE Annual Conference on Intelligent Transport Systems (IEEE-ITSC). IEEE, The Hague, The Netherlands, pp. 2355–2360, 2013.
Lopez, P. A., Behrisch, M., Bieker-Walz, L., Erdmann, J., Flötteröd, Y.-P., Hilbrich, R., Lücken, L., Rummel, J., Wagner, P., and Wießner, E. Microscopic traffic simulation using sumo. In The 21st IEEE International Conference on Intelligent Transportation Systems, 2018.
Ortúzar, J. d. D. and Willumsen, L. G. Modelling transport. John Wiley & Sons, Chichester, UK, 2011.
Ramos, G. de. O. and Grunitzki, R. An improved learning automata approach for the route choice problem. In Agent Technology for Intelligent Mobile Services and Smart Societies, F. Koch, F. Meneguzzi, and K. Lakkaraju (Eds.). Communications in Computer and Information Science, vol. 498. Springer Berlin Heidelberg, pp. 56–67, 2015.
Tumer, K., Welch, Z. T., and Agogino, A. Aligning social welfare and agent preferences to alleviate traffic congestion. In Proceedings of the 7th Int. Conference on Autonomous Agents and Multiagent Systems, L. Padgham, D. Parkes, J. Müller, and S. Parsons (Eds.). IFAAMAS, Estoril, pp. 655–662, 2008.
Wardrop, J. G. Some theoretical aspects of road traffic research. Proceedings of the Institution of Civil Engineers, Part II 1 (36): 325–362, 1952.
Yu, Y., Han, K., and Ochieng, W. Day-to-day dynamic traffic assignment with imperfect information, bounded rationality and information sharing. Transportation Research Part C: Emerging Technologies vol. 114, pp. 59 – 83, 2020.
Zhou, B., Song, Q., Zhao, Z., and Liu, T. A reinforcement learning scheme for the equilibrium of the in-vehicle route choice problem based on congestion game. Applied Mathematics and Computation vol. 371, pp. 124895, 2020.
Bazzan, A. L. C., Fehler, M., and Klügl, F. Learning to coordinate in a network of social drivers: The role of information. In Proceedings of the International Workshop on Learning and Adaptation in MAS (LAMAS 2005), K. Tuyls, P. J. Hoen, K. Verbeeck, and S. Sen (Eds.). Number 3898 in Lecture Notes in Artificial Intelligence. Utrecht, pp. 115–128, 2006.
Bazzan, A. L. C. and Grunitzki, R. A multiagent reinforcement learning approach to en-route trip building. In 2016 International Joint Conference on Neural Networks (IJCNN). pp. 5288–5295, 2016.
Grunitzki, R. and Bazzan, A. L. C. Combining car-to-infrastructure communication and multi-agent reinforcement learning in route choice. In Proceedings of the Ninth Workshop on Agents in Traffic and Transportation (ATT-2016), A. L. C. Bazzan, F. Klügl, S. Ossowski, and G. Vizzari (Eds.). CEUR-WS.org, New York, 2016.
Grunitzki, R. and Bazzan, A. L. C. Comparing two multiagent reinforcement learning approaches for the traffic assignment problem. In Intelligent Systems (BRACIS), 2017 Brazilian Conference on, 2017.
Koster, A., Tettamanzi, A., Bazzan, A. L. C., and Pereira, C. d. C. Using trust and possibilistic reasoning to deal with untrustworthy communication in VANETs. In Proceedings of the 16th IEEE Annual Conference on Intelligent Transport Systems (IEEE-ITSC). IEEE, The Hague, The Netherlands, pp. 2355–2360, 2013.
Lopez, P. A., Behrisch, M., Bieker-Walz, L., Erdmann, J., Flötteröd, Y.-P., Hilbrich, R., Lücken, L., Rummel, J., Wagner, P., and Wießner, E. Microscopic traffic simulation using sumo. In The 21st IEEE International Conference on Intelligent Transportation Systems, 2018.
Ortúzar, J. d. D. and Willumsen, L. G. Modelling transport. John Wiley & Sons, Chichester, UK, 2011.
Ramos, G. de. O. and Grunitzki, R. An improved learning automata approach for the route choice problem. In Agent Technology for Intelligent Mobile Services and Smart Societies, F. Koch, F. Meneguzzi, and K. Lakkaraju (Eds.). Communications in Computer and Information Science, vol. 498. Springer Berlin Heidelberg, pp. 56–67, 2015.
Tumer, K., Welch, Z. T., and Agogino, A. Aligning social welfare and agent preferences to alleviate traffic congestion. In Proceedings of the 7th Int. Conference on Autonomous Agents and Multiagent Systems, L. Padgham, D. Parkes, J. Müller, and S. Parsons (Eds.). IFAAMAS, Estoril, pp. 655–662, 2008.
Wardrop, J. G. Some theoretical aspects of road traffic research. Proceedings of the Institution of Civil Engineers, Part II 1 (36): 325–362, 1952.
Yu, Y., Han, K., and Ochieng, W. Day-to-day dynamic traffic assignment with imperfect information, bounded rationality and information sharing. Transportation Research Part C: Emerging Technologies vol. 114, pp. 59 – 83, 2020.
Zhou, B., Song, Q., Zhao, Z., and Liu, T. A reinforcement learning scheme for the equilibrium of the in-vehicle route choice problem based on congestion game. Applied Mathematics and Computation vol. 371, pp. 124895, 2020.
Publicado
20/10/2020
Como Citar
SANTOS, Guilherme; BAZZAN, Ana.
Accelerating Learning of Route Choices With C2I: A Preliminary Investigation. In: SYMPOSIUM ON KNOWLEDGE DISCOVERY, MINING AND LEARNING (KDMILE), 8. , 2020, Evento Online.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2020
.
p. 41-48.
ISSN 2763-8944.
DOI: https://doi.org/10.5753/kdmile.2020.11957.
