More Knowledge, More Efficiency: Using Non-Local Information on Multiple Traffic Attributes

• Ana L. C. Bazzan Universidade Federal do Rio Grande do Sul
• Henrique Uhlmann Gobbi Universidade Federal do Rio Grande do Sul
• Guilherme D. dos Santos Universidade Federal do Rio Grande do Sul

DOI: https://doi.org/10.5753/kdmile.2022.227737

Resumo

New technologies have the potential to transform urban mobility. Among the contributions, providing timely information to drivers via, e.g., apps, is proving valuable. However, providing the same information to nearly everyone is counterproductive. In this paper we extend previous works in which vehicles and the road infrastructure exchange information to allow drivers to make better informed decisions when using reinforcement learning. Here, we use non-local information to augment the knowledge elements of the infrastructure have. Moreover, we connect these elements when they have similar patterns related to multiple attributes, including emission of gases. Our results show that using augmented information leads to more efficiency.

Palavras-chave: Multi-Agent Reinforcement Learning, Route Choice, Non-Local Information

Referências

Auld, J., Verbas, O., and Stinson, M. Agent-based dynamic trac assignment with information mixing. Procedia Computer Science vol. 151, pp. 864-869, 2019.

Bazzan, A. L. C. and Grunitzki, R. A multiagent reinforcement learning approach to en-route trip building. In 2016 International Joint Conference on Neural Networks (IJCNN). pp. 5288-5295, 2016.

Cui, K., Tahir, A., Ekinci, G., Elshamanhory, A., Eich, Y., Li, M., and Koeppl, H. A survey on large-population systems and scalable multi-agent reinforcement learning, 2022.

Grunitzki, R. and Bazzan, A. L. C. Combining car-to-infrastructure communication and multi-agent reinforcement learning in route choice. In Proceedings of the Ninth Workshop on Agents in Trac and Transportation (ATT-2016), A. L. C. Bazzan, F. Klügl, S. Ossowski, and G. Vizzari (Eds.). CEUR Workshop Proceedings, vol. 1678. CEUR-WS.org, New York, 2016.

Huanca-Anquise, C. A. Multi-objective reinforcement learning methods for action selection: dealing with multiple objectives and non-stationarity. M.S. thesis, Instituto de Informática, UFRGS, Porto Alegre, Brazil, 2021.

Lopez, P. A., Behrisch, M., Bieker-Walz, L., Erdmann, J., Flötteröd, Y.-P., Hilbrich, R., Lücken, L., Rummel, J., Wagner, P., and Wieÿner, E. Microscopic trac simulation using SUMO. In The 21st IEEE International Conference on Intelligent Transportation Systems, 2018.

Mahmassani, H. S. Autonomous vehicles and connected vehicle systems: Flow and operations considerations. Transp. Sci. 50 (4): 1140-1162, 2016.

Maimaris, A. and Papageorgiou, G. A review of intelligent transportation systems from a communications technology perspective. In 2016 IEEE 19th International Conference on Intelligent Transportation Systems (ITSC). pp. 54-59, 2016.

Ortúzar, J. d. D. and Willumsen, L. G. Modelling transport. John Wiley & Sons, Chichester, UK, 2011.

Ramos, G. de. O., da Silva, B. C., and Bazzan, A. L. C. Learning to minimise regret in route choice. In Proc. of the 16th International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2017), S. Das, E. Durfee, K. Larson, and M. Winiko (Eds.). IFAAMAS, São Paulo, pp. 846-855, 2017.

Ramos, G. de. O. and Grunitzki, R. An improved learning automata approach for the route choice problem. In Agent Technology for Intelligent Mobile Services and Smart Societies, F. Koch, F. Meneguzzi, and K. Lakkaraju (Eds.). Communications in Computer and Information Science, vol. 498. Springer, pp. 56-67, 2015.

Santos, G. D. dos. and Bazzan, A. L. C. Accelerating learning of route choices with C2I: A preliminary investigation. In Proc. of the VIII Symposium on Knowledge Discovery, Mining and Learning. SBC, Rio Grande, pp. 41-48, 2020.

Santos, G. D. dos. and Bazzan, A. L. C. Sharing diverse information gets driver agents to learn faster: an application in en route trip building. PeerJ Computer Science vol. 7, pp. e428, March, 2021.

Santos, G. D. dos. and Bazzan, A. L. C. A multiobjective reinforcement learning approach to trip building. In Proc. of the 12th International Workshop on Agents in Trac and Transportation (ATT 2022), A. L. Bazzan, I. Dusparic, M. Lujak, and G. Vizzari (Eds.). Vol. 3173. CEUR-WS.org, pp. 160-174, 2022.

Santos, G. D. dos., Bazzan, A. L. C., and Baumgardt, A. P. Using car to infrastructure communication to accelerate learning in route choice. Journal of Information and Data Management 12 (2), 2021.

Yu, Y., Han, K., and Ochieng, W. Day-to-day dynamic trac assignment with imperfect information, bounded rationality and information sharing. Transportation Research Part C: Emerging Technologies vol. 114, pp. 59-83, 2020.

Zhou, B., Song, Q., Zhao, Z., and Liu, T. A reinforcement learning scheme for the equilibrium of the in-vehicle route choice problem based on congestion game. Applied Mathematics and Computation vol. 371, pp. 124895, 2020.