AnthillRL: Multi-agent Reinforcement Learning

• Anibal Sólon Heinsfeld PUCRS
• Felipe Meneguzzi PUCRS

Resumo

Ants are essentially social insects, and their organizational depen dency reflects directly on their survival. Due to their social nature, ant society provides a rich model for analysing properties of multiagent systems such as col laboration and effectiveness of collective action. Modelling natural behaviour of ants can help understand their actions, as well as studying better forms of co operation and competition for other multiagent models. In this paper, we model an ant society within a stochastic environment in which ant behaviour is gen erated using reinforcement learning to generate paths towards their goal. We test this model using a variable number of agents and learning parameters and report on the results.

Referências

Boissier, O., Bordini, R. H., Hübner, J. F., Ricci, A., and Santi, A. (2013). Multi-agent oriented programming with jacamo. Sci. Comput. Program., 78(6):747–761.

Bordini, R. H., Hübner, J. F., and Wooldridge, M. (2007). Programming Multi-Agent Systems in AgentSpeak using Jason.

Busoniu, L., Babuska, R., and De Schutter, B. (2008). A comprehensive survey of multiagent reinforcement learning. Systems, Man, and Cybernetics, Part C: Applications and Reviews, IEEE Transactions on, 38(2):156–172.

Gambardella, L. M., Dorigo, M., and Bruxelles, U. L. D. (1995). Ant-q: A reinforcement learning approach to the traveling salesman problem. pages 252–260. Morgan Kaufmann.

Hübner, J., Boissier, O., Kitio, R., and Ricci, A. (2010). Instrumenting multi-agent organisations with organisational artifacts and agents. Autonomous Agents and Multi-Agent Systems, 20(3):369–400.

Kaelbling, L. P., Littman, M. L., and Moore, A. W. (1996). Reinforcement learning: A survey. J. Artif. Int. Res., 4(1):237–285.

Mirjalili, S., Mirjalili, S. M., and Lewis, A. (2014). Grey wolf optimizer. Adv. Eng. Softw., 69:46–61.

Ricci, A., Piunti, M., and Viroli, M. (2011). Environment programming in multi-agent systems: an artifact-based perspective. Autonomous Agents and Multi-Agent Systems, 23(2):158–192.

Russell, S. and Norvig, P. (2010). Artificial Intelligence: A Modern Approach. Prentice Hall series in artificial intelligence. Prentice Hall.

Shtovba, S. (2005). Ant algorithms: Theory and applications. Programming and Computer Software, 31(4):167–178.

Sonmez, M. (2011). Artificial bee colony algorithm for optimization of truss structures. Applied Soft Computing, 11(2):2406 – 2418. The Impact of Soft Computing for the Progress of Artificial Intelligence.

Sorici, A., Picard, G., Boissier, O., Santi, A., and Hübner, J. F. (2012). Multi-agent oriented reorganisation within the jacamo infrastructure. In Proceedings of The Third International Workshop on Iinfraestructures and tools for multiagent systems: ITMAS, pages 135–148.

Subramanian, D., Druschel, P., and Chen, J. (1998). Ants and reinforcement learning: A case study in routing in dynamic networks. In In IJCAI (2, pages 832–838. Morgan Kaufmann.

Sutton, R. S. and Barto, A. G. (1998). Introduction to Reinforcement Learning. MIT Press, Cambridge, MA, USA, 1st edition.

Tan, M. (1993). Multi-Agent Reinforcement Learning Independent vs Cooperative Agents. PhD thesis, GTE Laboratories Incorporated.

Wilson, E. O. and Hölldobler, B. (2005). Eusociality: origin and consequences. Proceedings of the National Academy of Sciences of the United States of America, 102(38):13367–13371.

Yang, X.-S. and He, X. (2013). Bat algorithm: Literature review and applications. Int. J. Bio-Inspired Comput., 5(3):141–149.

Zhang, Q., Li, M., Wang, X., and Zhang, Y. (2012). Reinforcement learning in robot path optimization. Journal of Software, 7(3):657–662.