A Brief Overview of Social Dilemmas: From Social Sciences to Multiagent Based Simulations

• Rafael M. Cheang USP
• Anarosa A. F. Brandão USP
• Jaime S. Sichman USP

Resumo

Social dilemma is the name given to a set of games with conflicting individual and communal incentives and characterized by the existence of deficient equilibria. This area of research has for a long time drawn the attention of the social sciences, but not as much from the multiagent community. This paper aims to introduce this multidisciplinary area of research to computer scientists and engineers with a particular interest in multiagent systems. To this end, we present an overview of how the field has evolved over the last 40 years that includes: an introduction, key learning points coming from the social sciences, how social dilemmas are being simulated in multiagent Markov game settings, and the road ahead; difficulties and perspectives.

Referências

Axelrod, R. (1980a). Effective Choice in the Prisoner‘s Dilemma. Journal of Conflict Resolution, 24(1):3–25.

Axelrod, R. (1980b). More Effective Choice in the Prisoner‘s Dilemma. Journal of Conflict Resolution, 24(3):379–403.

Axelrod, R. (1984). The Evolution of Cooperation. Basic, New York.

Barbosa, J. V., Costa, A. H. R., Melo, F. S., Sichman, J. S., and Santos, F. C. (2020). Emergence of Cooperation in N-Person Dilemmas through Actor-Critic Reinforcement Learning. Adaptive Learning Workshop (ALA), AAMAS 2020 - Autonomous Agents and Multi-Agent Systems.

Dafoe, A., Hughes, E., Bachrach, Y., Collins, T., McKee, K. R., Leibo, J. Z., Larson, K., and Graepel, T. (2020). Open problems in cooperative ai.

Dawes, R. M. (1980). Social Dilemmas. Annual Review of Psychology, 31(1):169–193.

Deadman, P. J. (1999). Modelling individual behaviour and group performance in an intelligent agent-based simulation of the tragedy of the commons. Journal of Environmental Management, 56:159–172.

Eccles, T., Hughes, E., Kramár, J., Wheelwright, S., and Leibo, J. Z. (2019). Learning reciprocity in complex sequential social dilemmas.

Gotts, N. M., Polhill, J. G., and Law, A. N. (2003). Agent-based simulation in the study of social dilemmas. Artificial Intelligence Review, 19:3–92.

Guerberoff, I., Queiroz, D., and Sichman, J. (2011). Studies on the effect of the expressiveness of two strategy representation languages for the iterated n-player prisoner’s dilemma. Revue d’Intelligence Artificielle, 25:69–82.

Hardin, G. (1968). The tragedy of the commons. Science, 162(3859):1243–1248.

Hughes, E., Leibo, J. Z., Phillips, M., Tuyls, K., Dueñez Guzman, E., García Castañeda, A., Dunning, I., Zhu, T., McKee, K., Koster, R., Roff, H., and Graepel, T. (2018). Inequity aversion improves cooperation in intertemporal social dilemmas. In Bengio, S., Wallach, H., Larochelle, H., Grauman, K., Cesa-Bianchi, N., and Garnett, R., editors, Advances in Neural Information Processing Systems, volume 31. Curran Associates, Inc.

Izquierdo, S., Izquierdo, L., and Gotts, N. (2008). Reinforcement learning dynamics in social dilemmas. Journal of Artificial Societies and Social Simulation, 11.

Kollock, P. (1998). Social dilemmas: The Anatomy of Cooperation. Annual Review of Sociology, 24(1):183–214.

Leibo, J. Z., Zambaldi, V., Lanctot, M., Marecki, J., and Graepel, T. (2017). Multi-agent reinforcement learning in sequential social dilemmas. In Proceedings of the 16th Conference on Autonomous Agents and MultiAgent Systems, AAMAS ’17, page 464–473, Richland, SC. International Foundation for Autonomous Agents and Multiagent Systems.

Lerer, A. and Peysakhovich, A. (2018). Maintaining cooperation in complex social dilemmas using deep reinforcement learning.

Littman, M. L. (1994). Markov games as a framework for multi-agent reinforcement learning. In Proceedings of the Eleventh International Conference on International Conference on Machine Learning, ICML’94, page 157–163, San Francisco, CA, USA. Morgan Kaufmann Publishers Inc.

McKee, K. R., Gemp, I., McWilliams, B., Duèñez Guzmán, E. A., Hughes, E., and Leibo, J. Z. (2020). Social diversity and social preferences in mixed-motive reinforcement learning. In Proceedings of the 19th International Conference on Autonomous Agents and MultiAgent Systems, AAMAS ’20, page 869–877, Richland, SC. International Foundation for Autonomous Agents and Multiagent Systems.

Ostrom, E. (1999). Coping with tragedies of the commons. Annual Review of Political Science, 2(1):493–535.

Ostrom, E. (2000). Collective action and the evolution of social norms. Journal of Economic Perspectives, 14(3):137–158.

Pérolat, J., Leibo, J. Z., Zambaldi, V., Beattie, C., Tuyls, K., and Graepel, T. (2017). A multi-agent reinforcement learning model of common-pool resource appropriation. In Guyon, I., Luxburg, U. V., Bengio, S., Wallach, H., Fergus, R., Vishwanathan, S., and Garnett, R., editors, Advances in Neural Information Processing Systems, volume 30. Curran Associates, Inc.

Queiroz, D. and Sichman, J. (2016). Parallel simulations of the iterated n-player prisoner’s dilemma. In Gaudou, B. and Sichman, J. S., editors, Multi-Agent Based Simulation XVI, pages 87–105, Cham. Springer International Publishing.

Silver, D., Huang, A., Maddison, C., Guez, A., Sifre, L., Driessche, G., Schrittwieser, J., Antonoglou, I., Panneershelvam, V., Lanctot, M., Dieleman, S., Grewe, D., Nham, J., Kalchbrenner, N., Sutskever, I., Lillicrap, T., Leach, M., Kavukcuoglu, K., Graepel, T., and Hassabis, D. (2016). Mastering the game of go with deep neural networks and tree search. Nature, 529:484–489.

Silver, D., Hubert, T., Schrittwieser, J., Antonoglou, I., Lai, M., Guez, A., Lanctot, M., Sifre, L., Kumaran, D., Graepel, T., Lillicrap, T., Simonyan, K., and Hassabis, D. (2018). A general reinforcement learning algorithm that masters chess, shogi, and go through self-play. Science, 362(6419):1140–1144.