Skip to main content
SpringerLink
Log in

Applying Theory of Mind to Multi-agent Systems: A Systematic Review

  • Conference paper
  • First Online:
Intelligent Systems (BRACIS 2023)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 14195))

Included in the following conference series:

Abstract

Life in society requires constant communication and coordination. These abilities are efficiently achieved through sophisticated cognitive processes in which individuals are able to reason about the mental attitudes and actions of others. This ability is known as Theory of Mind. Inspired by human intelligence, the field of Artificial Intelligence aims to reproduce these sophisticated cognitive processes in intelligent software agents. In the field of multi-agent systems, intelligent agents are defined not only to execute reasoning cycles inspired by human reasoning but also to work similarly to human society, including aspects of communication, coordination, and organisation. Consequently, it is essential to explore the use of these sophisticated cognitive processes, such as Theory of Mind, in intelligent agents and multi-agent systems. In this paper, we conducted a literature review on how Theory of Mind has been applied to multi-agent systems, and summarise the contributions in this field.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 59.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 79.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    https://www.rayyan.ai/.

References

  1. Akata, Z., et al.: A research agenda for hybrid intelligence: augmenting human intellect with collaborative, adaptive, responsible, and explainable artificial intelligence. Computer 53(08), 18–28 (2020)

    Article  Google Scholar 

  2. Albrecht, S.V., Crandall, J.W., Ramamoorthy, S.: Belief and truth in hypothesised behaviours. Artif. Intell. 235, 63–94 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  3. Aylett, R., et al.: Werewolves, cheats, and cultural sensitivity. In: International Conference on Autonomous Agents and Multi-agent Systems (2014)

    Google Scholar 

  4. Bara, C.P., CH-Wang, S., Chai, J.: MindCraft: theory of mind modeling for situated dialogue in collaborative tasks. arXiv preprint arXiv:2109.06275 (2021)

  5. Bard, N., et al.: The Hanabi challenge: a new frontier for AI research. Artif. Intell. 280, 103216 (2020)

    Article  MathSciNet  MATH  Google Scholar 

  6. Baron-Cohen, S.: Mindblindness: An Essay on Autism and Theory of Mind. MIT Press (1997)

    Google Scholar 

  7. Baron-Cohen, S., Leslie, A.M., Frith, U.: Does the autistic child have a “theory of mind?’’. Cognition 21(1), 37–46 (1985)

    Article  Google Scholar 

  8. Berthiaume, V.G., Shultz, T.R., Onishi, K.H.: A constructivist connectionist model of transitions on false-belief tasks. Cognition 126(3), 441–458 (2013)

    Article  Google Scholar 

  9. Boella, G., Van der Torre, L., et al.: From the theory of mind to the construction of social reality. In: Proceedings of the Annual Conference on the Cognitive Science Society, vol. 5, pp. 298–303 (2005)

    Google Scholar 

  10. Bratman, M.: Intention, Plans, and Practical Reason. Harvard University Press, Cambridge (1987)

    Google Scholar 

  11. Bringsjord, S., et al.: Toward logic-based cognitively robust synthetic characters in digital environments. Front. Artif. Intell. Appl. 171, 87 (2008)

    Google Scholar 

  12. Cantucci, F., Falcone, R.: Towards trustworthiness and transparency in social human-robot interaction. In: 2020 IEEE International Conference on Human-Machine Systems (ICHMS), pp. 1–6. IEEE (2020)

    Google Scholar 

  13. Cantucci, F., Falcone, R.: Collaborative autonomy: human-robot interaction to the test of intelligent help. Electronics 11(19), 3065 (2022)

    Article  Google Scholar 

  14. Chang, H.M., Soo, V.W.: Simulation-based story generation with a theory of mind. In: Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment, vol. 4(1), pp. 16–21 (2008)

    Google Scholar 

  15. Cooke, A., Smith, D., Booth, A.: Beyond PICO: the SPIDER tool for qualitative evidence synthesis. Qual. Health Res. 22(10), 1435–1443 (2012)

    Article  Google Scholar 

  16. de Weerd, H., Verbrugge, R., Verheij, B.: Higher-order theory of mind in negotiations under incomplete information. In: Boella, G., Elkind, E., Savarimuthu, B.T.R., Dignum, F., Purvis, M.K. (eds.) PRIMA 2013. LNCS (LNAI), vol. 8291, pp. 101–116. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-44927-7_8

    Chapter  MATH  Google Scholar 

  17. De Weerd, H., Verbrugge, R., Verheij, B.: Negotiating with other minds: the role of recursive theory of mind in negotiation with incomplete information. Auton. Agent. Multi-Agent Syst. 31, 250–287 (2017)

    Article  Google Scholar 

  18. De Weerd, H., Verbrugge, R., Verheij, B.: Higher-order theory of mind is especially useful in unpredictable negotiations. Auton. Agent. Multi-Agent Syst. 36(2), 30 (2022)

    Article  Google Scholar 

  19. Dissing, L., Bolander, T.: Implementing theory of mind on a robot using dynamic epistemic logic. In: IJCAI, pp. 1615–1621 (2020)

    Google Scholar 

  20. Djerroud, H., Chérif, A.A.: VICA: a vicarious cognitive architecture environment model for navigation among movable obstacles. In: ICAART, vol. 2, pp. 298–305 (2021)

    Google Scholar 

  21. Doshi, P., Qu, X., Goodie, A.S., Young, D.L.: Modeling human recursive reasoning using empirically informed interactive partially observable Markov decision processes. IEEE Trans. Syst. Man Cybern. Part A Syst. Hum. 42(6), 1529–1542 (2012)

    Article  Google Scholar 

  22. Feng, D., Carstensdottir, E., El-Nasr, M.S., Marsella, S.: Exploring improvisational approaches to social knowledge acquisition. In: 18th International Conference on Autonomous Agents and MultiAgent Systems, AAMAS 2019 (2019)

    Google Scholar 

  23. Gebhard, P., Schneeberger, T., Baur, T., André, E.: MARSSI: model of appraisal, regulation, and social signal interpretation. In: International Conference on Autonomous Agents and Multi-agent Systems (2018)

    Google Scholar 

  24. Goldman, A.I.: Theory of Mind. Oxford University Press, United Kingdom (2012)

    Google Scholar 

  25. Gouidis, F., Vassiliades, A., Basina, N., Patkos, T.: Towards a formal framework for social robots with theory of mind. In: ICAART, vol. 3, pp. 689–696 (2022)

    Google Scholar 

  26. Harbers, M., van den Bosch, K., Meyer, J.J.C.: Agents with a theory of mind in virtual training. In: Multi-agent Systems for Education and Interactive Entertainment: Design, Use and Experience, pp. 172–187. IGI Global (2011)

    Google Scholar 

  27. Hoogendoorn, M., Merk, R.-J.: Action selection using theory of mind: a case study in the domain of fighter pilot training. In: Jiang, H., Ding, W., Ali, M., Wu, X. (eds.) IEA/AIE 2012. LNCS (LNAI), vol. 7345, pp. 521–533. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-31087-4_54

    Chapter  Google Scholar 

  28. Hoogendoorn, M., Soumokil, J.: Evaluation of virtual agents utilizing theory of mind in a real time action game. In: Proceedings of the 9th International Conference on Autonomous Agents and Multiagent Systems, vol. 1, pp. 59–66 (2010)

    Google Scholar 

  29. Hu, Q., Lu, Y., Pan, Z., Gong, Y., Yang, Z.: Can AI artifacts influence human cognition? The effects of artificial autonomy in intelligent personal assistants. Int. J. Inf. Manage. 56, 102250 (2021)

    Article  Google Scholar 

  30. Husemann, S., Pöppel, J., Kopp, S.: Differences and biases in mentalizing about humans and robots. In: 2022 31st IEEE International Conference on Robot and Human Interactive Communication (RO-MAN), pp. 490–497. IEEE (2022)

    Google Scholar 

  31. Kaufmann, R., Gupta, P., Taylor, J.: An active inference model of collective intelligence. Entropy 23(7), 830 (2021)

    Article  Google Scholar 

  32. Kelley, R., Tavakkoli, A., King, C., Nicolescu, M., Nicolescu, M., Bebis, G.: Understanding human intentions via hidden Markov models in autonomous mobile robots. In: Proceedings of the 3rd ACM/IEEE International Conference on Human Robot Interaction, pp. 367–374 (2008)

    Google Scholar 

  33. Lerer, A., Hu, H., Foerster, J., Brown, N.: Improving policies via search in cooperative partially observable games. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34, no. 05, pp. 7187–7194 (2020)

    Google Scholar 

  34. Lin, B., Bouneffouf, D., Cecchi, G.: Predicting human decision making with LSTM. In: 2022 International Joint Conference on Neural Networks (IJCNN), pp. 1–8. IEEE (2022)

    Google Scholar 

  35. Lorini, E.: Rethinking epistemic logic with belief bases. Artif. Intell. 282, 103233 (2020)

    Article  MathSciNet  MATH  Google Scholar 

  36. Lorini, E., Jimenez, B.F.R.: Decision procedures for epistemic logic exploiting belief bases. In: 18th International Conference on Autonomous Agents and MultiAgent Systems, AAMAS 2019, pp. 944–952 (2019)

    Google Scholar 

  37. Malle, B.F.: The relation between language and theory of mind in development and evolution. In: The Evolution of Language Out of Pre-language, vol. 18, pp. 265–284 (2002)

    Google Scholar 

  38. Marsella, S.C., Pynadath, D.V.: Modeling influence and theory of mind. In: Virtual Social Agents, p. 199 (2005)

    Google Scholar 

  39. Narang, S., Best, A., Manocha, D.: Inferring user intent using Bayesian theory of mind in shared avatar-agent virtual environments. IEEE Trans. Vis. Comput. Graph. 25(5), 2113–2122 (2019)

    Article  Google Scholar 

  40. Nguyen, D., Nguyen, P., Le, H., Do, K., Venkatesh, S., Tran, T.: Learning theory of mind via dynamic traits attribution. arXiv preprint arXiv:2204.09047 (2022)

  41. Page, M.J., et al.: The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ 372, n71 (2021)

    Article  Google Scholar 

  42. Panisson, A.R., et al.: On the formal semantics of theory of mind in agent communication. In: Lujak, M. (ed.) AT 2018. LNCS (LNAI), vol. 11327, pp. 18–32. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-17294-7_2

    Chapter  Google Scholar 

  43. Pantelis, P.C., et al.: Agency and rationality: adopting the intentional stance toward evolved virtual agents. Decision 3(1), 40 (2016)

    Article  Google Scholar 

  44. Persiani, M., Hellström, T.: Inference of the intentions of unknown agents in a theory of mind setting. In: Dignum, F., Corchado, J.M., De La Prieta, F. (eds.) PAAMS 2021. LNCS (LNAI), vol. 12946, pp. 188–200. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-85739-4_16

    Chapter  Google Scholar 

  45. Pöppel, J., Kahl, S., Kopp, S.: Resonating minds-emergent collaboration through hierarchical active inference. Cogn. Comput. 14(2), 581–601 (2022)

    Article  Google Scholar 

  46. Premack, D., Woodruff, G.: Does the Chimpanzee have a theory of mind? Behav. Brain Sci. 1(4), 515–526 (1978)

    Article  Google Scholar 

  47. Pynadath, D.V., et al.: Disaster world: decision-theoretic agents for simulating population responses to hurricanes. Comput. Math. Organ. Theor. 29(1), 84–117 (2022)

    Article  Google Scholar 

  48. Pynadath, D.V., Marsella, S.C.: PsychSim: modeling theory of mind with decision-theoretic agents. In: IJCAI, vol. 5, pp. 1181–1186 (2005)

    Google Scholar 

  49. Pynadath, D.V., Marsella, S.C.: Socio-cultural modeling through decision-theoretic agents with theory of mind. In: Advances in Design for Cross-Cultural Activities, pp. 417–426 (2012)

    Google Scholar 

  50. Pynadath, D.V., Rosenbloom, P.S., Marsella, S.C.: Reinforcement learning for adaptive theory of mind in the sigma cognitive architecture. In: Goertzel, B., Orseau, L., Snaider, J. (eds.) AGI 2014. LNCS (LNAI), vol. 8598, pp. 143–154. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-09274-4_14

    Chapter  Google Scholar 

  51. Pynadath, D.V., Si, M., Marsella, S.C.: Modeling theory of mind and cognitive appraisal with decision-theoretic agents. In: Social Emotions in Nature and Artifact: Emotions in Human and Human-Computer Interaction, pp. 70–87 (2011)

    Google Scholar 

  52. Reisenzein, R., et al.: Computational modeling of emotion: toward improving the inter-and intradisciplinary exchange. IEEE Trans. Affect. Comput. 4(3), 246–266 (2013)

    Article  Google Scholar 

  53. Rumbell, T., Barnden, J., Denham, S., Wennekers, T.: Emotions in autonomous agents: comparative analysis of mechanisms and functions. Auton. Agent. Multi-Agent Syst. 25, 1–45 (2012)

    Article  Google Scholar 

  54. Sarkadi, Ş, Panisson, A.R., Bordini, R.H., McBurney, P., Parsons, S.: Towards an approach for modelling uncertain theory of mind in multi-agent systems. In: Lujak, M. (ed.) AT 2018. LNCS (LNAI), vol. 11327, pp. 3–17. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-17294-7_1

    Chapter  Google Scholar 

  55. Sarkadi, S., Panisson, A.R., Bordini, R.H., McBurney, P., Parsons, S., Chapman, M.: Modelling deception using theory of mind in multi-agent systems. AI Commun. 32(4), 287–302 (2019)

    Article  MathSciNet  MATH  Google Scholar 

  56. Sarthou, G., Clodic, A., Alami, R.: Ontologenius: a long-term semantic memory for robotic agents. In: 2019 28th IEEE International Conference on Robot and Human Interactive Communication (RO-MAN), pp. 1–8. IEEE (2019)

    Google Scholar 

  57. Scassellati, B.: Theory of mind for a humanoid robot. Auton. Robot. 12, 13–24 (2002)

    Article  MATH  Google Scholar 

  58. Shum, M., Kleiman-Weiner, M., Littman, M.L., Tenenbaum, J.B.: Theory of minds: understanding behavior in groups through inverse planning. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 33(1), pp. 6163–6170 (2019)

    Google Scholar 

  59. Si, M., Marsella, S.C., Pynadath, D.V.: Modeling appraisal in theory of mind reasoning. Auton. Agent. Multi-Agent Syst. 20, 14–31 (2010)

    Article  Google Scholar 

  60. Stevens, C., de Weerd, H., Cnossen, F., Taatgen, N.: A metacognitive agent for training negotiation skills. In: Proceedings of the 14th International Conference on Cognitive Modeling, ICCM 2016 (2016)

    Google Scholar 

  61. Tekülve, J., Schöner, G.: Neural dynamic concepts for intentional systems. In: CogSci, pp. 1090–1096 (2019)

    Google Scholar 

  62. Veltman, K., de Weerd, H., Verbrugge, R.: Training the use of theory of mind using artificial agents. J. Multimodal User Interfaces 13, 3–18 (2019)

    Article  Google Scholar 

  63. Von Der Osten, F.B., Kirley, M., Miller, T.: The minds of many: Opponent modeling in a stochastic game. In: IJCAI, pp. 3845–3851 (2017)

    Google Scholar 

  64. Walton, D.: Using argumentation schemes to find motives and intentions of a rational agent. Argument Computat. 10(3), 233–275 (2019)

    Article  Google Scholar 

  65. Wang, R.E., Wu, S.A., Evans, J.A., Tenenbaum, J.B., Parkes, D.C., Kleiman-Weiner, M.: Too many cooks: coordinating multi-agent collaboration through inverse planning. In: Proceedings of the 19th International Conference on Autonomous Agents and MultiAgent Systems, pp. 2032–2034 (2020)

    Google Scholar 

  66. Ware, S.G., Siler, C.: Sabre: a narrative planner supporting intention and deep theory of mind. In: Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment, vol. 17, no. 1, pp. 99–106 (2021)

    Google Scholar 

  67. Whiten, A., Byrne, R.: Natural Theories of Mind: Evolution, Development and Simulation of Everyday Mindreading. B. Blackwell Oxford, UK (1991)

    Google Scholar 

  68. Wooldridge, M.: An Introduction to Multiagent Systems. Wiley, Hoboken (2009)

    Google Scholar 

  69. Yousefi, Z., Heinke, D., Apperly, I., Siebers, P.-O.: An agent-based model for false belief tasks: belief representation systematic approach (BRSA). In: Thomson, R., Dancy, C., Hyder, A., Bisgin, H. (eds.) SBP-BRiMS 2018. LNCS, vol. 10899, pp. 111–126. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-93372-6_14

    Chapter  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computing, UFSC, Florianópolis, Brazil

    Michele Rocha, Heitor Henrique da Silva, Analúcia Schiaffino Morales & Alison R. Panisson

  2. Department of Informatics, KCL, London, UK

    Stefan Sarkadi

Authors
  1. Michele Rocha
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Heitor Henrique da Silva
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Analúcia Schiaffino Morales
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Stefan Sarkadi
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Alison R. Panisson
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Alison R. Panisson .

Editor information

Editors and Affiliations

  1. Federal University of São Carlos, São Carlos, Brazil

    Murilo C. Naldi

  2. Centro Universitario da FEI, São Bernardo do Campo, Brazil

    Reinaldo A. C. Bianchi

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Rocha, M., da Silva, H.H., Morales, A.S., Sarkadi, S., Panisson, A.R. (2023). Applying Theory of Mind to Multi-agent Systems: A Systematic Review. In: Naldi, M.C., Bianchi, R.A.C. (eds) Intelligent Systems. BRACIS 2023. Lecture Notes in Computer Science(), vol 14195. Springer, Cham. https://doi.org/10.1007/978-3-031-45368-7_24

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-45368-7_24

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-45367-0

  • Online ISBN: 978-3-031-45368-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation