Modeling an Empathetic Embodied Conversational Agent

  • Paulo R. Knob PUCRS
  • Soraia R. Musse PUCRS


Empathy can be seen as a complex socio-emotional behavior, which is a result from the interaction between both cognitive and affective devices and is responsible, for instance, for one person being able to identify and mimicry others emotion. Moreover, the human memory is a powerful tool which allows for each person to be able to store and retrieve information about almost everything that happens. Equipping an embodied conversational agent (ECAs) with the ability of empathy, as well other features like memory, can help to make the interaction with humans smoother and more natural. This work aims to propose an empathetic conversational agent endowed with a human-like memory and the ability to show some extent of empathy. Also, this model endow the virtual agent with a few other abilities, like recognizing the person it is talking to by its face and store/retrieve information based on a human memory model. Some experiments were conducted to gather both quantitative and qualitative information, which show that the proposed model works as intended. Finally, a few avenues for future work are also presented, elucidating what it is planned to do aiming to improve the quality of this work.


J. Cassell, J. Sullivan, E. Churchill, and S. Prevost, Embodied conversational agents. MIT press, 2000.

F. B. de Waal and S. D. Preston, “Mammalian empathy: behavioural manifestations and neural basis,” Nature Reviews Neuroscience, vol. 18, no. 8, pp. 498–509, 2017.

J. Cassell, C. Pelachaud, N. Badler, M. Steedman, B. Achorn, T. Becket, B. Douville, S. Prevost, and M. Stone, “Animated conversation: rulebased generation of facial expression, gesture & spoken intonation for multiple conversational agents,” in Proceedings of the 21st annual conference on Computer graphics and interactive techniques, 1994, pp. 413–420.

M. Argyle and M. Cook, “Gaze and mutual gaze.” 1976.

G. Collier and G. J. Collier, Emotional expression. Psychology Press, 2014.

Ö . N. Yalçın, “Empathy framework for embodied conversational agents,” Cognitive Systems Research, vol. 59, pp. 123–132, 2020.

B. Biancardi, C. Wang, M. Mancini, A. Cafaro, G. Chanel, and C. Pelachaud, “A computational model for managing impressions of an embodied conversational agent in real-time,” in 2019 8th International Conference on Affective Computing and Intelligent Interaction (ACII). IEEE, 2019, pp. 1–7.

P. Sajjadi, L. Hoffmann, P. Cimiano, and S. Kopp, “A personalitybased emotional model for embodied conversational agents: Effects on perceived social presence and game experience of users,” Entertainment Computing, vol. 32, p. 100313, 2019.

L. Kramer, S. ter Stal, B. Mulder, E. de Vet, and L. van Velsen, “Developing embodied conversational agents for healthy lifestyles: a scoping review,” in ARPH, 2019.

M. Spitale, F. Catania, P. Crovari, and F. Garzotto, “Multicriteria decision analysis and conversational agents for children with autism,” in Proceedings of the 53rd Hawaii International Conference on System Sciences, 2020.

K. S. J. Das, T. Beinema, H. Op Den Akker, and H. Hermens, “Generation of multi-party dialogues among embodied conversational agents to promote active living and healthy diet for subjects suffering from type 2 diabetes,” in 5th International Conference on Information and Communication Technologies for Ageing Well and e-Health, ICT4AWE 2019. SCITEPRESS, 2019, pp. 297–304.

P. Philip, L. Dupuy, M. Auriacombe, F. Serre, E. de Sevin, A. Sauteraud, and J.-A. Micoulaud-Franchi, “Trust and acceptance of a virtual psychiatric interview between embodied conversational agents and outpatients,” npj Digital Medicine, vol. 3, no. 1, pp. 1–7, 2020.

J. Martínez-Miranda, A. Martínez, R. Ramos, H. Aguilar, L. Jiménez, H. Arias, G. Rosales, and E. Valencia, “Assessment of users’ acceptability of a mobile-based embodied conversational agent for the prevention and detection of suicidal behaviour,” Journal of medical systems, vol. 43, no. 8, p. 246, 2019.

G. Chetty and M. White, “Embodied conversational agents and interactive virtual humans for training simulators,” in Proc. The 15th International Conference on Auditory-Visual Speech Processing, 2019, pp. 73–77.

E. Ayedoun, Y. Hayashi, and K. Seta, “Adding communicative and affective strategies to an embodied conversational agent to enhance second language learners’ willingness to communicate,” International Journal of Artificial Intelligence in Education, vol. 29, no. 1, pp. 29–57, 2019.

Y. Zhang, X. Gao, S. Lee, C. Brockett, M. Galley, J. Gao, and B. Dolan, “Consistent dialogue generation with self-supervised feature learning,” arXiv preprint arXiv:1903.05759, 2019.

E. A. Croes and M. L. Antheunis, “Can we be friends with mitsuku? a longitudinal study on the process of relationship formation between humans and a social chatbot,” Journal of Social and Personal Relationships, vol. 38, no. 1, pp. 279–300, 2021.

S. Bluck and L. J. Levine, “Reminiscence as autobiographical memory: A catalyst for reminiscence theory development,” Ageing & Society, vol. 18, no. 2, pp. 185–208, 1998.

M. A. Conway and C. W. Pleydell-Pearce, “The construction of autobiographical memories in the self-memory system.” Psychological review, vol. 107, no. 2, p. 261, 2000.

A. Mehrabian and N. Epstein, “A measure of emotional empathy.” Journal of personality, 1972.

R. N. Spreng*, M. C. McKinnon*, R. A. Mar, and B. Levine, “The toronto empathy questionnaire: Scale development and initial validation of a factor-analytic solution to multiple empathy measures,” Journal of personality assessment, vol. 91, no. 1, pp. 62–71, 2009.

F. B. De Waal, “Putting the altruism back into altruism: the evolution of empathy,” Annu. Rev. Psychol., vol. 59, pp. 279–300, 2008.

J. K. Melgare, S. R. Musse, N. R. Schneider, and R. B. Queiroz, “Investigating emotion style in human faces and avatars,” in 2019 18th Brazilian Symposium on Computer Games and Digital Entertainment (SBGames). IEEE, 2019, pp. 115–124.
KNOB, Paulo R.; MUSSE, Soraia R.. Modeling an Empathetic Embodied Conversational Agent. In: WORKSHOP DE TESES E DISSERTAÇÕES - CONFERENCE ON GRAPHICS, PATTERNS AND IMAGES (SIBGRAPI), 36. , 2023, Rio Grande/RS. Anais [...]. Porto Alegre: Sociedade Brasileira de Computação, 2023 . p. 1-6. DOI: