Guidelines for Modeling Expressivity in Virtual Patients for Healthcare Training
Resumo
Virtual patients are increasingly being used in healthcare education to support the development of communication and social skills. However, their effectiveness depends on the integration of expressive realism across verbal and nonverbal communication modalities. This paper proposes a set of guidelines for modeling emotional expressivity in virtual patients by combining facial expressions, body posture, and vocal prosody. Grounded in established affective science literature—including Ekman’s basic emotions, the Facial Action Coding System (FACS), the Body Action and Posture (BAP) framework, and GeMAPS acoustic parameters—these guidelines synthesize expressive features for six core emotions: joy, sadness, anger, fear, disgust, and surprise. Our goal is to support the development of emotionally expressive virtual patients for medical simulations and virtual reality training.Referências
D. A. Cook, P. J. Erwin, and M. M. Triola, “Computerized virtual patients in health professions education: a systematic review and meta-analysis,” Academic Medicine, vol. 85, no. 10, pp. 1589–1602, 2010.
J. Cleland, K. Abe, and J.-J. Rethans, “The use of simulated patients in medical education: Amee guide no 42,” Medical Teacher, vol. 31, no. 6, pp. 477–486, 2009.
P. Kenny, T. D. Parsons, J. Gratch, A. Leuski, and A. Rizzo, “Virtual humans for assisted health care training,” in Studies in health technology and informatics, vol. 125. IOS Press, 2007, pp. 133–138.
A. Rizzo and S. T. Koenig, “Is clinical virtual reality ready for primetime?” Neuropsychology, vol. 31, no. 8, pp. 877–899, 2017.
D. Ring, I. Barry, J. Totzke, and B. Lok, “Designing virtual patients to elicit social and empathic responses from medical students,” Frontiers in Virtual Reality, vol. 3, p. 846470, 2022.
W. Ferreira, M. D. Negrão, A. Maciel, R. P. Torchelsen, and L. Nedel, “Llm vs. human driven conversation: A user study in virtual reality medical simulation,” in Proceedings of the Symposium on Virtual and Augmented Reality 2025 (SVR). Brazil: IEEE, 2025, to appear.
P. Ekman and W. V. Friesen, “Constants across cultures in the face and emotion,” Journal of Personality and Social Psychology, vol. 17, no. 2, pp. 124–129, 1971.
——, Facial Action Coding System: A technique for the measurement of facial movement. Palo Alto, CA: Consulting Psychologists Press, 1978.
P. Ekman, W. V. Friesen, and P. C. Ellsworth, Emotion in the Human Face: Guidelines for Research and an Integration of Findings, 3rd ed. Dallas, TX: Malor Books, 2013.
T. Bänziger and K. R. Scherer, “Introducing the geneva multimodal emotion portrayal (gemep) corpus,” in A Blueprint for Affective Computing: A sourcebook and manual, K. R. Scherer, T. Bänziger, and E. Roesch, Eds. Oxford, UK: Oxford University Press, 2010, pp. 271–294.
N. Dael, M. Mortillaro, and K. R. Scherer, “The body action and posture coding system (bap): Development and reliability,” Journal of Nonverbal Behavior, vol. 36, pp. 97–121, 6 2012.
——, “Emotion expression in body action and posture.” Emotion, vol. 12, pp. 1085–1101, 2012. [Online]. Available: [link]
F. Eyben, K. R. Scherer, B. W. Schuller, J. Sundberg, E. André, C. Busso, L. Devillers, J. Epps, P. Laukka, S. S. Narayanan et al., “The geneva minimalistic acoustic parameter set (gemaps) for voice research and affective computing,” IEEE Transactions on Affective Computing, vol. 7, no. 2, pp. 190–202, 2016.
M. Coulson, “Attributing emotion to static body postures: Recognition accuracy, confusions, and viewpoint dependence,” Journal of Nonverbal Behavior, vol. 28, no. 2, pp. 117–139, 2004.
M. de Meijer, “The contribution of general features of body movement to the attribution of emotions,” Journal of Nonverbal Behavior, vol. 13, no. 4, pp. 247–268, 1989.
S. R. Livingstone and F. A. Russo, “The ryerson audio-visual database of emotional speech and song (ravdess),” PLOS ONE, vol. 13, no. 5, p. e0196391, 2018. [Online]. DOI: 10.1371/journal.pone.0196391
J. Cleland, K. Abe, and J.-J. Rethans, “The use of simulated patients in medical education: Amee guide no 42,” Medical Teacher, vol. 31, no. 6, pp. 477–486, 2009.
P. Kenny, T. D. Parsons, J. Gratch, A. Leuski, and A. Rizzo, “Virtual humans for assisted health care training,” in Studies in health technology and informatics, vol. 125. IOS Press, 2007, pp. 133–138.
A. Rizzo and S. T. Koenig, “Is clinical virtual reality ready for primetime?” Neuropsychology, vol. 31, no. 8, pp. 877–899, 2017.
D. Ring, I. Barry, J. Totzke, and B. Lok, “Designing virtual patients to elicit social and empathic responses from medical students,” Frontiers in Virtual Reality, vol. 3, p. 846470, 2022.
W. Ferreira, M. D. Negrão, A. Maciel, R. P. Torchelsen, and L. Nedel, “Llm vs. human driven conversation: A user study in virtual reality medical simulation,” in Proceedings of the Symposium on Virtual and Augmented Reality 2025 (SVR). Brazil: IEEE, 2025, to appear.
P. Ekman and W. V. Friesen, “Constants across cultures in the face and emotion,” Journal of Personality and Social Psychology, vol. 17, no. 2, pp. 124–129, 1971.
——, Facial Action Coding System: A technique for the measurement of facial movement. Palo Alto, CA: Consulting Psychologists Press, 1978.
P. Ekman, W. V. Friesen, and P. C. Ellsworth, Emotion in the Human Face: Guidelines for Research and an Integration of Findings, 3rd ed. Dallas, TX: Malor Books, 2013.
T. Bänziger and K. R. Scherer, “Introducing the geneva multimodal emotion portrayal (gemep) corpus,” in A Blueprint for Affective Computing: A sourcebook and manual, K. R. Scherer, T. Bänziger, and E. Roesch, Eds. Oxford, UK: Oxford University Press, 2010, pp. 271–294.
N. Dael, M. Mortillaro, and K. R. Scherer, “The body action and posture coding system (bap): Development and reliability,” Journal of Nonverbal Behavior, vol. 36, pp. 97–121, 6 2012.
——, “Emotion expression in body action and posture.” Emotion, vol. 12, pp. 1085–1101, 2012. [Online]. Available: [link]
F. Eyben, K. R. Scherer, B. W. Schuller, J. Sundberg, E. André, C. Busso, L. Devillers, J. Epps, P. Laukka, S. S. Narayanan et al., “The geneva minimalistic acoustic parameter set (gemaps) for voice research and affective computing,” IEEE Transactions on Affective Computing, vol. 7, no. 2, pp. 190–202, 2016.
M. Coulson, “Attributing emotion to static body postures: Recognition accuracy, confusions, and viewpoint dependence,” Journal of Nonverbal Behavior, vol. 28, no. 2, pp. 117–139, 2004.
M. de Meijer, “The contribution of general features of body movement to the attribution of emotions,” Journal of Nonverbal Behavior, vol. 13, no. 4, pp. 247–268, 1989.
S. R. Livingstone and F. A. Russo, “The ryerson audio-visual database of emotional speech and song (ravdess),” PLOS ONE, vol. 13, no. 5, p. e0196391, 2018. [Online]. DOI: 10.1371/journal.pone.0196391
Publicado
30/09/2025
Como Citar
FERREIRA, Wesley Ferreira de; NEDEL, Luciana.
Guidelines for Modeling Expressivity in Virtual Patients for Healthcare Training. In: WORKSHOP ON VIRTUAL HUMANS - CONFERENCE ON GRAPHICS, PATTERNS AND IMAGES (SIBGRAPI), 38. , 2025, Salvador/BA.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2025
.
p. 345-348.
