Debriefing Tool for Immersive Analytics in Medical Training Simulations
Resumo
Immersive technologies enhance training and performance evaluation in fields like medical education. By enabling detailed skill assessment through immersive analytics, Virtual reality (VR) systems also allow instructors to capture and study user interactions in simulated environments, linking theoretical metrics to real-world skill evaluations via behavioral visualization. This paper proposes a system for collecting, storing, and reconstructing user interaction data in medical VR simulations, recording hand movements, 6-DoF positioning, and object manipulation. Data is synchronized to a cloud platform, where a reconstruction process generates a real-time 'ghost avatar' replay with spatial fidelity. Instructors and trainees review sessions immersively using interactive controls to analyze procedural steps, clinical decisions, and deviations from protocol, such as incorrect instrument handling, missteps in sequence, or safety violations within the original spatial context. To validate the tool, a user experiment evaluates the impact of 3D reconstructed scenes on perceived learning and self-assessment. Participants perform predefined tasks, e.g., instrument interaction, and qualitatively compare traditional debriefing with immersive replay, reporting on comprehension and perceived improvements. Results demonstrate that spatially grounded replay enhances perceived feedback precision and can improve training effectiveness compared to conventional methods. Cloud storage ensures scalability for asynchronous review, broadening adoption in resource-constrained settings.
Palavras-chave:
immersive analytics, virtual reality, human-computer interaction, medical training, medical debriefing
Referências
Li, S. Immersive technologies in health professions education: A bibliometric analysis. Computers & Education: X Reality, 4, 100051, 2024. Available: [link]
Levett-Jones, T. and Lapkin, S. A systematic review of the effectiveness of simulation debriefing in health professional education. Nurse Education Today, 34(6), e58–e63, 2014. Available: [link]
Savoldelli, G. L. Naik, V. N. Park, J. Joo, H. S. Chow, R. and Hamstra, S. J. Value of debriefing during simulated crisis management: oral versus video-assisted oral feedback. Anesthesiology, 105(2), 279–285, 2006.
Issenberg, S. B. Mcgaghie, W. C. Petrusa, E. R. Gordon, D. L. and Scalese, R. J. Features and uses of high-fidelity medical simulations that lead to effective learning: a beme systematic review. Medical Teacher, 27(1), 10–28, 2005.
Arafeh, J. M. R. Hansen, S. and Nichols, A. A. Debriefing in simulated-based learning: Facilitating a reflective discussion. The Journal of Perinatal & Neonatal Nursing, 24, 302–309, 2010.
Shinnick, M. A. Woo, M. Horwich, T. B. and Steadman, R. Debriefing: The most important component in simulation? Clinical Simulation in Nursing, 7(3), e105–e111, 2011. Available: [link]
Menin, A. Torchelsen, R. and Nedel, L. The effects of VR in training simulators: Exploring perception and knowledge gain. Computers & Graphics, 102, 402–412, 2022. Available: [link]
Dieckmann, P. Friis, S. M. Lippert, A. and Østergaard, D. The art and science of debriefing in simulation: Ideal and practice. Medical Teacher, 31(7), e287–e294, 2009.
Van Heukelom, J. N. Begaz, T. and Treat, R. Comparison of post-simulation debriefing versus in-simulation debriefing in medical simulation. Simulation in Healthcare, 5(2), 91–97, 2010.
Qu, Z. Lau, C. W. Simoff, S. J. Kennedy, P. J. Nguyen, Q. V. and Catchpoole, D. R. Review of innovative immersive technologies for healthcare applications. Innovations in Digital Health, Diagnostics, and Biomarkers, 2(2022), 27–39, 2022. DOI: 10.36401/IDDB-21-04
Negrao, M. D. Ferreira, W. Bohrer, B. Freitas, C. M. Maciel, A. and Nedel, L. Design and think-aloud study of an immersive interface for training health professionals in clinical skills. Proceedings of the 25th Symposium on Virtual and Augmented Reality (SVR ’23), ACM, 157–165, 2024. DOI: 10.1145/3625008.3625037
Sparks, D. Begum, R. Aqlan, F. Saleem, J. and S. D. M. Exploring team dynamics in virtual reality environments. IISE Transactions on Occupational Ergonomics and Human Factors, 2025. DOI: 10.1080/24725838.2025.2461481
Orland, M. D. Patetta, M. J. Wieser, M. Kayupov, E. and Gonzalez, M. H. Does virtual reality improve procedural completion and accuracy in an intramedullary tibial nail procedure? Clinical Orthopaedics and Related Research, 478(9), 2170–2177, 2020.
Sawyer, T. Eppich, W. Brett-Fleegler, M. Grant, V. and Cheng, A. More than one way to debrief: a critical review of healthcare simulation debriefing methods. Simulation in Healthcare, 11(3), 209–217, 2016.
Dine, C. J. Gersh, R. E. Leary, M. Riegel, B. J. Bellini, L. M. and Abella, B. S. Improving cardiopulmonary resuscitation quality and resuscitation training by combining audiovisual feedback and debriefing. Critical Care Medicine, 36, 2817–2822, 2008.
Minotti, K. Loup, G. Harquin, T. and Otmane, S. Exploring immersive debriefing in virtual reality training: A comparative study. Proceedings of the 30th ACM Symposium on Virtual Reality Software and Technology (VRST ’24), ACM, 2024. DOI: 10.1145/3641825.3687748
Minotti, K. Loup, G. Chellali, A. Ferrer, M.-H. and Otmane, S. Virtual reality learning optimization: Immersive debriefing with review and redo for effective, targeted training. 2025 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW), 1360–1361, 2025.
Degand, J. Loup, G. and Didier, J.-Y. Towards an immersive debriefing of serious games in virtual reality: A framework concept. Springer, 143–152, 2021.
Wolf, M. Teizer, J. Wolf, B. Bukru, S. and Solberg, A. Investigating hazard recognition in augmented virtuality for personalized feedback in construction safety education and training. Advanced Engineering Informatics, 51, 101469, 2022. Available: [link]
Krokos, E. Plaisant, C. and Varshney, A. Virtual memory palaces: immersion aids recall. Virtual Reality, 23(1), 1–15, 2019. DOI: 10.1007/s10055-018-0346-3
Cheng, A. Eppich, W. Grant, V. Sherbino, J. Zendejas, B. and Cook, D. A. Debriefing for technology-enhanced simulation: a systematic review and meta-analysis. Medical Education, 48(7), 657–666, 2014.
Kyaw, B. M. Saxena, N. Posadzki, P. Vseteckova, J. Nikolaou, C. K. George, P. P. Divakar, U. Masiello, I. Kononowicz, A. A. Zary, N. and Car, L. T. Virtual reality for health professions education: Systematic review and meta-analysis. Journal of Medical Internet Research, 21(1), e12959, 2019.
Howie, S. R. and Gilardi, M. Virtual observation of virtual reality simulations. Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems, 1–6, 2019. DOI: 10.1145/3290607.3312836
Nicholson, J. Gillespie, R. Bickerdike, S. Frith, G. and Hassan, T. Immersive video for simulation debriefing: record and review in 360-degrees using a virtual reality headset. 5, A9.1–A9, 2019.
Dudas, K. and Wheeler, J. Faculty re-enactment videos as a tool in simulation debriefing. Clinical Simulation in Nursing, 40, 25–30, 2020. Available: [link]
Cheng, A. Palaganas, J. C. Eppich, W. J. Rudolph, J. W. Robinson, T. and Grant, V. J. Co-debriefing for simulation-based education: a primer for facilitators. Simulation in Healthcare, 10(2), 69–75, 2015.
Elendu, C. Amaechi, D. C. Okatta, A. U. Amaechi, E. C. Elendu, T. C. Ezeh, C. P. and Elendu, I. D. The impact of simulation-based training in medical education: A review. Medicine, 103(27), e38813, 2024.
Motola, I. Devine, L. A. Chung, H. S. Sullivan, J. E. and Issenberg, S. B. Simulation in healthcare education: A best evidence practical guide. Medical Teacher, 35(10), e1511–e1530, 2013.
Adolph, K. and Kretch, K. Gibson’s theory of perceptual learning. International Encyclopedia of the Social & Behavioral Sciences, 10, 2015.
Mine, M. R. Brooks, F. P. and Sequin, C. H. Moving objects in space: exploiting proprioception in virtual-environment interaction. Proceedings of the 24th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH ’97), 19–26, 1997. DOI: 10.1145/258734.258747
Bartz-Kurycki, M. A. Anderson, K. T. Abraham, J. E. Masada, K. M. Wang, J. Kawaguchi, A. L. Lally, K. P. and Tsao, K. Debriefing: the forgotten phase of the surgical safety checklist. Journal of Surgical Research, 213, 222–227, 2017. Available: [link]
Brindle, M. Henrich, N. Foster, A. Marks, S. Rose, M. Welsh, R. and Berry, W. Implementation of surgical debriefing programs in large health systems: An exploratory qualitative analysis. BMC Health Services Research, 18, 210, 2018.
Seelandt, J. C. Walker, K. and Kolbe, M. A debriefer must be neutral and other debriefing myths: a systemic inquiry-based qualitative study. Advances in Simulation, 6, 1–15, 2021.
Wagner, J. Stuerzlinger, W. and Nedel, L. The effect of exploration mode and frame of reference in immersive analytics. IEEE Transactions on Visualization and Computer Graphics, 28(9), 3252–3264, 2022.
Lopez, T. Dumas, O. Danieau, F. Leroy, B. Mollet, N. and Vial, J.-F. A playback tool for reviewing VR experiences. Proceedings of the 23rd ACM Symposium on Virtual Reality Software and Technology (VRST ’17), 2017. DOI: 10.1145/3139131.3141776
Javerliat, C. Villenave, S. Raimbaud, P. and Lavoue, G. Plume: Record, replay, analyze and share user behavior in 6dof XR experiences. IEEE Transactions on Visualization and Computer Graphics, 30(5), 2087–2097, 2024.
Troncon, L. E. A. Clinical skills assessment: limitations to the introduction of an OSCE in a traditional Brazilian medical school. Sao Paulo Medical Journal, 122, 12–17, 2004.
Fernandez-Espinosa, M. Clouse, K. Sellars, D. Tong, D. Bsales, M. Alcindor, S. Hubbard, T. D. Villano, M. and Gomez-Zara, D. Breaking the familiarity bias: Employing virtual reality environments to enhance team formation and inclusion. Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems (CHI ’25), ACM, 2025. DOI: 10.1145/3706598.3714127
Jiang, H. Vimalesvaran, S. Wang, J. K. Lim, K. B. Mogali, S. R. and Car, L. T. Virtual reality in medical students’ education: Scoping review. JMIR Medical Education, 8(1), e34860, 2022. DOI: 10.2196/34860
Bracq, M.-S. Michinov, E. Arnaldi, B. Caillaud, B. Gibaud, B. Gouranton, V. and Jannin, P. Learning procedural skills with a virtual reality simulator: An acceptability study. Nurse Education Today, 79, 153–160, 2019.
Fealy, S. Jones, D. Hutton, A. Graham, K. McNeill, L. Sweet, L. and Hazelton, M. The integration of immersive virtual reality in tertiary nursing and midwifery education: A scoping review. Nurse Education Today, 79, 14–19, 2019.
Mekbib, D. B. Han, J. Zhang, L. Fang, S. Jiang, H. Zhu, J. Roe, A. W. and Xu, D. Virtual reality therapy for upper limb rehabilitation in patients with stroke: a meta-analysis of randomized clinical trials. Brain Injury, 34(4), 456–465, 2020.
Brett-Fleegler, M. Rudolph, J. Eppich, W. Monuteaux, M. Fleegler, E. Cheng, A. and Simon, R. Debriefing assessment for simulation in healthcare: development and psychometric properties. Simulation in Healthcare, 7(5), 288–294, 2012.
Levett-Jones, T. and Lapkin, S. A systematic review of the effectiveness of simulation debriefing in health professional education. Nurse Education Today, 34(6), e58–e63, 2014. Available: [link]
Savoldelli, G. L. Naik, V. N. Park, J. Joo, H. S. Chow, R. and Hamstra, S. J. Value of debriefing during simulated crisis management: oral versus video-assisted oral feedback. Anesthesiology, 105(2), 279–285, 2006.
Issenberg, S. B. Mcgaghie, W. C. Petrusa, E. R. Gordon, D. L. and Scalese, R. J. Features and uses of high-fidelity medical simulations that lead to effective learning: a beme systematic review. Medical Teacher, 27(1), 10–28, 2005.
Arafeh, J. M. R. Hansen, S. and Nichols, A. A. Debriefing in simulated-based learning: Facilitating a reflective discussion. The Journal of Perinatal & Neonatal Nursing, 24, 302–309, 2010.
Shinnick, M. A. Woo, M. Horwich, T. B. and Steadman, R. Debriefing: The most important component in simulation? Clinical Simulation in Nursing, 7(3), e105–e111, 2011. Available: [link]
Menin, A. Torchelsen, R. and Nedel, L. The effects of VR in training simulators: Exploring perception and knowledge gain. Computers & Graphics, 102, 402–412, 2022. Available: [link]
Dieckmann, P. Friis, S. M. Lippert, A. and Østergaard, D. The art and science of debriefing in simulation: Ideal and practice. Medical Teacher, 31(7), e287–e294, 2009.
Van Heukelom, J. N. Begaz, T. and Treat, R. Comparison of post-simulation debriefing versus in-simulation debriefing in medical simulation. Simulation in Healthcare, 5(2), 91–97, 2010.
Qu, Z. Lau, C. W. Simoff, S. J. Kennedy, P. J. Nguyen, Q. V. and Catchpoole, D. R. Review of innovative immersive technologies for healthcare applications. Innovations in Digital Health, Diagnostics, and Biomarkers, 2(2022), 27–39, 2022. DOI: 10.36401/IDDB-21-04
Negrao, M. D. Ferreira, W. Bohrer, B. Freitas, C. M. Maciel, A. and Nedel, L. Design and think-aloud study of an immersive interface for training health professionals in clinical skills. Proceedings of the 25th Symposium on Virtual and Augmented Reality (SVR ’23), ACM, 157–165, 2024. DOI: 10.1145/3625008.3625037
Sparks, D. Begum, R. Aqlan, F. Saleem, J. and S. D. M. Exploring team dynamics in virtual reality environments. IISE Transactions on Occupational Ergonomics and Human Factors, 2025. DOI: 10.1080/24725838.2025.2461481
Orland, M. D. Patetta, M. J. Wieser, M. Kayupov, E. and Gonzalez, M. H. Does virtual reality improve procedural completion and accuracy in an intramedullary tibial nail procedure? Clinical Orthopaedics and Related Research, 478(9), 2170–2177, 2020.
Sawyer, T. Eppich, W. Brett-Fleegler, M. Grant, V. and Cheng, A. More than one way to debrief: a critical review of healthcare simulation debriefing methods. Simulation in Healthcare, 11(3), 209–217, 2016.
Dine, C. J. Gersh, R. E. Leary, M. Riegel, B. J. Bellini, L. M. and Abella, B. S. Improving cardiopulmonary resuscitation quality and resuscitation training by combining audiovisual feedback and debriefing. Critical Care Medicine, 36, 2817–2822, 2008.
Minotti, K. Loup, G. Harquin, T. and Otmane, S. Exploring immersive debriefing in virtual reality training: A comparative study. Proceedings of the 30th ACM Symposium on Virtual Reality Software and Technology (VRST ’24), ACM, 2024. DOI: 10.1145/3641825.3687748
Minotti, K. Loup, G. Chellali, A. Ferrer, M.-H. and Otmane, S. Virtual reality learning optimization: Immersive debriefing with review and redo for effective, targeted training. 2025 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW), 1360–1361, 2025.
Degand, J. Loup, G. and Didier, J.-Y. Towards an immersive debriefing of serious games in virtual reality: A framework concept. Springer, 143–152, 2021.
Wolf, M. Teizer, J. Wolf, B. Bukru, S. and Solberg, A. Investigating hazard recognition in augmented virtuality for personalized feedback in construction safety education and training. Advanced Engineering Informatics, 51, 101469, 2022. Available: [link]
Krokos, E. Plaisant, C. and Varshney, A. Virtual memory palaces: immersion aids recall. Virtual Reality, 23(1), 1–15, 2019. DOI: 10.1007/s10055-018-0346-3
Cheng, A. Eppich, W. Grant, V. Sherbino, J. Zendejas, B. and Cook, D. A. Debriefing for technology-enhanced simulation: a systematic review and meta-analysis. Medical Education, 48(7), 657–666, 2014.
Kyaw, B. M. Saxena, N. Posadzki, P. Vseteckova, J. Nikolaou, C. K. George, P. P. Divakar, U. Masiello, I. Kononowicz, A. A. Zary, N. and Car, L. T. Virtual reality for health professions education: Systematic review and meta-analysis. Journal of Medical Internet Research, 21(1), e12959, 2019.
Howie, S. R. and Gilardi, M. Virtual observation of virtual reality simulations. Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems, 1–6, 2019. DOI: 10.1145/3290607.3312836
Nicholson, J. Gillespie, R. Bickerdike, S. Frith, G. and Hassan, T. Immersive video for simulation debriefing: record and review in 360-degrees using a virtual reality headset. 5, A9.1–A9, 2019.
Dudas, K. and Wheeler, J. Faculty re-enactment videos as a tool in simulation debriefing. Clinical Simulation in Nursing, 40, 25–30, 2020. Available: [link]
Cheng, A. Palaganas, J. C. Eppich, W. J. Rudolph, J. W. Robinson, T. and Grant, V. J. Co-debriefing for simulation-based education: a primer for facilitators. Simulation in Healthcare, 10(2), 69–75, 2015.
Elendu, C. Amaechi, D. C. Okatta, A. U. Amaechi, E. C. Elendu, T. C. Ezeh, C. P. and Elendu, I. D. The impact of simulation-based training in medical education: A review. Medicine, 103(27), e38813, 2024.
Motola, I. Devine, L. A. Chung, H. S. Sullivan, J. E. and Issenberg, S. B. Simulation in healthcare education: A best evidence practical guide. Medical Teacher, 35(10), e1511–e1530, 2013.
Adolph, K. and Kretch, K. Gibson’s theory of perceptual learning. International Encyclopedia of the Social & Behavioral Sciences, 10, 2015.
Mine, M. R. Brooks, F. P. and Sequin, C. H. Moving objects in space: exploiting proprioception in virtual-environment interaction. Proceedings of the 24th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH ’97), 19–26, 1997. DOI: 10.1145/258734.258747
Bartz-Kurycki, M. A. Anderson, K. T. Abraham, J. E. Masada, K. M. Wang, J. Kawaguchi, A. L. Lally, K. P. and Tsao, K. Debriefing: the forgotten phase of the surgical safety checklist. Journal of Surgical Research, 213, 222–227, 2017. Available: [link]
Brindle, M. Henrich, N. Foster, A. Marks, S. Rose, M. Welsh, R. and Berry, W. Implementation of surgical debriefing programs in large health systems: An exploratory qualitative analysis. BMC Health Services Research, 18, 210, 2018.
Seelandt, J. C. Walker, K. and Kolbe, M. A debriefer must be neutral and other debriefing myths: a systemic inquiry-based qualitative study. Advances in Simulation, 6, 1–15, 2021.
Wagner, J. Stuerzlinger, W. and Nedel, L. The effect of exploration mode and frame of reference in immersive analytics. IEEE Transactions on Visualization and Computer Graphics, 28(9), 3252–3264, 2022.
Lopez, T. Dumas, O. Danieau, F. Leroy, B. Mollet, N. and Vial, J.-F. A playback tool for reviewing VR experiences. Proceedings of the 23rd ACM Symposium on Virtual Reality Software and Technology (VRST ’17), 2017. DOI: 10.1145/3139131.3141776
Javerliat, C. Villenave, S. Raimbaud, P. and Lavoue, G. Plume: Record, replay, analyze and share user behavior in 6dof XR experiences. IEEE Transactions on Visualization and Computer Graphics, 30(5), 2087–2097, 2024.
Troncon, L. E. A. Clinical skills assessment: limitations to the introduction of an OSCE in a traditional Brazilian medical school. Sao Paulo Medical Journal, 122, 12–17, 2004.
Fernandez-Espinosa, M. Clouse, K. Sellars, D. Tong, D. Bsales, M. Alcindor, S. Hubbard, T. D. Villano, M. and Gomez-Zara, D. Breaking the familiarity bias: Employing virtual reality environments to enhance team formation and inclusion. Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems (CHI ’25), ACM, 2025. DOI: 10.1145/3706598.3714127
Jiang, H. Vimalesvaran, S. Wang, J. K. Lim, K. B. Mogali, S. R. and Car, L. T. Virtual reality in medical students’ education: Scoping review. JMIR Medical Education, 8(1), e34860, 2022. DOI: 10.2196/34860
Bracq, M.-S. Michinov, E. Arnaldi, B. Caillaud, B. Gibaud, B. Gouranton, V. and Jannin, P. Learning procedural skills with a virtual reality simulator: An acceptability study. Nurse Education Today, 79, 153–160, 2019.
Fealy, S. Jones, D. Hutton, A. Graham, K. McNeill, L. Sweet, L. and Hazelton, M. The integration of immersive virtual reality in tertiary nursing and midwifery education: A scoping review. Nurse Education Today, 79, 14–19, 2019.
Mekbib, D. B. Han, J. Zhang, L. Fang, S. Jiang, H. Zhu, J. Roe, A. W. and Xu, D. Virtual reality therapy for upper limb rehabilitation in patients with stroke: a meta-analysis of randomized clinical trials. Brain Injury, 34(4), 456–465, 2020.
Brett-Fleegler, M. Rudolph, J. Eppich, W. Monuteaux, M. Fleegler, E. Cheng, A. and Simon, R. Debriefing assessment for simulation in healthcare: development and psychometric properties. Simulation in Healthcare, 7(5), 288–294, 2012.
Publicado
30/09/2025
Como Citar
GUTHS, Leonardo S.; NEGRÃO, Matheus D.; FREITAS, Carla M. D. S.; MACIEL, Anderson; TORCHELSEN, Rafael; NEDEL, Luciana.
Debriefing Tool for Immersive Analytics in Medical Training Simulations. In: SIMPÓSIO DE REALIDADE VIRTUAL E AUMENTADA (SVR), 27. , 2025, Salvador/BA.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2025
.
p. 588-597.
