Development of an innovative mixed reality methodology for teaching medical students complex pathologies

• Ruan Toledano UFSC https://orcid.org/0009-0000-9981-4570
• Cecília Giordano UFSC http://orcid.org/0009-0003-0491-7940
• Eliane Pozzebon UFSC https://orcid.org/0000-0002-4237-6589
• Josete Manzon UFSC https://orcid.org/0000-0002-5800-1351

Resumo

The inclusion of mixed reality (MR) and other types of immersion technology into medical education has emerged as a transformative and innovative approach for teaching complex pathological concepts. This paper presents the methodological framework for developing an MR-based educational tool designed to enhance disease understanding through interactive three-dimensional models. The development process included bibliographic research, 3D modeling using Blender® and Substance 3D Painter®, animation techniques, and integration via Unity® onto mixed reality glasses. Preliminary classroom studies demonstrated comparable knowledge retention with improved spatial reasoning and engagement when using the proposed technology. Both groups showed significant learning gains after the intervention (p < 0.001); the control group improved from 52% to 69%, and the test group from 61% to 71%, with the latter achieving higher final scores and reduced score dispersion. The methodology also addresses key barriers to scalable MR implementation in education, providing a replicable approach for academic institutions. Furthermore, this study introduces a structured development pipeline that addresses technological challenges in MR content creation, such as anatomical fidelity, intuitive interaction, performance optimization, and collaborative use. The contribution extends beyond medical education, offering a replicable MR deployment framework relevant to the MR research communities.

Palavras-chave: mixed reality, medical education, pathology teaching, 3D modeling, mixed reality glasses

Referências

R. Andersson, M. Anker, A. Dunford, J. Lundqvist, and A. Weiss. Hamlin an augmented reality solution to visualise abstract concepts for science education. Proc. SIDeR16 Student Interaction Design Research Conference, 2016.

R. Tori, C. Kirner, and R. Siscouto. Fundamentals and Technology of Virtual Reality. Brazilian Computer Society (SBC), Porto Alegre, 2006.

C. Moro, Z. Stromberga, A. Raikos, and A. Stirling. The effectiveness of virtual and augmented reality in health sciences and medical anatomy. Anatomical Sciences Education, 10:549–559, 2017.

C. Stromberga, C. Phelps, J. Smith, and C. Moro. Teaching with disruptive technology the use of augmented, virtual, and mixed reality (HoloLens) for disease education. Biomedical Visualisation, P. M. Rea, Ed., vol. 1317, 105–115, Springer, Cham, 2021. DOI: 10.1007/978-3-030-61125-5_8.

C. Ekstrand et al. Immersive and interactive virtual reality to enhance learning and retention of neuroanatomy in medical students a randomized controlled study. CMAJ Open, 6(1):E103–E109, 2018. DOI: 10.9778/cmajo.20170110.

D. A. Kolb. Experiential Learning Experience as the Source of Learning and Development. Prentice-Hall, 1984.

J. Sweller. Cognitive load theory. Psychology of Learning and Motivation, 55:37–76, 2011. DOI: 10.1016/B978-0-12-387691-1.00002-8.

I. Bianchi, A. L. Zanatta, and R. Rieder. Augmented reality in the medical teaching-learning process a systematic review. Proc. Symposium on Virtual and Augmented Reality (SVR), 22nd ed., Porto de Galinhas, Brazil, 2020, 1–33. DOI: 10.1109/SVR51698.2020.00032.

G. Biglino et al. Piloting the use of patient-specific cardiac models as a novel tool to facilitate communication during clinical consultations. Pediatric Cardiology, 38(4):813–818, 2017.

F. Incekara, M. Smits, C. Dirven, and A. Vincent. Clinical feasibility of a wearable mixed reality device in neurosurgery. World Neurosurgery, 118:1422–1427, 2018.

M. K. Southworth, J. R. Silva, and J. N. A. Silva. Use of extended realities in cardiology. Trends in Cardiovascular Medicine, 30(3):143–148, 2020. DOI: 10.1016/j.tcm.2019.04.005.

M. McBain, L. Panerai, G. F. T. Silva, R. G. R. Lima, and C. E. S. Pires. A scoping review on augmented reality applications for anatomy learning. Proc. XXI Symposium on Virtual and Augmented Reality (SVR), Porto de Galinhas, Brazil, 2022, 120–129. DOI: 10.1109/SVR54801.2022.00023.

G. R. Teixeira et al. Analysis of serious games with immersive technologies in health training a systematic review of Brazilian studies. Proc. XIX Symposium on Virtual and Augmented Reality (SVR), Online, Brazil, 2020, 1–8. DOI: 10.1109/SVR51698.2020.00032.

J. L. Renner et al. Students experiences of MR-based bedside teaching with HoloLens 2 a mixed-methods study. BMC Medical Education, 24(150), 2024. Available [link].

J. P. Rees et al. Longitudinal implementation of immersive XR simulation training in medical education effects on learning outcomes and engagement. BMC Medical Education, 24(235), 2024. Available [link].

S. Zechner, T. Hann et al. Mixed reality-based tactical training with real-time performance feedback for emergency medicine. JMIR XR, 2(1):e57655, 2024. DOI: 10.2196/57655.

M. Slater, M. van Gelder, and H. Muller. Volumetric capture for spatially immersive medical training a user study. arXiv preprint arXiv:2312.09382, Dec. 2023. Available [link].

J. Ekins. What is STATdx South African Journal of Radiology, 11, 2007. DOI: 10.4102/sajr.v11i4.28.

K. Schwaber and J. Sutherland. The Scrum Guide The Rules of the Game. Available [link], 2020. Accessed Feb. 22, 2025.

A. F. D’Aiello et al. The effect of holographic heart models and mixed reality for anatomy learning in congenital heart disease an exploratory study. Journal of Medical Systems, 47(64), 2023. DOI: 10.1007/s10916-023-01959-8.

R. Likert. A technique for the measurement of attitudes. Archives of Psychology, 140:1–55, 1932.

J. A. Frank and V. Kapila. Mixed-reality learning environments integrating mobile interfaces with laboratory test-beds. Computers & Education, 110:88–104, 2017. DOI: 10.1016/j.compedu.2017.02.009.

R. Kolecki et al. Assessment of the utility of mixed reality in medical education. Translational Research in Anatomy, 28:100214, 2022. DOI: 10.1016/j.tria.2022.100214.

T. A. Gormley et al. Effect of mixed reality support on clinical errors, team performance, and workload during emergency simulations. JAMA Network Open, 6(8):e2328474, 2023. DOI: 10.1001/jamanetworkopen.2023.28474.

M. B. P. C. Bonde et al. Use of mixed reality to enhance anatomy teaching in remote learning contexts HoloAnatomy during the COVID-19 pandemic. JAMA Network Open, 3(10):e2022606, 2020. DOI: 10.1001/jamanetworkopen.2020.22606.

Y. M. Tang et al. A systematic review of immersive technology applications for medical practice and education trends, application areas, recipients, teaching contents, evaluation methods, and performance. Educational Research Review, 35:100429, 2022. DOI: 10.1016/j.edurev.2021.100429.

J. Sweller. Cognitive load theory and educational technology. Educational Technology Research and Development, 66(1):283–290, 2018. DOI: 10.1007/s11423-017-9558-z.

L. Sultan et al. An experimental study on usefulness of virtual reality 360 in undergraduate medical education. Advances in Medical Education and Practice, 10:907–916, 2019. DOI: 10.2147/AMEP.S219344.

K. Wilson and C. Peterson. Measuring long-term knowledge retention in medical education a review of current practices and future directions. Medical Teacher, 43(1):25–31, 2021. DOI: 10.1080/0142159X.2020.1798912.

M. Lee, J. Kim, and S. Park. Impact of individual differences in technology readiness on learning outcomes in mixed reality environments. Computers & Education, 185:104528, 2022. DOI: 10.1016/j.compedu.2022.104528.