Modeling and Designing Clinical Sessions with Hybrid Environments in the Metaverse
Resumo
The Metaverse, an immersive virtual environment enabling realtime interactions, presents a novel opportunity to enhance workflows in healthcare, particularly in clinical sessions. The necessity for immersive, collaborative, and device-friendly environments has become increasingly apparent in the age of remote work and the rapid advancement of XR technologies. This paper explored how to model and design clinical sessions using hybrid environments in the Metaverse. We considered users’ needs across diverse devices like HMDs, notebooks, and smartphones and integrated interactions available in the Metaverse. The study began by outlining the existing protocol followed by healthcare professionals during clinical sessions. Our research identified specific tasks within the clinical session process that could be improved by integrating Metaverse interactions. Based on these findings, we proposed and developed a prototype, which was evaluated by HCI, XR experts and stakeholders. Our findings provide valuable insights for healthcare professionals and Metaverse designers, pointing directions for future research.
Palavras-chave:
Metaverse, medical dataset, hybrid environments, BPMN, thinking aloud
Referências
2015. Technology Readiness Level. [link].
2023. Custom Prefab Objects | Spatial Creator Toolkit. [link]
2023. Empty Frame. [link]
Rosalinda Alfaro-Lefevre. 2016. Critical thinking, clinical reasoning and clinical judgment: A practical approach, pageburst E-book on kno. Elsevier Health Sciences.
Aaron Bangor, Philip Kortum, and James Miller. 2009. Determining what individual SUS scores mean: Adding an adjective rating scale. Journal of usability studies 4, 3 (2009), 114–123.
Douglas Bates, Martin Mächler, Benjamin M. Bolker, and Steven C. Walker. 2015. Fitting linear mixed-effects models using lme4. Journal of Statistical Software 67, 1 (2015). https://doi.org/10.18637/jss.v067.i01 Cited by: 37248; All Open Access, Gold Open Access, Green Open Access.
Krishna Bharadwaj and Andrew E Johnson. 2021. A Hybrid Collaborative Virtual Environment with Heterogeneous Representations for Architectural Planning. (2021), 184–199.
John Brooke. 2013. SUS: a retrospective. Journal of usability studies 8, 2 (2013), 29–40.
Donghua Chen and Runtong Zhang. 2022. Exploring research trends of emerging technologies in health metaverse: A bibliometric analysis. Available at SSRN 3998068 (05 Jan 2022). https://doi.org/10.2139/ssrn.3998068
Ruizhi Cheng, Nan Wu, Songqing Chen, and Bo Han. 2022. Will metaverse be nextg internet? vision, hype, and reality. IEEE Network 36, 5 (2022), 197–204.
Ruizhi Cheng, Nan Wu, Matteo Varvello, Songqing Chen, and Bo Han. 2022. Are We Ready for Metaverse? A Measurement Study of Social Virtual Reality Platforms. In Proceedings of the 22nd ACM Internet Measurement Conference. Association for Computing Machinery, New York, NY, USA, 504–518.
Michelene TH Chi. 1997. Quantifying qualitative analyses of verbal data: A practical guide. The journal of the learning sciences 6, 3 (1997), 271–315.
Kyle Couperus, Scott Young, Ryan Walsh, Christopher Kang, Carl Skinner, Robyn Essendrop, Kristin Fiala, Jillian F Phelps, Zachary Sletten, Matthew T Esposito, et al. 2020. Immersive virtual reality medical simulation: autonomous trauma training simulator. Cureus 12, 5 (2020).
Barbara Czarniawska. 2007. Shadowing: and other techniques for doing fieldwork in modern societies. Copenhagen Business School Press DK.
Alan Dix, Janet Finlay, Gregory D Abowd, and Russell Beale. 2003. Humancomputer interaction. Pearson Education.
Youcef Djenouri, Asma Belhadi, Gautam Srivastava, and Jerry Chun-Wei Lin. 2021. Secure collaborative augmented reality framework for biomedical informatics. IEEE Journal of Biomedical and Health Informatics 26, 6 (2021), 2417–2424.
Helen Forbes, Tracey K. Bucknall, and Alison M. Hutchinson. 2016. Piloting the feasibility of head-mounted video technology to augment student feedback during simulated clinical decision-making: An observational design pilot study. Nurse Education Today 39 (2016), 116–121. https://doi.org/10.1016/j.nedt.2016.01.012
Alba García-Viola, José Miguel Garrido-Molina, Verónica V Márquez-Hernández, Genoveva Granados-Gámez, Gabriel Aguilera-Manrique, and Lorena GutiérrezPuertas. 2019. The influence of gamification on decision making in nursing students. Journal of Nursing Education 58, 12 (2019), 718–722.
Wenzel Jakob, Marco Tarini, Daniele Panozzo, Olga Sorkine-Hornung, et al. 2015. Instant field-aligned meshes. ACM Trans. Graph. 34, 6 (2015), 189–1.
Carley Jans, Fiona Bogossian, Patrea Andersen, and Tracy Levett-Jones. 2023. Examining the impact of virtual reality on clinical decision making–An integrative review. Nurse Education Today (2023), 105767.
Yuen C Law, Wilken Wehrt, Sabine Sonnentag, and Benjamin Weyers. 2023. Obtaining semi-formal models from qualitative data: From interviews into bpmn models in user-centered design processes. International Journal of Human–Computer Interaction 39, 3 (2023), 476–493.
Clayton Lewis and Cathleen Wharton. 1997. Cognitive walkthroughs. In Handbook of human-computer interaction. Elsevier, 717–732.
Tahir Mahmood, Willis Fulmer, Neelesh Mungoli, Jian Huang, and Aidong Lu. 2019. Improving information sharing and collaborative analysis for remote geospatial visualization using mixed reality. In 2019 IEEE International Symposium on Mixed and Augmented Reality (ISMAR). IEEE, 236–247.
Tara Matthews, Steve Whittaker, Thomas Moran, and Sandra Yuen. 2011. Collaboration personas: A new approach to designing workplace collaboration tools. In Proceedings of the SIGCHI conference on human factors in computing systems.
Jacqueline McCallum, Valerie Ness, and Theresa Price. 2011. Exploring nursing students’ decision-making skills whilst in a Second Life clinical simulation laboratory. Nurse Education Today 31, 7 (2011), 699–704.
Jillian L McGrath, Jeffrey M Taekman, Parvati Dev, Douglas R Danforth, Deepika Mohan, Nicholas Kman, Amanda Crichlow, William F Bond, Shiela Riker, AJ Lemheney, et al. 2018. Using virtual reality simulation environments to assess competence for emergency medicine learners. Academic Emergency Medicine 25, 2 (2018), 186–195.
Yanni Mei, Jie Li, Huib De Ridder, and Pablo Cesar. 2021. Cakevr: A social virtual reality (vr) tool for co-designing cakes. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–14.
Radu Papara, Ramona Galatus, and Loredana Buzura. 2020. Virtual reality as cost effective tool for distance healthcare. In 2020 22nd International conference on transparent optical networks (ICTON). IEEE, 1–6.
Hyanghee Park, Daehwan Ahn, and Joonhwan Lee. 2023. Towards a Metaverse Workspace: Opportunities, Challenges, and Design Implications. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–20.
Monica Peddle. 2015. Virtual simulation developing non-technical skills in student nurses and midwives. Australian Nursing and Midwifery Journal 23, 1 (2015), 41.
Catlin Pidel and Philipp Ackermann. 2020. Collaboration in virtual and augmented reality: a systematic overview. In Augmented Reality, Virtual Reality, and Computer Graphics: 7th International Conference, AVR 2020, Lecce, Italy, September 7–10, 2020, Proceedings, Part I 7. Springer, 141–156.
Csaba Pinter, Andras Lasso, Saleh Choueib, Mark Asselin, Jean-Christophe FillionRobin, Jean-Baptiste Vimort, Ken Martin, Matthew A Jolley, and Gabor Fichtinger. 2020. SlicerVR for medical intervention training and planning in immersive virtual reality. IEEE transactions on medical robotics and bionics 2, 2 (2020).
Abhinesh Prabhakaran, Abdul-Majeed Mahamadu, Lamine Mahdjoubi, and Pawel Boguslawski. 2022. BIM-based immersive collaborative environment for furniture, fixture and equipment design. Automation in Construction 142 (2022), 104489.
Sandy Q Qu and John Dumay. 2011. The qualitative research interview. Qualitative research in accounting & management 8, 3 (2011), 238–264.
John Rieman, Marita Franzke, and David Redmiles. 1995. Usability Evaluation with the Cognitive Walkthrough. In Conference Companion on Human Factors in Computing Systems (Denver, Colorado, USA) (CHI ’95). Association for Computing Machinery, New York, NY, USA, 387–388. https://doi.org/10.1145/223355.223735
Victor Romero, Romain Pinquié, and Frédéric Noël. 2021. An immersive virtual environment for reviewing model-centric designs. Proceedings of the Design Society 1 (2021), 447–456.
Mattias Roupé, Mikael Johansson, Laura Maftei, Rikard Lundstedt, and Mikael Viklund-Tallgren. 2020. Virtual collaborative design environment: supporting seamless integration of multitouch table and immersive VR. Journal of Construction Engineering and Management 146, 12 (2020), 04020132.
Sharad Sharma. 2020. Improving emergency response training and decision making using a collaborative virtual reality environment for building evacuation. In HCI International 2020–Late Breaking Papers: Virtual and Augmented Reality: 22nd HCI International Conference, HCII 2020, Copenhagen, Denmark, July 19–24, 2020, Proceedings 22. Springer, 213–224.
Jae-Eun Shin, Boram Yoon, Dooyoung Kim, Hyung-Il Kim, and Woontack Woo. 2022. The Effects of Device and Spatial Layout on Social Presence During a Dynamic Remote Collaboration Task in Mixed Reality. In 2022 IEEE International Symposium on Mixed and Augmented Reality (ISMAR). IEEE, 394–403.
Neal Stephenson. 2003. Snow crash: A novel. Spectra.
David R Thomas. 2006. A general inductive approach for analyzing qualitative evaluation data. American journal of evaluation 27, 2 (2006), 237–246.
Arnold POS Vermeeren, Effie Lai-Chong Law, Virpi Roto, Marianna Obrist, Jettie Hoonhout, and Kaisa Väänänen-Vainio-Mattila. 2010. User experience evaluation methods: current state and development needs. In Proceedings of the 6th Nordic conference on human-computer interaction: Extending boundaries. 521–530.
Jacob O. Wobbrock, Leah Findlater, Darren Gergle, and James J. Higgins. 2011. The Aligned Rank Transform for nonparametric factorial analyses using only ANOVA procedures. Conference on Human Factors in Computing Systems - Proceedings (2011), 143 – 146. https://doi.org/10.1145/1978942.1978963 Cited by: 1112; All Open Access, Green Open Access.
Hemendra Worlikar, Sean Coleman, Jack Kelly, Sadhbh O’Connor, Aoife Murray, Terri McVeigh, Jennifer Doran, Ian McCabe, and Derek O’Keeffe. 2023. Mixed Reality Platforms in Telehealth Delivery: Scoping Review. JMIR Biomed Eng 8 (24 Mar 2023), e42709. https://doi.org/10.2196/42709
Hemendra Worlikar, Sean Coleman, Jack Kelly, Sadhbh O’Connor, Aoife Murray, Terri McVeigh, Jennifer Doran, Ian McCabe, and Derek O’Keeffe. 2023. Mixed Reality Platforms in Telehealth Delivery: Scoping Review. JMIR Biomedical Engineering 8 (2023), e42709.
Dawei Yang, Jian Zhou, Rongchang Chen, Yuanlin Song, Zhenju Song, Xiaoju Zhang, Qi Wang, Kai Wang, Chengzhi Zhou, Jiayuan Sun, Lichuan Zhang, Li Bai, Yuehong Wang, Xu Wang, Yeting Lu, Hongyi Xin, Charles A. Powell, Christoph Thüemmler, Niels H. Chavannes, Wei Chen, Lian Wu, and Chunxue Bai. 2022. Expert consensus on the metaverse in medicine. Clinical eHealth 5 (2022), 1–9.
2023. Custom Prefab Objects | Spatial Creator Toolkit. [link]
2023. Empty Frame. [link]
Rosalinda Alfaro-Lefevre. 2016. Critical thinking, clinical reasoning and clinical judgment: A practical approach, pageburst E-book on kno. Elsevier Health Sciences.
Aaron Bangor, Philip Kortum, and James Miller. 2009. Determining what individual SUS scores mean: Adding an adjective rating scale. Journal of usability studies 4, 3 (2009), 114–123.
Douglas Bates, Martin Mächler, Benjamin M. Bolker, and Steven C. Walker. 2015. Fitting linear mixed-effects models using lme4. Journal of Statistical Software 67, 1 (2015). https://doi.org/10.18637/jss.v067.i01 Cited by: 37248; All Open Access, Gold Open Access, Green Open Access.
Krishna Bharadwaj and Andrew E Johnson. 2021. A Hybrid Collaborative Virtual Environment with Heterogeneous Representations for Architectural Planning. (2021), 184–199.
John Brooke. 2013. SUS: a retrospective. Journal of usability studies 8, 2 (2013), 29–40.
Donghua Chen and Runtong Zhang. 2022. Exploring research trends of emerging technologies in health metaverse: A bibliometric analysis. Available at SSRN 3998068 (05 Jan 2022). https://doi.org/10.2139/ssrn.3998068
Ruizhi Cheng, Nan Wu, Songqing Chen, and Bo Han. 2022. Will metaverse be nextg internet? vision, hype, and reality. IEEE Network 36, 5 (2022), 197–204.
Ruizhi Cheng, Nan Wu, Matteo Varvello, Songqing Chen, and Bo Han. 2022. Are We Ready for Metaverse? A Measurement Study of Social Virtual Reality Platforms. In Proceedings of the 22nd ACM Internet Measurement Conference. Association for Computing Machinery, New York, NY, USA, 504–518.
Michelene TH Chi. 1997. Quantifying qualitative analyses of verbal data: A practical guide. The journal of the learning sciences 6, 3 (1997), 271–315.
Kyle Couperus, Scott Young, Ryan Walsh, Christopher Kang, Carl Skinner, Robyn Essendrop, Kristin Fiala, Jillian F Phelps, Zachary Sletten, Matthew T Esposito, et al. 2020. Immersive virtual reality medical simulation: autonomous trauma training simulator. Cureus 12, 5 (2020).
Barbara Czarniawska. 2007. Shadowing: and other techniques for doing fieldwork in modern societies. Copenhagen Business School Press DK.
Alan Dix, Janet Finlay, Gregory D Abowd, and Russell Beale. 2003. Humancomputer interaction. Pearson Education.
Youcef Djenouri, Asma Belhadi, Gautam Srivastava, and Jerry Chun-Wei Lin. 2021. Secure collaborative augmented reality framework for biomedical informatics. IEEE Journal of Biomedical and Health Informatics 26, 6 (2021), 2417–2424.
Helen Forbes, Tracey K. Bucknall, and Alison M. Hutchinson. 2016. Piloting the feasibility of head-mounted video technology to augment student feedback during simulated clinical decision-making: An observational design pilot study. Nurse Education Today 39 (2016), 116–121. https://doi.org/10.1016/j.nedt.2016.01.012
Alba García-Viola, José Miguel Garrido-Molina, Verónica V Márquez-Hernández, Genoveva Granados-Gámez, Gabriel Aguilera-Manrique, and Lorena GutiérrezPuertas. 2019. The influence of gamification on decision making in nursing students. Journal of Nursing Education 58, 12 (2019), 718–722.
Wenzel Jakob, Marco Tarini, Daniele Panozzo, Olga Sorkine-Hornung, et al. 2015. Instant field-aligned meshes. ACM Trans. Graph. 34, 6 (2015), 189–1.
Carley Jans, Fiona Bogossian, Patrea Andersen, and Tracy Levett-Jones. 2023. Examining the impact of virtual reality on clinical decision making–An integrative review. Nurse Education Today (2023), 105767.
Yuen C Law, Wilken Wehrt, Sabine Sonnentag, and Benjamin Weyers. 2023. Obtaining semi-formal models from qualitative data: From interviews into bpmn models in user-centered design processes. International Journal of Human–Computer Interaction 39, 3 (2023), 476–493.
Clayton Lewis and Cathleen Wharton. 1997. Cognitive walkthroughs. In Handbook of human-computer interaction. Elsevier, 717–732.
Tahir Mahmood, Willis Fulmer, Neelesh Mungoli, Jian Huang, and Aidong Lu. 2019. Improving information sharing and collaborative analysis for remote geospatial visualization using mixed reality. In 2019 IEEE International Symposium on Mixed and Augmented Reality (ISMAR). IEEE, 236–247.
Tara Matthews, Steve Whittaker, Thomas Moran, and Sandra Yuen. 2011. Collaboration personas: A new approach to designing workplace collaboration tools. In Proceedings of the SIGCHI conference on human factors in computing systems.
Jacqueline McCallum, Valerie Ness, and Theresa Price. 2011. Exploring nursing students’ decision-making skills whilst in a Second Life clinical simulation laboratory. Nurse Education Today 31, 7 (2011), 699–704.
Jillian L McGrath, Jeffrey M Taekman, Parvati Dev, Douglas R Danforth, Deepika Mohan, Nicholas Kman, Amanda Crichlow, William F Bond, Shiela Riker, AJ Lemheney, et al. 2018. Using virtual reality simulation environments to assess competence for emergency medicine learners. Academic Emergency Medicine 25, 2 (2018), 186–195.
Yanni Mei, Jie Li, Huib De Ridder, and Pablo Cesar. 2021. Cakevr: A social virtual reality (vr) tool for co-designing cakes. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–14.
Radu Papara, Ramona Galatus, and Loredana Buzura. 2020. Virtual reality as cost effective tool for distance healthcare. In 2020 22nd International conference on transparent optical networks (ICTON). IEEE, 1–6.
Hyanghee Park, Daehwan Ahn, and Joonhwan Lee. 2023. Towards a Metaverse Workspace: Opportunities, Challenges, and Design Implications. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–20.
Monica Peddle. 2015. Virtual simulation developing non-technical skills in student nurses and midwives. Australian Nursing and Midwifery Journal 23, 1 (2015), 41.
Catlin Pidel and Philipp Ackermann. 2020. Collaboration in virtual and augmented reality: a systematic overview. In Augmented Reality, Virtual Reality, and Computer Graphics: 7th International Conference, AVR 2020, Lecce, Italy, September 7–10, 2020, Proceedings, Part I 7. Springer, 141–156.
Csaba Pinter, Andras Lasso, Saleh Choueib, Mark Asselin, Jean-Christophe FillionRobin, Jean-Baptiste Vimort, Ken Martin, Matthew A Jolley, and Gabor Fichtinger. 2020. SlicerVR for medical intervention training and planning in immersive virtual reality. IEEE transactions on medical robotics and bionics 2, 2 (2020).
Abhinesh Prabhakaran, Abdul-Majeed Mahamadu, Lamine Mahdjoubi, and Pawel Boguslawski. 2022. BIM-based immersive collaborative environment for furniture, fixture and equipment design. Automation in Construction 142 (2022), 104489.
Sandy Q Qu and John Dumay. 2011. The qualitative research interview. Qualitative research in accounting & management 8, 3 (2011), 238–264.
John Rieman, Marita Franzke, and David Redmiles. 1995. Usability Evaluation with the Cognitive Walkthrough. In Conference Companion on Human Factors in Computing Systems (Denver, Colorado, USA) (CHI ’95). Association for Computing Machinery, New York, NY, USA, 387–388. https://doi.org/10.1145/223355.223735
Victor Romero, Romain Pinquié, and Frédéric Noël. 2021. An immersive virtual environment for reviewing model-centric designs. Proceedings of the Design Society 1 (2021), 447–456.
Mattias Roupé, Mikael Johansson, Laura Maftei, Rikard Lundstedt, and Mikael Viklund-Tallgren. 2020. Virtual collaborative design environment: supporting seamless integration of multitouch table and immersive VR. Journal of Construction Engineering and Management 146, 12 (2020), 04020132.
Sharad Sharma. 2020. Improving emergency response training and decision making using a collaborative virtual reality environment for building evacuation. In HCI International 2020–Late Breaking Papers: Virtual and Augmented Reality: 22nd HCI International Conference, HCII 2020, Copenhagen, Denmark, July 19–24, 2020, Proceedings 22. Springer, 213–224.
Jae-Eun Shin, Boram Yoon, Dooyoung Kim, Hyung-Il Kim, and Woontack Woo. 2022. The Effects of Device and Spatial Layout on Social Presence During a Dynamic Remote Collaboration Task in Mixed Reality. In 2022 IEEE International Symposium on Mixed and Augmented Reality (ISMAR). IEEE, 394–403.
Neal Stephenson. 2003. Snow crash: A novel. Spectra.
David R Thomas. 2006. A general inductive approach for analyzing qualitative evaluation data. American journal of evaluation 27, 2 (2006), 237–246.
Arnold POS Vermeeren, Effie Lai-Chong Law, Virpi Roto, Marianna Obrist, Jettie Hoonhout, and Kaisa Väänänen-Vainio-Mattila. 2010. User experience evaluation methods: current state and development needs. In Proceedings of the 6th Nordic conference on human-computer interaction: Extending boundaries. 521–530.
Jacob O. Wobbrock, Leah Findlater, Darren Gergle, and James J. Higgins. 2011. The Aligned Rank Transform for nonparametric factorial analyses using only ANOVA procedures. Conference on Human Factors in Computing Systems - Proceedings (2011), 143 – 146. https://doi.org/10.1145/1978942.1978963 Cited by: 1112; All Open Access, Green Open Access.
Hemendra Worlikar, Sean Coleman, Jack Kelly, Sadhbh O’Connor, Aoife Murray, Terri McVeigh, Jennifer Doran, Ian McCabe, and Derek O’Keeffe. 2023. Mixed Reality Platforms in Telehealth Delivery: Scoping Review. JMIR Biomed Eng 8 (24 Mar 2023), e42709. https://doi.org/10.2196/42709
Hemendra Worlikar, Sean Coleman, Jack Kelly, Sadhbh O’Connor, Aoife Murray, Terri McVeigh, Jennifer Doran, Ian McCabe, and Derek O’Keeffe. 2023. Mixed Reality Platforms in Telehealth Delivery: Scoping Review. JMIR Biomedical Engineering 8 (2023), e42709.
Dawei Yang, Jian Zhou, Rongchang Chen, Yuanlin Song, Zhenju Song, Xiaoju Zhang, Qi Wang, Kai Wang, Chengzhi Zhou, Jiayuan Sun, Lichuan Zhang, Li Bai, Yuehong Wang, Xu Wang, Yeting Lu, Hongyi Xin, Charles A. Powell, Christoph Thüemmler, Niels H. Chavannes, Wei Chen, Lian Wu, and Chunxue Bai. 2022. Expert consensus on the metaverse in medicine. Clinical eHealth 5 (2022), 1–9.
Publicado
06/11/2023
Como Citar
OLIVEIRA, Eder et al.
Modeling and Designing Clinical Sessions with Hybrid Environments in the Metaverse. In: SIMPÓSIO DE REALIDADE VIRTUAL E AUMENTADA (SVR), 25. , 2023, Rio Grande/RS.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2023
.
p. 183–193.
