Understanding affordances in XR interactions through a design space

Abstract


This paper presents a qualitative analysis of a literature review on affordances in XR environments. Affordances refer to the way how the system communicates to the user how to perform a task. Preliminary studies showed that there was no consensus and neither a standard taxonomy for describing affordances in XR. The goal of this study was to investigate how affordances were used as a strategy to promote discovery and usage in XR environments. We collected and analyzed works that describe affordance concepts and carry out experiments in 3D environments describing the applied affordances. We mapped the affordances usage in 8 dimensions: kind of environment, modality, location, temporal aspect, learning aspect, category of tasks, domain tasks and kind of affordance. Additionally, findings were categorized according to VR tasks and Bloom’s learning existing taxonomies. Finally, the relationship between the dimensions was analyzed and discussed as a tentative reflection and guidance on the future design and development of affordance taxonomy for XR environments.

Keywords: affordances, XR environments, discovery, strategies, taxonomy

References

[n. d.]. Afford. Merriam-Webster. Accessed October, 2023. [link]

Lorin W Anderson and David R Krathwohl. 2001. A taxonomy for learning, teaching, and assessing: A revision of Bloom’s taxonomy of educational objectives: complete edition. Addison Wesley Longman, Inc.

Rodrigo Assaf, Daniel Mendes, and Rui Rodrigues. 2024. Cues to fast-forward collaboration: A Survey of Workspace Awareness and Visual Cues in XR Collaborative Systems. In Computer Graphics Forum. Wiley Online Library, e15066.

Sarune Baceviciute, Thomas Terkildsen, and Guido Makransky. 2021. Remediating learning from non-immersive to immersive media: Using EEG to investigate the effects of environmental embeddedness on reading in Virtual Reality. Computers & Education 164 (2021), 104122. DOI: 10.1016/j.compedu.2020.104122

George P Banky and Aaron S Blicblau. 2017. Doing it with a glass onion: Investigating affordances provided by a head-mounted augmented reality immersive device for the real-time online supervision of experimental learning. In 2017 IEEE 3rd International Conference on Engineering Technologies and Social Sciences (ICETSS). IEEE, 1–6. DOI: 10.1109/ICETSS.2017.8324168

Manshul Belani, Harsh Vardhan Singh, Aman Parnami, and Pushpendra Singh. 2023. Investigating Spatial Representation of Learning Content in Virtual Reality Learning Environments. In 2023 IEEE Conference Virtual Reality and 3D User Interfaces (VR). IEEE, 33–43. DOI: 10.1109/VR55154.2023.00019

Benjamin S Bloom, Max D Engelhart, Edward J Furst, Walker H Hill, and David R Krathwohl. 1956. Taxonomy of educational objectives: The classification of educational goals. Handbook 1: Cognitive domain. McKay New York.

Vanessa Camilleri, Matthew Montebello, Alexiei Dingli, and Vince Briffa. 2017. Walking in small shoes: Investigating the power of VR on empathising with children’s difficulties. In 2017 23rd International Conference on Virtual System & Multimedia (VSMM). IEEE, 1–6. DOI: 10.1109/VSMM.2017.8346253

Joe Cecil, Sam Kauffman, Avinash Gupta, Vern McKinney, and MD Miguel Pirela-Cruz. 2021. Design of a human centered computing (HCC) based virtual reality simulator to train first responders involved in the Covid-19 pandemic. Understanding affordances in XR interactions through a design space IHC ’24, October 07–11, 2024, Brasília, DF In 2021 IEEE International Systems Conference (SysCon). IEEE, 1–7. DOI: 10.1109/SysCon48628.2021.9447090

Joe Cecil, Mary Sweet-Darter, Aaron Cecil-Xavier, and Avinash Gunta. 2021. Role of Affordance, Visual Density and Other HCC Criteria in Designing Virtual Learning Environments to Support STEM Learning for Autistic Students. In 2021 IEEE Frontiers in Education Conference (FIE). IEEE, 1–7. DOI: 10.1109/FIE49875.2021.9637130

Michelene TH Chi. 1997. Quantifying qualitative analyses of verbal data: A practical guide. The journal of the learning sciences 6, 3 (1997), 271–315. DOI: 10.1207/s15327809jls0603_1

Andreas Dengel. 2020. How important is immersion for learning in computer science replugged games?. In Proceedings of the 51st ACM Technical Symposium on Computer Science Education. 1165–1171. DOI: 10.1145/3328778.3366837

Paloma Díaz, Telmo Zarraonandía, Mónica Sánchez-Francisco, Ignacio Aedo, and Teresa Onorati. 2019. Do low cost virtual reality devices support learning acquisition? a comparative study of two different vr devices. In Proceedings of the xx international conference on human computer interaction. 1–8. DOI: 10.1145/3335595.3335629

Connor Daniel Flick, Courtney J Harris, Nikolas T Yonkers, Nahal Norouzi, Austin Erickson, Zubin Choudhary, Matt Gottsacker, Gerd Bruder, and Greg Welch. 2021. Trade-offs in augmented reality user interfaces for controlling a smart environment. In Proceedings of the 2021 ACM Symposium on Spatial User Interaction. 1–11. DOI: 10.1145/3485279.3485288

Francois R Foerster and Jeremy Goslin. 2021. Tool use and function knowledge shape visual object processing. Biological Psychology 164 (2021), 108143. DOI: 10.1016/j.biopsycho.2021.108143

William W Gaver. 1991. Technology affordances. In Proceedings of the SIGCHI conference on Human factors in computing systems. 79–84.

William W Gaver. 1991. Technology affordances. In Proceedings of the SIGCHI conference on Human factors in computing systems. 79–84.

James Jerome Gibson. 1979. The Ecological Approach to Visual Perception. [link]

Katelyn M Grasse, Marjorie Ann M Cuerdo, and Edward F Melcer. 2021. Mad Mixologist: exploring how object placement in tangible play spaces affects collaborative interaction strategies. In 2021 IEEE Conference on Games (CoG). IEEE, 1–8. DOI: 10.1109/CoG52621.2021.9619086

Peidi Gu, Xinhao Xu, Xueqin Qian, and Tsung-Han Weng. 2022. Leveraging Extended Reality for Autistic Individuals: A Scoping Review of Technical Features and Technology Affordances. IEEE Transactions on Learning Technologies (2022). DOI: 10.1109/TLT.2022.3197430

Jie Guan, Jay Irizawa, and Alexis Morris. 2022. Extended reality and internet of things for hyper-connected metaverse environments. In 2022 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW). IEEE, 163–168. DOI: 10.1109/VRW55335.2022.00043

Negin Hamzeheinejad, Daniel Roth, Samantha Monty, Julian Breuer, Anuschka Rodenberg, and Marc Erich Latoschik. 2021. The impact of implicit and explicit feedback on performance and experience during VR-supported motor rehabilitation. In 2021 IEEE Virtual Reality and 3D User Interfaces (VR). IEEE, 382–391. DOI: 10.1109/VR50410.2021.00061

Julia Hertel, Sukran Karaosmanoglu, Susanne Schmidt, Julia Bräker, Martin Semmann, and Frank Steinicke. 2021. A taxonomy of interaction techniques for immersive augmented reality based on an iterative literature review. In 2021 IEEE international symposium on mixed and augmented reality (ISMAR). IEEE, 431–440. DOI: 10.1109/ISMAR52148.2021.00060

Staffs Keele et al. 2007. Guidelines for performing systematic literature reviews in software engineering.

Barbara Kitchenham. 2004. Procedures for performing systematic reviews. Keele, UK, Keele University 33, 2004 (2004), 1–26.

Joseph J LaViola Jr, Ernst Kruijff, Ryan P McMahan, Doug Bowman, and Ivan P Poupyrev. 2017. 3D user interfaces: theory and practice. Addison-Wesley Professional.

Tsung-Chi Lin, Achyuthan Unni Krishnan, and Zhi Li. 2022. Comparison of haptic and augmented reality visual cues for assisting tele-manipulation. In 2022 International Conference on Robotics and Automation (ICRA). IEEE, 9309–9316. DOI: 10.1109/ICRA46639.2022.9811669

Vítor Neves Lopes, Isabel Dantas, José Pedro Barbosa, and José Barbosa. 2022. Telestration in the teaching of basic surgical skills: a randomized trial. Journal of Surgical Education 79, 4 (2022), 1031–1042. DOI: 10.1016/j.jsurg.2022.02.013

Heng Luo, Xu Han, Qinna Feng, and Gege Li. 2022. Integrating Inquiry-Based Pedagogy with Mixed Reality: Theories and Practices. In 2022 4th International Conference on Computer Science and Technologies in Education (CSTE). IEEE, 82–86. DOI: 10.1109/CSTE55932.2022.00021

Aline Menin, Ricardo Cava, Carla Maria Dal Sasso Freitas, Olivier Corby, and Marco Winckler. 2021. Towards a visual approach for representing analytical provenance in exploration processes. In 2021 25th International Conference Information Visualisation (IV). IEEE, 21–28.

Mohsen Moghaddam, Nicholas C Wilson, Alicia Sasser Modestino, Kemi Jona, and Stacy C Marsella. 2021. Exploring augmented reality for worker assistance versus training. Advanced Engineering Informatics 50 (2021), 101410. DOI: 10.1016/j.aei.2021.101410

Simone Morgagni. 2012. Affordances as possible actions: elements for a semiotic approach. (2012).

Peyman Najafi, Masi Mohammadi, Pieter van Wesemael, and Pascale M Le Blanc. 2023. A user-centred virtual city information model for inclusive Community design: State-of-art. Cities 134 (2023), 104203. DOI: 10.1016/j.cities.2023.104203

Don Norman. 2013. The design of everyday things: Revised and expanded edition. Basic books.

Donald A Norman. 1999. Affordance, conventions, and design. interactions 6, 3 (1999), 38–43.

Omobolanle R Ogunseiju, Nihar Gonsalves, Abiola A Akanmu, Diana Bairaktarova, Doug A Bowman, and Farrokh Jazizadeh. 2022. Mixed reality environment for learning sensing technology applications in Construction: A usability study. Advanced Engineering Informatics 53 (2022), 101637. DOI: 10.1016/j.aei.2022.101637

Seungjae Oh, Hyo-Jeong So, and Matthew Gaydos. 2018. Hybrid augmented reality for participatory learning: The hidden efficacy of multi-user game-based simulation. IEEE Transactions on Learning Technologies 11, 1 (2018), 115–127. DOI: 10.1109/TLT.2017.2750673

Anthony J Onwuegbuzie, Nancy L Leech, and Kathleen MT Collins. 2012. Qualitative analysis techniques for the review of the literature. Qualitative Report 17 (2012), 56.

Amos Sunday Oyelere, Friday Joseph Agbo, and Solomon Sunday Oyelere. 2023. Formative evaluation of immersive virtual reality expedition mini-games to facilitate computational thinking. Computers & Education: X Reality 2 (2023), 100016. DOI: 10.1016/j.cexr.2023.100016

Jocelyn Parong, Kimberly A Pollard, Benjamin T Files, Ashley H Oiknine, Anne M Sinatra, Jason D Moss, Antony Passaro, and Peter Khooshabeh. 2020. The mediating role of presence differs across types of spatial learning in immersive technologies. Computers in human behavior 107 (2020), 106290. DOI: 10.1016/j.chb.2020.106290

Shalaunda M Reeves, Kent J Crippen, and Erica D McCray. 2021. The varied experience of undergraduate students learning chemistry in virtual reality laboratories. Computers & Education 175 (2021), 104320. DOI: 10.1016/j.compedu.2021.104320

Yvonne Rogers. 2022. HCI theory: classical, modern, and contemporary. Springer Nature.

Helen Sharp, Jenny Preece, and Yvonne Rogers. 2019. Interaction design: beyond human-computer interaction.

Donghee Shin. 2022. The actualization of meta affordances: Conceptualizing affordance actualization in the metaverse games. Computers in human behavior 133 (2022), 107292. DOI: 10.1016/j.chb.2022.107292

Jae-Eun Shin, Boram Yoon, Dooyoung Kim, Hyung-Il Kim, and Woontack Woo. 2022. The Effects of Device and Spatial Layout on Social Presence During a Dynamic Remote Collaboration Task in Mixed Reality. In 2022 IEEE International Symposium on Mixed and Augmented Reality (ISMAR). IEEE, 394–403. DOI: 10.1109/ISMAR55827.2022.00055

Kay M Stanney, Hannah Nye, Sam Haddad, Kelly S Hale, Christina K Padron, and Joseph V Cohn. 2021. Extended reality (XR) environments. Handbook of human factors and ergonomics (2021), 782–815. DOI: 10.1002/9781119636113.ch30

Jacob H Steffen, James E Gaskin, Thomas O Meservy, Jeffrey L Jenkins, and Iopa Wolman. 2019. Framework of affordances for virtual reality and augmented reality. Journal of Management Information Systems 36, 3 (2019), 683–729.

Anselm Strauss and Juliet M Corbin. 1997. Grounded theory in practice. Sage.

Anchen Sun, Yudong Tao, Mei-Ling Shyu, Shu-Ching Chen, Angela Blizzard, William Andrew Rothenberg, Dainelys Garcia, and Jason F Jent. 2021. Multimodal data integration for interactive and realistic avatar simulation in augmented reality. In 2021 IEEE 22nd International Conference on Information Reuse and Integration for Data Science (IRI). IEEE, 362–369. DOI: 10.1109/IRI51335.2021.00056

Yang Tian, Chi-Wing Fu, Shengdong Zhao, Ruihui Li, Xiao Tang, Xiaowei Hu, and Pheng-Ann Heng. 2019. Enhancing augmented vr interaction via egocentric scene analysis. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 3, 3 (2019), 1–24. DOI: 10.1145/3351263

Mythreye Venkatesan, Harini Mohan, Justin R Ryan, Christian M Schürch, Garry P Nolan, David H Frakes, and Ahmet F Coskun. 2021. Virtual and augmented reality for biomedical applications. Cell reports medicine 2, 7 (2021). DOI: 10.1016/j.xcrm.2021.100348

William H Warren. 1984. Perceiving affordances: visual guidance of stair climbing. Journal of experimental psychology: Human perception and performance 10, 5 (1984), 683.

Michael Williams and Tami Moser. 2019. The art of coding and thematic exploration in qualitative research. International Management Review 15, 1 (2019), 45–55. IHC ’24, October 07–11, 2024, Brasília, DF Vieira et al.

Ezequiel Roberto Zorzal, José Miguel Campos Gomes, Maurício Sousa, Pedro Belchior, Pedro Garcia da Silva, Nuno Figueiredo, Daniel Simões Lopes, and Joaquim Jorge. 2020. Laparoscopy with augmented reality adaptations. Journal of biomedical informatics 107 (2020), 103463. DOI: 10.1016/j.jbi.2020.103463
Published
2024-11-07
VIEIRA, Victor; OLIVEIRA, Eder; MENIN, Aline; CLUA, Esteban; WINCKLER, Marco; TREVISAN, Daniela. Understanding affordances in XR interactions through a design space. In: BRAZILIAN SYMPOSIUM ON HUMAN FACTORS IN COMPUTATIONAL SYSTEMS (IHC), 23. , 2024, Brasília/DF. Anais [...]. Porto Alegre: Sociedade Brasileira de Computação, 2024 . p. 886-899.