Designing a Multimodal Feedback Component for VR Applications
Resumo
Virtual Reality (VR) environments have shown great potential across a wide range of applications that benefit from interaction through touch and haptic feedback, such as entertainment, education, and rehabilitation. By simulating realistic tactile sensations, such as contact, texture, and pressure, VR systems can enhance user engagement, accessibility, and understanding of virtual worlds. In this context, haptic-enabled VR applications can play a fundamental role in diverse scenarios that demand precise object manipulation and spatial awareness. This paper presents a multimodal feedback system for VR applications to provide precision in object recognition using headset controllers and their collision with virtual objects. The proposed system generates haptic feedback regarding three types of collision: touch, stroke, and entering. It is intended to increase the user experience and sense of presence. In the evaluation, most users reported being satisfied with the application and stated that it provided a good sense of presence. They also found that, among the vibration patterns, the irregular one was the most successful in conveying the intended texture.
Palavras-chave:
Multimodal, Multisensory, Virtual Reality, Acessibility
Referências
Fabio Buttussi and Luca Chittaro. 2023. Acquisition and retention of spatial knowledge through virtual reality experiences: Effects of VR setup and locomotion technique. International Journal of Human-Computer Studies 177 (2023), 103067.
Ahmed Choueib and Mehmet İlker Berkman. 2023. Comparing Performance and Experience in VR vs. Real-World Through a Puzzle Game. In International Conference on Videogame Sciences and Arts. Springer Nature Switzerland, 72–85.
Alexandra Covaci, Anne-Hélène Olivier, and Franck Multon. 2014. Third person view and guidance for more natural motor behaviour in immersive basketball playing. In Proceedings of the 20th ACM Symposium on Virtual Reality Software and Technology. 55–64.
Dixuan Cui and Christos Mousas. 2022. Estimating the Just Noticeable Difference of Tactile Feedback in Oculus Quest 2 Controllers. In 2022 IEEE International Symposium on Mixed and Augmented Reality (ISMAR). DOI: 10.1109/ISMAR55827.2022.00013
Arlindo Fernando Paiva de Carvalho Junrio. 2022. Orientation and mobility in physical education classes: knowledge experiences (des) built in the Benjamin Constant Institute. Revista Diálogos e Perspectivas em Educação Especial (2022).
Massimiliano Di Luca and Arash Mahnan. 2019. Perceptual Limits of Visual-Haptic Simultaneity in Virtual Reality Interactions. In 2019 IEEE World Haptics Conference (WHC). IEEE. DOI: 10.1109/WHC.2019.8816173
Aviv Elor et al. 2020. On shooting stars: Comparing cave and hmd immersive virtual reality exergaming for adults with mixed ability. ACM Transactions on Computing for Healthcare 1, 4 (2020), 1–22.
Mariusz P. Furmanek, Madhur Mangalam, Kyle Lockwood, Andrea Smith, Mathew Yarossi, and Eugene Tunik. 2021. Effects of Sensory Feedback and Collider Size on Reach-to-Grasp Coordination in Haptic-Free Virtual Reality. Frontiers in Virtual Reality (2021). DOI: 10.3389/frvir.2021.648529
Chulwoo Ha, Sanghun Nam, and Joungheum Kwon. 2024. Object Manipulation Using Haptic Feedback–Based Six Hand Areas in Virtual Reality. Edelweiss Applied Science and Technology (2024). DOI: 10.55214/25768484.v8i6.2822
Howard Kaplan and Anna Pyayt. 2024. Fully Digital Audio Haptic Maps for Individuals with Blindness. Disabilities 4, 1 (2024), 64–78. DOI: 10.3390/disabilities4010005
Sukran Karaosmanoglu et al. 2022. Canoe VR: An immersive exergame to support cognitive and physical exercises of older adults. In CHI Conference on Human Factors in Computing Systems Extended Abstracts. 1–7.
Hooman Khosravi, Katayoon Etemad, and Faramarz F. Samavati. 2021. Simulating Mass in Virtual Reality using Physically-Based Hand-Object Interactions with Vibration Feedback. In Graphics Interface 2021. DOI: 10.20380/GI2021.27
Orly Lahav. 2022. Virtual Reality Systems as an Orientation Aid for People Who Are Blind to Acquire New Spatial Information. Sensors 22, 4 (2022). DOI: 10.3390/s22041307
Bettina Laugwitz, Theo Held, and Martin Schrepp. 2008. Construction and evaluation of a user experience questionnaire. In Symposium of the Austrian HCI and usability engineering group. Springer, 63–76.
Jack M Loomis, Roberta L Klatzky, and Nicholas A Giudice. 2013. Representing 3D space in working memory: Spatial images from vision, hearing, touch, and language. Multisensory imagery (2013), 131–155.
World Health Organization. 2019. World report on vision. World Health Organization. 160 p. pages.
Grégoire Richard, Thomas Pietrzak, Ferran Argelaguet, Anatole Lécuyer, and Géry Casiez. 2021. Studying the Role of Haptic Feedback on Virtual Embodiment in a Drawing Task. Frontiers in Virtual Reality (2021). DOI: 10.3389/frvir.2020.573167
Lucas H. Sallaberry, Romero Tori, and Fatima L. S. Nunes. 2020. Avaliação automática de habilidades sensório-motoras em simulador odontológico. Interlab - EPUSP, USP (2020).
Jaime Sánchez and Javiera Mascaró. 2011. Audiopolis, Navigation through a Virtual City Using Audio and Haptic Interfaces for People Who Are Blind. In Universal Access in Human-Computer Interaction. Users Diversity, Constantine Stephanidis (Ed.). Springer Berlin Heidelberg, Berlin, Heidelberg, 362–371.
Forrest Smith. 2019. Measuring the perception of latency with a haptic glove. Technical Report. Meta Reality Labs. [link].
Lauren Thevin, Carine Briant, and Anke M Brock. 2020. X-road: virtual reality glasses for orientation and mobility training of people with visual impairments. ACM Transactions on Accessible Computing (TACCESS) 13, 2 (2020), 1–47.
Dimitrios Tzovaras, Konstantinos Moustakas, Georgios Nikolakis, and Michael G. Strintzis. 2009. Interactive mixed reality white cane simulation for the training of the blind and the visually impaired. Personal and Ubiquitous Computing (2009). DOI: 10.1007/s00779-007-0171-2
José Vasconcelos-Raposo, Miguel Melo, Luís Barbosa, C Teixeira, Luciana Cabral, and Maximino Bessa. 2021. Assessing presence in virtual environments: adaptation of the psychometric properties of the Presence Questionnaire to the Portuguese populations. Behaviour & Information Technology 40, 13 (2021), 1417–1427.
Hong-In Won, Byeong Hak Kim, Jun Ha Lee, Sebastian Merchel, and M. Ercan Altinsoy. 2022. Changes in audio intensity perception according to touch recognition in a virtual environment. In Proceedings of the 24th International Congress on Acoustics.
Ben CB Yip and David WK Man. 2009. Virtual reality (VR)-based community living skills training for people with acquired brain injury: A pilot study. Brain injury 23, 13-14 (2009), 1017–1026.
Ahmed Choueib and Mehmet İlker Berkman. 2023. Comparing Performance and Experience in VR vs. Real-World Through a Puzzle Game. In International Conference on Videogame Sciences and Arts. Springer Nature Switzerland, 72–85.
Alexandra Covaci, Anne-Hélène Olivier, and Franck Multon. 2014. Third person view and guidance for more natural motor behaviour in immersive basketball playing. In Proceedings of the 20th ACM Symposium on Virtual Reality Software and Technology. 55–64.
Dixuan Cui and Christos Mousas. 2022. Estimating the Just Noticeable Difference of Tactile Feedback in Oculus Quest 2 Controllers. In 2022 IEEE International Symposium on Mixed and Augmented Reality (ISMAR). DOI: 10.1109/ISMAR55827.2022.00013
Arlindo Fernando Paiva de Carvalho Junrio. 2022. Orientation and mobility in physical education classes: knowledge experiences (des) built in the Benjamin Constant Institute. Revista Diálogos e Perspectivas em Educação Especial (2022).
Massimiliano Di Luca and Arash Mahnan. 2019. Perceptual Limits of Visual-Haptic Simultaneity in Virtual Reality Interactions. In 2019 IEEE World Haptics Conference (WHC). IEEE. DOI: 10.1109/WHC.2019.8816173
Aviv Elor et al. 2020. On shooting stars: Comparing cave and hmd immersive virtual reality exergaming for adults with mixed ability. ACM Transactions on Computing for Healthcare 1, 4 (2020), 1–22.
Mariusz P. Furmanek, Madhur Mangalam, Kyle Lockwood, Andrea Smith, Mathew Yarossi, and Eugene Tunik. 2021. Effects of Sensory Feedback and Collider Size on Reach-to-Grasp Coordination in Haptic-Free Virtual Reality. Frontiers in Virtual Reality (2021). DOI: 10.3389/frvir.2021.648529
Chulwoo Ha, Sanghun Nam, and Joungheum Kwon. 2024. Object Manipulation Using Haptic Feedback–Based Six Hand Areas in Virtual Reality. Edelweiss Applied Science and Technology (2024). DOI: 10.55214/25768484.v8i6.2822
Howard Kaplan and Anna Pyayt. 2024. Fully Digital Audio Haptic Maps for Individuals with Blindness. Disabilities 4, 1 (2024), 64–78. DOI: 10.3390/disabilities4010005
Sukran Karaosmanoglu et al. 2022. Canoe VR: An immersive exergame to support cognitive and physical exercises of older adults. In CHI Conference on Human Factors in Computing Systems Extended Abstracts. 1–7.
Hooman Khosravi, Katayoon Etemad, and Faramarz F. Samavati. 2021. Simulating Mass in Virtual Reality using Physically-Based Hand-Object Interactions with Vibration Feedback. In Graphics Interface 2021. DOI: 10.20380/GI2021.27
Orly Lahav. 2022. Virtual Reality Systems as an Orientation Aid for People Who Are Blind to Acquire New Spatial Information. Sensors 22, 4 (2022). DOI: 10.3390/s22041307
Bettina Laugwitz, Theo Held, and Martin Schrepp. 2008. Construction and evaluation of a user experience questionnaire. In Symposium of the Austrian HCI and usability engineering group. Springer, 63–76.
Jack M Loomis, Roberta L Klatzky, and Nicholas A Giudice. 2013. Representing 3D space in working memory: Spatial images from vision, hearing, touch, and language. Multisensory imagery (2013), 131–155.
World Health Organization. 2019. World report on vision. World Health Organization. 160 p. pages.
Grégoire Richard, Thomas Pietrzak, Ferran Argelaguet, Anatole Lécuyer, and Géry Casiez. 2021. Studying the Role of Haptic Feedback on Virtual Embodiment in a Drawing Task. Frontiers in Virtual Reality (2021). DOI: 10.3389/frvir.2020.573167
Lucas H. Sallaberry, Romero Tori, and Fatima L. S. Nunes. 2020. Avaliação automática de habilidades sensório-motoras em simulador odontológico. Interlab - EPUSP, USP (2020).
Jaime Sánchez and Javiera Mascaró. 2011. Audiopolis, Navigation through a Virtual City Using Audio and Haptic Interfaces for People Who Are Blind. In Universal Access in Human-Computer Interaction. Users Diversity, Constantine Stephanidis (Ed.). Springer Berlin Heidelberg, Berlin, Heidelberg, 362–371.
Forrest Smith. 2019. Measuring the perception of latency with a haptic glove. Technical Report. Meta Reality Labs. [link].
Lauren Thevin, Carine Briant, and Anke M Brock. 2020. X-road: virtual reality glasses for orientation and mobility training of people with visual impairments. ACM Transactions on Accessible Computing (TACCESS) 13, 2 (2020), 1–47.
Dimitrios Tzovaras, Konstantinos Moustakas, Georgios Nikolakis, and Michael G. Strintzis. 2009. Interactive mixed reality white cane simulation for the training of the blind and the visually impaired. Personal and Ubiquitous Computing (2009). DOI: 10.1007/s00779-007-0171-2
José Vasconcelos-Raposo, Miguel Melo, Luís Barbosa, C Teixeira, Luciana Cabral, and Maximino Bessa. 2021. Assessing presence in virtual environments: adaptation of the psychometric properties of the Presence Questionnaire to the Portuguese populations. Behaviour & Information Technology 40, 13 (2021), 1417–1427.
Hong-In Won, Byeong Hak Kim, Jun Ha Lee, Sebastian Merchel, and M. Ercan Altinsoy. 2022. Changes in audio intensity perception according to touch recognition in a virtual environment. In Proceedings of the 24th International Congress on Acoustics.
Ben CB Yip and David WK Man. 2009. Virtual reality (VR)-based community living skills training for people with acquired brain injury: A pilot study. Brain injury 23, 13-14 (2009), 1017–1026.
Publicado
10/11/2025
Como Citar
MAGALHÃES, Gabriel; FONSECA, Davi; AMORIM, Glauco; VIANA, Windson; SANTOS, Joel dos.
Designing a Multimodal Feedback Component for VR Applications. In: BRAZILIAN SYMPOSIUM ON MULTIMEDIA AND THE WEB (WEBMEDIA), 31. , 2025, Rio de Janeiro/RJ.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2025
.
p. 194-201.
DOI: https://doi.org/10.5753/webmedia.2025.15302.
