Exploring Crossmodal Interaction of Tactile and Visual Cues on Temperature Perception in Virtual Reality: a Preliminary Study
Abstract
Thanks to the digital revolution, virtual reality (VR) has getting popularity due to its capacity to immerse users into virtual environments (VEs). VEs are typically limited to visual and auditory cues; however, recent results show that multiple sensory modalities increase the user’s immersion. In this study, an experimental protocol is proposed to recreate multiple tactile, in particular thermal, sensations in VR. The aim is twofold: (1) studying the performance of different devices for creating warm and cold sensations with regards to their efficiency and acoustic disturbance; and (2) investigating the interdependency between visual and tactile stimuli in the perception of temperature. 14 participants performed two experimental studies. Our results show no acoustic disturbance of the materials used. Spot projector is more efficient than fan heater to create a warm sensation; fan + water spray is more efficient than fan alone to create cold sensation. Moreover, no significant contribution of visual cue on the thermal perception was found except for the extremely cold simulation (snow visualization and thermal stimulation performed with fan + water spray).
Keywords:
Virtual environment, Thermal sensation, Virtual reality
References
Yasin Farmani and Robert J. Teather. 2018. Viewpoint Snapping to Reduce Cybersickness in Virtual Reality. In Proc. Graphics Interface Conference. Canadian Human-Computer Communications Society, Waterloo, CAN, 168–175. https://doi.org/10.20380/GI2018.23
Weina Jin, Jianyu Fan, Diane Gromala, and Philippe Pasquier. 2018. AAutomatic Prediction of Cybersickness for Virtual Reality Games. In IEEE Games, Entertainment, Media Conference. IEEE, Galway, Ireland, 1–9. https://doi.org/10.1109/GEM.2018.8516469
Miguel Melo, Guilherme Gonçalves, Pedro Monteiro, Hugo Coelho, José Vasconcelos-Raposo, and Maximino Bessa. 2020. Do Multisensory stimuli benefit the virtual reality experience? A systematic review. IEEE Transactions on Visualization and Computer Graphics. https://doi.org/10.1109/TVCG.2020.3010088
Marianna Obrist, Elia Gatti, Emanuela Maggioni, Chi Thanh Vi, and Carlos Velasco. 2017. Multisensory Experiences in HCI. IEEE MultiMedia 24.https://doi.org/10.1109/MMUL.2017.33
Felix Hülsmann, Julia Fröhlich, Nikita Mattar, and Ipke Wachsmuth. 2014. Wind and warmth in wirtual weality: implementation and evaluation. In Proc. Virtual Reality International Conference. ACM, New York,1-8. https://doi.org/10.1145/2617841.2620712
Daisuke Iwai, Mei Aoki, and Kosuke Sato. 2019. Non-Contact Thermo-Visual Augmentation by IR-RGB Projection. IEEE Transactions on Visualization and Computer Graphics 25, 4 (2019), 1707–1716. https://doi.org/10.1109/TVCG.2018.2820121
Nimesha Ranasinghe, Pravar Jain, Shienny Karwita, David Tolley, and Ellen Yi-Luen Do. 2017. Ambiotherm: Enhancing Sense of Presence in Virtual Reality by Simulating Real-World Environmental Conditions. In Proc. CHI’17. ACM, New York, 1731–1742. https://doi.org/10.1145/3025453.3025723
Francis McGlone and David Reilly. 2009. The cutaneous sensory system. Neuroscience & Biobehavioral Reviews 34 (2009). https://doi.org/10.1016/j.neubiorev.2009.08.004
Ping-Hsuan Han, Yang-Sheng Chen, Kong-Chang Lee, Hao-Cheng Wang, Chiao-En Hsieh, Jui-Chun Hsiao, Chien-Hsing Chou, and YiPing Hung. 2018. Haptic around: multiple tactile Sensations for immersive environment and interaction in virtual reality. In Proc. Symposium on Virtual Reality Software and Technology. ACM, New York, 1–10. https://doi.org/10.1145/3281505.3281507
Mitsuru Minakuchi and Satoshi Nakamura. 2007. Collaborative Ambient Systems by Blow Displays. In Proc. International Conference on Tangible and Embedded Interaction. ACM, New York, 105–108. https://doi.org/10.1145/1226969.1226992
Taeyong Moon and Gerard J. Kim. 2004. Design and Evaluation of a Wind Display for Virtual Reality. In Proc. Virtual Reality Software and Technology. ACM, New York, 122–128. https://doi.org/10.1145/1077534.1077558
Jouke C. Verlinden, Fabian A. Mulder, Joris S. Vergeest, Anna de Jonge, Darina Krutiy, Zsuzsa Nagy, Bob J. Logeman, and Paul Schouten. 2013. Enhancement of Presence in a Virtual Sailing Environment through Localized Wind Simulation. Procedia Engineering 60 (2013). https://doi.org/10.1016/j.proeng.2013.07.050
Dhruv Jain, Misha Sra, Jingru Guo, Rodrigo Marques, Raymond Wu, Justin Chiu, and Chris Schmandt. 2016. Immersive Terrestrial Scuba Diving Using Virtual Reality. In Proc. CHI’16. ACM, New York, 1563–1569. https://doi.org/10.1145/2851581.2892503
Roshan Lalitha Peiris, Yuan-Ling Feng, Liwei Chan, and Kouta Minamizawa. 2019. ThermalBracelet: Exploring Thermal Haptic Feedback Around the Wrist. In Proc. CHI’19. ACM, New York, 1–11.
Sebastian Günther, Florian Müller, Dominik Schön, Omar Elmoghazy, Max Mühlhäuser, and Martin Schmitz. 2020. Therminator: Understanding the Interdependency of Visual and On-Body Thermal Feedback in Virtual Reality. In Proc. CHI’20. ACM, New York, 1–14. https://doi.org/10.1145/3313831.3376195
Roshan Lalintha Peiris, Wei Peng, Zikun Chen, Liwei Chan, and Kouta Minamizawa. 2017. ThermoVR: Exploring Integrated Thermal Haptic Feedback with Head Mounted Displays. In Proc CHI’17. ACM, New York, 5452–5456. https://doi.org/10.1145/3025453.3025824
Weina Jin, Jianyu Fan, Diane Gromala, and Philippe Pasquier. 2018. AAutomatic Prediction of Cybersickness for Virtual Reality Games. In IEEE Games, Entertainment, Media Conference. IEEE, Galway, Ireland, 1–9. https://doi.org/10.1109/GEM.2018.8516469
Miguel Melo, Guilherme Gonçalves, Pedro Monteiro, Hugo Coelho, José Vasconcelos-Raposo, and Maximino Bessa. 2020. Do Multisensory stimuli benefit the virtual reality experience? A systematic review. IEEE Transactions on Visualization and Computer Graphics. https://doi.org/10.1109/TVCG.2020.3010088
Marianna Obrist, Elia Gatti, Emanuela Maggioni, Chi Thanh Vi, and Carlos Velasco. 2017. Multisensory Experiences in HCI. IEEE MultiMedia 24.https://doi.org/10.1109/MMUL.2017.33
Felix Hülsmann, Julia Fröhlich, Nikita Mattar, and Ipke Wachsmuth. 2014. Wind and warmth in wirtual weality: implementation and evaluation. In Proc. Virtual Reality International Conference. ACM, New York,1-8. https://doi.org/10.1145/2617841.2620712
Daisuke Iwai, Mei Aoki, and Kosuke Sato. 2019. Non-Contact Thermo-Visual Augmentation by IR-RGB Projection. IEEE Transactions on Visualization and Computer Graphics 25, 4 (2019), 1707–1716. https://doi.org/10.1109/TVCG.2018.2820121
Nimesha Ranasinghe, Pravar Jain, Shienny Karwita, David Tolley, and Ellen Yi-Luen Do. 2017. Ambiotherm: Enhancing Sense of Presence in Virtual Reality by Simulating Real-World Environmental Conditions. In Proc. CHI’17. ACM, New York, 1731–1742. https://doi.org/10.1145/3025453.3025723
Francis McGlone and David Reilly. 2009. The cutaneous sensory system. Neuroscience & Biobehavioral Reviews 34 (2009). https://doi.org/10.1016/j.neubiorev.2009.08.004
Ping-Hsuan Han, Yang-Sheng Chen, Kong-Chang Lee, Hao-Cheng Wang, Chiao-En Hsieh, Jui-Chun Hsiao, Chien-Hsing Chou, and YiPing Hung. 2018. Haptic around: multiple tactile Sensations for immersive environment and interaction in virtual reality. In Proc. Symposium on Virtual Reality Software and Technology. ACM, New York, 1–10. https://doi.org/10.1145/3281505.3281507
Mitsuru Minakuchi and Satoshi Nakamura. 2007. Collaborative Ambient Systems by Blow Displays. In Proc. International Conference on Tangible and Embedded Interaction. ACM, New York, 105–108. https://doi.org/10.1145/1226969.1226992
Taeyong Moon and Gerard J. Kim. 2004. Design and Evaluation of a Wind Display for Virtual Reality. In Proc. Virtual Reality Software and Technology. ACM, New York, 122–128. https://doi.org/10.1145/1077534.1077558
Jouke C. Verlinden, Fabian A. Mulder, Joris S. Vergeest, Anna de Jonge, Darina Krutiy, Zsuzsa Nagy, Bob J. Logeman, and Paul Schouten. 2013. Enhancement of Presence in a Virtual Sailing Environment through Localized Wind Simulation. Procedia Engineering 60 (2013). https://doi.org/10.1016/j.proeng.2013.07.050
Dhruv Jain, Misha Sra, Jingru Guo, Rodrigo Marques, Raymond Wu, Justin Chiu, and Chris Schmandt. 2016. Immersive Terrestrial Scuba Diving Using Virtual Reality. In Proc. CHI’16. ACM, New York, 1563–1569. https://doi.org/10.1145/2851581.2892503
Roshan Lalitha Peiris, Yuan-Ling Feng, Liwei Chan, and Kouta Minamizawa. 2019. ThermalBracelet: Exploring Thermal Haptic Feedback Around the Wrist. In Proc. CHI’19. ACM, New York, 1–11.
Sebastian Günther, Florian Müller, Dominik Schön, Omar Elmoghazy, Max Mühlhäuser, and Martin Schmitz. 2020. Therminator: Understanding the Interdependency of Visual and On-Body Thermal Feedback in Virtual Reality. In Proc. CHI’20. ACM, New York, 1–14. https://doi.org/10.1145/3313831.3376195
Roshan Lalintha Peiris, Wei Peng, Zikun Chen, Liwei Chan, and Kouta Minamizawa. 2017. ThermoVR: Exploring Integrated Thermal Haptic Feedback with Head Mounted Displays. In Proc CHI’17. ACM, New York, 5452–5456. https://doi.org/10.1145/3025453.3025824
Published
2021-06-21
How to Cite
HELFENSTEIN-DIDIER, Clémentine; DHOUIB, Amira; FAVRE, Florent; PASCAL, Jonathan; BAERT, Patrick.
Exploring Crossmodal Interaction of Tactile and Visual Cues on Temperature Perception in Virtual Reality: a Preliminary Study . In: WORKSHOP ON MULTISENSORY EXPERIENCES (SENSORYX), 1. , 2021, New York.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2021
.
DOI: https://doi.org/10.5753/sensoryx.2021.15684.
