Cooking in the dark: a mixed reality empathy experience for the embodiment of blindness
Abstract
In the context of promoting a sense of empathy for the difference in people without disabilities, we propose a gaming experience that allows users to embody having a visual impairment. By occluding the user’s vision and providing spatialized audio and passive haptic feedback, allied with a speech recognition digital assistant, our goal is to offer a multi-sensory experience to enhance the user’s sense of embodiment inside a mixed reality blindness simulation. Inside the game environment, while expecting a guest to arrive, the player is required to cook a meal completely in the dark. Being aided solely by their remaining senses and a digital assistant, players must go through several tasks as to prepare dinner in time, risking to lose a love interest.
Keywords:
AR, Digital Empathy, Interactive Games, Embodiment, Spatial audio, Passive Haptics, HCI
References
Sun Joo Ahn. 2011. Embodied experiences in immersive virtual environments: Effects on pro-environmental attitude and behavior. Ph.D. Dissertation. Citeseer.
Robert Albrecht, Riitta Väänänen, and Tapio Lokki. 2016. Guided by Music: Pedestrian and Cyclist Navigation with Route and Beacon Guidance. Personal Ubiquitous Comput. 20, 1 (Feb. 2016), 121–145. https://doi.org/10.1007/s00779-016-0906-z
Daniel Bacchus. 2019. Life is Beautiful. Always: Using virtual reality to share the experience of disability. (2019).
Stacy M. Branham and Shaun K. Kane. 2015. Collaborative Accessibility: How Blind and Sighted Companions Co-Create Accessible Home Spaces. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (Seoul, Republic of Korea) (CHI ’15). Association for Computing Machinery, New York, NY, USA, 2373–2382. https://doi.org/10.1145/2702123.2702511
John Brooke. 1996. SUS: A quick and dirty usability scale.
Aimee Calepso, Renan Guarese, Rafael Valer, Yhonatan Jesus Iquiapaza Ccama, Victor Adriel Oliveira, Luciana Nedel, and Anderson Maciel. 2020. 3DUI and the Phantom Limb: multisensory experience for embodiment of amputation. In 2020 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW). 517–518. https://doi.org/10.1109/VRW50115.2020.00107
Jennifer Edson Escalas and Barbara B. Stern. 2003. Sympathy and Empathy: Emotional Responses to Advertising Dramas. Journal of Consumer Research 29, 4 (03 2003), 566–578. https://doi.org/10.1086/346251 arXiv:https://academic.oup.com/jcr/article-pdf/29/4/566/17927772/29-4-566.pdf
Sylvain Ferrand, François Alouges, and Matthieu Aussal. 2017. Binaural spatialization methods for indoor navigation. In Audio Engineering Society Convention 142. Berlin, Germany. https://hal.archives-ouvertes.fr/hal-02948560
Sylvain Ferrand, Francois Alouges, and Matthieu Aussal. 2018. An Augmented Reality Audio Device Helping Blind People Navigation. In Computers Helping People with Special Needs, Klaus Miesenberger and Georgios Kouroupetroglou (Eds.). Springer International Publishing, Cham, 28–35.
Borko Furht (Ed.). 2008. Virtual Presence. Springer US, Boston, MA, 967–968. https://doi.org/10.1007/978-0-387-78414-4_254
Victoria Groom, Jeremy N Bailenson, and Clifford Nass. 2009. The influence of racial embodiment on racial bias in immersive virtual environments. Social Influence 4, 3 (2009), 231–248.
R. Guarese, V. d. J. Oliveira, A. Calepso, , R. Valer, Y. Iquiapaza, L. Nedel, and A. Maciel. 2020. E-mpathy and the Phantom Limb Sensation: a multisensory experience for embodiment of amputation. In Proceedings of the XChange Reality! 2020 St. Pölten, Austria, April 27-30, 2020. 13–16. http://ceur-ws.org/Vol-2618/
Blake Hannaford and Allison M Okamura. 2016. Haptics. Springer International Publishing, Cham, 1063–1084. https://doi.org/10.1007/978-3-319- 32552-1_42
Sandra G. Hart and Lowell E. Staveland. 1988. Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research. In Human Mental Workload, Peter A. Hancock and Najmedin Meshkati (Eds.). Advances in Psychology, Vol. 52. North-Holland, 139 – 183. https: //doi.org/10.1016/S0166-4115(08)62386-9
Brent Edward Insko. 2001. Passive Haptics Significantly Enhances Virtual Environments. Ph.D. Dissertation. Advisor(s) Brooks, Frederick P. AAI3007820.
Yasha Iravantchi, Mayank Goel, and Chris Harrison. 2020. Digital Ventriloquism: Giving Voice to Everyday Objects. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI ’20). Association for Computing Machinery, New York, NY, USA, 1–10. https://doi.org/10.1145/3313831.3376503
Redouane Kachach, Pablo Perez, Alvaro Villegas, and Ester Gonzalez-Sosa. 2020. Blindness Visualizer: A Simulated Navigation Experience. Proceedings - 2020 IEEE Conference on Virtual Reality and 3D User Interfaces, VRW 2020, 504–506. https://doi.org/10.1109/VRW50115.2020.00107
Robert S. Kennedy, Norman E. Lane, Kevin S. Berbaum, and Michael G. Lilienthal. 1993. Simulator Sickness Questionnaire: An Enhanced Method for Quantifying Simulator Sickness. The International Journal of Aviation Psychology 3, 3 (1993), 203–220. https://doi.org/10.1207/s15327108ijap0303_3
Rohit Ashok Khot and Florian Mueller. 2019. Human-Food Interaction. Foundations and Trends® in Human–Computer Interaction 12, 4 (2019), 238–415. https://doi.org/10.1561/1100000074
David L Penn, James D Ivory, Abigail Judge, et al. 2010. The virtual doppelganger: Effects of a virtual reality simulator on perceptions of schizophrenia. The Journal of nervous and mental disease 198, 6 (2010), 437–443.
Kevin M. Storer, Tejinder K. Judge, and Stacy M. Branham. 2020. "All in the Same Boat": Tradeoffs of Voice Assistant Ownership for Mixed-Visual- Ability Families. Conference on Human Factors in Computing Systems - Proceedings. https://doi.org/10.1145/3313831.3376225
Anja Thieme, Cynthia L. Bennett, Cecily Morrison, Edward Cutrell, and Alex S. Taylor. 2018. "I can do everything but see!" - How people with vision impairments negotiate their abilities in social contexts. Conference on Human Factors in Computing Systems - Proceedings 2018-April. https://doi.org/10.1145/3173574.3173777
Francesco Vona, Emanuele Torelli, Eleonora Beccaluva, and Franca Garzotto. 2020. Exploring the Potential of Speech-based Virtual Assistants in Mixed Reality Applications for People with Cognitive Disabilities. Proceedings of the International Conference on Advanced Visual Interfaces, 1–9. https://doi.org/10.1145/3399715.3399845
Zhiquan Wang, Huimin Liu, Yucong Pan, and Christos Mousas. 2020. Experiencing and Navigating Virtual Reality without Sight. Proceedings - 2020 IEEE Conference on Virtual Reality and 3D User Interfaces, VRW 2020, 519–520. https://doi.org/10.1109/VRW50115.2020.00114
Nick Yee and Jeremy N Bailenson. 2007. The Proteus effect: Self transformations in virtual reality. Human Communication Research 33, 3 (2007), 271–90.
Nick Yee, Nicolas Ducheneaut, Mike Yao, and Les Nelson. 2011. Do men heal more when in drag? Conflicting identity cues between user and avatar. In Proceedings of the SIGCHI conference on Human factors in computing systems. 773–776.
Robert Albrecht, Riitta Väänänen, and Tapio Lokki. 2016. Guided by Music: Pedestrian and Cyclist Navigation with Route and Beacon Guidance. Personal Ubiquitous Comput. 20, 1 (Feb. 2016), 121–145. https://doi.org/10.1007/s00779-016-0906-z
Daniel Bacchus. 2019. Life is Beautiful. Always: Using virtual reality to share the experience of disability. (2019).
Stacy M. Branham and Shaun K. Kane. 2015. Collaborative Accessibility: How Blind and Sighted Companions Co-Create Accessible Home Spaces. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (Seoul, Republic of Korea) (CHI ’15). Association for Computing Machinery, New York, NY, USA, 2373–2382. https://doi.org/10.1145/2702123.2702511
John Brooke. 1996. SUS: A quick and dirty usability scale.
Aimee Calepso, Renan Guarese, Rafael Valer, Yhonatan Jesus Iquiapaza Ccama, Victor Adriel Oliveira, Luciana Nedel, and Anderson Maciel. 2020. 3DUI and the Phantom Limb: multisensory experience for embodiment of amputation. In 2020 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW). 517–518. https://doi.org/10.1109/VRW50115.2020.00107
Jennifer Edson Escalas and Barbara B. Stern. 2003. Sympathy and Empathy: Emotional Responses to Advertising Dramas. Journal of Consumer Research 29, 4 (03 2003), 566–578. https://doi.org/10.1086/346251 arXiv:https://academic.oup.com/jcr/article-pdf/29/4/566/17927772/29-4-566.pdf
Sylvain Ferrand, François Alouges, and Matthieu Aussal. 2017. Binaural spatialization methods for indoor navigation. In Audio Engineering Society Convention 142. Berlin, Germany. https://hal.archives-ouvertes.fr/hal-02948560
Sylvain Ferrand, Francois Alouges, and Matthieu Aussal. 2018. An Augmented Reality Audio Device Helping Blind People Navigation. In Computers Helping People with Special Needs, Klaus Miesenberger and Georgios Kouroupetroglou (Eds.). Springer International Publishing, Cham, 28–35.
Borko Furht (Ed.). 2008. Virtual Presence. Springer US, Boston, MA, 967–968. https://doi.org/10.1007/978-0-387-78414-4_254
Victoria Groom, Jeremy N Bailenson, and Clifford Nass. 2009. The influence of racial embodiment on racial bias in immersive virtual environments. Social Influence 4, 3 (2009), 231–248.
R. Guarese, V. d. J. Oliveira, A. Calepso, , R. Valer, Y. Iquiapaza, L. Nedel, and A. Maciel. 2020. E-mpathy and the Phantom Limb Sensation: a multisensory experience for embodiment of amputation. In Proceedings of the XChange Reality! 2020 St. Pölten, Austria, April 27-30, 2020. 13–16. http://ceur-ws.org/Vol-2618/
Blake Hannaford and Allison M Okamura. 2016. Haptics. Springer International Publishing, Cham, 1063–1084. https://doi.org/10.1007/978-3-319- 32552-1_42
Sandra G. Hart and Lowell E. Staveland. 1988. Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research. In Human Mental Workload, Peter A. Hancock and Najmedin Meshkati (Eds.). Advances in Psychology, Vol. 52. North-Holland, 139 – 183. https: //doi.org/10.1016/S0166-4115(08)62386-9
Brent Edward Insko. 2001. Passive Haptics Significantly Enhances Virtual Environments. Ph.D. Dissertation. Advisor(s) Brooks, Frederick P. AAI3007820.
Yasha Iravantchi, Mayank Goel, and Chris Harrison. 2020. Digital Ventriloquism: Giving Voice to Everyday Objects. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI ’20). Association for Computing Machinery, New York, NY, USA, 1–10. https://doi.org/10.1145/3313831.3376503
Redouane Kachach, Pablo Perez, Alvaro Villegas, and Ester Gonzalez-Sosa. 2020. Blindness Visualizer: A Simulated Navigation Experience. Proceedings - 2020 IEEE Conference on Virtual Reality and 3D User Interfaces, VRW 2020, 504–506. https://doi.org/10.1109/VRW50115.2020.00107
Robert S. Kennedy, Norman E. Lane, Kevin S. Berbaum, and Michael G. Lilienthal. 1993. Simulator Sickness Questionnaire: An Enhanced Method for Quantifying Simulator Sickness. The International Journal of Aviation Psychology 3, 3 (1993), 203–220. https://doi.org/10.1207/s15327108ijap0303_3
Rohit Ashok Khot and Florian Mueller. 2019. Human-Food Interaction. Foundations and Trends® in Human–Computer Interaction 12, 4 (2019), 238–415. https://doi.org/10.1561/1100000074
David L Penn, James D Ivory, Abigail Judge, et al. 2010. The virtual doppelganger: Effects of a virtual reality simulator on perceptions of schizophrenia. The Journal of nervous and mental disease 198, 6 (2010), 437–443.
Kevin M. Storer, Tejinder K. Judge, and Stacy M. Branham. 2020. "All in the Same Boat": Tradeoffs of Voice Assistant Ownership for Mixed-Visual- Ability Families. Conference on Human Factors in Computing Systems - Proceedings. https://doi.org/10.1145/3313831.3376225
Anja Thieme, Cynthia L. Bennett, Cecily Morrison, Edward Cutrell, and Alex S. Taylor. 2018. "I can do everything but see!" - How people with vision impairments negotiate their abilities in social contexts. Conference on Human Factors in Computing Systems - Proceedings 2018-April. https://doi.org/10.1145/3173574.3173777
Francesco Vona, Emanuele Torelli, Eleonora Beccaluva, and Franca Garzotto. 2020. Exploring the Potential of Speech-based Virtual Assistants in Mixed Reality Applications for People with Cognitive Disabilities. Proceedings of the International Conference on Advanced Visual Interfaces, 1–9. https://doi.org/10.1145/3399715.3399845
Zhiquan Wang, Huimin Liu, Yucong Pan, and Christos Mousas. 2020. Experiencing and Navigating Virtual Reality without Sight. Proceedings - 2020 IEEE Conference on Virtual Reality and 3D User Interfaces, VRW 2020, 519–520. https://doi.org/10.1109/VRW50115.2020.00114
Nick Yee and Jeremy N Bailenson. 2007. The Proteus effect: Self transformations in virtual reality. Human Communication Research 33, 3 (2007), 271–90.
Nick Yee, Nicolas Ducheneaut, Mike Yao, and Les Nelson. 2011. Do men heal more when in drag? Conflicting identity cues between user and avatar. In Proceedings of the SIGCHI conference on Human factors in computing systems. 773–776.
Published
2021-06-21
How to Cite
GUARESE, Renan et al.
Cooking in the dark: a mixed reality empathy experience for the embodiment of blindness. In: XR IN GAMES WORKSHOP (XR IN GAMES), 1. , 2021, New York.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2021
.
DOI: https://doi.org/10.5753/xr_in_games.2021.15680.
