Aleph Silveira
Fotios Spyridonis
Celso A. S. Santos
ABSTRACT
This paper presents the conclusions drawn from an ongoing experiment investigating the performance, usability, and impact of multisensory stimuli in the virtual environment, with a focus on proposing guidelines for conducting biofeedback-enhanced Quality of Experience (QoE) studies in multimedia-enhanced Virtual Reality (VR). The study evaluated various devices, including the Polar H10 and Grove GSR for biosignal measurement and an EEG device for brainwave analysis. Concerns related to participant comfort were highlighted, such as discomfort caused by tight electrodes and difficulties in achieving consistent contact with the scalp. Ergonomic issues with Head-Mounted Displays (HMDs) were also identified, emphasizing the need for comfortable and immersive experiences. The paper recommends addressing these concerns through inclusive design and user-friendly adaptation of devices. The findings emphasize the importance of integrated devices and user-friendly design to enhance QoE and facilitate the adoption of biofeedback technologies outside of the lab. By following the proposed guidelines, researchers and developers can improve the immersive experience and advance the field of biofeedback in VR environments.
- Mohamed Abdelazez, Sreeraman Rajan, and Adrian D.C. Chan. 2021. Automated Biosignal Quality Analysis of Electrocardiograms. IEEE Instrumentation & Measurement Magazine 24, 2 (2021), 37–44. https://doi.org/10.1109/MIM.2021.9400951Google Scholar
Cross Ref
- Oluwakemi A. Ademoye, Niall Murray, Gabriel-Miro Muntean, and Gheorghita Ghinea. 2016. Audio Masking Effect on Inter-Component Skews in Olfaction-Enhanced Multimedia Presentations. ACM Trans. Multimedia Comput. Commun. Appl. 12, 4, Article 51 (aug 2016), 14 pages. https://doi.org/10.1145/2957753Google ScholarDigital Library
- Tayebeh Baniasadi, Seyed Mohammad Ayyoubzadeh, and Niloofar Mohammadzadeh. 2020. Challenges and practical considerations in applying virtual reality in medical education and treatment. Oman medical journal 35, 3 (2020), e125.Google Scholar
- Jessica Oliveira Brito, Celso A. S. Santos, Rodrigo Laiola Guimarães, and Thiago Felipe Corrêa Borges. 2019. Toward Understanding the Quality of Subtitle Synchronization to Improve the Viewer Experience. In Proceedings of the 25th Brazillian Symposium on Multimedia and the Web (Rio de Janeiro, Brazil) (WebMedia ’19). Association for Computing Machinery, New York, NY, USA, 209–216. https://doi.org/10.1145/3323503.3349565Google ScholarDigital Library
- Gemma A. Calvert and Thomas Thesen. 2004. Multisensory integration: methodological approaches and emerging principles in the human brain. Journal of Physiology-Paris 98, 1 (2004), 191–205. https://doi.org/10.1016/j.jphysparis.2004.03.018 Representation of 3-D Space Using Different Senses In Different Species.Google ScholarCross Ref
- Antonio Calvo-Morata, Manuel Freire, Iván Martínez-Ortiz, and Baltasar Fernández-Manjón. 2022. Scoping review of bioelectrical signals uses in videogames for evaluation purposes.IEEE Access (2022).Google ScholarCross Ref
- Sandra Cano, Jonathan Soto, Laura Acosta, Victor M. Peñeñory, and Fernando Moreira. 2022. Using Brain -Computer Interface to evaluate the User eXperience in interactive systems. Computer Methods in Biomechanics and Biomedical Engineering: Imaging & Visualization (May 2022), 1–9. https://doi.org/10.1080/21681163.2022.2072398Google ScholarCross Ref
- Ioan-Sorin Comşa, Estêvão Bissoli Saleme, Alexandra Covaci, Gebremariam Mesfin Assres, Ramona Trestian, Celso A. S. Santos, and Gheorghiţă Ghinea. 2020. Do I Smell Coffee? The Tale of a 360° Mulsemedia Experience. IEEE MultiMedia 27, 1 (2020), 27–36. https://doi.org/10.1109/MMUL.2019.2954405Google ScholarCross Ref
- Alexandra Covaci, Estêvão B Saleme, Gebremariam Mesfin, Ioan-Sorin Comsa, Ramona Trestian, Celso AS Santos, and George Ghinea. 2022. Multisensory 360 videos under varying resolution levels enhance presence. IEEE Transactions on Visualization and Computer Graphics 29, 4 (2022), 2093–2101.Google ScholarDigital Library
- Alexandra Covaci, Estêvão Bissoli Saleme, Gebremariam Mesfin, Nadia Hussain, Elahe Kani-Zabihi, and Gheorghita Ghinea. 2020. How Do We Experience Crossmodal Correspondent Mulsemedia Content?IEEE Transactions on Multimedia 22, 5 (2020), 1249–1258. https://doi.org/10.1109/TMM.2019.2941274Google ScholarCross Ref
- Sebastião Baümberg Tavares da Silva. 2021. A Framework for Supporting Privacy in the Computation of Biosignals. Ph.D. Dissertation. Universidade de Lisboa (Portugal).Google Scholar
- Tarak Das, Camellia Mitra, Hritwika Paul, and Sahil Banerjee. 2022. Study the Changes of Bioelectrical Skin Impedance of Human body Associated with Different Physiological Parameters. Journal homepage: www. ijrpr. com ISSN 2582 (2022), 7421.Google Scholar
- Darragh Egan, Sean Brennan, John Barrett, Yuansong Qiao, Christian Timmerer, and Niall Murray. 2016. An evaluation of Heart Rate and ElectroDermal Activity as an objective QoE evaluation method for immersive virtual reality environments. In 2016 eighth international conference on quality of multimedia experience (QoMEX). IEEE, 1–6.Google Scholar
- Jing Gao, Peng Li, Zhikui Chen, and Jianing Zhang. 2020. A Survey on Deep Learning for Multimodal Data Fusion. Neural Computation 32, 5 (05 2020), 829–864. https://doi.org/10.1162/neco_a_01273 arXiv:https://direct.mit.edu/neco/article-pdf/32/5/829/1865303/neco_a_01273.pdfGoogle ScholarDigital Library
- George Ghinea, Frederic Andres, and Stephen Gulliver. 2012. Multiple Sensorial Media Advances and Applications: New Developments in. (2012).Google Scholar
- Giorgos Giannakakis, Dimitris Grigoriadis, Katerina Giannakaki, Olympia Simantiraki, Alexandros Roniotis, and Manolis Tsiknakis. 2019. Review on psychological stress detection using biosignals. IEEE Transactions on Affective Computing 13, 1 (2019), 440–460.Google ScholarCross Ref
- Kunal Gupta, Yuewei Zhang, Tamil Selvan Gunasekaran, Prasanth Sasikumar, Nanditha Krishna, Yun Suen Pai, and Mark Billinghurst. 2023. VRdoGraphy: An Empathic VR Photography Experience. In 2023 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW). IEEE, 1013–1014.Google Scholar
- Marc Janeras, Joan Roca, Josep A Gili, Oriol Pedraza, Gerald Magnusson, M Amparo Núñez-Andrés, and Kathryn Franklin. 2022. Using Mixed Reality for the Visualization and Dissemination of Complex 3D Models in Geosciences—Application to the Montserrat Massif (Spain). Geosciences 12, 10 (2022), 370.Google ScholarCross Ref
- Dana Lahat, Tülay Adali, and Christian Jutten. 2015. Multimodal Data Fusion: An Overview of Methods, Challenges, and Prospects. Proc. IEEE 103, 9 (2015), 1449–1477. https://doi.org/10.1109/JPROC.2015.2460697Google ScholarCross Ref
- Yiming Liu, Chun Ki Yiu, Zhao Zhao, Wooyoung Park, Rui Shi, Xingcan Huang, Yuyang Zeng, Kuan Wang, Tsz Hung Wong, Shengxin Jia, Jingkun Zhou, Zhan Gao, Ling Zhao, Kuanming Yao, Jian Li, Chuanlu Sha, Yuyu Gao, Guangyao Zhao, Ya Huang, Dengfeng Li, Qinglei Guo, Yuhang Li, and Xinge Yu. 2023. Soft, miniaturized, wireless olfactory interface for virtual reality. Nature Communications 14, 1 (May 2023). https://doi.org/10.1038/s41467-023-37678-4Google ScholarCross Ref
- Olibário Machado Neto and Maria da Graça Pimentel. 2013. Heuristics for the Assessment of Interfaces of Mobile Devices. In Proceedings of the 19th Brazilian Symposium on Multimedia and the Web (Salvador, Brazil) (WebMedia ’13). Association for Computing Machinery, New York, NY, USA, 93–96. https://doi.org/10.1145/2526188.2526237Google ScholarDigital Library
- Arian Mehrfard, Javad Fotouhi, Giacomo Taylor, Tess Forster, Nassir Navab, and Bernhard Fuerst. 2019. A comparative analysis of virtual reality head-mounted display systems. arXiv preprint arXiv:1912.02913 (2019).Google Scholar
- Mina Mikhail, Khaled El-Ayat, Rana El Kaliouby, James Coan, and John J. B. Allen. 2010. Emotion Detection Using Noisy EEG Data. In Proceedings of the 1st Augmented Human International Conference (Megève, France) (AH ’10). Association for Computing Machinery, New York, NY, USA, Article 7, 7 pages. https://doi.org/10.1145/1785455.1785462Google ScholarDigital Library
- Niall Murray, Oluwakemi A. Ademoye, Gheorghita Ghinea, and Gabriel-Miro Muntean. 2017. A Tutorial for Olfaction-Based Multisensorial Media Application Design and Evaluation. ACM Comput. Surv. 50, 5, Article 67 (sep 2017), 30 pages. https://doi.org/10.1145/3108243Google ScholarDigital Library
- Niall Murray, Brian Lee, Yuansong Qiao, and Gabriel-Miro Muntean. 2014. Multiple-Scent Enhanced Multimedia Synchronization. ACM Trans. Multimedia Comput. Commun. Appl. 11, 1s, Article 12 (oct 2014), 28 pages. https://doi.org/10.1145/2637293Google ScholarDigital Library
- Diogo Kionori Cândido Nishikawa, Roberta Pereira Brandão, and Vânia Paula de Almeida Neris. 2020. Um estudo empírico sobre reações emocionais de usuários na interação com interfaces web pautadas na Gestalt. In Anais do XI Workshop sobre Aspectos da Interação Humano-Computador para a Web Social. SBC, 25–32.Google Scholar
- Pratheep Kumar Paranthaman, Nikesh Bajaj, Nicholas Solovey, and David Jennings. 2021. Comparative Evaluation of the EEG Performance Metrics and Player Ratings on the Virtual Reality Games. In 2021 IEEE Conference on Games (CoG). IEEE, 1–8.Google Scholar
- Szczepan Paszkiel and Szczepan Paszkiel. 2020. Using BCI and VR technology in neurogaming. Analysis and Classification of EEG Signals for Brain–Computer Interfaces (2020), 93–99.Google Scholar
- Jiali Qian, Daniel J McDonough, and Zan Gao. 2020. The effectiveness of virtual reality exercise on individual’s physiological, psychological and rehabilitative outcomes: a systematic review. International journal of environmental research and public health 17, 11 (2020), 4133.Google ScholarCross Ref
- Plínio M.S. Ramos, Caio B.S. Maior, Márcio C. Moura, and Isis D. Lins. 2022. Automatic drowsiness detection for safety-critical operations using ensemble models and EEG signals. Process Safety and Environmental Protection 164 (2022), 566–581. https://doi.org/10.1016/j.psep.2022.06.039Google ScholarCross Ref
- Giuseppe Riva, Brenda K Wiederhold, and Fabrizia Mantovani. 2019. Neuroscience of virtual reality: from virtual exposure to embodied medicine. Cyberpsychology, behavior, and social networking 22, 1 (2019), 82–96.Google Scholar
- Theerat Saichoo, Poonpong Boonbrahm, and Yunyong Punsawad. 2021. Facial-Machine interface-based virtual Reality Wheelchair control using EEG artifacts of Emotiv neuroheadset. In 2021 18th International Conference on Electrical Engineering/Electronics, Computer, Telecommunications and Information Technology (ECTI-CON). IEEE, 781–784.Google ScholarCross Ref
- Estêvão B. Saleme, Alexandra Covaci, Gebremariam Mesfin, Celso A. S. Santos, and Gheorghita Ghinea. 2019. Mulsemedia DIY: A Survey of Devices and a Tutorial for Building Your Own Mulsemedia Environment. ACM Comput. Surv. 52, 3, Article 58 (jun 2019), 29 pages. https://doi.org/10.1145/3319853Google ScholarDigital Library
- Débora Pereira Salgado, Felipe Roque Martins, Thiago Braga Rodrigues, Conor Keighrey, Ronan Flynn, Eduardo Lázaro Martins Naves, and Niall Murray. 2018. A QoE assessment method based on EDA, heart rate and EEG of a virtual reality assistive technology system. In Proceedings of the 9th ACM Multimedia Systems Conference. 517–520.Google ScholarDigital Library
- Marcelle Schaffarczyk, Bruce Rogers, Rüdiger Reer, and Thomas Gronwald. 2022. Validity of the polar H10 sensor for heart rate variability analysis during resting state and incremental exercise in recreational men and women. Sensors 22, 17 (2022), 6536.Google ScholarCross Ref
- Catriona L. Scrivener and Arran T. Reader. 2022. Variability of EEG electrode positions and their underlying brain regions: visualizing gel artifacts from a simultaneous EEG-fMRI dataset. Brain and Behavior 12, 2 (Jan. 2022). https://doi.org/10.1002/brb3.2476Google ScholarCross Ref
- Jungryul Seo, Teemu H Laine, and Kyung-Ah Sohn. 2019. An exploration of machine learning methods for robust boredom classification using EEG and GSR data. Sensors 19, 20 (2019), 4561.Google ScholarCross Ref
- Tanuja Subba and Tejbanta Singh Chingtham. 2022. A Review on Types of Machine Learning Techniques for Biosignal Evaluation for Human Computer Interaction. Advanced Computational Paradigms and Hybrid Intelligent Computing: Proceedings of ICACCP 2021 (2022), 457–466.Google ScholarCross Ref
- Moritz Tacke, Katharina Janson, Katharina Vill, Florian Heinen, Lucia Gerstl, Karl Reiter, and Ingo Borggraefe. 2022. Effects of a reduction of the number of electrodes in the EEG montage on the number of identified seizure patterns. Scientific Reports 12, 1 (2022), 1–7.Google ScholarCross Ref
- Tijana Vuletic, Alex Duffy, Laura Hay, Chris McTeague, Gerard Campbell, and Madeleine Grealy. 2019. Systematic literature review of hand gestures used in human computer interaction interfaces. International Journal of Human-Computer Studies 129 (2019), 74–94.Google ScholarDigital Library
- Efy Yosrita, Yaya Heryadi, Lili Ayu Wulandhari, and Widodo Budiharto. 2019. EEG Based Identification of Words on Exam Models with Yes-No Answers for Students with Visual Impairments. In 2019 IEEE International Conference on Engineering, Technology and Education (TALE). IEEE, 1–5.Google Scholar
Index Terms
- Guidelines for conducting biofeedback-enhanced QoE studies in mulsemedia-enhanced virtual reality
Recommendations
Subjective Evaluation of an Olfaction Enhanced Immersive Virtual Reality Environment
AltMM '17: Proceedings of the 2nd International Workshop on Multimedia Alternate RealitiesRecent research efforts have reported findings on user Quality of Experience (QoE) of immersive virtual reality (VR) experiences. Truly immersive multimedia experiences also include multisensory components such as factional, tactile etc., in addition to ...
Enhanced Virtual Reality: Exploring an Immersive and Realistic Virtual Reality Training for Nursing
SenSys '21: Proceedings of the 19th ACM Conference on Embedded Networked Sensor SystemsVirtual Reality (VR) training is an emerging method, which is widely deployed in more and more applications. Compared with traditional physical training and video games-based training, VR training can not only provide a sense of realism and immersion ...
Virtual reality-supported biofeedback for stress management: Beneficial effects on heart rate variability and user experience
AbstractHeart rate variability biofeedback (HRV-BF) is frequently used for stress management. Recently, virtual reality technology has gained attention for delivery, promising higher immersion, motivation, and attention than classical screens. ...
Highlights- Heart rate variability biofeedback in virtual reality reduces psychological stress.
Comments