Reviewing the Role of Physiological Data in Immersive Environments: A Rapid Systematic Review
Resumo
This systematic literature review investigates the use of physiological signals for evaluating user experience (UX) in immersive environments such as virtual reality (VR), augmented reality, and extended reality (XR). Drawing from 118 peer-reviewed articles published between 2019 and 2024, the study identifies the most commonly used biosignals, collection devices, evaluation criteria, and supporting instruments. Key physiological measures include electrodermal activity (EDA), heart rate (HR), heart rate variability (HRV), and electroencephalography (EEG), which are frequently associated with emotional responses, cognitive load, and presence. The analysis, conducted through rapid review, highlights a strong reliance on hybrid approaches that combine physiological data with subjective questionnaires and behavioral metrics. While emotion and cybersickness emerge as dominant UX evaluation dimensions, the study also reveals ethical gaps in research reporting. By mapping the interconnections between technologies, signals, and evaluation frameworks, this review offers a comprehensive overview of current trends and provides practical insights for researchers aiming to develop more adaptive and user-centered immersive systems.
Palavras-chave:
physiological signals, user experience, immersive environments, virtual reality, UX evaluation, ethical aspects, HCI
Referências
da Silveira, A. C. Lima de Souza, M. Ghinea, G. and Saibel Santos, C. A. Physiological data for user experience and quality of experience A systematic review (2018–2022). International Journal of Human–Computer Interaction, 2024, 1–30.
Radianti, J. Majchrzak, T. A. Fromm, J. and Wohlgenannt, I. A systematic review of immersive virtual reality applications for higher education Design elements, lessons learned, and research agenda. Computers & Education, 147, 2020, 103778.
Goncalves, G. Coelho, H. M. Pedro, M. and Bessa, M. Systematic review of comparative studies of the impact of realism in immersive virtual experiences. ACM Computing Surveys, 55(6):115, 2022.
Putze, S. Alexandrovsky, D. Putze, F. Hoffner, S. Smeddinck, J. D. and Malaka, R. Breaking the experience Effects of questionnaires in VR user studies. CHI Conference on Human Factors in Computing Systems (CHI 20), 2020.
Safikhani, S. Holly, M. Kainz, A. and Pirker, J. The influence of in-VR questionnaire design on the user experience. ACM Symposium on Virtual Reality Software and Technology (VRST 21), 2021.
Certicky, M. Sincak, P. Magyar, G. Vascak, J. and Cavallo, F. Psychophysiological indicators for modeling user experience in interactive digital entertainment. Sensors, 19:989, 2019.
Holloman, A. K. and Crawford, C. S. Defining scents A systematic literature review of olfactory-based computing systems. ACM Transactions on Multimedia Computing, Communications, and Applications, 18(1):15, 2022.
Cartaxo, B. Pinto, G. and Soares, S. Rapid reviews in software engineering. In Contemporary Empirical Methods in Software Engineering, 357–384. Springer, Cham, 2020.
Santos, C. M. C. Pimenta, C. A. M. and Nobre, M. R. C. A estrategia PICO para a construcao da pergunta de pesquisa e busca de evidencias. Revista Latino-Americana de Enfermagem, 15(3):508–511, 2007.
Tabbaa, L. et al. VREED Virtual Reality Emotion Recognition Dataset using Eye Tracking and Physiological Measures. Proceedings of the ACM Interactive, Mobile, Wearable and Ubiquitous Technologies, 5(4), 2022.
Gnacek, M. et al. Heart rate detection from the supratrochlear vessels using a virtual reality headset integrated PPG sensor. Proceedings of the ACM International Conference on Multimodal Interaction (ICMI 20 Companion), 2020.
Chiossi, F. Ou, C. and Mayer, S. Optimizing visual complexity for physiologically-adaptive VR systems Evaluating a multimodal dataset using ECG and EEG features. Proceedings of the International Conference on Advanced Visual Interfaces (AVI 24), ACM, 2024.
Dietz, D. et al. Walk this beam Impact of different balance assistance strategies and height exposure on performance and physiological arousal in VR. ACM Symposium on Virtual Reality Software and Technology (VRST 22), 2022.
Samery, J. et al. Collaborative cooking in VR Effects of network distortion in multi-user virtual environments. ACM Multimedia Systems Conference, 509–515, 2024.
Miner, N. et al. Stairway to heaven A gamified VR journey for breath awareness. CHI Conference on Human Factors in Computing Systems (CHI 24), ACM, 2024.
Delachambre, J. et al. AMD Journee A patient co-designed VR experience to raise awareness towards the impact of AMD on social interactions. ACM International Conference on Interactive Media Experiences (IMX 24), 2024.
Terfurth, L. Gramann, K. and Gehrke, L. Decoding realism of virtual objects Exploring behavioral and ocular reactions to inaccurate interaction feedback. ACM Transactions on Computer–Human Interaction, 31(3), 2024.
Gupta, K. Lazarevic, J. Pai, Y. S. and Billinghurst, M. AffectivelyVR Towards VR personalized emotion recognition. ACM Symposium on Virtual Reality Software and Technology (VRST 20), 2020.
Barathi, S. C. Proulx, M. O’Neill, E. and Lutteroth, C. Affect recognition using psychophysiological correlates in high-intensity VR exergaming. CHI Conference on Human Factors in Computing Systems (CHI 20), ACM, 2020.
Li, Y. Ch’ng, E. and Cobb, S. Factors influencing engagement in hybrid virtual and augmented reality. ACM Transactions on Computer–Human Interaction, 30(4), 2023.
Arjun, S. Hebbar, A. Sanjana, and Biswas, P. VR cognitive load dashboard for flight simulator. Proceedings of the ACM Symposium on Eye Tracking Research and Applications (ETRA 22), 2022.
Ahmad, M. and Alzahrani, A. Crucial clues Investigating psychophysiological behaviors for measuring trust in human-robot interaction. Proceedings of the ACM International Conference on Multimodal Interaction (ICMI 23), 2023.
Gumilar, I. et al. Connecting the brains via virtual eyes Eye-gaze directions and inter-brain synchrony in VR. CHI Extended Abstracts (CHI EA 21), ACM, 2021.
He, Z. et al. Understanding user immersion in online short video interaction. Proceedings of the ACM International Conference on Information and Knowledge Management (CIKM 23), 2023.
Hirway, A. Qiao, Y. and Murray, N. A quality of experience and visual attention evaluation for 360 videos with non-spatial and spatial audio. ACM Transactions on Multimedia Computing, Communications, and Applications, 20(9), 2024.
Whitmore, N. W. et al. Improving attention using wearables via haptic and multimodal rhythmic stimuli. CHI Conference on Human Factors in Computing Systems (CHI 24), ACM, 2024.
Ferrari, A. et al. Using voice and biofeedback to predict user engagement during product feedback interviews. ACM Transactions on Software Engineering and Methodology, 33(4), 2024.
Zaman, A. et al. BraidFlow A flow-annotated dataset of kumihimo braidmaking activity. Designing Interactive Systems Conference (DIS 23), ACM, 2023.
Amores Fernandez, J. Mehra, N. Rasch, B. and Maes, P. Olfactory wearables for mobile targeted memory reactivation. CHI Conference on Human Factors in Computing Systems (CHI 23), ACM, 2023.
Torres, A. et al. Eye-tracking as a measure of cognitive load and attention in immersive virtual environments A systematic review. Frontiers in Virtual Reality, 5:1430012, 2024.
Kucuk, S. Goktas, Y. and Yilmaz, R. Exploring physiological signals and learning outcomes in VR-supported education A systematic review. Computers & Education, 198:104726, 2024.
Wang, F. and Yin, J. Emotion-aware adaptive VR environments using multimodal physiological feedback. IEEE Transactions on Affective Computing, 15(3):511–523, 2024.
Katsigiannis, S. and Ramzan, N. DREAMER A database for emotion recognition through EEG and ECG signals from wireless low-cost off-the-shelf devices. IEEE Journal of Biomedical and Health Informatics, 22(1):98–107, 2018.
Mottelson, A. et al. Data-driven virtual reality applications using affective computing. Proceedings of the 2021 ACM Symposium on Virtual Reality Software and Technology (VRST 21), 2021.
Peixoto, B. Rodrigues, J. and Bessa, M. Physiological computing frameworks for affect recognition in immersive environments. Sensors, 21(22):7504, 2021.
Thammasan, N. Fukui, K. and Numao, M. Continuous EEG-based emotion recognition in VR environments with virtual agents. Frontiers in Neuroscience, 14:618754, 2020.
Huang, X. et al. Combining EEG and eye tracking for attention and workload evaluation in immersive virtual reality. IEEE Transactions on Human-Machine Systems, 53(1):80–90, 2023.
Gashi, S. et al. Brain-computer interfaces for affective feedback in VR games. Entertainment Computing, 43:100494, 2022.
Kreibig, S. D. Autonomic nervous system activity in emotion a review. Biological Psychology, 84(3):394–421, 2010.
Fairclough, S. H. and Gilleade, K. R. Construction of the biocybernetic loop theory, methods and applications. IEEE Transactions on Human-Machine Systems, 42(6):839–853, 2012.
Rebenitsch, L. and Owen, C. Causes and consequences of cybersickness in virtual reality. ACM Computing Surveys, 50(4):68, 2017.
Stanney, K. M. et al. Identifying causes of and solutions for cybersickness in immersive technology a comprehensive review. International Journal of Human–Computer Interaction, 36(17):1658–1682, 2020.
Weech, S. Kenny, S. and Barnett-Cowan, M. Presence and cybersickness in virtual reality are negatively related a review. Frontiers in Psychology, 10:158, 2019.
Grassini, S. and Laumann, K. Are modern head-mounted displays sexist exploring gender differences in virtual reality cybersickness susceptibility. Virtual Reality, 24:1–8, 2020.
Liu, J. et al. Adaptive motion-to-photon latency reduction for VR sickness mitigation. IEEE Transactions on Visualization and Computer Graphics, 29(4):1903–1915, 2023.
Dennison, M. S. Wisti, A. Z. and D’Zmura, M. Use of physiological signals to predict cybersickness. Displays, 44:42–52, 2016.
Weidner, F. et al. Real-time physiological measurement in VR-based stress experiments. Frontiers in Virtual Reality, 3:921254, 2022.
Toet, A. and van Erp, J. B. F. The psychophysiology of immersive virtual reality. Frontiers in Virtual Reality, 2:695342, 2021.
Marin-Morales, J. et al. Affective computing in virtual reality emotion recognition from brain and heart rate signals using machine learning. Sensors, 18(12):4082, 2018.
Gomez, P. et al. Measuring affective responses in virtual reality comparison between physiological and self-report measures. Frontiers in Virtual Reality, 4:1171138, 2023.
Vlemincx, E. et al. Respiratory variability and stress exploring breathing as a physiological index of emotional state. Biological Psychology, 129:86–97, 2017.
Lazarus, R. S. Emotion and Adaptation. Oxford University Press, 1991.
Ekman, P. An argument for basic emotions. Cognition and Emotion, 6(3-4):169–200, 1992.
Russell, J. A. Core affect and the psychological construction of emotion. Psychological Review, 110(1):145–172, 2003.
Picard, R. W. Affective Computing. MIT Press, 1997.
Mandryk, R. L. and Atkins, M. S. A fuzzy physiological approach for continuously modeling emotion during interaction with play technologies. International Journal of Human–Computer Studies, 65(4):329–347, 2007.
Kim, J. and Andre, E. Emotion recognition based on physiological changes in music listening. IEEE Transactions on Pattern Analysis and Machine Intelligence, 30(12):2067–2083, 2008.
D’mello, S. and Kory, J. A review and meta-analysis of multimodal affect detection systems. ACM Computing Surveys, 47(3):43, 2015.
Calvo, R. A. and D’Mello, S. A. Affect detection an interdisciplinary review of models, methods, and their applications. IEEE Transactions on Affective Computing, 1(1):18–37, 2010.
Canento, F. et al. Psychophysiological indicators for user modeling in affective interactive environments. IEEE Transactions on Affective Computing, 4(4):398–411, 2013.
Moghimi, M. et al. Predicting affective states using multimodal physiological data in virtual environments. Sensors, 23(3):1212, 2023.
Katsis, C. D. et al. Emotion recognition in car racing drivers using biosignals. Expert Systems with Applications, 38(6):3897–3904, 2011.
Jerritta, S. et al. Physiological signals for emotion recognition in affective computing a review. IEEE Sensors Journal, 12(11):1974–1987, 2012.
Koelstra, S. et al. DEAP a database for emotion analysis using physiological signals. IEEE Transactions on Affective Computing, 3(1):18–31, 2012.
Haouij, N. et al. PhysioZoo a novel physiological dataset for emotion recognition. PLOS ONE, 17(5):e0267653, 2022.
Wagner, J. et al. Machine learning for emotion recognition from physiological signals toward affective computing in virtual environments. IEEE Transactions on Affective Computing, 11(4):509–523, 2020.
Zhao, Y. et al. Deep affective computing for emotion recognition from multimodal physiological signals. Neurocomputing, 517:302–313, 2023.
Duan, J. et al. Fusion of EEG and eye-tracking data for affective computing in immersive virtual environments. Frontiers in Neuroscience, 17:1018327, 2023.
Liu, X. et al. Emotion classification in immersive VR using deep learning on multimodal physiological data. IEEE Access, 10:92314–92327, 2022.
Correa, J. A. et al. Emotion recognition in VR through galvanic skin response and eye movement analysis. Virtual Reality, 28(2):357–370, 2024.
Kim, H. et al. Affective state estimation from physiological signals in immersive environments. Sensors, 22(21):8229, 2022.
Yang, C. et al. Deep multimodal emotion recognition framework based on EEG and physiological data. IEEE Transactions on Neural Networks and Learning Systems, 35(5):6158–6171, 2024.
Chanel, G. Krone, T. and Pun, T. Emotion assessment based on physiological signals and fuzzy logic for VR users. Applied Soft Computing, 13(9):3959–3971, 2013.
Bhattacharyya, S. and Konar, A. Physiological computing for human–computer interaction applications. Cognitive Computation, 13(1):211–231, 2021.
Ariza, A. et al. Neural network models for emotion prediction from physiological data in VR. IEEE Access, 11:90118–90130, 2023.
Li, T. et al. Physiological-based metrics for adaptive VR user interfaces a survey. ACM Computing Surveys, 56(8):166, 2024.
Dey, A. and Schmidt, A. Context-aware computing for the design of adaptive virtual environments. Pervasive and Mobile Computing, 101:101670, 2024.
Yamazaki, T. et al. Emotion recognition based on pupil dilation and skin conductance in immersive settings. Frontiers in Psychology, 13:1054251, 2022.
Huang, R. et al. Human emotion understanding based on physiological data a deep learning perspective. IEEE Transactions on Affective Computing, 15(2):277–290, 2024.
Balters, S. and Steinert, M. Capturing emotion rethinking paradigms for physiological data in immersive systems. Frontiers in Human Neuroscience, 16:836829, 2022.
Jiang, Y. et al. Real-time affective adaptation in VR using physiological data a reinforcement learning approach. IEEE Access, 12:150221–150235, 2024.
Vogel, B. and Grafe, S. Physiological computing in immersive learning environments systematic review and implications for design. Computers & Education, 205:104868, 2024.
Goncalves, G. et al. Biofeedback in immersive learning a review of physiological signal integration in VR training. IEEE Transactions on Learning Technologies, 17(4):543–559, 2024.
Jensen, L. and Konradsen, F. A review of the use of virtual reality head-mounted displays in education and training. Education and Information Technologies, 23:1515–1529, 2018.
Jiang, W. et al. Integrating multimodal physiological sensors in immersive systems for real-time feedback. Sensors, 23(4):1882, 2023.
Nourbakhsh, N. et al. Wearable sensors for physiological monitoring in immersive environments a review. IEEE Sensors Journal, 23(12):11544–11562, 2023.
Serrano, B. and Blesius, C. Physiological synchrony in collaborative VR systems challenges and opportunities. Frontiers in Virtual Reality, 5:1353229, 2024.
Marques, A. et al. Bio-adaptive feedback for improving user performance in VR-based rehabilitation. Computers in Biology and Medicine, 163:107145, 2023.
Keller, C. et al. Adaptive training systems in VR using real-time physiological data. IEEE Transactions on Human–Machine Systems, 54(2):178–191, 2024.
Ferreira, F. et al. Personalized virtual learning systems using physiological and behavioral data a systematic mapping. Education and Information Technologies, 29(2):845–865, 2024.
Freeman, D. et al. Virtual reality in the assessment, understanding, and treatment of mental health disorders. Psychological Medicine, 47(14):2393–2400, 2017.
Maples-Keller, J. L. et al. The use of virtual reality technology in the treatment of anxiety and other psychiatric disorders. Harvard Review of Psychiatry, 25(3):103–113, 2017.
Reger, G. M. and Gahm, G. A. Virtual reality exposure therapy for active duty soldiers. Journal of Clinical Psychology, 64(8):940–946, 2008.
Kahlon, S. et al. Physiological synchrony and presence in VR a meta-analytic study. Frontiers in Virtual Reality, 4:1102957, 2023.
Kassner, M. et al. PupilLabs Eye Tracker Open source tool for real-time gaze analysis in immersive environments. Behavior Research Methods, 46(4):1083–1099, 2014.
Pekala, R. J. and Wenger, C. D. Psychophysiological correlates of hypnotic susceptibility. Journal of Abnormal Psychology, 98(4):431–439, 1989.
Winkler, I. et al. Integrating biosignals for cognitive workload assessment in VR tasks. IEEE Access, 10:59321–59336, 2022.
Schmidt, E. A. et al. EEG-based mental workload metrics in immersive simulation training. Frontiers in Neuroergonomics, 2:651423, 2021.
Chen, J. et al. Physiological measures for adaptive gaming systems. ACM Computing Surveys, 55(12):245, 2023.
Sousa, R. et al. Cognitive state assessment using multimodal physiological data. Sensors, 22(8):3119, 2022.
Kim, Y. et al. EEG-based workload evaluation for complex control tasks in mixed reality. Frontiers in Virtual Reality, 3:943265, 2022.
Bal, P. et al. Towards affect-driven adaptive VR interfaces leveraging biosignal data. Computers & Graphics, 115:1–11, 2023.
Herrera, F. et al. Physiological correlates of presence in immersive virtual environments. Frontiers in Psychology, 10:1296, 2019.
Miller, M. et al. Influence of environmental cues on physiological arousal in VR. Presence Teleoperators and Virtual Environments, 31(2):95–111, 2022.
Lee, J. et al. Combining EEG and GSR for predicting user engagement in VR learning. IEEE Transactions on Learning Technologies, 16(1):44–58, 2023.
Alonso, A. et al. The relationship between presence and physiological arousal during virtual exposure therapy. Cyberpsychology, Behavior, and Social Networking, 25(6):389–398, 2022.
Perez, J. et al. Physiological measures in VR games towards adaptive difficulty adjustment. Entertainment Computing, 46:100547, 2023.
Martinez, G. et al. Real-time heart rate and EEG feedback to regulate anxiety in immersive therapy. IEEE Transactions on Affective Computing, 15(5):689–700, 2024.
Yu, H. et al. Eye-tracking and ECG correlations for attention and emotion recognition in VR driving simulators. IEEE Access, 12:62152–62167, 2024.
Lopez, D. et al. Predictive modeling of cybersickness using multimodal physiological features. ACM Transactions on Applied Perception, 20(2):12, 2023.
Yoon, J. et al. Exploring empathy through physiological synchrony in shared VR experiences. Frontiers in Virtual Reality, 5:1362457, 2024.
Karacan, H. et al. Using physiological sensors for adaptive storytelling in immersive experiences. Computers in Human Behavior, 145:107833, 2024.
Gonzalez, V. et al. Real-time stress detection using HRV and GSR in immersive training. Sensors, 23(9):4122, 2023.
Torres, J. et al. Influence of auditory stimuli on physiological arousal in immersive soundscapes. Frontiers in Human Neuroscience, 17:1008276, 2023.
Farhan, A. et al. Multimodal physiological signal fusion for emotion recognition in immersive systems. IEEE Transactions on Affective Computing, 14(6):1550–1563, 2023.
Ramos, P. et al. Integrating biosignal-driven feedback in VR exergaming. Entertainment Computing, 52:100664, 2024.
Palacios, M. et al. EEG-based detection of attention lapses in VR simulations. Virtual Reality, 28(1):167–182, 2024.
Lozano, A. et al. Neural and physiological data for presence estimation in immersive environments. IEEE Transactions on Visualization and Computer Graphics, 30(2):245–258, 2024.
Bessa, M. et al. Physiological measures and emotional response in immersive storytelling a systematic review. Frontiers in Virtual Reality, 6:1470023, 2024.
Radianti, J. Majchrzak, T. A. Fromm, J. and Wohlgenannt, I. A systematic review of immersive virtual reality applications for higher education Design elements, lessons learned, and research agenda. Computers & Education, 147, 2020, 103778.
Goncalves, G. Coelho, H. M. Pedro, M. and Bessa, M. Systematic review of comparative studies of the impact of realism in immersive virtual experiences. ACM Computing Surveys, 55(6):115, 2022.
Putze, S. Alexandrovsky, D. Putze, F. Hoffner, S. Smeddinck, J. D. and Malaka, R. Breaking the experience Effects of questionnaires in VR user studies. CHI Conference on Human Factors in Computing Systems (CHI 20), 2020.
Safikhani, S. Holly, M. Kainz, A. and Pirker, J. The influence of in-VR questionnaire design on the user experience. ACM Symposium on Virtual Reality Software and Technology (VRST 21), 2021.
Certicky, M. Sincak, P. Magyar, G. Vascak, J. and Cavallo, F. Psychophysiological indicators for modeling user experience in interactive digital entertainment. Sensors, 19:989, 2019.
Holloman, A. K. and Crawford, C. S. Defining scents A systematic literature review of olfactory-based computing systems. ACM Transactions on Multimedia Computing, Communications, and Applications, 18(1):15, 2022.
Cartaxo, B. Pinto, G. and Soares, S. Rapid reviews in software engineering. In Contemporary Empirical Methods in Software Engineering, 357–384. Springer, Cham, 2020.
Santos, C. M. C. Pimenta, C. A. M. and Nobre, M. R. C. A estrategia PICO para a construcao da pergunta de pesquisa e busca de evidencias. Revista Latino-Americana de Enfermagem, 15(3):508–511, 2007.
Tabbaa, L. et al. VREED Virtual Reality Emotion Recognition Dataset using Eye Tracking and Physiological Measures. Proceedings of the ACM Interactive, Mobile, Wearable and Ubiquitous Technologies, 5(4), 2022.
Gnacek, M. et al. Heart rate detection from the supratrochlear vessels using a virtual reality headset integrated PPG sensor. Proceedings of the ACM International Conference on Multimodal Interaction (ICMI 20 Companion), 2020.
Chiossi, F. Ou, C. and Mayer, S. Optimizing visual complexity for physiologically-adaptive VR systems Evaluating a multimodal dataset using ECG and EEG features. Proceedings of the International Conference on Advanced Visual Interfaces (AVI 24), ACM, 2024.
Dietz, D. et al. Walk this beam Impact of different balance assistance strategies and height exposure on performance and physiological arousal in VR. ACM Symposium on Virtual Reality Software and Technology (VRST 22), 2022.
Samery, J. et al. Collaborative cooking in VR Effects of network distortion in multi-user virtual environments. ACM Multimedia Systems Conference, 509–515, 2024.
Miner, N. et al. Stairway to heaven A gamified VR journey for breath awareness. CHI Conference on Human Factors in Computing Systems (CHI 24), ACM, 2024.
Delachambre, J. et al. AMD Journee A patient co-designed VR experience to raise awareness towards the impact of AMD on social interactions. ACM International Conference on Interactive Media Experiences (IMX 24), 2024.
Terfurth, L. Gramann, K. and Gehrke, L. Decoding realism of virtual objects Exploring behavioral and ocular reactions to inaccurate interaction feedback. ACM Transactions on Computer–Human Interaction, 31(3), 2024.
Gupta, K. Lazarevic, J. Pai, Y. S. and Billinghurst, M. AffectivelyVR Towards VR personalized emotion recognition. ACM Symposium on Virtual Reality Software and Technology (VRST 20), 2020.
Barathi, S. C. Proulx, M. O’Neill, E. and Lutteroth, C. Affect recognition using psychophysiological correlates in high-intensity VR exergaming. CHI Conference on Human Factors in Computing Systems (CHI 20), ACM, 2020.
Li, Y. Ch’ng, E. and Cobb, S. Factors influencing engagement in hybrid virtual and augmented reality. ACM Transactions on Computer–Human Interaction, 30(4), 2023.
Arjun, S. Hebbar, A. Sanjana, and Biswas, P. VR cognitive load dashboard for flight simulator. Proceedings of the ACM Symposium on Eye Tracking Research and Applications (ETRA 22), 2022.
Ahmad, M. and Alzahrani, A. Crucial clues Investigating psychophysiological behaviors for measuring trust in human-robot interaction. Proceedings of the ACM International Conference on Multimodal Interaction (ICMI 23), 2023.
Gumilar, I. et al. Connecting the brains via virtual eyes Eye-gaze directions and inter-brain synchrony in VR. CHI Extended Abstracts (CHI EA 21), ACM, 2021.
He, Z. et al. Understanding user immersion in online short video interaction. Proceedings of the ACM International Conference on Information and Knowledge Management (CIKM 23), 2023.
Hirway, A. Qiao, Y. and Murray, N. A quality of experience and visual attention evaluation for 360 videos with non-spatial and spatial audio. ACM Transactions on Multimedia Computing, Communications, and Applications, 20(9), 2024.
Whitmore, N. W. et al. Improving attention using wearables via haptic and multimodal rhythmic stimuli. CHI Conference on Human Factors in Computing Systems (CHI 24), ACM, 2024.
Ferrari, A. et al. Using voice and biofeedback to predict user engagement during product feedback interviews. ACM Transactions on Software Engineering and Methodology, 33(4), 2024.
Zaman, A. et al. BraidFlow A flow-annotated dataset of kumihimo braidmaking activity. Designing Interactive Systems Conference (DIS 23), ACM, 2023.
Amores Fernandez, J. Mehra, N. Rasch, B. and Maes, P. Olfactory wearables for mobile targeted memory reactivation. CHI Conference on Human Factors in Computing Systems (CHI 23), ACM, 2023.
Torres, A. et al. Eye-tracking as a measure of cognitive load and attention in immersive virtual environments A systematic review. Frontiers in Virtual Reality, 5:1430012, 2024.
Kucuk, S. Goktas, Y. and Yilmaz, R. Exploring physiological signals and learning outcomes in VR-supported education A systematic review. Computers & Education, 198:104726, 2024.
Wang, F. and Yin, J. Emotion-aware adaptive VR environments using multimodal physiological feedback. IEEE Transactions on Affective Computing, 15(3):511–523, 2024.
Katsigiannis, S. and Ramzan, N. DREAMER A database for emotion recognition through EEG and ECG signals from wireless low-cost off-the-shelf devices. IEEE Journal of Biomedical and Health Informatics, 22(1):98–107, 2018.
Mottelson, A. et al. Data-driven virtual reality applications using affective computing. Proceedings of the 2021 ACM Symposium on Virtual Reality Software and Technology (VRST 21), 2021.
Peixoto, B. Rodrigues, J. and Bessa, M. Physiological computing frameworks for affect recognition in immersive environments. Sensors, 21(22):7504, 2021.
Thammasan, N. Fukui, K. and Numao, M. Continuous EEG-based emotion recognition in VR environments with virtual agents. Frontiers in Neuroscience, 14:618754, 2020.
Huang, X. et al. Combining EEG and eye tracking for attention and workload evaluation in immersive virtual reality. IEEE Transactions on Human-Machine Systems, 53(1):80–90, 2023.
Gashi, S. et al. Brain-computer interfaces for affective feedback in VR games. Entertainment Computing, 43:100494, 2022.
Kreibig, S. D. Autonomic nervous system activity in emotion a review. Biological Psychology, 84(3):394–421, 2010.
Fairclough, S. H. and Gilleade, K. R. Construction of the biocybernetic loop theory, methods and applications. IEEE Transactions on Human-Machine Systems, 42(6):839–853, 2012.
Rebenitsch, L. and Owen, C. Causes and consequences of cybersickness in virtual reality. ACM Computing Surveys, 50(4):68, 2017.
Stanney, K. M. et al. Identifying causes of and solutions for cybersickness in immersive technology a comprehensive review. International Journal of Human–Computer Interaction, 36(17):1658–1682, 2020.
Weech, S. Kenny, S. and Barnett-Cowan, M. Presence and cybersickness in virtual reality are negatively related a review. Frontiers in Psychology, 10:158, 2019.
Grassini, S. and Laumann, K. Are modern head-mounted displays sexist exploring gender differences in virtual reality cybersickness susceptibility. Virtual Reality, 24:1–8, 2020.
Liu, J. et al. Adaptive motion-to-photon latency reduction for VR sickness mitigation. IEEE Transactions on Visualization and Computer Graphics, 29(4):1903–1915, 2023.
Dennison, M. S. Wisti, A. Z. and D’Zmura, M. Use of physiological signals to predict cybersickness. Displays, 44:42–52, 2016.
Weidner, F. et al. Real-time physiological measurement in VR-based stress experiments. Frontiers in Virtual Reality, 3:921254, 2022.
Toet, A. and van Erp, J. B. F. The psychophysiology of immersive virtual reality. Frontiers in Virtual Reality, 2:695342, 2021.
Marin-Morales, J. et al. Affective computing in virtual reality emotion recognition from brain and heart rate signals using machine learning. Sensors, 18(12):4082, 2018.
Gomez, P. et al. Measuring affective responses in virtual reality comparison between physiological and self-report measures. Frontiers in Virtual Reality, 4:1171138, 2023.
Vlemincx, E. et al. Respiratory variability and stress exploring breathing as a physiological index of emotional state. Biological Psychology, 129:86–97, 2017.
Lazarus, R. S. Emotion and Adaptation. Oxford University Press, 1991.
Ekman, P. An argument for basic emotions. Cognition and Emotion, 6(3-4):169–200, 1992.
Russell, J. A. Core affect and the psychological construction of emotion. Psychological Review, 110(1):145–172, 2003.
Picard, R. W. Affective Computing. MIT Press, 1997.
Mandryk, R. L. and Atkins, M. S. A fuzzy physiological approach for continuously modeling emotion during interaction with play technologies. International Journal of Human–Computer Studies, 65(4):329–347, 2007.
Kim, J. and Andre, E. Emotion recognition based on physiological changes in music listening. IEEE Transactions on Pattern Analysis and Machine Intelligence, 30(12):2067–2083, 2008.
D’mello, S. and Kory, J. A review and meta-analysis of multimodal affect detection systems. ACM Computing Surveys, 47(3):43, 2015.
Calvo, R. A. and D’Mello, S. A. Affect detection an interdisciplinary review of models, methods, and their applications. IEEE Transactions on Affective Computing, 1(1):18–37, 2010.
Canento, F. et al. Psychophysiological indicators for user modeling in affective interactive environments. IEEE Transactions on Affective Computing, 4(4):398–411, 2013.
Moghimi, M. et al. Predicting affective states using multimodal physiological data in virtual environments. Sensors, 23(3):1212, 2023.
Katsis, C. D. et al. Emotion recognition in car racing drivers using biosignals. Expert Systems with Applications, 38(6):3897–3904, 2011.
Jerritta, S. et al. Physiological signals for emotion recognition in affective computing a review. IEEE Sensors Journal, 12(11):1974–1987, 2012.
Koelstra, S. et al. DEAP a database for emotion analysis using physiological signals. IEEE Transactions on Affective Computing, 3(1):18–31, 2012.
Haouij, N. et al. PhysioZoo a novel physiological dataset for emotion recognition. PLOS ONE, 17(5):e0267653, 2022.
Wagner, J. et al. Machine learning for emotion recognition from physiological signals toward affective computing in virtual environments. IEEE Transactions on Affective Computing, 11(4):509–523, 2020.
Zhao, Y. et al. Deep affective computing for emotion recognition from multimodal physiological signals. Neurocomputing, 517:302–313, 2023.
Duan, J. et al. Fusion of EEG and eye-tracking data for affective computing in immersive virtual environments. Frontiers in Neuroscience, 17:1018327, 2023.
Liu, X. et al. Emotion classification in immersive VR using deep learning on multimodal physiological data. IEEE Access, 10:92314–92327, 2022.
Correa, J. A. et al. Emotion recognition in VR through galvanic skin response and eye movement analysis. Virtual Reality, 28(2):357–370, 2024.
Kim, H. et al. Affective state estimation from physiological signals in immersive environments. Sensors, 22(21):8229, 2022.
Yang, C. et al. Deep multimodal emotion recognition framework based on EEG and physiological data. IEEE Transactions on Neural Networks and Learning Systems, 35(5):6158–6171, 2024.
Chanel, G. Krone, T. and Pun, T. Emotion assessment based on physiological signals and fuzzy logic for VR users. Applied Soft Computing, 13(9):3959–3971, 2013.
Bhattacharyya, S. and Konar, A. Physiological computing for human–computer interaction applications. Cognitive Computation, 13(1):211–231, 2021.
Ariza, A. et al. Neural network models for emotion prediction from physiological data in VR. IEEE Access, 11:90118–90130, 2023.
Li, T. et al. Physiological-based metrics for adaptive VR user interfaces a survey. ACM Computing Surveys, 56(8):166, 2024.
Dey, A. and Schmidt, A. Context-aware computing for the design of adaptive virtual environments. Pervasive and Mobile Computing, 101:101670, 2024.
Yamazaki, T. et al. Emotion recognition based on pupil dilation and skin conductance in immersive settings. Frontiers in Psychology, 13:1054251, 2022.
Huang, R. et al. Human emotion understanding based on physiological data a deep learning perspective. IEEE Transactions on Affective Computing, 15(2):277–290, 2024.
Balters, S. and Steinert, M. Capturing emotion rethinking paradigms for physiological data in immersive systems. Frontiers in Human Neuroscience, 16:836829, 2022.
Jiang, Y. et al. Real-time affective adaptation in VR using physiological data a reinforcement learning approach. IEEE Access, 12:150221–150235, 2024.
Vogel, B. and Grafe, S. Physiological computing in immersive learning environments systematic review and implications for design. Computers & Education, 205:104868, 2024.
Goncalves, G. et al. Biofeedback in immersive learning a review of physiological signal integration in VR training. IEEE Transactions on Learning Technologies, 17(4):543–559, 2024.
Jensen, L. and Konradsen, F. A review of the use of virtual reality head-mounted displays in education and training. Education and Information Technologies, 23:1515–1529, 2018.
Jiang, W. et al. Integrating multimodal physiological sensors in immersive systems for real-time feedback. Sensors, 23(4):1882, 2023.
Nourbakhsh, N. et al. Wearable sensors for physiological monitoring in immersive environments a review. IEEE Sensors Journal, 23(12):11544–11562, 2023.
Serrano, B. and Blesius, C. Physiological synchrony in collaborative VR systems challenges and opportunities. Frontiers in Virtual Reality, 5:1353229, 2024.
Marques, A. et al. Bio-adaptive feedback for improving user performance in VR-based rehabilitation. Computers in Biology and Medicine, 163:107145, 2023.
Keller, C. et al. Adaptive training systems in VR using real-time physiological data. IEEE Transactions on Human–Machine Systems, 54(2):178–191, 2024.
Ferreira, F. et al. Personalized virtual learning systems using physiological and behavioral data a systematic mapping. Education and Information Technologies, 29(2):845–865, 2024.
Freeman, D. et al. Virtual reality in the assessment, understanding, and treatment of mental health disorders. Psychological Medicine, 47(14):2393–2400, 2017.
Maples-Keller, J. L. et al. The use of virtual reality technology in the treatment of anxiety and other psychiatric disorders. Harvard Review of Psychiatry, 25(3):103–113, 2017.
Reger, G. M. and Gahm, G. A. Virtual reality exposure therapy for active duty soldiers. Journal of Clinical Psychology, 64(8):940–946, 2008.
Kahlon, S. et al. Physiological synchrony and presence in VR a meta-analytic study. Frontiers in Virtual Reality, 4:1102957, 2023.
Kassner, M. et al. PupilLabs Eye Tracker Open source tool for real-time gaze analysis in immersive environments. Behavior Research Methods, 46(4):1083–1099, 2014.
Pekala, R. J. and Wenger, C. D. Psychophysiological correlates of hypnotic susceptibility. Journal of Abnormal Psychology, 98(4):431–439, 1989.
Winkler, I. et al. Integrating biosignals for cognitive workload assessment in VR tasks. IEEE Access, 10:59321–59336, 2022.
Schmidt, E. A. et al. EEG-based mental workload metrics in immersive simulation training. Frontiers in Neuroergonomics, 2:651423, 2021.
Chen, J. et al. Physiological measures for adaptive gaming systems. ACM Computing Surveys, 55(12):245, 2023.
Sousa, R. et al. Cognitive state assessment using multimodal physiological data. Sensors, 22(8):3119, 2022.
Kim, Y. et al. EEG-based workload evaluation for complex control tasks in mixed reality. Frontiers in Virtual Reality, 3:943265, 2022.
Bal, P. et al. Towards affect-driven adaptive VR interfaces leveraging biosignal data. Computers & Graphics, 115:1–11, 2023.
Herrera, F. et al. Physiological correlates of presence in immersive virtual environments. Frontiers in Psychology, 10:1296, 2019.
Miller, M. et al. Influence of environmental cues on physiological arousal in VR. Presence Teleoperators and Virtual Environments, 31(2):95–111, 2022.
Lee, J. et al. Combining EEG and GSR for predicting user engagement in VR learning. IEEE Transactions on Learning Technologies, 16(1):44–58, 2023.
Alonso, A. et al. The relationship between presence and physiological arousal during virtual exposure therapy. Cyberpsychology, Behavior, and Social Networking, 25(6):389–398, 2022.
Perez, J. et al. Physiological measures in VR games towards adaptive difficulty adjustment. Entertainment Computing, 46:100547, 2023.
Martinez, G. et al. Real-time heart rate and EEG feedback to regulate anxiety in immersive therapy. IEEE Transactions on Affective Computing, 15(5):689–700, 2024.
Yu, H. et al. Eye-tracking and ECG correlations for attention and emotion recognition in VR driving simulators. IEEE Access, 12:62152–62167, 2024.
Lopez, D. et al. Predictive modeling of cybersickness using multimodal physiological features. ACM Transactions on Applied Perception, 20(2):12, 2023.
Yoon, J. et al. Exploring empathy through physiological synchrony in shared VR experiences. Frontiers in Virtual Reality, 5:1362457, 2024.
Karacan, H. et al. Using physiological sensors for adaptive storytelling in immersive experiences. Computers in Human Behavior, 145:107833, 2024.
Gonzalez, V. et al. Real-time stress detection using HRV and GSR in immersive training. Sensors, 23(9):4122, 2023.
Torres, J. et al. Influence of auditory stimuli on physiological arousal in immersive soundscapes. Frontiers in Human Neuroscience, 17:1008276, 2023.
Farhan, A. et al. Multimodal physiological signal fusion for emotion recognition in immersive systems. IEEE Transactions on Affective Computing, 14(6):1550–1563, 2023.
Ramos, P. et al. Integrating biosignal-driven feedback in VR exergaming. Entertainment Computing, 52:100664, 2024.
Palacios, M. et al. EEG-based detection of attention lapses in VR simulations. Virtual Reality, 28(1):167–182, 2024.
Lozano, A. et al. Neural and physiological data for presence estimation in immersive environments. IEEE Transactions on Visualization and Computer Graphics, 30(2):245–258, 2024.
Bessa, M. et al. Physiological measures and emotional response in immersive storytelling a systematic review. Frontiers in Virtual Reality, 6:1470023, 2024.
Publicado
30/09/2025
Como Citar
MELO, Edel Da Silva; TREVISAN, Daniela Gorski; SAADE, Débora Christina Muchaluat.
Reviewing the Role of Physiological Data in Immersive Environments: A Rapid Systematic Review. In: SIMPÓSIO DE REALIDADE VIRTUAL E AUMENTADA (SVR), 27. , 2025, Salvador/BA.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2025
.
p. 331-342.
