Towards Emotion-Aware Olfactory VR: Preliminary Results of a Multimodal Biofeedback Approach for Sense of Presence
Resumo
This work-in-progress investigates the application of artificial intelligence (AI) and multimodal biofeedback for emotion recognition and objective validation of the sense of presence in virtual reality (VR). Using synchronized physiological signals, such as eye tracking, electrocardiogram (ECG), and galvanic skin response (GSR) from the VREED dataset, we trained models based on Russell’s affective model. Preliminary results indicate that eye-tracking metrics are particularly effective in emotion prediction and also show strong potential for objectively assessing the users’ subjective sense of presence. These models are being integrated into the SOFIA Framework as a foundation for adaptive and dynamic virtual environments. Although this stage of the project does not yet include olfactory experiments, the findings provide the basis for future integration of scent delivery to further enhance realism and user immersion. Future work will also explore real-time validation tests, integrate EEG data for deeper analysis, and expand the repertoire of olfactory stimuli.Referências
Adhanom, I. B., MacNeilage, P., and Folmer, E. (2023). Eye tracking in virtual reality: A broad review of applications and challenges. Virtual Reality, 27(2):1481–1505.
Ahmad, Z., Rabbani, S., Zafar, M. R., Ishaque, S., Krishnan, S., and Khan, N. (2023). Multilevel stress assessment from ecg in a virtual reality environment using multimodal fusion. IEEE Sensors Journal, 23(23):29559–29570.
Archer, N. S., Bluff, A., Eddy, A., Nikhil, C. K., Hazell, N., Frank, D., and Johnston, A. (2022). Odour enhances the sense of presence in a virtual reality environment. PLoS ONE, 17(3):1–20.
Kim, J., Park, J., and Park, J. (2020). Development of a statistical model to classify driving stress levels using galvanic skin responses. Human Factors and Ergonomics in Manufacturing & Service Industries, 30(5):321–328.
Lopes, M. K. S. and Falk, T. H. (2024). Audio-visual-olfactory immersive digital nature exposure for stress and anxiety reduction: A systematic review on systems, outcomes, and challenges. Frontiers in Virtual Reality, 5.
Lundberg, S. M. and Lee, S.-I. (2017). A unified approach to interpreting model predictions. In Advances in Neural Information Processing Systems (NeurIPS), volume 30, pages 4765–4774.
Niedenthal, S., Fredborg, W., Lunden, P., Ehrndal, M., and Olofsson, J. K. (2023). A graspable olfactory display for virtual reality. International Journal of Human-Computer Studies, 169:102928.
Pratviel, Y., Bouny, P., and Deschodt-Arsac, V. (2024). Immersion in a relaxing virtual reality environment is associated with similar effects on stress and anxiety as heart rate variability biofeedback. Frontiers in Virtual Reality, 5.
Riva, G., Wiederhold, B. K., and Mantovani, F. (2019). Neuroscience of virtual reality: From virtual exposure to embodied medicine. Cyberpsychology, Behavior, and Social Networking, 22(1):82–96.
Russell, J. (1980). A circumplex model of affect. Journal of Personality and Social Psychology, 39:1161–1178.
Slater, M., Banakou, D., Beacco, A., Gallego, J., Macia-Varela, F., and Oliva, R. (2022). A separate reality: An update on place illusion and plausibility in virtual reality. Frontiers in Virtual Reality, 3:914392.
Steed, A., Archer, D., Izzouzi, L., Numan, N., Shapiro, K., Swapp, D., Lammiman, D., and Lindeman, R. W. (2023). Immersive competence and immersive literacy: Exploring how users learn about immersive experiences. Frontiers in Virtual Reality, 4.
Tabbaa, L., Searle, R., Bafti, S. M., Hossain, M. M., Intarasisrisawat, J., Glancy, M., and Ang, C. S. (2021). Vreed: Virtual reality emotion recognition dataset using eye tracking and physiological measures. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, 5(4):1–20. Article 178.
Yildirim, M., Globa, A., Gocer, O., and Brambilla, A. (2025). Digital smell technologies for the built environment: Evaluating human responses to multisensory stimuli in immersive virtual reality. Building and Environment, 271:112608.
Ahmad, Z., Rabbani, S., Zafar, M. R., Ishaque, S., Krishnan, S., and Khan, N. (2023). Multilevel stress assessment from ecg in a virtual reality environment using multimodal fusion. IEEE Sensors Journal, 23(23):29559–29570.
Archer, N. S., Bluff, A., Eddy, A., Nikhil, C. K., Hazell, N., Frank, D., and Johnston, A. (2022). Odour enhances the sense of presence in a virtual reality environment. PLoS ONE, 17(3):1–20.
Kim, J., Park, J., and Park, J. (2020). Development of a statistical model to classify driving stress levels using galvanic skin responses. Human Factors and Ergonomics in Manufacturing & Service Industries, 30(5):321–328.
Lopes, M. K. S. and Falk, T. H. (2024). Audio-visual-olfactory immersive digital nature exposure for stress and anxiety reduction: A systematic review on systems, outcomes, and challenges. Frontiers in Virtual Reality, 5.
Lundberg, S. M. and Lee, S.-I. (2017). A unified approach to interpreting model predictions. In Advances in Neural Information Processing Systems (NeurIPS), volume 30, pages 4765–4774.
Niedenthal, S., Fredborg, W., Lunden, P., Ehrndal, M., and Olofsson, J. K. (2023). A graspable olfactory display for virtual reality. International Journal of Human-Computer Studies, 169:102928.
Pratviel, Y., Bouny, P., and Deschodt-Arsac, V. (2024). Immersion in a relaxing virtual reality environment is associated with similar effects on stress and anxiety as heart rate variability biofeedback. Frontiers in Virtual Reality, 5.
Riva, G., Wiederhold, B. K., and Mantovani, F. (2019). Neuroscience of virtual reality: From virtual exposure to embodied medicine. Cyberpsychology, Behavior, and Social Networking, 22(1):82–96.
Russell, J. (1980). A circumplex model of affect. Journal of Personality and Social Psychology, 39:1161–1178.
Slater, M., Banakou, D., Beacco, A., Gallego, J., Macia-Varela, F., and Oliva, R. (2022). A separate reality: An update on place illusion and plausibility in virtual reality. Frontiers in Virtual Reality, 3:914392.
Steed, A., Archer, D., Izzouzi, L., Numan, N., Shapiro, K., Swapp, D., Lammiman, D., and Lindeman, R. W. (2023). Immersive competence and immersive literacy: Exploring how users learn about immersive experiences. Frontiers in Virtual Reality, 4.
Tabbaa, L., Searle, R., Bafti, S. M., Hossain, M. M., Intarasisrisawat, J., Glancy, M., and Ang, C. S. (2021). Vreed: Virtual reality emotion recognition dataset using eye tracking and physiological measures. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, 5(4):1–20. Article 178.
Yildirim, M., Globa, A., Gocer, O., and Brambilla, A. (2025). Digital smell technologies for the built environment: Evaluating human responses to multisensory stimuli in immersive virtual reality. Building and Environment, 271:112608.
Publicado
30/09/2025
Como Citar
SILVA JUNIOR, Jonas G. da; SILVA, Meryck F. B. da; MOTTIN, Melina; ANDRADE, Carolina H.; GALVÃO FILHO, Arlindo Rodrigues.
Towards Emotion-Aware Olfactory VR: Preliminary Results of a Multimodal Biofeedback Approach for Sense of Presence. In: WORKSHOP DE TRABALHOS EM ANDAMENTO - SIMPÓSIO DE REALIDADE VIRTUAL E AUMENTADA (SVR), 27. , 2025, Salvador/BA.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2025
.
p. 227-233.
DOI: https://doi.org/10.5753/svr_estendido.2025.15755.
