Towards Emotion-Aware Olfactory VR: Preliminary Results of a Multimodal Biofeedback Approach for Sense of Presence - Silva Junior, Jonas G. da; Silva, Meryck F. B. da; Mottin, Melina; Andrade, Carolina H.; Galvão Filho, Arlindo Rodrigues (Anais Estendidos do Simpósio de Realidade Virtual e Aumentada (SVR), 2025)
This work-in-progress investigates the application of artificial intelligence (AI) and multimodal biofeedback for emotion recognition and objective validation of the sense of presence in virtual reality (VR). Using synchronized physiological signals, such as eye tracking, electrocardiogram (ECG), and galvanic skin response (GSR) from the VREED dataset, we trained models based on Russell’s affective model. Preliminary results indicate that eye-tracking metrics are particularly effective in emotion prediction and also show strong potential for objectively assessing the users’ subjective sense of presence. These models are being integrated into the SOFIA Framework as a foundation for adaptive and dynamic virtual environments. Although this stage of the project does not yet include olfactory experiments, the findings provide the basis for future integration of scent delivery to further enhance realism and user immersion. Future work will also explore real-time validation tests, integrate EEG data for deeper analysis, and expand the repertoire of olfactory stimuli.