Analysis of Cybersickness through Biosignals: an approach with Symbolic Machine Learning
Resumo
Cybersickness represents one of the main obstacles to the use of Virtual Reality, often triggered by the use of Head-mounted Display devices. The symptoms associated with cybersickness can vary among individuals and include nausea, dizziness, eye strain, and headache, which may persist for minutes or even hours after exposure to Virtual Reality. According to the literature, cybersickness has a considerable impact on physiological signals such as delta waves in the Electroencephalogram; Heart Rate and Heart Rate Variability, derived from the Electrocardiogram; Electrodermal Activity; and Electrogastrography, all of which show a significant correlation with this condition. In this study, we investigated the use of biosignals to identify the possible causes associated with cybersickness in Virtual Reality. Our main hypothesis is that the combination of quantitative and subjective assessments, combined with Symbolic Machine Learning techniques, is effective in creating a ranking of the main causative/indicative factors of this condition. The results of this study highlight significant contributions to the understanding of factors influencing cybersickness symptoms. Statistical analyses confirmed the relationship between physiological changes and cybersickness symptoms. By including biosignals in our model, we achieved a significant gain, with an AUC of 0.95. The rankings of the main factors, both for the model without and with the inclusion of biosignals, confirmed previous research described in the literature. To the best of our knowledge, this is the first work to employ Symbolic Machine Learning models to detect the causes of cybersickness and generate a ranking of the most relevant factors.
Palavras-chave:
Virtual Reality, Cybersickness, Biosignals, HMD Devices, Symbolic Machine Learning, Decision Tree, Random Forest
Referências
Ane Alberdi, Asier Aztiria, and Adrian Basarab. 2016. Towards an automatic early stress recognition system for office environments based on multimodal measurements: A review. Journal of biomedical informatics 59 (2016), 49–75.
Matej Andrejašic. 2008. Mems accelerometers. In University of Ljubljana. Faculty for mathematics and physics, Department of physics, Seminar, Vol. 49.
Davide Anguita, Luca Ghelardoni, Alessandro Ghio, Luca Oneto, Sandro Ridella, et al. 2012. The ‘K’ in K-fold Cross Validation.. In ESANN. 441–446.
Oluleke Bamodu and Xu Ming Ye. 2013. Virtual reality and virtual reality system components. In Advanced materials research, Vol. 765. Trans Tech Publ, 1169–1172.
David R Bassett and Dinesh John. 2010. Use of pedometers and accelerometers in clinical populations: validity and reliability issues. Physical therapy reviews 15, 3 (2010), 135–142.
Flávia Cristina Bernardini. 2006. Combinação de classificadores simbólicos utilizando medidas de regras de conhecimento e algoritmos genéticos. Ph. D. Dissertation. Universidade de São Paulo.
Plux Biosignals. 2023. Electrocardiography (ECG) Sensor Data Sheet. (2023). [link]
Plux Biosignals. 2023. Getting Started: BITalino Electrodermal Activity (EDA) Sensor. (2023). [link]
Plux Biosignals. 2024. BITalino (r)evolution Lab Guide. (2024). [link]
Frederick Bonato, Andrea Bubka, and Stephen Palmisano. 2009. Combined pitch and roll and cybersickness in a virtual environment. Aviation, space, and environmental medicine 80, 11 (2009), 941–945.
Guillaume Bouyer, Amine Chellali, and Anatole Lécuyer. 2017. Inducing self-motion sensations in driving simulators using force-feedback and haptic motion. In 2017 IEEE Virtual Reality (VR). IEEE, 84–90.
Pulkit Budhiraja, Mark Roman Miller, Abhishek K Modi, and David Forsyth. 2017. Rotation blurring: use of artificial blurring to reduce cybersickness in virtual reality first person shooters. arXiv preprint arXiv:1710.02599 (2017).
Eunhee Chang, Hyun Taek Kim, and Byounghyun Yoo. 2021. Predicting cybersickness based on user’s gaze behaviors in HMD-based virtual reality. Journal of Computational Design and Engineering 8, 2 (2021), 728–739.
T Daniya, M Geetha, and K Suresh Kumar. 2020. Classification and regression trees with gini index. Advances in Mathematics: Scientific Journal 9, 10 (2020), 8237–8247.
Sašo Džeroski. 2001. Applications of symbolic machine learning to ecological modelling. Ecological modelling 146, 1-3 (2001), 263–273.
Ivan Fomenko and Taninwat Kaewpankan. 2022. Thrill vs. Cybersickness: A study on camera settings’ impact on immersion and cybersickness in VR Racing Games.
Augusto Garcia-Agundez, Christian Reuter, Polona Caserman, Robert Konrad, and Stefan Göbel. 2019. Identifying cybersickness through heart rate variability alterations. International Journal of Virtual Reality 19, 1 (2019), 1–10.
Rifatul Islam, Kevin Desai, and John Quarles. 2021. Cybersickness prediction from integrated hmd’s sensors: A multimodal deep fusion approach using eye-tracking and head-tracking data. In 2021 IEEE international symposium on mixed and augmented reality (ISMAR). IEEE, 31–40.
Rifatul Islam, Yonggun Lee, Mehrad Jaloli, Imtiaz Muhammad, Dakai Zhu, Paul Rad, Yufei Huang, and John Quarles. 2020. Automatic detection and prediction of cybersickness severity using deep neural networks from user’s physiological signals. In 2020 IEEE international symposium on mixed and augmented reality (ISMAR). IEEE, 400–411.
Dayoung Jeong and Kyungsik Han. 2024. PRECYSE: Predicting Cybersickness using Transformer for Multimodal Time-Series Sensor Data. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 8, 2 (2024), 1–24.
J. Jerald. 2015. The VR book: Human-centered design for virtual reality. Accessed on 7 September 2023.
Sungchul Jung, Richard Li, Ryan McKee, Mary C Whitton, and Robert W Lindeman. 2021. Floor-vibration vr: mitigating cybersickness using whole-body tactile stimuli in highly realistic vehicle driving experiences. IEEE Transactions on Visualization & Computer Graphics 27, 05 (2021), 2669–2680.
Robert S Kennedy and Michael G Lilienthal. 1995. Implications of balance disturbances following exposure to virtual reality systems. In Proceedings Virtual Reality Annual International Symposium’95. IEEE, 35–39.
Behrang Keshavarz, Katlyn Peck, Sia Rezaei, and Babak Taati. 2022. Detecting and predicting visually induced motion sickness with physiological measures in combination with machine learning techniques. International Journal of Psychophysiology 176 (2022), 14–26.
Wojciech Kotlowski, Krzysztof J Dembczynski, and Eyke Huellermeier. 2011. Bipartite ranking through minimization of univariate loss. In Proceedings of the 28th International Conference on Machine Learning (ICML-11). Citeseer, 1113–1120.
Panagiotis Kourtesis, Simona Collina, Leonidas AA Doumas, and Sarah E MacPherson. 2019. Technological competence is a pre-condition for effective implementation of virtual reality head mounted displays in human neuroscience: a technological review and meta-analysis. Frontiers in Human Neuroscience 13 (2019), 342.
Eric Krokos and Amitabh Varshney. 2022. Quantifying VR cybersickness using EEG. Virtual Reality 26, 1 (2022), 77–89.
Joseph J LaViola Jr. 2000. A discussion of cybersickness in virtual environments. ACM Sigchi Bulletin 32, 1 (2000), 47–56.
Sergey Edward Lyshevski. 2018. MEMS and NEMS: systems, devices, and structures. CRC press.
Maria Carolina Monard and José Augusto Baranauskas. 2003. Indução de regras e árvores de decisão. Sistemas Inteligentes-fundamentos e aplicações 1 (2003), 115–139.
Ngoc Dang Nguyen, Wei Tan, Lan Du, Wray Buntine, Richard Beare, and Changyou Chen. 2023. AUC maximization for low-resource named entity recognition. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 37. 13389–13399.
Heeseok Oh and Wookho Son. 2022. Cybersickness and its severity arising from virtual reality content: A comprehensive study. Sensors 22, 4 (2022), 1314.
Elham Pashaei and Elnaz Pashaei. 2019. Gene selection using intelligent dynamic genetic algorithm and random forest. In 2019 11th international conference on electrical and electronics engineering (ELECO). IEEE, 470–474.
Thiago Porcino, Erick O Rodrigues, Flavia Bernardini, Daniela Trevisan, and Esteban Clua. 2022. Identifying cybersickness causes in virtual reality games using symbolic machine learning algorithms. Entertainment Computing 41 (2022), 100473.
Thiago Porcino, Daniela Trevisan, and Esteban Clua. 2020. Minimizing cybersickness in head-mounted display systems: causes and strategies review. In 2020 22nd Symposium on Virtual and Augmented Reality (SVR). IEEE, 154–163.
Thiago Malheiros Porcino. 2021. Cybersickness Analysis Using Symbolic Machine Learning Algorithms. Ph. D. Dissertation. Universidade Federal Fluminense - UFF, Niterói-RJ, Brasil.
Chenxin Qu, Xiaoping Che, Siqi Ma, and Shuqin Zhu. 2022. Bio-physiological-signals-based vr cybersickness detection. CCF Transactions on Pervasive Computing and Interaction 4, 3 (2022), 268–284.
James T Reason and Joseph John Brand. 1975. Motion sickness. Academic press.
Marco Recenti, Carlo Ricciardi, Romain Aubonnet, Ilaria Picone, Deborah Jacob, Halldór ÁR Svansson, Sólveig Agnarsdóttir, Gunnar H Karlsson, Valdís Baeringsdóttir, Hannes Petersen, et al. 2021. Toward predicting motion sickness using virtual reality and a moving platform assessing brain, muscles, and heart signals. Frontiers in Bioengineering and Biotechnology 9 (2021), 635661.
Lior Rokach and Oded Maimon. 2005. Decision trees. Data mining and knowledge discovery handbook (2005), 165–192.
Dimitrios Saredakis, Ancret Szpak, Brandon Birckhead, Hannah AD Keage, Albert Rizzo, and Tobias Loetscher. 2020. Factors associated with virtual reality sickness in head-mounted displays: a systematic review and meta-analysis. Frontiers in human neuroscience 14 (2020), 96.
Jinkai Tan, Hexiang Liu, Mengya Li, and Jun Wang. 2018. A prediction scheme of tropical cyclone frequency based on lasso and random forest. Theoretical and applied climatology 133 (2018), 973–983.
Nana Tian and Ronan Boulic. [n. d.]. Who says you are so sick? An investigation on individual susceptibility to cybersickness triggers using EEG, EGG and ECG. ([n. d.]).
Mira Valkonen, Kimmo Kartasalo, Kaisa Liimatainen, Matti Nykter, Leena Latonen, and Pekka Ruusuvuori. 2017. Metastasis detection from whole slide images using local features and random forests. Cytometry Part A 91, 6 (2017), 555–565.
John Vince. 2004. Introduction to virtual reality. Springer Science & Business Media.
Dengju Yao, Xiaojuan Zhan, and Chee-Keong Kwoh. 2019. An improved random forest-based computational model for predicting novel miRNA-disease associations. BMC bioinformatics 20 (2019), 1–14.
Yongheng Zhao and Yanxia Zhang. 2008. Comparison of decision tree methods for finding active objects. Advances in Space Research 41, 12 (2008), 1955–1959.
Matej Andrejašic. 2008. Mems accelerometers. In University of Ljubljana. Faculty for mathematics and physics, Department of physics, Seminar, Vol. 49.
Davide Anguita, Luca Ghelardoni, Alessandro Ghio, Luca Oneto, Sandro Ridella, et al. 2012. The ‘K’ in K-fold Cross Validation.. In ESANN. 441–446.
Oluleke Bamodu and Xu Ming Ye. 2013. Virtual reality and virtual reality system components. In Advanced materials research, Vol. 765. Trans Tech Publ, 1169–1172.
David R Bassett and Dinesh John. 2010. Use of pedometers and accelerometers in clinical populations: validity and reliability issues. Physical therapy reviews 15, 3 (2010), 135–142.
Flávia Cristina Bernardini. 2006. Combinação de classificadores simbólicos utilizando medidas de regras de conhecimento e algoritmos genéticos. Ph. D. Dissertation. Universidade de São Paulo.
Plux Biosignals. 2023. Electrocardiography (ECG) Sensor Data Sheet. (2023). [link]
Plux Biosignals. 2023. Getting Started: BITalino Electrodermal Activity (EDA) Sensor. (2023). [link]
Plux Biosignals. 2024. BITalino (r)evolution Lab Guide. (2024). [link]
Frederick Bonato, Andrea Bubka, and Stephen Palmisano. 2009. Combined pitch and roll and cybersickness in a virtual environment. Aviation, space, and environmental medicine 80, 11 (2009), 941–945.
Guillaume Bouyer, Amine Chellali, and Anatole Lécuyer. 2017. Inducing self-motion sensations in driving simulators using force-feedback and haptic motion. In 2017 IEEE Virtual Reality (VR). IEEE, 84–90.
Pulkit Budhiraja, Mark Roman Miller, Abhishek K Modi, and David Forsyth. 2017. Rotation blurring: use of artificial blurring to reduce cybersickness in virtual reality first person shooters. arXiv preprint arXiv:1710.02599 (2017).
Eunhee Chang, Hyun Taek Kim, and Byounghyun Yoo. 2021. Predicting cybersickness based on user’s gaze behaviors in HMD-based virtual reality. Journal of Computational Design and Engineering 8, 2 (2021), 728–739.
T Daniya, M Geetha, and K Suresh Kumar. 2020. Classification and regression trees with gini index. Advances in Mathematics: Scientific Journal 9, 10 (2020), 8237–8247.
Sašo Džeroski. 2001. Applications of symbolic machine learning to ecological modelling. Ecological modelling 146, 1-3 (2001), 263–273.
Ivan Fomenko and Taninwat Kaewpankan. 2022. Thrill vs. Cybersickness: A study on camera settings’ impact on immersion and cybersickness in VR Racing Games.
Augusto Garcia-Agundez, Christian Reuter, Polona Caserman, Robert Konrad, and Stefan Göbel. 2019. Identifying cybersickness through heart rate variability alterations. International Journal of Virtual Reality 19, 1 (2019), 1–10.
Rifatul Islam, Kevin Desai, and John Quarles. 2021. Cybersickness prediction from integrated hmd’s sensors: A multimodal deep fusion approach using eye-tracking and head-tracking data. In 2021 IEEE international symposium on mixed and augmented reality (ISMAR). IEEE, 31–40.
Rifatul Islam, Yonggun Lee, Mehrad Jaloli, Imtiaz Muhammad, Dakai Zhu, Paul Rad, Yufei Huang, and John Quarles. 2020. Automatic detection and prediction of cybersickness severity using deep neural networks from user’s physiological signals. In 2020 IEEE international symposium on mixed and augmented reality (ISMAR). IEEE, 400–411.
Dayoung Jeong and Kyungsik Han. 2024. PRECYSE: Predicting Cybersickness using Transformer for Multimodal Time-Series Sensor Data. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 8, 2 (2024), 1–24.
J. Jerald. 2015. The VR book: Human-centered design for virtual reality. Accessed on 7 September 2023.
Sungchul Jung, Richard Li, Ryan McKee, Mary C Whitton, and Robert W Lindeman. 2021. Floor-vibration vr: mitigating cybersickness using whole-body tactile stimuli in highly realistic vehicle driving experiences. IEEE Transactions on Visualization & Computer Graphics 27, 05 (2021), 2669–2680.
Robert S Kennedy and Michael G Lilienthal. 1995. Implications of balance disturbances following exposure to virtual reality systems. In Proceedings Virtual Reality Annual International Symposium’95. IEEE, 35–39.
Behrang Keshavarz, Katlyn Peck, Sia Rezaei, and Babak Taati. 2022. Detecting and predicting visually induced motion sickness with physiological measures in combination with machine learning techniques. International Journal of Psychophysiology 176 (2022), 14–26.
Wojciech Kotlowski, Krzysztof J Dembczynski, and Eyke Huellermeier. 2011. Bipartite ranking through minimization of univariate loss. In Proceedings of the 28th International Conference on Machine Learning (ICML-11). Citeseer, 1113–1120.
Panagiotis Kourtesis, Simona Collina, Leonidas AA Doumas, and Sarah E MacPherson. 2019. Technological competence is a pre-condition for effective implementation of virtual reality head mounted displays in human neuroscience: a technological review and meta-analysis. Frontiers in Human Neuroscience 13 (2019), 342.
Eric Krokos and Amitabh Varshney. 2022. Quantifying VR cybersickness using EEG. Virtual Reality 26, 1 (2022), 77–89.
Joseph J LaViola Jr. 2000. A discussion of cybersickness in virtual environments. ACM Sigchi Bulletin 32, 1 (2000), 47–56.
Sergey Edward Lyshevski. 2018. MEMS and NEMS: systems, devices, and structures. CRC press.
Maria Carolina Monard and José Augusto Baranauskas. 2003. Indução de regras e árvores de decisão. Sistemas Inteligentes-fundamentos e aplicações 1 (2003), 115–139.
Ngoc Dang Nguyen, Wei Tan, Lan Du, Wray Buntine, Richard Beare, and Changyou Chen. 2023. AUC maximization for low-resource named entity recognition. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 37. 13389–13399.
Heeseok Oh and Wookho Son. 2022. Cybersickness and its severity arising from virtual reality content: A comprehensive study. Sensors 22, 4 (2022), 1314.
Elham Pashaei and Elnaz Pashaei. 2019. Gene selection using intelligent dynamic genetic algorithm and random forest. In 2019 11th international conference on electrical and electronics engineering (ELECO). IEEE, 470–474.
Thiago Porcino, Erick O Rodrigues, Flavia Bernardini, Daniela Trevisan, and Esteban Clua. 2022. Identifying cybersickness causes in virtual reality games using symbolic machine learning algorithms. Entertainment Computing 41 (2022), 100473.
Thiago Porcino, Daniela Trevisan, and Esteban Clua. 2020. Minimizing cybersickness in head-mounted display systems: causes and strategies review. In 2020 22nd Symposium on Virtual and Augmented Reality (SVR). IEEE, 154–163.
Thiago Malheiros Porcino. 2021. Cybersickness Analysis Using Symbolic Machine Learning Algorithms. Ph. D. Dissertation. Universidade Federal Fluminense - UFF, Niterói-RJ, Brasil.
Chenxin Qu, Xiaoping Che, Siqi Ma, and Shuqin Zhu. 2022. Bio-physiological-signals-based vr cybersickness detection. CCF Transactions on Pervasive Computing and Interaction 4, 3 (2022), 268–284.
James T Reason and Joseph John Brand. 1975. Motion sickness. Academic press.
Marco Recenti, Carlo Ricciardi, Romain Aubonnet, Ilaria Picone, Deborah Jacob, Halldór ÁR Svansson, Sólveig Agnarsdóttir, Gunnar H Karlsson, Valdís Baeringsdóttir, Hannes Petersen, et al. 2021. Toward predicting motion sickness using virtual reality and a moving platform assessing brain, muscles, and heart signals. Frontiers in Bioengineering and Biotechnology 9 (2021), 635661.
Lior Rokach and Oded Maimon. 2005. Decision trees. Data mining and knowledge discovery handbook (2005), 165–192.
Dimitrios Saredakis, Ancret Szpak, Brandon Birckhead, Hannah AD Keage, Albert Rizzo, and Tobias Loetscher. 2020. Factors associated with virtual reality sickness in head-mounted displays: a systematic review and meta-analysis. Frontiers in human neuroscience 14 (2020), 96.
Jinkai Tan, Hexiang Liu, Mengya Li, and Jun Wang. 2018. A prediction scheme of tropical cyclone frequency based on lasso and random forest. Theoretical and applied climatology 133 (2018), 973–983.
Nana Tian and Ronan Boulic. [n. d.]. Who says you are so sick? An investigation on individual susceptibility to cybersickness triggers using EEG, EGG and ECG. ([n. d.]).
Mira Valkonen, Kimmo Kartasalo, Kaisa Liimatainen, Matti Nykter, Leena Latonen, and Pekka Ruusuvuori. 2017. Metastasis detection from whole slide images using local features and random forests. Cytometry Part A 91, 6 (2017), 555–565.
John Vince. 2004. Introduction to virtual reality. Springer Science & Business Media.
Dengju Yao, Xiaojuan Zhan, and Chee-Keong Kwoh. 2019. An improved random forest-based computational model for predicting novel miRNA-disease associations. BMC bioinformatics 20 (2019), 1–14.
Yongheng Zhao and Yanxia Zhang. 2008. Comparison of decision tree methods for finding active objects. Advances in Space Research 41, 12 (2008), 1955–1959.
Publicado
30/09/2024
Como Citar
NUNES DA SILVA, Wedrey; PORCINO, Thiago Malheiros; CASTANHO, Carla Denise; JACOBI, Ricardo Pezzuol.
Analysis of Cybersickness through Biosignals: an approach with Symbolic Machine Learning. In: SIMPÓSIO DE REALIDADE VIRTUAL E AUMENTADA (SVR), 26. , 2024, Manaus/AM.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2024
.
p. 11-20.
