Towards analysing user attention using electroencephalography in immersive multisensory virtual environments
Abstract
This study explores attention detection using EEG signals and its application in immersive environments, combining insights from different approaches. We assessed the attention levels of 12 participants through 2D digital games, highlighting that tasks without time limitswere more effective in maintaining user engagement.We intend to run another experiment with PhysioDrum, a multisensory drum system that combines physical and digital elements for immersive musical experiences. Equipped with sensors and actuators, PhysioDrum ensures real-time synchronization and multimodal feedback, demonstrating potential for creative and educational uses. Future research aims to analyze the interaction between attention and immersive environments, focusing on the differences in engagement between neurotypical and neurodivergent users, and promoting the development of therapeutic tools for cognitive and emotional enrichment.
Keywords:
Attention detection, physiological signals, Wavelet transform, Io3MT, virtual drum
References
Carlos P Amaral, Marco A Simões, Susana Mouga, João Andrade, and Miguel Castelo-Branco. 2017. A novel brain computer interface for classification of social joint attention in autism and comparison of 3 experimental setups: a feasibility study. Journal of neuroscience methods 290 (2017), 105–115.
John Brooke et al. 1996. SUS-A quick and dirty usability scale. Usability evaluation in industry 189, 194 (1996), 4–7.
P. Comon. 1994. Independent Component Analysis, A new concept? Signal Processing 36, 3 (1994), 287–314.
Bruna Santos Da Silva, Eugenio Horacio Grevet, Luiza Carolina Fagundes Silva, João Kleber Neves Ramos, Diego Luiz Rovaris, and Claiton Henrique Dotto Bau. 2023. An overview on neurobiology and therapeutics of attentiondeficit/hyperactivity disorder. Discover Mental Health 3, 1 (2023), 2.
R.C. Gonzalez and R.E.Woods. 2009. Processamento Digital De Imagens. ADDISON WESLEY BRA. [link]
Sandra G Hart and Lowell E Staveland. 1988. Development of NASA-TLX (Task Load Index): Results of empirical and theoretical research. In Advances in psychology. Vol. 52. Elsevier, 139–183.
R. Hassan, M. S. Hasan, J. Hasan, M. R. Jamader, D. Eisenberg, and T. Pias. 2020. Machine learning based human attention recognition from brain-EEG signals. (2020).
E. M. Imah, E. S. Dewi, and I. G. P. A. Buditjahjanto. 2021. A Comparative Analysis of Machine Learning Methods for Joint Attention Classification in Autism Spectrum Disorder Using Electroencephalography Brain Computer Interface. International Journal of Intelligent Engineering & Systems 14, 3 (2021).
Robert S Kennedy, Norman E Lane, Kevin S Berbaum, and Michael G Lilienthal. 1993. Simulator sickness questionnaire: An enhanced method for quantifying simulator sickness. The international journal of aviation psychology 3, 3 (1993), 203–220.
George H Klem. 1999. The ten-twenty electrode system of the international federation. The international federation of clinical neurophysiology. Electroencephalogr. Clin. Neurophysiol. Suppl. 52 (1999), 3–6.
Colin M. MacLeod. 1991. Half a century of research on the Stroop effect: An integrative review. Psychological Bulletin 109, 2 (1991), 163–203. DOI: 10.1037/0033-2909.109.2.163
Stéphane Mallat. 2008. A Wavelet Tour of Signal Processing: The Sparse Way (3rd ed.). Academic Press.
S.A. PLUX Wireless Biosignals. 2021. BITalino (r)evolution User Manual. Available at [link].
R. Ramos, B. Valdez-Salas, R. Zlatev, M. S. Wiener, and J. M. B. Rull. 2017. The discrete wavelet transform and its application for noise removal in localized corrosion measurements. International Journal of Corrosion 2017 (2017). DOI: 10.1155/2017/7925404
Ralph M. Reitan. 1958. Validity of the Trail Making Test as an indicator of organic brain damage. Vol. 8. Perceptual and Motor Skills. 271–276 pages. DOI: 10.2466/pms.1958.8.3.271
Suji Sathiyamurthy, Melody Lui, Erin Kim, and Oliver Schneider. 2021. Measuring Haptic Experience: Elaborating the HX model with scale development. In 2021 IEEE World Haptics Conference (WHC). IEEE, 979–984.
A. Smith. 1982. Symbol Digit Modalities Test: Manual. Western Psychological Services.
J. Ridley Stroop. 1935. Studies of interference in serial verbal reactions. Journal of Experimental Psychology 18, 6 (1935), 643–662. DOI: 10.1037/h0054651
Christopher Torrence and Gilbert P. Compo. 1998. A Practical Guide to Wavelet Analysis. Bulletin of the American Meteorological Society 79, 1 (1998), 61–78. DOI: 10.1175/1520-0477(1998)079<0061:APGTWA>2.0.CO;2
Luca Turchet. 2023. Musical Metaverse: vision, opportunities, and challenges. Personal and Ubiquitous Computing 27, 5 (2023), 1811–1827.
Rômulo Vieira, Débora C. Muchaluat-Saade, and Pablo César. 2023. Towards an Internet of Multisensory, Multimedia and Musical Things (Io3MT) Environment. In 2023 4th International Symposium on the Internet of Sounds. 1–10. DOI: 10.1109/IEEECONF59510.2023.10335383
Rômulo Vieira, Shu Wei, Thomas Röggla, Débora C. Muchaluat-Saade, and Pablo César. 2024. Immersive Io3MT Environments: Design Guidelines, Use Cases and Future Directions. In 2024 IEEE 5th International Symposium on the Internet of Sounds (IS2). 1–10. DOI: 10.1109/IS262782.2024.10704141
Bob G Witmer and Michael J Singer. 1998. Measuring presence in virtual environments: A presence questionnaire. Presence 7, 3 (1998), 225–240.
John Brooke et al. 1996. SUS-A quick and dirty usability scale. Usability evaluation in industry 189, 194 (1996), 4–7.
P. Comon. 1994. Independent Component Analysis, A new concept? Signal Processing 36, 3 (1994), 287–314.
Bruna Santos Da Silva, Eugenio Horacio Grevet, Luiza Carolina Fagundes Silva, João Kleber Neves Ramos, Diego Luiz Rovaris, and Claiton Henrique Dotto Bau. 2023. An overview on neurobiology and therapeutics of attentiondeficit/hyperactivity disorder. Discover Mental Health 3, 1 (2023), 2.
R.C. Gonzalez and R.E.Woods. 2009. Processamento Digital De Imagens. ADDISON WESLEY BRA. [link]
Sandra G Hart and Lowell E Staveland. 1988. Development of NASA-TLX (Task Load Index): Results of empirical and theoretical research. In Advances in psychology. Vol. 52. Elsevier, 139–183.
R. Hassan, M. S. Hasan, J. Hasan, M. R. Jamader, D. Eisenberg, and T. Pias. 2020. Machine learning based human attention recognition from brain-EEG signals. (2020).
E. M. Imah, E. S. Dewi, and I. G. P. A. Buditjahjanto. 2021. A Comparative Analysis of Machine Learning Methods for Joint Attention Classification in Autism Spectrum Disorder Using Electroencephalography Brain Computer Interface. International Journal of Intelligent Engineering & Systems 14, 3 (2021).
Robert S Kennedy, Norman E Lane, Kevin S Berbaum, and Michael G Lilienthal. 1993. Simulator sickness questionnaire: An enhanced method for quantifying simulator sickness. The international journal of aviation psychology 3, 3 (1993), 203–220.
George H Klem. 1999. The ten-twenty electrode system of the international federation. The international federation of clinical neurophysiology. Electroencephalogr. Clin. Neurophysiol. Suppl. 52 (1999), 3–6.
Colin M. MacLeod. 1991. Half a century of research on the Stroop effect: An integrative review. Psychological Bulletin 109, 2 (1991), 163–203. DOI: 10.1037/0033-2909.109.2.163
Stéphane Mallat. 2008. A Wavelet Tour of Signal Processing: The Sparse Way (3rd ed.). Academic Press.
S.A. PLUX Wireless Biosignals. 2021. BITalino (r)evolution User Manual. Available at [link].
R. Ramos, B. Valdez-Salas, R. Zlatev, M. S. Wiener, and J. M. B. Rull. 2017. The discrete wavelet transform and its application for noise removal in localized corrosion measurements. International Journal of Corrosion 2017 (2017). DOI: 10.1155/2017/7925404
Ralph M. Reitan. 1958. Validity of the Trail Making Test as an indicator of organic brain damage. Vol. 8. Perceptual and Motor Skills. 271–276 pages. DOI: 10.2466/pms.1958.8.3.271
Suji Sathiyamurthy, Melody Lui, Erin Kim, and Oliver Schneider. 2021. Measuring Haptic Experience: Elaborating the HX model with scale development. In 2021 IEEE World Haptics Conference (WHC). IEEE, 979–984.
A. Smith. 1982. Symbol Digit Modalities Test: Manual. Western Psychological Services.
J. Ridley Stroop. 1935. Studies of interference in serial verbal reactions. Journal of Experimental Psychology 18, 6 (1935), 643–662. DOI: 10.1037/h0054651
Christopher Torrence and Gilbert P. Compo. 1998. A Practical Guide to Wavelet Analysis. Bulletin of the American Meteorological Society 79, 1 (1998), 61–78. DOI: 10.1175/1520-0477(1998)079<0061:APGTWA>2.0.CO;2
Luca Turchet. 2023. Musical Metaverse: vision, opportunities, and challenges. Personal and Ubiquitous Computing 27, 5 (2023), 1811–1827.
Rômulo Vieira, Débora C. Muchaluat-Saade, and Pablo César. 2023. Towards an Internet of Multisensory, Multimedia and Musical Things (Io3MT) Environment. In 2023 4th International Symposium on the Internet of Sounds. 1–10. DOI: 10.1109/IEEECONF59510.2023.10335383
Rômulo Vieira, Shu Wei, Thomas Röggla, Débora C. Muchaluat-Saade, and Pablo César. 2024. Immersive Io3MT Environments: Design Guidelines, Use Cases and Future Directions. In 2024 IEEE 5th International Symposium on the Internet of Sounds (IS2). 1–10. DOI: 10.1109/IS262782.2024.10704141
Bob G Witmer and Michael J Singer. 1998. Measuring presence in virtual environments: A presence questionnaire. Presence 7, 3 (1998), 225–240.
Published
2025-06-03
How to Cite
SILVA, Carla Estefany Caetano; VIEIRA, Rômulo; TREVISAN, Daniela Gorski; MUCHALUAT-SAADE, Débora C..
Towards analysing user attention using electroencephalography in immersive multisensory virtual environments. In: ACM INTERNATIONAL CONFERENCE ON INTERACTIVE MEDIA EXPERIENCES WORKSHOPS (IMXW), 25. , 2025, Niterói/RJ.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2025
.
p. 91-95.
DOI: https://doi.org/10.5753/imxw.2025.2092.
