Investigando os Efeitos do Feedback Háptico em uma Experiência Musical Imersiva: Um Estudo com a PhysioDrum
Resumo
The use of haptic stimuli in musical practices has received increasing attention in the literature, particularly in applications related to expressiveness, aesthetic appreciation, and motor control. However, significant gaps still remain regarding the use and evaluation of haptic elements in immersive musical applications, as well as the lack of open-source, rapid-prototyping, and modular tools for experimentation in such environments. To address these limitations, this paper presents PhysioDrum, an immersive musical platform based on the phygital (physical + digital) paradigm, consisting of a virtual reality drum kit integrated with physical devices, responsive haptic feedback, and real-time sound processing. An experimental study was conducted with 30 participants divided into two groups: one exposed to uniform haptic feedback and the other to differentiated tactile patterns. The evaluation adopted a mixed-methods approach, combining standardized quantitative questionnaires and non-parametric statistical analyses. The results indicate that both haptic strategies positively contributed to the immersive experience, with uniform feedback favoring perceptual stability, immersion, and realism, while differentiated responses enhanced haptic realism, expressiveness, and multisensory harmony. As contributions, this work provides empirical evidence regarding the effects of vibrotactile feedback in immersive musical applications and proposes an open-source, modular, and rapid-prototyping platform for musical experimentation in immersive environments.
Palavras-chave:
Feedback háptico, Realidade Virtual, Io3MT, Metaverso Musical, Bateria virtual
Referências
Alfonso Balandra, Hironori Mitake, Takako Yoshida, and Shoichi Hasegawa. 2019. Selective listening attention enhancement, using a simultaneous visual and haptic stimuli. Journal of New Music Research 48, 3 (2019), 294–308.
Alexandre Bouënard, Marcelo MM Wanderley, and Sylvie Gibet. 2010. Gesture control of sound synthesis: Analysis and classification of percussion gestures. Acta Acustica united with Acustica 96, 4 (2010), 668–677.
John Brooke. 2013. SUS: a retrospective. Journal of usability studies 8, 2 (2013), 1–12.
Jack Cuzick. 1985. A Wilcoxon-type test for trend. Statistics in medicine 4, 1 (1985), 87–90.
Donald Degraen, Bruno Fruchard, Frederik Smolders, Emmanouil Potetsianakis, Seref Güngör, Antonio Krüger, and Jürgen Steimle. 2021. Weirding haptics: In-situ prototyping of vibrotactile feedback in virtual reality through vocalization. In The 34th Annual ACM symposium on user interface software and technology. ACM Digital Library, New York, USA, 936–953.
Laura Faulkner. 2003. Beyond the Five-User Assumption: Benefits of Increased Sample Sizes in Usability Testing. Behavior Research Methods, Instruments, & Computers 35, 3 (2003), 379–383. DOI: 10.3758/BF03195514
Daniella Briotto Faustino, Sandra Gabriele, Rami Ibrahim, Anna-Lena Theus, and Audrey Girouard. 2017. SensArt demo: A multisensory prototype for engaging with visual art. In Proceedings of the 2017 ACM international conference on interactive surfaces and spaces. ACM Digital Library, New York, USA, 462–465.
Rebecca Grier, Aaron Bangor, Philip Kortum, and S Camille Peres. 2013. The system usability scale: Beyond standard usability testing. In Proceedings of the human factors and ergonomics society annual meeting, Vol. 57. SAGE Publications Sage CA: Los Angeles, CA, 187–191.
Robert S Kennedy, Norman E Lane, Kevin S Berbaum, and Michael G Lilienthal. 1993. Simulator sickness questionnaire: An enhanced method for quantifying simulator sickness. The international journal of aviation psychology 3, 3 (1993), 203–220.
Mark T Marshall and Marcelo M Wanderley. 2011. Examining the effects of embedded vibrotactile feedback on the feel of a digital musical instrument. In Proceedings of the International Conference on New Interfaces for Musical Expression. NIME, Oslo, Norway, 399–404.
John McDowell and Dermont J Furlong. 2018. Haptic-Listening and the Classical Guitar. In Proceedings of the International Conference on New Interfaces for Musical Expression. NIME, Virginia, USA, 293–298.
Patrick E McKnight and Julius Najab. 2010. Mann-whitney U test. The Corsini encyclopedia of psychology 1, 1 (2010), 1–1.
Cristina Mele, Tiziana Russo Spena, Marialuiza Marzullo, and Irene Di Bernardo. 2023. The phygital transformation: a systematic review and a research agenda. Italian Journal of Marketing 2023, 3 (2023), 323–349.
Axel Mulder. 1994. Virtual musical instruments: Accessing the sound synthesis universe as a performer. In Proceedings of the First Brazilian Symposium on Computer Music. Sociedade Brasileira de Computação (SBC), Porto Alegre, Brazil, 243–250.
Suranga Chandima Nanayakkara, Lonce Wyse, Sim Heng Ong, and Elizabeth A Taylor. 2013. Enhancing musical experience for the hearing-impaired using visual and haptic displays. Human–Computer Interaction 28, 2 (2013), 115–160.
Jakob Nielsen and Thomas K. Landauer. 1993. A Mathematical Model of the Finding of Usability Problems. In Proceedings of the INTERCHI ’93 Conference on Human Factors in Computing Systems. ACM, 206–213. DOI: 10.1145/169059.169166
Sile O’Modhrain and Chris Chafe. 2000. Incorporating haptic feedback into interfaces for music applications. In Proceedings of the International Symposium on Robotics with Applications, World Automation Conference. Stanford University, Stanford, USA, 1–6.
SOMNUK Phon-Amnuaisuk, KIARASH Rezahanjani, HAMID REZA Momeni, and KC Khor. 2009. Virtual musical instruments: air drums. In Proceedings of the Sixth International Conference on Information Technology in Asia (CITA 2009), Kuching, Sarawak. Springer, Kuching, Malasya, 95–99.
Suji Sathiyamurthy, Melody Lui, Erin Kim, and Oliver Schneider. 2021. Measuring Haptic Experience: Elaborating the HX model with scale development. In 2021 IEEE World Haptics Conference (WHC). IEEE, New York, USA, 979–984.
Oliver Schneider, Karon MacLean, Colin Swindells, and Kellogg Booth. 2017. Haptic experience design: What hapticians do and where they need help. International Journal of Human-Computer Studies 107 (2017), 5–21.
Hasti Seifi and Karon E MacLean. 2017. Exploiting haptic facets: Users’ sensemaking schemas as a path to design and personalization of experience. International Journal of Human-Computer Studies 107 (2017), 38–61.
Paul Strohmeier, Seref Güngör, Luis Herres, Dennis Gudea, Bruno Fruchard, and Jürgen Steimle. 2020. BARefoot: Generating virtual materials using motion coupled vibration in shoes. In Proceedings of the 33rd Annual ACM Symposium on User Interface Software and Technology. ACM Digital Library, New York, USA, 579–593.
Carl Timothy Tolentino, Agatha Uy, and Prospero Naval. 2019. Air Drums: Playing Drums Using Computer Vision. In 2019 International Symposium on Multimedia and Communication Technology (ISMAC). IEEE, New York, USA, 1–6.
Luca Turchet. 2018. Smart Musical Instruments: vision, design principles, and future directions. IEEE Access 7 (2018), 8944–8963.
Luca Turchet. 2023. Musical Metaverse: vision, opportunities, and challenges. Personal and Ubiquitous Computing 27, 5 (2023), 1811–1827.
Luca Turchet, Raffaele Rosaia, Alessandro Diodati, and Marco Carner. 2025. Exposure to vibrotactile music improves audiometric performances in individuals with cochlear implants. Scientific Reports 15, 1 (2025), 20054.
Luca Turchet, Christian Sassi, Davide Vecchia, and Gian Pietro Picco. 2025. Real-Time Musical Haptics With Ultra-Wideband: A Study on Latency, Reliability, and Perception. IEEE Transactions on Haptics 18, 1 (2025), 12.
Luca Turchet, TravisWest, and Marcelo MWanderley. 2019. Smart mandolin and musical haptic gilet: effects of vibro-tactile stimuli during live music performance. In Proceedings of the 14th International Audio Mostly Conference: A Journey in Sound. ACM Digital Library, New York, USA, 168–175.
Luca Turchet, Travis West, and Marcelo M Wanderley. 2021. Touching the audience: musical haptic wearables for augmented and participatory live music performances. Personal and Ubiquitous Computing 25, 4 (2021), 749–769.
Tara Venkatesan and Qian Janice Wang. 2023. Feeling connected: the role of haptic feedback in VR concerts and the impact of haptic music players on the music listening experience. In Arts, Vol. 12. MDPI, Basel, Switzerland, 148.
Rômulo Vieira, Débora C Muchaluat-Saade, and Pablo César. 2023. Towards an internet of multisensory, multimedia and musical things (Io3MT) environment. In 2023 4th International Symposium on the Internet of Sounds. IEEE, New York, USA, 1–10.
Rômulo Vieira, Debora Christina Muchaluat-Saade, and Pablo Cesar. 2025. PhysioDrum: Bridging Physical and Digital Realms in Immersive Musical Interaction. In Proceedings of the 2025 ACM International Conference on Interactive Media Experiences. ACM Digital Library, New York, USA, 356–358.
Rômulo Vieira, Débora C. Muchaluat-Saade, Marcelo Rocha, Célio Albuquerque, and Pablo César. 2023. RemixDrum: A Smart Musical Instrument for Music and Visual Art Remix. In 2023 IEEE 9th World Forum on Internet of Things (WF-IoT). IEEE XPlore, New York, USA, 1–7. DOI: 10.1109/WF-IoTXXX
Rômulo Vieira, Shu Wei, Thomas Röggla, Débora C Muchaluat-Saade, and Pablo César. 2024. Immersive Io3MT Environments: Design Guidelines, Use Cases and Future Directions. In 2024 IEEE 5th International Symposium on the Internet of Sounds (IS2). IEEE, New York, USA, 1–10.
Robert A. Virzi. 1992. Refining the Test Phase of Usability Evaluation: How Many Subjects Is Enough? Human Factors 34, 4 (1992), 457–468. DOI: 10.1177/001872089203400407
Bob G Witmer and Michael J Singer. 1998. Measuring presence in virtual environments: A presence questionnaire. Presence 7, 3 (1998), 225–240.
Harel Yadid, Almog Algranti, Mark Levin, and Ayal Taitler. 2023. A2d: Anywhere anytime drumming. In 2023 IEEE Region 10 Symposium (TENSYMP). IEEE, New York, USA, 1–6.
Gareth Young, David Murphy, and JeffreyWeeter. 2017. A Qualitative Analysis of Haptic Feedback in Music Focused Exercises. In Proceedings of the New Interfaces for Musical Expression. NIME, Maynooth, Ireland.
Dishant Zaveri, Rajvi Parekh, Russel Lobo, Sanika Ardekar, Vismay Vora, and Aniket Kore. 2022. Aero Drums-Augmented Virtual Drums. In 2022 IEEE 4th International Conference on Cybernetics, Cognition and Machine Learning Applications (ICCCMLA). IEEE, New York, USA, 516–520.
Alexandre Bouënard, Marcelo MM Wanderley, and Sylvie Gibet. 2010. Gesture control of sound synthesis: Analysis and classification of percussion gestures. Acta Acustica united with Acustica 96, 4 (2010), 668–677.
John Brooke. 2013. SUS: a retrospective. Journal of usability studies 8, 2 (2013), 1–12.
Jack Cuzick. 1985. A Wilcoxon-type test for trend. Statistics in medicine 4, 1 (1985), 87–90.
Donald Degraen, Bruno Fruchard, Frederik Smolders, Emmanouil Potetsianakis, Seref Güngör, Antonio Krüger, and Jürgen Steimle. 2021. Weirding haptics: In-situ prototyping of vibrotactile feedback in virtual reality through vocalization. In The 34th Annual ACM symposium on user interface software and technology. ACM Digital Library, New York, USA, 936–953.
Laura Faulkner. 2003. Beyond the Five-User Assumption: Benefits of Increased Sample Sizes in Usability Testing. Behavior Research Methods, Instruments, & Computers 35, 3 (2003), 379–383. DOI: 10.3758/BF03195514
Daniella Briotto Faustino, Sandra Gabriele, Rami Ibrahim, Anna-Lena Theus, and Audrey Girouard. 2017. SensArt demo: A multisensory prototype for engaging with visual art. In Proceedings of the 2017 ACM international conference on interactive surfaces and spaces. ACM Digital Library, New York, USA, 462–465.
Rebecca Grier, Aaron Bangor, Philip Kortum, and S Camille Peres. 2013. The system usability scale: Beyond standard usability testing. In Proceedings of the human factors and ergonomics society annual meeting, Vol. 57. SAGE Publications Sage CA: Los Angeles, CA, 187–191.
Robert S Kennedy, Norman E Lane, Kevin S Berbaum, and Michael G Lilienthal. 1993. Simulator sickness questionnaire: An enhanced method for quantifying simulator sickness. The international journal of aviation psychology 3, 3 (1993), 203–220.
Mark T Marshall and Marcelo M Wanderley. 2011. Examining the effects of embedded vibrotactile feedback on the feel of a digital musical instrument. In Proceedings of the International Conference on New Interfaces for Musical Expression. NIME, Oslo, Norway, 399–404.
John McDowell and Dermont J Furlong. 2018. Haptic-Listening and the Classical Guitar. In Proceedings of the International Conference on New Interfaces for Musical Expression. NIME, Virginia, USA, 293–298.
Patrick E McKnight and Julius Najab. 2010. Mann-whitney U test. The Corsini encyclopedia of psychology 1, 1 (2010), 1–1.
Cristina Mele, Tiziana Russo Spena, Marialuiza Marzullo, and Irene Di Bernardo. 2023. The phygital transformation: a systematic review and a research agenda. Italian Journal of Marketing 2023, 3 (2023), 323–349.
Axel Mulder. 1994. Virtual musical instruments: Accessing the sound synthesis universe as a performer. In Proceedings of the First Brazilian Symposium on Computer Music. Sociedade Brasileira de Computação (SBC), Porto Alegre, Brazil, 243–250.
Suranga Chandima Nanayakkara, Lonce Wyse, Sim Heng Ong, and Elizabeth A Taylor. 2013. Enhancing musical experience for the hearing-impaired using visual and haptic displays. Human–Computer Interaction 28, 2 (2013), 115–160.
Jakob Nielsen and Thomas K. Landauer. 1993. A Mathematical Model of the Finding of Usability Problems. In Proceedings of the INTERCHI ’93 Conference on Human Factors in Computing Systems. ACM, 206–213. DOI: 10.1145/169059.169166
Sile O’Modhrain and Chris Chafe. 2000. Incorporating haptic feedback into interfaces for music applications. In Proceedings of the International Symposium on Robotics with Applications, World Automation Conference. Stanford University, Stanford, USA, 1–6.
SOMNUK Phon-Amnuaisuk, KIARASH Rezahanjani, HAMID REZA Momeni, and KC Khor. 2009. Virtual musical instruments: air drums. In Proceedings of the Sixth International Conference on Information Technology in Asia (CITA 2009), Kuching, Sarawak. Springer, Kuching, Malasya, 95–99.
Suji Sathiyamurthy, Melody Lui, Erin Kim, and Oliver Schneider. 2021. Measuring Haptic Experience: Elaborating the HX model with scale development. In 2021 IEEE World Haptics Conference (WHC). IEEE, New York, USA, 979–984.
Oliver Schneider, Karon MacLean, Colin Swindells, and Kellogg Booth. 2017. Haptic experience design: What hapticians do and where they need help. International Journal of Human-Computer Studies 107 (2017), 5–21.
Hasti Seifi and Karon E MacLean. 2017. Exploiting haptic facets: Users’ sensemaking schemas as a path to design and personalization of experience. International Journal of Human-Computer Studies 107 (2017), 38–61.
Paul Strohmeier, Seref Güngör, Luis Herres, Dennis Gudea, Bruno Fruchard, and Jürgen Steimle. 2020. BARefoot: Generating virtual materials using motion coupled vibration in shoes. In Proceedings of the 33rd Annual ACM Symposium on User Interface Software and Technology. ACM Digital Library, New York, USA, 579–593.
Carl Timothy Tolentino, Agatha Uy, and Prospero Naval. 2019. Air Drums: Playing Drums Using Computer Vision. In 2019 International Symposium on Multimedia and Communication Technology (ISMAC). IEEE, New York, USA, 1–6.
Luca Turchet. 2018. Smart Musical Instruments: vision, design principles, and future directions. IEEE Access 7 (2018), 8944–8963.
Luca Turchet. 2023. Musical Metaverse: vision, opportunities, and challenges. Personal and Ubiquitous Computing 27, 5 (2023), 1811–1827.
Luca Turchet, Raffaele Rosaia, Alessandro Diodati, and Marco Carner. 2025. Exposure to vibrotactile music improves audiometric performances in individuals with cochlear implants. Scientific Reports 15, 1 (2025), 20054.
Luca Turchet, Christian Sassi, Davide Vecchia, and Gian Pietro Picco. 2025. Real-Time Musical Haptics With Ultra-Wideband: A Study on Latency, Reliability, and Perception. IEEE Transactions on Haptics 18, 1 (2025), 12.
Luca Turchet, TravisWest, and Marcelo MWanderley. 2019. Smart mandolin and musical haptic gilet: effects of vibro-tactile stimuli during live music performance. In Proceedings of the 14th International Audio Mostly Conference: A Journey in Sound. ACM Digital Library, New York, USA, 168–175.
Luca Turchet, Travis West, and Marcelo M Wanderley. 2021. Touching the audience: musical haptic wearables for augmented and participatory live music performances. Personal and Ubiquitous Computing 25, 4 (2021), 749–769.
Tara Venkatesan and Qian Janice Wang. 2023. Feeling connected: the role of haptic feedback in VR concerts and the impact of haptic music players on the music listening experience. In Arts, Vol. 12. MDPI, Basel, Switzerland, 148.
Rômulo Vieira, Débora C Muchaluat-Saade, and Pablo César. 2023. Towards an internet of multisensory, multimedia and musical things (Io3MT) environment. In 2023 4th International Symposium on the Internet of Sounds. IEEE, New York, USA, 1–10.
Rômulo Vieira, Debora Christina Muchaluat-Saade, and Pablo Cesar. 2025. PhysioDrum: Bridging Physical and Digital Realms in Immersive Musical Interaction. In Proceedings of the 2025 ACM International Conference on Interactive Media Experiences. ACM Digital Library, New York, USA, 356–358.
Rômulo Vieira, Débora C. Muchaluat-Saade, Marcelo Rocha, Célio Albuquerque, and Pablo César. 2023. RemixDrum: A Smart Musical Instrument for Music and Visual Art Remix. In 2023 IEEE 9th World Forum on Internet of Things (WF-IoT). IEEE XPlore, New York, USA, 1–7. DOI: 10.1109/WF-IoTXXX
Rômulo Vieira, Shu Wei, Thomas Röggla, Débora C Muchaluat-Saade, and Pablo César. 2024. Immersive Io3MT Environments: Design Guidelines, Use Cases and Future Directions. In 2024 IEEE 5th International Symposium on the Internet of Sounds (IS2). IEEE, New York, USA, 1–10.
Robert A. Virzi. 1992. Refining the Test Phase of Usability Evaluation: How Many Subjects Is Enough? Human Factors 34, 4 (1992), 457–468. DOI: 10.1177/001872089203400407
Bob G Witmer and Michael J Singer. 1998. Measuring presence in virtual environments: A presence questionnaire. Presence 7, 3 (1998), 225–240.
Harel Yadid, Almog Algranti, Mark Levin, and Ayal Taitler. 2023. A2d: Anywhere anytime drumming. In 2023 IEEE Region 10 Symposium (TENSYMP). IEEE, New York, USA, 1–6.
Gareth Young, David Murphy, and JeffreyWeeter. 2017. A Qualitative Analysis of Haptic Feedback in Music Focused Exercises. In Proceedings of the New Interfaces for Musical Expression. NIME, Maynooth, Ireland.
Dishant Zaveri, Rajvi Parekh, Russel Lobo, Sanika Ardekar, Vismay Vora, and Aniket Kore. 2022. Aero Drums-Augmented Virtual Drums. In 2022 IEEE 4th International Conference on Cybernetics, Cognition and Machine Learning Applications (ICCCMLA). IEEE, New York, USA, 516–520.
Publicado
27/10/2026
Como Citar
VIEIRA, Rômulo; MUCHALUAT-SAADE, Débora C.; CÉSAR, Pablo.
Investigando os Efeitos do Feedback Háptico em uma Experiência Musical Imersiva: Um Estudo com a PhysioDrum. In: BRAZILIAN SYMPOSIUM ON MULTIMEDIA AND THE WEB (WEBMEDIA), 32. , 2026, Lavras/MG.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2026
.
p. 262-271.
DOI: https://doi.org/10.5753/webmedia.2026.30152.
