Spatial Audio in Virtual Reality: A systematic review
Resumo
Spatial Audio represents sound across a full sphere around the listener’s position and is being widely implemented in Virtual Reality and games. However there’s not an extensive body of work in the full benefits of immersive audio in virtual reality, as well as it’s applications through these systems in the literature. This work aims, through a systematic review on the theme, to point how and where can spatial audio be of immense value to virtual reality applications, opening up a world to develop future works in mixing spatial audio content for virtual reality.
Palavras-chave:
virtual reality, spatial audio, systematic review, immersive audio
Referências
Jamieson Brettle and Jan Skoglund. 2016. Open-Source Spatial Audio Compression for VR Content. SMPTE 2016 Annual Technical Conference and Exhibition, 1–9. https://doi.org/10.5594/M001712
Ji-Ho Chang and Wan-Ho Cho. 2018. Impairments of Binaural Sound Based on Ambisonics for Virtual Reality Audio. 2018 IEEE 10th Sensor Array and Multichannel Signal Processing Workshop (SAM), 341–345. https://doi.org/10.1109/SAM.2018.8448493
Fang-Yi Chao, Cagri Ozcinar, Chen Wang, Emin Zerman, Lu Zhang, Wassim Hamidouche, Olivier Deforges, and Aljosa Smolic. 2020. Audio-Visual Perception of Omnidirectional Video for Virtual Reality Applications. 2020 IEEE International Conference on Multimedia & Expo Workshops (ICMEW), 1–6. https://doi.org/10.1109/ICMEW46912.2020.9105956
Fang-Yi Chao, Cagri Ozcinar, Lu Zhang, Wassim Hamidouche, Olivier Deforges, and Aljosa Smolic. 2020. Towards Audio-Visual Saliency Prediction for Omnidirectional Video with Spatial Audio. 2020 IEEE International Conference on Visual Communications and Image Processing (VCIP), 355–358. https://doi.org/10.1109/VCIP49819.2020.9301766
Brenda Chapman and Mark Andrews et al.2012. Brave.
Randy Frans Fela, Nick Zacharov, and Soren Forchhammer. 2020. Towards a Perceived Audiovisual Quality Model for Immersive Content. 2020 Twelfth International Conference on Quality of Multimedia Experience (QoMEX), 1–6. https://doi.org/10.1109/QoMEX48832.2020.9123134
Amit Hirway, Yuansong Qiao, and Niall Murray. 2020. A QoE and Visual Attention Evaluation on the Influence of Spatial Audio in 360 Videos. 2020 IEEE International Conference on Artificial Intelligence and Virtual Reality (AIVR), 345–350. https://doi.org/10.1109/AIVR50618.2020.00071
D Hong, H.-J. Kwon, C G Kim, and W.-C. Park. 2018. Real-time 3D audio downmixing system based on sound rendering for the immersive sound of mobile virtual reality applications. KSII Transactions on Internet and Information Systems 12 (2018), 5936–5954. Issue 12. https://doi.org/10.3837/tiis.2018.12.018 cited By 1.
Takashi Kasuya, Manabu Tsukada, Yu Komohara, Shigeki Takasaka, Takuhiro Mizuno, Yoshitaka Nomura, Yuta Ueda, and Hiroshi Esaki. 2019. LiVRation: Remote VR live platform with interactive 3D audio-visual service. 2019 IEEE Games, Entertainment, Media Conference (GEM), 1–7. https://doi.org/10.1109/GEM.2019.8811549 cited By 5.
S Kato, T Ikeda, M Kawamorita, M Tsukada, and H Esaki. 2020. Web3602: An interactive web application for viewing 3D audio-visual Contents. Proceedings of the Sound and Music Computing Conferences 2020-June, 32–39. [link]. cited By 1.
Victoria Korshunova, Gerard B. Remijn, Synes Elischka, and Catarina Mendonca. 2019. The Impact of Sound Systems on the Perception of Cinematic Content in Immersive Audiovisual Productions. 2019 12th Asia Pacific Workshop on Mixed and Augmented Reality (APMAR), 1–5. https://doi.org/10.1109/APMAR.2019.8709163
Ademir Marques, Rafael Kenji Horota, Eniuce Menezes de Souza, Lucas Kupssinskü, Pedro Rossa, Alysson Soares Aires, Leonardo Bachi, Mauricio Roberto Veronez, Luiz Gonzaga, and Caroline Lessio Cazarin. 2020. Virtual and digital outcrops in the petroleum industry: A systematic review. https://doi.org/10.1016/j.earscirev.2020.103260
Bartlomiej Mroz, Marek Kabacinski, Tomasz Ciotucha, Andrzej Ruminski, and Tomasz Zernicki. 2021. Production of six-degrees-of-freedom (6DoF) navigable audio using 30 Ambisonic microphones. 2021 Immersive and 3D Audio: from Architecture to Automotive (I3DA), 1–5. https://doi.org/10.1109/I3DA48870.2021.9610970
Schuyler R. Quackenbush and Jurgen Herre. 2021. MPEG Standards for Compressed Representation of Immersive Audio. Proc. IEEE 109 (9 2021), 1578–1589. Issue 9. https://doi.org/10.1109/JPROC.2021.3075390
Vincent Russell, Rachel Barry, and David Murphy. 2018. HAVE Experience: An Investigation into VR Empathy for Panic Disorder. 2018 IEEE Games, Entertainment, Media Conference (GEM), 1–9. https://doi.org/10.1109/GEM.2018.8516461
Anderson Augusto Simiscuka, Mohammed Amine Togou, Rohit Verma, Mikel Zorrilla, Noel E. O’Connor, and Gabriel-Miro Muntean. 2022. An Evaluation of 360° Video and Audio Quality in an Artistic-Oriented Platform. 2022 IEEE International Symposium on Broadband Multimedia Systems and Broadcasting (BMSB) 2022-June, 1–5. https://doi.org/10.1109/BMSB55706.2022.9828745
Robin Stockert and Aleksander Tidemann. 2022. Hybrid Learning Spaces with Spatial Audio. 2022 29th International Conference on Systems, Signals and Image Processing (IWSSIP) CFP2255E-A, 1–4. https://doi.org/10.1109/IWSSIP55020.2022.9854439
Chi Wang and Chien-Wen Cheng. 2019. Perceiving the World with Hearing in Virtual Reality. 2019 IEEE International Conference on Consumer Electronics - Taiwan (ICCE-TW), 1–2. https://doi.org/10.1109/ICCE-TW46550.2019.8991888
Shu-Nung Yao. 2017. Headphone-based immersive audio for virtual reality headsets. IEEE Transactions on Consumer Electronics 63 (8 2017), 300–308. Issue 3. https://doi.org/10.1109/TCE.2017.014951
Ji-Ho Chang and Wan-Ho Cho. 2018. Impairments of Binaural Sound Based on Ambisonics for Virtual Reality Audio. 2018 IEEE 10th Sensor Array and Multichannel Signal Processing Workshop (SAM), 341–345. https://doi.org/10.1109/SAM.2018.8448493
Fang-Yi Chao, Cagri Ozcinar, Chen Wang, Emin Zerman, Lu Zhang, Wassim Hamidouche, Olivier Deforges, and Aljosa Smolic. 2020. Audio-Visual Perception of Omnidirectional Video for Virtual Reality Applications. 2020 IEEE International Conference on Multimedia & Expo Workshops (ICMEW), 1–6. https://doi.org/10.1109/ICMEW46912.2020.9105956
Fang-Yi Chao, Cagri Ozcinar, Lu Zhang, Wassim Hamidouche, Olivier Deforges, and Aljosa Smolic. 2020. Towards Audio-Visual Saliency Prediction for Omnidirectional Video with Spatial Audio. 2020 IEEE International Conference on Visual Communications and Image Processing (VCIP), 355–358. https://doi.org/10.1109/VCIP49819.2020.9301766
Brenda Chapman and Mark Andrews et al.2012. Brave.
Randy Frans Fela, Nick Zacharov, and Soren Forchhammer. 2020. Towards a Perceived Audiovisual Quality Model for Immersive Content. 2020 Twelfth International Conference on Quality of Multimedia Experience (QoMEX), 1–6. https://doi.org/10.1109/QoMEX48832.2020.9123134
Amit Hirway, Yuansong Qiao, and Niall Murray. 2020. A QoE and Visual Attention Evaluation on the Influence of Spatial Audio in 360 Videos. 2020 IEEE International Conference on Artificial Intelligence and Virtual Reality (AIVR), 345–350. https://doi.org/10.1109/AIVR50618.2020.00071
D Hong, H.-J. Kwon, C G Kim, and W.-C. Park. 2018. Real-time 3D audio downmixing system based on sound rendering for the immersive sound of mobile virtual reality applications. KSII Transactions on Internet and Information Systems 12 (2018), 5936–5954. Issue 12. https://doi.org/10.3837/tiis.2018.12.018 cited By 1.
Takashi Kasuya, Manabu Tsukada, Yu Komohara, Shigeki Takasaka, Takuhiro Mizuno, Yoshitaka Nomura, Yuta Ueda, and Hiroshi Esaki. 2019. LiVRation: Remote VR live platform with interactive 3D audio-visual service. 2019 IEEE Games, Entertainment, Media Conference (GEM), 1–7. https://doi.org/10.1109/GEM.2019.8811549 cited By 5.
S Kato, T Ikeda, M Kawamorita, M Tsukada, and H Esaki. 2020. Web3602: An interactive web application for viewing 3D audio-visual Contents. Proceedings of the Sound and Music Computing Conferences 2020-June, 32–39. [link]. cited By 1.
Victoria Korshunova, Gerard B. Remijn, Synes Elischka, and Catarina Mendonca. 2019. The Impact of Sound Systems on the Perception of Cinematic Content in Immersive Audiovisual Productions. 2019 12th Asia Pacific Workshop on Mixed and Augmented Reality (APMAR), 1–5. https://doi.org/10.1109/APMAR.2019.8709163
Ademir Marques, Rafael Kenji Horota, Eniuce Menezes de Souza, Lucas Kupssinskü, Pedro Rossa, Alysson Soares Aires, Leonardo Bachi, Mauricio Roberto Veronez, Luiz Gonzaga, and Caroline Lessio Cazarin. 2020. Virtual and digital outcrops in the petroleum industry: A systematic review. https://doi.org/10.1016/j.earscirev.2020.103260
Bartlomiej Mroz, Marek Kabacinski, Tomasz Ciotucha, Andrzej Ruminski, and Tomasz Zernicki. 2021. Production of six-degrees-of-freedom (6DoF) navigable audio using 30 Ambisonic microphones. 2021 Immersive and 3D Audio: from Architecture to Automotive (I3DA), 1–5. https://doi.org/10.1109/I3DA48870.2021.9610970
Schuyler R. Quackenbush and Jurgen Herre. 2021. MPEG Standards for Compressed Representation of Immersive Audio. Proc. IEEE 109 (9 2021), 1578–1589. Issue 9. https://doi.org/10.1109/JPROC.2021.3075390
Vincent Russell, Rachel Barry, and David Murphy. 2018. HAVE Experience: An Investigation into VR Empathy for Panic Disorder. 2018 IEEE Games, Entertainment, Media Conference (GEM), 1–9. https://doi.org/10.1109/GEM.2018.8516461
Anderson Augusto Simiscuka, Mohammed Amine Togou, Rohit Verma, Mikel Zorrilla, Noel E. O’Connor, and Gabriel-Miro Muntean. 2022. An Evaluation of 360° Video and Audio Quality in an Artistic-Oriented Platform. 2022 IEEE International Symposium on Broadband Multimedia Systems and Broadcasting (BMSB) 2022-June, 1–5. https://doi.org/10.1109/BMSB55706.2022.9828745
Robin Stockert and Aleksander Tidemann. 2022. Hybrid Learning Spaces with Spatial Audio. 2022 29th International Conference on Systems, Signals and Image Processing (IWSSIP) CFP2255E-A, 1–4. https://doi.org/10.1109/IWSSIP55020.2022.9854439
Chi Wang and Chien-Wen Cheng. 2019. Perceiving the World with Hearing in Virtual Reality. 2019 IEEE International Conference on Consumer Electronics - Taiwan (ICCE-TW), 1–2. https://doi.org/10.1109/ICCE-TW46550.2019.8991888
Shu-Nung Yao. 2017. Headphone-based immersive audio for virtual reality headsets. IEEE Transactions on Consumer Electronics 63 (8 2017), 300–308. Issue 3. https://doi.org/10.1109/TCE.2017.014951
Publicado
06/11/2023
Como Citar
DE ALMEIDA, Gustavo Corrêa; DE SOUZA, Vinicius Costa; DA SILVEIRA JÚNIOR, Luiz Gonzaga; VERONEZ, Maurício Roberto.
Spatial Audio in Virtual Reality: A systematic review. In: SIMPÓSIO DE REALIDADE VIRTUAL E AUMENTADA (SVR), 25. , 2023, Rio Grande/RS.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2023
.
p. 264–268.
