Motus 4: new developments on a web-deployed software for composing real-time audiovisual interactions
Resumo
Camera-based audiovisual interactions have been greatly used in art installations. For such, it is often necessary to use specialized hardware and software. This restricts audiovisual installations to areas that can both afford them financially and be physically accessed by the audience. In this work, we present the development of a web-deployed composition tool for audiovisual interactions that runs on the client side and does not require installing any additional software. The tool allows quickly configuring areas for movement capture, and customizing the corresponding audio feedback. Simultaneously, it provides visual feedback that can aid the audience to understand the experience. Consequently, the tool can be used to compose audiovisual interactions that reach a large audience via web. Moreover, its virtual and ubiquitous nature can foster a diversity of artistic explorations related to the remote characteristic of the relationship between audience and artwork.
Palavras-chave:
Computer Music and Creative processes, Real-time Interactive Systems, Sensors and Multimodal Signal Processing
Referências
Clayton Mamedes. Participação em instalações interativas (in portuguese). Revista Interdisciplinar Internacional de Artes Visuais, 5(1), 2018.
Raul Paiva and Tiago Fernandes Tavares. Impact of algorithmic composition on player immersion in computer games: A case study using markov chains. Revista Música Hodie, 18(1):61–73, 2018.
Todd Winkler. Motion-sensing music: Artistic and technical challenges in two works for dance. In Proceedings of the International Computer Music Conference, 01 1998.
Tiago Fernandes Tavares, Gabriel Rimoldi, Vânia Eger Pontes, and Jônatas Manzoli. Cooperative musical creation using kinect, wiimote, epoc and microphones: a case study with mindsounds. In Sound and Music Computing 2015, 2015.
João Araújo, Avner de Paulo, Igino Silva Junior, Flávio Schiavoni, Mauro César Canito, and Rômulo Costa. A technical approach of the audience participation in the performance ’o chaos das 5’. In Anais do XVII Simpósio Brasileiro de Computação Musical, pages 28–34, Porto Alegre, RS, Brasil, 2019. SBC.
S. Trail, M. Dean, T. F. Tavares, G. Odowichuk, P. Driessen, A. W. Schloss, and G. Tzanetakis. Noninvasive sensing and gesture control for pitched percussion hyper-instruments using the kinect. In New Interfaces for Musical Expression - NIME 2012, Ann Arbour, Michigan, U.S.A., May 2012.
Damián Keller, Cláudio Gomes, and Luzilei Aliel. The handy metaphor: Bimanual, touchless interaction for the internet of musical things. Journal of New Music Research, 48(4):385–396, 2019.
Filipe Calegario, João Tragtenberg, Giordano Cabral, and Geber Ramalho. Batebit controller: Popularizing digital musical instruments’ technical development process. In Anais do XVII Simpósio Brasileiro de Computação Musical, pages 217–218, Porto Alegre, RS, Brasil, 2019. SBC.
Cristiano Figueiró, Guilherme Soares, and Bruno Rohde. Esmeril - an interactive audio player and composition system for collaborative experimental music netlabels. In NIME, 2019.
Tiago Fernandes Tavares. An interactive audio-visual installation using ubiquitous hardware and web-based software deployment. PeerJ Computer Science, 1:e5, 5 2015.
Clayton Rosa Mamedes, Denise H. L. Garcia, José Fornari, and Jônatas Manzolli. Abstrações: An audiovisual installation based on motion recognizing. In 2011 4th International Congress on Image and Signal Processing, volume 2, pages 586–589, 2011.
Rômulo Vieira and Flávio Schiavoni. Fliperama: An affordable arduino based midi controller. In Proceedings of Nime 2020, 2020.
Cristiano Severo Figueiró. Desobediência sonora: selos de música experimental e suas tecnologias de sustentabilidade. EDUFBA, 2019.
NIME 2020. ”installations” accessed jul. 15, 2021. https://nime2020.bcu.ac.uk/installations/.
Mutek. ”mutek virtual” accessed jul. 15, 2021. https://virtual.mutek.org/en/mutek.
Ash Turner. ”how many smartphones are in the world?” accessed jun. 23, 2021. https://www.bankmycell.com/blog/how-many-phones-are-in-the-world. 18th Brazilian Symposium on Computer Music - SBCM 2021
zugara.com. ”webcam penetration rates adoption” accessed jun. 23, 2021. http://zugara.com/webcam-penetration-rates-adoption.
Shirlene Vianna Moreira and Francis Ricardo dos Reis Justi and Marcos Moreira. Intervenção musical pode melhorar a memória em pacientes com doença de alzheimer? uma revisão sistemática. 2018.
Thiago Holanda and Paulo José Moté Barboza and Dayana Priscila Maia Mejia. Tratamento fisioterapêutico em pacientes acometidos por alzheimer: Referência bibliográfica. 2012.
Wyzowl. ”why video is exploding on social media in 2021” accessed jul. 14, 2021. https://www.wyzowl.com/video-social-media-2020/.
Zhang Yiming. ”homepage” accessed jul. 14, 2021. https://www.tiktok.com/pt-BR/.
JOYO TECHNOLOGY PTE. LTD. ”about us” accessed jul. 14, 2021. https://www.kwai.com/about/.
Heather Shipman. Smart art for smart cities. In Miroslav Mateev and Panikkos Poutziouris, editors, Creative Business and Social Innovations for a Sustainable Future, pages 251–253, Cham, 2019. Springer International Publishing.
Processing Foundation. ”overview. a short introduction to the processing software and projects from the community.” accessed jun. 07, 2021. https://www.processing.org/overview/.
Processing Foundation. ”homepage” accessed jun. 07, 2021. https://www.processingfoundation.org/.
Raul Paiva and Tiago Fernandes Tavares. Impact of algorithmic composition on player immersion in computer games: A case study using markov chains. Revista Música Hodie, 18(1):61–73, 2018.
Todd Winkler. Motion-sensing music: Artistic and technical challenges in two works for dance. In Proceedings of the International Computer Music Conference, 01 1998.
Tiago Fernandes Tavares, Gabriel Rimoldi, Vânia Eger Pontes, and Jônatas Manzoli. Cooperative musical creation using kinect, wiimote, epoc and microphones: a case study with mindsounds. In Sound and Music Computing 2015, 2015.
João Araújo, Avner de Paulo, Igino Silva Junior, Flávio Schiavoni, Mauro César Canito, and Rômulo Costa. A technical approach of the audience participation in the performance ’o chaos das 5’. In Anais do XVII Simpósio Brasileiro de Computação Musical, pages 28–34, Porto Alegre, RS, Brasil, 2019. SBC.
S. Trail, M. Dean, T. F. Tavares, G. Odowichuk, P. Driessen, A. W. Schloss, and G. Tzanetakis. Noninvasive sensing and gesture control for pitched percussion hyper-instruments using the kinect. In New Interfaces for Musical Expression - NIME 2012, Ann Arbour, Michigan, U.S.A., May 2012.
Damián Keller, Cláudio Gomes, and Luzilei Aliel. The handy metaphor: Bimanual, touchless interaction for the internet of musical things. Journal of New Music Research, 48(4):385–396, 2019.
Filipe Calegario, João Tragtenberg, Giordano Cabral, and Geber Ramalho. Batebit controller: Popularizing digital musical instruments’ technical development process. In Anais do XVII Simpósio Brasileiro de Computação Musical, pages 217–218, Porto Alegre, RS, Brasil, 2019. SBC.
Cristiano Figueiró, Guilherme Soares, and Bruno Rohde. Esmeril - an interactive audio player and composition system for collaborative experimental music netlabels. In NIME, 2019.
Tiago Fernandes Tavares. An interactive audio-visual installation using ubiquitous hardware and web-based software deployment. PeerJ Computer Science, 1:e5, 5 2015.
Clayton Rosa Mamedes, Denise H. L. Garcia, José Fornari, and Jônatas Manzolli. Abstrações: An audiovisual installation based on motion recognizing. In 2011 4th International Congress on Image and Signal Processing, volume 2, pages 586–589, 2011.
Rômulo Vieira and Flávio Schiavoni. Fliperama: An affordable arduino based midi controller. In Proceedings of Nime 2020, 2020.
Cristiano Severo Figueiró. Desobediência sonora: selos de música experimental e suas tecnologias de sustentabilidade. EDUFBA, 2019.
NIME 2020. ”installations” accessed jul. 15, 2021. https://nime2020.bcu.ac.uk/installations/.
Mutek. ”mutek virtual” accessed jul. 15, 2021. https://virtual.mutek.org/en/mutek.
Ash Turner. ”how many smartphones are in the world?” accessed jun. 23, 2021. https://www.bankmycell.com/blog/how-many-phones-are-in-the-world. 18th Brazilian Symposium on Computer Music - SBCM 2021
zugara.com. ”webcam penetration rates adoption” accessed jun. 23, 2021. http://zugara.com/webcam-penetration-rates-adoption.
Shirlene Vianna Moreira and Francis Ricardo dos Reis Justi and Marcos Moreira. Intervenção musical pode melhorar a memória em pacientes com doença de alzheimer? uma revisão sistemática. 2018.
Thiago Holanda and Paulo José Moté Barboza and Dayana Priscila Maia Mejia. Tratamento fisioterapêutico em pacientes acometidos por alzheimer: Referência bibliográfica. 2012.
Wyzowl. ”why video is exploding on social media in 2021” accessed jul. 14, 2021. https://www.wyzowl.com/video-social-media-2020/.
Zhang Yiming. ”homepage” accessed jul. 14, 2021. https://www.tiktok.com/pt-BR/.
JOYO TECHNOLOGY PTE. LTD. ”about us” accessed jul. 14, 2021. https://www.kwai.com/about/.
Heather Shipman. Smart art for smart cities. In Miroslav Mateev and Panikkos Poutziouris, editors, Creative Business and Social Innovations for a Sustainable Future, pages 251–253, Cham, 2019. Springer International Publishing.
Processing Foundation. ”overview. a short introduction to the processing software and projects from the community.” accessed jun. 07, 2021. https://www.processing.org/overview/.
Processing Foundation. ”homepage” accessed jun. 07, 2021. https://www.processingfoundation.org/.
Publicado
24/10/2021
Como Citar
AMSTALDEN, Augusto Piato; TAVARES, Tiago Fernandes.
Motus 4: new developments on a web-deployed software for composing real-time audiovisual interactions. In: SIMPÓSIO BRASILEIRO DE COMPUTAÇÃO MUSICAL (SBCM), 18. , 2021, Recife.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2021
.
p. 109-113.
DOI: https://doi.org/10.5753/sbcm.2021.19434.
