A preliminary study of Augmented Musical Instruments for Study (AMIS) using research through design
Resumo
The emergence of Digital Musical Instruments (DMIs) in the computer music field has been providing new means of interaction with music performances. Particularly, with the advancements in the Internet of Things (IoT) area, there has been an increase in augmented musical instruments containing LEDs within their own bodies to support music learning. These instruments can help musicians by having the capabilities to display musical notes, chords and scales. This research uses research through design to present a preliminary study of some of the challenges related to the design of these instruments and attempts to provide some insights associated with their usability and development, particularly related to the development of an augmented acoustic guitar, the VioLED, developed alongside the company Daccord Music. The system provides three modes of operation: song mode, solo/improvising mode and animation mode. The preliminary results present challenges and obstacles pertinent to usability and development of these instruments identified in different iterations of design. These challenges are associated with areas such hardware-software co-design, usability, latency/jitter, energy consumption, etc.
Palavras-chave:
Computer Music and Creative processes, Real-time Interactive Systems
Referências
Aumeo. (n.d.). https://aumeoaudio.com/.
E. R. Miranda and M. M. Wanderley. (2006). New digital musical instruments: control and interaction beyond the keyboard (Vol. 21). AR Editions, Inc.
L. Turchet, C. Fischione and M. Barthet. (2017). Towards the Internet of Musical Things.
K. L. Lueth. (2020). State of the IoT 2020: 12 billion IoT connections, surpassing non-IoT for the first time. IoT Analytics - Market Insights for the Internet of Things. https://bit.ly/3yrD2ac.
The Fretlight Guitar Store. (n.d.). The Fretlight Wireless Guitar Store. Retrieved August 6, 2021, from https://fretlight.com/
Fret Zealot. (2021). Best Way To Learn Guitar | How To Learn Guitar At Home. https://www.fretzealot.com/
Populele - Smart Ukulele (n.d.). Populele Smart Ukulele by PopuBand Music Inc. Retrieved August 6, 2021, from https://popuband.com/
Yamaha Clavinova CSP. (n.d.). Clavinova CSP. Retrieved August 6, 2021, from https://yamaha.io/3yujPED
S. Seol, Y. Shin and K. Lee. (2016). Learning Guitar with an Embedded System. Contemporary Engineering Sciences, 9(12), 553-560.
M. Löchtefeld, S. Gehring, R. Jung, and A. Krüger, guitar: Supporting Guitar Learning through Mobile Projection, CHI 2011 Extended Abstracts on Human Factors in Computing Systems, (2011), 1447-1452.
L. Raymaekers, J. Vermeulen, K. Luyten and K. Coninx (2014). Game of tones: Learning to play songs on a piano using projected instructions and games. In CHI'14 Extended Abstracts on Human Factors in Computing Systems (pp. 411-414).
Music Everywhere. Music Everywhere - Instant Musician AR Piano for iPhone. (n.d.). Retrieved August 6, 2021, from https://www.musiceverywhere.co/home
T. Issa and P. Isaias (2015). Usability and human computer interaction (HCI). In Sustainable design (pp. 19-36). Springer, London.
Dix, A., Finlay, J. E., Abowd, G. D., & Beale, R. (2003). Human-Computer Interaction (3rd ed.). Pearson.
Nielsen, J. (2012). Usability 101: Introduction to Usability. Nielsen Norman Group. https://bit.ly/3ysRyhQ.
K. Y. Huang (2009, October). Challenges in humancomputer interaction design for mobile devices. In Proceedings of the World Congress on Engineering and Computer Science (Vol. 1, No. 6, pp. 1-6).
G. De Michell and R. K. Gupta (1997). Hardware/software co-design. Proceedings of the IEEE, 85(3), 349-365.
D. Chen, J. Cong, S. Gurumani, W. M. Hwu, K. Rupnow and Z. Zhang (2016). Platform choices and design demands for IoT platforms: cost, power, and performance tradeoffs. IET Cyber-Physical Systems: Theory & Applications, 1(1), 70-77.
M. M. Wanderley and N. Orio (2002). Evaluation of input devices for musical expression: Borrowing tools from hci. Computer Music Journal, 26(3), 62-76.
S. Jordà (2005). Digital Lutherie Crafting musical computers for new musics' performance and improvisation. Department of Information and Communication Technologies.
C. Palmer and R. K. Meyer (2000). Conceptual and motor learning in music performance. Psychological Science, 11(1), 63-68.
N. MIKROS (2004) Using Software Prototypes In Game Design. In Game Design Workshop–Designing, Prototyping, and Playtesting Games. [s.l.] CMP Books, 2004. p. 460.
S. Oore (2005). Learning advanced skills on new instruments. In Proceedings of the 2005 conference on New interfaces for musical expression (pp. 60-64).
G. McPherson (Ed.). (2015). The child as musician: A handbook of musical development. Oxford University Press.
H. Pinto (1978). Iniciação ao violão. São Paulo: Ricordi, 1.
A. C. N. do Couto (2009). Música popular e aprendizagem: algumas considerações. Opus, 15(2), 89-104.
S. Jordà (2004, June). Digital instruments and players: part I---efficiency and apprenticeship. In Proceedings of the 2004 conference on New interfaces for musical expression (pp. 59-63).
D. Wessel and M. Wright (2002). Problems and prospects for intimate musical control of computers. Computer music journal, 26(3), 11-22.
J. Zimmerman, J. Forlizzi and S. Evenson (2007). Research through design as a method for interaction design research in HCI. In Proceedings of the SIGCHI conference on Human factors in computing systems (pp. 493-502).
D. Fallman (2003). Design-oriented human-computer interaction. In Proceedings of the SIGCHI conference on Human factors in computing systems (pp. 225- 232).
R. Sigrist, G. Rauter, R. Riener and P. Wolf (2013). Augmented visual, auditory, haptic, and multimodal feedback in motor learning: a review. Psychonomic bulletin & review, 20(1), 21-53.
I. S. MacKenzie and C. Ware (1993). Lag as a determinant of human performance in interactive systems. In Proceedings of the INTERACT'93 and CHI'93 conference on Human factors in computing systems (pp. 488-493).
E. R. Miranda and M. M. Wanderley. (2006). New digital musical instruments: control and interaction beyond the keyboard (Vol. 21). AR Editions, Inc.
L. Turchet, C. Fischione and M. Barthet. (2017). Towards the Internet of Musical Things.
K. L. Lueth. (2020). State of the IoT 2020: 12 billion IoT connections, surpassing non-IoT for the first time. IoT Analytics - Market Insights for the Internet of Things. https://bit.ly/3yrD2ac.
The Fretlight Guitar Store. (n.d.). The Fretlight Wireless Guitar Store. Retrieved August 6, 2021, from https://fretlight.com/
Fret Zealot. (2021). Best Way To Learn Guitar | How To Learn Guitar At Home. https://www.fretzealot.com/
Populele - Smart Ukulele (n.d.). Populele Smart Ukulele by PopuBand Music Inc. Retrieved August 6, 2021, from https://popuband.com/
Yamaha Clavinova CSP. (n.d.). Clavinova CSP. Retrieved August 6, 2021, from https://yamaha.io/3yujPED
S. Seol, Y. Shin and K. Lee. (2016). Learning Guitar with an Embedded System. Contemporary Engineering Sciences, 9(12), 553-560.
M. Löchtefeld, S. Gehring, R. Jung, and A. Krüger, guitar: Supporting Guitar Learning through Mobile Projection, CHI 2011 Extended Abstracts on Human Factors in Computing Systems, (2011), 1447-1452.
L. Raymaekers, J. Vermeulen, K. Luyten and K. Coninx (2014). Game of tones: Learning to play songs on a piano using projected instructions and games. In CHI'14 Extended Abstracts on Human Factors in Computing Systems (pp. 411-414).
Music Everywhere. Music Everywhere - Instant Musician AR Piano for iPhone. (n.d.). Retrieved August 6, 2021, from https://www.musiceverywhere.co/home
T. Issa and P. Isaias (2015). Usability and human computer interaction (HCI). In Sustainable design (pp. 19-36). Springer, London.
Dix, A., Finlay, J. E., Abowd, G. D., & Beale, R. (2003). Human-Computer Interaction (3rd ed.). Pearson.
Nielsen, J. (2012). Usability 101: Introduction to Usability. Nielsen Norman Group. https://bit.ly/3ysRyhQ.
K. Y. Huang (2009, October). Challenges in humancomputer interaction design for mobile devices. In Proceedings of the World Congress on Engineering and Computer Science (Vol. 1, No. 6, pp. 1-6).
G. De Michell and R. K. Gupta (1997). Hardware/software co-design. Proceedings of the IEEE, 85(3), 349-365.
D. Chen, J. Cong, S. Gurumani, W. M. Hwu, K. Rupnow and Z. Zhang (2016). Platform choices and design demands for IoT platforms: cost, power, and performance tradeoffs. IET Cyber-Physical Systems: Theory & Applications, 1(1), 70-77.
M. M. Wanderley and N. Orio (2002). Evaluation of input devices for musical expression: Borrowing tools from hci. Computer Music Journal, 26(3), 62-76.
S. Jordà (2005). Digital Lutherie Crafting musical computers for new musics' performance and improvisation. Department of Information and Communication Technologies.
C. Palmer and R. K. Meyer (2000). Conceptual and motor learning in music performance. Psychological Science, 11(1), 63-68.
N. MIKROS (2004) Using Software Prototypes In Game Design. In Game Design Workshop–Designing, Prototyping, and Playtesting Games. [s.l.] CMP Books, 2004. p. 460.
S. Oore (2005). Learning advanced skills on new instruments. In Proceedings of the 2005 conference on New interfaces for musical expression (pp. 60-64).
G. McPherson (Ed.). (2015). The child as musician: A handbook of musical development. Oxford University Press.
H. Pinto (1978). Iniciação ao violão. São Paulo: Ricordi, 1.
A. C. N. do Couto (2009). Música popular e aprendizagem: algumas considerações. Opus, 15(2), 89-104.
S. Jordà (2004, June). Digital instruments and players: part I---efficiency and apprenticeship. In Proceedings of the 2004 conference on New interfaces for musical expression (pp. 59-63).
D. Wessel and M. Wright (2002). Problems and prospects for intimate musical control of computers. Computer music journal, 26(3), 11-22.
J. Zimmerman, J. Forlizzi and S. Evenson (2007). Research through design as a method for interaction design research in HCI. In Proceedings of the SIGCHI conference on Human factors in computing systems (pp. 493-502).
D. Fallman (2003). Design-oriented human-computer interaction. In Proceedings of the SIGCHI conference on Human factors in computing systems (pp. 225- 232).
R. Sigrist, G. Rauter, R. Riener and P. Wolf (2013). Augmented visual, auditory, haptic, and multimodal feedback in motor learning: a review. Psychonomic bulletin & review, 20(1), 21-53.
I. S. MacKenzie and C. Ware (1993). Lag as a determinant of human performance in interactive systems. In Proceedings of the INTERACT'93 and CHI'93 conference on Human factors in computing systems (pp. 488-493).
Publicado
24/10/2021
Como Citar
SILVA, Eduardo Santos; CABRAL, Giordano Ribeiro Eulalio; PEREIRA, Rodrigo Mendes de Carvalho.
A preliminary study of Augmented Musical Instruments for Study (AMIS) using research through design. In: SIMPÓSIO BRASILEIRO DE COMPUTAÇÃO MUSICAL (SBCM), 18. , 2021, Recife.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2021
.
p. 136-143.
DOI: https://doi.org/10.5753/sbcm.2021.19438.
