AutoTab: An Interactive System for Automatic Guitar Tablature Transcription
Resumo
This work introduces AutoTab, an interactive system designed to simplify the process of transcribing music into guitar tablature. By combining sound event detection, Convolutional Neural Networks (CNNs), and graph-based optimization algorithms, AutoTab offers an intuitive and accessible solution for musicians of all skill levels. The system employs precise note and chord identification through CNNs, a WebSocket-based server for real-time audio analysis, and specialized algorithms such as the Audio Window Analyser for improved sound event detection and TabGen for optimized tablature generation. Initial results demonstrate robust performance, with the Notes Model achieving 87.8% accuracy in note identification and the Chords Model achieving a 78% Macro F1-score in chord recognition. AutoTab aims to democratize access to high-quality transcription technology, enhancing musical practice and learning for guitarists worldwide. Future work will focus on addressing the challenges in polyphonic transcription and expanding the system’s functionality.
Palavras-chave:
Audio processing, Convolutional Neural Networks for music, Analysis of musical elements, Tablature generation, Interactive web application, Note and chord recognition
Referências
Martín Abadi, Ashish Agarwal, Paul Barham, Eugene Brevdo, Zhifeng Chen, Craig Citro, Greg S. Corrado, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Ian Goodfellow, Andrew Harp, Geoffrey Irving, Michael Isard, Yangqing Jia, Rafal Jozefowicz, Lukasz Kaiser, Manjunath Kudlur, Josh Levenberg, Dandelion Mané, Rajat Monga, Sherry Moore, Derek Murray, Chris Olah, Mike Schuster, Jonathon Shlens, Benoit Steiner, Ilya Sutskever, Kunal Talwar, Paul Tucker, Vincent Vanhoucke, Vijay Vasudevan, Fernanda Viégas, Oriol Vinyals, Pete Warden, Martin Wattenberg, Martin Wicke, Yuan Yu, and Xiaoqiang Zheng. TensorFlow: Large-scale machine learning on heterogeneous systems, 2015. Software available from [link].
M. Alonso, G. Richard, and B. David. Extracting note onsets from musical recordings. 2005 IEEE International Conference on Multimedia and Expo, pages 4 pp.–, 2005.
J.P. Bello, L. Daudet, S. Abdallah, C. Duxbury, M. Davies, and M.B. Sandler. A tutorial on onset detection in music signals. IEEE Transactions on Speech and Audio Processing, 13(5):1035–1047, 2005.
Emmanouil Benetos, S. Dixon, Z. Duan, and S. Ewert. Automatic music transcription: An overview. IEEE Signal Processing Magazine, 36:20–30, 2019.
J. Bondy and U. Murty. Graph theory. pages 1–582, 2008.
V. Chiesa, P. Coughlan, and C. Voss. Development of a technical innovation audit. Journal of Product Innovation Management, 13:105–136, 1996.
Frank Cwitkowitz, T. Hirvonen, and A. Klapuri. Fretnet: Continuous-valued pitch contour streaming for polyphonic guitar tablature transcription. ArXiv, abs/2212.03023, 2022.
Maria Luisa de Mattos Priolli. Princípios básicos da música para juventude, 1999.
L. Frenzel. Chapter 10 – audio electronics: Digital voice and music dominate. pages 229–246, 2010.
Ian Goodfellow, Yoshua Bengio, and Aaron Courville. Deep Learning. MIT Press, 2016. [link].
Simon S. Haykin. Neural networks and learning machines. Pearson Education, Upper Saddle River, NJ, third edition, 2009.
Sicong Huang, Qiyang Li, Cem Anil, Xuchan Bao, Sageev Oore, and Roger B. Grosse. Timbretron: A wavenet(cyclegan(cqt(audio))) pipeline for musical timbre transfer, 2023.
Yogesh Jadhav, Ashish Patel, Rutvij H. Jhaveri, and Roshani Raut. Transfer learning for audio waveform to guitar chord spectrograms using the convolution neural network. Mobile Information Systems, 2022:1–11, 2022.
Yogesh Jadhav, Ashish Patel, Rutvij H. Jhaveri, and Roshani Raut. Transfer learning for audio waveform to guitar chord spectrograms using the convolution neural network. Mobile Information Systems, 2022:11, 2022.
Christian Kehling, Jakob Abeßer, Christian Dittmar, and Gerald Schuller. Automatic tablature transcription of electric guitar recordings by estimation of score-and instrument-related parameters. In Proceedings of the 17th International Conference on Digital Audio Effects (DAFx), Erlangen, Germany, 2014.
Jong Wook Kim, Justin Salamon, Peter Li, and Juan Pablo Bello. Crepe: A convolutional representation for pitch estimation, 2018.
Jackson Loth, Pedro Sarmento, CJ Carr, Zack Zukowski, and Mathieu Barthet. Proggp: From guitarpro tablature neural generation to progressive metal production, 2023.
Rock Out Loud. Tablature vs sheet music explained, 2025. Accessed: 2025-03-10.
Meinard Müller. Fundamentals of Music Processing: Audio, Analysis, Algorithms, Applications. Springer Cham, 1 edition, 2015.
Huy Phan, Lars Hertel, Marco Maass, and Alfred Mertins. Robust audio event recognition with 1-max pooling convolutional neural networks. arXiv preprint arXiv:1604.06338, 2016.
Polifono. Como aprender a ler partitura e por qual motivo, 2020. Acessado em: 18 de Janeiro de 2024.
Hendrik Purwins, Bo Li, Tuomas Virtanen, Jan Schluter, Shuo-Yiin Chang, and Tara Sainath. Deep learning for audio signal processing. IEEE Journal of Selected Topics in Signal Processing, 13(2):206–219, may 2019.
Miguel A. Román, Antonio Pertusa, and Jorge Calvo-Zaragoza. A holistic approach to polyphonic music transcription with neural networks, 2019.
Jan Schlüter and Sebastian Böck. Improved musical onset detection with convolutional neural networks. In 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 6979–6983, 2014.
A. C. P. R. Silva. A música no processo de ensino e aprendizagem. Trabalho de Conclusão de Curso (Especialização em Educação: Métodos e Técnicas de Ensino) - Universidade Tecnológica Federal do Paraná, 2021.
Julius O. Smith. Spectral Audio Signal Processing. [link], accessed 2024. online book, 2011 edition.
D.R. Tuohy and W.D. Potter. Ga-based music arranging for guitar. In 2006 IEEE International Conference on Evolutionary Computation, pages 1065–1070, 2006.
G. Tzanetakis and P. Cook. Musical genre classification of audio signals. IEEE Transactions on Speech and Audio Processing, 10(5):293–302, 2002.
Marcel A. Vélez Vásquez, Mariëlle Baelemans, Jonathan Driedger, Willem Zuidema, and John Ashley Burgoyne. Quantifying the ease of playing song chords on the guitar. In Proceedings of the 24th International Society for Music Information Retrieval Conference, pages 725–732, Milan, Italy, 2023. ISMIR. © M.A. Vélez Vásquez, M.C.E. Baelemans, J. Driedger, W.H. Zuidema, and J.A. Burgoyne. Licensed under a Creative Commons Attribution 4.0 International License (CC BY 4.0).
Q. Xi, R. Bittner, J. Pauwels, X. Ye, and J. P. Bello. Guitarset: A dataset for guitar transcription. In 19th International Society for Music Information Retrieval Conference, Paris, France, Sept 2018.
Jin Y Yen. An algorithm for finding shortest routes from all source nodes to a given destination in general networks. Quarterly of Applied Mathematics, 27(4):526–530, 1970.
R. Zhou and J. Reiss. Music onset detection. pages 297–316, 2011.
M. Alonso, G. Richard, and B. David. Extracting note onsets from musical recordings. 2005 IEEE International Conference on Multimedia and Expo, pages 4 pp.–, 2005.
J.P. Bello, L. Daudet, S. Abdallah, C. Duxbury, M. Davies, and M.B. Sandler. A tutorial on onset detection in music signals. IEEE Transactions on Speech and Audio Processing, 13(5):1035–1047, 2005.
Emmanouil Benetos, S. Dixon, Z. Duan, and S. Ewert. Automatic music transcription: An overview. IEEE Signal Processing Magazine, 36:20–30, 2019.
J. Bondy and U. Murty. Graph theory. pages 1–582, 2008.
V. Chiesa, P. Coughlan, and C. Voss. Development of a technical innovation audit. Journal of Product Innovation Management, 13:105–136, 1996.
Frank Cwitkowitz, T. Hirvonen, and A. Klapuri. Fretnet: Continuous-valued pitch contour streaming for polyphonic guitar tablature transcription. ArXiv, abs/2212.03023, 2022.
Maria Luisa de Mattos Priolli. Princípios básicos da música para juventude, 1999.
L. Frenzel. Chapter 10 – audio electronics: Digital voice and music dominate. pages 229–246, 2010.
Ian Goodfellow, Yoshua Bengio, and Aaron Courville. Deep Learning. MIT Press, 2016. [link].
Simon S. Haykin. Neural networks and learning machines. Pearson Education, Upper Saddle River, NJ, third edition, 2009.
Sicong Huang, Qiyang Li, Cem Anil, Xuchan Bao, Sageev Oore, and Roger B. Grosse. Timbretron: A wavenet(cyclegan(cqt(audio))) pipeline for musical timbre transfer, 2023.
Yogesh Jadhav, Ashish Patel, Rutvij H. Jhaveri, and Roshani Raut. Transfer learning for audio waveform to guitar chord spectrograms using the convolution neural network. Mobile Information Systems, 2022:1–11, 2022.
Yogesh Jadhav, Ashish Patel, Rutvij H. Jhaveri, and Roshani Raut. Transfer learning for audio waveform to guitar chord spectrograms using the convolution neural network. Mobile Information Systems, 2022:11, 2022.
Christian Kehling, Jakob Abeßer, Christian Dittmar, and Gerald Schuller. Automatic tablature transcription of electric guitar recordings by estimation of score-and instrument-related parameters. In Proceedings of the 17th International Conference on Digital Audio Effects (DAFx), Erlangen, Germany, 2014.
Jong Wook Kim, Justin Salamon, Peter Li, and Juan Pablo Bello. Crepe: A convolutional representation for pitch estimation, 2018.
Jackson Loth, Pedro Sarmento, CJ Carr, Zack Zukowski, and Mathieu Barthet. Proggp: From guitarpro tablature neural generation to progressive metal production, 2023.
Rock Out Loud. Tablature vs sheet music explained, 2025. Accessed: 2025-03-10.
Meinard Müller. Fundamentals of Music Processing: Audio, Analysis, Algorithms, Applications. Springer Cham, 1 edition, 2015.
Huy Phan, Lars Hertel, Marco Maass, and Alfred Mertins. Robust audio event recognition with 1-max pooling convolutional neural networks. arXiv preprint arXiv:1604.06338, 2016.
Polifono. Como aprender a ler partitura e por qual motivo, 2020. Acessado em: 18 de Janeiro de 2024.
Hendrik Purwins, Bo Li, Tuomas Virtanen, Jan Schluter, Shuo-Yiin Chang, and Tara Sainath. Deep learning for audio signal processing. IEEE Journal of Selected Topics in Signal Processing, 13(2):206–219, may 2019.
Miguel A. Román, Antonio Pertusa, and Jorge Calvo-Zaragoza. A holistic approach to polyphonic music transcription with neural networks, 2019.
Jan Schlüter and Sebastian Böck. Improved musical onset detection with convolutional neural networks. In 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 6979–6983, 2014.
A. C. P. R. Silva. A música no processo de ensino e aprendizagem. Trabalho de Conclusão de Curso (Especialização em Educação: Métodos e Técnicas de Ensino) - Universidade Tecnológica Federal do Paraná, 2021.
Julius O. Smith. Spectral Audio Signal Processing. [link], accessed 2024. online book, 2011 edition.
D.R. Tuohy and W.D. Potter. Ga-based music arranging for guitar. In 2006 IEEE International Conference on Evolutionary Computation, pages 1065–1070, 2006.
G. Tzanetakis and P. Cook. Musical genre classification of audio signals. IEEE Transactions on Speech and Audio Processing, 10(5):293–302, 2002.
Marcel A. Vélez Vásquez, Mariëlle Baelemans, Jonathan Driedger, Willem Zuidema, and John Ashley Burgoyne. Quantifying the ease of playing song chords on the guitar. In Proceedings of the 24th International Society for Music Information Retrieval Conference, pages 725–732, Milan, Italy, 2023. ISMIR. © M.A. Vélez Vásquez, M.C.E. Baelemans, J. Driedger, W.H. Zuidema, and J.A. Burgoyne. Licensed under a Creative Commons Attribution 4.0 International License (CC BY 4.0).
Q. Xi, R. Bittner, J. Pauwels, X. Ye, and J. P. Bello. Guitarset: A dataset for guitar transcription. In 19th International Society for Music Information Retrieval Conference, Paris, France, Sept 2018.
Jin Y Yen. An algorithm for finding shortest routes from all source nodes to a given destination in general networks. Quarterly of Applied Mathematics, 27(4):526–530, 1970.
R. Zhou and J. Reiss. Music onset detection. pages 297–316, 2011.
Publicado
20/07/2025
Como Citar
MATOS, Athos; TORRES, Daniel Teodolino Barbosa; TAKAHASHI, Adriana; PARADEDA, Raul Benites.
AutoTab: An Interactive System for Automatic Guitar Tablature Transcription. In: SEMINÁRIO INTEGRADO DE SOFTWARE E HARDWARE (SEMISH), 52. , 2025, Maceió/AL.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2025
.
p. 275-286.
ISSN 2595-6205.
DOI: https://doi.org/10.5753/semish.2025.8431.
