Recuperação Vocal de Pacientes Traqueostomizados Utilizando Processamento de Áudio e Aprendizado de Máquina em Cenário Zero-Shot - Freitas Filho, Mario Pinto; Almeida, João Dallyson Sousa de; Braz Junior, Geraldo (Proceedings of the Brazilian Symposium on Multimedia and the Web (WebMedia); 2025: Proceedings of the 31st Brazilian Symposium on Multimedia and the Web, 2025)
This work proposes a method for voice reconstruction in individuals who have undergone laryngectomy, integrating advanced audio processing and machine learning techniques. The approach aims to restore features such as timbre, intonation, and prosody, which are often lost when using an electronic larynx, whose sound is constrained by a constant fundamental frequency (F0). To address the lack of public datasets containing voices of tracheostomized patients, a synthetic dataset was created to simulate the acoustic properties of these devices. The developed pipeline comprises three stages: (i) speech analysis, involving the extraction of linguistic content and style; (ii) mapping, combining this information with the mel-spectrogram through techniques such as conditional modulation and diffusion networks, with a particular focus on Flow Matching; and (iii) reconstruction and synthesis, using highfidelity vocoders. Experiments compared two preprocessing methods— timbre shifter and F0 fixation—evaluated in four training and testing combinations. Results show that the F0→F0 configuration outperformed the alternative in three out of the four analyzed metrics (MCD of 444.04, LSD of 0.47, and PSNR of 42.27), suggesting that F0 fixation favors voice reconstruction that more closely matches the original signal. These findings highlight the potential of the proposed approach to improve the naturalness and intelligibility of synthesized speech for laryngectomized patients.