Improving Soundscape Retrieval for Bioacoustic Monitoring: An Analysis of Fusion Techniques with Pre-trained Embeddings - Peralta, Andrés D.; Santos, Eulanda Miranda dos; Gordo, Marcelo; Xie, Jie; Colonna, Juan G. (Proceedings of the Brazilian Symposium on Multimedia and the Web (WebMedia), 2025)
The retrieval of similar soundscapes is essential for bioacoustic and ecoacoustic monitoring, yet it remains challenging due to the large volume of unlabeled data, environmental noise, and the complexity of acoustic scenes. To overcome the limitations of traditional, feature-based methods, this study proposes an efficient system that integrates embeddings extracted from a pretrained deep learning model, combined with a noise reduction technique and feature fusion strategies within a vector database to enable similarity-based retrieval. We evaluated the system using bird, amphibian, and mammal recordings across four experimental methodologies, including a use case focused on endangered species. Results show that embedding vectors consistently outperform traditional MFCC (Mel-frequency cepstral coefficients) features in capturing acoustic similarity, and that approximate search algorithms (HNSW) significantly improve both retrieval precision and query efficiency. Additionally, the system effectively retrieves recordings of the critically endangered species Crax alberti and maps their geographic distribution, highlighting its potential for conservation planning and early-warning monitoring.