Evaluating Vector Fusion Techniques for Unsupervised Soundscape Retrieval using Deep Embeddings

• Andrés D. Peralta UFAM
• Alejandro C. Frery VUW
• Jie Xie Nanjing Normal University
• Juan G. Colonna UFAM / VUW

DOI: https://doi.org/10.5753/wcama.2026.22589

Resumo

The efficient storage, management, and retrieval of vast volumes of bioacoustic data represent a critical bottleneck for long-term biodiversity monitoring. To address this challenge, this work proposes an unsupervised soundscape retrieval system that integrates high-dimensional embeddings from the pre-trained Perch V2 model with a vector fusion strategy that handles variable-length recordings. We systematically evaluate retrieval efficiency and accuracy by comparing two retrieval algorithms and four vector fusion techniques on a vector database. The methodology was validated using a multitaxonomic dataset comprising birds, mammals, and amphibians. A case study involving eleven species of conservation interest shows that the system significantly outperforms traditional MFCC-based approaches, offering a scalable, robust solution for autonomous biodiversity inventorying.

Palavras-chave: Biodiversity Monitoring, Vector Fusion, Vector Databases, Acoustic Similarity Retrieval

Referências

M. J. Bianco, P. Gerstoft, J. Traer, E. Ozanich, M. A. Roch, S. Gannot, and C.-A. Deledalle. Machine learning in acoustics: Theory and applications. The Journal of the Acoustical Society of America, 146:3590–3628, 2019. DOI: 10.1121/1.5133944.

J. Bjorck, B. H. Rappazzo, D. Chen, R. Bernstein, P. H. Wrege, and C. P. Gomes. Automatic Detection and Compression for Passive Acoustic Monitoring of the African Forest Elephant. Proceedings of the AAAI Conference on Artificial Intelligence, pages 476–484, 2019. DOI: 10.1609/aaai.v33i01.3301476.

Y. Chen, Z. Lai, Y. Ding, K. Lin, and W. K. Wong. Deep supervised hashing with anchor graph. In Proceedings of the IEEE/CVF international conference on computer vision, pages 9796–9804, 2019.

B. Ghani, T. Denton, S. Kahl, and H. Klinck. Global birdsong embeddings enable superior transfer learning for bioacoustic classification. Scientific Reports, 2023. DOI: 10.1038/s41598-023-49989-z.

J. Hamer, E. Triantafillou, B. van Merriënboer, T. Denton, V. Dumoulin, S. Kahl, and H. Klinck. Birb: A generalization benchmark for information retrieval in bioacoustics. Preprint under review, 2023. URL [link].

iNaturalist community. iNaturalist – Citizen science platform for biodiversity observations. Online at [link], 2025.

A. Jati and D. Emmanouilidou. Supervised deep hashing for efficient audio event retrieval. In IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 4497–4501, 2020. DOI: 10.1109/ICASSP40776.2020.9053766.

H. Klinck, J. S. Cañas, M. Demkin, S. Dane, S. Kahl, and T. Denton. Birdclef+ 2025. [link], 2025. Kaggle.

O. Krauss, M. Balbino, and C. Nobre. Evaluation of methods of counterfactual explanation - a qualitative and quantitative analysis. In Anais do XI Symposium on Knowledge Discovery, Mining and Learning. SBC, 2023. DOI: 10.5753/kdmile.2023.232932.

Y. Liang, S. Zhang, L. K. Li, and X. Wang. Unleashing the full potential of product quantization for large-scale image retrieval. Advances in Neural Information Processing Systems, pages 61712–61724, 2023.

Y. A. Malkov and D. A. Yashunin. Efficient and Robust Approximate Nearest Neighbor Search Using Hierarchical Navigable Small World Graphs. IEEE Transactions on Pattern Analysis and Machine Intelligence, pages 824–836, 2020. DOI: 10.1109/TPAMI.2018.2889473.

D. Murillo Bedoya, A. Buitrago-Cardona, O. Acevedo-Charry, and J. M. Ochoa-Quintero. Colección de sonidos ambientales mauricio Álvarez-rebolledo (iavh-csa). Instituto Humboldt (Colombia), 2021. URL [link].

B. C. Pijanowski, L. J. Villanueva-Rivera, S. L. Dumyahn, A. Farina, B. L. Krause, B. M. Napoletano, S. H. Gage, and N. Pieretti. Soundscape Ecology: The Science of Sound in the Landscape. BioScience, 61:203–216, 2011. DOI: 10.1525/bio.2011.61.3.6.

M. Ravanelli, B. Elizalde, K. Ni, and G. Friedland. Audio concept classification with hierarchical deep neural networks. In 22nd European Signal Processing Conference (EUSIPCO), pages 606–610, 2014.

M. Sert and A. M. Başbuğ. Combining acoustic and semantic similarity for acoustic scene retrieval. In 2019 IEEE International Symposium on Multimedia (ISM), 2019. DOI: 10.1109/ISM46123.2019.00036.

The IUCN Red List of Threatened Species. Summary statistics – the iucn red list of threatened species, July 2025. URL [link].

I. Tolkova. Feature representations for conservation bioacoustics: Review and discussion. Harvard University, 2019.

B. van Merriënboer, V. Dumoulin, J. Hamer, L. Harrell, A. Burns, and T. Denton. Perch 2.0: The bittern lesson for bioacoustics. arXiv, 2025. URL [link].

Vellinga, W.P. and Planqué, R. Xeno-canto – Bird sounds from around the world. GBIF Occurrence Dataset, 2025. URL [link].

A. L.-C. Wang. An industrial strength audio search algorithm. In Proceedings of the 4th International Conference on Music Information Retrieval (ISMIR), 2003.

G. Wichern, J. Xue, H. Thornburg, B. Mechtley, and A. Spanias. Segmentation, indexing, and retrieval for environmental and natural sounds. IEEE Transactions on Audio, Speech, and Language Processing, 18(3):688–707, 2010. DOI: 10.1109/TASL.2010.2041384.

J. Xie and M. Zhu. Acoustic classification of bird species using an early fusion of deep features. Birds, page 11, 2023. DOI: 10.3390/birds4010011. URL [link].