Seleção de amostras baseada em transfer learning e autoaprendizagem para tarefas zero-shot

• Matheus V. Todescato UFRGS
• Joel L. Carbonera UFRGS

DOI: https://doi.org/10.5753/eramiars.2025.16741

Resumo

Em ambientes com escassez de dados visuais rotulados, como no contexto médico ou geológico, modelos de aprendizado profundo sofrem com problemas de desempenho. Nesse contexto, desenvolvemos um pipeline utilizando transfer learning e auto-aprendizagem para viabilizar o treinamento de um classificador, podendo ser utilizado para diferentes tarefas zero-shot como classificação de imagens ou detecção de amostras de fora da distribuição (OOD). Avaliações experimentais demonstram que nossa abordagem supera o estado-da-arte dessas tarefas.

Referências

Esmaeilpour, S., Liu, B., Robertson, E., and Shu, L. (2022). Zero-shot out-of-distribution detection based on the pre-trained model clip. In Proceedings of the AAAI conference on artificial intelligence, volume 36, pages 6568–6576.

Ming, Y., Cai, Z., Gu, J., Sun, Y., Li, W., and Li, Y. (2022). Delving into out-of-distribution detection with vision-language representations. Advances in neural information processing systems, 35:35087–35102.

Mirza, M. J., Karlinsky, L., Lin, W., Possegger, H., Kozinski, M., Feris, R., and Bischof, H. (2023). Lafter: Label-free tuning of zero-shot classifier using language and unlabeled image collections. Advances in Neural Information Processing Systems, 36:5765–5777.

Miyai, A., Yu, Q., Irie, G., and Aizawa, K. (2025). Gl-mcm: Global and local maximum concept matching for zero-shot out-of-distribution detection. International Journal of Computer Vision, pages 1–11.

Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al. (2021). Learning transferable visual models from natural language supervision. In International conference on machine learning, pages 8748–8763. PmLR.

Saha, O., Van Horn, G., and Maji, S. (2024). Improved zero-shot classification by adapting vlms with text descriptions. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 17542–17552.

Todescato, M. V. and Carbonera, J. L. (2024). Investigating performance patterns of pre-trained models for feature extraction in image classification. In 2024 IEEE 36th International Conference on Tools with Artificial Intelligence (ICTAI), pages 1024–1031. IEEE.

Torrey, L. and Shavlik, J. (2010). Transfer learning. In Handbook of research on machine learning applications and trends: algorithms, methods, and techniques, pages 242–264. IGI global.

Zhuang, F. et al. (2020). A comprehensive survey on transfer learning. In Proceedings of the IEEE 109, pages 43–76, 1.