Prototypical Few-Shot Classification with Vision-Language Models: A Taxonomy and Systematic Study
Resumo
Prototypical few-shot methods on frozen vision-language models combine a feature transformation, a similarity metric, and a visual-text fusion strategy in ways that resist direct comparison. We organize the quadratic-distance methods as a single scoring family parametrized by a feature transformation and a geometry matrix, recovering Prototypical Networks, cosine similarity, and SimpleShot’s L2N variant; Tip-Adapter sits in a separate kernel-based sibling family. Inside it we propose a diagonal Mahalanobis variant whose geometry is estimated from the support set in closed form, with no base-class data and no gradient training, gaining about 2 points over cosine on CUB-200-2011 and 4 on EuroSAT at 1-shot, with overlapping intervals on miniImageNet. All methods are released in one open-source library.
Referências
Alayrac, J.-B., Donahue, J., Luc, P., Miech, A., Barr, I., Hasson, Y., Lenc, K., Mensch, A., Millican, K., Reynolds, M., et al. (2022). Flamingo: a visual language model for few-shot learning. NeurIPS, 35:23716–23736.
Arnold, S. M. R., Mahajan, P., Datta, D., Bunner, I., and Zarkadas, K. S. (2020). learn2learn: A library for meta-learning research. In ICML AutoML Workshop.
Bateni, P., Goyal, R., Masrani, V., Wood, F., and Sigal, L. (2020). Improved few-shot visual classification. In CVPR.
Deleu, T., Würfl, T., Samiei, M., Cohen, J. P., and Bengio, Y. (2019). Torchmeta: A meta-learning library for PyTorch. arXiv:1909.06576.
Fisher, R. A. (1936). The use of multiple measurements in taxonomic problems. Annals of Eugenics, 7(2):179–188.
Gao, P., Geng, S., Zhang, R., Ma, T., Fang, R., Zhang, Y., Li, H., and Qiao, Y. (2024). Clip-adapter: Better vision-language models with feature adapters. IJCV, 132(2):581–595.
Helber, P., Bischke, B., Dengel, A., and Borth, D. (2019). Eurosat: A novel dataset and deep learning benchmark for land use and land cover classification. IEEE J. Sel. Top. Appl. Earth Obs., 12(7):2217–2226.
Hilliard, N., Phillips, L., Howland, S., Yankov, A., Corley, C. D., and Hodas, N. O. (2018). Few-shot learning with metric-agnostic conditional embeddings. arXiv:1802.04376.
Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al. (2021). Learning transferable visual models from natural language supervision. In ICML, pages 8748–8763. PMLR.
Ravi, S. and Larochelle, H. (2017). Optimization as a model for few-shot learning. In ICLR.
Snell, J., Swersky, K., and Zemel, R. (2017). Prototypical networks for few-shot learning. In Guyon, I., Luxburg, U. V., Bengio, S., Wallach, H., Fergus, R., Vishwanathan, S., and Garnett, R., editors, NeurIPS, volume 30. Curran Associates, Inc.
Udandarao, V., Gupta, A., and Albanie, S. (2023). SuS-X: Training-Free Name-Only Transfer of Vision-Language Models. In ICCV, pages 2725–2736.
Vinyals, O., Blundell, C., Lillicrap, T., Kavukcuoglu, K., and Wierstra, D. (2016). Matching networks for one shot learning. In NeurIPS, volume 29.
Wah, C., Branson, S., Welinder, P., Perona, P., and Belongie, S. (2011). The caltech-ucsd birds-200-2011 dataset. Technical Report CNS-TR-2011-001, California Institute of Technology.
Wang, F., Xiang, X., Cheng, J., and Yuille, A. L. (2017). Normface: L2 hypersphere embedding for face verification. In ACM Multimedia, MM ’17, pages 1041–1049. ACM.
Wang, Y., Chao, W.-L., Weinberger, K. Q., and Maaten, L. v. d. (2019). SimpleShot: Revisiting nearest-neighbor classification for few-shot learning. arXiv:1911.04623.
Wortsman, M., Ilharco, G., Kim, J. W., Li, M., Kornblith, S., Roelofs, R., Gontijo-Lopes, R., Hajishirzi, H., Farhadi, A., Namkoong, H., and Schmidt, L. (2022). Robust fine-tuning of zero-shot models. arXiv:2109.01903.
Zhang, L., Yang, X., Cheng, X., Cheng, W., and Lin, Y. (2025). Few-Shot Image Classification Algorithm Based on Global–Local Feature Fusion. AI, 6(10):265.
Zhang, R., Hu, X., Li, B., Huang, S., Deng, H., Qiao, Y., Gao, P., and Li, H. (2023). Prompt, generate, then cache: Cascade of foundation models makes strong few-shot learners. In CVPR, pages 15211–15222.
Zhang, R., Zhang, W., Fang, R., Gao, P., Li, K., Dai, J., Qiao, Y., and Li, H. (2022). Tip-Adapter: Training-Free Adaption of CLIP for Few-Shot Classification. In Avidan, S., Brostow, G., Cissé, M., Farinella, G. M., and Hassner, T., editors, ECCV, volume 13695, pages 493–510. Springer Nature Switzerland, Cham. Series Title: Lecture Notes in Computer Science.
Zhou, K., Yang, J., Loy, C. C., and Liu, Z. (2022). Learning to prompt for vision-language models. IJCV, 130(9):2337–2348.
Zhu, B., Niu, Y., Han, Y., Wu, Y., and Zhang, H. (2023a). Prompt-aligned gradient for prompt tuning. In ICCV, pages 15659–15669.
Zhu, H. and Koniusz, P. (2023). Transductive few-shot learning with prototype-based label propagation by iterative graph refinement. In CVPR.
Zhu, X., Zhang, R., He, B., Zhou, A., Wang, D., Zhao, B., and Gao, P. (2023b). Not All Features Matter: Enhancing Few-shot CLIP with Adaptive Prior Refinement. In ICCV, pages 2605–2615.
Arnold, S. M. R., Mahajan, P., Datta, D., Bunner, I., and Zarkadas, K. S. (2020). learn2learn: A library for meta-learning research. In ICML AutoML Workshop.
Bateni, P., Goyal, R., Masrani, V., Wood, F., and Sigal, L. (2020). Improved few-shot visual classification. In CVPR.
Deleu, T., Würfl, T., Samiei, M., Cohen, J. P., and Bengio, Y. (2019). Torchmeta: A meta-learning library for PyTorch. arXiv:1909.06576.
Fisher, R. A. (1936). The use of multiple measurements in taxonomic problems. Annals of Eugenics, 7(2):179–188.
Gao, P., Geng, S., Zhang, R., Ma, T., Fang, R., Zhang, Y., Li, H., and Qiao, Y. (2024). Clip-adapter: Better vision-language models with feature adapters. IJCV, 132(2):581–595.
Helber, P., Bischke, B., Dengel, A., and Borth, D. (2019). Eurosat: A novel dataset and deep learning benchmark for land use and land cover classification. IEEE J. Sel. Top. Appl. Earth Obs., 12(7):2217–2226.
Hilliard, N., Phillips, L., Howland, S., Yankov, A., Corley, C. D., and Hodas, N. O. (2018). Few-shot learning with metric-agnostic conditional embeddings. arXiv:1802.04376.
Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al. (2021). Learning transferable visual models from natural language supervision. In ICML, pages 8748–8763. PMLR.
Ravi, S. and Larochelle, H. (2017). Optimization as a model for few-shot learning. In ICLR.
Snell, J., Swersky, K., and Zemel, R. (2017). Prototypical networks for few-shot learning. In Guyon, I., Luxburg, U. V., Bengio, S., Wallach, H., Fergus, R., Vishwanathan, S., and Garnett, R., editors, NeurIPS, volume 30. Curran Associates, Inc.
Udandarao, V., Gupta, A., and Albanie, S. (2023). SuS-X: Training-Free Name-Only Transfer of Vision-Language Models. In ICCV, pages 2725–2736.
Vinyals, O., Blundell, C., Lillicrap, T., Kavukcuoglu, K., and Wierstra, D. (2016). Matching networks for one shot learning. In NeurIPS, volume 29.
Wah, C., Branson, S., Welinder, P., Perona, P., and Belongie, S. (2011). The caltech-ucsd birds-200-2011 dataset. Technical Report CNS-TR-2011-001, California Institute of Technology.
Wang, F., Xiang, X., Cheng, J., and Yuille, A. L. (2017). Normface: L2 hypersphere embedding for face verification. In ACM Multimedia, MM ’17, pages 1041–1049. ACM.
Wang, Y., Chao, W.-L., Weinberger, K. Q., and Maaten, L. v. d. (2019). SimpleShot: Revisiting nearest-neighbor classification for few-shot learning. arXiv:1911.04623.
Wortsman, M., Ilharco, G., Kim, J. W., Li, M., Kornblith, S., Roelofs, R., Gontijo-Lopes, R., Hajishirzi, H., Farhadi, A., Namkoong, H., and Schmidt, L. (2022). Robust fine-tuning of zero-shot models. arXiv:2109.01903.
Zhang, L., Yang, X., Cheng, X., Cheng, W., and Lin, Y. (2025). Few-Shot Image Classification Algorithm Based on Global–Local Feature Fusion. AI, 6(10):265.
Zhang, R., Hu, X., Li, B., Huang, S., Deng, H., Qiao, Y., Gao, P., and Li, H. (2023). Prompt, generate, then cache: Cascade of foundation models makes strong few-shot learners. In CVPR, pages 15211–15222.
Zhang, R., Zhang, W., Fang, R., Gao, P., Li, K., Dai, J., Qiao, Y., and Li, H. (2022). Tip-Adapter: Training-Free Adaption of CLIP for Few-Shot Classification. In Avidan, S., Brostow, G., Cissé, M., Farinella, G. M., and Hassner, T., editors, ECCV, volume 13695, pages 493–510. Springer Nature Switzerland, Cham. Series Title: Lecture Notes in Computer Science.
Zhou, K., Yang, J., Loy, C. C., and Liu, Z. (2022). Learning to prompt for vision-language models. IJCV, 130(9):2337–2348.
Zhu, B., Niu, Y., Han, Y., Wu, Y., and Zhang, H. (2023a). Prompt-aligned gradient for prompt tuning. In ICCV, pages 15659–15669.
Zhu, H. and Koniusz, P. (2023). Transductive few-shot learning with prototype-based label propagation by iterative graph refinement. In CVPR.
Zhu, X., Zhang, R., He, B., Zhou, A., Wang, D., Zhao, B., and Gao, P. (2023b). Not All Features Matter: Enhancing Few-shot CLIP with Adaptive Prior Refinement. In ICCV, pages 2605–2615.
Publicado
19/10/2026
Como Citar
RESMER, João Pedro Perez; KAMASSURY, Jorge Kysnney Santos; GEHLEN, Alexsandro.
Prototypical Few-Shot Classification with Vision-Language Models: A Taxonomy and Systematic Study. In: ENCONTRO NACIONAL DE INTELIGÊNCIA ARTIFICIAL E COMPUTACIONAL (ENIAC), 23. , 2026, Cuiabá/MT.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2026
.
p. 3487-3498.
ISSN 2763-9061.
DOI: https://doi.org/10.5753/eniac.2026.31445.
