Um Estudo Comparativo de Estratégias de Seleção de Exemplos para In-Context Learning aplicado à Classificação Automática de Texto com Grandes Modelos de Linguagem
Resumo
A Classificação Automática de Texto (CAT) com Grandes Modelos de Linguagem (LLMs) pode ser feita via zero-shot (baixo custo, menor efetividade) ou fine-tuning (alto custo, maior efetividade). Este estudo investiga o in-context learning, abordagem intermediária que insere poucos exemplos no prompt, avaliando estratégias de seleção de exemplos. Comparamos seleção aleatória com métodos baseados em representações vetoriais (TF-IDF, RoBERTa, SBERT e LLM2Vec). Os resultados reforçam que o fine-tuning de Small Language Models (SLMs), como RoBERTa, oferece melhor custo-benefício. Ainda assim, o incontext learning mostra-se promissor, superando o zero-shot em efetividade sem exigir os altos custos do fine-tuning, sobretudo com boas estratégias de seleção.
Palavras-chave:
Classificação Automática de Texto, LLMs, In-context Learning, Zero-shot, Fine-tuning, Seleção de exemplos, Representações vetoriais, SLMs
Referências
BehnamGhader, P., Adlakha, V., Mosbach, M., Bahdanau, D., Chapados, N., and Reddy, S. (2024). Llm2vec: Large language models are secretly powerful text encoders.
Chandra, M., Ganguly, D., and Ounis, I. (2025). One size doesn’t fit all: Predicting the number of examples for in-context learning. In Advances in Information Retrieval, pages 67–84, Cham.
Cunha, W., França, C., Fonseca, G., Rocha, L., and Gonçalves, M. A. (2023). An effective, efficient, and scalable confidence-based instance selection framework for transformer-based text classification. In Proceedings of the 46th ACM SIGIR, SIGIR ’23, page 665–674.
Cunha, W., Rocha, L., and Gonçalves, M. A. (2025). A thorough benchmark of automatic text classification: From traditional approaches to large language models.
Devlin, J., Chang, M.-W., Lee, K., and Toutanova, K. (2019). Bert: Pre-training of deep bidirectional transformers for language understanding.
Edwards, A. and Camacho-Collados, J. (2024). Language models for text classification: Is in-context learning enough? In Proceedings of the 2024 LREC-COLING 2024, pages 10058–10072, Torino, Italia.
Fonseca, G., Prenassi, G., Cunha, W., Gonçalves, M., and Rocha, L. (2024). Estratégias de undersampling para redução de viés em classificação de texto baseada em transformers. In Proceedings of the 30th Brazilian Symposium on Multimedia and the Web, pages 144–152, Porto Alegre, RS, Brasil. SBC.
Grattafiori, A. et al. (2024). The llama 3 herd of models.
Hochberg, Y. (1988). A sharper bonferroni procedure for multiple tests of significance. Biometrika, 75(4):800–802.
Hu, E. J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., and Chen, W. (2021). Lora: Low-rank adaptation of large language models.
Kumar, S. and Talukdar, P. (2021). Reordering examples helps during priming-based few-shot learning. In Findings of ACL-IJCNLP 2021, pages 4507–4518, Online. Association for Computational Linguistics.
Liu, J., Shen, D., Zhang, Y., Dolan, B., Carin, L., and Chen, W. (2022). What makes good in-context examples for GPT-3? In Proceedings of Deep Learning Inside Out (DeeLIO 2022): The 3rd Workshop on Knowledge Extraction and Integration for Deep Learning Architectures, pages 100–114.
Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., and Stoyanov, V. (2019). Roberta: A robustly optimized bert pretraining approach.
Lu, Z., Tian, J., Wei, W., Qu, X., Cheng, Y., Xie, W., and Chen, D. (2024). Mitigating boundary ambiguity and inherent bias for text classification in the era of large language models. In Findings of ACL 2024, pages 7841–7864.
Luhn, H. P. (1957). A statistical approach to mechanized encoding and searching of literary information. IBM Journal of Research and Development, 1(4):309–317.
OpenAI et al. (2024). Gpt-4 technical report.
Reimers, N. and Gurevych, I. (2019). Sentence-bert: Sentence embeddings using siamese bert-networks. In Proceedings of the 2019 EMNLP.
Xu, C., Xu, Y., Wang, S., Liu, Y., Zhu, C., and McAuley, J. (2024). Small models are valuable plug-ins for large language models. In Findings of ACL 2024, pages 283–294.
Chandra, M., Ganguly, D., and Ounis, I. (2025). One size doesn’t fit all: Predicting the number of examples for in-context learning. In Advances in Information Retrieval, pages 67–84, Cham.
Cunha, W., França, C., Fonseca, G., Rocha, L., and Gonçalves, M. A. (2023). An effective, efficient, and scalable confidence-based instance selection framework for transformer-based text classification. In Proceedings of the 46th ACM SIGIR, SIGIR ’23, page 665–674.
Cunha, W., Rocha, L., and Gonçalves, M. A. (2025). A thorough benchmark of automatic text classification: From traditional approaches to large language models.
Devlin, J., Chang, M.-W., Lee, K., and Toutanova, K. (2019). Bert: Pre-training of deep bidirectional transformers for language understanding.
Edwards, A. and Camacho-Collados, J. (2024). Language models for text classification: Is in-context learning enough? In Proceedings of the 2024 LREC-COLING 2024, pages 10058–10072, Torino, Italia.
Fonseca, G., Prenassi, G., Cunha, W., Gonçalves, M., and Rocha, L. (2024). Estratégias de undersampling para redução de viés em classificação de texto baseada em transformers. In Proceedings of the 30th Brazilian Symposium on Multimedia and the Web, pages 144–152, Porto Alegre, RS, Brasil. SBC.
Grattafiori, A. et al. (2024). The llama 3 herd of models.
Hochberg, Y. (1988). A sharper bonferroni procedure for multiple tests of significance. Biometrika, 75(4):800–802.
Hu, E. J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., and Chen, W. (2021). Lora: Low-rank adaptation of large language models.
Kumar, S. and Talukdar, P. (2021). Reordering examples helps during priming-based few-shot learning. In Findings of ACL-IJCNLP 2021, pages 4507–4518, Online. Association for Computational Linguistics.
Liu, J., Shen, D., Zhang, Y., Dolan, B., Carin, L., and Chen, W. (2022). What makes good in-context examples for GPT-3? In Proceedings of Deep Learning Inside Out (DeeLIO 2022): The 3rd Workshop on Knowledge Extraction and Integration for Deep Learning Architectures, pages 100–114.
Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., and Stoyanov, V. (2019). Roberta: A robustly optimized bert pretraining approach.
Lu, Z., Tian, J., Wei, W., Qu, X., Cheng, Y., Xie, W., and Chen, D. (2024). Mitigating boundary ambiguity and inherent bias for text classification in the era of large language models. In Findings of ACL 2024, pages 7841–7864.
Luhn, H. P. (1957). A statistical approach to mechanized encoding and searching of literary information. IBM Journal of Research and Development, 1(4):309–317.
OpenAI et al. (2024). Gpt-4 technical report.
Reimers, N. and Gurevych, I. (2019). Sentence-bert: Sentence embeddings using siamese bert-networks. In Proceedings of the 2019 EMNLP.
Xu, C., Xu, Y., Wang, S., Liu, Y., Zhu, C., and McAuley, J. (2024). Small models are valuable plug-ins for large language models. In Findings of ACL 2024, pages 283–294.
Publicado
29/09/2025
Como Citar
PRENASSI, Gabriel Silva; FONSECA, Guilherme; REIS, Davi; CUNHA, Washington; GONÇALVES, Marcos André; ROCHA, Leonardo.
Um Estudo Comparativo de Estratégias de Seleção de Exemplos para In-Context Learning aplicado à Classificação Automática de Texto com Grandes Modelos de Linguagem. In: SIMPÓSIO BRASILEIRO DE BANCO DE DADOS (SBBD), 40. , 2025, Fortaleza/CE.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2025
.
p. 921-927.
ISSN 2763-8979.
DOI: https://doi.org/10.5753/sbbd.2025.247808.
