Comparando Modelos Compactos e Especializados para Análise de Sentimento em Tweets
Resumo
A análise de sentimentos de alta performance é hoje dominada por grandes modelos de linguagem, cujo alto custo computacional impõe um trade-off crítico entre desempenho e eficiência. Este trabalho investiga essa relação ao comparar empiricamente modelos representativos de duas estratégias de otimização — compressão (DistilBERT, ALBERT) e especialização de domínio (BERTweet) — na tarefa de classificação de sentimentos em tweets. Os resultados demonstram que, embora a especialização de domínio alcance o maior desempenho preditivo, os modelos compactos, especialmente o DistilBERT, apresentam um balanço custo-benefício altamente competitivo. Este estudo, portanto, oferece um guia quantitativo para a seleção de arquiteturas em cenários práticos, elucidando as vantagens relativas de cada abordagem.
Palavras-chave:
Inteligência Artificial, Análise de Sentimentos, Processamento da Linguagem Natural (PLN)
Referências
Adu-Acheampong, F., Okyere-Dankwa, B., and Yeboah-Akowuah, B. (2021). Transformer models for text-based sentiment analysis: a review of bert-based approaches. Journal of Smart Systems and Technologies, 1(1):1–16.
Bender, E. M., Gebru, T., McMillan-Major, A., and Shmitchell, S. (2021). On the dangers of stochastic parrots: Can language models be too big? In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency (FAccT), pages 610–623.
Bhatt, N., Patel, K., Chotai, Y., and Shah, V. (2020). A comparative study of pre-trained language models for task-oriented dialogue system. In Proceedings of the 1st Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 10th International Joint Conference on Natural Language Processing, pages 487–492, Suzhou, China. Association for Computational Linguistics.
Blalock, D., Gonzalez Ortiz, J. J., Frankle, J., and Guttag, J. (2020). What is the state of neural network pruning? Proceedings of Machine Learning and Systems, 2:129–146.
Cai, H., Gan, C., Wang, T., Zhang, Z., and Han, S. (2020). Once-for-all: Train one network and specialize it for efficient deployment. In Proceedings of the International Conference on Learning Representations (ICLR). arXiv:1908.09791.
Cambria, E., Schuller, B., Xia, Y., and Havasi, C. (2013). New avenues in opinion mining and sentiment analysis. IEEE Intelligent Systems, 28(2):15–21.
Devlin, J., Chang, M.-W., Lee, K., and Toutanova, K. (2019a). Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of NAACL-HLT, pages 4171–4186.
Devlin, J., Chang, M.-W., Lee, K., and Toutanova, K. (2019b). BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4171–4186, Minneapolis, Minnesota. Association for Computational Linguistics.
Frankle, J. and Carbin, M. (2019). The lottery ticket hypothesis: Finding sparse, trainable neural networks. In Proceedings of the International Conference on Learning Representations (ICLR).
Ganesh, P., Chen, Y., Le, D., Suda, N., Su, H., and M., R. (2021). Compressing BERT: A case study in Deep Model Compression. In 2021 IEEE High Performance Extreme Computing Conference (HPEC), pages 1–7.
Go, A., Bhayani, R., and Huang, L. (2009a). Twitter sentiment analysis. Technical report, Stanford University. CS224N Project Report.
Go, A., Bhayani, R., and Huang, L. (2009b). Twitter sentiment classification using distant supervision. Technical Report CS224N Project Report, Stanford University.
Gururangan, S., Marasović, A., Swayamdipta, S., Lo, K., Beltagy, I., Downey, D., and Smith, N. A. (2020). Don’t stop pretraining: Adapt language models to domains and tasks. In Proceedings of ACL, pages 8342–8360.
Hinton, G., Vinyals, O., and Dean, J. (2015). Distilling the knowledge in a neural network. arXiv preprint, arXiv:1503.02531.
Kavitha, P., Janaki, D., Ayathri, M., Kirubatini, G., and Kaveri (2025). Applications, challenges, and emerging trends of twitter sentiment analysis. Zenodo.
Lan, Z., Chen, M., Goodman, S., Gimpel, K., Sharma, P., and Soricut, R. (2020). Albert: A lite bert for self-supervised learning of language representations. In International Conference on Learning Representations.
Liu, B. (2015). Sentiment Analysis: Mining Opinions, Sentiments, and Emotions. Cambridge University Press.
Nguyen, D. Q., Vu, T., and Nguyen, A. (2020). BERTweet: A pre-trained language model for English tweets. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 9–14, Online. Association for Computational Linguistics.
Pak, A. and Paroubek, P. (2010). Twitter as a corpus for sentiment analysis and opinion mining. In Proceedings of the Seventh International Conference on Language Resources and Evaluation (LREC’10), pages 1320–1326.
Pang, B. and Lee, L. (2008). Opinion mining and sentiment analysis. Foundations and Trends in Information Retrieval, 2(1-2):1–135.
Patterson, D. A., Gonzalez, J., Le, Q. V., Liang, C., Munguia, L.-M., Rothchild, D., So, D. R., Texier, M., and Dean, J. (2021). Carbon emissions and large neural network training. arXiv preprint, arXiv:2104.10350.
Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V., Vanderplas, J., Passos, A., Cournapeau, D., Brucher, M., Perrot, M., and Duchesnay, E. (2011). Scikit-learn: Machine learning in python. Journal of Machine Learning Research, 12:2825–2830.
Ramesh, K., Chavan, A., Pandit, S., and Sitaram, S. (2023). A comparative study on the impact of model compression techniques on fairness in language models. In Rogers, A., Boyd-Graber, J., and Okazaki, N., editors, Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 15762–15782, Toronto, Canada. Association for Computational Linguistics.
Rosenthal, S., Farra, N., and Nakov, P. (2017). Semeval-2017 task 4: Sentiment analysis in twitter. In Proceedings of the 11th International Workshop on Semantic Evaluation (SemEval-2017), pages 502–518.
Sanh, V., Debut, L., Chaumond, J., and Wolf, T. (2019). Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter. ArXiv, abs/1910.01108.
Strubell, E., Ganesh, A., and McCallum, A. (2019). Energy and policy considerations for deep learning in NLP. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 3645–3650.
Sun, C., Qiu, X., Xu, Y., and Huang, X. (2019). How to fine-tune BERT for text classification? In Chinese Computational Linguistics, pages 194–206. Springer International Publishing.
Tay, Y., Dehghani, M., Bahri, D., and Metzler, D. (2022a). Efficient transformers: A survey. ACM Computing Surveys, 55(6):1–28.
Tay, Y., Dehghani, M., Bahri, D., and Metzler, D. (2022b). Efficient transformers: A survey. ACM Computing Surveys, 55(6):1–28.
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, Ł., and Polosukhin, I. (2017). Attention is all you need. In Advances in Neural Information Processing Systems 30 (NIPS 2017), pages 5998–6008. Curran Associates, Inc.
Wolf, T., Debut, L., Sanh, V., Chaumond, J., Delangue, C., Moi, A., Cistac, P., Rault, T., Louf, R., Funtowicz, M., Davison, J., Shleifer, S., von Platen, P., Ma, C., Jernite, Y., Plu, J., Xu, C., Scao, T. L., Gugger, S., Drame, M., Lhoest, Q., and Rush, A. M. (2020). Transformers: State-of-the-art natural language processing. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 38–45, Online. Association for Computational Linguistics.
Xie, Y., Aggarwal, K., and Ahmad, A. (2024). Efficient continual pre-training for building domain specific large language models. In Ku, L.-W., Martins, A., and Srikumar, V., editors, Findings of the Association for Computational Linguistics: ACL 2024, pages 10184–10201, Bangkok, Thailand. Association for Computational Linguistics.
Zhou, H., Lan, J., Liu, R., and Yosinski, J. (2019). Deconstructing lottery tickets: Zeros, signs, and the supermask. In Advances in Neural Information Processing Systems (NeurIPS), volume 32.
Bender, E. M., Gebru, T., McMillan-Major, A., and Shmitchell, S. (2021). On the dangers of stochastic parrots: Can language models be too big? In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency (FAccT), pages 610–623.
Bhatt, N., Patel, K., Chotai, Y., and Shah, V. (2020). A comparative study of pre-trained language models for task-oriented dialogue system. In Proceedings of the 1st Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 10th International Joint Conference on Natural Language Processing, pages 487–492, Suzhou, China. Association for Computational Linguistics.
Blalock, D., Gonzalez Ortiz, J. J., Frankle, J., and Guttag, J. (2020). What is the state of neural network pruning? Proceedings of Machine Learning and Systems, 2:129–146.
Cai, H., Gan, C., Wang, T., Zhang, Z., and Han, S. (2020). Once-for-all: Train one network and specialize it for efficient deployment. In Proceedings of the International Conference on Learning Representations (ICLR). arXiv:1908.09791.
Cambria, E., Schuller, B., Xia, Y., and Havasi, C. (2013). New avenues in opinion mining and sentiment analysis. IEEE Intelligent Systems, 28(2):15–21.
Devlin, J., Chang, M.-W., Lee, K., and Toutanova, K. (2019a). Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of NAACL-HLT, pages 4171–4186.
Devlin, J., Chang, M.-W., Lee, K., and Toutanova, K. (2019b). BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4171–4186, Minneapolis, Minnesota. Association for Computational Linguistics.
Frankle, J. and Carbin, M. (2019). The lottery ticket hypothesis: Finding sparse, trainable neural networks. In Proceedings of the International Conference on Learning Representations (ICLR).
Ganesh, P., Chen, Y., Le, D., Suda, N., Su, H., and M., R. (2021). Compressing BERT: A case study in Deep Model Compression. In 2021 IEEE High Performance Extreme Computing Conference (HPEC), pages 1–7.
Go, A., Bhayani, R., and Huang, L. (2009a). Twitter sentiment analysis. Technical report, Stanford University. CS224N Project Report.
Go, A., Bhayani, R., and Huang, L. (2009b). Twitter sentiment classification using distant supervision. Technical Report CS224N Project Report, Stanford University.
Gururangan, S., Marasović, A., Swayamdipta, S., Lo, K., Beltagy, I., Downey, D., and Smith, N. A. (2020). Don’t stop pretraining: Adapt language models to domains and tasks. In Proceedings of ACL, pages 8342–8360.
Hinton, G., Vinyals, O., and Dean, J. (2015). Distilling the knowledge in a neural network. arXiv preprint, arXiv:1503.02531.
Kavitha, P., Janaki, D., Ayathri, M., Kirubatini, G., and Kaveri (2025). Applications, challenges, and emerging trends of twitter sentiment analysis. Zenodo.
Lan, Z., Chen, M., Goodman, S., Gimpel, K., Sharma, P., and Soricut, R. (2020). Albert: A lite bert for self-supervised learning of language representations. In International Conference on Learning Representations.
Liu, B. (2015). Sentiment Analysis: Mining Opinions, Sentiments, and Emotions. Cambridge University Press.
Nguyen, D. Q., Vu, T., and Nguyen, A. (2020). BERTweet: A pre-trained language model for English tweets. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 9–14, Online. Association for Computational Linguistics.
Pak, A. and Paroubek, P. (2010). Twitter as a corpus for sentiment analysis and opinion mining. In Proceedings of the Seventh International Conference on Language Resources and Evaluation (LREC’10), pages 1320–1326.
Pang, B. and Lee, L. (2008). Opinion mining and sentiment analysis. Foundations and Trends in Information Retrieval, 2(1-2):1–135.
Patterson, D. A., Gonzalez, J., Le, Q. V., Liang, C., Munguia, L.-M., Rothchild, D., So, D. R., Texier, M., and Dean, J. (2021). Carbon emissions and large neural network training. arXiv preprint, arXiv:2104.10350.
Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V., Vanderplas, J., Passos, A., Cournapeau, D., Brucher, M., Perrot, M., and Duchesnay, E. (2011). Scikit-learn: Machine learning in python. Journal of Machine Learning Research, 12:2825–2830.
Ramesh, K., Chavan, A., Pandit, S., and Sitaram, S. (2023). A comparative study on the impact of model compression techniques on fairness in language models. In Rogers, A., Boyd-Graber, J., and Okazaki, N., editors, Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 15762–15782, Toronto, Canada. Association for Computational Linguistics.
Rosenthal, S., Farra, N., and Nakov, P. (2017). Semeval-2017 task 4: Sentiment analysis in twitter. In Proceedings of the 11th International Workshop on Semantic Evaluation (SemEval-2017), pages 502–518.
Sanh, V., Debut, L., Chaumond, J., and Wolf, T. (2019). Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter. ArXiv, abs/1910.01108.
Strubell, E., Ganesh, A., and McCallum, A. (2019). Energy and policy considerations for deep learning in NLP. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 3645–3650.
Sun, C., Qiu, X., Xu, Y., and Huang, X. (2019). How to fine-tune BERT for text classification? In Chinese Computational Linguistics, pages 194–206. Springer International Publishing.
Tay, Y., Dehghani, M., Bahri, D., and Metzler, D. (2022a). Efficient transformers: A survey. ACM Computing Surveys, 55(6):1–28.
Tay, Y., Dehghani, M., Bahri, D., and Metzler, D. (2022b). Efficient transformers: A survey. ACM Computing Surveys, 55(6):1–28.
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, Ł., and Polosukhin, I. (2017). Attention is all you need. In Advances in Neural Information Processing Systems 30 (NIPS 2017), pages 5998–6008. Curran Associates, Inc.
Wolf, T., Debut, L., Sanh, V., Chaumond, J., Delangue, C., Moi, A., Cistac, P., Rault, T., Louf, R., Funtowicz, M., Davison, J., Shleifer, S., von Platen, P., Ma, C., Jernite, Y., Plu, J., Xu, C., Scao, T. L., Gugger, S., Drame, M., Lhoest, Q., and Rush, A. M. (2020). Transformers: State-of-the-art natural language processing. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 38–45, Online. Association for Computational Linguistics.
Xie, Y., Aggarwal, K., and Ahmad, A. (2024). Efficient continual pre-training for building domain specific large language models. In Ku, L.-W., Martins, A., and Srikumar, V., editors, Findings of the Association for Computational Linguistics: ACL 2024, pages 10184–10201, Bangkok, Thailand. Association for Computational Linguistics.
Zhou, H., Lan, J., Liu, R., and Yosinski, J. (2019). Deconstructing lottery tickets: Zeros, signs, and the supermask. In Advances in Neural Information Processing Systems (NeurIPS), volume 32.
Publicado
04/12/2025
Como Citar
FREITAS, Leandro I.; PAIVA, Anselmo C.; DE PAIVA, Fael Faray; PAIVA NETO, Arnaud Guedes de.
Comparando Modelos Compactos e Especializados para Análise de Sentimento em Tweets. In: ESCOLA REGIONAL DE COMPUTAÇÃO DO CEARÁ, MARANHÃO E PIAUÍ (ERCEMAPI), 13. , 2025, São Luís/MA.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2025
.
p. 121-132.
DOI: https://doi.org/10.5753/ercemapi.2025.17615.
