Benchmarking Text Classifiers for Economic News in Brazilian Portuguese: In-Domain Quality and Out-of-Distribution Generalization

• Diego de Almeida Miranda UNIFESP
• João Assaoka UNIFESP
• Lilian Berton UNIFESP

DOI: https://doi.org/10.5753/eniac.2026.30771

Resumo

Filtering economic news is an upstream triage step for finance-oriented NLP, yet for Brazilian Portuguese there is no standardised comparison of the available model families. We benchmark TF–IDF linear models, Portuguese BERT encoders (BERTimbau, FinBERT-PT-BR, NorBERTo, DeB3RTa), and instruction-tuned LLMs on a corpus of Folha de São Paulo articles, under two shared-split formulations: a binary mercado-vs-rest task and a 7+other multiclass task over editorial sections. Every system is evaluated on identical 5-fold splits with a calibrationand significance-aware protocol (Brier, ECE, Bonferroni-corrected McNemar) and a cross-validation-only selection funnel that never consults the test set. A stacking ensemble is the strongest in-domain filter (multiclass macro-F1 0.94) and BERT leads among individual models, but a TF–IDF model recovers most of that performance at one-to-two orders of magnitude lower cost and stays competitive on a zero-shot out-of-domain probe over four external corpora — the trade-off that matters for an upstream triage stage.

Referências

Araci, D. (2019). FinBERT: Financial sentiment analysis with pre-trained language models. arXiv preprint.

Corrêa, N. K., Sen, A., Falk, S., and Fatimah, S. (2025). Tucano: Advancing neural text generation for Portuguese. Patterns, 6(6):101325.

Garcia, G. L., Afonso, L. C. S., and Papa, J. P. (2022). FakeRecogna: A new Brazilian corpus for fake news detection. In Computational Processing of the Portuguese Language (PROPOR 2022), volume 13208 of Lecture Notes in Computer Science, pages 57–67. Springer.

Garcia, G. L., Paiola, P. H., Morelli, L. H., Candido, G., Candido Júnior, A., Jodas, D. S., Afonso, L. C. S., Guilherme, I. R., Penteado, B. E., and Papa, J. a. P. (2024a). Introducing bode: A fine-tuned large language model for Portuguese prompt-based task. arXiv preprint.

Garcia, K., Shiguihara, P., and Berton, L. (2024b). Breaking news: Unveiling a new dataset for Portuguese news classification and comparative analysis of approaches. PLOS ONE, 19(1):e0296929.

Gretton, A., Borgwardt, K. M., Rasch, M. J., Schölkopf, B., and Smola, A. (2012). A kernel two-sample test. Journal of Machine Learning Research, 13(25):723–773.

Jiang, A. Q., Sablayrolles, A., Mensch, A., Bamford, C., Chaplot, D. S., de las Casas, D., Bressand, F., Lengyel, G., Lample, G., Saulnier, L., et al. (2023). Mistral 7B. arXiv preprint.

Monteiro, R. A., Santos, R. L. S., Pardo, T. A. S., de Almeida, T. A., Ruiz, E. E. S., and Vale, O. A. (2018). Contributions to the study of fake news in Portuguese: New corpus and automatic detection results. In Computational Processing of the Portuguese Language (PROPOR), pages 324–334. Springer.

Paiola, P. H., Garcia, G. L., Jodas, D. S., Correia, J. V. M., Sugi, L. A., and Papa, J. P. (2024). RecognaSumm: A novel Brazilian summarization dataset. In Proceedings of the 16th International Conference on Computational Processing of Portuguese (PROPOR 2024), pages 575–579, Santiago de Compostela, Spain. Association for Computational Linguistics.

Pires, H., Paucar, L., and Carvalho, J. a. P. (2025). DeB3RTa: A transformer-based model for the Portuguese financial domain. Big Data and Cognitive Computing, 9(3):51.

Pires, R., Abonizio, H., Almeida, T. S., and Nogueira, R. (2023). Sabiá: Portuguese large language models. In Intelligent Systems — 12th Brazilian Conference (BRACIS), volume 14197 of Lecture Notes in Computer Science. Springer.

Santana, M. (2018). News of the site Folha de São Paulo (Brazilian Newspaper). [link]. Accessed 2026-05-21.

Santos, L. L., Bianchi, R. A. C., and Costa, A. H. R. (2023). FinBERT-PT-BR: Análise de sentimentos de textos em português do mercado financeiro. In Anais do II Brazilian Workshop on Artificial Intelligence in Finance (BWAIF), pages 144–155. SBC.

Silva, E. S. N., Costa, P. B., Vlasman, R. C., Costa, R. P., Silva, H. L. P., Pellicer, L. F. A. O., Rinaldo, G., Almeida, R. A., Rabbani, D. S. R., Oestreich, C. O., and Caridá, V. F. (2026). NorBERTo: A ModernBERT model trained for Portuguese with 331 billion tokens corpus. In Souza, M., de Dios-Flores, I., Santos, D., Freitas, L., Souza, J. W. d. C., and Ribeiro, E., editors, Proceedings of the 17th International Conference on Computational Processing of Portuguese (PROPOR 2026) - Vol. 1, pages 183–193, Salvador, Brazil. Association for Computational Linguistics.

Souza, F., Nogueira, R., and Lotufo, R. (2020). BERTimbau: Pretrained BERT models for Brazilian Portuguese. In Intelligent Systems — 9th Brazilian Conference (BRACIS), volume 12320 of Lecture Notes in Computer Science. Springer.

Yang, A., Yang, B., Zhang, B., Hui, B., Zheng, B., Yu, B., Li, C., Liu, D., Huang, F., Wei, H., et al. (2024). Qwen2.5 Technical Report. arXiv preprint.

Yang, Y., Uy, M. C. S., and Huang, A. (2020). FinBERT: A pretrained language model for financial communications. arXiv preprint.