Desempenho alto, generalização frágil: lacunas de generalização e aprendizado por atalho em detecção de fake news em portugues brasileiro
Resumo
Avalia-se a capacidade de generalização de modelos de classificação textual na detecção de fake news em português brasileiro, com dezessete configurações, de baselines clássicos a Transformers, avaliadas no FakeRecogna 2.0. Dentro da distribuição, os melhores modelos atingem F1-macro próximo de 0,96 e são indistinguíveis entre si; na avaliação cross-dataset (base Fake.br), o F1-macro cai 27–28 pontos, ficando entre 0,68 e 0,70 nas duas direções de transferência. Análises de explicabilidade e de estresse apontam regularidades do corpus compatíveis com aprendizado por atalho, tratado como hipótese; um classificador trivial que usa apenas metadados (categoria e ano) iguala o ensemble neural dentro da distribuição.Referências
Abonizio, H. Q., de Morais, J. I., Tavares, G. M., and Barbon Junior, S. (2020). Language-independent fake news detection: English, Portuguese, and Spanish mutual features. Future Internet, 12(5):87.
Conneau, A., Khandelwal, K., Goyal, N., Chaudhary, V., Wenzek, G., Guzmán, F., Grave, E., Ott, M., Zettlemoyer, L., and Stoyanov, V. (2020). Unsupervised cross-lingual representation learning at scale. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (ACL), pages 8440–8451. Association for Computational Linguistics.
Fischer, M., Haque, R., Stynes, P., and Pathak, P. (2022). Identifying fake news in Brazilian Portuguese. In Natural Language Processing and Information Systems (NLDB 2022), volume 13286 of Lecture Notes in Computer Science, pages 110–122. Springer.
Garcia, G. L., Paiola, P. H., Jodas, D. S., Sugi, L. A., and Papa, J. P. (2024). Text summarization and temporal learning models applied to Portuguese fake news detection in a novel Brazilian corpus dataset. In Proceedings of the 16th International Conference on Computational Processing of Portuguese (PROPOR 2024), volume 1, pages 86–96. Association for Computational Linguistics.
Geirhos, R., Jacobsen, J.-H., Michaelis, C., Zemel, R., Brendel, W., Bethge, M., and Wichmann, F. A. (2020). Shortcut learning in deep neural networks. Nature Machine Intelligence, 2(11):665–673.
Guedes, R., Barros, B., and do Nascimento, H. (2026). Identificação de notícias falsas em português: um olhar sobre a generalização de modelos. In Proceedings of the 17th International Conference on Computational Processing of Portuguese (PROPOR 2026), volume 1, pages 717–726. Association for Computational Linguistics.
Gururangan, S., Swayamdipta, S., Levy, O., Schwartz, R., Bowman, S. R., and Smith, N. A. (2018). Annotation artifacts in natural language inference data. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT), volume 2, pages 107–112. Association for Computational Linguistics.
He, P., Gao, J., and Chen, W. (2021). DeBERTaV3: Improving DeBERTa using ELECTRA-style pre-training with gradient-disentangled embedding sharing. arXiv:2111.09543.
Kapoor, S. and Narayanan, A. (2023). Leakage and the reproducibility crisis in machine-learning-based science. Patterns, 4(9):100804.
Krishna, S., Han, T., Gu, A., Pombra, J., Jabbari, S., Wu, Z. S., and Lakkaraju, H. (2022). The disagreement problem in explainable machine learning: A practitioner’s perspective. arXiv:2202.01602.
Mihalcea, R. and Tarau, P. (2004). TextRank: Bringing order into texts. In Proceedings of the 2004 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 404–411. Association for Computational Linguistics.
Monroe, B. L., Colaresi, M. P., and Quinn, K. M. (2008). Fightin’ words: Lexical feature selection and evaluation for identifying the content of political conflict. Political Analysis, 16(4):372–403.
Monteiro, R. A., Santos, R. L. S., Pardo, T. A. S., de Almeida, T. A., Ruiz, E. E. S., and Vale, O. A. (2018). Contributions to the study of fake news in Portuguese: New corpus and automatic detection results. In Computational Processing of the Portuguese Language (PROPOR 2018), volume 11122 of Lecture Notes in Computer Science, pages 324–334. Springer.
Okano, E. Y., Liu, Z., Ji, D., and Ruiz, E. E. S. (2020). Fake news detection on Fake.Br using hierarchical attention networks. In Computational Processing of the Portuguese Language (PROPOR 2020), volume 12037 of Lecture Notes in Computer Science, pages 143–152. Springer.
Oshikawa, R., Qian, J., and Wang, W. Y. (2020). A survey on natural language processing for fake news detection. In Proceedings of the 12th Language Resources and Evaluation Conference (LREC), pages 6086–6093. European Language Resources Association.
Ovadia, Y., Fertig, E., Ren, J., Nado, Z., Sculley, D., Nowozin, S., Dillon, J. V., Lakshminarayanan, B., and Snoek, J. (2019). Can you trust your model’s uncertainty? evaluating predictive uncertainty under dataset shift. In Advances in Neural Information Processing Systems (NeurIPS), volume 32.
Quiñonero-Candela, J., Sugiyama, M., Schwaighofer, A., and Lawrence, N. D. (2009). Dataset Shift in Machine Learning. MIT Press.
Ribeiro, M. T., Singh, S., and Guestrin, C. (2016). “why should i trust you?”: Explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD), pages 1135–1144. ACM.
Silva, R. M., Amamou, H., Ferraz, L. B. S., da Silva, F. K. A., and Avila, A. R. (2025). Fake news detection in Portuguese under large language model-generated content. Journal of the Brazilian Computer Society, 31(1):1149–1166.
Silva, R. M., Santos, R. L. S., Almeida, T. A., and Pardo, T. A. S. (2020). Towards automatically filtering fake news in Portuguese. Expert Systems with Applications, 146:113199.
Soares, F. B., Recuero, R., Volcan, T., Fagundes, G., and Sodré, G. (2021). Research note: Bolsonaro’s firehose: How Covid-19 disinformation on WhatsApp was used to fight a government political crisis in Brazil. Harvard Kennedy School Misinformation Review, 2(1):1–17.
Souza, F., Nogueira, R., and Lotufo, R. (2020). BERTimbau: Pretrained BERT models for Brazilian Portuguese. In Brazilian Conference on Intelligent Systems (BRACIS 2020), volume 12319 of Lecture Notes in Computer Science, pages 403–417. Springer.
Tiedemann, J. and Thottingal, S. (2020). OPUS-MT – building open translation services for the world. In Proceedings of the 22nd Annual Conference of the European Association for Machine Translation (EAMT), pages 479–480.
Vicentini, J., Rodrigues, R. B. d. M., Candido Junior, A., and Guilherme, I. R. (2026). Comparing explainable AI techniques in language models: A case study for fake news detection in Portuguese. Journal of the Brazilian Computer Society, 32(1):1–12.
Vosoughi, S., Roy, D., and Aral, S. (2018). The spread of true and false news online. Science, 359(6380):1146–1151.
Wanderley, M. G., Ferraz, L. B. S., de Almeida, T. A., and Silva, R. M. (2025). A moving target: Detecting concept drift in Brazilian Portuguese fake news. In Anais do XVI Simpósio Brasileiro de Tecnologia da Informação e da Linguagem Humana (STIL), pages 490–501. SBC.
Wardle, C. and Derakhshan, H. (2017). Information disorder: Toward an interdisciplinary framework for research and policy making. Technical Report DGI(2017)09, Council of Europe.
Conneau, A., Khandelwal, K., Goyal, N., Chaudhary, V., Wenzek, G., Guzmán, F., Grave, E., Ott, M., Zettlemoyer, L., and Stoyanov, V. (2020). Unsupervised cross-lingual representation learning at scale. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (ACL), pages 8440–8451. Association for Computational Linguistics.
Fischer, M., Haque, R., Stynes, P., and Pathak, P. (2022). Identifying fake news in Brazilian Portuguese. In Natural Language Processing and Information Systems (NLDB 2022), volume 13286 of Lecture Notes in Computer Science, pages 110–122. Springer.
Garcia, G. L., Paiola, P. H., Jodas, D. S., Sugi, L. A., and Papa, J. P. (2024). Text summarization and temporal learning models applied to Portuguese fake news detection in a novel Brazilian corpus dataset. In Proceedings of the 16th International Conference on Computational Processing of Portuguese (PROPOR 2024), volume 1, pages 86–96. Association for Computational Linguistics.
Geirhos, R., Jacobsen, J.-H., Michaelis, C., Zemel, R., Brendel, W., Bethge, M., and Wichmann, F. A. (2020). Shortcut learning in deep neural networks. Nature Machine Intelligence, 2(11):665–673.
Guedes, R., Barros, B., and do Nascimento, H. (2026). Identificação de notícias falsas em português: um olhar sobre a generalização de modelos. In Proceedings of the 17th International Conference on Computational Processing of Portuguese (PROPOR 2026), volume 1, pages 717–726. Association for Computational Linguistics.
Gururangan, S., Swayamdipta, S., Levy, O., Schwartz, R., Bowman, S. R., and Smith, N. A. (2018). Annotation artifacts in natural language inference data. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT), volume 2, pages 107–112. Association for Computational Linguistics.
He, P., Gao, J., and Chen, W. (2021). DeBERTaV3: Improving DeBERTa using ELECTRA-style pre-training with gradient-disentangled embedding sharing. arXiv:2111.09543.
Kapoor, S. and Narayanan, A. (2023). Leakage and the reproducibility crisis in machine-learning-based science. Patterns, 4(9):100804.
Krishna, S., Han, T., Gu, A., Pombra, J., Jabbari, S., Wu, Z. S., and Lakkaraju, H. (2022). The disagreement problem in explainable machine learning: A practitioner’s perspective. arXiv:2202.01602.
Mihalcea, R. and Tarau, P. (2004). TextRank: Bringing order into texts. In Proceedings of the 2004 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 404–411. Association for Computational Linguistics.
Monroe, B. L., Colaresi, M. P., and Quinn, K. M. (2008). Fightin’ words: Lexical feature selection and evaluation for identifying the content of political conflict. Political Analysis, 16(4):372–403.
Monteiro, R. A., Santos, R. L. S., Pardo, T. A. S., de Almeida, T. A., Ruiz, E. E. S., and Vale, O. A. (2018). Contributions to the study of fake news in Portuguese: New corpus and automatic detection results. In Computational Processing of the Portuguese Language (PROPOR 2018), volume 11122 of Lecture Notes in Computer Science, pages 324–334. Springer.
Okano, E. Y., Liu, Z., Ji, D., and Ruiz, E. E. S. (2020). Fake news detection on Fake.Br using hierarchical attention networks. In Computational Processing of the Portuguese Language (PROPOR 2020), volume 12037 of Lecture Notes in Computer Science, pages 143–152. Springer.
Oshikawa, R., Qian, J., and Wang, W. Y. (2020). A survey on natural language processing for fake news detection. In Proceedings of the 12th Language Resources and Evaluation Conference (LREC), pages 6086–6093. European Language Resources Association.
Ovadia, Y., Fertig, E., Ren, J., Nado, Z., Sculley, D., Nowozin, S., Dillon, J. V., Lakshminarayanan, B., and Snoek, J. (2019). Can you trust your model’s uncertainty? evaluating predictive uncertainty under dataset shift. In Advances in Neural Information Processing Systems (NeurIPS), volume 32.
Quiñonero-Candela, J., Sugiyama, M., Schwaighofer, A., and Lawrence, N. D. (2009). Dataset Shift in Machine Learning. MIT Press.
Ribeiro, M. T., Singh, S., and Guestrin, C. (2016). “why should i trust you?”: Explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD), pages 1135–1144. ACM.
Silva, R. M., Amamou, H., Ferraz, L. B. S., da Silva, F. K. A., and Avila, A. R. (2025). Fake news detection in Portuguese under large language model-generated content. Journal of the Brazilian Computer Society, 31(1):1149–1166.
Silva, R. M., Santos, R. L. S., Almeida, T. A., and Pardo, T. A. S. (2020). Towards automatically filtering fake news in Portuguese. Expert Systems with Applications, 146:113199.
Soares, F. B., Recuero, R., Volcan, T., Fagundes, G., and Sodré, G. (2021). Research note: Bolsonaro’s firehose: How Covid-19 disinformation on WhatsApp was used to fight a government political crisis in Brazil. Harvard Kennedy School Misinformation Review, 2(1):1–17.
Souza, F., Nogueira, R., and Lotufo, R. (2020). BERTimbau: Pretrained BERT models for Brazilian Portuguese. In Brazilian Conference on Intelligent Systems (BRACIS 2020), volume 12319 of Lecture Notes in Computer Science, pages 403–417. Springer.
Tiedemann, J. and Thottingal, S. (2020). OPUS-MT – building open translation services for the world. In Proceedings of the 22nd Annual Conference of the European Association for Machine Translation (EAMT), pages 479–480.
Vicentini, J., Rodrigues, R. B. d. M., Candido Junior, A., and Guilherme, I. R. (2026). Comparing explainable AI techniques in language models: A case study for fake news detection in Portuguese. Journal of the Brazilian Computer Society, 32(1):1–12.
Vosoughi, S., Roy, D., and Aral, S. (2018). The spread of true and false news online. Science, 359(6380):1146–1151.
Wanderley, M. G., Ferraz, L. B. S., de Almeida, T. A., and Silva, R. M. (2025). A moving target: Detecting concept drift in Brazilian Portuguese fake news. In Anais do XVI Simpósio Brasileiro de Tecnologia da Informação e da Linguagem Humana (STIL), pages 490–501. SBC.
Wardle, C. and Derakhshan, H. (2017). Information disorder: Toward an interdisciplinary framework for research and policy making. Technical Report DGI(2017)09, Council of Europe.
Publicado
19/10/2026
Como Citar
SILVA, Guilherme O.; BIANCHI, Reinaldo A. C..
Desempenho alto, generalização frágil: lacunas de generalização e aprendizado por atalho em detecção de fake news em portugues brasileiro. In: ENCONTRO NACIONAL DE INTELIGÊNCIA ARTIFICIAL E COMPUTACIONAL (ENIAC), 23. , 2026, Cuiabá/MT.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2026
.
p. 503-514.
ISSN 2763-9061.
DOI: https://doi.org/10.5753/eniac.2026.30035.
