Automated Fact-Checking in Brazilian Portuguese: Resources and Baselines

• Marcelo M. Delucis PUCRS
• Lucas Fraga PUCRS
• Otávio Parraga PUCRS
• Christian Mattjie PUCRS
• Rafaela Ravazio PUCRS
• Rodrigo C. Barros PUCRS
• Lucas S. Kupssinskü PUCRS

DOI: https://doi.org/10.5753/stil.2025.37820

Abstract

The spread of misinformation presents a growing societal challenge, particularly in low-resource languages such as Brazilian Portuguese (PTBR), where the scarcity of high-quality datasets limits automated fact-checking tools. In this work, we introduce translated PTBR versions of two influential English-language fact-checking datasets: LIAR and AVERITEC. These resources support multi-class veracity classification and incorporate evidence-based reasoning. We also establish baseline results for both datasets using a range of model configurations, including zero-shot and few-shot prompting with Gemma 3, and fine-tuning of encoder-based models such as mBERT, BERT-Large, and BERTimbau-Large. Across both datasets, fine-tuned encoder-based models consistently outperformed Gemma 3 in zero-shot and few-shot settings. Our results underscore the importance of task-specific fine-tuning and evidence inclusion for veracity classification in PTBR. All datasets, translation scripts, and evaluation protocols are publicly released to support further research in this area.

References

Achiam, J., Adler, S., Agarwal, S., Ahmad, L., Akkaya, I., Aleman, F. L., Almeida, D., Altenschmidt, J., Altman, S., Anadkat, S., et al. (2023). GPT-4 Technical Report. arXiv preprint arXiv:2303.08774.

Boididou, C., Andreadou, K., Papadopoulos, S., Dang Nguyen, D. T., Boato, G., Riegler, M., Kompatsiaris, Y., et al. (2015). Verifying multimedia use at MediaEval 2015. In MediaEval 2015, volume 1436. CEUR-WS.

Chavarro, J., Carvalho, J., Portela, T., and Silva, J. (2023). FakeTrueBR: Um corpus brasileiro de notícias falsas. In Anais da XVIII Escola Regional de Banco de Dados, pages 108–117, Porto Alegre, RS, Brasil. SBC.

Cohen, J. (1960). A coefficient of agreement for nominal scales. Educational and Psychological Measurement, 20:37 – 46.

Cohen, J. (1968). Weighted Kappa: Nominal scale agreement with provision for scaled disagreement or partial credit. Psychol. Bull., 70(4):213–220.

Del Vicario, M., Bessi, A., Zollo, F., Petroni, F., Scala, A., Caldarelli, G., Stanley, H. E., and Quattrociocchi, W. (2016). The spreading of misinformation online. Proceedings of the national academy of Sciences, 113(3):554–559.

Derczynski, L., Bontcheva, K., Liakata, M., Procter, R., Hoi, G. W. S., and Zubiaga, A. (2017). SemEval-2017 Task 8 RumourEval: Determining rumour veracity and support for rumours. arXiv preprint arXiv:1704.05972.

Devlin, J., Chang, M.-W., Lee, K., and Toutanova, K. (2019). BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies, volume 1 (long and short papers), pages 4171–4186.

Domingues, L. (2021). Infodemia: uma ameaça à saúde pública global durante e após a pandemia de covid-19. Revista Eletrônica de Comunicação, Informação & Inovação em Saúde, 15(1).

Garcia, G. L., Afonso, L. C. S., and Papa, J. P. (2022). FakeRecogna: A New Brazilian Corpus for Fake News Detection. In Computational Processing of the Portuguese Language, pages 57–67. Springer International Publishing.

Guo, Z., Schlichtkrull, M., and Vlachos, A. (2022). A survey on automated fact-checking. Transactions of the Association for Computational Linguistics, 10:178–206.

Jurafsky, D. and Martin, J. (2000). Speech and Language Processing: An Introduction to Natural Language Processing, Computational Linguistics, and Speech Recognition. Practical Resources for the Mental Health Professionals Series. Prentice Hall.

Kamath, A., Ferret, J., Pathak, S., Vieillard, N., Merhej, R., Perrin, S., Matejovicova, T., Ramé, A., Rivière, M., Rouillard, L., et al. (2025). Gemma 3 Technical Report. CoRR.

Landis, J. R. and Koch, G. G. (1977). The measurement of observer agreement for categorical data. Biometrics, 33(1):159.

Li, J. and Chang, X. (2023). Combating misinformation by sharing the truth: a study on the spread of fact-checks on social media. Information systems frontiers, 25(4):1479–1493.

Lundberg, S. M. and Lee, S.-I. (2017). A unified approach to interpreting model predictions. Advances in neural information processing systems, 30.

Moreno, J. a. and Bressan, G. (2019). FACTCK.BR: A new dataset to study fake news. In Proceedings of the 25th Brazillian Symposium on Multimedia and the Web, WebMedia ’19, page 525–527, New York, NY, USA. Association for Computing Machinery.

Pomerleau, D. and Rao, D. (2017). Fake news challenge. Available at: [link].

Ribeiro, M. T., Singh, S., and Guestrin, C. (2016). “Why should I trust you?” Explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, pages 1135–1144.

Schlichtkrull, M., Guo, Z., and Vlachos, A. (2023). AVERITEC: A dataset for real-world claim verification with evidence from the web. Advances in Neural Information Processing Systems, 36:65128–65167.

Silva, R. M., Santos, R. L., Almeida, T. A., and Pardo, T. A. (2020). Towards automatically filtering fake news in portuguese. Expert Systems with Applications, 146:113199.

Southwell, B. G., Niederdeppe, J., Cappella, J. N., Gaysynsky, A., Kelley, D. E., Oh, A., Peterson, E. B., and Chou, W.-Y. S. (2019). Misinformation as a misunderstood challenge to public health. American journal of preventive medicine, 57(2):282–285.

Souza, F., Nogueira, R., and Lotufo, R. (2020). BERTimbau: Pretrained BERT Models for Brazilian Portuguese. In Intelligent Systems, pages 403–417. Springer International Publishing.

Thorne, J. and Vlachos, A. (2018). Automated Fact Checking: Task formulations, methods and future directions. arXiv:1806.07687 [cs].

Thorne, J., Vlachos, A., Christodoulopoulos, C., and Mittal, A. (2018). FEVER: a large-scale dataset for fact extraction and VERification. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pages 809–819. Association for Computational Linguistics.

Vargas, F., Jaidka, K., Pardo, T., and Benevenuto, F. (2023). Predicting sentence-level factuality of news and bias of media outlets. In Proceedings of the 14th International Conference on Recent Advances in Natural Language Processing, pages 1197–1206. INCOMA Ltd., Shoumen, Bulgaria.

Vargas, F., Salles, I., Alves, D., Agrawal, A., Pardo, T. A. S., and Benevenuto, F. (2024). Improving explainable fact-checking via sentence-level factual reasoning. In Proceedings of the Seventh Fact Extraction and VERification Workshop (FEVER), pages 192–204. Association for Computational Linguistics.

Wang, W. Y. (2017). “liar, liar pants on fire”: A new benchmark dataset for fake news detection. arXiv preprint arXiv:1705.00648.