Semantic Indicators in Prompt Engineering: An Explainable Approach to Fake News Detection

• Camilla B. Quincozes UFU
• Diego Molinos UFU
• Rafael D. Araújo UFU
• Silvio E. Quincozes UFU / UNIPAMPA

DOI: https://doi.org/10.5753/sbseg.2025.11513

Abstract

The detection of fake news has benefited from the use of Large Language Models (LLMs). However, the decisions made by these models lack explainability. This work aims to fill this gap by constructing indicators (red flags) from news through prompt engineering, which enable explainable decisionmaking. As a result, a dataset was created with 16 red flags concerning true and false news. Experiments with the Random Forest classifier and the SHAP tool revealed an F1-Score of 95.38% in the explainable detection of fake news.

References

Anjos, L. S. d. (2023). Análise experimental do desempenho de grandes modelos de linguagem na detecção de notícias falsas. In Anais do Trabalho de Conclusão de Curso - Universidade Federal de Uberlândia, pages 1–10. Universidade Federal de Uberlândia.

Aslam, N., Ullah Khan, I., Alotaibi, F. S., Aldaej, L. A., and Aldubaikil, A. K. (2021). Fake detect: A deep learning ensemble model for fake news detection. Complexity.

Baarir, N. F. and Djeffal, A. (2021). Fake news detection using machine learning. 2020 Workshop on Human-Centric Smart Environments for Health and Well-being (IHSH).

BBC News (2024). Pesquisa aponta que 25% das pessoas não sabem identificar fake news. [link]. Acesso em: 14 maio 2025.

Hu, B., Sheng, Q., Cao, J., Shi, Y., Li, Y., Wang, D., and Qi, P. (2024). Bad actor, good advisor: Exploring the role of large language models in fake news detection. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 22105–22113.

KaabOmeir, F., Khademizadeh, S., Seifadini, R., Balani, S. O., and Khazaneha, M. (2024). Overview of Misinformation and Disinformation Research from 1971 to 2022. Journal of Scientometric Research, 13(2):430–447.

Kanashina, O., Huertas García, R., and Jiménez-Zarco, A. I. (2023). Fake news and youngsters’ decision journey: An evaluation of the influence of misinformation on social media. Intangible Capital, 19(4):534–554.

Khanam, Z., Alwasel, B. N., Sirafi, H., and Rashid, M. (2021). Fake news detection using machine learning approaches. In IOP Conference Series: Materials Science and Engineering, volume 1099, page 012040. IOP Publishing.

Khivasara, Y., Khare, Y., and Bhadane, T. (2020). Fake news detection system using web-extension. 2020 IEEE Pune Section International Conference.

Kong, S. H., Tan, L. M., Gan, K. H., and Samsudin, N. H. (2020). Fake news detection using deep learning. In 2020 IEEE 10th symposium on computer applications & industrial electronics (ISCAIE), pages 102–107. IEEE.

Meta & Broadminded (2025). Relatório de percepção sobre fake news no brasil. [link]. Acesso em: 14 maio 2025.

OCDE (2024). The OECD Truth Quest Survey: Methodology and findings. OECD Digital Economy Papers, 369.

Papageorgiou, E., Chronis, C., Varlamis, I., and Himeur, Y. (2024). A survey on the use of large language models (llms) in fake news. Future Internet, 16(8):298.

Raza, S. and Ding, C. (2022). Fake news detection based on news content and social contexts: a transformer-based approach. Int. Jrnl. of Data Science and Analytics.

Wazlawick, R. S. (2009). Metodologia de pesquisa para ciência da computação, volume 2. Elsevier Rio de Janeiro.

Özbay, F. and Alatas, B. (2019). A novel approach for detection of fake news on social media using metaheuristic optimization algorithms. Elektronika ir Elektrotechnika.