Investigating the Performance of the GPT-3.5 Model in Fake News Detection: An Experimental Analysis
Resumo
The dissemination of fake news has become a significant concern in the current society. This problem is evident on social media platforms, where the spread of misinformation has become a constant presence in the daily lives of many individuals. In this work, we investigate the performance of the GPT-3.5 model in classifying fake and real news, considering 200 newspaper articles and two strategies for question formulation. Our results reveal that using a well-formulated question is crucial to obtain more precise responses. In particular, we observed an improvement of 21.1% in the F1-Score metric by directing the question to focus on the characteristics of a fake text.
Referências
Aslam, N., Ullah Khan, I., Alotaibi, F. S., Aldaej, L. A., and Aldubaikil, A. K. (2021). Fake detect: A deep learning ensemble model for fake news detection. Complexity.
Baarir, N. F. and Djeffal, A. (2021). Fake news detection using machine learning. 2020 Workshop on Human-Centric Smart Environments for Health and Well-being (IHSH).
Government, A. (2023). Is it true? Can COVID-19 vaccines alter my DNA? [link]. Acessed: August 11, 2023.
Khivasara, Y., Khare, Y., and Bhadane, T. (2020). Fake news detection system using web-extension. 2020 IEEE Pune Section International Conference.
OpenAI (2023). Models OpenAI API. Disponível em: [link]. Acessed: August 11, 2023.
Raza, S. and Ding, C. (2022). Fake news detection based on news content and social contexts: a transformer-based approach. Int. Jrnl. of Data Science and Analytics.
Rocha, Y. M., de Moura, G. A., Desidério, G. A., de Oliveira, C. H., Lourenço, F. D., and de Figueiredo Nicolete, L. D. (2021). The impact of fake news on social media and its influence on health during the covid-19 pandemic: a systematic review. Journal of Public Health.
Tribunal Superior Eleitoral (2022). Pílulas contra a desinformação: notícias falsas circulam 70% mais rápido do que as verdadeiras. Disponível em: https://bit.ly/43DYRmy. Acessed: August 11, 2023.
Özbay, F. and Alatas, B. (2019). A novel approach for detection of fake news on social media using metaheuristic optimization algorithms. Elektronika ir Elektrotechnika.
Baarir, N. F. and Djeffal, A. (2021). Fake news detection using machine learning. 2020 Workshop on Human-Centric Smart Environments for Health and Well-being (IHSH).
Government, A. (2023). Is it true? Can COVID-19 vaccines alter my DNA? [link]. Acessed: August 11, 2023.
Khivasara, Y., Khare, Y., and Bhadane, T. (2020). Fake news detection system using web-extension. 2020 IEEE Pune Section International Conference.
OpenAI (2023). Models OpenAI API. Disponível em: [link]. Acessed: August 11, 2023.
Raza, S. and Ding, C. (2022). Fake news detection based on news content and social contexts: a transformer-based approach. Int. Jrnl. of Data Science and Analytics.
Rocha, Y. M., de Moura, G. A., Desidério, G. A., de Oliveira, C. H., Lourenço, F. D., and de Figueiredo Nicolete, L. D. (2021). The impact of fake news on social media and its influence on health during the covid-19 pandemic: a systematic review. Journal of Public Health.
Tribunal Superior Eleitoral (2022). Pílulas contra a desinformação: notícias falsas circulam 70% mais rápido do que as verdadeiras. Disponível em: https://bit.ly/43DYRmy. Acessed: August 11, 2023.
Özbay, F. and Alatas, B. (2019). A novel approach for detection of fake news on social media using metaheuristic optimization algorithms. Elektronika ir Elektrotechnika.
Publicado
18/09/2023
Como Citar
ANJOS, Lucas S.; QUINCOZES, Silvio E.; KAZIENKO, Juliano F.; QUINCOZES, Vagner E..
Investigating the Performance of the GPT-3.5 Model in Fake News Detection: An Experimental Analysis. In: SIMPÓSIO BRASILEIRO DE SEGURANÇA DA INFORMAÇÃO E DE SISTEMAS COMPUTACIONAIS (SBSEG), 23. , 2023, Juiz de Fora/MG.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2023
.
p. 552-557.
DOI: https://doi.org/10.5753/sbseg.2023.233647.
