Evaluating social bots detection approaches in different domains

• Daniel Morais Universidade de São Paulo
• Luciano Antonio Digiampietri Universidade de São Paulo https://orcid.org/0000-0003-4890-1548

Resumo

Context: Social bots are automated users who make use of social networks to publish and interact with network users, mimicking or attempting to alter user behaviors with purposes such as spreading spam, malicious content, or misleading information, with various negative effects. Problem: Detecting these bots is a major challenge since, as detection mechanisms evolve, they are also enhanced to avoid such mechanisms, either by improving strategies for emulating real users or by organizing groups of bots in networks with the same purpose (botnets). IS Theory: The paper was developed considering Social Network Theory and Social Information Processing Theory. Method: The paper evaluates bots detection techniques by comparing the classifiers trained against three distinct datasets, aiming to emulate the behavior of a social network through time, to verify the performance of the classifiers in distinct conditions and the resilience of those techniques. Contributions and Impact in the IS area: The objective is to evaluate the effectiveness of the most common techniques in the domain in a variety of conditions based on the datasets used, an important challenge in the development and deployment of information systems. Summary of Results: The performance of the classifiers, when confronted against other datasets, was poor, showing that the classifiers trained for this purpose require constant maintenance to remain effective, reinforcing the need for improved techniques that are more resilient to changes over time and subject of messages. Proposed Solution: To counter those weaknesses, techniques that explore other characteristics, such as the message content, could be explored to improve the resilience of the classifiers.

Palavras-chave: Bots detection, Social networks, Social bots

Referências

Leo Breiman. 2001. Random Forests. Machine Learning 45, 01 (Oct 2001), 5–32. https://doi.org/10.1023/A:1010933404324

Daniel Marques Gomes de Morais and Luciano Antonio Digiampietri. 2021. Methods and Challenges in Social Bots Detection: A Systematic Review. In XVII Brazilian Symposium on Information Systems (Uberlândia, Brazil) (SBSI 2021). Association for Computing Machinery, New York, NY, USA, Article 38, 8 pages. https://doi.org/10.1145/3466933.3466973

Rodrigo Igawa e Alex Almeida e Bruno Zarpelão e Sylvio Barbon Jr. 2016. Recognition on Online Social Network by user's writing style. iSys - Revista Brasileira de Sistemas de Informação 8, 3(2016), 64–85. 

Juan Echeverria and Shi Zhou. 2017. Discovery, Retrieval, and Analysis of the “Star Wars” Botnet in Twitter. In Proceedings of the 2017 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining 2017 (Sydney, Australia) (ASONAM ’17). Association for Computing Machinery, New York, NY, USA, 1–8. https://doi.org/10.1145/3110025.3110074

J. Fernquist, L. Kaati, and R. Schroeder. 2018. Political Bots and the Swedish General Election. In 2018 IEEE International Conference on Intelligence and Security Informatics (ISI). IEEE, Miami, FL, USA, 124–129. https://doi.org/10.1109/ISI.2018.8587347

Emilio Ferrara, Onur Varol, Clayton Davis, Filippo Menczer, and Alessandro Flammini. 2016. The Rise of Social Bots. Commun. ACM 59, 7 (June 2016), 96–104. https://doi.org/10.1145/2818717

Rodrigo Igawa, Alex Almeida, Bruno Zarpelão, and Sylvio Jr. 2015. Recognition of Compromised Accounts on Twitter. In Anais do XI Simpósio Brasileiro de Sistemas de Informação (Goiania). SBC, Porto Alegre, RS, Brasil, 9–14. 

S. Khaled, N. El-Tazi, and H. M. O. Mokhtar. 2018. Detecting Fake Accounts on Social Media. In 2018 IEEE International Conference on Big Data (Big Data). IEEE, Seattle, WA, USA, 3672–3681. 

Diego Lira, Fernando Xavier, and Luciano Digiampietri. 2021. Combining clustering and classification algorithms for automatic bot detection: a case study on posts about COVID-19. In Anais do XVII Simpósio Brasileiro de Sistemas de Informação (Uberlândia). SBC, Porto Alegre, RS, Brasil, 1–7. https://sol.sbc.org.br/index.php/sbsi/article/view/17726

Michele Mazza, Stefano Cresci, Marco Avvenuti, Walter Quattrociocchi, and Maurizio Tesconi. 2019. RTbust: Exploiting Temporal Patterns for Botnet Detection on Twitter. arxiv:1902.04506 [cs.SI] 

Bjorn Ross, Laura Pilz, Benjamin Cabrera, Florian Brachten, German Neubaum, and Stefan Stieglitz. 2019. Are social bots a real threat? An agent-based model of the spiral of silence to analyse the impact of manipulative actors in social networks. European Journal of Information Systems 28, 4 (2019), 394–412. 

S. Sadiq, Y. Yan, A. Taylor, M. Shyu, S. Chen, and D. Feaster. 2017. AAFA: Associative Affinity Factor Analysis for Bot Detection and Stance Classification in Twitter. In 2017 IEEE International Conference on Information Reuse and Integration (IRI). IEEE, San Diego, CA, USA, 356–365. https://doi.org/10.1109/IRI.2017.25

E. Shaabani, R. Guo, and P. Shakarian. 2018. Detecting Pathogenic Social Media Accounts without Content or Network Structure. In 2018 1st International Conference on Data Intelligence and Security (ICDIS). IEEE, South Padre Island, TX, USA, 57–64. https://doi.org/10.1109/ICDIS.2018.00016

V. S. Subrahmanian, A. Azaria, S. Durst, V. Kagan, A. Galstyan, K. Lerman, L. Zhu, E. Ferrara, A. Flammini, and F. Menczer. 2016. The DARPA Twitter Bot Challenge. Computer 49, 6 (June 2016), 38–46. https://doi.org/10.1109/MC.2016.183

Gabriel Tavares, Saulo Mastelini, and Sylvio Jr.2017. User Classification on Online Social Networks by Post Frequency. In Anais do XIII Simpósio Brasileiro de Sistemas de Informação (Lavras). SBC, Porto Alegre, RS, Brasil, 464–471. 

Onur Varol, Emilio Ferrara, Clayton A. Davis, Filippo Menczer, and Alessandro Flammini. 2017. Online Human-Bot Interactions: Detection, Estimation, and Characterization. arxiv:1703.03107 [cs.SI] 

Kai-Cheng Yang, Onur Varol, Pik-Mai Hui, and Filippo Menczer. 2020. Scalable and Generalizable Social Bot Detection through Data Selection. Proceedings of the AAAI Conference on Artificial Intelligence 34, 01 (Apr 2020), 1096–1103. https://doi.org/10.1609/aaai.v34i01.5460