Malware Classification using Transfer Learning through the GPT-2 model

• Matheus Vanzan IME
• Julio Cesar Duarte IME

DOI: https://doi.org/10.5753/sbseg.2023.233086

Resumo

Malware detection and classification pose critical challenges in the field of cybersecurity. In recent years, deep learning techniques have made remarkable progress in addressing the classification problem, outperforming traditional methods. Moreover, Natural Language Processing has proven successful in extending its applications beyond natural language texts across numerous semantic domains. This research work focuses on presenting a proposal that extends the Transfer Learning from OpenAI’s GPT-2 model to identify different malware families, without prior knowledge of their behaviors. The achieved results are highly promising, with an exceptional accuracy rate of 99.72%, close to state-of-the-art results reported for the problem.

Referências

AV-Test (2023). Av-test statistics: Malware. [link]. Accessed on May 12, 2023.

Bisong, E. (2019). Google colaboratory.

Cakir, B. and Dogdu, E. (2018). Malware classification using deep learning methods. In Proceedings of the ACMSE 2018 Conference, pages 1–5.

Chang, J., Venkatasubramanian, K. K., West, A. G., and Lee, I. (2013). Analyzing and defending against web-based malware. ACM Computing Surveys (CSUR), 45(4):1– 35.

Dai, A. M. and Le, Q. V. (2015). Semi-supervised sequence learning. Advances in neural information processing systems, 28.

Damodaran, A., Troia, F. D., Corrado, V. A., Austin, T. H., and Stamp, M. (2022). A comparison of static, dynamic, and hybrid analysis for malware detection. Journal of Computer Virology and Hacking Techniques.

de Albuquerque, D. G., Vieira, L. d. Q., Sant’Ana, R., and Duarte, J. C. (2020). Análise de comportamento de malware utilizando redes neurais recorrentes - uma abordagem por intermédio da previsão de opcodes. Revista Militar de Ciência e Tecnologia, 37(3).

Eagle, C. (2011). The IDA Pro Book. William Pollock, San Francisco, second edition edition.

Gers, F. A. (1999). Learning to forget: Continual prediction with lstm. In 9th International Conference on Artificial Neural Networks: ICANN ’99. IET.

Hugging Face (2021). Hugging face transformers: Gpt-2 documentation. [link]. Accessed on March 20, 2023.

Kaggle Team (2022). Microsoft malware winners’ interview: 1st place, “no to overfitting!”. [link]. Accessed on June 20, 2022.

Kalash, M., Rochan, M., Mohammed, N., Bruce, N. D., Wang, Y., and Iqbal, F. (2018). Malware classification with deep convolutional neural networks. In 2018 9th IFIP international conference on new technologies, mobility and security (NTMS), pages 1–5. IEEE.

Kim, J.-Y., Bu, S.-J., and Cho, S.-B. (2017). Malware detection using deep transferred generative adversarial networks. In International Conference on Neural Information Processing.

Kramer, S. and Bradfield, J. C. (2010). A general definition of malware. Communications of the ACM.

Radford, A., Narasimhan, K., Salimans, T., Sutskever, I., et al. (2018). Improving language understanding by generative pre-training.

Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., Sutskever, I., et al. (2019). Language models are unsupervised multitask learners. OpenAI blog, 1(8):9.

Ronen, R., Radu, M., Feuerstein, C., Yom-Tov, E., and Ahmadi, M. (2018). Microsoft malware classification challenge. [link].

Şahin, N. (2021). Malware detection using transformers-based model gpt-2. Master’s thesis, Middle East Technical University.

Simonyan, K. and Zisserman, A. (2015). Very deep convolutional networks for large-scale image recognition.

Van Rossum, G. and Drake, F. L. (2009). Python 3 Reference Manual. CreateSpace, Scotts Valley, CA.

Vanzan, M. (2023). GPT-2 Malware Classification Github Repository. [link].

Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, Ł., and Polosukhin, I. (2017). Attention is all you need. Advances in neural information processing systems, 30.

Wang, N., Wang, J., and Zhang, X. (2018). Ynu-hpcc at semeval-2018 task 2: Multiensemble bi-gru model with attention mechanism for multilingual emoji prediction. In Proceedings of The 12th International Workshop on Semantic Evaluation, pages 459–465.

Wolf, T., Debut, L., Sanh, V., Chaumond, J., Delangue, C., Moi, A., Cistac, P., Rault, T., Louf, R., Funtowicz, M., Davison, J., Shleifer, S., von Platen, P., Ma, C., Jernite, Y., Plu, J., Xu, C., Scao, T. L., Gugger, S., Drame, M., Lhoest, Q., and Rush, A. M. (2020). Transformers: State-of-the-art natural language processing. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 38–45, Online. Association for Computational Linguistics.

Yang, S., Yu, X., and Zhou, Y. (2020). LSTM and GRU neural network performance comparison study: Taking Yelp review dataset as an example. In 2020 International workshop on electronic communication and artificial intelligence (IWECAI), pages 98–101. IEEE.

Zhang, G. P. (2000). Neural networks for classification: a survey. IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews), 30(4):451–462.