Machine Learning for Malware Detection: Beyond Accuracy Rates
ResumoToday's world is supported by connected, electronic systems, thus ensuring their secure operation is essential to our daily lives. A major threat to system's security is malware infections, which cause ﬁnancial and image losses to corporate and end-users, thus motivating the development of malware detectors. In this scenario, Machine Learning (ML) has been demonstrated to be a powerful technique to develop classiﬁers able to distinguish malware from goodware samples. However, many ML research work on malware detection focus only on the ﬁnal detection accuracy rate and overlook other important aspects of classiﬁer's implementation and evaluation, such as feature extraction and parameter selection. In this paper, we shed light to these aspects to highlight the challenges and drawbacks of ML-based malware classiﬁers development. We trained 25 distinct classiﬁcation models and applied them to 2,800 real x86, Linux ELF malware binaries. Our results shows that: (i) dynamic features outperforms static features when the same classiﬁers are considered; (ii) Discrete-bounded features present smaller accuracy variance over time in comparison to continuous features, at the cost of some time-localized accuracy loss; (iii) Datasets presenting distinct characteristics (e.g., temporal changes) impose generalization challenges to ML models; and (iv) Feature analysis can be used as feedback information for malware detection and infection prevention. We expect that our work could help other researchers when developing their ML-based malware classiﬁcation solutions.
Babaagba, K. O. and Adesanya, S. O. (2019). A study on the effect of feature selection on malware analysis using machine learning. ACM ICEIT 2019.
Duncan, B. (2019). Shade ransomware hits high-tech, wholesale, education sectors in u.s, japan, india, thailand, canada. https://bit.ly/2X2beX5.
Feizollah, A., Anuar, N. B., Salleh, R., and Wahab, A. W. A. (2015). A review on feature selection in mobile malware detection. Digit. Investig., 13(C):22–37.
Galante, L. B., Botacin, M. F., Grégio, A. R. A., , and de Geus, P. L. (2018). Malicious linux binaries: A landscape. XVIII SBSeg.
Garcia, F. C. C. and II, F. P. M. (2016). Random forest for malware classication.
Imran, M., Afzal, M., and Qadir, M. A. (2016). Malware classication using dynamic features and hidden markov model. Journal of Intelligent & Fuzzy Systems.
Kruczkowski, M. and Szynkiewicz, E. N. (2014). Support vector machine for malware analysis and classication. WI-IAT. IEEE.
Liangboonprakong, C. and Sornil, O. (2013). Classication of malware families based on n-grams sequential pattern features. In IEEE ICIEA.
Menahem, E., Shabtai, A., and Levhar, A. (2013). Poster: Detecting malware through temporal function-based features. CCS '13. ACM.
Rezende, E., Ruppert, G., Carvalho, T., Theophilo, A., Ramos, F., and de Geus, P. (2018).
Malicious software classication using vgg16 deep neural network's bottleneck fea- tures. In ITNG. Springer.
Stewart, R. (2019). New backdoor malware found infecting wordpress and joomla web- sites. https://bit.ly/2QzWpbQ.