Empirical evaluation of classifiers and balancing methods for fraud detection in credit card transactions

• Victor Nicola University of São Paulo
• Marcelo Lauretto EACH-USP
• Karina Valdivia Delgado University of São Paulo

DOI: https://doi.org/10.5753/eniac.2020.12118

Abstract

Machine learning algorithms are widely used in credit card fraud detection systems due to their ability to distinguish between legitimate and fraudulent transactions. A known problem in this field is the high imbalance usually found in the classes, which can compromise the performance of the classifiers. The empirical studies found in the literature apply, at most, two sampling techniques. This article presents a comparative study of five classification models under five different methods of balancing the training sets. The best performance was obtained by random forest, which in addition to having the highest average F-score (0.867), proved to be considerably more robust than the other classifiers in relation to the choice of the balancing technique and attribute selection.

Keywords: Fraud Detection, Credit Card, Balancing, Supervised Learning, Random Forest

References

Altman, N. S. (1992). An introduction to kernel and nearest-neighbor nonparametric regression. The American Statistician, 46(3):175–185.

Awoyemi, J. O., Adetunmbi, A. O., and Oluwadare, S. A. (2017). Credit card fraud detection using machine learning techniques: A comparative analysis. In 2017 International Conference on Computing Networking and Informatics (ICCNI), pages 1–9.

Batista, G. E. A. P. A., Prati, R. C., and Monard, M. C. (2004). A study of the behavior of several methods for balancing machine learning training data. SIGKDD Explor. Newsl., 6(1):20–29.

Bowyer, K. W., Chawla, N. V., Hall, L. O., and Kegelmeyer, W. P. (2011). SMOTE: synthetic minority over-sampling technique. CoRR, abs/1106.1813.

Breiman, L. (2001). Random forests. Mach. Learn., 45(1):5–32.

Chan, P. K., Fan, W., Prodromidis, A. L., and Stolfo, S. J. (1999). Distributed data mining in credit card fraud detection. IEEE Intelligent Systems and their Applications, 14(6):67–74.

Cortes, C. and Vapnik, V. (1995). Support-vector networks. Mach. Learn., 20(3):273–297.

Dhankhad, S., Mohammed, E., and Far, B. (2018). Supervised machine learning algorithms for credit card fraudulent transaction detection: A comparative study. In 2018 IEEE International Conference on Information Reuse and Integration (IRI), pages 122–125.

Dupret, G. and Koda, M. (2001). Bootstrap re-sampling for unbalanced data in supervised learning. European Journal of Operational Research, 134(1):141 – 156.

Hesterberg, T., Monaghan, S., Moore, D., Clipson, A., and Epstein, R. (2003). Bootstrap Methods and Permutation Tests: Companion Chapter 18 to the Practice of Business Statistics. W.H.Freeman and Company, New York.

Khatri, S., Arora, A., and Agrawal, A. P. (2020). Supervised machine learning algorithms for credit card fraud detection: A comparison. In 2020 10th International Conference on Cloud Computing, Data Science Engineering (Confluence), pages 680–683.

Kuhn, M. and Johnson, K. (2013). Applied Predictive Learning. Springer, New York, NY, USA.

Maron, M. E. (1961). Automatic indexing: An experimental inquiry. J. ACM, 8(3):404–417.

Mishra, A. and Ghorpade, C. (2018). Credit card fraud detection on the skewed data using various classification and ensemble techniques. In 2018 IEEE International Students’ Conference on Electrical, Electronics and Computer Science (SCEECS), pages 1–5.

Neter, J., Kutner, M. H., Nachtsheim, C. J., and Wasserman, W. (1996). Applied Linear Statistical Models. Irwin.

Niu, X., Wang, L., and Yang, X. (2019). A comparison study of credit card fraud detection: Supervised versus unsupervised. CoRR, abs/1904.10604.

Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V., Vanderplas, J., Passos, A., Cournapeau, D., Brucher, M., Perrot, M., and Duchesnay, E. (2011). Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, 12:2825–2830.

Ren, H. and Yang, B. (2019). Clustering-based prototype generation for imbalance classification. In 2019 International Conference on Smart Grid and Electrical Automation (ICSGEA), pages 422–426.

Sahin, Y. and Duman, E. (2011). Detecting credit card fraud by decision trees and support vector machines. IMECS 2011 - International MultiConference of Engineers and Computer Scientists 2011, 1:442–447.

Schmidt, M., Le Roux, N., and Bach, F. (2017). Minimizing finite sums with the stochastic average gradient. Math. Program., 162(1–2):83–112.

Tomek, I. (1976). Two modifications of cnn. IEEE Transactions on Systems, Man, and Cybernetics, SMC-6(11):769–772.

Varmedja, D., Karanovic, M., Sladojevic, S., Arsenovic, M., and Anderla, A. (2019). Credit card fraud detection - machine learning methods. pages 1–5.