Comparative study between classification tree algorithms and support vector machines based on ensembles of classifiers

• Melina Brilhadori Universidade de São Paulo
• Marcelo Lauretto Universidade de São Paulo

DOI: https://doi.org/10.5753/sbsi.2013.5679

Abstract

This article presents a comparative analysis of the performance of one algorithm of the classification tree family (J48) and one of the support vector machines (SMO), when combined as ensembles bagging and boosting. The two main questions are: a) For a certain algorithm, which ensemble setting (between Bagging and Boosting) achieves higher accuracy? b) Is there any evidence that a particular classifier performs consistently better than the other under the ensemble setting? Results suggest that J48 tends to achieve a higher accuracy under the Boosting configuration, while SMO seems less sensitive to the ensemble adopted. Nonetheless, both algorithms attained similar numbers of victories among the used datasets

References

Basgalupp, M.P. (2010). LEGAL-Tree: Um algoritmo genetico multi-objetivo lexicográfico para indução de árvores de decisão. Tese de Doutorado. Instituto de Ciências Matemáticas e de Computação. Universidade de São Paulo, São Carlos.

Bauer,E.; Kohavi R. (1998) An Empirical Comparison of Voting Classification Algorithms: Bagging, Boosting, and Variants. Kluwer Academic Publishers, Boston: Machine Learning, vv, 1-38Bouckaert, R.R., Frank, E., Hall, M., Kirkby, R., Reutemann, P., Seewald, A., Scuse, D. (2012) WEKA Manual for Version 3-7-7. Hamilton: The University of Waikato.

Breiman, L. (1996) Bagging predictors. Machine Learning, 24, 123–140.

Coelho, G.P. (2006). Geração, Seleção e Combinação de Componentes para Ensembles de Redes Neurais Aplicadas a Problemas de Classificação. Dissertação de Mestrado. Faculdade de Engenharia Elétrica e de Computação. Universidade Estadual de Campinas.

Frank, A. and Asuncion, A. (2010). UCI Machine Learning Repository [http://archive.ics.uci.edu/ml]. Irvine, CA: University of California, School of Information and Computer Science.

Freund, Y.; Schapire, E.R. (1996); Experiments with a new boosting algorithm. In Proceedings of the International Conference on Machine Learning, p.148–156. Morgan Kaufmann, San Francisco.

Hall, M.; Frank, E.; Holmes, G.; Pfahringer, B.; Reutemann, P.; Witten, I. H. (2009); The WEKA Data Mining Software: An Update; SIGKDD Explorations, Volume 11, Issue 1.

Hosmer, D.W., Lemeshow, S. (2000). Applied Logistic Regression. 2nd ed. New York: Wiley.

Lehmann,C.; Koenig, T.; Jelic,V.; Prichep,L.; John,R.E.; Wahlund,L.; Dodgee,Y.; Dierks,T. (2007) Application and comparison of classification algorithms for recognition of Alzheimer’s disease in electrical brain activity (EEG). Journal of Neuroscience Methods, 161, 342–350

Loh,W.; Shih,Y. (2000) A Comparison of Prediction Accuracy, Complexity, and Training Time of Thirty-three Old and New Classification Algorithms. Kluwer Academic Publishers, Boston: Machine Learning, 40, 203-229.

Maclin, R.; Opitz, D. (1999) Popular Ensemble Methods: An Empirical Study. Journal Of Artificial Intelligence Research, Vol. 11, pag. 169-198.Magalhaes, R. M. (2007); Uma investigação sobre a utilização de processamento paralelo no projeto de máquinas de comitê centro de tecnologia. Dissertação de Mestrado. Universidade Federal do Rio Grande do Norte, Natal, RN.

Mitchell, T.M. (1997). Machine Learning. McGraw-Hill.

Nascimento, D.S.C. (2009) Configuração Heterogênea de Ensembles de Classificadores: Investigação em Bagging, Boosting e MultiBoosting. Dissertação de Mestrado, Universidade de Fortaleza, Fortaleza, CE.

Noether, G.E. (1983). Introdução à Estatística: Uma Abordagem Não Paramétrica. 2ed. Rio de Janeiro: Guanabara Dois.

Platt, J. C. (1998) Sequential Minimal Optimization: A Fast Algorithm for Training Support Vector Machines. Microsoft Research, Technical Report MSR-TR-98-14, USA.

Quinlan, R. (1993) C4.5: Programs for Machine Learning Morgan Kaufmann Publishers Inc., San Francisco, CA, USA.

R Core Team (2011). R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna, Austria. http://www.R-project.org

Witten, I. H.; Frank, E. (2005) Data Mining: Practical Machine Learning Tools and Techniques. Segunda Edição. Ed. Elsevier.