A New Approach for Obtain Reduced Sets in Least Squares Support Vector Machine: Lengthen via Levenberg-Marquardt

• Felipe P. Marinho UFC
• Wellington D. de Almeida UFC
• Victor O. Santos University of Guelph
• Ajalmar R. R. Neto IFCE

Resumo

In this work, a new approach for obtaining reduced sets for training least squares support vector machine (LSSVM) in binary classification tasks is presented. The methodology consists of carrying out a vector quantization procedure, forming clusters in the original feature space, subsequently it is determined which of the clusters presents greater heterogeneity, that is, which one presents more patterns of distinct classes. Starting from the premise that the patterns with the greatest chance of being support vectors are those closest to the decision boundary, the patterns from this cluster are used as candidates for initial support vectors and then the feature space is divided into the most heterogeneous cluster and its complementary. Then, using only complementary set, the LSSVM problem is formulated and solved iteratively via the Levenberg-Marquardt algorithm. This algorithm, in each iteration, considers the lagrange multipliers with the largest absolute values and transfer them from the complementary set to the most heterogenous one. This process continues until the stopping criterion is met. This results in a reduced set formed by the final support vectors for the LSSVM containing both the closest and furthest patterns from the decision boundary which carry great significance for the model. Numerical simulations are conducted to demonstrate the predictive capacity of the proposal, as well as its efficiency in terms of processing time.