On Model Complexity Reduction in Instance-Based Learners - Oliveira, Saulo A. F.; Rocha Neto, Ajalmar R.; Gomes, João P. P. (Anais Estendidos da Conference on Graphics, Patterns and Images (SIBGRAPI); 2022: Anais Estendidos da XXXV Conference on Graphics, Patterns and Images, 2022)
Instance-based learners habitually adopt instance selection techniques to reduce complexity and avoid overfitting. Such learners’ most recent and well-known formulations seek to impose some sparsity in their training and prediction structure alongside regularization to meet such a result. Due to the variety of such instance-based learners, we will draw attention to the Least-Squares Support Vector Machines and Minimal Learning Machines because they embody additional information beyond the stored instances to perform predictions. Later, this thesis proposes variants constraining candidate solutions within a specific functional space where we avoid overfitting and reduce model complexity. The central core of such variants is related to penalizing samples with a specific condition during learning. For regressors, we adopted strategies based on random and observed linearity conditions related to the data. At the same time, we borrowed definitions from the computer vision field for classification tasks to derive a concept we call the classcorner relationship (in which we designed an instance selection algorithm). In the Least-Squares Support Vector Machines context, this thesis follows the pruning fashion by adopting the samples that share such a class-corner relationship. As for the Minimal Learning Machine model, this thesis introduces a new proposal called the Lightweight Minimal Learning Machine, a faster model for out-of-sample prediction due to the reduced number of computations inherent in the original proposal’s multilateration process. Another remarkable feature is that it derives a unique solution when other formulations rely on overdetermined systems.