Solução de Regressão Regularizada com Vetores Suporte através de Programação Linear
Resumo
Nesse trabalho foram introduzidos dois métodos de regressão baseados na teoria de vetor suporte que visam encontrar soluções esparsas. Esses métodos são solucionáveis por programação linear. Um deles só é aplicável a regressão linear entretanto o outro pode ser estendido para o caso não linear através de métodos kernel. Os métodos propostos obtiveram resultados numéricos próximos dos métodos considerados estado da arte.
Palavras-chave:
Máquinas de Vetores Suporte para Regressão, SVR, Regularização, Seleção de Variáveis, Programação Linear, PL
Referências
Alves, A., Chaparro Pinzon, A., Costa, R., Silva, M., Vieira, E., Mendonça, I., Viana, V., and Lôbo, R. (2019). Multiple regression and machine learning based methods for cacass traits and saleable meat cuts prediction using non-invasive in vivo measurements in commercial lambs. Small Ruminant Research, 171:49–56. cited By 0.
Bi, J., Bennett, K., Embrechts, M., Breneman, C., and Song, M. (2003). Dimensionality reduction via sparse support vector machines. J. Mach. Learn. Res., 3:1229–1243.
Boser, B. E., Guyon, I. M., and Vapnik, V. N. (1992). A training algorithm for optimal margin classifiers. In Proceedings of the Fifth Annual Workshop on Computational Learning Theory, COLT ’92, pages 144–152, New York, NY, USA. ACM.
Boucher, T., Ozanne, M., Carmosino, M., Dyar, M., Mahadevan, S., Breves, E., Lepore, K., and Clegg, S. (2015). A study of machine learning regression methods for major elemental analysis of rocks using laser-induced breakdown spectroscopy. Spectrochimica Acta - Part B Atomic Spectroscopy, 107:1–10. cited By 30.
Chang, C.-C. and Lin, C.-J. (2011). LIBSVM: A library for support vector machines. ACM Transactions on Intelligent Systems and Technology, 2:27:1–27:27. Software available at http://www.csie.ntu.edu.tw/˜cjlin/libsvm.
De Rivas, B., Vivancos, J.-L., Ordieres-Meré, J., and Capuz-Rizo, S. (2017). Determnation of the total acid number (tan) of used mineral oils in aviation engines by ftir using regression models. Chemometrics and Intelligent Laboratory Systems, 160:32– 39. cited By 6.
Drucker, H., Burges, C. J. C., Kaufman, L., Smola, A. J., and Vapnik, V. (1997). Support vector regression machines. In Mozer, M. C., Jordan, M. I., and Petsche, T., editors, Advances in Neural Information Processing Systems 9, pages 155–161. MIT Press.
Dua, D. and Graff, C. (2017). UCI machine learning repository.
Friedman, J., Hastie, T., and Tibshirani, R. (2010). Regularization paths for generalized linear models via coordinate descent. Journal of Statistical Software, 33(1):1–22.
Gelius-Dietrich, G. (2017). cplexAPI: R Interface to C API of IBM ILOG CPLEX. R package version 1.3.3.
Jaggi, M. (2013). An equivalence between the lasso and support vector machines. Regularization, optimization, kernels, and support vector machines, chap 1.
Mercer, J. (1909). Functions of positive and negative type, and their connection with the theory of integral equations. Philosophical Transactions of the Royal Society of London. Series A, Containing Papers of a Mathematical or Physical Character, 209:415– 446.
Meyer, D., Dimitriadou, E., Hornik, K., Weingessel, A., and Leisch, F. (2017). e1071: Misc Functions of the Department of Statistics, Probability Theory Group (Formerly: E1071), TU Wien. R package version 1.6-8.
Smola, A., Scholkopf, B., and Ratsch, G. (1999). Linear programs for automatic accuracy control in regression. In 1999 Ninth International Conference on Artificial Neural Networks ICANN 99. (Conf. Publ. No. 470), volume 2, pages 575–580 vol.2.
Smola, A. J. and Schölkopf, B. (2004). A tutorial on support vector regression. Statistics and Computing, 14(3):199–222.
Tibshirani, R. (1996). Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society. Series B (Methodological), 58(1):267–288.
Tikhonov, A. N. (1963). On the solution of ill-posed problems and the method of regularization. Doklady Akademii Nauk SSSR, 151(3):501–504.
Zhou, Q., Chen, W., Song, S., Gardner, J. R., Weinberger, K. Q., and Chen, Y. (2015). A reduction of the elastic net to support vector machines with an application to gpu computing. In Proceedings of the Twenty-Ninth AAAI Conference on Artificial Intelligence, AAAI’15, pages 3210–3216. AAAI Press.
Zou, H. and Hastie, T. (2005). Regularization and variable selection via the elastic net. Journal of the Royal Statistical Society, Series B, 67:301–320.
Bi, J., Bennett, K., Embrechts, M., Breneman, C., and Song, M. (2003). Dimensionality reduction via sparse support vector machines. J. Mach. Learn. Res., 3:1229–1243.
Boser, B. E., Guyon, I. M., and Vapnik, V. N. (1992). A training algorithm for optimal margin classifiers. In Proceedings of the Fifth Annual Workshop on Computational Learning Theory, COLT ’92, pages 144–152, New York, NY, USA. ACM.
Boucher, T., Ozanne, M., Carmosino, M., Dyar, M., Mahadevan, S., Breves, E., Lepore, K., and Clegg, S. (2015). A study of machine learning regression methods for major elemental analysis of rocks using laser-induced breakdown spectroscopy. Spectrochimica Acta - Part B Atomic Spectroscopy, 107:1–10. cited By 30.
Chang, C.-C. and Lin, C.-J. (2011). LIBSVM: A library for support vector machines. ACM Transactions on Intelligent Systems and Technology, 2:27:1–27:27. Software available at http://www.csie.ntu.edu.tw/˜cjlin/libsvm.
De Rivas, B., Vivancos, J.-L., Ordieres-Meré, J., and Capuz-Rizo, S. (2017). Determnation of the total acid number (tan) of used mineral oils in aviation engines by ftir using regression models. Chemometrics and Intelligent Laboratory Systems, 160:32– 39. cited By 6.
Drucker, H., Burges, C. J. C., Kaufman, L., Smola, A. J., and Vapnik, V. (1997). Support vector regression machines. In Mozer, M. C., Jordan, M. I., and Petsche, T., editors, Advances in Neural Information Processing Systems 9, pages 155–161. MIT Press.
Dua, D. and Graff, C. (2017). UCI machine learning repository.
Friedman, J., Hastie, T., and Tibshirani, R. (2010). Regularization paths for generalized linear models via coordinate descent. Journal of Statistical Software, 33(1):1–22.
Gelius-Dietrich, G. (2017). cplexAPI: R Interface to C API of IBM ILOG CPLEX. R package version 1.3.3.
Jaggi, M. (2013). An equivalence between the lasso and support vector machines. Regularization, optimization, kernels, and support vector machines, chap 1.
Mercer, J. (1909). Functions of positive and negative type, and their connection with the theory of integral equations. Philosophical Transactions of the Royal Society of London. Series A, Containing Papers of a Mathematical or Physical Character, 209:415– 446.
Meyer, D., Dimitriadou, E., Hornik, K., Weingessel, A., and Leisch, F. (2017). e1071: Misc Functions of the Department of Statistics, Probability Theory Group (Formerly: E1071), TU Wien. R package version 1.6-8.
Smola, A., Scholkopf, B., and Ratsch, G. (1999). Linear programs for automatic accuracy control in regression. In 1999 Ninth International Conference on Artificial Neural Networks ICANN 99. (Conf. Publ. No. 470), volume 2, pages 575–580 vol.2.
Smola, A. J. and Schölkopf, B. (2004). A tutorial on support vector regression. Statistics and Computing, 14(3):199–222.
Tibshirani, R. (1996). Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society. Series B (Methodological), 58(1):267–288.
Tikhonov, A. N. (1963). On the solution of ill-posed problems and the method of regularization. Doklady Akademii Nauk SSSR, 151(3):501–504.
Zhou, Q., Chen, W., Song, S., Gardner, J. R., Weinberger, K. Q., and Chen, Y. (2015). A reduction of the elastic net to support vector machines with an application to gpu computing. In Proceedings of the Twenty-Ninth AAAI Conference on Artificial Intelligence, AAAI’15, pages 3210–3216. AAAI Press.
Zou, H. and Hastie, T. (2005). Regularization and variable selection via the elastic net. Journal of the Royal Statistical Society, Series B, 67:301–320.
Publicado
15/10/2019
Como Citar
TEIXEIRA, Lucas; F. NETO, Raul.
Solução de Regressão Regularizada com Vetores Suporte através de Programação Linear. In: ENCONTRO NACIONAL DE INTELIGÊNCIA ARTIFICIAL E COMPUTACIONAL (ENIAC), 16. , 2019, Salvador.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2019
.
p. 1032-1043.
ISSN 2763-9061.
DOI: https://doi.org/10.5753/eniac.2019.9355.
