A formulation for the minimum learning machine based on linear programming
Abstract
Minimal Learning Machine (MLM) is a supervised learning method whose basic principle is based on a linear mapping between distances in the input and output spaces, followed by an optimization process to, based on estimated distances, provide an estimate for the output in a typical regression case. The MLM test step involves solving a non-convex optimization problem and it may suffer from local minima problems. In this paper, we present a formulation for the out-of-sample step using linear programming.The experiments show that the proposed method achieves similar performance to that obtained with the original algorithm, additionally producing results with small variance.
References
Bertsimas, D. and Tsitsiklis, J. N. (1997). Introduction to Linear Optimization.
Bishop, C. M. et al. (2006). Pattern recognition and machine learning, volume 1. Springer New York.
Hastie, T., Tibshirani, R., and Friedman, J. (2001). Elements of Statistical Learning: Data Mining, Inference and Prediction. Springer.
Hinton, G. E. and Salakhutdinov, R. R. (2006). Reducing the dimensionality of data with neural networks. Science, 313(5786):504-507.
Krizhevsky, A., Sutskever, I., and Hinton, G. E. (2012). Imagenet classification with deep convolutional neural networks. In Pereira, F., Burges, C. J. C., Bottou, L., and Weinberger, K. Q., editors, Advances in Neural Information Processing Systems 25, pages 1097-1105. Curran Associates, Inc.
Lenz, I., Lee, H., and Saxena, A. (2015). Deep learning for detecting robotic grasps. International Journal of Robotics Research (IJRR) Special Issue on Robot Vision.
Sainath, T., Kingsbury, B., Saon, G., Soltau, H., rahman Mohamed, A., Dahl, G., and Ramabhadran, B. (2015). Deep convolutional neural networks for large-scale speech tasks. Neural Networks, 64:39-48.
Silver, D., Huang, A., Maddison, C. J., Guez, A., Sifre, L., van den Driessche, G., Schrittwieser, J., Antonoglou, I., Panneershelvam, V., Lanctot, M., Dieleman, S., Grewe, D., Nham, J., Kalchbrenner, N., Sutskever, I., Lillicrap, T., Leach, M., Kavukcuoglu, K., Graepel, T., and Hassabis, D. (2016). Mastering the game of go with deep neural networks and tree search. Nature, 529:484-503.
SOUZA JUNIOR, A. H., Corona, F., Miché, Y., Lendasse, A., Barreto, G., and Simula, O. (2013). Minimal learning machine: A new distance-based method for supervised learning. In Proceedings of the 12th International Work Conference on Artificial Neural Networks (IWANN'2013), volume 7902 of Lecture Notes in Computer Science, pages 408-416. Springer.
Souza Júnior, A. H., Corona, F., Barreto, G. A., Miche, Y., and Lendasse, A. (2015). Minimal learning machine: A novel supervised distance-based approach for regression and classification. Neurocomputing, 164:34 - 44.
Bishop, C. M. et al. (2006). Pattern recognition and machine learning, volume 1. Springer New York.
Hastie, T., Tibshirani, R., and Friedman, J. (2001). Elements of Statistical Learning: Data Mining, Inference and Prediction. Springer.
Hinton, G. E. and Salakhutdinov, R. R. (2006). Reducing the dimensionality of data with neural networks. Science, 313(5786):504-507.
Krizhevsky, A., Sutskever, I., and Hinton, G. E. (2012). Imagenet classification with deep convolutional neural networks. In Pereira, F., Burges, C. J. C., Bottou, L., and Weinberger, K. Q., editors, Advances in Neural Information Processing Systems 25, pages 1097-1105. Curran Associates, Inc.
Lenz, I., Lee, H., and Saxena, A. (2015). Deep learning for detecting robotic grasps. International Journal of Robotics Research (IJRR) Special Issue on Robot Vision.
Sainath, T., Kingsbury, B., Saon, G., Soltau, H., rahman Mohamed, A., Dahl, G., and Ramabhadran, B. (2015). Deep convolutional neural networks for large-scale speech tasks. Neural Networks, 64:39-48.
Silver, D., Huang, A., Maddison, C. J., Guez, A., Sifre, L., van den Driessche, G., Schrittwieser, J., Antonoglou, I., Panneershelvam, V., Lanctot, M., Dieleman, S., Grewe, D., Nham, J., Kalchbrenner, N., Sutskever, I., Lillicrap, T., Leach, M., Kavukcuoglu, K., Graepel, T., and Hassabis, D. (2016). Mastering the game of go with deep neural networks and tree search. Nature, 529:484-503.
SOUZA JUNIOR, A. H., Corona, F., Miché, Y., Lendasse, A., Barreto, G., and Simula, O. (2013). Minimal learning machine: A new distance-based method for supervised learning. In Proceedings of the 12th International Work Conference on Artificial Neural Networks (IWANN'2013), volume 7902 of Lecture Notes in Computer Science, pages 408-416. Springer.
Souza Júnior, A. H., Corona, F., Barreto, G. A., Miche, Y., and Lendasse, A. (2015). Minimal learning machine: A novel supervised distance-based approach for regression and classification. Neurocomputing, 164:34 - 44.
Published
2017-07-02
How to Cite
PEREIRA, Tamara Arruda; DE SOUZA JUNIOR, Amauri Holanda.
A formulation for the minimum learning machine based on linear programming. In: INTEGRATED SOFTWARE AND HARDWARE SEMINAR (SEMISH), 44. , 2017, São Paulo.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2017
.
p. 2566-2574.
ISSN 2595-6205.
DOI: https://doi.org/10.5753/semish.2017.3364.
