On the Use of Fuzzy ART and SOM Networks in Ensemble Classifiers: A Performance Comparison

• César L. C. Mattos UFC
• Guilherme A. Barreto UFC

Resumo

In this paper we introduce two novel ensemble models that are built using Fuzzy ART (FA) and SOM networks as base classifiers. For this purpose, we first describe three different strategies to convert these unsupervised competitive learning algorithms to supervised ones to allow them to be applied to pattern classification tasks. Then, a metaheuristic solution based on a hybrid PSO algorithm is devised for parameter optimization of the proposed ensemble classifiers. A comprehensive performance comparison using 10 benchmarking data sets indicates that the FAand SOM-based ensemble classifiers consistently outperform ensembles built from standard supervised neural networks, such as the Fuzzy ARTMAP and the Extreme Learning Machine.

Referências

Barreto, G. A. and Aguayo, L. (2009). Time series clustering for anomaly detection using competitive neural networks. In Principe, J. C. and Miikkulainen, R., editors, Proceedings of the 7th International Workshop on Advances in Self-Organizing Maps (WSOM’09), volume LNCS-5629, pages 28–36. Springer.

Biebelmann, E., Köppen, M., and Nickolay, B. (1996). Pratical aplications of neural networks in texture analysis. Neurocomputing, 13(2–4):261–279.

Bratton, D. and Kennedy, J. (2007). Defining a standard for particle swarm optimization. In Proceedings of the IEEE Swarm Intelligence Symposium, pages 120–127, Honolulu, Hawaii.

Breiman, L. (1996). Bagging predictors. Machine Learning, 24(2):123–140.

Carpenter, G. and Grossberg, S. (1987a). Stable self-organization of pattern recognition codes for analog input patterns. Applied Optics, 26:4919–4930.

Carpenter, G., Grossberg, S., and Reynolds, J. H. (1991a). ARTMAP: Supervised real-time learning and classification of nonstationary data by a self-organizing neural network. Neural Networks, 4(5):565–588.

Carpenter, G. A. and Grossberg, S. (1987b). A massively parallel architecture for a self-organizing neural pattern recognition machine. Computer Vision, Graphics, and Image Processing, 37(1):54–115.

Carpenter, G. A., Grossberg, S., and Rosen, D. B. (1991b). Fuzzy ART: Fast stable learning, categorization of analog patterns by an adaptive resonance system. Neural Networks, 4(6):759–771.

Christodoulou, C. I., Michaelides, S. C., and Pattichis, C. S. (2003). Multifeature texture analysis for the classification of clouds in satellite imagery. IEEE Transactions on Geoscience and Remote Sensing, 41(11):2662–2668.

Corchado, E., Baruque, B., and Yin, H. (2007). Boosting unsupervised competitive learning ensembles. In de Sá, J. M., Alexandre, L. A., Duch, W., and Mandic, D. P., editors, Proceedings of the 17th International Conference on Artificial Neural Networks (ICANN’07), Part I, volume LNCS 4668, pages 339–348. Springer.

del-Hoyo, R., Buldain, D., and Marco, A. (2003). Supervised classification with associative SOM. In Proceedings of the 7th International Work-Conference on Artificial and Neural Networks, (IWANN)’03, pages 334–341.

Dietterich, T. G. (1998). Approximate statistical tests for comparing supervised classification learning algorithms. Neural Computation, 10(7):1895–1923.

Dietterich, T. G. (2000). An experimental comparison of three methods for constructing ensembles of decision trees: Bagging, boosting, and randomization. Machine Learning, 40(2):139–157.

Everitt, B. (1977). The Analysis of Contingency Tables. Chapman & Hall/CRC.

Frank, A. and Asuncion, A. (2010). UCI machine learning repository.

Hansen, L. K. and Salamon, P. (2002). Neural network ensembles. IEEE Transactions on Pattern Analysis and Machine Intelligence, 12(10):993–1001.

Haykin, S. (2008). Neural Networks and Learning Machines. Prentice Hall, 3rd edition.

He, Q. and Wang, L. (2007). A hybrid particle swarm optimization with a feasibility-based rule for constrained optimization. Applied Mathematics, Computation, 186(2):1407–1422.

Huang, G. B., Zhu, Q. Y., and Ziew, C. K. (2006). Extreme learning machine: Theory and applications. Neurocomputing, 70(1–3):489–501.

Kennedy, J. and Eberhart, R. C. (1995). Particle swarm optimization. In Proceedings of the IEEE International Conference on Neural Networks, volume 4, pages 1942–1948, Piscataway, NJ, USA.

Keskin, G. A. and Özkan, C. (2009). An alternative evaluation of FMEA: Fuzzy art algorithm. Quality and Reliability Engineering International, 25(6):647–661.

Kohonen, T. (1982). Self-organized formation of topologically correct feature maps. Biological Cybernetics, 43(1):59–69.

Kohonen, T. (1988). The “neural” phonetic typewriter. Computer, 21(3):11–22.

Krogh, A. and Vedelsby, J. (1995). Neural network ensembles, cross validation, active learning. Advances in Neural Information Processing Systems, pages 231–238.

Loo, C. K., Law, A., Lim, W. S., and Rao, M. V. C. (2006). Probabilistic ensemble simplified fuzzy ARTMAP for sonar target differentiation. Neural Computing & Applications, 15(1):79–90.

Pedersen, M. E. H. and Chipperfield, A. J. (2010). Simplifying particle swarm optimization. Applied Soft Computing, 10(2):618–628.

Petrikieva, L. and Fyfe, C. (2002). Bagging and bumping self-organising maps. Computing and Information Systems, 9(2):69.

Rocha Neto, A. R. and Barreto, G. A. (2009). On the application of ensembles of classifiers to the diagnosis of pathologies of the vertebral column: A comparative analysis. IEEE Latin America Transactions, 7(4):487–496.

Salzberg, S. L. (1997). On comparing classifiers: Pitfalls to avoid and a recommended approach. Data Mining and Knowledge Discovery, 1(3):317–328.

Santos, A. M. and Canuto, A. M. P. (2008). Using ARTMAP-based ensemble systems designed by three variants of boosting. In Proceedings of the International Conference on Artificial Neural Networks (ICANN’08), pages 562–571.

Souza Júnior, A. H., Barreto, G. A., and Varela, A. T. (2011). A speech recognition system for embedded applications using the SOM and TS-SOM networks. In Mwasiagi, J. I., editor, Self-Organizing Maps - Applications and Novel Algorithm Design. InTech.

Wyns, B., Sette, S., Boullart, L., Baeten, D., Hoffman, I. E. A., and De Keyser, F. (2004). Prediction of diagnosis in patients with early arthritis using a combined Kohonen mapping and instance-based evaluation criterion. Artificial Intelligence in Medicine, 31(1):45–55.

Xiao, Y.-D., Clauset, A., Harris, R., Bayram, E., Santago, P., and Schmitt, J. D. (2005). Supervised self-organizing maps in drug discovery. 1. robust behavior with overdetermined data sets. Journal of Chemical Information and Modeling, 45(6):1749–1758.