Hybrid Swarm Enhanced Classifier Ensembles
Resumo
Os Sistemas de Multi-Classificadores (MCSs) constituem um dos paradigmas mais competitivos para a obtenção de classificações precisas no campo do aprendizado de máquina. Este artigo busca avaliar se a utilização de algoritmos híbridos de enxames pode melhorar a performance dos MCSs por meio da otimização de pesos em combinações por voto majoritário ponderado. A metodologia proposta rendeu resultados competitivos em 25 conjuntos de dados de referência. Adotou-se a acurácia como função objetivo a ser maximizada pelas seguintes meta-heurísticas: otimização do exame de partículas (PSO), a colônia artificial de abelhas (ABC), e a alternativa híbrida das anteriores usando a técnica de multi enxames dinâmicos (DM-PSO-ABC).
Referências
Aburomman, A. A. and Ibne Reaz, M. B. (2016). A novel svm-knn-pso ensemble method for intrusion detection system. Applied Soft Computing, 38:360–372.
Aksela, M. (2003). Comparison of classifier selection methods for improving committee performance. In Proceedings of the 4th International Conference on Multiple Classifier Systems, MCS’03, page 84–93, Berlin, Heidelberg. Springer-Verlag.
Albuquerque Filho, F. S. d., Madeiro, F., Fernandes, S. M., de Mattos Neto, P. S., and Ferreira, T. A. (2013). Time-series forecasting of pollutant concentration levels using particle swarm optimization and artificial neural networks. Química Nova, 36(6):783–789.
Bello-Orgaz, G., Jung, J. J., and Camacho, D. (2016). Social big data: Recent achievements and new challenges. Information Fusion, 28:45–59.
Cruz, R. M. O., Hafemann, L. G., Sabourin, R., and Cavalcanti, G. D. C. (2020). Deslib: A dynamic ensemble selection library in python. Journal of Machine Learning Research, 21(8):1–5.
Demsar, J. (2006). Statistical comparisons of classifiers over multiple data sets. The Journal of Machine Learning Research, 7:1–30.
Dorigo, M., Birattari, M., and Stutzle, T. (2006). Ant colony optimization. IEEE Computational Intelligence Magazine, 1(4):28–39.
Feng, X., Xiao, Z., Zhong, B., Dong, Y., and Qiu, J. (2019). Dynamic weighted ensemble classification for credit scoring using markov chain. Applied Intelligence, 49(2):555–568.
Fierrez, J., Morales, A., Vera-Rodriguez, R., and Camacho, D. (2018). Multiple classifiers in biometrics. part 1: Fundamentals and review. Information Fusion, 44:57–64.
García-Martín, and Martínez, J. M. (2015). People detection in surveillance: classification and evaluation. IET Computer Vision, 9(5):779–788.
Geman, S., Bienenstock, E., and Doursat, R. (1992). Neural Networks and the Bias/Variance Dilemma. Neural Computation, 4(1):1–58.
Hassanien, A. E. and Emary, E. (2018). Swarm intelligence: principles, advances, and applications. CRC Press.
Karaboga, D. (2005). An idea based on honey bee swarm for numerical optimization. Technical report, Technical report-tr06, Erciyes university, engineering faculty, computer engineering department.
Kennedy, J. and Eberhart, R. (1995). Particle swarm optimization. In Proceedings of ICNN’95 International Conference on Neural Networks, volume 4, pages 1942–1948 vol.4.
Ksieniewicz, P., Krawczyk, B., and Wózniak, M. (2018). Ensemble of extreme learning machines with trained classifier combination and statistical features for hyperspectral data. Neurocomputing, 271:28–37.
Kuncheva, L. I. (2004). Combining Pattern Classifiers: Methods and Algorithms. WileyInterscience, USA.
Kuncheva, L. I. and Rodríguez, J. J. (2014). A weighted voting framework for classifiers ensembles. Knowledge and Information Systems, 38(2):259–275.
Lima Junior, A. R., Silva, D. A., Mattos Neto, P. S., and Ferreira, T. A. (2010). An experimental study of fitness function and time series forecasting using artificial neural networks. In Proceedings of the 12th annual conference companion on Genetic and evolutionary computation, pages 2015–2018.
Mirjalili, S., Mirjalili, S. M., and Lewis, A. (2014). Grey wolf optimizer. Advances in Engineering Software, 69:46–61.
Mohammed, A. M., Onieva, E., and Wózniak, M. (2020). Training set selection and swarm intelligence for enhanced integration in multiple classifier systems. Applied Soft Computing, 95:106568.
Moreno-Seco, F., Iñesta, J. M., de León, P. J. P., and Micó, L. (2006). Comparison of classifier fusion methods for classification in pattern recognition tasks. In Yeung, D.Y., Kwok, J. T., Fred, A., Roli, F., and de Ridder, D., editors, Structural, Syntactic, and Statistical Pattern Recognition, pages 705–713, Berlin, Heidelberg. Springer Berlin Heidelberg.
Niu, G., Han, T., Yang, B.-S., and Tan, A. C. C. (2007). Multi-agent decision fusion for motor fault diagnosis. Mechanical Systems and Signal Processing, 21(3):1285–1299.
Parker, J. (2001). Rank and response combination from confusion matrix data. Information Fusion, 2(2):113–120.
Sagi, O. and Rokach, L. (2018). Ensemble learning: A survey. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, 8:e1249.
Terpilowski, M. (2019). scikit-posthocs: Pairwise multiple comparison tests in python. The Journal of Open Source Software, 4(36):1169.
Wózniak, M., Graña, M., and Corchado, E. (2014). A survey of multiple classifier systems as hybrid systems. Information Fusion, 16:3–17. Special Issue on Information Fusion in Hybrid Intelligent Fusion Systems.
Wu, D. and Zheng, J. (2012). A Dynamic Multistage Hybrid Swarm Intelligence Optimization Algorithm for Function Optimization. Discrete Dynamics in Nature and Society, 2012:578064.
Xu, L., Krzyzak, A., and Suen, C. (1992). Methods of combining multiple classifiers and IEEE Transactions on Systems, Man, their applications to handwriting recognition. and Cybernetics, 22(3):418–435.
Yang, X.-S. (2010). A New Metaheuristic Bat-Inspired Algorithm, pages 65–74. Springer Berlin Heidelberg, Berlin, Heidelberg.
Zhou, Z.-H. (2012). Ensemble Methods: Foundations and Algorithms. Chapman amp; Hall/CRC, 1st edition.
Aksela, M. (2003). Comparison of classifier selection methods for improving committee performance. In Proceedings of the 4th International Conference on Multiple Classifier Systems, MCS’03, page 84–93, Berlin, Heidelberg. Springer-Verlag.
Albuquerque Filho, F. S. d., Madeiro, F., Fernandes, S. M., de Mattos Neto, P. S., and Ferreira, T. A. (2013). Time-series forecasting of pollutant concentration levels using particle swarm optimization and artificial neural networks. Química Nova, 36(6):783–789.
Bello-Orgaz, G., Jung, J. J., and Camacho, D. (2016). Social big data: Recent achievements and new challenges. Information Fusion, 28:45–59.
Cruz, R. M. O., Hafemann, L. G., Sabourin, R., and Cavalcanti, G. D. C. (2020). Deslib: A dynamic ensemble selection library in python. Journal of Machine Learning Research, 21(8):1–5.
Demsar, J. (2006). Statistical comparisons of classifiers over multiple data sets. The Journal of Machine Learning Research, 7:1–30.
Dorigo, M., Birattari, M., and Stutzle, T. (2006). Ant colony optimization. IEEE Computational Intelligence Magazine, 1(4):28–39.
Feng, X., Xiao, Z., Zhong, B., Dong, Y., and Qiu, J. (2019). Dynamic weighted ensemble classification for credit scoring using markov chain. Applied Intelligence, 49(2):555–568.
Fierrez, J., Morales, A., Vera-Rodriguez, R., and Camacho, D. (2018). Multiple classifiers in biometrics. part 1: Fundamentals and review. Information Fusion, 44:57–64.
García-Martín, and Martínez, J. M. (2015). People detection in surveillance: classification and evaluation. IET Computer Vision, 9(5):779–788.
Geman, S., Bienenstock, E., and Doursat, R. (1992). Neural Networks and the Bias/Variance Dilemma. Neural Computation, 4(1):1–58.
Hassanien, A. E. and Emary, E. (2018). Swarm intelligence: principles, advances, and applications. CRC Press.
Karaboga, D. (2005). An idea based on honey bee swarm for numerical optimization. Technical report, Technical report-tr06, Erciyes university, engineering faculty, computer engineering department.
Kennedy, J. and Eberhart, R. (1995). Particle swarm optimization. In Proceedings of ICNN’95 International Conference on Neural Networks, volume 4, pages 1942–1948 vol.4.
Ksieniewicz, P., Krawczyk, B., and Wózniak, M. (2018). Ensemble of extreme learning machines with trained classifier combination and statistical features for hyperspectral data. Neurocomputing, 271:28–37.
Kuncheva, L. I. (2004). Combining Pattern Classifiers: Methods and Algorithms. WileyInterscience, USA.
Kuncheva, L. I. and Rodríguez, J. J. (2014). A weighted voting framework for classifiers ensembles. Knowledge and Information Systems, 38(2):259–275.
Lima Junior, A. R., Silva, D. A., Mattos Neto, P. S., and Ferreira, T. A. (2010). An experimental study of fitness function and time series forecasting using artificial neural networks. In Proceedings of the 12th annual conference companion on Genetic and evolutionary computation, pages 2015–2018.
Mirjalili, S., Mirjalili, S. M., and Lewis, A. (2014). Grey wolf optimizer. Advances in Engineering Software, 69:46–61.
Mohammed, A. M., Onieva, E., and Wózniak, M. (2020). Training set selection and swarm intelligence for enhanced integration in multiple classifier systems. Applied Soft Computing, 95:106568.
Moreno-Seco, F., Iñesta, J. M., de León, P. J. P., and Micó, L. (2006). Comparison of classifier fusion methods for classification in pattern recognition tasks. In Yeung, D.Y., Kwok, J. T., Fred, A., Roli, F., and de Ridder, D., editors, Structural, Syntactic, and Statistical Pattern Recognition, pages 705–713, Berlin, Heidelberg. Springer Berlin Heidelberg.
Niu, G., Han, T., Yang, B.-S., and Tan, A. C. C. (2007). Multi-agent decision fusion for motor fault diagnosis. Mechanical Systems and Signal Processing, 21(3):1285–1299.
Parker, J. (2001). Rank and response combination from confusion matrix data. Information Fusion, 2(2):113–120.
Sagi, O. and Rokach, L. (2018). Ensemble learning: A survey. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, 8:e1249.
Terpilowski, M. (2019). scikit-posthocs: Pairwise multiple comparison tests in python. The Journal of Open Source Software, 4(36):1169.
Wózniak, M., Graña, M., and Corchado, E. (2014). A survey of multiple classifier systems as hybrid systems. Information Fusion, 16:3–17. Special Issue on Information Fusion in Hybrid Intelligent Fusion Systems.
Wu, D. and Zheng, J. (2012). A Dynamic Multistage Hybrid Swarm Intelligence Optimization Algorithm for Function Optimization. Discrete Dynamics in Nature and Society, 2012:578064.
Xu, L., Krzyzak, A., and Suen, C. (1992). Methods of combining multiple classifiers and IEEE Transactions on Systems, Man, their applications to handwriting recognition. and Cybernetics, 22(3):418–435.
Yang, X.-S. (2010). A New Metaheuristic Bat-Inspired Algorithm, pages 65–74. Springer Berlin Heidelberg, Berlin, Heidelberg.
Zhou, Z.-H. (2012). Ensemble Methods: Foundations and Algorithms. Chapman amp; Hall/CRC, 1st edition.
Publicado
29/11/2021
Como Citar
BARBOSA, José Matheus Lacerda; LIMA, Adriano Marabuco de Albuquerque; MATTOS NETO, Paulo Salgado Gomes de; OLIVEIRA, Adriano Lorena Inácio de.
Hybrid Swarm Enhanced Classifier Ensembles. In: ENCONTRO NACIONAL DE INTELIGÊNCIA ARTIFICIAL E COMPUTACIONAL (ENIAC), 18. , 2021, Evento Online.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2021
.
p. 314-325.
ISSN 2763-9061.
DOI: https://doi.org/10.5753/eniac.2021.18263.
