Ant Colony Optimization Architecture for the CASH Problem in the Context of Machine Learning Classification

• Diego D. Fernandes UECE
• Thalyson G. N. da Silva UECE
• Gustavo A. L. de Campos UECE
• Ana Luiza B. de Paula Barros UECE
• Felipe G. Marajo UECE

DOI: https://doi.org/10.5753/latinoware.2025.16561

Resumo

This paper presents ACOMV-CASH, an automated machine learning (AutoML) approach based on Ant Colony Optimization (ACO) to select a classifier from K-Nearest Neighbors (KNN), Multi-Layer Perceptron (MLP), Decision Tree, Random Forest Classifier (RFC), and Support Vector Machine (SVM), while also optimizing model-specific hyperparameters. The framework incorporates a preprocessing pipeline that automatically prepares the dataset. Finally, the proposed algorithm brings the traditional ACO architecture back into the spotlight by structuring the search space as a directed graph (digraph) and incorporating novel strategies, such as the generation of continuous variables and hierarchical modeling. Evaluation of eight UCI datasets demonstrates competitive performance compared to existing AutoML frameworks, highlighting the robustness of ACOMV-CASH.

Palavras-chave: AutoML Ant Colony Optimization Classification

Referências

F. Hutter, L. Kotthoff, and J. Vanschoren, Automated machine learning: methods, systems, challenges. Springer Nature, 2019.

C. Thornton, F. Hutter, H. H. Hoos, and K. Leyton-Brown, “Auto-weka: Combined selection and hyperparameter optimization of classification algorithms,” in Proceedings of the 19th ACM SIGKDD international conference on Knowledge discovery and data mining, 2013, pp. 847–855.

M. Feurer, A. Klein, K. Eggensperger, J. Springenberg, M. Blum, and F. Hutter, “Efficient and robust automated machine learning,” Advances in neural information processing systems, vol. 28, 2015.

H. Jin, Q. Song, and X. Hu, “Auto-keras: An efficient neural architecture search system,” in Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining, 2019, pp. 1946–1956.

N. K. Kasabov, Foundations of neural networks, fuzzy systems, and knowledge engineering. Marcel Alencar, 1996.

J. Waring, C. Lindvall, and R. Umeton, “Automated machine learning: Review of the state-of-the-art and opportunities for healthcare,” Artificial intelligence in medicine, vol. 104, p. 101822, 2020.

D. Whitley, “A genetic algorithm tutorial,” Statistics and computing, vol. 4, pp. 65–85, 1994.

Y. Sun, B. Xue, M. Zhang, G. G. Yen, and J. Lv, “Automatically designing cnn architectures using the genetic algorithm for image classification,” IEEE transactions on cybernetics, vol. 50, no. 9, pp. 3840–3854, 2020.

S. Nematzadeh, F. Kiani, M. Torkamanian-Afshar, and N. Aydin, “Tuning hyperparameters of machine learning algorithms and deep neural networks using metaheuristics: A bioinformatics study on biomedical and biological cases,” Computational biology and chemistry, vol. 97, p. 107619, 2022.

Y. Wang, H. Zhang, and G. Zhang, “cpso-cnn: An efficient pso-based algorithm for fine-tuning hyper-parameters of convolutional neural networks,” Swarm and Evolutionary Computation, vol. 49, pp. 114–123, 2019.

H. J. Escalante, M. Montes, and L. E. Sucar, “Particle swarm model selection.” Journal of Machine Learning Research, vol. 10, no. 2, 2009.

B. Shahriari, K. Swersky, Z. Wang, R. P. Adams, and N. De Freitas, “Taking the human out of the loop: A review of bayesian optimization,” Proceedings of the IEEE, vol. 104, no. 1, pp. 148–175, 2015.

P. Liashchynskyi and P. Liashchynskyi, “Grid search, random search, genetic algorithm: a big comparison for nas,” arXiv preprint arXiv:1912.06059, 2019.

T. Domhan, J. T. Springenberg, and F. Hutter, “Speeding up automatic hyperparameter optimization of deep neural networks by extrapolation of learning curves.” in IJCAI, vol. 15, 2015, pp. 3460–8.

D. Corne, M. Dorigo, F. Glover, D. Dasgupta, P. Moscato, R. Poli, and K. V. Price, New ideas in optimization. McGraw-Hill Ltd., UK, 1999.

T. M. Shami, A. A. El-Saleh, M. Alswaitti, Q. Al-Tashi, M. A. Summakieh, and S. Mirjalili, “Particle swarm optimization: A comprehensive survey,” Ieee Access, vol. 10, pp. 10 031–10 061, 2022.

B. Komer, J. Bergstra, and C. Eliasmith, “Hyperopt-sklearn: Automatic hyperparameter configuration for scikit-learn.” in Scipy, 2014, pp. 32–37.

I. Brezina Jr and Z. Čičková, “Solving the travelling salesman problem using the ant colony optimization,” Management Information Systems, vol. 6, no. 4, pp. 10–14, 2011.

K. Socha and M. Dorigo, “Ant colony optimization for continuous domains,” European journal of operational research, vol. 185, no. 3, pp. 1155–1173, 2008.

T. Liao, K. Socha, M. A. M. de Oca, T. Stützle, and M. Dorigo, “Ant colony optimization for mixed-variable optimization problems,” IEEE Transactions on evolutionary computation, vol. 18, no. 4, pp. 503–518, 2013.

V. O. Costa and C. R. Rodrigues, “Hierarchical ant colony for simultaneous classifier selection and hyperparameter optimization,” in 2018 IEEE congress on evolutionary computation (CEC). IEEE, 2018, pp. 1–8.

V. Rossum, “Python 3 reference manual,” 2009.

F. Chollet et al., “Keras. [link] 2015.

P. Fabian, “Scikit-learn: Machine learning in python,” Journal of machine learning research 12, p. 2825, 2011.

C. R. Harris, K. J. Millman, S. J. Van Der Walt, R. Gommers, P. Virtanen, D. Cournapeau, E. Wieser, J. Taylor, S. Berg, N. J. Smith et al., “Array programming with numpy,” Nature, vol. 585, no. 7825, pp. 357–362, 2020.

W. McKinney et al., “Data structures for statistical computing in python.” SciPy, vol. 445, no. 1, pp. 51–56, 2010.

E. LeDell and S. Poirier, “H2o automl: Scalable automatic machine learning,” in Proceedings of the AutoML Workshop at ICML, vol. 2020, 2020, p. 24.

A. Płónska and P. Płónski, “Mljar: State-of-the-art automated machine learning framework for tabular data. version 0.10.3,” Łapy, Poland, 2021. [Online]. Available: [link]