Assessing Multi-Objective Search Engines for GE: A Case Study in CNN Generation
Resumo
In recent years, the number of available Convolutional Neural Networks (CNNs) has increased significantly, making it difficult to select an appropriate CNN for a specific problem. To address this challenge, researchers have proposed automated techniques for optimizing CNN architectures, with Grammatical Evolution (GE) being one of the most promising approaches. GE uses context-free grammar to generate programs (e.g., CNNs) and a search engine to find the best solutions. Although several grammars have been proposed for CNN generation, there has been no research evaluating the impact of different search engines in the GE optimization process. This study treats the CNN generation as a multi-objective problem by optimizing accuracy and F1-score, and evaluates seven different multi-objective optimizers listed in the literature as potential search engines. The goal is to investigate the strengths and weaknesses of each optimizer in CNN generation. The experiments were performed on the widely-used CIFAR-10 image classification dataset, and the results showed that selecting the right optimizer for the task is crucial and can have a significant impact on the final result, especially when the number of generations is limited.
Palavras-chave:
CNN generation, Grammatical Evolution, AutoML, Neural Architecture Search
Referências
Assunção, F., Lourenço, N., Machado, P., and Ribeiro, B. (2018). Evolving the topology of large scale deep neural networks. In European Conference on Genetic Programming, pages 19–34.
Corne, D. W., Jerram, N. R., Knowles, J. D., and Oates, M. J. (2001). Pesa-ii: Region-based selection in evolutionary multi-objective optimization. In Proceedings of the 3rd Annual Conference on Genetic and Evolutionary Computation, GECCO’01, page 283–290, San Francisco, CA, USA. Morgan Kaufmann Publishers Inc.
da Silva, C. A., Miranda, P. B., and Cordeiro, F. R. (2021a). A new grammar for creating convolutional neural networks applied to medical image classification. In SIBGRAPI 2021 34th Conference on Graphics, Patterns and Images.
da Silva, C. A., Rosa, D. C., Miranda, P. B., Cordeiro, F. R., Si, T., Nascimento, A. C., Mello, R. F., and de Mattos Neto, P. S. (2021b). A multi-objective grammatical evolution framework to generate convolutional neural network architectures. In 2021 IEEE Congress on Evolutionary Computation (CEC), pages 2187–2194. IEEE.
da Silva, C. A., Rosa, D. C., Miranda, P. B., Cordeiro, F. R., Si, T., Nascimento, A. C., Mello, R. F., and de Mattos Neto, P. S. (2023). A novel multi-objective grammar-based framework for the generation of convolutional neural networks. Expert Systems with Applications, 212:118670.
de Lima, R. H. R., Pozo, A., and Santana, R. (2019). Automatic design of convolutional neural networks using grammatical evolution. In Brazilian Conference on Intelligent Systems (BRACIS), pages 329–334.
Deb, K., Pratap, A., Agarwal, S., and Meyarivan, T. (2002). A fast and elitist multiobjective genetic algorithm: Nsga-ii. IEEE transactions on evolutionary computation, 6(2):182–197.
Diniz, J. B., Cordeiro, F. R., Miranda, P. B., and da Silva, L. A. T. (2018). A grammar-based genetic programming approach to optimize convolutional neural network architectures. In Encontro Nacional de Inteligência Artificial e Computacional, pages 82–93.
Knowles, J. and Corne, D. (1999). The pareto archived evolution strategy: A new baseline algorithm for pareto multiobjective optimisation. volume 1.
Koza, J., Koza, J., and Rice, J. (1992). Genetic Programming: On the Programming of Computers by Means of Natural Selection. A Bradford book. Bradford.
Li, K. (2021). Decomposition multi-objective evolutionary optimization: From state-of-the-art to future opportunities.
Lima, R. H., Magalhães, D., Pozo, A., Mendiburu, A., and Santana, R. (2022). A grammar-based gp approach applied to the design of deep neural networks. Genetic Programming and Evolvable Machines, 23(3):427–452.
Lima, R. H. and Pozo, A. T. (2019). Evolving convolutional neural networks through grammatical evolution. In Proceedings of the Genetic and Evolutionary Computation Conference (GECCO 2019), pages 179–180.
Liu, L., Li, M., and Lin, D. (2007). A novel epsilon-dominance multi-objective evolutionary algorithms for solving drs multi-objective optimization problems. In Third International Conference on Natural Computation (ICNC 2007), volume 4, pages 96–100.
Mariani, T., Guizzo, G., Vergilio, S. R., and Pozo, A. T. (2016). Grammatical evolution for the multi-objective integration and test order problem. In Proceedings of the Genetic and Evolutionary Computation Conference 2016 (GECCO 2016), pages 1069–1076.
Naujoks, B., Hochstrate, N., and Emmerich, M. (2005). Multi-objective optimisation using s-metric selection: application to three-dimensional solution spaces. volume 2, pages 1282 – 1289 Vol. 2.
Neto, G., Miranda, P. B., Cavalcanti, G. D., Si, T., Cordeiro, F., and Castro, M. (2020). Layers sequence optimizing for deep neural networks using multiples objectives. In 2020 IEEE Congress on Evolutionary Computation (CEC), pages 1–8. IEEE.
O’Neill, M. and Ryan, C. (2001). Grammatical evolution. IEEE Transactions on Evolutionary Computation, 5(4):349–358.
Orouskhani, M., Teshnehlab, M., and Nekoui, M. A. (2017). Evolutionary dynamic multiobjective optimization algorithm based on borda count method. International Journal of Machine Learning and Cybernetics, pages 1–29.
O’Neill, M. and Ryan, C. (2004). Grammatical evolution by grammatical evolution: The evolution of grammar and genetic code. In European Conference on Genetic Programming, pages 138–149. Springer.
Ryan, C., Collins, J. J., and Neill, M. O. (1998). Grammatical evolution: Evolving programs for an arbitrary language. In European Conference on Genetic Programming, pages 83–96. Springer.
Zitzler, E., Laumanns, M., and Thiele, L. (2001). Spea2: Improving the strength pareto evolutionary algorithm.
Corne, D. W., Jerram, N. R., Knowles, J. D., and Oates, M. J. (2001). Pesa-ii: Region-based selection in evolutionary multi-objective optimization. In Proceedings of the 3rd Annual Conference on Genetic and Evolutionary Computation, GECCO’01, page 283–290, San Francisco, CA, USA. Morgan Kaufmann Publishers Inc.
da Silva, C. A., Miranda, P. B., and Cordeiro, F. R. (2021a). A new grammar for creating convolutional neural networks applied to medical image classification. In SIBGRAPI 2021 34th Conference on Graphics, Patterns and Images.
da Silva, C. A., Rosa, D. C., Miranda, P. B., Cordeiro, F. R., Si, T., Nascimento, A. C., Mello, R. F., and de Mattos Neto, P. S. (2021b). A multi-objective grammatical evolution framework to generate convolutional neural network architectures. In 2021 IEEE Congress on Evolutionary Computation (CEC), pages 2187–2194. IEEE.
da Silva, C. A., Rosa, D. C., Miranda, P. B., Cordeiro, F. R., Si, T., Nascimento, A. C., Mello, R. F., and de Mattos Neto, P. S. (2023). A novel multi-objective grammar-based framework for the generation of convolutional neural networks. Expert Systems with Applications, 212:118670.
de Lima, R. H. R., Pozo, A., and Santana, R. (2019). Automatic design of convolutional neural networks using grammatical evolution. In Brazilian Conference on Intelligent Systems (BRACIS), pages 329–334.
Deb, K., Pratap, A., Agarwal, S., and Meyarivan, T. (2002). A fast and elitist multiobjective genetic algorithm: Nsga-ii. IEEE transactions on evolutionary computation, 6(2):182–197.
Diniz, J. B., Cordeiro, F. R., Miranda, P. B., and da Silva, L. A. T. (2018). A grammar-based genetic programming approach to optimize convolutional neural network architectures. In Encontro Nacional de Inteligência Artificial e Computacional, pages 82–93.
Knowles, J. and Corne, D. (1999). The pareto archived evolution strategy: A new baseline algorithm for pareto multiobjective optimisation. volume 1.
Koza, J., Koza, J., and Rice, J. (1992). Genetic Programming: On the Programming of Computers by Means of Natural Selection. A Bradford book. Bradford.
Li, K. (2021). Decomposition multi-objective evolutionary optimization: From state-of-the-art to future opportunities.
Lima, R. H., Magalhães, D., Pozo, A., Mendiburu, A., and Santana, R. (2022). A grammar-based gp approach applied to the design of deep neural networks. Genetic Programming and Evolvable Machines, 23(3):427–452.
Lima, R. H. and Pozo, A. T. (2019). Evolving convolutional neural networks through grammatical evolution. In Proceedings of the Genetic and Evolutionary Computation Conference (GECCO 2019), pages 179–180.
Liu, L., Li, M., and Lin, D. (2007). A novel epsilon-dominance multi-objective evolutionary algorithms for solving drs multi-objective optimization problems. In Third International Conference on Natural Computation (ICNC 2007), volume 4, pages 96–100.
Mariani, T., Guizzo, G., Vergilio, S. R., and Pozo, A. T. (2016). Grammatical evolution for the multi-objective integration and test order problem. In Proceedings of the Genetic and Evolutionary Computation Conference 2016 (GECCO 2016), pages 1069–1076.
Naujoks, B., Hochstrate, N., and Emmerich, M. (2005). Multi-objective optimisation using s-metric selection: application to three-dimensional solution spaces. volume 2, pages 1282 – 1289 Vol. 2.
Neto, G., Miranda, P. B., Cavalcanti, G. D., Si, T., Cordeiro, F., and Castro, M. (2020). Layers sequence optimizing for deep neural networks using multiples objectives. In 2020 IEEE Congress on Evolutionary Computation (CEC), pages 1–8. IEEE.
O’Neill, M. and Ryan, C. (2001). Grammatical evolution. IEEE Transactions on Evolutionary Computation, 5(4):349–358.
Orouskhani, M., Teshnehlab, M., and Nekoui, M. A. (2017). Evolutionary dynamic multiobjective optimization algorithm based on borda count method. International Journal of Machine Learning and Cybernetics, pages 1–29.
O’Neill, M. and Ryan, C. (2004). Grammatical evolution by grammatical evolution: The evolution of grammar and genetic code. In European Conference on Genetic Programming, pages 138–149. Springer.
Ryan, C., Collins, J. J., and Neill, M. O. (1998). Grammatical evolution: Evolving programs for an arbitrary language. In European Conference on Genetic Programming, pages 83–96. Springer.
Zitzler, E., Laumanns, M., and Thiele, L. (2001). Spea2: Improving the strength pareto evolutionary algorithm.
Publicado
25/09/2023
Como Citar
CHAGAS, Amerson; ROSA, Daniel; SILVA, Cleber; SI, Tapas; MIRANDA, Péricles B. C..
Assessing Multi-Objective Search Engines for GE: A Case Study in CNN Generation. In: ENCONTRO NACIONAL DE INTELIGÊNCIA ARTIFICIAL E COMPUTACIONAL (ENIAC), 20. , 2023, Belo Horizonte/MG.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2023
.
p. 43-57.
ISSN 2763-9061.
DOI: https://doi.org/10.5753/eniac.2023.233537.
