Otimização multi-objetivo da sequência de camadas em arquiteturas de aprendizagem profunda
Resumo
Selecionar a melhor arquitetura para uma Rede Neural Profunda (DNN) não é uma tarefa trivial, pois há uma enorme quantidade de configurações possíveis (camadas e parâmetros) e grande dificuldade em como escolhê-las. A fim de tornar essa tarefa mais independente da interação humana, este trabalho propõe um método inteligente para otimizar a arquitetura (sequência de camadas) de uma DNN estruturada em cadeia, levando em consideração múltiplos critérios: acurácia e F1 score. O método foi avaliado quanto ao desempenho e comparado às abordagens exaustiva e aleatória. Os resultados obtidos são promissores, mostrando o potencial do método proposto.
Palavras-chave:
Aprendizagem profunda, otimização multi-objetivo
Referências
Assunção, F., Lourenço, N., Machado, P., and Ribeiro, B. (2018). Evolving the topology of large scale deep neural networks. In European Conference on Genetic Programming, pages 19–34. Springer.
Black, D. (1976). Partial justication of the borda count. Public Choice, 28(1):1–15.
Chollet, F. et al. (2015). Keras. https://keras.io/. Accessed: 2019-06-12.
David, O. E. and Greental, I. (2014). Genetic algorithms for evolving deep neural In Proceedings of the Annual Conference on Genetic and Evolutionary networks. Computation, pages 1451–1452. ACM.
Deb, K., Pratap, A., Agarwal, S., and Meyarivan, T. (2002). A fast and elitist multiobjective genetic algorithm: Nsga-ii. IEEE transactions on evolutionary computation, 6(2):182–197.
Deng, L. (2012). The mnist database of handwritten digit images for machine learning research [best of the web]. IEEE Signal Processing Magazine, 29(6):141–142.
Diniz, J. B., Cordeiro, F. R., Miranda, P. B., and da Silva, L. A. T. (2018). A grammarbased genetic programming approach to optimize convolutional neural network architectures. In Anais do XV Encontro Nacional de Inteligência Articial e Computacional, pages 82–93. SBC.
Elsken, T., Metzen, J. H., and Hutter, F. (2018). Neural architecture search: A survey.arXiv preprint arXiv:1808.05377.
Hadka, D. (2017). Platypus: A free and open source python library for multiobjective optimization. Available on Github, vol. https://github.com/Project-Platypus/Platypus.
Ishibuchi, H., Tsukamoto, N., and Nojima, Y. (2008). Evolutionary many-objective optimization: A short review. In IEEE World Congress on Computational Intelligence)., pages 2419–2426. IEEE.
Krizhevsky, A. and Hinton, G. (2009). Learning multiple layers of features from tiny images. Technical report, Citeseer.
LeCun, Y., Bengio, Y., and Hinton, G. (2015). Deep learning. nature, 521(7553):436.
Liu, S., Yang, J., Huang, C., and Yang, M.-H. (2015). Multi-objective convolutional learning for face labeling. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 3451–3459.
Lorenzo, P. R., Nalepa, J., Kawulok, M., Ramos, L. S., and Pastor, J. R. (2017). Particle swarm optimization for hyper-parameter selection in deep neural networks. In Proceedings of the Genetic and Evolutionary Computation Conference, pages 481–488. ACM.
Miikkulainen, R., Liang, J., Meyerson, E., Rawal, A., Fink, D., Francon, O., Raju, B., Shahrzad, H., Navruzyan, A., Duffy, N., et al. (2019). Evolving deep neural networks. In Articial Intelligence in the Age of Neural Networks and Brain Computing, pages 293–312. Elsevier.
Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V., et al. (2011). Scikit-learn: Machine learning in python. Journal of machine learning research, 12(Oct):2825–2830.
Schaffer, J. D., Whitley, D., and Eshelman, L. J. (1992). Combinations of genetic alIn [Proceedings] gorithms and neural networks: A survey of the state of the art. COGANN-92: International Workshop on Combinations of Genetic Algorithms and Neural Networks, pages 1–37. IEEE.
Wang, W., Zhang, L.-L., Chen, J.-J., and Wang, J.-H. (2015). Parameter estimation for coupled hydromechanical simulation of dynamic compaction based on pareto multiobjective optimization. Shock and Vibration, 2015.
Yang, C., An, Z., Li, C., Diao, B., and Xu, Y. (2019). Multi-objective pruning for cnns using genetic algorithm. arXiv preprint arXiv:1906.00399.
Young, S. R., Rose, D. C., Karnowski, T. P., Lim, S.-H., and Patton, R. M. (2015). Optimizing deep learning hyper-parameters through an evolutionary algorithm. In Proceedings of the Workshop on Machine Learning in High-Performance Computing Environments, page 4. ACM.
Black, D. (1976). Partial justication of the borda count. Public Choice, 28(1):1–15.
Chollet, F. et al. (2015). Keras. https://keras.io/. Accessed: 2019-06-12.
David, O. E. and Greental, I. (2014). Genetic algorithms for evolving deep neural In Proceedings of the Annual Conference on Genetic and Evolutionary networks. Computation, pages 1451–1452. ACM.
Deb, K., Pratap, A., Agarwal, S., and Meyarivan, T. (2002). A fast and elitist multiobjective genetic algorithm: Nsga-ii. IEEE transactions on evolutionary computation, 6(2):182–197.
Deng, L. (2012). The mnist database of handwritten digit images for machine learning research [best of the web]. IEEE Signal Processing Magazine, 29(6):141–142.
Diniz, J. B., Cordeiro, F. R., Miranda, P. B., and da Silva, L. A. T. (2018). A grammarbased genetic programming approach to optimize convolutional neural network architectures. In Anais do XV Encontro Nacional de Inteligência Articial e Computacional, pages 82–93. SBC.
Elsken, T., Metzen, J. H., and Hutter, F. (2018). Neural architecture search: A survey.arXiv preprint arXiv:1808.05377.
Hadka, D. (2017). Platypus: A free and open source python library for multiobjective optimization. Available on Github, vol. https://github.com/Project-Platypus/Platypus.
Ishibuchi, H., Tsukamoto, N., and Nojima, Y. (2008). Evolutionary many-objective optimization: A short review. In IEEE World Congress on Computational Intelligence)., pages 2419–2426. IEEE.
Krizhevsky, A. and Hinton, G. (2009). Learning multiple layers of features from tiny images. Technical report, Citeseer.
LeCun, Y., Bengio, Y., and Hinton, G. (2015). Deep learning. nature, 521(7553):436.
Liu, S., Yang, J., Huang, C., and Yang, M.-H. (2015). Multi-objective convolutional learning for face labeling. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 3451–3459.
Lorenzo, P. R., Nalepa, J., Kawulok, M., Ramos, L. S., and Pastor, J. R. (2017). Particle swarm optimization for hyper-parameter selection in deep neural networks. In Proceedings of the Genetic and Evolutionary Computation Conference, pages 481–488. ACM.
Miikkulainen, R., Liang, J., Meyerson, E., Rawal, A., Fink, D., Francon, O., Raju, B., Shahrzad, H., Navruzyan, A., Duffy, N., et al. (2019). Evolving deep neural networks. In Articial Intelligence in the Age of Neural Networks and Brain Computing, pages 293–312. Elsevier.
Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V., et al. (2011). Scikit-learn: Machine learning in python. Journal of machine learning research, 12(Oct):2825–2830.
Schaffer, J. D., Whitley, D., and Eshelman, L. J. (1992). Combinations of genetic alIn [Proceedings] gorithms and neural networks: A survey of the state of the art. COGANN-92: International Workshop on Combinations of Genetic Algorithms and Neural Networks, pages 1–37. IEEE.
Wang, W., Zhang, L.-L., Chen, J.-J., and Wang, J.-H. (2015). Parameter estimation for coupled hydromechanical simulation of dynamic compaction based on pareto multiobjective optimization. Shock and Vibration, 2015.
Yang, C., An, Z., Li, C., Diao, B., and Xu, Y. (2019). Multi-objective pruning for cnns using genetic algorithm. arXiv preprint arXiv:1906.00399.
Young, S. R., Rose, D. C., Karnowski, T. P., Lim, S.-H., and Patton, R. M. (2015). Optimizing deep learning hyper-parameters through an evolutionary algorithm. In Proceedings of the Workshop on Machine Learning in High-Performance Computing Environments, page 4. ACM.
Publicado
15/10/2019
Como Citar
CASTRO, Mayara; F. NETO, Giuseppe; MIRANDA, Péricles de; CORDEIRO, Filipe; PRUDÊNCIO, Ricardo.
Otimização multi-objetivo da sequência de camadas em arquiteturas de aprendizagem profunda. In: ENCONTRO NACIONAL DE INTELIGÊNCIA ARTIFICIAL E COMPUTACIONAL (ENIAC), 16. , 2019, Salvador.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2019
.
p. 25-36.
ISSN 2763-9061.
DOI: https://doi.org/10.5753/eniac.2019.9269.
