A Grammar-based Genetic Programming Approach to Optimize Convolutional Neural Network Architectures

• Jessica Barbosa Diniz UFRPE
• Filipe R. Cordeiro UFRPE
• Pericles B. C. Miranda UFRPE
• Laura A. Tomaz da Silva PUCRS

DOI: https://doi.org/10.5753/eniac.2018.4406

Resumo

Deep Learning is a research area under the spotlight in recent years due to its successful application to many domains, such as computer vision and image recognition. The most prominent technique derived from Deep Learning is Convolutional Neural Network, which allows the network to automatically learn representations needed for detection or classification tasks. However, Convolutional Neural Networks have some limitations, as designing these networks are not easy to master and require expertise and insight. In this work, we present the use of Genetic Algorithm associated to Grammar-based Genetic Programming to optimize Convolution Neural Network architectures. To evaluate our proposed approach, we adopted CIFAR-10 dataset to validate the evolution of the generated architectures, using the metric of accuracy to evaluate its classification performance in the test dataset. The results demonstrate that our method using Grammar-based Genetic Programming can easily produce optimized CNN architectures that are competitive and achieve high accuracy results.

Referências

Ahmadizar, F., Soltanian, K., AkhlaghianTab, F., and Tsoulos, I. (2015). Artificial neural network development by means of a novel combination of grammatical evolution and genetic algorithm. Engineering Applications of Artificial Intelligence, 39:1–13.

Assunção, F., Lourenço, N., Machado, P., and Ribeiro, B. (2018). Evolving the topology of large scale deep neural networks. In European Conference on Genetic Programming, pages 19–34. Springer.

Chollet, F. et al. (2015). Keras. https://keras.io.

Deng, L., Hinton, G., and Kingsbury, B. (2013). New types of deep neural network learning for speech recognition and related applications: An overview. In Acoustics, Speech and Signal Processing (ICASSP), 2013 IEEE International Conference on, pages 8599–8603. IEEE.

Farfade, S. S., Saberian, M. J., and Li, L.-J. (2015). Multi-view face detection using deep convolutional neural networks. In Proceedings of the 5th ACM on International Conference on Multimedia Retrieval, pages 643–650. ACM.

Fenton, M., McDermott, J., Fagan, D., Forstenlechner, S., Hemberg, E., and O’Neill, M. (2017). Ponyge2: Grammatical evolution in python. In Proceedings of the Genetic and Evolutionary Computation Conference Companion, pages 1194–1201. ACM.

Glorot, X., Bordes, A., and Bengio, Y. (2011). Deep sparse rectifier neural networks. In Proceedings of the fourteenth international conference on artificial intelligence and statistics, pages 315–323.

Goodfellow, I., Bengio, Y., Courville, A., and Bengio, Y. (2016). Deep learning, volume 1. MIT press Cambridge.

Graves, A., Mohamed, A.-r., and Hinton, G. (2013). Speech recognition with deep recurrent neural networks. In Acoustics, speech and signal processing (icassp), 2013 ieee international conference on, pages 6645–6649. IEEE.

Hingee, K. and Hutter, M. (2008). Equivalence of probabilistic tournament and polynomial ranking selection. In Evolutionary Computation, 2008. CEC 2008.(IEEE World Congress on Computational Intelligence). IEEE Congress on, pages 564–571. IEEE.

Hopcroft, J. E. (2008). Introduction to automata theory, languages, and computation. Pearson Education India.

Koza, J. R. (1992). Genetic programming: on the programming of computers by means of natural selection, volume 1. MIT Press.

Krizhevsky, A. and Hinton, G. (2009). Learning multiple layers of features from tiny images.

Krizhevsky, A., Sutskever, I., and Hinton, G. E. (2012). Imagenet classification with deep convolutional neural networks. In Advances in neural information processing systems, pages 1097–1105.

LeCun, Y., Bengio, Y., and Hinton, G. (2015). Deep learning. nature, 521(7553):436.

LeCun, Y., Boser, B. E., Denker, J. S., Henderson, D., Howard, R. E., Hubbard,W. E., and Jackel, L. D. (1990). Handwritten digit recognition with a back-propagation network. In Advances in neural information processing systems, pages 396–404.

Long, J., Shelhamer, E., and Darrell, T. (2015). Fully convolutional networks for semantic segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 3431–3440.

Miikkulainen, R., Liang, J., Meyerson, E., Rawal, A., Fink, D., Francon, O., Raju, B.,

Shahrzad, H., Navruzyan, A., Duffy, N., et al. (2017). Evolving deep neural networks. arXiv preprint arXiv:1703.00548. O’Neill, M. and Ryan, C. (2001). Grammatical evolution. IEEE Transactions on Evolutionary Computation, 5(4):349–358.

O’Neil, M. and Ryan, C. (2003). Grammatical evolution. In Grammatical Evolution, pages 33–47. Springer.

Plis, S. M., Hjelm, D. R., Salakhutdinov, R., Allen, E. A., Bockholt, H. J., Long, J. D., Johnson, H. J., Paulsen, J. S., Turner, J. A., and Calhoun, V. D. (2014). Deep learning for neuroimaging: a validation study. Frontiers in neuroscience, 8:229.

Scherer, D., Müller, A., and Behnke, S. (2010). Evaluation of pooling operations in convolutional architectures for object recognition. In Artificial Neural Networks–ICANN 2010, pages 92–101. Springer.

Simonyan, K. and Zisserman, A. (2014). Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556.

Soni, N. and Kumar, T. (2014). Study of various crossover operators in genetic algorithms. International Journal of Computer Science and Information Technologies, 5(6):7235–7238.

Tsoulos, I., Gavrilis, D., and Glavas, E. (2008). Neural network construction and training using grammatical evolution. Neurocomputing, 72(1-3):269–277.

Zhang, J. and Zong, C. (2015). Deep neural networks in machine translation: An overview. IEEE Intelligent Systems, 30(5):16–25.

Zhou, H., Alvarez, J. M., and Porikli, F. (2016). Less is more: Towards compact cnns. In European Conference on Computer Vision, pages 662–677. Springer.