A Thorough Evaluation of Kernel Order in CNN Based Traffic Signs Recognition
Resumo
Convolutional Neural Network is an important deep learning architecture for computer vision. Alongside with its variations, it brought image analysis applications to a new performance level. However, despite its undoubted quality, the evaluation of the performance presented in the literature is mostly restricted to accuracy measurements. So, considering the stochastic characteristics of neural networks training and the impact of the architectures configuration, research is still needed to affirm if such architectures reached the optimal configuration for their focused problems. Statistical significance is a powerful tool for a more accurate experimental evaluation of stochastic processes. This paper is dedicated to perform a thorough evaluation of kernel order influence over convolutional neural networks in the context of traffic signs recognition. Experiments for distinct kernels sizes were performed using the most well accepted database, the socalled German Traffic Sign Recognition Benchmark.
Palavras-chave:
autonomous vehicles, CNNs, kernel size, statistical evaluation
Referências
M. G. Speranza, “Trends in transportation and logistics,” European Journal of Operational Research, vol. 264, no. 3, pp. 830 – 836, 2018. [Online]. Available: http://www.sciencedirect.com/science/article/pii/S0377221716306713
F. Favaró, S. O. Eurich, and N. Nader, “Autonomous vehicles’ disengagements: Trends, triggers, and regulatory limitations.” Accident; analysis and prevention, vol. 110, pp. 136–148, 2018.
O. Abiodun, A. Jantan, O. Omolara, K. Dada, N. Mohamed, and H. Ar- shed, "State-of-the-art in articial neural network applications: A survey," Heliyon, vol. 4, p. e00938, 11 2018.
Y. LeCun, P. Haffner, L. Bottou, and Y. Bengio, Object Recognition Berlin, Heidelberg: Springer Berlin with Gradient-Based Learning. Heidelberg, 1999, pp. 319–345.
Q. Tang and K. Jo, "Analysis of various trafc sign detectors based on deep convolution network," in 2019 IEEE/SICE International Symposium on System Integration (SII), Jan 2019, pp. 507–511.
A. Jain, A. Mishra, A. Shukla, and R. Tiwari, "A novel genetically optimized convolutional neural network for trafc sign recognition: A new benchmark on belgium and chinese trafc sign datasets," Neural Processing Letters, vol. 50, no. 3, pp. 3019–3043, 02 2019.
V. Sichkar and S. Kolyubin, "Effect of various dimension convolutional layer lters on trafc sign classication accuracy," Scientic and Techni- cal Journal of Information Technologies, Mechanics and Optics, vol. 19, pp. 546–552, 06 2019.
J. Stallkamp, M. Schlipsing, J. Salmen, and C. Igel, "The german trafc sign recognition benchmark: A multi-class classication competition," in The 2011 International Joint Conference on Neural Networks, IJCNN 2011, San Jose, California, USA, July 31 August 5, 2011. IEEE, 2011, pp. 1453–1460.
S. Houben, J. Stallkamp, J. Salmen, M. Schlipsing, and C. Igel, "Detection of trafc signs in real-world images: The german trafc sign detection benchmark," in The 2013 International Joint Conference on Neural Networks (IJCNN).IEEE, aug 2013.
R. Timofte, K. Zimmermann, and L. Van Gool, “Multi-view traffic sign detection, recognition, and 3d localisation,” Mach. Vision Appl., vol. 25, no. 3, p. 633–647, Apr. 2014. [Online]. Available: https://doi.org/10.1007/s00138-011-0391-3
S. Segvic, K. Brkic, Z. Kalafatic, V. Stanisavljevic, M. Sevrovic, D. Budimir, and I. Dadic, “A computer vision assisted geoinformation inventory for traffic infrastructure,” in 13th International IEEE Conference on Intelligent Transportation Systems, Sep. 2010, pp. 66–73.
N. Paparoditis, J.-P. Papelard, B. Cannelle, A. Devaux, B. Soheilian, N. David, and E. Houzay, "Stereopolis ii: A multi-purpose and multisensor 3d mobile mapping system for street visualisation and 3d metrology," Revue française de photogrammétrie et de télédétection, vol. 200, no. 1, pp. 69–79, 2012.
C. Grigorescu and N. Petkov, "Distance sets for shape lters and shape recognition," IEEE Transactions on Image Processing, vol. 12, no. 10, pp. 1274–1286, Oct 2003.
F. Larsson and M. Felsberg, "Using fourier descriptors and spatial models for trafc sign recognition," in Scandinavian conference on image analysis. Springer, 2011, pp. 238–249.
H. S. Lee and K. Kim, "Simultaneous trafc sign detection and boundary estimation using convolutional neural network," IEEE Transactions on Intelligent Transportation Systems, vol. 19, no. 5, pp. 1652–1663, may 2018.
W. Liu, D. Anguelov, D. Erhan, C. Szegedy, S. Reed, C.-Y. Fu, and A. C. Berg, "Ssd: Single shot multibox detector," in Computer Vision – ECCV 2016, B. Leibe, J. Matas, N. Sebe, and M. Welling, Eds. Cham: Springer International Publishing, 2016, pp. 21–37.
S. Song, Z. Que, J. Hou, S. Du, and Y. Song, "An efcient convolutional trafc sign detection," Journal of Systems neural network for small Architecture, jan 2019.
T. Yang, X. Long, A. K. Sangaiah, Z. Zheng, and C. Tong, "Deep detec- tion network for real-life trafc sign in vehicular networks," Computer Networks, vol. 136, pp. 95–104, may 2018.
X. Xu, J. Jin, S. Zhang, L. Zhang, S. Pu, and Z. Chen, "Smart data driven trafc sign detection method based on adaptive color threshold and shape symmetry," Future Generation Computer Systems, vol. 94, pp. 381–391, may 2019.
S. Salti, A. Petrelli, F. Tombari, N. Fioraio, and L. D. Stefano, “Traffic sign detection via interest region extraction,” Pattern Recognition, vol. 48, no. 4, pp. 1039–1049, apr 2015.
H. Li, F. Sun, L. Liu, and L. Wang, “A novel traffic sign detection method via color segmentation and robust shape matching,” Neurocomputing, vol. 169, pp. 77–88, dec 2015.
Á. Arcos-García, J. A. Álvarez-García, and L. M. Soria-Morillo, “Deep neural network for traffic sign recognition systems: An analysis of spatial transformers and stochastic optimisation methods,” Neural Networks, vol. 99, pp. 158–165, mar 2018.
A. A. Minai, "2011 international joint conference on neural networks (ijcnn 2011) [conference reports]," IEEE Computational Intelligence Magazine, vol. 7, no. 1, pp. 13–15, Feb 2012.
D. Ciresan, U. Meier, J. Masci, and J. Schmidhuber, "Multi-column deep neural network for trafc sign classication," Neural Networks, vol. 32, pp. 333–338, aug 2012.
P. Sermanet and Y. LeCun, "Trafc sign recognition with multi-scale convolutional networks," in The 2011 International Joint Conference on Neural Networks, 2011, pp. 2809–2813.
S. Aziz, E. A. Mohamed, and F. Youssef, "Trafc sign recognition based on multi-feature fusion and ELM classier," Procedia Computer Science, vol. 127, pp. 146–153, 2018.
S. Kaplan Berkaya, H. Gunduz, O. Ozsen, C. Akinlar, and S. Gunal, "On circular trafc sign detection and recognition," Expert Systems with Applications, vol. 48, 12 2015.
P. H. Kassani and A. B. J. Teoh, "A new sparse model for trafc sign classication using soft histogram of oriented gradients," Applied Soft Computing, vol. 52, pp. 231–246, mar 2017.
J. Pizarro, E. Guerrero, and P. Galindo, "Multiple comparison procedures applied to model selection," Neurocomputing, vol. 48, pp. 155–173, 10 2002.
J. Demsar, "Statistical comparisons of classiers over multiple data sets," The Journal of Machine Learning Research, vol. 7, pp. 1–30, 2006.
T. G. Dietterich, "Approximate statistical tests for comparing supervised classication learning algorithms," Neural computation, vol. 10, no. 7, pp. 1895–1923, 1998.
M. Friedman, "The use of ranks to avoid the assumption of normality implicit in the analysis of variance," Journal of the American Statistical Association, vol. 32, no. 200, pp. 675–701, 1937. [Online]. Available: http://www.jstor.org/stable/2279372
P. Nemenyi, Distribution-free Multiple Comparisons. Princeton University, 1963. [Online]. Available: https://books.google.com.br/books?id=nhDMtgAACAAJ
D. J. Sheskin, Handbook of Parametric and Nonparametric Statistical Procedures, 4th ed. Chapman & Hall/CRC, 2007.
K. Sichkar V. N. (2019) Traffic signs preprocessed data. [Online]. Available: https://www.kaggle.com/valentynsichkar/trafficsigns-preprocessed
M. Friedman, “A comparison of alternative tests of significance for the problem of m rankings,” The Annals of Mathematical Statistics, vol. 11, no. 1, pp. 86–92, 1940. [Online]. Available: http://www.jstor.org/stable/2235971
F. Favaró, S. O. Eurich, and N. Nader, “Autonomous vehicles’ disengagements: Trends, triggers, and regulatory limitations.” Accident; analysis and prevention, vol. 110, pp. 136–148, 2018.
O. Abiodun, A. Jantan, O. Omolara, K. Dada, N. Mohamed, and H. Ar- shed, "State-of-the-art in articial neural network applications: A survey," Heliyon, vol. 4, p. e00938, 11 2018.
Y. LeCun, P. Haffner, L. Bottou, and Y. Bengio, Object Recognition Berlin, Heidelberg: Springer Berlin with Gradient-Based Learning. Heidelberg, 1999, pp. 319–345.
Q. Tang and K. Jo, "Analysis of various trafc sign detectors based on deep convolution network," in 2019 IEEE/SICE International Symposium on System Integration (SII), Jan 2019, pp. 507–511.
A. Jain, A. Mishra, A. Shukla, and R. Tiwari, "A novel genetically optimized convolutional neural network for trafc sign recognition: A new benchmark on belgium and chinese trafc sign datasets," Neural Processing Letters, vol. 50, no. 3, pp. 3019–3043, 02 2019.
V. Sichkar and S. Kolyubin, "Effect of various dimension convolutional layer lters on trafc sign classication accuracy," Scientic and Techni- cal Journal of Information Technologies, Mechanics and Optics, vol. 19, pp. 546–552, 06 2019.
J. Stallkamp, M. Schlipsing, J. Salmen, and C. Igel, "The german trafc sign recognition benchmark: A multi-class classication competition," in The 2011 International Joint Conference on Neural Networks, IJCNN 2011, San Jose, California, USA, July 31 August 5, 2011. IEEE, 2011, pp. 1453–1460.
S. Houben, J. Stallkamp, J. Salmen, M. Schlipsing, and C. Igel, "Detection of trafc signs in real-world images: The german trafc sign detection benchmark," in The 2013 International Joint Conference on Neural Networks (IJCNN).IEEE, aug 2013.
R. Timofte, K. Zimmermann, and L. Van Gool, “Multi-view traffic sign detection, recognition, and 3d localisation,” Mach. Vision Appl., vol. 25, no. 3, p. 633–647, Apr. 2014. [Online]. Available: https://doi.org/10.1007/s00138-011-0391-3
S. Segvic, K. Brkic, Z. Kalafatic, V. Stanisavljevic, M. Sevrovic, D. Budimir, and I. Dadic, “A computer vision assisted geoinformation inventory for traffic infrastructure,” in 13th International IEEE Conference on Intelligent Transportation Systems, Sep. 2010, pp. 66–73.
N. Paparoditis, J.-P. Papelard, B. Cannelle, A. Devaux, B. Soheilian, N. David, and E. Houzay, "Stereopolis ii: A multi-purpose and multisensor 3d mobile mapping system for street visualisation and 3d metrology," Revue française de photogrammétrie et de télédétection, vol. 200, no. 1, pp. 69–79, 2012.
C. Grigorescu and N. Petkov, "Distance sets for shape lters and shape recognition," IEEE Transactions on Image Processing, vol. 12, no. 10, pp. 1274–1286, Oct 2003.
F. Larsson and M. Felsberg, "Using fourier descriptors and spatial models for trafc sign recognition," in Scandinavian conference on image analysis. Springer, 2011, pp. 238–249.
H. S. Lee and K. Kim, "Simultaneous trafc sign detection and boundary estimation using convolutional neural network," IEEE Transactions on Intelligent Transportation Systems, vol. 19, no. 5, pp. 1652–1663, may 2018.
W. Liu, D. Anguelov, D. Erhan, C. Szegedy, S. Reed, C.-Y. Fu, and A. C. Berg, "Ssd: Single shot multibox detector," in Computer Vision – ECCV 2016, B. Leibe, J. Matas, N. Sebe, and M. Welling, Eds. Cham: Springer International Publishing, 2016, pp. 21–37.
S. Song, Z. Que, J. Hou, S. Du, and Y. Song, "An efcient convolutional trafc sign detection," Journal of Systems neural network for small Architecture, jan 2019.
T. Yang, X. Long, A. K. Sangaiah, Z. Zheng, and C. Tong, "Deep detec- tion network for real-life trafc sign in vehicular networks," Computer Networks, vol. 136, pp. 95–104, may 2018.
X. Xu, J. Jin, S. Zhang, L. Zhang, S. Pu, and Z. Chen, "Smart data driven trafc sign detection method based on adaptive color threshold and shape symmetry," Future Generation Computer Systems, vol. 94, pp. 381–391, may 2019.
S. Salti, A. Petrelli, F. Tombari, N. Fioraio, and L. D. Stefano, “Traffic sign detection via interest region extraction,” Pattern Recognition, vol. 48, no. 4, pp. 1039–1049, apr 2015.
H. Li, F. Sun, L. Liu, and L. Wang, “A novel traffic sign detection method via color segmentation and robust shape matching,” Neurocomputing, vol. 169, pp. 77–88, dec 2015.
Á. Arcos-García, J. A. Álvarez-García, and L. M. Soria-Morillo, “Deep neural network for traffic sign recognition systems: An analysis of spatial transformers and stochastic optimisation methods,” Neural Networks, vol. 99, pp. 158–165, mar 2018.
A. A. Minai, "2011 international joint conference on neural networks (ijcnn 2011) [conference reports]," IEEE Computational Intelligence Magazine, vol. 7, no. 1, pp. 13–15, Feb 2012.
D. Ciresan, U. Meier, J. Masci, and J. Schmidhuber, "Multi-column deep neural network for trafc sign classication," Neural Networks, vol. 32, pp. 333–338, aug 2012.
P. Sermanet and Y. LeCun, "Trafc sign recognition with multi-scale convolutional networks," in The 2011 International Joint Conference on Neural Networks, 2011, pp. 2809–2813.
S. Aziz, E. A. Mohamed, and F. Youssef, "Trafc sign recognition based on multi-feature fusion and ELM classier," Procedia Computer Science, vol. 127, pp. 146–153, 2018.
S. Kaplan Berkaya, H. Gunduz, O. Ozsen, C. Akinlar, and S. Gunal, "On circular trafc sign detection and recognition," Expert Systems with Applications, vol. 48, 12 2015.
P. H. Kassani and A. B. J. Teoh, "A new sparse model for trafc sign classication using soft histogram of oriented gradients," Applied Soft Computing, vol. 52, pp. 231–246, mar 2017.
J. Pizarro, E. Guerrero, and P. Galindo, "Multiple comparison procedures applied to model selection," Neurocomputing, vol. 48, pp. 155–173, 10 2002.
J. Demsar, "Statistical comparisons of classiers over multiple data sets," The Journal of Machine Learning Research, vol. 7, pp. 1–30, 2006.
T. G. Dietterich, "Approximate statistical tests for comparing supervised classication learning algorithms," Neural computation, vol. 10, no. 7, pp. 1895–1923, 1998.
M. Friedman, "The use of ranks to avoid the assumption of normality implicit in the analysis of variance," Journal of the American Statistical Association, vol. 32, no. 200, pp. 675–701, 1937. [Online]. Available: http://www.jstor.org/stable/2279372
P. Nemenyi, Distribution-free Multiple Comparisons. Princeton University, 1963. [Online]. Available: https://books.google.com.br/books?id=nhDMtgAACAAJ
D. J. Sheskin, Handbook of Parametric and Nonparametric Statistical Procedures, 4th ed. Chapman & Hall/CRC, 2007.
K. Sichkar V. N. (2019) Traffic signs preprocessed data. [Online]. Available: https://www.kaggle.com/valentynsichkar/trafficsigns-preprocessed
M. Friedman, “A comparison of alternative tests of significance for the problem of m rankings,” The Annals of Mathematical Statistics, vol. 11, no. 1, pp. 86–92, 1940. [Online]. Available: http://www.jstor.org/stable/2235971
Publicado
07/10/2020
Como Citar
DE OLIVEIRA, Lucas; MOTA, Guilherme; VIDAL, Vitor.
A Thorough Evaluation of Kernel Order in CNN Based Traffic Signs Recognition. In: WORKSHOP DE VISÃO COMPUTACIONAL (WVC), 16. , 2020, Evento Online.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2020
.
p. 81-86.
DOI: https://doi.org/10.5753/wvc.2020.13485.
