Improving the performance of a SVM+HOG classifier for detection and tracking of wagon components by using geometric constraints
Resumo
The inspection of train and railway components that can cause derailment plays a key role in rail maintenance. To improve productivity and safety, service providers look for automatic and reliable inspection solutions. Although automatic inspection based on computer vision is a standard concept, such an application challenges development community due to the environmental and logistic factors to be considered. Previous publications presented automatic classifiers to evaluate integrity and placement of wagon components. Although the high classification accuracy reported, ineffective object detection affected the general performance. Our object detector/tracker consists of a descriptor based on the histogram of oriented gradients, a support vector machine classifier, and a set of geometric constraints, which takes in account the ideal trajectory path of the wagon’s components of interest and the distances between them. We detail training and validation procedures, together with the metrics used to assess the performance of the system. Presented results compare two other techniques with our approach, which exhibits a fair trade-off between reliability and computational complexity for the application of wagon component detection.
Palavras-chave:
Computer vision, Object tracking, Object detection, Wagon inspection system
Referências
A. M. A. Talab, Z. Huang, F. Xi, and L. HaiMing, “Detection crack in image using Otsu method and multiple filtering in image processing techniques,” Optik, vol. 127, no. 3, pp. 1030–1033, feb 2016. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S0030402615012164.
S. Iyer and S. K. Sinha, “A robust approach for automatic detection and segmentation of cracks in underground pipeline images,” Image and Vision Computing, vol. 23, no. 10, pp. 921–933, sep 2005. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S0262885605000764
S. Alam, A. Loukili, F. Grondin, and E. Rozi`ere, “Use of the digital image correlation and acoustic emission technique to study the effect of structural size on cracking of reinforced concrete,” Engineering Fracture Mechanics, vol. 143, pp. 17–31, jul 2015. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S0013794415003045
L. M. Bergasa, D. Almer´ıa, J. Almaz´an, J. J. Yebes, and R. Arroyo, “DriveSafe: An app for alerting inattentive drivers and scoring driving behaviors,” in 2014 IEEE Intelligent Vehicles Symposium Proceedings, 2014, pp. 240–245.
A. Bender, G. Agamennoni, J. R. Ward, S. Worrall, and E. M. Nebot, “An Unsupervised Approach for Inferring Driver Behavior From Naturalistic Driving Data,” IEEE Transactions on Intelligent Transportation Systems, vol. 16, no. 6, pp. 3325–3336, 2015.
K. Seshadri, F. Juefei-Xu, D. K. Pal, M. Savvides, and C. P. Thor, “Driver cell phone usage detection on Strategic Highway Research Program (SHRP2) face view videos,” in 2015 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), 2015, pp. 35–43.
Y. Lu, X. Xu, and J. Xu, “Development of a Hybrid Manufacturing Cloud,” Journal of Manufacturing Systems, vol. 33, no. 4, pp. 551–566, oct 2014. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S027861251400048X
L. Wang, M. T¨orngren, and M. Onori, “Current status and advancement of cyber-physical systems in manufacturing,” Journal of Manufacturing Systems, vol. 37, pp. 517–527, oct 2015. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S0278612515000400
D. Wu, D. W. Rosen, L. Wang, and D. Schaefer, “Cloud-based design and manufacturing: A new paradigm in digital manufacturing and design innovation,” Computer-Aided Design, vol. 59, pp. 1–14, feb 2015. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S0010448514001560
K. Kapach, E. Barnea, R. Mairon, Y. Edan, and O. Shahar, “Computer vision for fruit harvesting robots—State of the art and challenges ahead,” International Journal of Computational Vision and Robotics, vol. 3, pp. 4–34, 2012.
H. Zareiforoush, S. Minaei, M. R. Alizadeh, and A. Banakar, “Potential Applications of Computer Vision in Quality Inspection of Rice: A Review,” Food Engineering Reviews, vol. 7, no. 3, pp. 321–345, 2015. [Online]. Available: https://doi.org/10.1007/s12393-014-9101-z
J. G. Arnal Barbedo, “Digital image processing techniques for detecting, quantifying and classifying plant diseases,” SpringerPlus, vol. 2, no. 1, p. 660, 2013. [Online]. Available: https://doi.org/10.1186/2193-1801-2-660
E. Resendiz, J. M. Hart, and N. Ahuja, “Automated Visual Inspection of Railroad Tracks,” IEEE Transactions on Intelligent Transportation Systems, vol. 14, no. 2, pp. 751–760, 2013.
I. Tang and T. P. Breckon, “Automatic Road Environment Classification,” IEEE Transactions on Intelligent Transportation Systems, vol. 12, no. 2, pp. 476–484, 2011.
M. Singh, S. Singh, J. Jaiswal, and J. Hempshall, “Autonomous Rail Track Inspection using Vision Based System,” in 2006 IEEE International Conference on Computational Intelligence for Homeland Security and Personal Safety, 2006, pp. 56–59.
R. L. Rocha, A. C. Q. Siravenha, A. C. S. Gomes, G. L. Serejo, A. F. B. Silva, L. M. Rodrigues, J. Braga, G. Dias, S. R. Carvalho, and C. R. B. de Souza, “A Deep-learning-based Approach for Automated Wagon Component Inspection,” in Proceedings of the 33rd Annual ACM Symposium on Applied Computing, ser. SAC ’18. New York, NY, USA: ACM, 2018, pp. 276–283. [Online]. Available: http://doi.acm.org/10.1145/3167132.3167157
E. Fernandes, R. L. Rocha, B. Ferreira, E. Carvalho, A. C. Siravenha, A. C. S. Gomes, S. Carvalho, and C. R. B. de Souza, “An Ensemble of Convolutional Neural Networks for Unbalanced Datasets: A case Study with Wagon Component Inspection,” in 2018 International Joint Conference on Neural Networks (IJCNN), 2018, pp. 1–6.
R. Rocha, A. Siravenha, A. C. S. Gomes, G. L. Serejo, A. Silva, L. Mousinho Rodrigues, J. Braga, G. Dias, and S. Carvalho, “Avaliaçãoo de técnicas de Deep Learning aplicadas à identificacão de peças defeituosas em vagões de trem,” in Workshop of Industry Applications (WIA) in the 30th Conference on Graphics, Patterns and Images (SIBGRAPI’17), E. Clua and F. L. C. P´adua, Eds., Niterói, RJ, Brazil, 2017. [Online]. Available: http://sibgrapi2017.ic.uff.br/eproceedings/assets/papers/WIA/WIA6.pdf
N. Dalal and B. Triggs, “Histograms of oriented gradients for human detection,” in 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’05), vol. 1, 2005, pp. 886–893
Y. LeCun, P. Haffner, L. Bottou, and Y. Bengio, “Object Recognition with Gradient-Based Learning,” in Shape, Contour and Grouping in Computer Vision. London, UK, UK: Springer-Verlag, 1999, pp. 319. [Online]. Available: http://dl.acm.org/citation.cfm?id=646469.691875
C. Cortes and V. Vapnik, “Support-Vector Networks,” Machine Learning, vol. 20, no. 3, pp. 273–297, 1995. [Online]. Available: https://doi.org/10.1007/BF00994018
P. Viola and M. Jones, “Rapid object detection using a boosted cascade of simple features,” in Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001, vol. 1, pp. I-511 - I-518, 2001.
R. B. Girshick, J. Donahue, T. Darrell, and J. Malik, “Rich feature hierarchies for accurate object detection and semantic segmentation,” CoRR, vol. abs/1311.2, 2013. [Online]. Available: http://arxiv.org/abs/1311.2524
R. B. Girshick, “Fast R-CNN,” CoRR, vol. abs/1504.0, 2015. [Online]. Available: http://arxiv.org/abs/1504.08083
S. Ren, K. He, R. B. Girshick, and J. Sun, “Faster RCNN: Towards Real-Time Object Detection with Region Proposal Networks,” CoRR, vol. abs/1506.0, 2015. [Online]. Available: http://arxiv.org/abs/1506.01497
J. Redmon, S. K. Divvala, R. B. Girshick, and A. Farhadi, “You Only Look Once: Unified, Real-Time Object Detection,” CoRR, vol. abs/1506.0, 2015. [Online]. Available: http://arxiv.org/abs/1506.02640
J. Redmon and A. Farhadi, “YOLO9000: Better, Faster, Stronger,” CoRR, vol. abs/1612.0, 2016. [Online]. Available: http://arxiv.org/abs/1612.08242
J. Redmon and A. Farhadi,“YOLOv3: An Incremental Improvement,” CoRR, vol. abs/1804.0, 2018. [Online]. Available: http://arxiv.org/abs/1804.02767
W. Liu, D. Anguelov, D. Erhan, C. Szegedy, S. E. Reed, C.-Y. Fu, and A. C. Berg, “SSD: Single Shot MultiBox Detector,” CoRR, vol. abs/1512.0, 2015. [Online]. Available: http://arxiv.org/abs/1512.02325
R. Kohavi, “A Study of Cross-Validation and Bootstrap for Accuracy Estimation and Model Selection.” Morgan Kaufmann, 1995, pp. 1137–1143.
J. F. Henriques, R. Caseiro, P. Martins, and J. Batista, “High-Speed Tracking with Kernelized Correlation Filters,” CoRR, vol. abs/1404.7, 2014. [Online]. Available: http://arxiv.org/abs/1404.7584
F. Suard, A. Rakotomamonjy, A. Bensrhair, and A. Broggi, “Pedestrian Detection using Infrared images and Histograms of Oriented Gradients,” in 2006 IEEE Intelligent Vehicles Symposium, 2006, pp. 206–212.
T. Kutschbach, E. Bochinski, V. Eiselein, and T. Sikora, “Sequential sensor fusion combining probability hypothesis density and kernelized correlation filters for multi-object tracking in video data,” in 2017 14th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS), 2017, pp. 1–5.
R. Xu, S. Y. Nikouei, Y. Chen, A. Polunchenko, S. Song, C. Deng, and T. R. Faughnan, “Real-Time Human Objects Tracking for Smart Surveillance at the Edge,” in 2018 IEEE International Conference on Communications (ICC), 2018, pp. 1–6.
H. Cheng, L. Lin, Z. Zheng, Y. Guan, and Z. Liu, “An autonomous vision-based target tracking system for rotorcraft unmanned aerial vehicles,” in 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2017, pp. 1732–1738.
J. Greenhalgh and M. Mirmehdi, “Real-Time Detection and Recognition of Road Traffic Signs,” IEEE Transactions on Intelligent Transportation Systems, vol. 13, no. 4, pp. 1498–1506, 2012.
D. Held, S. Thrun, and S. Savarese, “Learning to Track at 100 FPS with Deep Regression Networks,” in Computer Vision – ECCV 2016, B. Leibe, J. Matas, N. Sebe, and M. Welling, Eds. Cham: Springer International Publishing, 2016, pp. 749–765.
S. Iyer and S. K. Sinha, “A robust approach for automatic detection and segmentation of cracks in underground pipeline images,” Image and Vision Computing, vol. 23, no. 10, pp. 921–933, sep 2005. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S0262885605000764
S. Alam, A. Loukili, F. Grondin, and E. Rozi`ere, “Use of the digital image correlation and acoustic emission technique to study the effect of structural size on cracking of reinforced concrete,” Engineering Fracture Mechanics, vol. 143, pp. 17–31, jul 2015. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S0013794415003045
L. M. Bergasa, D. Almer´ıa, J. Almaz´an, J. J. Yebes, and R. Arroyo, “DriveSafe: An app for alerting inattentive drivers and scoring driving behaviors,” in 2014 IEEE Intelligent Vehicles Symposium Proceedings, 2014, pp. 240–245.
A. Bender, G. Agamennoni, J. R. Ward, S. Worrall, and E. M. Nebot, “An Unsupervised Approach for Inferring Driver Behavior From Naturalistic Driving Data,” IEEE Transactions on Intelligent Transportation Systems, vol. 16, no. 6, pp. 3325–3336, 2015.
K. Seshadri, F. Juefei-Xu, D. K. Pal, M. Savvides, and C. P. Thor, “Driver cell phone usage detection on Strategic Highway Research Program (SHRP2) face view videos,” in 2015 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), 2015, pp. 35–43.
Y. Lu, X. Xu, and J. Xu, “Development of a Hybrid Manufacturing Cloud,” Journal of Manufacturing Systems, vol. 33, no. 4, pp. 551–566, oct 2014. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S027861251400048X
L. Wang, M. T¨orngren, and M. Onori, “Current status and advancement of cyber-physical systems in manufacturing,” Journal of Manufacturing Systems, vol. 37, pp. 517–527, oct 2015. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S0278612515000400
D. Wu, D. W. Rosen, L. Wang, and D. Schaefer, “Cloud-based design and manufacturing: A new paradigm in digital manufacturing and design innovation,” Computer-Aided Design, vol. 59, pp. 1–14, feb 2015. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S0010448514001560
K. Kapach, E. Barnea, R. Mairon, Y. Edan, and O. Shahar, “Computer vision for fruit harvesting robots—State of the art and challenges ahead,” International Journal of Computational Vision and Robotics, vol. 3, pp. 4–34, 2012.
H. Zareiforoush, S. Minaei, M. R. Alizadeh, and A. Banakar, “Potential Applications of Computer Vision in Quality Inspection of Rice: A Review,” Food Engineering Reviews, vol. 7, no. 3, pp. 321–345, 2015. [Online]. Available: https://doi.org/10.1007/s12393-014-9101-z
J. G. Arnal Barbedo, “Digital image processing techniques for detecting, quantifying and classifying plant diseases,” SpringerPlus, vol. 2, no. 1, p. 660, 2013. [Online]. Available: https://doi.org/10.1186/2193-1801-2-660
E. Resendiz, J. M. Hart, and N. Ahuja, “Automated Visual Inspection of Railroad Tracks,” IEEE Transactions on Intelligent Transportation Systems, vol. 14, no. 2, pp. 751–760, 2013.
I. Tang and T. P. Breckon, “Automatic Road Environment Classification,” IEEE Transactions on Intelligent Transportation Systems, vol. 12, no. 2, pp. 476–484, 2011.
M. Singh, S. Singh, J. Jaiswal, and J. Hempshall, “Autonomous Rail Track Inspection using Vision Based System,” in 2006 IEEE International Conference on Computational Intelligence for Homeland Security and Personal Safety, 2006, pp. 56–59.
R. L. Rocha, A. C. Q. Siravenha, A. C. S. Gomes, G. L. Serejo, A. F. B. Silva, L. M. Rodrigues, J. Braga, G. Dias, S. R. Carvalho, and C. R. B. de Souza, “A Deep-learning-based Approach for Automated Wagon Component Inspection,” in Proceedings of the 33rd Annual ACM Symposium on Applied Computing, ser. SAC ’18. New York, NY, USA: ACM, 2018, pp. 276–283. [Online]. Available: http://doi.acm.org/10.1145/3167132.3167157
E. Fernandes, R. L. Rocha, B. Ferreira, E. Carvalho, A. C. Siravenha, A. C. S. Gomes, S. Carvalho, and C. R. B. de Souza, “An Ensemble of Convolutional Neural Networks for Unbalanced Datasets: A case Study with Wagon Component Inspection,” in 2018 International Joint Conference on Neural Networks (IJCNN), 2018, pp. 1–6.
R. Rocha, A. Siravenha, A. C. S. Gomes, G. L. Serejo, A. Silva, L. Mousinho Rodrigues, J. Braga, G. Dias, and S. Carvalho, “Avaliaçãoo de técnicas de Deep Learning aplicadas à identificacão de peças defeituosas em vagões de trem,” in Workshop of Industry Applications (WIA) in the 30th Conference on Graphics, Patterns and Images (SIBGRAPI’17), E. Clua and F. L. C. P´adua, Eds., Niterói, RJ, Brazil, 2017. [Online]. Available: http://sibgrapi2017.ic.uff.br/eproceedings/assets/papers/WIA/WIA6.pdf
N. Dalal and B. Triggs, “Histograms of oriented gradients for human detection,” in 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’05), vol. 1, 2005, pp. 886–893
Y. LeCun, P. Haffner, L. Bottou, and Y. Bengio, “Object Recognition with Gradient-Based Learning,” in Shape, Contour and Grouping in Computer Vision. London, UK, UK: Springer-Verlag, 1999, pp. 319. [Online]. Available: http://dl.acm.org/citation.cfm?id=646469.691875
C. Cortes and V. Vapnik, “Support-Vector Networks,” Machine Learning, vol. 20, no. 3, pp. 273–297, 1995. [Online]. Available: https://doi.org/10.1007/BF00994018
P. Viola and M. Jones, “Rapid object detection using a boosted cascade of simple features,” in Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001, vol. 1, pp. I-511 - I-518, 2001.
R. B. Girshick, J. Donahue, T. Darrell, and J. Malik, “Rich feature hierarchies for accurate object detection and semantic segmentation,” CoRR, vol. abs/1311.2, 2013. [Online]. Available: http://arxiv.org/abs/1311.2524
R. B. Girshick, “Fast R-CNN,” CoRR, vol. abs/1504.0, 2015. [Online]. Available: http://arxiv.org/abs/1504.08083
S. Ren, K. He, R. B. Girshick, and J. Sun, “Faster RCNN: Towards Real-Time Object Detection with Region Proposal Networks,” CoRR, vol. abs/1506.0, 2015. [Online]. Available: http://arxiv.org/abs/1506.01497
J. Redmon, S. K. Divvala, R. B. Girshick, and A. Farhadi, “You Only Look Once: Unified, Real-Time Object Detection,” CoRR, vol. abs/1506.0, 2015. [Online]. Available: http://arxiv.org/abs/1506.02640
J. Redmon and A. Farhadi, “YOLO9000: Better, Faster, Stronger,” CoRR, vol. abs/1612.0, 2016. [Online]. Available: http://arxiv.org/abs/1612.08242
J. Redmon and A. Farhadi,“YOLOv3: An Incremental Improvement,” CoRR, vol. abs/1804.0, 2018. [Online]. Available: http://arxiv.org/abs/1804.02767
W. Liu, D. Anguelov, D. Erhan, C. Szegedy, S. E. Reed, C.-Y. Fu, and A. C. Berg, “SSD: Single Shot MultiBox Detector,” CoRR, vol. abs/1512.0, 2015. [Online]. Available: http://arxiv.org/abs/1512.02325
R. Kohavi, “A Study of Cross-Validation and Bootstrap for Accuracy Estimation and Model Selection.” Morgan Kaufmann, 1995, pp. 1137–1143.
J. F. Henriques, R. Caseiro, P. Martins, and J. Batista, “High-Speed Tracking with Kernelized Correlation Filters,” CoRR, vol. abs/1404.7, 2014. [Online]. Available: http://arxiv.org/abs/1404.7584
F. Suard, A. Rakotomamonjy, A. Bensrhair, and A. Broggi, “Pedestrian Detection using Infrared images and Histograms of Oriented Gradients,” in 2006 IEEE Intelligent Vehicles Symposium, 2006, pp. 206–212.
T. Kutschbach, E. Bochinski, V. Eiselein, and T. Sikora, “Sequential sensor fusion combining probability hypothesis density and kernelized correlation filters for multi-object tracking in video data,” in 2017 14th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS), 2017, pp. 1–5.
R. Xu, S. Y. Nikouei, Y. Chen, A. Polunchenko, S. Song, C. Deng, and T. R. Faughnan, “Real-Time Human Objects Tracking for Smart Surveillance at the Edge,” in 2018 IEEE International Conference on Communications (ICC), 2018, pp. 1–6.
H. Cheng, L. Lin, Z. Zheng, Y. Guan, and Z. Liu, “An autonomous vision-based target tracking system for rotorcraft unmanned aerial vehicles,” in 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2017, pp. 1732–1738.
J. Greenhalgh and M. Mirmehdi, “Real-Time Detection and Recognition of Road Traffic Signs,” IEEE Transactions on Intelligent Transportation Systems, vol. 13, no. 4, pp. 1498–1506, 2012.
D. Held, S. Thrun, and S. Savarese, “Learning to Track at 100 FPS with Deep Regression Networks,” in Computer Vision – ECCV 2016, B. Leibe, J. Matas, N. Sebe, and M. Welling, Eds. Cham: Springer International Publishing, 2016, pp. 749–765.
Publicado
28/10/2019
Como Citar
GONÇALVES, Camilo Lélis A. et al.
Improving the performance of a SVM+HOG classifier for detection and tracking of wagon components by using geometric constraints. In: WORKSHOP DE APLICAÇÕES INDUSTRIAIS - CONFERENCE ON GRAPHICS, PATTERNS AND IMAGES (SIBGRAPI), 32. , 2019, Rio de Janeiro.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2019
.
p. 230-236.
DOI: https://doi.org/10.5753/sibgrapi.est.2019.8336.
