An Implementation Based on Mask R-CNN for Insect Detection in Digital Images
Abstract
The manual task of counting and identifying small insects, such as aphids and parasitoids, captured in yellow traps on the field, is an exhaustive, time-consuming, and non-scalable activity in agricultural research centers. The tasks involve complexity in the screening of the elements of interest and the use of magnifying glasses and microscopes. Recent advances in artificial intelligence and high-performance computing have enabled the development of efficient computer vision solutions for monitoring pests and identifying diseases in plants. In this context, this paper presents a routine for automatic counting and identification of insects in images, scanned from samples captured by the traps in the Embrapa Wheat experimental sites. For the implementation a small data set was used, image processing techniques and the convolutional two-stage Mask R-CNN approach were applied. The preliminary results indicate a mean accuracy (mAP) of 60.4% and show some details to increase the efficiency of the proposed method.References
L. C. Wright and W. W. Cone, "Population Dynamics of Brachycorynella asparagi (Homoptera: Aphididae) on Undisturbed Asparagus in Washington State," Environmental Entomology, vol. 17, no. 5, pp. 878– 886, 10 1988.
P. R. V. d. S. Pereira, J. R. Salvadori, and D. Lau, "Identificação de adultos ápteros e alados das principais espécies de afídeos (hemiptera: Aphididae) associadas acereais de inverno no brasil," Embrapa Trigo, Comunicado Técnico online, 258, Tech. Rep., 2009. [Online]. Available: http://www.cnpt.embrapa.br/biblio/co/p co258.htm
E. A. Lins, J. P. M. Rodriguez, S. I. Scoloski, J. Pivato, M. B. Lima, J. M. C. Fernandes, P. R. V. da Silva Pereira, D. Lau, and R. Rieder, "A method for counting and classifying aphids using computer vision," Computers and Electronics in Agriculture, vol. 169, p. 105200, 2020.
A. Picon, A. Alvarez-Gila, M. Seitz, A. Ortiz-Barredo, J. Echazarra, and A. Johannes, "Deep convolutional neural networks for mobile capture device-based crop disease classification in the wild," Computers and Electronics in Agriculture, 2018. [Online]. Available: https: //doi.org/10.1016/j.compag.2018.04.002
L. Liu, R. Wang, C. Xie, P. Yang, F. Wang, S. Sudirman, and W. Liu, "PestNet: An End-to-End Deep Learning Approach for Large-Scale Multi-Class Pest Detection and Classification," IEEE Access, vol. 7, pp. 45 301–45 312, 2019. [Online]. Available: https://doi.org/10.1109/ACCESS.2019.2909522
M. C. Bakkay, S. Chambon, H. A. Rashwan, C. Lubat, and S. Barsotti, "Automatic detection of individual and touching moths from trap images by combining contour-based and region-based segmentation," IET Computer Vision, vol. 12, no. 2, pp. 138–145, 2018. [Online]. Available: https://doi.org/10.1049/iet-cvi.2017.0086.
V. Partel, L. Nunes, P. Stansly, and Y. Ampatzidis, "Automated visionbased system for monitoring Asian citrus psyllid in orchards utilizing artificial intelligence," Computers and Electronics in Agriculture, vol. 162, pp. 328–336, jul 2019.
Y. Sun, X. Liu, M. Yuan, L. Ren, J. Wang, and Z. Chen, "Automatic in-trap pest detection using learning for pheromone-based Dendroctonus valens monitoring," Biosystems Engineering, vol. 176, pp. 140–150, dec 2018.
A. Kamilaris and F. X. Prenafeta-Boldu, "Deep learning in agriculture: A survey," Computers and Electronics in Agriculture, vol. 147, pp. 70–90, 2018. [Online]. Available: https://doi.org/10.1016/j.compag.2018.02.016.
Y. Zhong, J. Gao, Q. Lei, and Y. Zhou, "A vision-based counting and recognition system for flying insects in intelligent agriculture," Sensors, vol. 18, no. 5, p. 1489, 2018.
W. Li, P. Chen, B. Wang, and C. Xie, "Automatic Localization and Count of Agricultural Crop Pests Based on an Improved Deep Learning Pipeline," Scientific Reports, vol. 9, no. 1, 2019.
A. Fuentes, S. Yoon, S. C. Kim, and D. S. Park, "A robust deep-learningbased detector for real-time tomato plant diseases and pests recognition," Sensors (Switzerland), vol. 17, no. 9, 2017.
J. Chen, Y. Fan, T. Wang, C. Zhang, Z. Qiu, and Y. He, "Automatic Segmentation and Counting of Aphid Nymphs on Leaves Using Convolutional Neural Networks," Agronomy, vol. 8, no. 8, 2018.
A. Krizhevsky, I. Sutskever, and G. Hinton, "Imagenet classification with deep convolutional neural networks," Neural Information Processing Systems, vol. 25, 01 2012.
S. Ren, K. He, R. Girshick, and J. Sun, "Faster r-cnn: Towards real-time object detection with region proposal networks," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 39, no. 6, pp. 1137– 1149, 2016.
J. Long, E. Shelhamer, and T. Darrell, "Fully convolutional networks for semantic segmentation," in 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015, pp. 3431–3440.
P. Fischer, A. Dosovitskiy, and T. Brox, "Descriptor matching with convolutional neural networks: a comparison to SIFT," CoRR, vol. abs/1405.5769, 2014, withdrawn. [Online]. Available: http: //arxiv.org/abs/1405.5769.
Y. LeCun, Y. Bengio, and G. Hinton, "Deep learning," Nature, vol. 521, no. 7553, pp. 436–444, 2015.
J. Dai, Y. Li, K. He, and J. Sun, "R-fcn: Object detection via region-based fully convolutional networks," in Proceedings of the 30th International Conference on Neural Information Processing Systems, ser. NIPS’16. Red Hook, NY, USA: Curran Associates Inc., 2016, p. 379–387. [Online]. Available: https://dl.acm.org/doi/10.5555/3157096. 3157139.
K. He, G. Gkioxari, P. Dollar, and R. Girshick, "Mask r-cnn," in 2017 IEEE International Conference on Computer Vision (ICCV), 2017, pp. 2980–2988. [Online]. Available: https://doi.org/10.1109/ICCV.2017.322.
W. Liu, D. Anguelov, D. Erhan, C. Szegedy, S. Reed, C.-Y. Fu, and A. C. Berg, "SSD: Single Shot MultiBox Detector," in Computer Vision – ECCV 2016, B. Leibe, J. Matas, N. Sebe, and M. Welling, Eds. Cham: Springer International Publishing, 2016, pp. 21–37.
J. Redmon, S. Divvala, R. Girshick, and A. Farhadi, "You only look once: Unified, real-time object detection," in 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016, pp. 779–788. [Online]. Available: https://doi.org/10.1109/CVPR.2016.91.
T. Lin, P. Goyal, R. Girshick, K. He, and P. Dollar, "Focal loss for ´ dense object detection," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 42, no. 2, pp. 318–327, 2018. [Online]. Available: https://doi.org/10.1109/TPAMI.2018.2858826.
A. Nazri, N. Mazlan, and F. Muharam, "PENYEK: Automated brown planthopper detection from imperfect sticky pad images using deep convolutional neural network," PLOS ONE, vol. 13, no. 12, p. e0208501, dec 2018. [Online]. Available: http://dx.plos.org/10.1371/ journal.pone.0208501.
W. Abdulla, "Mask r-cnn for object detection and instance segmentation on keras and tensorflow," 2017. [Online]. Available: https://github.com/ matterport/Mask RCNN
P. R. V. d. S. Pereira, J. R. Salvadori, and D. Lau, "Identificação de adultos ápteros e alados das principais espécies de afídeos (hemiptera: Aphididae) associadas acereais de inverno no brasil," Embrapa Trigo, Comunicado Técnico online, 258, Tech. Rep., 2009. [Online]. Available: http://www.cnpt.embrapa.br/biblio/co/p co258.htm
E. A. Lins, J. P. M. Rodriguez, S. I. Scoloski, J. Pivato, M. B. Lima, J. M. C. Fernandes, P. R. V. da Silva Pereira, D. Lau, and R. Rieder, "A method for counting and classifying aphids using computer vision," Computers and Electronics in Agriculture, vol. 169, p. 105200, 2020.
A. Picon, A. Alvarez-Gila, M. Seitz, A. Ortiz-Barredo, J. Echazarra, and A. Johannes, "Deep convolutional neural networks for mobile capture device-based crop disease classification in the wild," Computers and Electronics in Agriculture, 2018. [Online]. Available: https: //doi.org/10.1016/j.compag.2018.04.002
L. Liu, R. Wang, C. Xie, P. Yang, F. Wang, S. Sudirman, and W. Liu, "PestNet: An End-to-End Deep Learning Approach for Large-Scale Multi-Class Pest Detection and Classification," IEEE Access, vol. 7, pp. 45 301–45 312, 2019. [Online]. Available: https://doi.org/10.1109/ACCESS.2019.2909522
M. C. Bakkay, S. Chambon, H. A. Rashwan, C. Lubat, and S. Barsotti, "Automatic detection of individual and touching moths from trap images by combining contour-based and region-based segmentation," IET Computer Vision, vol. 12, no. 2, pp. 138–145, 2018. [Online]. Available: https://doi.org/10.1049/iet-cvi.2017.0086.
V. Partel, L. Nunes, P. Stansly, and Y. Ampatzidis, "Automated visionbased system for monitoring Asian citrus psyllid in orchards utilizing artificial intelligence," Computers and Electronics in Agriculture, vol. 162, pp. 328–336, jul 2019.
Y. Sun, X. Liu, M. Yuan, L. Ren, J. Wang, and Z. Chen, "Automatic in-trap pest detection using learning for pheromone-based Dendroctonus valens monitoring," Biosystems Engineering, vol. 176, pp. 140–150, dec 2018.
A. Kamilaris and F. X. Prenafeta-Boldu, "Deep learning in agriculture: A survey," Computers and Electronics in Agriculture, vol. 147, pp. 70–90, 2018. [Online]. Available: https://doi.org/10.1016/j.compag.2018.02.016.
Y. Zhong, J. Gao, Q. Lei, and Y. Zhou, "A vision-based counting and recognition system for flying insects in intelligent agriculture," Sensors, vol. 18, no. 5, p. 1489, 2018.
W. Li, P. Chen, B. Wang, and C. Xie, "Automatic Localization and Count of Agricultural Crop Pests Based on an Improved Deep Learning Pipeline," Scientific Reports, vol. 9, no. 1, 2019.
A. Fuentes, S. Yoon, S. C. Kim, and D. S. Park, "A robust deep-learningbased detector for real-time tomato plant diseases and pests recognition," Sensors (Switzerland), vol. 17, no. 9, 2017.
J. Chen, Y. Fan, T. Wang, C. Zhang, Z. Qiu, and Y. He, "Automatic Segmentation and Counting of Aphid Nymphs on Leaves Using Convolutional Neural Networks," Agronomy, vol. 8, no. 8, 2018.
A. Krizhevsky, I. Sutskever, and G. Hinton, "Imagenet classification with deep convolutional neural networks," Neural Information Processing Systems, vol. 25, 01 2012.
S. Ren, K. He, R. Girshick, and J. Sun, "Faster r-cnn: Towards real-time object detection with region proposal networks," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 39, no. 6, pp. 1137– 1149, 2016.
J. Long, E. Shelhamer, and T. Darrell, "Fully convolutional networks for semantic segmentation," in 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015, pp. 3431–3440.
P. Fischer, A. Dosovitskiy, and T. Brox, "Descriptor matching with convolutional neural networks: a comparison to SIFT," CoRR, vol. abs/1405.5769, 2014, withdrawn. [Online]. Available: http: //arxiv.org/abs/1405.5769.
Y. LeCun, Y. Bengio, and G. Hinton, "Deep learning," Nature, vol. 521, no. 7553, pp. 436–444, 2015.
J. Dai, Y. Li, K. He, and J. Sun, "R-fcn: Object detection via region-based fully convolutional networks," in Proceedings of the 30th International Conference on Neural Information Processing Systems, ser. NIPS’16. Red Hook, NY, USA: Curran Associates Inc., 2016, p. 379–387. [Online]. Available: https://dl.acm.org/doi/10.5555/3157096. 3157139.
K. He, G. Gkioxari, P. Dollar, and R. Girshick, "Mask r-cnn," in 2017 IEEE International Conference on Computer Vision (ICCV), 2017, pp. 2980–2988. [Online]. Available: https://doi.org/10.1109/ICCV.2017.322.
W. Liu, D. Anguelov, D. Erhan, C. Szegedy, S. Reed, C.-Y. Fu, and A. C. Berg, "SSD: Single Shot MultiBox Detector," in Computer Vision – ECCV 2016, B. Leibe, J. Matas, N. Sebe, and M. Welling, Eds. Cham: Springer International Publishing, 2016, pp. 21–37.
J. Redmon, S. Divvala, R. Girshick, and A. Farhadi, "You only look once: Unified, real-time object detection," in 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016, pp. 779–788. [Online]. Available: https://doi.org/10.1109/CVPR.2016.91.
T. Lin, P. Goyal, R. Girshick, K. He, and P. Dollar, "Focal loss for ´ dense object detection," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 42, no. 2, pp. 318–327, 2018. [Online]. Available: https://doi.org/10.1109/TPAMI.2018.2858826.
A. Nazri, N. Mazlan, and F. Muharam, "PENYEK: Automated brown planthopper detection from imperfect sticky pad images using deep convolutional neural network," PLOS ONE, vol. 13, no. 12, p. e0208501, dec 2018. [Online]. Available: http://dx.plos.org/10.1371/ journal.pone.0208501.
W. Abdulla, "Mask r-cnn for object detection and instance segmentation on keras and tensorflow," 2017. [Online]. Available: https://github.com/ matterport/Mask RCNN
Published
2020-11-07
How to Cite
DE CESARO JÚNIOR, Telmo; RIEDER, Rafael.
An Implementation Based on Mask R-CNN for Insect Detection in Digital Images. In: WORKSHOP OF WORKS IN PROGRESS - CONFERENCE ON GRAPHICS, PATTERNS AND IMAGES (SIBGRAPI), 33. , 2020, Evento Online.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2020
.
p. 139-142.
DOI: https://doi.org/10.5753/sibgrapi.est.2020.12996.
