Graph-based Multibeam Forward Looking Acoustic Image Classification
Resumo
Multibeam sonar imaging has many applications, such as mine-like detection and navigation tasks, motivating interest in the automatic classification of sonar images. Recent works have proposed graph neural networks (GNNs) as an alternative to convolutional neural networks (CNNs) to address this task. This paper focuses on combining the strengths of both models to enhance the performance of GNNs when classifying sonar images. This proposal exploits a superpixel algorithm for image segmentation and graph formation. Comprehensive experiments with an MFLS open dataset evaluate the effect of model design parameters on the performance of the proposed approach. Using CNN-extracted features as initial node embeddings significantly improved the graph-based image classification performance.
Palavras-chave:
Computer Vision, Graph-Oriented Machine Learning, Hybrid Intelligent Systems
Referências
Achanta, R., Shaji, A., Smith, K., Lucchi, A., Fua, P., and Süsstrunk, S. (2012). SLIC superpixels compared to state-of-the-art superpixel methods. IEEE transactions on pattern analysis and machine intelligence, 34(11):2274–2282.
Avelar, P. H., Tavares, A. R., da Silveira, T. L., Jung, C. R., and Lamb, L. C. (2020). Superpixel image classification with graph attention networks. In 2020 33rd SIBGRAPI Conference on Graphics, Patterns and Images (SIBGRAPI), pages 203–209. IEEE.
Bjørnø, L., Neighbors, T., and Bradley, D. (2017). Applied underwater acoustics. Elsevier.
Belkin, M. and Niyogi, P. (2001). Laplacian eigenmaps and spectral techniques for embedding and clustering. Advances in neural information processing systems, 14.
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., and Fei-Fei, L. (2009). Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pages 248–255. IEEE.
Ding, Y., Zhang, Z., Zhao, X., Hong, D., Cai, W., Yu, C., Yang, N., and Cai, W. (2022). Multi-feature fusion: graph neural network and CNN combining for hyperspectral image classification. Neurocomputing, 501:246–257.
Dos Santos, M. M., De Giacomo, G. G., Drews-Jr, P. L., and Botelho, S. S. (2022). Crossview and cross-domain underwater localization based on optical aerial and acoustic underwater images. IEEE Robotics and Automation Letters, 7(2):4969–4974.
Dwivedi, V. P., Joshi, C. K., Luu, A. T., Laurent, T., Bengio, Y., and Bresson, X. (2023). Benchmarking graph neural networks. Journal of Machine Learning Research, 24(43):1–48.
Galceran, E., Djapic, V., Carreras, M., and Williams, D. P. (2012). A real-time underwater object detection algorithm for multi-beam forward looking sonar. IFAC Proceedings Volumes, 45(5):306–311.
Girshick, R. (2015). Fast R-CNN. Proceedings of the IEEE International Conference on Computer Vision (ICCV), pages 1440–1448.
He, K., Zhang, X., Ren, S., and Sun, J. (2016). Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 770–778.
Huo, G., Wu, Z., and Li, J. (2020). Underwater object classification in sidescan sonar images using deep transfer learning and semisynthetic training data. IEEE access, 8:47407–47418.
Kipf, T. N. and Welling, M. (2017). Semi-supervised classification with graph convolutional networks. In International Conference on Learning Representations (ICLR).
Knyazev, B., Taylor, G. W., and Amer, M. (2019). Understanding attention and generalization in graph neural networks. Advances in neural information processing systems, 32.
Lei, C., Wang, H., and Lei, J. (2022). SI-GAT: A method based on improved graph attention network for sonar image classification. arXiv preprint arXiv:2211.15133.
Mitrokhin, A., Hua, Z., Fermuller, C., and Aloimonos, Y. (2020). Learning visual motion segmentation using event surfaces. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14414–14423.
Ren, S., He, K., Girshick, R., and Sun, J. (2015). Faster R-CNN: Towards real-time object detection with region proposal networks. Advances in neural information processing systems, 28.
Schaefer, S., Gehrig, D., and Scaramuzza, D. (2022). AEGNN: asynchronous event-based graph neural networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12371–12381.
Shi, W. and Rajkumar, R. (2020). Point-GNN: graph neural network for 3D object detection in a point cloud. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 1711–1719.
Sinai, A., Amar, A., and Gilboa, G. (2016). Mine-like objects detection in side-scan sonar images using a shadows-highlights geometrical features space. In OCEANS 2016 MTS/IEEE Monterey, pages 1–6. IEEE.
Singh, D. and Valdenegro-Toro, M. (2021). The marine debris dataset for forward-looking sonar semantic segmentation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 3741–3749.
Steiniger, Y., Kraus, D., and Meisen, T. (2022). Survey on deep learning based computer vision for sonar imagery. Engineering Applications of Artificial Intelligence, 114:105157.
Vasudevan, V., Bassenne, M., Islam, M. T., and Xing, L. (2023). Image classification using graph neural network and multiscale wavelet superpixels. Pattern Recognition Letters.
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, Ł., and Polosukhin, I. (2017). Attention is all you need. Advances in neural information processing systems, 30.
Veličković, P., Cucurull, G., Casanova, A., Romero, A., Lio, P., and Bengio, Y. (2018). Graph attention networks. In International Conference on Learning Representations (ICLR).
Xie, K., Yang, J., and Qiu, K. (2022). A dataset with multibeam forward-looking sonar for underwater object detection. Scientific Data, 9(1):739.
Xu, K., Hu, W., Leskovec, J., and Jegelka, S. (2019). How powerful are graph neural networks? In International Conference on Learning Representations (ICLR).
Yang, D., Cheng, C., Wang, C., Pan, G., and Zhang, F. (2022). Side-scan sonar image segmentation based on multi-channel CNN for AUV navigation. Frontiers in Neuro-robotics, 16:928206.
Ye, X., Li, C., Zhang, S., Yang, P., and Li, X. (2018). Research on side-scan sonar image target classification method based on transfer learning. In OCEANS 2018 MTS/IEEE Charleston, pages 1–6. IEEE.
You, J., Ying, Z., and Leskovec, J. (2020). Design space for graph neural networks. Advances in Neural Information Processing Systems, 33:17009–17021.
Yu, Y., Zhao, J., Gong, Q., Huang, C., Zheng, G., and Ma, J. (2021). Real-time underwater maritime object detection in side-scan sonar images based on Transformer-YOLOv5. Remote Sensing, 13(18):3555.
Avelar, P. H., Tavares, A. R., da Silveira, T. L., Jung, C. R., and Lamb, L. C. (2020). Superpixel image classification with graph attention networks. In 2020 33rd SIBGRAPI Conference on Graphics, Patterns and Images (SIBGRAPI), pages 203–209. IEEE.
Bjørnø, L., Neighbors, T., and Bradley, D. (2017). Applied underwater acoustics. Elsevier.
Belkin, M. and Niyogi, P. (2001). Laplacian eigenmaps and spectral techniques for embedding and clustering. Advances in neural information processing systems, 14.
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., and Fei-Fei, L. (2009). Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pages 248–255. IEEE.
Ding, Y., Zhang, Z., Zhao, X., Hong, D., Cai, W., Yu, C., Yang, N., and Cai, W. (2022). Multi-feature fusion: graph neural network and CNN combining for hyperspectral image classification. Neurocomputing, 501:246–257.
Dos Santos, M. M., De Giacomo, G. G., Drews-Jr, P. L., and Botelho, S. S. (2022). Crossview and cross-domain underwater localization based on optical aerial and acoustic underwater images. IEEE Robotics and Automation Letters, 7(2):4969–4974.
Dwivedi, V. P., Joshi, C. K., Luu, A. T., Laurent, T., Bengio, Y., and Bresson, X. (2023). Benchmarking graph neural networks. Journal of Machine Learning Research, 24(43):1–48.
Galceran, E., Djapic, V., Carreras, M., and Williams, D. P. (2012). A real-time underwater object detection algorithm for multi-beam forward looking sonar. IFAC Proceedings Volumes, 45(5):306–311.
Girshick, R. (2015). Fast R-CNN. Proceedings of the IEEE International Conference on Computer Vision (ICCV), pages 1440–1448.
He, K., Zhang, X., Ren, S., and Sun, J. (2016). Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 770–778.
Huo, G., Wu, Z., and Li, J. (2020). Underwater object classification in sidescan sonar images using deep transfer learning and semisynthetic training data. IEEE access, 8:47407–47418.
Kipf, T. N. and Welling, M. (2017). Semi-supervised classification with graph convolutional networks. In International Conference on Learning Representations (ICLR).
Knyazev, B., Taylor, G. W., and Amer, M. (2019). Understanding attention and generalization in graph neural networks. Advances in neural information processing systems, 32.
Lei, C., Wang, H., and Lei, J. (2022). SI-GAT: A method based on improved graph attention network for sonar image classification. arXiv preprint arXiv:2211.15133.
Mitrokhin, A., Hua, Z., Fermuller, C., and Aloimonos, Y. (2020). Learning visual motion segmentation using event surfaces. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14414–14423.
Ren, S., He, K., Girshick, R., and Sun, J. (2015). Faster R-CNN: Towards real-time object detection with region proposal networks. Advances in neural information processing systems, 28.
Schaefer, S., Gehrig, D., and Scaramuzza, D. (2022). AEGNN: asynchronous event-based graph neural networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12371–12381.
Shi, W. and Rajkumar, R. (2020). Point-GNN: graph neural network for 3D object detection in a point cloud. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 1711–1719.
Sinai, A., Amar, A., and Gilboa, G. (2016). Mine-like objects detection in side-scan sonar images using a shadows-highlights geometrical features space. In OCEANS 2016 MTS/IEEE Monterey, pages 1–6. IEEE.
Singh, D. and Valdenegro-Toro, M. (2021). The marine debris dataset for forward-looking sonar semantic segmentation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 3741–3749.
Steiniger, Y., Kraus, D., and Meisen, T. (2022). Survey on deep learning based computer vision for sonar imagery. Engineering Applications of Artificial Intelligence, 114:105157.
Vasudevan, V., Bassenne, M., Islam, M. T., and Xing, L. (2023). Image classification using graph neural network and multiscale wavelet superpixels. Pattern Recognition Letters.
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, Ł., and Polosukhin, I. (2017). Attention is all you need. Advances in neural information processing systems, 30.
Veličković, P., Cucurull, G., Casanova, A., Romero, A., Lio, P., and Bengio, Y. (2018). Graph attention networks. In International Conference on Learning Representations (ICLR).
Xie, K., Yang, J., and Qiu, K. (2022). A dataset with multibeam forward-looking sonar for underwater object detection. Scientific Data, 9(1):739.
Xu, K., Hu, W., Leskovec, J., and Jegelka, S. (2019). How powerful are graph neural networks? In International Conference on Learning Representations (ICLR).
Yang, D., Cheng, C., Wang, C., Pan, G., and Zhang, F. (2022). Side-scan sonar image segmentation based on multi-channel CNN for AUV navigation. Frontiers in Neuro-robotics, 16:928206.
Ye, X., Li, C., Zhang, S., Yang, P., and Li, X. (2018). Research on side-scan sonar image target classification method based on transfer learning. In OCEANS 2018 MTS/IEEE Charleston, pages 1–6. IEEE.
You, J., Ying, Z., and Leskovec, J. (2020). Design space for graph neural networks. Advances in Neural Information Processing Systems, 33:17009–17021.
Yu, Y., Zhao, J., Gong, Q., Huang, C., Zheng, G., and Ma, J. (2021). Real-time underwater maritime object detection in side-scan sonar images based on Transformer-YOLOv5. Remote Sensing, 13(18):3555.
Publicado
25/09/2023
Como Citar
EVANGELISTA, Gabriel Arruda; OLIVEIRA E SOUZA FILHO, João Baptista de.
Graph-based Multibeam Forward Looking Acoustic Image Classification. In: ENCONTRO NACIONAL DE INTELIGÊNCIA ARTIFICIAL E COMPUTACIONAL (ENIAC), 20. , 2023, Belo Horizonte/MG.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2023
.
p. 756-770.
ISSN 2763-9061.
DOI: https://doi.org/10.5753/eniac.2023.234431.
