Superpixel Segmentation Effect on Hierarchical GNN applied to Image Classification

• João Pedro de Melo Murta PUC Minas
• João Pedro O. Batisteli PUC Minas
• Silvio Jamil F. Guimarães PUC Minas
• Zenilton K. G. Patrocínio Jr PUC Minas

Resumo

In the image processing field, Graph Neural Networks (GNNs) are employed to learn from the graph structure to its full potential. Before model training, the data transformation converts the image into a graph structure. In this sense, many superpixel algorithms such as SLIC, DISF, SCALP, and ODISF have been adopted for reducing complexity and creating meaningful pixel regions based on some criteria. As these algorithms offer distinct characteristics and superpixel aspects, this work-inprogress presents a study on the performance of the HiErarchical Layered Multigraph Network (HELMNet) according to the variation of superpixel methods used to generate the base graph of the HiErarchical Layered Multigraph (HELM) representation with images from the STL-10 dataset. An attention function called Region Graph Readout (RGR), used for classification, is also set to guide the clarification of the divergent results. Furthermore, this study aims to highlight the nonexistence of a general rule for selecting a superpixel method given a specific visual task.

Referências

F. Scarselli, M. Gori, A. C. Tsoi, M. Hagenbuchner, and G. Monfardini, “The graph neural network model,” IEEE Transactions on Neural Networks, vol. 20, no. 1, pp. 61–80, 2009.

V. P. Dwivedi, C. K. Joshi, A. T. Luu, T. Laurent, Y. Bengio, and X. Bresson, “Benchmarking graph neural networks,” J. Mach. Learn. Res., vol. 24, no. 1, Jan. 2023.

J. P. O. Batisteli, S. J. F. Guimarães, and Z. K. G. do Patrocínio Júnior, “Multi-scale image graph representation: a novel gnn approach for image classification through scale importance estimation,” in IEEE International Symposium on Multimedia (ISM), 2023.

J. P. O. Batisteli, S. J. F. Guimarães, and Z. K. G. Patrocínio, Jr, “Hierarchical graph neural networks with scale-aware readout for image classification,” International Journal of Semantic Computing, vol. 18, no. 04, pp. 713–738, 2024.

J. P. O. Batisteli, S. J. F. Guimarães, and Z. K. G. do Patrocínio Júnior, “Hierarchical layered multigraph network with scale importance estimation for image classification,” Applied Soft Computing, 2025, submitted.

P. H. Avelar, A. R. Tavares, T. L. da Silveira, C. R. Jung, and L. C. Lamb, “Superpixel image classification with graph attention networks,” in 33rd SIBGRAPI Conference on Graphics, Patterns and Images (SIBGRAPI), 2020, pp. 203–209.

V. Vasudevan, M. Bassenne, M. T. Islam, and L. Xing, “Image classification using graph neural network and multiscale wavelet superpixels,” Pattern Recognition Letters, vol. 166, pp. 89–96, 2023.

J. Rodrigues and J. Carbonera, “Graph convolutional networks for image classification: Comparing approaches for building graphs from images,” in Proceedings of the 26th International Conference on Enterprise Information Systems-Volume 1: ICEIS, 2024, pp. 437–446.

K. Li, D. DeTone, Y. F. S. Chen, M. Vo, I. Reid, H. Rezatofighi, C. Sweeney, J. Straub, and R. Newcombe, “Odam: Object detection, association, and mapping using posed rgb video,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021, pp. 5998–6008.

M. Khademi and O. Schulte, “Deep generative probabilistic graph neural networks for scene graph generation,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34, no. 07, 2020, pp. 11 237–11 245.

W. Liang, Y. Jiang, and Z. Liu, “GraghVQA: Language-guided graph neural networks for graph-based visual question answering,” in Proceedings of the Third Workshop on Multimodal Artificial Intelligence, Jun. 2021, pp. 79–86.

R. Achanta, A. Shaji, K. Smith, A. Lucchi, P. Fua, and S. Süsstrunk, “Slic superpixels compared to state-of-the-art superpixel methods,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 34, no. 11, pp. 2274–2282, 2012.

F. C. Belém, S. J. F. Guimarães, and A. X. Falcão, “Superpixel segmentation using dynamic and iterative spanning forest,” IEEE Signal Processing Letters, vol. 27, pp. 1440–1444, 2020.

D. Stutz, A. Hermans, and B. Leibe, “Superpixels: An evaluation of the state-of-the-art,” Computer Vision and Image Understanding, vol. 166, pp. 1–27, 2018.

I. B. Barcelos, F. D. C. Belém, L. D. M. João, Z. K. G. D. Patrocínio, A. X. Falcão, and S. J. F. Guimarães, “A comprehensive review and new taxonomy on superpixel segmentation,” ACM Comput. Surv., vol. 56, no. 8, Apr. 2024.

B. Knyazev, X. Lin, M. Amer, and G. Taylor, “Image classification with hierarchical multigraph networks,” in Proceedings of the British Machine Vision Conference (BMVC), 2019, pp. 223.1–223.13.

M. Munir, W. Avery, M. M. Rahman, and R. Marculescu, “GreedyViG: Dynamic axial graph construction for efficient vision GNNs,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 6118–6127.

K. Han, Y. Wang, J. Guo, Y. Tang, and E. Wu, “Vision gnn: an image is worth graph of nodes,” in Proceedings of the 36th International Conference on Neural Information Processing Systems, ser. NIPS ’22. Red Hook, NY, USA: Curran Associates Inc., 2022.

G. Nikolentzos, M. Thomas, A. R. Rivera, and M. Vazirgiannis, “Image classification using graph-based representations and graph neural networks,” in Complex Networks & Their Applications IX: Volume 2, Proceedings of the Ninth International Conference on Complex Networks and Their Applications, 2021, pp. 142–153.

R. A. Cosma, L. Knobel, P. van der Linden, D. M. Knigge, and E. J. Bekkers, “Geometric superpixel representations for efficient image classification with graph neural networks,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2023, pp. 109–118.

J. Long, Z. yan, and H. chen, “A graph neural network for superpixel image classification,” Journal of Physics: Conference Series, vol. 1871, no. 1, p. 012071, apr 2021.

J. Cousty and L. Najman, “Incremental algorithm for hierarchical minimum spanning forests and saliency of watershed cuts,” in ISMM, 2011, pp. 272–283.

F. C. Belém, B. Perret, J. Cousty, S. J. F. Guimarães, and A. X. Falcão, “Towards a simple and efficient object-based superpixel delineation framework,” in 2021 34th SIBGRAPI Conference on Graphics, Patterns and Images (SIBGRAPI), 2021, pp. 346–353.

R. Giraud, V.-T. Ta, and N. Papadakis, “Scalp: Superpixels with contour adherence using linear path,” in 2016 23rd International Conference on Pattern Recognition (ICPR), 2016, pp. 2374–2379.