Towards an Agnostic Superpixel Segmentation Framework
Resumo
Superpixel segmentation partitions multiple objects into disjoint parts so that their delineation can be accurately achieved by their grouping, and it has been used as an intermediary step for solving multiple problems. However, state-of-the-art algorithms face a significant challenge of effective and efficient segmentation irrespective of the problem’s domain (object and background characteristics, and user’s desires). In this work, we address such challenge by proposing several contributions. One of such is a novel superpixel segmentation framework, named Superpixels through Iterative CLEarcutting (SICLE), which generalizes two other contributions of this work. In SICLE, three independent steps are defined: (i) seed oversampling; (ii) superpixel generation using the Image Foresting Transform (IFT) framework; and (iii) seed removal. From (i), where a significantly high amount of seeds is selected, steps (ii) and (iii) are performed for generating superpixels from a refined seed set until achieving the desired number of superpixels. SICLE overcomes domain shifts primarily through steps (ii) and (iii), where the user may provide an objective function for optimization. Experimental results show that SICLE variants surpass several state-of-the-art algorithms concerning speed and accuracy for distinct domains while generating a series of segmentations in a single execution. Still, in SICLE, the contours from a preceding scale might not be present in the subsequent one leading to hierarchical violations. Thus, we studied eight possible cases when analyzing pairwise subsequent segmentations, and we conceived three measures for estimating the hierarchiness of a multiscale segmentation: (i) nestedness; (ii) inflation ratio; and (iii) refinement error. From our results, it is possible to verify if a multiscale is a hierarchy and, when it is not the case, to analyze and state the nature and extent of the hierarchical violations that prevent it from being hierarchical.Referências
C. Castelo-Fernandez and A. Falcão, “Learning visual dictionaries from class-specific superpixel segmentation,” in Computer Analysis of Images and Patterns, 2019, pp. 171–182.
S. Dhore and D. Abin, “Chest x-ray segmentation using watershed and super pixel segmentation technique,” in 4th International Conference on Communication information and Computing Technology (ICCICT), 2021, pp. 1–4.
Y. Yu, Y. Makihara, and Y. Yagi, “Pedestrian segmentation based on a spatio-temporally consistent graph-cut with optimal transport,” Transactions on Computer Vision and Applications, vol. 11, no. 1, p. 10, 2019.
R. Achanta, A. Shaji, K. Smith, A. Lucchi, P. Fua, and S. Süsstrunk, “SLIC superpixels compared to state-of-the-art superpixel methods,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 34, no. 11, pp. 2274–2282, 2012.
M. Liu, O. Tuzel, S. Ramalingam, and R. Chellappa, “Entropy rate superpixel segmentation,” in IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2011, pp. 2097–2104.
J. Wu, C. Liu, and B. Li, “Texture-aware and structure-preserving superpixel segmentation,” Computers and Graphics, vol. 94, pp. 152–163, 2021.
H. Peng, A. Aviles-Rivero, and C. Schonlieb, “Hers superpixels: Deep affinity learning for hierarchical entropy rate segmentation,” in IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), 2022, pp. 72–81.
F. Belém, S. Guimarães, and A. Falcão, “Superpixel segmentation by object-based iterative spanning forest,” in Progress in Pattern Recognition, Image Analysis, Computer Vision, and Applications, vol. 11401, 2019, pp. 334–341.
——, “Superpixel generation by the iterative spanning forest using object information,” in 33rd Conference on Graphics, Patterns and Images (SIBGRAPI), 2020, pp. 22–28, workshop of Thesis and Dissertations.
F. Belém, J. Cousty, B. Perret, S. Guimarães, and A. Falcão, “Towards a simple and efficient object-based superpixel delineation framework,” in 34th Conference on Graphics, Patterns and Images (SIBGRAPI), 2021, pp. 346–353.
X. Wei, Q. Yang, Y. Gong, N. Ahuja, and M. Yang, “Superpixel hierarchy,” IEEE Transactions on Image Processing, vol. 27, no. 10, pp. 4838–4849, 2018.
F. Belém, S. Guimarães, and A. Falcão, “Superpixel segmentation using dynamic and iterative spanning forest,” Signal Processing Letters, vol. 27, pp. 1440–1444, 2020.
S. Guimarães, Y. Kenmochi, J. Cousty, Z. Patrocínio, and L. Najman, “Hierarchizing graph-based image segmentation algorithms relying on region dissimilarity: the case of the felzenszwalb-huttenlocher method,” Mathematical Morphology - Theory and Applications, vol. 2, no. 1, pp. 55–75, 2017.
Z. Wu, L. He, Z. Hu, Y. Zhang, and G. Wu, “Hierarchical segmentation evaluation of region-based image hierarchy,” IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, vol. 12, no. 8, pp. 2718–2727, 2019.
S. Todorovic and M. Nechyba, “Dynamic trees for unsupervised segmentation and matching of image regions,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 27, no. 11, pp. 1762–1777, 2005.
J. Pont-Tuset and F. Marqués, “Supervised evaluation of image segmentation and object proposal techniques,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 38, pp. 1–1, 2015.
A. Falcão, J. Stolfi, and R. Lotufo, “The image foresting transform: Theory, algorithms, and applications,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 26, no. 1, pp. 19–29, 2004.
X. Qin, Z. Zhang, C. Huang, M. Dehghan, O. Zaiane, and M. Jagersand, “U2-net: Going deeper with nested u-structure for salient object detection,” Pattern Recognition, vol. 106, p. 107404, 2020.
F. Belém, L. Melo, S. Guimarães, and A. Falcão, “The importance of object-based seed sampling for superpixel segmentation,” in 32nd Conference on Graphics, Patterns and Images (SIBGRAPI), 2019, pp. 108–115.
P. Neubert and P. Protzel, “Superpixel benchmark and comparison,” in Forum Bildverarbeitung, vol. 6, 2012, pp. 1–12.
J. Shi, Q. Yan, L. Xu, and J. Jia, “Hierarchical image saliency detection on extended cssd,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 38, no. 4, pp. 717–729, 2015.
L. Mansilla and P. Miranda, “Oriented image foresting transform segmentation: Connectivity constraints with adjustable width,” in 29th Conference on Graphics, Patterns and Images (SIBGRAPI), 2016, pp. 289–296.
J. Vargas-Muñoz, A. Chowdhury, E. Alexandre, F. Galvão, P. Miranda, and A. Falcão, “An iterative spanning forest framework for superpixel segmentation,” IEEE Transactions on Image Processing, vol. 28, no. 7, pp. 3477–3489, 2019.
S. Bobbia, R. Macwan, Y. Benezeth, K. Nakamura, R. Gomez, and J. Dubois, “Iterative boundaries implicit identification for superpixels segmentation: A real-time approach,” IEEE Access, vol. 9, pp. 77 250–77 263, 2021.
Z. Li and J. Chen, “Superpixel segmentation using linear spectral clustering,” in IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015, pp. 1356–1363.
Z. Ban, J. Liu, and L. Cao, “Superpixel segmentation using gaussian mixture model,” IEEE Transactions on Image Processing, vol. 27, no. 8, pp. 4105–4117, 2018.
F. Galvão, A. Falcão, and A. Chowdhury, “RISF: recursive iterative spanning forest for superpixel segmentation,” in 31st Conference on Graphics, Patterns and Images (SIBGRAPI), 2018, pp. 408–415.
D. Stutz, A. Hermans, and B. Leibe, “Superpixels: An evaluation of the state-of-the-art,” Computer Vision and Image Understanding, vol. 166, pp. 1–27, 2018.
C. Jerônimo, F. Belém, S. Carneiro, Z. Patrocínio, L. Najman, A. Falcão, and S. Guimarães, “Graph-based supervoxel computation from iterative spanning forest,” in Discrete Geometry and Mathematical Morphology, 2021, pp. 404–415.
I. Barcelos, F. Belém, P. Miranda, A. Falcão, Z. Patrocínio, and S. Guimarães, “Towards interactive image segmentation by dynamic and iterative spanning forest,” in Discrete Geometry and Mathematical Morphology, 2021, pp. 351–364.
I. Barcelos, F. Belém, L. João, A. Falcão, and S. Guimarães, “Improving color homogeneity measure in superpixel segmentation assessment,” in 35th Conference on Graphics, Patterns and Images (SIBGRAPI), vol. 1, 2022, pp. 79–84.
F. Belém, I. Borlido, L. João, B. Perret, J. Cousty, S. Guimarães, and A. Falcão, “Fast and effective superpixel segmentation using accurate saliency estimation,” in Discrete Geometry and Mathematical Morphology, 2022, pp. 261–273.
——, “Novel arc-cost functions and seed relevance estimations for compact and accurate superpixels,” Journal of Mathematical Imaging and Vision, vol. 65, no. 5, pp. 770–786, 2023.
I. Barcelos, F. Belém, L. João, Z. Patrocínio, A. Falcão, and S. Guimarães, “A comprehensive review and new taxonomy on superpixel segmentation,” ACM Computing Surveys, vol. 56, no. 8, pp. 1–39, 2024.
D. Vieira, I. Borlido, , F. Belém, Z. Patrocínio, A. Falcão, and S. Guimarães, “Streaming graph-based supervoxel computation based on dynamic iterative spanning forest,” in Progress in Pattern Recognition, Image Analysis, Computer Vision, and Applications, 2024, pp. 90–104.
F. Belém, F. Faria, A. Falcão, Z. Patrocínio, S. Guimarães, B. Perret, and J. Cousty, “Measuring hierarchiness of image segmentations,” pp. 1–6, 2024, to appear.
F. Belém, B. Perret, J. Cousty, S. Guimarães, and A. Falcão, “Efficient multiscale object-based superpixel framework,” pp. 1–16, 2024, submitted to the Journal of the Brazilian Computer Society.
L. Lacerda, F. Belém, A. Falcão, Z. Patrocínio, and S. Guimarães, “Seed-based superpixel re-segmentation for improving object delineation,” pp. 1–13, 2024, submitted to the Iberoamerican Conference on Pattern Recognition.
S. Dhore and D. Abin, “Chest x-ray segmentation using watershed and super pixel segmentation technique,” in 4th International Conference on Communication information and Computing Technology (ICCICT), 2021, pp. 1–4.
Y. Yu, Y. Makihara, and Y. Yagi, “Pedestrian segmentation based on a spatio-temporally consistent graph-cut with optimal transport,” Transactions on Computer Vision and Applications, vol. 11, no. 1, p. 10, 2019.
R. Achanta, A. Shaji, K. Smith, A. Lucchi, P. Fua, and S. Süsstrunk, “SLIC superpixels compared to state-of-the-art superpixel methods,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 34, no. 11, pp. 2274–2282, 2012.
M. Liu, O. Tuzel, S. Ramalingam, and R. Chellappa, “Entropy rate superpixel segmentation,” in IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2011, pp. 2097–2104.
J. Wu, C. Liu, and B. Li, “Texture-aware and structure-preserving superpixel segmentation,” Computers and Graphics, vol. 94, pp. 152–163, 2021.
H. Peng, A. Aviles-Rivero, and C. Schonlieb, “Hers superpixels: Deep affinity learning for hierarchical entropy rate segmentation,” in IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), 2022, pp. 72–81.
F. Belém, S. Guimarães, and A. Falcão, “Superpixel segmentation by object-based iterative spanning forest,” in Progress in Pattern Recognition, Image Analysis, Computer Vision, and Applications, vol. 11401, 2019, pp. 334–341.
——, “Superpixel generation by the iterative spanning forest using object information,” in 33rd Conference on Graphics, Patterns and Images (SIBGRAPI), 2020, pp. 22–28, workshop of Thesis and Dissertations.
F. Belém, J. Cousty, B. Perret, S. Guimarães, and A. Falcão, “Towards a simple and efficient object-based superpixel delineation framework,” in 34th Conference on Graphics, Patterns and Images (SIBGRAPI), 2021, pp. 346–353.
X. Wei, Q. Yang, Y. Gong, N. Ahuja, and M. Yang, “Superpixel hierarchy,” IEEE Transactions on Image Processing, vol. 27, no. 10, pp. 4838–4849, 2018.
F. Belém, S. Guimarães, and A. Falcão, “Superpixel segmentation using dynamic and iterative spanning forest,” Signal Processing Letters, vol. 27, pp. 1440–1444, 2020.
S. Guimarães, Y. Kenmochi, J. Cousty, Z. Patrocínio, and L. Najman, “Hierarchizing graph-based image segmentation algorithms relying on region dissimilarity: the case of the felzenszwalb-huttenlocher method,” Mathematical Morphology - Theory and Applications, vol. 2, no. 1, pp. 55–75, 2017.
Z. Wu, L. He, Z. Hu, Y. Zhang, and G. Wu, “Hierarchical segmentation evaluation of region-based image hierarchy,” IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, vol. 12, no. 8, pp. 2718–2727, 2019.
S. Todorovic and M. Nechyba, “Dynamic trees for unsupervised segmentation and matching of image regions,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 27, no. 11, pp. 1762–1777, 2005.
J. Pont-Tuset and F. Marqués, “Supervised evaluation of image segmentation and object proposal techniques,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 38, pp. 1–1, 2015.
A. Falcão, J. Stolfi, and R. Lotufo, “The image foresting transform: Theory, algorithms, and applications,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 26, no. 1, pp. 19–29, 2004.
X. Qin, Z. Zhang, C. Huang, M. Dehghan, O. Zaiane, and M. Jagersand, “U2-net: Going deeper with nested u-structure for salient object detection,” Pattern Recognition, vol. 106, p. 107404, 2020.
F. Belém, L. Melo, S. Guimarães, and A. Falcão, “The importance of object-based seed sampling for superpixel segmentation,” in 32nd Conference on Graphics, Patterns and Images (SIBGRAPI), 2019, pp. 108–115.
P. Neubert and P. Protzel, “Superpixel benchmark and comparison,” in Forum Bildverarbeitung, vol. 6, 2012, pp. 1–12.
J. Shi, Q. Yan, L. Xu, and J. Jia, “Hierarchical image saliency detection on extended cssd,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 38, no. 4, pp. 717–729, 2015.
L. Mansilla and P. Miranda, “Oriented image foresting transform segmentation: Connectivity constraints with adjustable width,” in 29th Conference on Graphics, Patterns and Images (SIBGRAPI), 2016, pp. 289–296.
J. Vargas-Muñoz, A. Chowdhury, E. Alexandre, F. Galvão, P. Miranda, and A. Falcão, “An iterative spanning forest framework for superpixel segmentation,” IEEE Transactions on Image Processing, vol. 28, no. 7, pp. 3477–3489, 2019.
S. Bobbia, R. Macwan, Y. Benezeth, K. Nakamura, R. Gomez, and J. Dubois, “Iterative boundaries implicit identification for superpixels segmentation: A real-time approach,” IEEE Access, vol. 9, pp. 77 250–77 263, 2021.
Z. Li and J. Chen, “Superpixel segmentation using linear spectral clustering,” in IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015, pp. 1356–1363.
Z. Ban, J. Liu, and L. Cao, “Superpixel segmentation using gaussian mixture model,” IEEE Transactions on Image Processing, vol. 27, no. 8, pp. 4105–4117, 2018.
F. Galvão, A. Falcão, and A. Chowdhury, “RISF: recursive iterative spanning forest for superpixel segmentation,” in 31st Conference on Graphics, Patterns and Images (SIBGRAPI), 2018, pp. 408–415.
D. Stutz, A. Hermans, and B. Leibe, “Superpixels: An evaluation of the state-of-the-art,” Computer Vision and Image Understanding, vol. 166, pp. 1–27, 2018.
C. Jerônimo, F. Belém, S. Carneiro, Z. Patrocínio, L. Najman, A. Falcão, and S. Guimarães, “Graph-based supervoxel computation from iterative spanning forest,” in Discrete Geometry and Mathematical Morphology, 2021, pp. 404–415.
I. Barcelos, F. Belém, P. Miranda, A. Falcão, Z. Patrocínio, and S. Guimarães, “Towards interactive image segmentation by dynamic and iterative spanning forest,” in Discrete Geometry and Mathematical Morphology, 2021, pp. 351–364.
I. Barcelos, F. Belém, L. João, A. Falcão, and S. Guimarães, “Improving color homogeneity measure in superpixel segmentation assessment,” in 35th Conference on Graphics, Patterns and Images (SIBGRAPI), vol. 1, 2022, pp. 79–84.
F. Belém, I. Borlido, L. João, B. Perret, J. Cousty, S. Guimarães, and A. Falcão, “Fast and effective superpixel segmentation using accurate saliency estimation,” in Discrete Geometry and Mathematical Morphology, 2022, pp. 261–273.
——, “Novel arc-cost functions and seed relevance estimations for compact and accurate superpixels,” Journal of Mathematical Imaging and Vision, vol. 65, no. 5, pp. 770–786, 2023.
I. Barcelos, F. Belém, L. João, Z. Patrocínio, A. Falcão, and S. Guimarães, “A comprehensive review and new taxonomy on superpixel segmentation,” ACM Computing Surveys, vol. 56, no. 8, pp. 1–39, 2024.
D. Vieira, I. Borlido, , F. Belém, Z. Patrocínio, A. Falcão, and S. Guimarães, “Streaming graph-based supervoxel computation based on dynamic iterative spanning forest,” in Progress in Pattern Recognition, Image Analysis, Computer Vision, and Applications, 2024, pp. 90–104.
F. Belém, F. Faria, A. Falcão, Z. Patrocínio, S. Guimarães, B. Perret, and J. Cousty, “Measuring hierarchiness of image segmentations,” pp. 1–6, 2024, to appear.
F. Belém, B. Perret, J. Cousty, S. Guimarães, and A. Falcão, “Efficient multiscale object-based superpixel framework,” pp. 1–16, 2024, submitted to the Journal of the Brazilian Computer Society.
L. Lacerda, F. Belém, A. Falcão, Z. Patrocínio, and S. Guimarães, “Seed-based superpixel re-segmentation for improving object delineation,” pp. 1–13, 2024, submitted to the Iberoamerican Conference on Pattern Recognition.
Publicado
30/09/2025
Como Citar
BELÉM, Felipe; PERRET, Benjamin; GUIMARÃES, Silvio Jamil F.; COUSTY, Jean; FALCÃO, Alexandre X..
Towards an Agnostic Superpixel Segmentation Framework. In: WORKSHOP DE TESES E DISSERTAÇÕES - CONFERENCE ON GRAPHICS, PATTERNS AND IMAGES (SIBGRAPI), 38. , 2025, Salvador/BA.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2025
.
p. 1-7.
