Superpixel Generation by the Iterative Spanning Forest Using Object Information
Resumo
Superpixel segmentation methods aim to partition the image into homogeneous connected regions of pixels (i.e., superpixels) such that the union of its comprising superpixels precisely defines the objects of interest. However, the homogeneity criterion is often based solely on color, which, in certain conditions, might be insufficient for inferring the extension of the objects (e.g., low gradient regions). In this dissertation, we address such issue by incorporating prior object information — represented as monochromatic object saliency maps — into a state-of-the-art method, the Iterative Spanning Forest (ISF) framework, resulting in a novel framework named Object-based ISF (OISF). For a given saliency map, OISF-based methods are capable of increasing the superpixel resolution within the objects of interest, whilst permitting a higher adherence to the map’s borders, when color is insufficient for delineation. We compared our work with state-of-the-art methods, considering two classic superpixel segmentation metrics, in three datasets. Experimental results show that our approach presents effective object delineation with a significantly lower number of superpixels than the baselines, especially in terms of preventing superpixel leaking.Referências
Z. Li and J. Chen, "Superpixel segmentation using linear spectral clus- tering," in 28th Conference on Computer Vision and Pattern Recognition (CVPR), 2015, pp. 1356–1363.
Y. Li and L. Shen, "Skin lesion analysis towards melanoma detection using deep learning network," Sensors, vol. 18, no. 2, p. 556, 2018.
J. Zhou, J. Ruan, C. Wu, G. Ye, Z. Zhu, J. Yue, and Y. Zhang, "Super- pixel segmentation of breast cancer pathology images based on features extracted from the autoencoder," in 11th International Conference on Communication Software and Networks (ICCSN), 2019, pp. 366–370.
M. Liu, O. Tuzel, S. Ramalingam, and R. Chellappa, "Entropy rate superpixel segmentation," in 24th Conference on Computer Vision and Pattern Recognition (CVPR), 2011, pp. 2097–2104.
S. Zhang, H. Wang, W. Huang, and Z. You, "Plant diseased leaf segmentation and recognition by fusion of superpixel, K-means and PHOG," Optik, vol. 157, pp. 866–872, 2018.
W. Wu, A. Chen, L. Zhao, and J. Corso, "Brain tumor detection and segmentation in a CRF (conditional random fields) framework with pixel-pairwise affinity and superpixel-level features," International Journal of Computer Assisted Radiology and Surgery, vol. 9, no. 2, pp. 241–253, 2014.
J. Vargas-Múnoz, A. Chowdhury, E. Alexandre, F. Galvão, P. Miranda, and A. Falcão, "An iterative spanning forest framework for superpixel segmentation," Transactions on Image Processing, vol. 28, no. 7, pp. 3477–3489, 2019.
F. Belém, S. Guimarães, and A. Falcão, "Superpixel segmentation by object-based iterative spanning forest," in 23rd Iberoamerican Congress on Pattern Recognition, 2018, pp. 334–341.
F. Belém, L. Melo, S. Guimarães, and A. Falcão, "The importance of object-based seed sampling for superpixel segmentation," in 32nd Conference on Graphics, Patterns and Images (SIBGRAPI), 2019, pp. 108–115.
A. Schick, M. Fischer, and R. Stiefelhagen, "An evaluation of the compactness of superpixels," Pattern Recognition Letters, vol. 43, pp. 71–80, 2014.
D. Stutz, A. Hermans, and B. Leibe, "Superpixels: An evaluation of the state-of-the-art," Computer Vision and Image Understanding, vol. 166, pp. 1–27, 2018.
L. Wan, X. Xu, Q. Zhao, and W. Feng, "Spherical superpixels: Bench- mark and evaluation," in 14th Asian Conference on Computer Vision (ACCV), 2018, pp. 703–717.
X. Ren and J. Malik, "Learning a classification model for segmentation," in 9th International Conference on Computer Vision (ICCV), vol. 1, 2003, pp. 10–17.
Z. Ren and G. Shakhnarovich, "Image segmentation by cascaded region agglomeration," in 26th Conference on Computer Vision and Pattern Recognition (CVPR), 2013, pp. 2011–2018.
R. Achanta, A. Shaji, K. Smith, A. Lucchi, P. Fua, and S. S¨usstrunk, "SLIC superpixels compared to state-of-the-art superpixel methods," Transactions on Pattern Analysis and Machine Intelligence, vol. 34, no. 11, pp. 2274–2282, 2012.
R. Achanta and S. S¨usstrunk, "Superpixels and polygons using simple non-iterative clustering," in 30th Conference on Computer Vision and Pattern Recognition (CVPR), 2017, pp. 4651–4660.
Y. Liu, M. Yu, B. Li, and Y. He, "Intrinsic manifold SLIC: A simple and efficient method for computing content-sensitive superpixels," Transac- tions on Pattern Analysis and Machine Intelligence, vol. 40, no. 3, pp. 653–666, 2018.
A. Falcão, J. Stolfi, and R. Lotufo, "The image foresting transform: Theory, algorithms, and applications," Transactions on Pattern Analysis and Machine Intelligence, vol. 26, no. 1, pp. 19–29, 2004.
F. Galvão, A. Falcão, and A. Chowdhury, "RISF: recursive iterative spanning forest for superpixel segmentation," in 31st Conference on Graphics, Patterns and Images (SIBGRAPI), 2018, pp. 408–415.
C. Castelo-Fernandez and A. Falcão, "Learning visual dictionaries from class-specific superpixel segmentation," in 18th Conference on Computer Analysis of Images and Patterns (CAIP), 2019, pp. 171–182.
S. Martins, G. Ruppert, F. Reis, C. Yasuda, and A. Falcao, "A supervoxel-based approach for unsupervised abnormal asymmetry detection in MR images of the brain," in 16th International Symposium on Biomedical Imaging (ISBI), 2019, pp. 882–885.
M. Awaisu, L. Li, J. Peng, and J. Zhang, "Fast superpixel segmentation with deep features," in 36th Computer Graphics International Conference (CGI), 2019, pp. 410–416.
V. Jampani, D. Sun, M. Liu, M. Yang, and J. Kautz, "Superpixel sampling networks," in 18th European Conference on Computer Vision (ECCV), 2018, pp. 352–368.
W. Tu, M. Liu, V. Jampani, D. Sun, S. Chien, M. Yang, and J. Kautz, "Learning superpixels with segmentation-aware affinity loss," in 31st Conference on Computer Vision and Pattern Recognition (CVPR), 2018, pp. 568–576.
L. Xu, L. Zeng, and Z. Wang, "Saliency-based superpixels," Signal, Image and Video Processing, vol. 8, no. 1, pp. 181–190, 2014.
T. Zhao and X. Wu, "Pyramid feature attention network for saliency detection," in 32nd Conference on Computer Vision and Pattern Recognition (CVPR), 2019, pp. 3080–3089.
T. Spina, P. Miranda, and A. Falcao, "Intelligent understanding of user interaction in image segmentation," International Journal of Pattern Recognition and Artificial Intelligence, vol. 26, no. 02, p. 1265001, 2012.
A. Falcao, B. Cunha, and R. Lotufo, "Design of connected operators using the image foresting transform," in Medical Imaging (SPIE-MI), vol. 4322, 2001, pp. 468–479.
A. Falcao, L. da Costa, and B. Cunha, "Multiscale skeletons by image foresting transform and its application to neuromorphometry," Pattern Recognition, vol. 35, no. 7, pp. 1571–1582, 2002.
A. Sousa, S. Martins, A. Falcao, F. Reis, E. Bagatin, and K. Irion, "Altis: A fast and automatic lung and trachea ct-image segmentation method," Medical Physics, vol. 46, no. 11, pp. 4970–4982, 2019.
A. Tavares, P. Miranda, T. Spina, and A. Falcao, "A supervoxel-based solution to resume segmentation for interactive correction by differential image foresting transforms," in 13th International Symposium on Mathematical Morphology (ISMM), 2017, pp. 107–118.
L. Mansilla, P. Miranda, and F. Cappabianco, "Image segmentation by image foresting transform with non-smooth connectivity functions," in 26th Conference on Graphics, Patterns and Images (SIBGRAPI), 2013, pp. 147–154.
J. Shi, Q. Yan, L. Xu, and J. Jia, "Hierarchical image saliency detection on extended cssd," Transactions on Pattern Analysis and Machine Intelligence, vol. 38, no. 4, pp. 717–729, 2015.
C. Yang, L. Zhang, H. Lu, X. Ruan, and M. Yang, "Saliency detection via graph-based manifold ranking," in 26th Conference on Computer Vision and Pattern Recognition (CVPR), 2013, pp. 3166–3173.
J. Papa, A. Falcao, and C. Suzuki, "Supervised pattern classification based on optimum-path forest," International Journal of Imaging Systems and Technology, vol. 19, no. 2, pp. 120–131, 2009.
P. Buyssens, I. Gardin, S. Ruan, and A. Elmoataz, "Eikonal-based region growing for efficient clustering," Image and Vision Computing, vol. 32, no. 12, pp. 1045–1054, 2014.
P. Neubert and P. Protzel, "Superpixel benchmark and comparison," in Forum Bildverarbeitung, vol. 6, 2012, pp. 1–12.
Y. Li and L. Shen, "Skin lesion analysis towards melanoma detection using deep learning network," Sensors, vol. 18, no. 2, p. 556, 2018.
J. Zhou, J. Ruan, C. Wu, G. Ye, Z. Zhu, J. Yue, and Y. Zhang, "Super- pixel segmentation of breast cancer pathology images based on features extracted from the autoencoder," in 11th International Conference on Communication Software and Networks (ICCSN), 2019, pp. 366–370.
M. Liu, O. Tuzel, S. Ramalingam, and R. Chellappa, "Entropy rate superpixel segmentation," in 24th Conference on Computer Vision and Pattern Recognition (CVPR), 2011, pp. 2097–2104.
S. Zhang, H. Wang, W. Huang, and Z. You, "Plant diseased leaf segmentation and recognition by fusion of superpixel, K-means and PHOG," Optik, vol. 157, pp. 866–872, 2018.
W. Wu, A. Chen, L. Zhao, and J. Corso, "Brain tumor detection and segmentation in a CRF (conditional random fields) framework with pixel-pairwise affinity and superpixel-level features," International Journal of Computer Assisted Radiology and Surgery, vol. 9, no. 2, pp. 241–253, 2014.
J. Vargas-Múnoz, A. Chowdhury, E. Alexandre, F. Galvão, P. Miranda, and A. Falcão, "An iterative spanning forest framework for superpixel segmentation," Transactions on Image Processing, vol. 28, no. 7, pp. 3477–3489, 2019.
F. Belém, S. Guimarães, and A. Falcão, "Superpixel segmentation by object-based iterative spanning forest," in 23rd Iberoamerican Congress on Pattern Recognition, 2018, pp. 334–341.
F. Belém, L. Melo, S. Guimarães, and A. Falcão, "The importance of object-based seed sampling for superpixel segmentation," in 32nd Conference on Graphics, Patterns and Images (SIBGRAPI), 2019, pp. 108–115.
A. Schick, M. Fischer, and R. Stiefelhagen, "An evaluation of the compactness of superpixels," Pattern Recognition Letters, vol. 43, pp. 71–80, 2014.
D. Stutz, A. Hermans, and B. Leibe, "Superpixels: An evaluation of the state-of-the-art," Computer Vision and Image Understanding, vol. 166, pp. 1–27, 2018.
L. Wan, X. Xu, Q. Zhao, and W. Feng, "Spherical superpixels: Bench- mark and evaluation," in 14th Asian Conference on Computer Vision (ACCV), 2018, pp. 703–717.
X. Ren and J. Malik, "Learning a classification model for segmentation," in 9th International Conference on Computer Vision (ICCV), vol. 1, 2003, pp. 10–17.
Z. Ren and G. Shakhnarovich, "Image segmentation by cascaded region agglomeration," in 26th Conference on Computer Vision and Pattern Recognition (CVPR), 2013, pp. 2011–2018.
R. Achanta, A. Shaji, K. Smith, A. Lucchi, P. Fua, and S. S¨usstrunk, "SLIC superpixels compared to state-of-the-art superpixel methods," Transactions on Pattern Analysis and Machine Intelligence, vol. 34, no. 11, pp. 2274–2282, 2012.
R. Achanta and S. S¨usstrunk, "Superpixels and polygons using simple non-iterative clustering," in 30th Conference on Computer Vision and Pattern Recognition (CVPR), 2017, pp. 4651–4660.
Y. Liu, M. Yu, B. Li, and Y. He, "Intrinsic manifold SLIC: A simple and efficient method for computing content-sensitive superpixels," Transac- tions on Pattern Analysis and Machine Intelligence, vol. 40, no. 3, pp. 653–666, 2018.
A. Falcão, J. Stolfi, and R. Lotufo, "The image foresting transform: Theory, algorithms, and applications," Transactions on Pattern Analysis and Machine Intelligence, vol. 26, no. 1, pp. 19–29, 2004.
F. Galvão, A. Falcão, and A. Chowdhury, "RISF: recursive iterative spanning forest for superpixel segmentation," in 31st Conference on Graphics, Patterns and Images (SIBGRAPI), 2018, pp. 408–415.
C. Castelo-Fernandez and A. Falcão, "Learning visual dictionaries from class-specific superpixel segmentation," in 18th Conference on Computer Analysis of Images and Patterns (CAIP), 2019, pp. 171–182.
S. Martins, G. Ruppert, F. Reis, C. Yasuda, and A. Falcao, "A supervoxel-based approach for unsupervised abnormal asymmetry detection in MR images of the brain," in 16th International Symposium on Biomedical Imaging (ISBI), 2019, pp. 882–885.
M. Awaisu, L. Li, J. Peng, and J. Zhang, "Fast superpixel segmentation with deep features," in 36th Computer Graphics International Conference (CGI), 2019, pp. 410–416.
V. Jampani, D. Sun, M. Liu, M. Yang, and J. Kautz, "Superpixel sampling networks," in 18th European Conference on Computer Vision (ECCV), 2018, pp. 352–368.
W. Tu, M. Liu, V. Jampani, D. Sun, S. Chien, M. Yang, and J. Kautz, "Learning superpixels with segmentation-aware affinity loss," in 31st Conference on Computer Vision and Pattern Recognition (CVPR), 2018, pp. 568–576.
L. Xu, L. Zeng, and Z. Wang, "Saliency-based superpixels," Signal, Image and Video Processing, vol. 8, no. 1, pp. 181–190, 2014.
T. Zhao and X. Wu, "Pyramid feature attention network for saliency detection," in 32nd Conference on Computer Vision and Pattern Recognition (CVPR), 2019, pp. 3080–3089.
T. Spina, P. Miranda, and A. Falcao, "Intelligent understanding of user interaction in image segmentation," International Journal of Pattern Recognition and Artificial Intelligence, vol. 26, no. 02, p. 1265001, 2012.
A. Falcao, B. Cunha, and R. Lotufo, "Design of connected operators using the image foresting transform," in Medical Imaging (SPIE-MI), vol. 4322, 2001, pp. 468–479.
A. Falcao, L. da Costa, and B. Cunha, "Multiscale skeletons by image foresting transform and its application to neuromorphometry," Pattern Recognition, vol. 35, no. 7, pp. 1571–1582, 2002.
A. Sousa, S. Martins, A. Falcao, F. Reis, E. Bagatin, and K. Irion, "Altis: A fast and automatic lung and trachea ct-image segmentation method," Medical Physics, vol. 46, no. 11, pp. 4970–4982, 2019.
A. Tavares, P. Miranda, T. Spina, and A. Falcao, "A supervoxel-based solution to resume segmentation for interactive correction by differential image foresting transforms," in 13th International Symposium on Mathematical Morphology (ISMM), 2017, pp. 107–118.
L. Mansilla, P. Miranda, and F. Cappabianco, "Image segmentation by image foresting transform with non-smooth connectivity functions," in 26th Conference on Graphics, Patterns and Images (SIBGRAPI), 2013, pp. 147–154.
J. Shi, Q. Yan, L. Xu, and J. Jia, "Hierarchical image saliency detection on extended cssd," Transactions on Pattern Analysis and Machine Intelligence, vol. 38, no. 4, pp. 717–729, 2015.
C. Yang, L. Zhang, H. Lu, X. Ruan, and M. Yang, "Saliency detection via graph-based manifold ranking," in 26th Conference on Computer Vision and Pattern Recognition (CVPR), 2013, pp. 3166–3173.
J. Papa, A. Falcao, and C. Suzuki, "Supervised pattern classification based on optimum-path forest," International Journal of Imaging Systems and Technology, vol. 19, no. 2, pp. 120–131, 2009.
P. Buyssens, I. Gardin, S. Ruan, and A. Elmoataz, "Eikonal-based region growing for efficient clustering," Image and Vision Computing, vol. 32, no. 12, pp. 1045–1054, 2014.
P. Neubert and P. Protzel, "Superpixel benchmark and comparison," in Forum Bildverarbeitung, vol. 6, 2012, pp. 1–12.
Publicado
07/11/2020
Como Citar
BELÉM, Felipe C.; FALCÃO, Alexandre X.; GUIMARÃES, Silvio Jamil F..
Superpixel Generation by the Iterative Spanning Forest Using Object Information. In: WORKSHOP DE TESES E DISSERTAÇÕES - CONFERENCE ON GRAPHICS, PATTERNS AND IMAGES (SIBGRAPI), 33. , 2020, Evento Online.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2020
.
p. 22-28.
DOI: https://doi.org/10.5753/sibgrapi.est.2020.12979.
