Evaluation of machine learning applied to the realignment of hierarchies for image segmentation
Resumo
A hierarchical image segmentation is a set of image segmentations at different detail levels. However, objects can be located at different scales due to their size differences or to their distinct distances from the camera. In literature, many works have been developed to improve hierarchical image segmentation results. One possible solution is to realign the hierarchy such that every region containing an object (or its parts) is at the same level. In this work, we have explored the use of random forest and artificial neural network as regressors models to predict score values for regions belonging to a hierarchy of partitions, which are used to realign it. We have also proposed a new score calculation witch considering all user-defined segmentations that exist in the ground-truth. Experimental results are presented for two different hierarchical segmentation methods. Moreover, an analysis of the adoption of different combination of mid-level features to describe regions and different architectures from random forest and artificial neural network to train regressors models. Experimental results have point out that the use of new proposed score was able to improve final segmentation results.
Referências
S. J. F. Guimarães, Y. Kenmochi, J. Cousty, Z. K. G. P. Jr, and L. Naj-man, “Hierarchizing graph-based image segmentation algorithms relying on region dissimilarity: the case of the Felzenszwalb-Huttenlocher method,” Mathematical Morphology - Theory and Applications, vol. 2, pp. 55–75, 2017. https://doi.org/10.1515/mathm-2017-0004
P. Arbelaez, M. Maire, C. Fowlkes, and J. Malik, “Contour detection and hierarchical image segmentation,” IEEE TPAMI, vol. 33, no. 5, pp. 898–916, 2011. https://doi.org/10.1109/TPAMI.2010.161
L. S. Belo, C. A. Caetano, Z. K. G. Patrocnio Jr, and S. J. F. Guimares, “Summarizing video sequence using a graph-based hierarchical approach,” Neurocomputing, vol. 173, pp. 1001–1016, 2016. https://doi.org/10.1016/j.neucom.2015.08.057
F. Rodrigues, P. Leal, Y. Kenmochi, J. Cousty, L. Najman, S. Guimarães, and Z. Patrocnio Jr, “Graph-based hierarchical video cosegmentation,” in Image Analysis and Processing - ICIAP 2017, S. Battiato, G. Gallo, R. Schettini, and F. Stanco, Eds., 2017, pp. 15–26. https://doi.org/10.1007/978-3-319-68560-1_2
K. J. F. Souza, A. A. Arajo, Z. K. G. Patrocnio Jr, and S. J. F. Guimares, “Graph-based hierarchical video segmentation based on a simple dissimilarity measure,” PRL, vol. 47, pp. 85–92, 2014. https://doi.org/10.1016/j.patrec.2014.02.016
P. Arbelaez, “Boundary extraction in natural images using ultrametric contour maps,” in Computer Vision and Pattern Recognition Workshop, 2006. CVPRW’06. Conference on. IEEE, 2006, pp. 182–182. https://doi.org/10.1109/cvprw.2006.48
J. Cousty, L. Najman, Y. Kenmochi, and S. J. F. Guimarães, “Hierarchical segmentations with graphs: Quasi-flat zones, minimum spanning trees, and saliency maps,” Journal of Mathematical Imaging and Vision, vol. 60, no. 4, pp. 479–502, May 2018. https://doi.org/10.1007/s10851-017-0768-7
L. C. Chen, Y. Yang, J. Wang, W. Xu, and A. L. Yuille, “Attention to scale: Scale-aware semantic image segmentation,” in IEEE CVPR 2016, 2016, pp. 3640–3649. https://doi.org/10.1109/CVPR.2016.396
Z. Hao, Y. Liu, H. Qin, J. Yan, X. Li, and X. Hu, “Scale-aware face detection,” in IEEE CVPR 2017, 2017, pp. 1913–1922. https://doi.org/10.1109/cvpr.2017.207
Z. Jie, X. Liang, J. Feng, W. F. Lu, E. H. F. Tay, and S. Yan, “Scale-aware pixelwise object proposal networks,” IEEE TIP, vol. 25, no. 10, pp. 4525–4539, 2016. https://doi.org/10.1109/TIP.2016.2593342
J. Li, X. Liang, S. Shen, T. Xu, J. Feng, and S. Yan, “Scale-aware fast r-cnn for pedestrian detection,” IEEE Transactions on Multimedia, vol. 20, no. 4, pp. 985–996, 2018. https://doi.org/10.1109/tmm.2017.2759508
L. C. Chen, G. Papandreou, I. Kokkinos, K. Murphy, and A. L. Yuille, “Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs,” IEEE TPAMI, vol. 40, no. 4, pp. 834–848, 2018. https://doi.org/10.1109/TPAMI.2017.2699184
Y. Chen, D. Dai, J. Pont-Tuset, and L. Van Gool, “Scale-aware alignment of hierarchical image segmentation,” in IEEE CVPR 2016, 2016, pp. 364–372. https://doi.org/10.1109/cvpr.2016.46
R. C. Gonzalez and R. E. Woods, Digital Image Processing. Prentice-Hall, Incg, 2001.
D. R. Martin, C. C. Fowlkes, and J. Malik, “Learning to detect natural image boundaries using local brightness, color, and texture cues,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 26, no. 5, pp. 530–549, May 2004. https://doi.org/10.1109/tpami.2004.1273918
S. J. F. Guimarães, J. Cousty, Y. Kenmochi, and L. Najman, “A hierarchical image segmentation algorithm based on an observation scale,” in SSPR/SPR. Kluwer Academic Publishers, 2012, pp. 116–125. https://doi.org/10.1007/978-3-642-34166-3_13
S. Russell, S. Russell, P. Norvig, and E. Davis, Artificial Intelligence, ser. Prentice Hall series in artificial intelligence. Prentice Hall, 2010.
L. I. Kuncheva, Combining Pattern Classifiers: Methods and Algorithms, 2nd ed. Wiley Publishing, 2014. https://doi.org/10.1002/0471660264
K. Fu, Syntactic pattern recognition and applications, ser. Prentice-Hall advanced reference series: Computer science. Prentice-Hall, 1982.
C. Xu, S. Whitt, and J. J. Corso, “Flattening supervoxel hierarchies by the uniform entropy slice,” in IEEE ICCV 2013, 2013, pp. 2240–2247. https://doi.org/10.1109/iccv.2013.279
J. Cousty and L. Najman, “Morphological floodings and optimal cuts in hierarchies,” in IEEE ICIP 2014, 2014, pp. 4462–4466. https://doi.org/10.1109/icip.2014.7025905
L. Guiges, J. Cocquerez, and H. L. Men, “Scale-sets image analysis.” IJCV, vol. 68, no. 3, pp. 289–317, 2006. https://doi.org/10.1007/s11263-005-6299-0
M. M. Adão, S. J. F. Guimarães, and Z. K. G. Patrocı́nio, “Evaluation of scale-aware realignments of hierarchical image segmentation,” in Progress in Pattern Recognition, Image Analysis, Computer Vision, and Applications – CIARP 2018, R. Vera-Rodriguez, J. Fierrez, and A. Morales, Eds. Springer International Publishing, 2019, pp. 141–149. https://doi.org/10.1007/978-3-030-13469-3_17
J. Carreira and C. Sminchisescu, “Constrained parametric min-cuts for automatic object segmentation,” in IEEE CVPR 2010. IEEE, 2010, pp. 3241–3248. https://doi.org/10.1109/cvpr.2010.5540063
J. Pont-Tuset and F. Marques, “Supervised evaluation of image segmentation and object proposal techniques,” IEEE TPAMI, vol. 38, no. 7, pp. 1465–1478, 2016. https://doi.org/10.1109/tpami.2015.2481406
P. Arbelaez, M. Maire, C. Fowlkes, and J. Malik, “Contour detection and hierarchical image segmentation,” IEEE TPAMI, vol. 33, no. 5, pp. 898–916, 2011. https://doi.org/10.1109/TPAMI.2010.161
L. S. Belo, C. A. Caetano, Z. K. G. Patrocnio Jr, and S. J. F. Guimares, “Summarizing video sequence using a graph-based hierarchical approach,” Neurocomputing, vol. 173, pp. 1001–1016, 2016. https://doi.org/10.1016/j.neucom.2015.08.057
F. Rodrigues, P. Leal, Y. Kenmochi, J. Cousty, L. Najman, S. Guimarães, and Z. Patrocnio Jr, “Graph-based hierarchical video cosegmentation,” in Image Analysis and Processing - ICIAP 2017, S. Battiato, G. Gallo, R. Schettini, and F. Stanco, Eds., 2017, pp. 15–26. https://doi.org/10.1007/978-3-319-68560-1_2
K. J. F. Souza, A. A. Arajo, Z. K. G. Patrocnio Jr, and S. J. F. Guimares, “Graph-based hierarchical video segmentation based on a simple dissimilarity measure,” PRL, vol. 47, pp. 85–92, 2014. https://doi.org/10.1016/j.patrec.2014.02.016
P. Arbelaez, “Boundary extraction in natural images using ultrametric contour maps,” in Computer Vision and Pattern Recognition Workshop, 2006. CVPRW’06. Conference on. IEEE, 2006, pp. 182–182. https://doi.org/10.1109/cvprw.2006.48
J. Cousty, L. Najman, Y. Kenmochi, and S. J. F. Guimarães, “Hierarchical segmentations with graphs: Quasi-flat zones, minimum spanning trees, and saliency maps,” Journal of Mathematical Imaging and Vision, vol. 60, no. 4, pp. 479–502, May 2018. https://doi.org/10.1007/s10851-017-0768-7
L. C. Chen, Y. Yang, J. Wang, W. Xu, and A. L. Yuille, “Attention to scale: Scale-aware semantic image segmentation,” in IEEE CVPR 2016, 2016, pp. 3640–3649. https://doi.org/10.1109/CVPR.2016.396
Z. Hao, Y. Liu, H. Qin, J. Yan, X. Li, and X. Hu, “Scale-aware face detection,” in IEEE CVPR 2017, 2017, pp. 1913–1922. https://doi.org/10.1109/cvpr.2017.207
Z. Jie, X. Liang, J. Feng, W. F. Lu, E. H. F. Tay, and S. Yan, “Scale-aware pixelwise object proposal networks,” IEEE TIP, vol. 25, no. 10, pp. 4525–4539, 2016. https://doi.org/10.1109/TIP.2016.2593342
J. Li, X. Liang, S. Shen, T. Xu, J. Feng, and S. Yan, “Scale-aware fast r-cnn for pedestrian detection,” IEEE Transactions on Multimedia, vol. 20, no. 4, pp. 985–996, 2018. https://doi.org/10.1109/tmm.2017.2759508
L. C. Chen, G. Papandreou, I. Kokkinos, K. Murphy, and A. L. Yuille, “Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs,” IEEE TPAMI, vol. 40, no. 4, pp. 834–848, 2018. https://doi.org/10.1109/TPAMI.2017.2699184
Y. Chen, D. Dai, J. Pont-Tuset, and L. Van Gool, “Scale-aware alignment of hierarchical image segmentation,” in IEEE CVPR 2016, 2016, pp. 364–372. https://doi.org/10.1109/cvpr.2016.46
R. C. Gonzalez and R. E. Woods, Digital Image Processing. Prentice-Hall, Incg, 2001.
D. R. Martin, C. C. Fowlkes, and J. Malik, “Learning to detect natural image boundaries using local brightness, color, and texture cues,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 26, no. 5, pp. 530–549, May 2004. https://doi.org/10.1109/tpami.2004.1273918
S. J. F. Guimarães, J. Cousty, Y. Kenmochi, and L. Najman, “A hierarchical image segmentation algorithm based on an observation scale,” in SSPR/SPR. Kluwer Academic Publishers, 2012, pp. 116–125. https://doi.org/10.1007/978-3-642-34166-3_13
S. Russell, S. Russell, P. Norvig, and E. Davis, Artificial Intelligence, ser. Prentice Hall series in artificial intelligence. Prentice Hall, 2010.
L. I. Kuncheva, Combining Pattern Classifiers: Methods and Algorithms, 2nd ed. Wiley Publishing, 2014. https://doi.org/10.1002/0471660264
K. Fu, Syntactic pattern recognition and applications, ser. Prentice-Hall advanced reference series: Computer science. Prentice-Hall, 1982.
C. Xu, S. Whitt, and J. J. Corso, “Flattening supervoxel hierarchies by the uniform entropy slice,” in IEEE ICCV 2013, 2013, pp. 2240–2247. https://doi.org/10.1109/iccv.2013.279
J. Cousty and L. Najman, “Morphological floodings and optimal cuts in hierarchies,” in IEEE ICIP 2014, 2014, pp. 4462–4466. https://doi.org/10.1109/icip.2014.7025905
L. Guiges, J. Cocquerez, and H. L. Men, “Scale-sets image analysis.” IJCV, vol. 68, no. 3, pp. 289–317, 2006. https://doi.org/10.1007/s11263-005-6299-0
M. M. Adão, S. J. F. Guimarães, and Z. K. G. Patrocı́nio, “Evaluation of scale-aware realignments of hierarchical image segmentation,” in Progress in Pattern Recognition, Image Analysis, Computer Vision, and Applications – CIARP 2018, R. Vera-Rodriguez, J. Fierrez, and A. Morales, Eds. Springer International Publishing, 2019, pp. 141–149. https://doi.org/10.1007/978-3-030-13469-3_17
J. Carreira and C. Sminchisescu, “Constrained parametric min-cuts for automatic object segmentation,” in IEEE CVPR 2010. IEEE, 2010, pp. 3241–3248. https://doi.org/10.1109/cvpr.2010.5540063
J. Pont-Tuset and F. Marques, “Supervised evaluation of image segmentation and object proposal techniques,” IEEE TPAMI, vol. 38, no. 7, pp. 1465–1478, 2016. https://doi.org/10.1109/tpami.2015.2481406
Publicado
28/10/2019
Como Citar
ADÃO, Milena Menezes; GUIMARÃES, Silvio Jamil F.; PATROCÍNIO JR., Zenilton K. G..
Evaluation of machine learning applied to the realignment of hierarchies for image segmentation. In: WORKSHOP DE TESES E DISSERTAÇÕES - CONFERENCE ON GRAPHICS, PATTERNS AND IMAGES (SIBGRAPI), 32. , 2019, Rio de Janeiro.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2019
.
p. 119-125.
DOI: https://doi.org/10.5753/sibgrapi.est.2019.8311.
