An Efficient Hierarchical Layered Graph Approach for Multi-Region Segmentation
Resumo
We proposed a novel efficient seed-based method for the multiple region segmentation of images based on graphs, named Hierarchical Layered Oriented Image Foresting Transform (HLOIFT). It uses a tree of the relations between the image objects, represented by a node. Each tree node may contain different individual high-level priors and defines a weighted digraph, named as layer. The layer graphs are then integrated into a hierarchical graph, considering the hierarchical relations of inclusion and exclusion. A single energy optimization is performed in the hierarchical layered weighted digraph leading to globally optimal results satisfying all the high-level priors. The experimental evaluations of HLOIFT and its extensions, on medical, natural and synthetic images, indicate promising results comparable to the state-of-the-art methods, but with lower computational complexity. Compared to hierarchical segmentation by the min-cut/max-flow algorithm, our approach is less restrictive, leading to globally optimal results in more general scenarios, and has a better running time.
Referências
K. D. Toennies, Guide to medical image analysis: methods and algorithms. Springer Science & Business Media, 2012. https://doi.org/10.1007/978-1-4471-2751-2
E. Visser, M. C. Keuken, G. Douaud, V. Gaura, A.-C. Bachoud-Levi, P. Remy, B. U. Forstmann, and M. Jenkinson, “Automatic segmentation of the striatum and globus pallidus using mist: Multimodal image segmentation tool,” NeuroImage, vol. 125, pp. 479 – 497, 2016. https://doi.org/10.1016/j.neuroimage.2015.10.013
M. Jackowski and A. Goshtasby, A Computer-Aided Design System for Segmentation of Volumetric Images. Boston, MA: Springer US, 2005, pp. 251–272. https://doi.org/10.1007/0-306-48608-3_7
S. Golodetz, I. Voiculescu, and S. Cameron, “Simpler editing of graph-based segmentation hierarchies using zipping algorithms,” Pattern Recognition, vol. 70, pp. 44–59, 2017. https://doi.org/10.1016/j.patcog.2017.04.007
J. Cousty, G. Bertrand, L. Najman, and M. Couprie, “Watershed cuts: Thinnings, shortest path forests, and topological watersheds,” PAMI, IEEE Transactions on, vol. 32, no. 5, pp. 925–939, 2010. https://doi.org/10.1109/TPAMI.2009.71
L. Grady, “Random walks for image segmentation,” PAMI, IEEE Transactions on, vol. 28, no. 11, pp. 1768–1783, 2006. https://doi.org/10.1109/TPAMI.2006.233
K. Ciesielski, J. Udupa, P. Saha, and Y. Zhuge, “Iterative relative fuzzy connectedness for multiple objects with multiple seeds,” Computer Vision and Image Understanding, vol. 107, no. 3, pp. 160–182, 2007. https://doi.org/10.1016/j.cviu.2006.10.005
Y. Boykov and G. Funka-Lea, “Graph cuts and efficient ND image segmentation,” International journal of computer vision, vol. 70, no. 2, pp. 109–131, 2006. https://doi.org/10.1007/s11263-006-7934-5
X. Li, J. Chen, and H. Fan, “Interactive image segmentation based on grow cut of two scale graphs,” in Advances on Digital Television and Wireless Multimedia Communications, W. Zhang, X. Yang, Z. Xu, P. An, Q. Liu, and Y. Lu, Eds. Springer Berlin Heidelberg, 2012, pp. 90–95. https://doi.org/10.1007/978-3-642-34595-1_13
K. C. Ciesielski, R. Strand, F. Malmberg, and P. K. Saha, “Efficient algorithm for finding the exact minimum barrier distance,” Computer Vision and Image Understanding, vol. 123, pp. 53 – 64, 2014. https://doi.org/10.1016/j.cviu.2014.03.007
A. X. Falcão, J. Stolfi, and R. de Alencar Lotufo, “The image foresting transform: Theory, algorithms, and applications,” PAMI, IEEE Transactions on, vol. 26, no. 1, pp. 19–29, 2004. https://doi.org/10.1109/TPAMI.2004.1261076
K. C. Ciesielski, A. X. Falcão, and P. A. V. Miranda, “Path-value functions for which dijkstra’s algorithm returns optimal mapping,” Journal of Mathematical Imaging and Vision, Feb 2018. https://doi.org/10.1007/s10851-018-0793-1
H. Isack, O. Veksler, M. Sonka, and Y. Boykov, “Hedgehog shape priors for multi-object segmentation,” in Proceedings of the IEEE Conference on CVPR, 2016, pp. 2434–2442. https://doi.org/10.1109/CVPR.2016.267
V. Gulshan, C. Rother, A. Criminisi, A. Blake, and A. Zisserman, “Geodesic star convexity for interactive image segmentation,” in Pro-ceedings of CVPR, 2010, pp. 3129–3136. https://doi.org/10.1109/CVPR.2010.5540073
O. Veksler, “Star shape prior for graph-cut image segmentation,” Computer Vision–ECCV 2008, pp. 454–467, 2008. https://doi.org/10.1007/978-3-540-88690-7_34
L. Gorelick, O. Veksler, Y. Boykov, and C. Nieuwenhuis, “Convexity shape prior for binary segmentation,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 39, no. 2, pp. 258–271, 2017. https://doi.org/10.1109/TPAMI.2016.2547399
D. Singaraju, L. Grady, and R. Vidal, “Interactive image segmentation via minimization of quadratic energies on directed graphs,” in CVPR, 2008. CVPR 2008. IEEE Conference on. IEEE, 2008, pp. 1–8. https://doi.org/10.1109/CVPR.2008.4587485
P. A. Miranda and L. A. Mansilla, “Oriented image foresting transform segmentation by seed competition,” Image Processing, IEEE Transactions on, vol. 23, no. 1, pp. 389–398, 2014. https://doi.org/10.1109/TIP.2013.2288867
A. Delong and Y. Boykov, “Globally optimal segmentation of multiregion objects,” in Computer Vision, 2009 IEEE 12th International Conference on. IEEE, 2009, pp. 285–292. https://doi.org/10.1109/ICCV.2009.5459263
A. Delong, L. Gorelick, O. Veksler, and Y. Boykov, “Minimizing energies with hierarchical costs,” International journal of computer vision, vol. 100, no. 1, pp. 38–58, 2012. https://doi.org/10.1007/s11263-012-0531-x
J. Ulén, P. Strandmark, and F. Kahl, “An efficient optimization framework for multi-region segmentation based on lagrangian duality,” Medical Imaging, IEEE Transactions on, vol. 32, no. 2, pp. 178–188, 2013. https://doi.org/10.1109/TMI.2012.2218117
Y. Yin, X. Zhang, R. Williams, X. Wu, D. D. Anderson, and M. Sonka, “LOGISMOS–layered optimal graph image segmentation of multiple objects and surfaces: cartilage segmentation in the knee joint,” IEEE transactions on medical imaging, vol. 29, no. 12, pp. 2023–2037, 2010. https://doi.org/10.1109/TMI.2010.2058861
I. Oguz and M. Sonka, “LOGISMOS–B: layered optimal graph image segmentation of multiple objects and surfaces for the brain,” IEEE transactions on medical imaging, vol. 33, no. 6, pp. 1220–1235, 2014. https://doi.org/10.1109/TMI.2014.2304499
P. A. V. Miranda, A. X. Falcao, and J. K. Udupa, “Cloud bank: A multiple clouds model and its use in mr brain image segmentation,”in 2009 IEEE International Symposium on Biomedical Imaging: From Nano to Macro, June 2009, pp. 506–509. https://doi.org/10.1109/ISBI.2009.5193095
J. K. Udupa, D. Odhner, Y. Tong, M. M. S. Matsumoto, K. C. Ciesielski, P. Vaideeswaran, V. Ciesielski, B. Saboury, L. Zhao, S. Mohammadianrasanani, and D. Torigian, “Fuzzy model-based body-wide anatomy recognition in medical images,” 2013. https://doi.org/10.1117/12.2007983
L. Rittner, J. K. Udupa, and D. A. Torigian, “Multiple fuzzy object modeling improves sensitivity in automatic anatomy recognition,” in In Proceedings of SPIE on Medical Imaging: Image Processing, 2014. https://doi.org/10.1117/12.2044297
K. Sun, J. K. Udupa, D. Odhner, Y. Tong, and D. A. Torigian, “Automatic thoracic anatomy segmentation on ct images using hierarchical fuzzy models and registration,” in In Proceedings of SPIE on Medical Imaging: Image-Guided Procedures, Robotic Interventions, and Modeling, 2014. https://doi.org/10.1118/1.4942486
J. K. Udupa, D. Odhner, L. Zhao, Y. Tong, M. M. Matsumoto, K. C. Ciesielski, A. X. Falcao, P. Vaideeswaran, V. Ciesielski, B. Saboury, S. Mohammadianrasanani, S. Sin, R. Arens, and D. A. Torigian, “Body-wide hierarchical fuzzy modeling, recognition, and delineation of anatomy in medical images,” Medical Image Analysis, vol. 18, no. 5, pp. 752 – 771, 2014. https://doi.org/10.1016/j.media.2014.04.003
S. Mohammadianrasanani, “The use of a body-wide automatic anatomy recognition system in image analysis of kidneys,” Master’s thesis, School of Technology and Health, Royal Institute of Technology, 2013.
Y. Tong, J. K. Udupa, D. Odhner, S. Sin, and R. Arens, “Abdominal adiposity quantification at MRI via fuzzy model-based anatomy recognition,” in In Proceedings of SPIE on Medical Imaging: Biomedical Applications in Molecular, Structural, and Functional Imaging, 2013. https://doi.org/10.1117/12.2007938
L. M. C. Leon and P. A. V. D. Miranda, “Multi-object segmentation by hierarchical layered oriented image foresting transform,” in 2017 30th SIBGRAPI Conference, Oct 2017, pp. 79–86. https://doi.org/10.1109/SIBGRAPI.2017.17
L. M. C. Leon, K. Ciesielski, and P. A. V. Miranda, “Efficient hierarchical multi-object segmentation in layered graph (submitted),” p. https://www.math.wvu.edu/ kcies/SubmittedPapers/SS29.HLOIFT.pdf.
L. C. Leon and P. V. de Miranda, “Efficient interactive multi-object segmentation in medical images,” in Computer Vision – ECCV 2018 Workshops. Springer International Publishing, 2019, pp. 705–710. https://doi.org/10.1007/978-3-030-11018-5_61
L. M. C. Leon and P. A. V. Miranda, “A hierarchical layered graph approach for multi-label segmentation in 2d medical images,” 2018.
L. M. C. Leon, K. Ciesielski, and P. A. V. Miranda, “Extensions of the hierarchical graph approach in multi-region segmentation,” LatinX in AI Workshop @ ICML 2019, Long Beach, United States.
L. M. C. Leon, K. Ciesielski, and P. A. V. Miranda, “Extensions of the hierarchical graph approach in multi-region segmentation,” Women in Computer Vision Workshop @ CVPR 2019, Long Beach, United States.
E. W. Dijkstra, “A note on two problems in connexion with graphs,” NUMERISCHE MATHEMATIK, vol. 1, no. 1, pp. 269–271, 1959. https://doi.org/10.1007/BF01386390
P. de Miranda, A. Falcão, and J. Udupa, “Synergistic arc-weight estimation for interactive image segmentation using graphs,” Computer Vision and Image Understanding, vol. 114, no. 1, pp. 85 – 99, 2010. https://doi.org/10.1016/j.cviu.2009.08.001
K. C. Ciesielski and J. K. Udupa, “Affinity functions in fuzzy connectedness based image segmentation i: Equivalence of affinities,” Comput. Vis. Image Underst., vol. 114, no. 1, pp. 146–154, Jan. 2010. https://doi.org/10.1016/j.cviu.2009.09.006
L. Mansilla and P. Miranda, “Image segmentation by oriented image foresting transform with geodesic star convexity,” in CAIP, vol. 8047, York, UK, Aug 2013, pp. 572–579. https://doi.org/10.1007/978-3-642-40261-6_69
Y. Boykov and V. Kolmogorov, “An experimental comparison of mincut/max-flow algorithms for energy minimization in vision,” IEEE transactions on PAMI, vol. 26, no. 9, pp. 1124–1137, 2004. https://doi.org/10.1109/TPAMI.2004.60
E. B. Alexandre, A. S. Chowdhury, A. X. Falcao, and P. A. V. Miranda, “IFT-SLIC: A general framework for superpixel generation based on simple linear iterative clustering and image foresting transform,” in 2015 28th SIBGRAPI Conference, 2015, pp. 337–344. https://doi.org/10.1109/SIBGRAPI.2015.20
L. Soler, A. Hostettler, V. Agnus, A. Charnoz, J.-B. Fasquel, J. Moreau, A.-B. Osswald, M. Bouhadjar, and J. Marescaux, “3d image reconstruction for comparison of algorithm database : A patient-specific anatomical and medical image database,” 2012. [Online]. Available: https://www.ircad.fr/research/3d-ircadb-02/
P. A. Miranda and A. X. Falcão, “Links between image segmentation based on optimum-path forest and minimum cut in graph,” Journal of Mathematical Imaging and Vision, vol. 35, no. 2, pp. 128–142, 2009. https://doi.org/10.1007/s10851-009-0159-9
F. Malmberg, I. Nyström, A. Mehnert, C. Engstrom, and E. Bengtsson, “Relaxed image foresting transforms for interactive volume image segmentation,” in Proc.SPIE, vol. 7623, 2010, pp. 7623 – 7623 – 11. https://doi.org/10.1117/12.840019