Robust Point-Cloud Registration based on Dense Point Matching and Probabilistic Modeling
Resumo
We present techniques for 3D point-cloud registration that are suited for scenarios where robustness to outliers and missing regions is necessary, besides being applicable to both rigid and non-rigid configurations. Our techniques exploit advantages from deep learning models for dense point matching and from recent advances in probabilistic modeling of point-cloud registration. Such a combination produces context awareness and resilience to outliers and missing information. We demonstrate their effectiveness by comparing them to state-of-the-art methods and showing that ours achieves superior results in general. For example, our approaches achieve registration error up to 45% smaller than these methods in partial point clouds for non-rigid registration, and up to 49% smaller on rigid registration.Referências
G. Bresson, Z. Alsayed, L. Yu, and S. Glaser, “Simultaneous localization and mapping: A survey of current trends in autonomous driving,” IEEE Transactions on Intelligent Vehicles, vol. 2, no. 3, pp. 194–220, 2017.
M. Naseer, S. Khan, and F. Porikli, “Indoor scene understanding in 2.5/3d for autonomous agents: A survey,” IEEE access, vol. 7, pp. 1859–1887, 2018.
M. Berger, A. Tagliasacchi, L. M. Seversky, P. Alliez, G. Guennebaud, J. A. Levine, A. Sharf, and C. T. Silva, “A survey of surface reconstruction from point clouds,” in Comput. Graph. Forum, vol. 36, no. 1. Wiley Online Library, 2017, pp. 301–329.
Z. Wang, Y. Wu, and Q. Niu, “Multi-sensor fusion in automated driving: A survey,” IEEE Access, vol. 8, pp. 2847–2868, 2019.
J. Liu, H. Li, R. Wu, Q. Zhao, Y. Guo, and L. Chen, “A survey on deep learning methods for scene flow estimation,” Pattern Recognition, vol. 106, p. 107378, 2020.
P. J. Besl and N. D. McKay, “Method for registration of 3-d shapes,” in Sensor fusion IV: control paradigms and data structures, vol. 1611. International Society for Optics and Photonics, 1992, pp. 586–606.
W. Feng, J. Zhang, H. Cai, H. Xu, J. Hou, and H. Bao, “Recurrent multi-view alignment network for unsupervised surface registration,” in CVPR, 2021, pp. 10 297–10 307.
M. Eisenberger, D. Novotny, G. Kerchenbaum, P. Labatut, N. Neverova, D. Cremers, and A. Vedaldi, “Neuromorph: Unsupervised shape interpolation and correspondence in one go,” in CVPR, 2021, pp. 7473–7483.
O. Hirose, “A bayesian formulation of coherent point drift,” IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020.
A. W. Fitzgibbon, “Robust registration of 2d and 3d point sets,” Image and vision computing, vol. 21, no. 13-14, pp. 1145–1153, 2003.
S. Granger and X. Pennec, “Multi-scale em-icp: A fast and robust approach for surface registration,” in European Conference on Computer Vision. Springer, 2002, pp. 418–432.
A. Myronenko and X. Song, “Point set registration: Coherent point drift,” IEEE TPAMI, vol. 32, no. 12, pp. 2262–2275, 2010.
W. Gao and R. Tedrake, “Filterreg: Robust and efficient probabilistic point-set registration using gaussian filter and twist parameterization,” in CVPR, 2019, pp. 11 095–11 104.
B. Eckart, K. Kim, and J. Kautz, “Fast and accurate point cloud registration using trees of gaussian mixtures,” arXiv preprint arXiv:1807.02587, 2018.
J. Yang, H. Li, D. Campbell, and Y. Jia, “Go-icp: A globally optimal solution to 3d icp point-set registration,” IEEE TPAMI.
F. Tombari, S. Salti, and L. Di Stefano, “Unique signatures of histograms for local surface description,” in ECCV. Springer, 2010, pp. 356–369.
F. Pomerleau, F. Colas, and R. Siegwart, “A review of point cloud registration algorithms for mobile robotics,” 2015.
Y. Wang and J. M. Solomon, “Deep closest point: Learning representations for point cloud registration,” in ICCV, 2019, pp. 3523–3532.
Z. J. Yew and G. H. Lee, “Rpm-net: Robust point matching using learned features,” in CVPR, 2020, pp. 11 824–11 833.
S. Huang, Z. Gojcic, M. Usvyatsov, A. Wieser, and K. Schindler, “Predator: Registration of 3d point clouds with low overlap,” in CVPR, 2021, pp. 4267–4276.
H. Thomas, C. R. Qi, J.-E. Deschaud, B. Marcotegui, F. Goulette, and L. J. Guibas, “Kpconv: Flexible and deformable convolution for point clouds,” in ICCV, 2019, pp. 6411–6420.
X. Bai, Z. Luo, L. Zhou, H. Fu, L. Quan, and C.-L. Tai, “D3feat: Joint learning of dense detection and description of 3d local features,” in CVPR, 2020, pp. 6359–6367.
C. Choy, J. Park, and V. Koltun, “Fully convolutional geometric features,” in ICCV, 2019, pp. 8958–8966.
K. Fu, S. Liu, X. Luo, and M. Wang, “Robust point cloud registration framework based on deep graph matching,” in CVPR, 2021, pp. 8893–8902.
B. Amberg, S. Romdhani, and T. Vetter, “Optimal step nonrigid icp algorithms for surface registration,” in CVPR. IEEE, 2007, pp. 1–8.
B. Jian and B. C. Vemuri, “Robust point set registration using gaussian mixture models,” IEEE transactions on pattern analysis and machine intelligence, vol. 33, no. 8, pp. 1633–1645, 2010.
J. Ma, J. Wu, J. Zhao, J. Jiang, H. Zhou, and Q. Z. Sheng, “Nonrigid point set registration with robust transformation learning under manifold regularization,” IEEE transactions on neural networks and learning systems, 2018.
X. Liu, C. R. Qi, and L. J. Guibas, “Flownet3d: Learning scene flow in 3d point clouds,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2019, pp. 529–537.
H. Mittal, B. Okorn, and D. Held, “Just go with the flow: Self-supervised scene flow estimation,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020, pp. 11 177–11 185.
I. Tishchenko, S. Lombardi, M. R. Oswald, and M. Pollefeys, “Selfsupervised learning of non-rigid residual flow and ego-motion,” in Int. Conf. on 3D Vision (3DV). IEEE, 2020, pp. 150–159.
W. Wu, Z. Y. Wang, Z. Li, W. Liu, and L. Fuxin, “Pointpwc-net: Cost volume on point clouds for (self-) supervised scene flow estimation,” in ECCV. Springer, 2020, pp. 88–107.
Y. Kittenplon, Y. C. Eldar, and D. Raviv, “Flowstep3d: Model unrolling for self-supervised scene flow estimation,” in CVPR, 2021, pp. 4114–4123.
G. Puy, A. Boulch, and R. Marlet, “Flot: Scene flow on point clouds guided by optimal transport,” in ECCV. Springer, 2020, pp. 527–544.
B. Ouyang and D. Raviv, “Occlusion guided scene flow estimation on 3d point clouds,” in CVPR, 2021, pp. 2805–2814.
L. Wang, X. Li, J. Chen, and Y. Fang, “Coherent point drift networks: Unsupervised learning of non-rigid point set registration,” arXiv preprint arXiv:1906.03039, 2019.
L. Wang, J. Chen, X. Li, and Y. Fang, “Non-rigid point set registration networks,” arXiv preprint arXiv:1904.01428, 2019.
Y. Sahillioğlu, “Recent advances in shape correspondence,” The Visual Computer, vol. 36, no. 8, pp. 1705–1721, 2020.
P.-E. Sarlin, D. DeTone, T. Malisiewicz, and A. Rabinovich, “Superglue: Learning feature matching with graph neural networks,” in CVPR, 2020, pp. 4938–4947.
Y. Wang, Y. Sun, Z. Liu, S. E. Sarma, M. M. Bronstein, and J. M. Solomon, “Dynamic graph cnn for learning on point clouds,” ACM TOG, vol. 38, no. 5, pp. 1–12, 2019.
G. Mena, J. Snoek, S. Linderman, and D. Belanger, “Learning latent permutations with gumbel-sinkhorn networks,” in ICLR 2018 Conference Track, vol. 2018. OpenReview, 2018.
A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury, G. Chanan, T. Killeen, Z. Lin, N. Gimelshein, L. Antiga et al., “Pytorch: An imperative style, high-performance deep learning library,” in NeurIPS, 2019, pp. 8026–8037.
A. Ranjan, T. Bolkart, S. Sanyal, and M. J. Black, “Generating 3D faces using convolutional mesh autoencoders,” in ECCV, 2018, pp. 725–741. [Online]. Available: http://coma.is.tue.mpg.de/
O. Hirose, “Acceleration of non-rigid point set registration with downsampling and gaussian process regression,” IEEE TPAMI, vol. 43, no. 8, pp. 2858–2865, 2020.
A. M. Bronstein, M. M. Bronstein, and R. Kimmel, “Calculus of nonrigid surfaces for geometry and texture manipulation,” IEEE TVCG, vol. 13, no. 5, pp. 902–913, 2007.
Z. Wu, S. Song, A. Khosla, F. Yu, L. Zhang, X. Tang, and J. Xiao, “3d shapenets: A deep representation for volumetric shapes,” in CVPR, 2015, pp. 1912–1920.
M. Naseer, S. Khan, and F. Porikli, “Indoor scene understanding in 2.5/3d for autonomous agents: A survey,” IEEE access, vol. 7, pp. 1859–1887, 2018.
M. Berger, A. Tagliasacchi, L. M. Seversky, P. Alliez, G. Guennebaud, J. A. Levine, A. Sharf, and C. T. Silva, “A survey of surface reconstruction from point clouds,” in Comput. Graph. Forum, vol. 36, no. 1. Wiley Online Library, 2017, pp. 301–329.
Z. Wang, Y. Wu, and Q. Niu, “Multi-sensor fusion in automated driving: A survey,” IEEE Access, vol. 8, pp. 2847–2868, 2019.
J. Liu, H. Li, R. Wu, Q. Zhao, Y. Guo, and L. Chen, “A survey on deep learning methods for scene flow estimation,” Pattern Recognition, vol. 106, p. 107378, 2020.
P. J. Besl and N. D. McKay, “Method for registration of 3-d shapes,” in Sensor fusion IV: control paradigms and data structures, vol. 1611. International Society for Optics and Photonics, 1992, pp. 586–606.
W. Feng, J. Zhang, H. Cai, H. Xu, J. Hou, and H. Bao, “Recurrent multi-view alignment network for unsupervised surface registration,” in CVPR, 2021, pp. 10 297–10 307.
M. Eisenberger, D. Novotny, G. Kerchenbaum, P. Labatut, N. Neverova, D. Cremers, and A. Vedaldi, “Neuromorph: Unsupervised shape interpolation and correspondence in one go,” in CVPR, 2021, pp. 7473–7483.
O. Hirose, “A bayesian formulation of coherent point drift,” IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020.
A. W. Fitzgibbon, “Robust registration of 2d and 3d point sets,” Image and vision computing, vol. 21, no. 13-14, pp. 1145–1153, 2003.
S. Granger and X. Pennec, “Multi-scale em-icp: A fast and robust approach for surface registration,” in European Conference on Computer Vision. Springer, 2002, pp. 418–432.
A. Myronenko and X. Song, “Point set registration: Coherent point drift,” IEEE TPAMI, vol. 32, no. 12, pp. 2262–2275, 2010.
W. Gao and R. Tedrake, “Filterreg: Robust and efficient probabilistic point-set registration using gaussian filter and twist parameterization,” in CVPR, 2019, pp. 11 095–11 104.
B. Eckart, K. Kim, and J. Kautz, “Fast and accurate point cloud registration using trees of gaussian mixtures,” arXiv preprint arXiv:1807.02587, 2018.
J. Yang, H. Li, D. Campbell, and Y. Jia, “Go-icp: A globally optimal solution to 3d icp point-set registration,” IEEE TPAMI.
F. Tombari, S. Salti, and L. Di Stefano, “Unique signatures of histograms for local surface description,” in ECCV. Springer, 2010, pp. 356–369.
F. Pomerleau, F. Colas, and R. Siegwart, “A review of point cloud registration algorithms for mobile robotics,” 2015.
Y. Wang and J. M. Solomon, “Deep closest point: Learning representations for point cloud registration,” in ICCV, 2019, pp. 3523–3532.
Z. J. Yew and G. H. Lee, “Rpm-net: Robust point matching using learned features,” in CVPR, 2020, pp. 11 824–11 833.
S. Huang, Z. Gojcic, M. Usvyatsov, A. Wieser, and K. Schindler, “Predator: Registration of 3d point clouds with low overlap,” in CVPR, 2021, pp. 4267–4276.
H. Thomas, C. R. Qi, J.-E. Deschaud, B. Marcotegui, F. Goulette, and L. J. Guibas, “Kpconv: Flexible and deformable convolution for point clouds,” in ICCV, 2019, pp. 6411–6420.
X. Bai, Z. Luo, L. Zhou, H. Fu, L. Quan, and C.-L. Tai, “D3feat: Joint learning of dense detection and description of 3d local features,” in CVPR, 2020, pp. 6359–6367.
C. Choy, J. Park, and V. Koltun, “Fully convolutional geometric features,” in ICCV, 2019, pp. 8958–8966.
K. Fu, S. Liu, X. Luo, and M. Wang, “Robust point cloud registration framework based on deep graph matching,” in CVPR, 2021, pp. 8893–8902.
B. Amberg, S. Romdhani, and T. Vetter, “Optimal step nonrigid icp algorithms for surface registration,” in CVPR. IEEE, 2007, pp. 1–8.
B. Jian and B. C. Vemuri, “Robust point set registration using gaussian mixture models,” IEEE transactions on pattern analysis and machine intelligence, vol. 33, no. 8, pp. 1633–1645, 2010.
J. Ma, J. Wu, J. Zhao, J. Jiang, H. Zhou, and Q. Z. Sheng, “Nonrigid point set registration with robust transformation learning under manifold regularization,” IEEE transactions on neural networks and learning systems, 2018.
X. Liu, C. R. Qi, and L. J. Guibas, “Flownet3d: Learning scene flow in 3d point clouds,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2019, pp. 529–537.
H. Mittal, B. Okorn, and D. Held, “Just go with the flow: Self-supervised scene flow estimation,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020, pp. 11 177–11 185.
I. Tishchenko, S. Lombardi, M. R. Oswald, and M. Pollefeys, “Selfsupervised learning of non-rigid residual flow and ego-motion,” in Int. Conf. on 3D Vision (3DV). IEEE, 2020, pp. 150–159.
W. Wu, Z. Y. Wang, Z. Li, W. Liu, and L. Fuxin, “Pointpwc-net: Cost volume on point clouds for (self-) supervised scene flow estimation,” in ECCV. Springer, 2020, pp. 88–107.
Y. Kittenplon, Y. C. Eldar, and D. Raviv, “Flowstep3d: Model unrolling for self-supervised scene flow estimation,” in CVPR, 2021, pp. 4114–4123.
G. Puy, A. Boulch, and R. Marlet, “Flot: Scene flow on point clouds guided by optimal transport,” in ECCV. Springer, 2020, pp. 527–544.
B. Ouyang and D. Raviv, “Occlusion guided scene flow estimation on 3d point clouds,” in CVPR, 2021, pp. 2805–2814.
L. Wang, X. Li, J. Chen, and Y. Fang, “Coherent point drift networks: Unsupervised learning of non-rigid point set registration,” arXiv preprint arXiv:1906.03039, 2019.
L. Wang, J. Chen, X. Li, and Y. Fang, “Non-rigid point set registration networks,” arXiv preprint arXiv:1904.01428, 2019.
Y. Sahillioğlu, “Recent advances in shape correspondence,” The Visual Computer, vol. 36, no. 8, pp. 1705–1721, 2020.
P.-E. Sarlin, D. DeTone, T. Malisiewicz, and A. Rabinovich, “Superglue: Learning feature matching with graph neural networks,” in CVPR, 2020, pp. 4938–4947.
Y. Wang, Y. Sun, Z. Liu, S. E. Sarma, M. M. Bronstein, and J. M. Solomon, “Dynamic graph cnn for learning on point clouds,” ACM TOG, vol. 38, no. 5, pp. 1–12, 2019.
G. Mena, J. Snoek, S. Linderman, and D. Belanger, “Learning latent permutations with gumbel-sinkhorn networks,” in ICLR 2018 Conference Track, vol. 2018. OpenReview, 2018.
A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury, G. Chanan, T. Killeen, Z. Lin, N. Gimelshein, L. Antiga et al., “Pytorch: An imperative style, high-performance deep learning library,” in NeurIPS, 2019, pp. 8026–8037.
A. Ranjan, T. Bolkart, S. Sanyal, and M. J. Black, “Generating 3D faces using convolutional mesh autoencoders,” in ECCV, 2018, pp. 725–741. [Online]. Available: http://coma.is.tue.mpg.de/
O. Hirose, “Acceleration of non-rigid point set registration with downsampling and gaussian process regression,” IEEE TPAMI, vol. 43, no. 8, pp. 2858–2865, 2020.
A. M. Bronstein, M. M. Bronstein, and R. Kimmel, “Calculus of nonrigid surfaces for geometry and texture manipulation,” IEEE TVCG, vol. 13, no. 5, pp. 902–913, 2007.
Z. Wu, S. Song, A. Khosla, F. Yu, L. Zhang, X. Tang, and J. Xiao, “3d shapenets: A deep representation for volumetric shapes,” in CVPR, 2015, pp. 1912–1920.
Publicado
06/11/2023
Como Citar
NETTO, Gustavo Marques; OLIVEIRA NETO, Manuel M. de.
Robust Point-Cloud Registration based on Dense Point Matching and Probabilistic Modeling. In: WORKSHOP DE TESES E DISSERTAÇÕES - CONFERENCE ON GRAPHICS, PATTERNS AND IMAGES (SIBGRAPI), 36. , 2023, Rio Grande/RS.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2023
.
p. 7-13.
DOI: https://doi.org/10.5753/sibgrapi.est.2023.27445.
