SLAM Visual Em Ambientes Dinâmicos Usando Segmentação Panóptica
Resumo
A maioria dos sistemas de SLAM visual não é robusta em cenários dinâmicos. Aqueles que lidam com conteúdo dinâmico nas cenas geralmente dependem de métodos baseados em aprendizado profundo para detectar e filtrar objetos dinâmicos. No entanto, esses métodos não conseguem lidar com objetos desconhecidos. Este trabalho apresenta o Panoptic-SLAM, um sistema de SLAM visual robusto para ambientes dinâmicos, mesmo na presença de objetos desconhecidos. Ele utiliza a Segmentação Panóptica para filtrar objetos dinâmicos da cena durante o processo de estimativa de estado. A metodologia proposta é baseada no ORB-SLAM3, um sistema SLAM estado-da-arte para ambientes estáticos. A implementação foi testada usando conjuntos de dados do mundo real e comparada com vários sistemas da literatura, incluindo DynaSLAM, DS-SLAM e SaD-SLAM e PVO.
Referências
Bescos, B., Fácil, J. M., Civera, J., and Neira, J. (2018). Dynaslam: Tracking, mapping, and inpainting in dynamic scenes. IEEE Robotics and Automation Letters, 3(4):4076–4083.
Bolya, D., Zhou, C., Xiao, F., and Lee, Y. J. (2019). Yolact: Real-time instance segmentation. In 2019 IEEE/CVF International Conference on Computer Vision (ICCV), pages 9156–9165.
Campos, C., Elvira, R., Rodríguez, J. J. G., Montiel, J. M. M., and Tardós, J. D. (2021). Orb-slam3: An accurate open-source library for visual, visual–inertial, and multimap slam. IEEE Transactions on Robotics, 37(6):1874–1890.
Engel, J., Koltun, V., and Cremers, D. (2018). Direct sparse odometry. IEEE Transactions on Pattern Analysis and Machine Intelligence, 40(3):611–625.
Engel, J., Schöps, T., and Cremers, D. (2014). Lsd-slam: Large-scale direct monocular slam. In Computer Vision – ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part II, pages 834–849.
He, K., Gkioxari, G., Dollár, P., and Girshick, R. (2017). Mask r-cnn. In Proceedings of the IEEE international conference on computer vision, pages 2961–2969.
Ji, T., Wang, C., and Xie, L. (2021). Towards real-time semantic rgb-d slam in dynamic environments. In 2021 IEEE International Conference on Robotics and Automation (ICRA), pages 11175–11181.
Kirillov, A., Girshick, R., He, K., and Dollar, P. (2019a). Panoptic feature pyramid networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 6392–6401.
Kirillov, A., He, K., Girshick, R., Rother, C., and Dollár, P. (2019b). Panoptic segmentation. In 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 9396–9405.
Li, G. and Chen, S. (2022). Visual slam in dynamic scenes based on object tracking and static points detection. Journal of Intelligent & Robotic Systems, 104(33).
Lin, T., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., and Zitnick, C. L. (2014). Microsoft coco: Common objects in context. In European conference on computer vision, pages 740–755. Springer.
Liu, Y. and Miura, J. (2021). Rds-slam: Real-time dynamic slam using semantic segmentation methods. IEEE Access, 9:23772–23785.
Mur-Artal, R., Montiel, J. M. M., and Tardós, J. D. (2015). Orb-slam: A versatile and accurate monocular slam system. IEEE Transactions on Robotics, 31(5):1147–1163.
Mur-Artal, R. and Tardós, J. D. (2017). Orb-slam2: An open-source slam system for monocular, stereo, and rgb-d cameras. IEEE Transactions on Robotics, 33(5):1255–1262.
Palazzolo, E., Behley, J., Lottes, P., Giguère, P., and Stachniss, C. (2019). Refusion: 3d reconstruction in dynamic environments for rgb-d cameras exploiting residuals. In 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 7855–7862.
Redmon, J. and Farhadi, A. (2018). Yolov3: An incremental improvement. arXiv preprint arXiv:1804.02767.
Soares, J. C. V., Gattass, M., and Meggiolaro, M. A. (2021). Crowd-slam: Visual slam towards crowded environments using object detection. Journal of Intelligent & Robotic Systems, 102(2).
Sturm, J., Engelhard, N., Endres, F., Burgard, W., and Cremers, D. (2012). A benchmark for the evaluation of rgb-d slam systems. In 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems, pages 573–580.
Teed, Z. and Deng, J. (2021). Droid-slam: Deep visual slam for monocular, stereo, and rgb-d cameras. In Neural Information Processing Systems.
Vincent, J., Labb’e, M., Lauzon, J., Grondin, F., Comtois-Rivet, P., and Michaud, F. (2020). Dynamic object tracking and masking for visual slam. 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 4974–4979.
Wu, Y., Kirillov, A., Massa, F., Lo, W.-Y., and Girshick, R. (2019). Detectron2. https://github.com/facebookresearch/detectron2.
Yang, S., Fan, G., Bai, L., Zhao, C., and Li, D. (2020). Sgc-vslam: A semantic and geometric constraints vslam for dynamic indoor environments. Sensors, 20(8).
Ye, W., Lan, X., Chen, S., Ming, Y., rong Yu, X., Bao, H., Cui, Z., and Zhang, G. (2022). Pvo: Panoptic visual odometry. ArXiv, abs/2207.01610.
Yu, C., Liu, Z., Liu, X.-J., Xie, F., Yang, Y., Wei, Q., and Fei, Q. (2018). Ds-slam: A semantic visual slam towards dynamic environments. In 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 1168–1174.
Yuan, X. and Chen, S. (2020). Sad-slam: A visual slam based on semantic and depth information. In 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 4930–4935.
Yuan, Z., Xu, K., Zhou, X., Deng, B., and Ma, Y. (2021). SVG-Loop : Semantic – Visual – Geometric Information-Based Loop Closure Detection.
Zhang, J., Gao, M., He, Z., and Yang, Y. (2022). Dcs-slam: A semantic slam with moving cluster towards dynamic environments. In 2022 IEEE International Conference on Robotics and Biomimetics (ROBIO), pages 1923–1928.
Zhu, H., Yao, C., Zhu, Z., Liu, Z., and Jia, Z. (2022). Fusing panoptic segmentation and geometry information for robust visual slam in dynamic environments. In IEEE 18th International Conference on Automation Science and Engineering.
Bolya, D., Zhou, C., Xiao, F., and Lee, Y. J. (2019). Yolact: Real-time instance segmentation. In 2019 IEEE/CVF International Conference on Computer Vision (ICCV), pages 9156–9165.
Campos, C., Elvira, R., Rodríguez, J. J. G., Montiel, J. M. M., and Tardós, J. D. (2021). Orb-slam3: An accurate open-source library for visual, visual–inertial, and multimap slam. IEEE Transactions on Robotics, 37(6):1874–1890.
Engel, J., Koltun, V., and Cremers, D. (2018). Direct sparse odometry. IEEE Transactions on Pattern Analysis and Machine Intelligence, 40(3):611–625.
Engel, J., Schöps, T., and Cremers, D. (2014). Lsd-slam: Large-scale direct monocular slam. In Computer Vision – ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part II, pages 834–849.
He, K., Gkioxari, G., Dollár, P., and Girshick, R. (2017). Mask r-cnn. In Proceedings of the IEEE international conference on computer vision, pages 2961–2969.
Ji, T., Wang, C., and Xie, L. (2021). Towards real-time semantic rgb-d slam in dynamic environments. In 2021 IEEE International Conference on Robotics and Automation (ICRA), pages 11175–11181.
Kirillov, A., Girshick, R., He, K., and Dollar, P. (2019a). Panoptic feature pyramid networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 6392–6401.
Kirillov, A., He, K., Girshick, R., Rother, C., and Dollár, P. (2019b). Panoptic segmentation. In 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 9396–9405.
Li, G. and Chen, S. (2022). Visual slam in dynamic scenes based on object tracking and static points detection. Journal of Intelligent & Robotic Systems, 104(33).
Lin, T., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., and Zitnick, C. L. (2014). Microsoft coco: Common objects in context. In European conference on computer vision, pages 740–755. Springer.
Liu, Y. and Miura, J. (2021). Rds-slam: Real-time dynamic slam using semantic segmentation methods. IEEE Access, 9:23772–23785.
Mur-Artal, R., Montiel, J. M. M., and Tardós, J. D. (2015). Orb-slam: A versatile and accurate monocular slam system. IEEE Transactions on Robotics, 31(5):1147–1163.
Mur-Artal, R. and Tardós, J. D. (2017). Orb-slam2: An open-source slam system for monocular, stereo, and rgb-d cameras. IEEE Transactions on Robotics, 33(5):1255–1262.
Palazzolo, E., Behley, J., Lottes, P., Giguère, P., and Stachniss, C. (2019). Refusion: 3d reconstruction in dynamic environments for rgb-d cameras exploiting residuals. In 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 7855–7862.
Redmon, J. and Farhadi, A. (2018). Yolov3: An incremental improvement. arXiv preprint arXiv:1804.02767.
Soares, J. C. V., Gattass, M., and Meggiolaro, M. A. (2021). Crowd-slam: Visual slam towards crowded environments using object detection. Journal of Intelligent & Robotic Systems, 102(2).
Sturm, J., Engelhard, N., Endres, F., Burgard, W., and Cremers, D. (2012). A benchmark for the evaluation of rgb-d slam systems. In 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems, pages 573–580.
Teed, Z. and Deng, J. (2021). Droid-slam: Deep visual slam for monocular, stereo, and rgb-d cameras. In Neural Information Processing Systems.
Vincent, J., Labb’e, M., Lauzon, J., Grondin, F., Comtois-Rivet, P., and Michaud, F. (2020). Dynamic object tracking and masking for visual slam. 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 4974–4979.
Wu, Y., Kirillov, A., Massa, F., Lo, W.-Y., and Girshick, R. (2019). Detectron2. https://github.com/facebookresearch/detectron2.
Yang, S., Fan, G., Bai, L., Zhao, C., and Li, D. (2020). Sgc-vslam: A semantic and geometric constraints vslam for dynamic indoor environments. Sensors, 20(8).
Ye, W., Lan, X., Chen, S., Ming, Y., rong Yu, X., Bao, H., Cui, Z., and Zhang, G. (2022). Pvo: Panoptic visual odometry. ArXiv, abs/2207.01610.
Yu, C., Liu, Z., Liu, X.-J., Xie, F., Yang, Y., Wei, Q., and Fei, Q. (2018). Ds-slam: A semantic visual slam towards dynamic environments. In 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 1168–1174.
Yuan, X. and Chen, S. (2020). Sad-slam: A visual slam based on semantic and depth information. In 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 4930–4935.
Yuan, Z., Xu, K., Zhou, X., Deng, B., and Ma, Y. (2021). SVG-Loop : Semantic – Visual – Geometric Information-Based Loop Closure Detection.
Zhang, J., Gao, M., He, Z., and Yang, Y. (2022). Dcs-slam: A semantic slam with moving cluster towards dynamic environments. In 2022 IEEE International Conference on Robotics and Biomimetics (ROBIO), pages 1923–1928.
Zhu, H., Yao, C., Zhu, Z., Liu, Z., and Jia, Z. (2022). Fusing panoptic segmentation and geometry information for robust visual slam in dynamic environments. In IEEE 18th International Conference on Automation Science and Engineering.
Publicado
09/10/2023
Como Citar
ABATI, Gabriel F.; SOARES, João Carlos V.; MEGGIOLARO, Marco Antonio.
SLAM Visual Em Ambientes Dinâmicos Usando Segmentação Panóptica. In: CONCURSO DE TESES E DISSERTAÇÕES EM ROBÓTICA - CTDR (MESTRADO) - SIMPÓSIO BRASILEIRO DE ROBÓTICA E SIMPÓSIO LATINO-AMERICANO DE ROBÓTICA (SBR/LARS), 15. , 2023, Salvador/BA.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2023
.
p. 61-72.
DOI: https://doi.org/10.5753/sbrlars_estendido.2023.235116.
