People Tracking Methods Applied to Planalto Palace Security Videos
Resumo
This paper presents a work in progress with comparative results for five state-of-the-art approaches for pedestrian tracking (Deep OC-SORT, OC-SORT, StrongSORT, BotSORT and ByteTrack) applied to a preliminary version of the UFPRPlanalto801 dataset, composed by footage taken from security cameras in Palácio do Planalto, the official office of the President of Brazil. The videos show images of the protesters invasion occurred on January 8, 2023. We used pieces of the public released footage in order to conduct the experiments in a realworld context. The trackers were evaluated by using IDF1, CLEAR and HOTA metrics. The results show a large number of ID switches and missed associations, and a maximum HOTA score of 0.46, achieved by StrongSORT and ByteTrack methods, which shows how challenging is this type of scenario.
Referências
Gabinete de Segurança Institucional, “Nota à imprensa,” 2023, last accessed 25 September 2023. [Online]. Available: [link].
J. Alikhanov and H. Kim, “Online Action Detection in Surveillance Scenarios: A Comprehensive Review and Comparative Study of State-of-the-Art Multi-Object Tracking Methods,” IEEE Access, vol. 11, pp. 68 079–68 092, 2023.
G. Maggiolino, A. Ahmad, J. Cao, and K. Kitani, “Deep OC-SORT: Multi-Pedestrian Tracking by Adaptive Re-Identification,” arXiv preprint arXiv:2302.11813, 2023.
J. Cao, J. Pang, X. Weng, R. Khirodkar, and K. Kitani, “Observation-Centric SORT: Rethinking SORT for Robust Multi-Object Tracking,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2023, pp. 9686–9696.
Y. Du, Z. Zhao, Y. Song, Y. Zhao, F. Su, T. Gong, and H. Meng, “StrongSORT: Make DeepSORT Great Again,” IEEE Transactions on Multimedia, 2023.
N. Aharon, R. Orfaig, and B.-Z. Bobrovsky, “BoT-SORT: Robust Associations Multi-Pedestrian Tracking,” arXiv preprint arXiv:2206.14651, 2022.
Y. Zhang, P. Sun, Y. Jiang, D. Yu, F. Weng, Z. Yuan, P. Luo, W. Liu, and X. Wang, “ByteTrack: Multi-Object Tracking by Associating Every Detection Box,” in Proceedings of the European Conference on Computer Vision (ECCV), 2022.
P. Sun, J. Cao, Y. Jiang, Z. Yuan, S. Bai, K. Kitani, and P. Luo, “Dance-Track: Multi-Object Tracking in Uniform Appearance and Diverse Motion,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2022, pp. 20 993–21 002.
P. Dendorfer, H. Rezatofighi, A. Milan, J. Shi, D. Cremers, I. Reid, S. Roth, K. Schindler, and L. Leal-Taixé, “CVPR19 tracking and detection challenge: How crowded can it get?” arXiv:1906.04567 [cs], Jun. 2019, arXiv: 1906.04567. [Online]. Available: [link]
——, “MOT20: A benchmark for multi object tracking in crowded scenes,” arXiv:2003.09003 [cs], Mar. 2020, arXiv: 2003.09003. [Online]. Available: [link]
E. Ristani, F. Solera, R. Zou, R. Cucchiara, and C. Tomasi, “Performance Measures and a Data Set for Multi-target, Multi-camera Tracking,” in Computer Vision – ECCV 2016 Workshops, G. Hua and H. Jégou, Eds. Cham: Springer International Publishing, 2016, pp. 17–35.
K. Bernardin and R. Stiefelhagen, “Evaluating Multiple Object Tracking Performance: The CLEAR MOT Metrics,” J. Image Video Process., vol. 2008, jan 2008. [Online]. Available: https://doi.org/10.1155/2008/246309
J. Luiten, A. Osep, P. Dendorfer, P. Torr, A. Geiger, L. Leal-Taixé, and B. Leibe, “HOTA: A Higher Order Metric for Evaluating Multi-Object Tracking,” International Journal of Computer Vision, pp. 1–31, 2020.
A. Bewley, Z. Ge, L. Ott, F. Ramos, and B. Upcroft, “Simple online and realtime tracking,” in 2016 IEEE International Conference on Image Processing (ICIP), 2016, pp. 3464–3468.
N. Wojke, A. Bewley, and D. Paulus, “Simple online and realtime tracking with a deep association metric,” in 2017 IEEE International Conference on Image Processing (ICIP), 2017, pp. 3645–3649.
Z. Ge, S. Liu, F. Wang, Z. Li, and J. Sun, “YOLOX: Exceeding YOLO Series in 2021,” 2021.
Y. Du, J. Wan, Y. Zhao, B. Zhang, Z. Tong, and J. Dong, “GIAOTracker: A comprehensive framework for MCMOT with global information and optimizing strategies in VisDrone 2021,” in 2021 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW), 2021, pp. 2809–2819.
J. Redmon, S. Divvala, R. Girshick, and A. Farhadi, “You only look once: Unified, real-time object detection,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2016.
P. Jiang, D. Ergu, F. Liu, Y. Cai, and B. Ma, “A Review of Yolo Algorithm Developments,” Procedia Computer Science, vol. 199, pp. 1066–1073, 2022. [Online]. Available: [link].
J. Terven and D. Cordova-Esparza, “A Comprehensive Review of YOLO: From YOLOv1 and Beyond,” 2023.
G. Jocher, A. Chaurasia, and J. Qiu, “YOLO by Ultralytics,” Jan. 2023. [Online]. Available: [link]
M. Broström, “BoxMOT: A collection of SOTA real-time, multiobject trackers for object detectors ,” 2023. [Online]. Available: [link]
A. H. Jonathon Luiten, “TrackEval,” [link], 2020.
J. Alikhanov and H. Kim, “Online Action Detection in Surveillance Scenarios: A Comprehensive Review and Comparative Study of State-of-the-Art Multi-Object Tracking Methods,” IEEE Access, vol. 11, pp. 68 079–68 092, 2023.
G. Maggiolino, A. Ahmad, J. Cao, and K. Kitani, “Deep OC-SORT: Multi-Pedestrian Tracking by Adaptive Re-Identification,” arXiv preprint arXiv:2302.11813, 2023.
J. Cao, J. Pang, X. Weng, R. Khirodkar, and K. Kitani, “Observation-Centric SORT: Rethinking SORT for Robust Multi-Object Tracking,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2023, pp. 9686–9696.
Y. Du, Z. Zhao, Y. Song, Y. Zhao, F. Su, T. Gong, and H. Meng, “StrongSORT: Make DeepSORT Great Again,” IEEE Transactions on Multimedia, 2023.
N. Aharon, R. Orfaig, and B.-Z. Bobrovsky, “BoT-SORT: Robust Associations Multi-Pedestrian Tracking,” arXiv preprint arXiv:2206.14651, 2022.
Y. Zhang, P. Sun, Y. Jiang, D. Yu, F. Weng, Z. Yuan, P. Luo, W. Liu, and X. Wang, “ByteTrack: Multi-Object Tracking by Associating Every Detection Box,” in Proceedings of the European Conference on Computer Vision (ECCV), 2022.
P. Sun, J. Cao, Y. Jiang, Z. Yuan, S. Bai, K. Kitani, and P. Luo, “Dance-Track: Multi-Object Tracking in Uniform Appearance and Diverse Motion,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2022, pp. 20 993–21 002.
P. Dendorfer, H. Rezatofighi, A. Milan, J. Shi, D. Cremers, I. Reid, S. Roth, K. Schindler, and L. Leal-Taixé, “CVPR19 tracking and detection challenge: How crowded can it get?” arXiv:1906.04567 [cs], Jun. 2019, arXiv: 1906.04567. [Online]. Available: [link]
——, “MOT20: A benchmark for multi object tracking in crowded scenes,” arXiv:2003.09003 [cs], Mar. 2020, arXiv: 2003.09003. [Online]. Available: [link]
E. Ristani, F. Solera, R. Zou, R. Cucchiara, and C. Tomasi, “Performance Measures and a Data Set for Multi-target, Multi-camera Tracking,” in Computer Vision – ECCV 2016 Workshops, G. Hua and H. Jégou, Eds. Cham: Springer International Publishing, 2016, pp. 17–35.
K. Bernardin and R. Stiefelhagen, “Evaluating Multiple Object Tracking Performance: The CLEAR MOT Metrics,” J. Image Video Process., vol. 2008, jan 2008. [Online]. Available: https://doi.org/10.1155/2008/246309
J. Luiten, A. Osep, P. Dendorfer, P. Torr, A. Geiger, L. Leal-Taixé, and B. Leibe, “HOTA: A Higher Order Metric for Evaluating Multi-Object Tracking,” International Journal of Computer Vision, pp. 1–31, 2020.
A. Bewley, Z. Ge, L. Ott, F. Ramos, and B. Upcroft, “Simple online and realtime tracking,” in 2016 IEEE International Conference on Image Processing (ICIP), 2016, pp. 3464–3468.
N. Wojke, A. Bewley, and D. Paulus, “Simple online and realtime tracking with a deep association metric,” in 2017 IEEE International Conference on Image Processing (ICIP), 2017, pp. 3645–3649.
Z. Ge, S. Liu, F. Wang, Z. Li, and J. Sun, “YOLOX: Exceeding YOLO Series in 2021,” 2021.
Y. Du, J. Wan, Y. Zhao, B. Zhang, Z. Tong, and J. Dong, “GIAOTracker: A comprehensive framework for MCMOT with global information and optimizing strategies in VisDrone 2021,” in 2021 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW), 2021, pp. 2809–2819.
J. Redmon, S. Divvala, R. Girshick, and A. Farhadi, “You only look once: Unified, real-time object detection,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2016.
P. Jiang, D. Ergu, F. Liu, Y. Cai, and B. Ma, “A Review of Yolo Algorithm Developments,” Procedia Computer Science, vol. 199, pp. 1066–1073, 2022. [Online]. Available: [link].
J. Terven and D. Cordova-Esparza, “A Comprehensive Review of YOLO: From YOLOv1 and Beyond,” 2023.
G. Jocher, A. Chaurasia, and J. Qiu, “YOLO by Ultralytics,” Jan. 2023. [Online]. Available: [link]
M. Broström, “BoxMOT: A collection of SOTA real-time, multiobject trackers for object detectors ,” 2023. [Online]. Available: [link]
A. H. Jonathon Luiten, “TrackEval,” [link], 2020.
Publicado
06/11/2023
Como Citar
OLIVEIRA, Cristiano B.; NEVES, João C.; RIBEIRO, Rafael O.; MENOTTI, David.
People Tracking Methods Applied to Planalto Palace Security Videos. In: WORKSHOP DE TRABALHOS EM ANDAMENTO - CONFERENCE ON GRAPHICS, PATTERNS AND IMAGES (SIBGRAPI), 36. , 2023, Rio Grande/RS.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2023
.
p. 115-120.
DOI: https://doi.org/10.5753/sibgrapi.est.2023.27462.
