Automating the audit of the Brazilian electronic ballot operation: A new dataset for 6DoF pose estimation of the voter terminal based on domain randomization
Resumo
Performing detection and pose estimation of objects in six degrees of freedom (6-DoF) is a widely studied challenge in virtual and augmented reality, robotics and computer vision. For simulation and testing of the Brazilian voter terminal, its pose could allow automatic testing/auditing with robotics arms or virtual reality applications to simulate the voting process. For pose estimation using deep learning, it is necessary to generate large amounts of annotated real data, which is a costly task in time and resources. One way to avoid this issue is to create synthetic data through domain randomization, using 3D object modeling, to train the pose estimation technique with a reduced amount of annotated real data. In this work, domain randomization was utilized to generate a synthetic dataset, starting from a 3D model of the voter terminal, varying the lighting settings, camera position and distract insertion, to verify what impact this randomization has on training a single shot algorithm to perform the detection and pose estimation of this terminal in a different scenario. The new dataset with real and synthetic data from the voter terminal was built and will be publicly available.
Palavras-chave:
Object pose estimation, Brazilian voter terminal, Domain randomization, Computer vision, Object tracking
Referências
Blender Foundation (2002). 2022. About Blender. [link].
Autodesk. 2022. Inventor: software avançado de projeto mecânico para suas ideias mais ambiciosas. [link].
João Borrego, Atabak Dehban, Rui Figueiredo, Plinio Moreno, Alexandre Bernardino, and José Santos-Victor. 2018. Applying Domain Randomization to Synthetic Data for Object Category Detection. arXiv:1807.09834 [cs.CV]
Kelvin Batista Da Cunha. 2019. Detecção de objetos em 6-DoF em tempo real utilizando técnicas de aprendizagem profunda. Master’s thesis. Universidade Federal de Pernambuco.
Kelvin B. Da Cunha, Caio Brito, Lucas Valença, Lucas Figueiredo, Francisco Simões, and Veronica Teichrieb. 2022. The impact of domain randomization on cross-device monocular deep 6DoF detection. Pattern Recognition Letters 159 (2022), 224–231. https://doi.org/10.1016/j.patrec.2022.04.008
K. B. da Cunha, C. Brito, L. Valenca, F. Simoes, and V. Teichrieb. 2020. A Study on the Impact of Domain Randomization for Monocular Deep 6DoF Pose Estimation. In 2020 33rd SIBGRAPI Conference on Graphics, Patterns and Images (SIBGRAPI). IEEE Computer Society, Los Alamitos, CA, USA, 332–339. https://doi.org/10.1109/SIBGRAPI51738.2020.00052
F2Wang. 2021. Object Dataset Tools. [link].
Daniel P Huttenlocher, Gregory A. Klanderman, and William J Rucklidge. 1993. Comparing images using the Hausdorff distance. IEEE Transactions on pattern analysis and machine intelligence 15, 9 (1993), 850–863.
A. Kendall, M. Grimes, and R. Cipolla. 2015. PoseNet: A Convolutional Network for Real-Time 6-DOF Camera Relocalization. In 2015 IEEE International Conference on Computer Vision (ICCV). IEEE Computer Society, Los Alamitos, CA, USA, 2938– 2946. https://doi.org/10.1109/ICCV.2015.336
K. Ramnath, S. N. Sinha, R. Szeliski, and E. Hsiao. 2014. Car make and model recognition using 3D curve alignment. In 2014 IEEE Winter Conference on Applications of Computer Vision (WACV). IEEE Computer Society, Los Alamitos, CA, USA, 285–292. https://doi.org/10.1109/WACV.2014.6836087
Joseph Redmon and Ali Farhadi. 2017. YOLO9000: Better, Faster, Stronger. In 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 6517–6525. https://doi.org/10.1109/CVPR.2017.690
Artem Rozantsev, Vincent Lepetit, and Pascal Fua. 2015. On rendering synthetic images for training an object detector. Computer Vision and Image Understanding 137 (2015), 24–37.
Mallick Satya. 2016. Head Pose Estimation using OpenCV and Dlib. [link].
Yongzhi Su, Jason Rambach, Alain Pagani, and Didier Stricker. 2021. Synponet—Accurate and fast CNN-based 6DoF object pose estimation using synthetic training. Sensors 21, 1 (2021), 300.
Bugra Tekin, Sudipta N. Sinha, and Pascal Fua. 2018. Real-Time Seamless Single Shot 6D Object Pose Prediction. arXiv:1711.08848 [cs.CV]
Josh Tobin, Rachel Fong, Alex Ray, Jonas Schneider, Wojciech Zaremba, and Pieter Abbeel. 2017. Domain randomization for transferring deep neural networks from simulation to the real world. In 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE Computer Society, Los Alamitos, CA, USA, 23–30. https://doi.org/10.1109/IROS.2017.8202133
J. Tremblay, A. Prakash, D. Acuna, M. Brophy, V. Jampani, C. Anil, T. To, E. Cameracci, S. Boochoon, and S. Birchfield. 2018. Training Deep Networks with Synthetic Data: Bridging the Reality Gap by Domain Randomization. In 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). IEEE Computer Society, Los Alamitos, CA, USA, 1082–10828. https://doi.org/10.1109/CVPRW.2018.00143
TSE. 2020. Veja como é feita a auditoria de funcionamento das urnas eletrônicas. [link]. (Accessed on 10/10/2022).
TSE. 2022. Plenário do TSE triplica número de urnas eletrônicas auditadas no dia da eleição. [link]. (Accessed on 2023/01/16).
A. Veeraraghavan, R. Chellappa, O. Tuzel, and M. Liu. 2010. Fast directional chamfer matching. In 2010 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE Computer Society, Los Alamitos, CA, USA, 1696–1703. https://doi.org/10.1109/CVPR.2010.5539837
Mei Wang and Weihong Deng. 2018. Deep visual domain adaptation: A survey. Neurocomputing 312 (2018), 135–153. https://doi.org/10.1016/j.neucom.2018.05.083
Y. Xu, K. Lin, G. Zhang, X. Wang, and H. Li. 2022. RNNPose: Recurrent 6- DoF Object Pose Refinement with Robust Correspondence Field Estimation and Pose Optimization. In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE Computer Society, Los Alamitos, CA, USA, 14860–14870. https://doi.org/10.1109/CVPR52688.2022.01446
Kaiyang Zhou, Ziwei Liu, Yu Qiao, Tao Xiang, and Chen Change Loy. 2023. Domain Generalization: A Survey. IEEE Transactions on Pattern Analysis and Machine Intelligence 45, 4 (2023), 4396–4415. https://doi.org/10.1109/TPAMI.2022.3195549
Autodesk. 2022. Inventor: software avançado de projeto mecânico para suas ideias mais ambiciosas. [link].
João Borrego, Atabak Dehban, Rui Figueiredo, Plinio Moreno, Alexandre Bernardino, and José Santos-Victor. 2018. Applying Domain Randomization to Synthetic Data for Object Category Detection. arXiv:1807.09834 [cs.CV]
Kelvin Batista Da Cunha. 2019. Detecção de objetos em 6-DoF em tempo real utilizando técnicas de aprendizagem profunda. Master’s thesis. Universidade Federal de Pernambuco.
Kelvin B. Da Cunha, Caio Brito, Lucas Valença, Lucas Figueiredo, Francisco Simões, and Veronica Teichrieb. 2022. The impact of domain randomization on cross-device monocular deep 6DoF detection. Pattern Recognition Letters 159 (2022), 224–231. https://doi.org/10.1016/j.patrec.2022.04.008
K. B. da Cunha, C. Brito, L. Valenca, F. Simoes, and V. Teichrieb. 2020. A Study on the Impact of Domain Randomization for Monocular Deep 6DoF Pose Estimation. In 2020 33rd SIBGRAPI Conference on Graphics, Patterns and Images (SIBGRAPI). IEEE Computer Society, Los Alamitos, CA, USA, 332–339. https://doi.org/10.1109/SIBGRAPI51738.2020.00052
F2Wang. 2021. Object Dataset Tools. [link].
Daniel P Huttenlocher, Gregory A. Klanderman, and William J Rucklidge. 1993. Comparing images using the Hausdorff distance. IEEE Transactions on pattern analysis and machine intelligence 15, 9 (1993), 850–863.
A. Kendall, M. Grimes, and R. Cipolla. 2015. PoseNet: A Convolutional Network for Real-Time 6-DOF Camera Relocalization. In 2015 IEEE International Conference on Computer Vision (ICCV). IEEE Computer Society, Los Alamitos, CA, USA, 2938– 2946. https://doi.org/10.1109/ICCV.2015.336
K. Ramnath, S. N. Sinha, R. Szeliski, and E. Hsiao. 2014. Car make and model recognition using 3D curve alignment. In 2014 IEEE Winter Conference on Applications of Computer Vision (WACV). IEEE Computer Society, Los Alamitos, CA, USA, 285–292. https://doi.org/10.1109/WACV.2014.6836087
Joseph Redmon and Ali Farhadi. 2017. YOLO9000: Better, Faster, Stronger. In 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 6517–6525. https://doi.org/10.1109/CVPR.2017.690
Artem Rozantsev, Vincent Lepetit, and Pascal Fua. 2015. On rendering synthetic images for training an object detector. Computer Vision and Image Understanding 137 (2015), 24–37.
Mallick Satya. 2016. Head Pose Estimation using OpenCV and Dlib. [link].
Yongzhi Su, Jason Rambach, Alain Pagani, and Didier Stricker. 2021. Synponet—Accurate and fast CNN-based 6DoF object pose estimation using synthetic training. Sensors 21, 1 (2021), 300.
Bugra Tekin, Sudipta N. Sinha, and Pascal Fua. 2018. Real-Time Seamless Single Shot 6D Object Pose Prediction. arXiv:1711.08848 [cs.CV]
Josh Tobin, Rachel Fong, Alex Ray, Jonas Schneider, Wojciech Zaremba, and Pieter Abbeel. 2017. Domain randomization for transferring deep neural networks from simulation to the real world. In 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE Computer Society, Los Alamitos, CA, USA, 23–30. https://doi.org/10.1109/IROS.2017.8202133
J. Tremblay, A. Prakash, D. Acuna, M. Brophy, V. Jampani, C. Anil, T. To, E. Cameracci, S. Boochoon, and S. Birchfield. 2018. Training Deep Networks with Synthetic Data: Bridging the Reality Gap by Domain Randomization. In 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). IEEE Computer Society, Los Alamitos, CA, USA, 1082–10828. https://doi.org/10.1109/CVPRW.2018.00143
TSE. 2020. Veja como é feita a auditoria de funcionamento das urnas eletrônicas. [link]. (Accessed on 10/10/2022).
TSE. 2022. Plenário do TSE triplica número de urnas eletrônicas auditadas no dia da eleição. [link]. (Accessed on 2023/01/16).
A. Veeraraghavan, R. Chellappa, O. Tuzel, and M. Liu. 2010. Fast directional chamfer matching. In 2010 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE Computer Society, Los Alamitos, CA, USA, 1696–1703. https://doi.org/10.1109/CVPR.2010.5539837
Mei Wang and Weihong Deng. 2018. Deep visual domain adaptation: A survey. Neurocomputing 312 (2018), 135–153. https://doi.org/10.1016/j.neucom.2018.05.083
Y. Xu, K. Lin, G. Zhang, X. Wang, and H. Li. 2022. RNNPose: Recurrent 6- DoF Object Pose Refinement with Robust Correspondence Field Estimation and Pose Optimization. In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE Computer Society, Los Alamitos, CA, USA, 14860–14870. https://doi.org/10.1109/CVPR52688.2022.01446
Kaiyang Zhou, Ziwei Liu, Yu Qiao, Tao Xiang, and Chen Change Loy. 2023. Domain Generalization: A Survey. IEEE Transactions on Pattern Analysis and Machine Intelligence 45, 4 (2023), 4396–4415. https://doi.org/10.1109/TPAMI.2022.3195549
Publicado
06/11/2023
Como Citar
NORBERTO, Jefferson Medeiros; DA CUNHA, Kelvin Batista; DA SILVA JÚNIOR, Marcondes Ricarte; SIMÕES, Francisco Paulo Magalhães.
Automating the audit of the Brazilian electronic ballot operation: A new dataset for 6DoF pose estimation of the voter terminal based on domain randomization. In: SIMPÓSIO DE REALIDADE VIRTUAL E AUMENTADA (SVR), 25. , 2023, Rio Grande/RS.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2023
.
p. 56–65.
