Jefferson Medeiros Norberto
ABSTRACT
Performing detection and pose estimation of objects in six degrees of freedom (6-DoF) is a widely studied challenge in virtual and augmented reality, robotics and computer vision. For simulation and testing of the Brazilian voter terminal, its pose could allow automatic testing/auditing with robotics arms or virtual reality applications to simulate the voting process. For pose estimation using deep learning, it is necessary to generate large amounts of annotated real data, which is a costly task in time and resources. One way to avoid this issue is to create synthetic data through domain randomization, using 3D object modeling, to train the pose estimation technique with a reduced amount of annotated real data. In this work, domain randomization was utilized to generate a synthetic dataset, starting from a 3D model of the voter terminal, varying the lighting settings, camera position and distract insertion, to verify what impact this randomization has on training a single shot algorithm to perform the detection and pose estimation of this terminal in a different scenario. The new dataset with real and synthetic data from the voter terminal was built and will be publicly available.
- Blender Foundation (2002). 2022. About Blender. https://www.blender.org/about/.Google Scholar
- Autodesk. 2022. Inventor: software avançado de projeto mecânico para suas ideias mais ambiciosas. https://www.autodesk.com.br/products/inventor/overview.Google Scholar
- João Borrego, Atabak Dehban, Rui Figueiredo, Plinio Moreno, Alexandre Bernardino, and José Santos-Victor. 2018. Applying Domain Randomization to Synthetic Data for Object Category Detection. arxiv:1807.09834 [cs.CV]Google Scholar
- Kelvin Batista Da Cunha. 2019. Detecção de objetos em 6-DoF em tempo real utilizando técnicas de aprendizagem profunda. Master’s thesis. Universidade Federal de Pernambuco.Google Scholar
- Kelvin B. Da Cunha, Caio Brito, Lucas Valença, Lucas Figueiredo, Francisco Simões, and Veronica Teichrieb. 2022. The impact of domain randomization on cross-device monocular deep 6DoF detection. Pattern Recognition Letters 159 (2022), 224–231. https://doi.org/10.1016/j.patrec.2022.04.008Google ScholarDigital Library
- K. B. da Cunha, C. Brito, L. Valenca, F. Simoes, and V. Teichrieb. 2020. A Study on the Impact of Domain Randomization for Monocular Deep 6DoF Pose Estimation. In 2020 33rd SIBGRAPI Conference on Graphics, Patterns and Images (SIBGRAPI). IEEE Computer Society, Los Alamitos, CA, USA, 332–339. https://doi.org/10.1109/SIBGRAPI51738.2020.00052Google ScholarCross Ref
- F2Wang. 2021. Object Dataset Tools. https://github.com/F2Wang/ObjectDatasetTools.Google Scholar
- Daniel P Huttenlocher, Gregory A. Klanderman, and William J Rucklidge. 1993. Comparing images using the Hausdorff distance. IEEE Transactions on pattern analysis and machine intelligence 15, 9 (1993), 850–863.Google ScholarDigital Library
- A. Kendall, M. Grimes, and R. Cipolla. 2015. PoseNet: A Convolutional Network for Real-Time 6-DOF Camera Relocalization. In 2015 IEEE International Conference on Computer Vision (ICCV). IEEE Computer Society, Los Alamitos, CA, USA, 2938–2946. https://doi.org/10.1109/ICCV.2015.336Google ScholarDigital Library
- K. Ramnath, S. N. Sinha, R. Szeliski, and E. Hsiao. 2014. Car make and model recognition using 3D curve alignment. In 2014 IEEE Winter Conference on Applications of Computer Vision (WACV). IEEE Computer Society, Los Alamitos, CA, USA, 285–292. https://doi.org/10.1109/WACV.2014.6836087Google ScholarCross Ref
- Joseph Redmon and Ali Farhadi. 2017. YOLO9000: Better, Faster, Stronger. In 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 6517–6525. https://doi.org/10.1109/CVPR.2017.690Google ScholarCross Ref
- Artem Rozantsev, Vincent Lepetit, and Pascal Fua. 2015. On rendering synthetic images for training an object detector. Computer Vision and Image Understanding 137 (2015), 24–37.Google ScholarDigital Library
- Mallick Satya. 2016. Head Pose Estimation using OpenCV and Dlib. https://learnopencv.com/head-pose-estimation-using-opencv-and-dlib/Google Scholar
- Yongzhi Su, Jason Rambach, Alain Pagani, and Didier Stricker. 2021. Synpo-net—Accurate and fast CNN-based 6DoF object pose estimation using synthetic training. Sensors 21, 1 (2021), 300.Google ScholarCross Ref
- Bugra Tekin, Sudipta N. Sinha, and Pascal Fua. 2018. Real-Time Seamless Single Shot 6D Object Pose Prediction. arxiv:1711.08848 [cs.CV]Google Scholar
- Josh Tobin, Rachel Fong, Alex Ray, Jonas Schneider, Wojciech Zaremba, and Pieter Abbeel. 2017. Domain randomization for transferring deep neural networks from simulation to the real world. In 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE Computer Society, Los Alamitos, CA, USA, 23–30. https://doi.org/10.1109/IROS.2017.8202133Google ScholarDigital Library
- J. Tremblay, A. Prakash, D. Acuna, M. Brophy, V. Jampani, C. Anil, T. To, E. Cameracci, S. Boochoon, and S. Birchfield. 2018. Training Deep Networks with Synthetic Data: Bridging the Reality Gap by Domain Randomization. In 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). IEEE Computer Society, Los Alamitos, CA, USA, 1082–10828. https://doi.org/10.1109/CVPRW.2018.00143Google ScholarCross Ref
- TSE. 2020. Veja como é feita a auditoria de funcionamento das urnas eletrônicas. https://www.tse.jus.br/comunicacao/noticias/2020/Dezembro/veja-como-funciona-a-auditoria-de-funcionamento-das-urnas-eletronicas. (Accessed on 10/10/2022).Google Scholar
- TSE. 2022. Plenário do TSE triplica número de urnas eletrônicas auditadas no dia da eleição. https://www.tse.jus.br/comunicacao/noticias/2022/Marco/plenario-do-tse-triplica-base-amostral-de-urnas-eletronicas-auditadas-no-dia-da-eleicao. (Accessed on 2023/01/16).Google Scholar
- A. Veeraraghavan, R. Chellappa, O. Tuzel, and M. Liu. 2010. Fast directional chamfer matching. In 2010 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE Computer Society, Los Alamitos, CA, USA, 1696–1703. https://doi.org/10.1109/CVPR.2010.5539837Google ScholarCross Ref
- Mei Wang and Weihong Deng. 2018. Deep visual domain adaptation: A survey. Neurocomputing 312 (2018), 135–153. https://doi.org/10.1016/j.neucom.2018.05.083Google ScholarDigital Library
- Y. Xu, K. Lin, G. Zhang, X. Wang, and H. Li. 2022. RNNPose: Recurrent 6-DoF Object Pose Refinement with Robust Correspondence Field Estimation and Pose Optimization. In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE Computer Society, Los Alamitos, CA, USA, 14860–14870. https://doi.org/10.1109/CVPR52688.2022.01446Google ScholarCross Ref
- Kaiyang Zhou, Ziwei Liu, Yu Qiao, Tao Xiang, and Chen Change Loy. 2023. Domain Generalization: A Survey. IEEE Transactions on Pattern Analysis and Machine Intelligence 45, 4 (2023), 4396–4415. https://doi.org/10.1109/TPAMI.2022.3195549Google ScholarDigital Library
Index Terms
- Automating the audit of the Brazilian electronic ballot operation: A new dataset for 6DoF pose estimation of the voter terminal based on domain randomization
Recommendations
The impact of domain randomization on cross-device monocular deep 6DoF detection
Highlights- We propose an evaluation of synthetic datas influence when training deep learning models to handle unseen scenarios.
AbstractThis work evaluates the use of synthetic data to train deep 6DoF pose estimation models that use a monocular RGB camera as input. We have compared different training strategies combining real and synthetic data (with domain ...
Benchmarking pose estimation for robot manipulation
AbstractRobot grasping and manipulation require estimation of 3D object poses. Recently, a number of methods and datasets for vision-based pose estimation have been proposed. However, it is unclear how well the performance measures developed ...
Highlights- We propose a probabilistic performance metric to evaluate the estimated object pose.
Deep Learning-based 6D pose estimation of texture less objects for Industrial Cobots
AIR '23: Proceedings of the 2023 6th International Conference on Advances in RoboticsRobotics and artificial intelligence have led to a significant improvement in the automation of various industrial processes. 6D pose estimation for industrial robotic arms refers to the process of accurately determining the position and orientation of ...
Comments