Deploying a Low-Cost Vision-Guided Cartesian Robot for Physical Interaction in Mobile Testing: An Industrial Case Study

• Matheus Lopes Motorola Mobility http://orcid.org/0009-0004-3998-1536
• Breno Miranda UFPE https://orcid.org/0000-0001-9608-9393

DOI: https://doi.org/10.5753/sast.2025.14437

Abstract

Automation of tests and related tasks is becoming increasingly essential, as it significantly reduces manual effort and improves efficiency. However, certain test cases cannot be fully automated due to the need for physical interaction either during or prior to their execution. In mobile testing scenarios, for instance, physical interaction may be required to enable USB debugging when it is not pre-authorized, preventing the full automation of test suites. Accordingly, the use of robotic mechanisms for software testing has emerged as a viable alternative to overcome the limitations of full automation. However, such solutions often involve high acquisition, development, and integration costs. To address these challenges, this paper proposes an alternative low-cost approach: a Cartesian robot controlled via Arduino, equipped with a 3D-printed finger. The robot’s movements and interactions with the device screen are guided by computer vision, enabling automated physical interaction during testing. This setup enables automated testing while significantly reducing development and operational costs compared to conventional robotic arms typically used for such tasks. As a preliminary evaluation, our proposed solution was deployed in an industrial setting to enable USB debugging and unblock the automation process within a software testing framework. The results indicate that the approach is highly accurate. Furthermore, when compared to the manual process, which takes approximately 120 minutes, our method reduces the time to 70 minutes, resulting in a time savings of approximately 42%.

Keywords: Software test automation, robotics, computer vision, USB debugging

References

Afsoon Afzal. 2021. Automated Testing of Robotic and Cyberphysical Systems. Ph.D. Dissertation. Carnegie Mellon Univeristy. [link]

Android Developers. 2025. Enable USB Debugging. [link] Accessed: 2025-06-03.

Android Developers. 2025. UI Automator. [link]. Accessed: 2025-06-03.

Appium Contributors. 2025. Appium: Mobile App Automation Made Awesome. [link]. Accessed: 2025-06-03.

Arduino LLC. 2025. Arduino — Open-source electronics platform. [link]. Accessed: 2025-06-03.

Jan Axelson. 2007. Serial Port Complete: COM Ports, USB Virtual COM Ports, and Ports for Embedded Systems. Lakeview Research.

CoreXY Project. 2025. CoreXY: A New Foundation for Motion Systems. [link]. Accessed: 2025-06-03.

GitLab Inc. 2024. What is CI/CD? [link]. Accessed: 2025-06-03.

Chris Liechti. 2025. pySerial - Python Serial Port Extension. [link] Accessed: 2025-06-03.

Lucas Maciel, Alice Oliveira, Riei Rodrigues, Williams Santiago, Andresa Silva, Gustavo Carvalho, and Breno Miranda. 2022. A systematic mapping study on robotic testing of mobile devices. In 2022 48th Euromicro conference on software engineering and advanced applications (SEAA). IEEE, 475–482.

Joseph Redmon, Santosh Divvala, Ross Girshick, and Ali Farhadi. 2016. You Only Look Once: Unified, Real-Time Object Detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

Mojtaba Shahin, Muhammad Ali Babar, and Liming Zhu. 2017. Continuous integration, delivery and deployment: a systematic review on approaches, tools, challenges and practices. IEEE access 5 (2017), 3909–3943.

Ultralytics. 2023. YOLOv8: Cutting-Edge Object Detection Models. [link]. Accessed: 2025-06-03.