Abda Myrria de Albuquerque
ABSTRACT
Software testing is one of the essential phases of software development. In this context, test automation has gained significant traction for meeting immediate requirements while upholding result quality. While both automated and manual methods are employed for software testing, manual approaches are susceptible to inaccuracies and human errors despite the benefits of automation. This research is focused on Google’s Android OS, pivotal for a variety of global mobile devices, which must adhere to specific quality requirements. Our study focuses on the Compatibility Test Suite tool, specifically the Rotation Vector Computer Vision Crosscheck (RV) Test case designed for evaluating smartphone sensor capabilities. We introduce a three-axis robotic arm designed to automate the RV test execution, thereby minimizing operational failures and expediting Android smartphones’ quality testing processes. We compared RV test execution in a real-world company using our automated solution against human testers. The proposed robotic arm demonstrated a 75% accuracy rate, surpassing the 37% accuracy achieved by human testers. This significant disparity underscores the potential of our automation approach to mitigate manual errors while ensuring robust and effective testing processes.
Supplemental Material
Available for Download
- Daniel Asfaw. 2015. Benefits of automated testing over manual testing. International Journal of Innovative Research in Information Security 2, 1 (2015), 5–13.Google Scholar
- Debdeep Banerjee and Kevin Yu. 2018. Robotic Arm-Based Face Recognition Software Test Automation. IEEE Access 6 (2018), 37858–37868. https://doi.org/10.1109/ACCESS.2018.2854754Google ScholarCross Ref
- Debdeep Banerjee, Kevin Yu, and Garima Aggarwal. 2018. Image Rectification Software Test Automation Using a Robotic ARM. IEEE Access 6 (2018), 34075–34085. https://doi.org/10.1109/ACCESS.2018.2846761Google ScholarCross Ref
- Stefan Berner, Roland Weber, and Rudolf K Keller. 2005. Observations and lessons learned from automated testing. In Proceedings of the 27th international conference on Software engineering. 571–579.Google ScholarDigital Library
- SM Bindu Bhargavi and V Suma. 2022. A Survey of the Software Test Methods and Identification of Critical Success Factors for Automation. SN Computer Science 3, 6 (2022), 449.Google ScholarDigital Library
- Android Developers. 2023. Compatibility Test Suite. url = https://source.android.com/compatibility/cts. "[Accessed: April/2023]".Google Scholar
- Android Developers. 2023. Compatibility Test Suite Verifier. url = https://source.android.com/docs/compatibility/cts/verifier. "[Accessed: April/2023]".Google Scholar
- Android Developers. 2023. Rotation Computer Vision Crosscheck Vector. url = https://source.android.com/compatibility/cts/rotation-vector. "[Accessed: April/2023]".Google Scholar
- Demian Frister, Andreas Oberweis, and Aleksandar Goranov. 2020. Automated Testing of Mobile Applications Using a Robotic Arm. In 2020 International Conference on Computational Science and Computational Intelligence (CSCI). 1729–1735. https://doi.org/10.1109/CSCI51800.2020.00321Google ScholarCross Ref
- Heidilyn Veloso Gamido and Marlon Viray Gamido. 2019. Comparative review of the features of automated software testing tools. International Journal of Electrical and Computer Engineering 9, 5 (2019), 4473.Google Scholar
- Klaus Haller. 2013. Mobile testing. ACM SIGSOFT Software Engineering Notes 38, 6 (2013), 1–8.Google ScholarDigital Library
- Mona Erfani Joorabchi, Ali Mesbah, and Philippe Kruchten. 2013. Real challenges in mobile app development. In 2013 ACM/IEEE International Symposium on Empirical Software Engineering and Measurement. IEEE, 15–24.Google ScholarCross Ref
- Wazir Zada Khan, Yang Xiang, Mohammed Y Aalsalem, and Quratulain Arshad. 2012. Mobile phone sensing systems: A survey. IEEE Communications Surveys & Tutorials 15, 1 (2012), 402–427.Google ScholarCross Ref
- Divya Kumar and Krishn Kumar Mishra. 2016. The impacts of test automation on software’s cost, quality and time to market. Procedia Computer Science 79 (2016), 8–15.Google ScholarCross Ref
- Pedro Ivo Pereira Lancellotta, Heryck Michael dos Santos Barbosa, João Gabriel Castro Santos, Klirssia Matos Isaac Sahdo, and Janislley Oliveira De Sousa. 2022. An Industry Case Study: Methodology Application to the Reviewing Process on Android Releases Homologation. In Anais Estendidos do XIII Congresso Brasileiro de Software: Teoria e Prática. SBC, 13–16.Google Scholar
- Mario Linares-Vásquez, Carlos Bernal-Cárdenas, Kevin Moran, and Denys Poshyvanyk. 2017. How do developers test android applications?. In 2017 IEEE International Conference on Software Maintenance and Evolution (ICSME). IEEE, 613–622.Google ScholarCross Ref
- Mika V Mäntylä, Bram Adams, Foutse Khomh, Emelie Engström, and Kai Petersen. 2015. On rapid releases and software testing: a case study and a semi-systematic literature review. Empirical Software Engineering 20 (2015), 1384–1425.Google ScholarDigital Library
- Ke Mao, Mark Harman, and Yue Jia. 2017. Robotic Testing of Mobile Apps for Truly Black-Box Automation. IEEE Software 34, 2 (2017), 11–16. https://doi.org/10.1109/MS.2017.49Google ScholarDigital Library
- Leckraj Nagowah and Gayeree Sowamber. 2012. A novel approach of automation testing on mobile devices. In 2012 international conference on computer & information science (ICCIS), Vol. 2. IEEE.Google ScholarCross Ref
- Dudekula Mohammad Rafi, Katam Reddy Kiran Moses, Kai Petersen, and Mika V. Mäntylä. 2012. Benefits and limitations of automated software testing: Systematic literature review and practitioner survey. (2012), 36–42. https://doi.org/10.1109/IWAST.2012.6228988Google ScholarCross Ref
- RM Sharma. 2014. Quantitative analysis of automation and manual testing. International journal of engineering and innovative technology 4, 1 (2014).Google Scholar
- Ian Sommerville. 2016. Software Engineering (10 ed.). Addison-Wesley, Harlow, England.Google Scholar
- Yuqing Wang, Mika V Mäntylä, Zihao Liu, and Jouni Markkula. 2022. Test automation maturity improves product quality—Quantitative study of open source projects using continuous integration. Journal of Systems and Software 188 (2022), 111259.Google ScholarDigital Library
Index Terms
- Automating Android Rotation Vector Testing in Google's Compatibility Test Suite Using a Robotic Arm
Recommendations
A compatibility testing platform for android multimedia applications
Along with the widespread use of smartphones, Android has become one of the major platforms for multimedia applications (apps). However, due to the fast evolution of Android operating system and the fragmentation of Android devices, it becomes important ...
Automating PBX System Testing
Private communication networks provide their customers with supplementary services that often have relations or dependencies to old, already-tested services. In the past these services were tested manually on terminal devices. This kind of testing is ...
Fault-based test suite prioritization for specification-based testing
Context: Existing test suite prioritization techniques usually rely on code coverage information or historical execution data that serve as indicators for estimating the fault-detecting ability of test cases. Such indicators are primarily empirical in ...
Comments