Gustavo Costa
Breno Miranda
ABSTRACT
Mobile applications have become an integral part of modern society, driving the demand for robust and efficient mobile user interface (UI) testing frameworks. With the growing adoption of Continuous Integration (CI) practices, mobile developers seek reliable solutions that can seamlessly integrate into their CI pipelines. This paper presents a comparative analysis of popular mobile UI testing frameworks concerning their suitability for integration into CI environments. The study investigates four widely used mobile UI testing frameworks: Detox, Appium, Calabash, and Maestro. A set of evaluation criteria is established, encompassing aspects such as popularity, ease of use, and test execution speed. The results highlight the strengths and limitations of each testing framework concerning its seamless integration into CI workflows. Overall, this comparative analysis aims to assist mobile developers and organizations in making informed decisions when selecting a mobile UI testing framework for their CI processes. The findings provide insights into the strengths and weaknesses of each framework, enabling developers to align their testing strategies with CI best practices and ultimately improve the quality of their mobile applications.
- 2023. Appium. https://appium.io/docs/en/2.0/. Acessado em 29/06/2023.Google Scholar
- 2023. Art Institute of Chicago API. https://api.artic.edu/docs/. Acessado em 29/06/2023.Google Scholar
- 2023. Bitrise. https://bitrise.io/. Acessado em 29/06/2023.Google Scholar
- 2023. Calabash. https://github.com/calabash. Acessado em 29/06/2023.Google Scholar
- 2023. Detox. https://wix.github.io/Detox/. Acessado em 29/06/2023.Google Scholar
- 2023. Difference between black-box testing and gray-box testing. https://www.geeksforgeeks.org/difference-between-black-box-testing-and-gray-box-testing/. Acessado em 09/07/2023.Google Scholar
- 2023. Github - Appium. https://github.com/appium/appium. Acessado em 09/07/2023.Google Scholar
- 2023. Github - Calabash-Android. https://github.com/calabash/calabash-android. Acessado em 09/07/2023.Google Scholar
- 2023. Github - Detox. https://github.com/wix/Detox. Acessado em 09/07/2023.Google Scholar
- 2023. Github - Maestro. https://github.com/mobile-dev-inc/maestro. Acessado em 09/07/2023.Google Scholar
- 2023. Maestro. https://maestro.mobile.dev/. Acessado em 09/07/2023.Google Scholar
- 2023. reqres.in. https://reqres.in. Acessado em 29/06/2023.Google Scholar
- 2023. Stack Overflow - Appium. https://stackoverflow.com/questions/tagged/appium. Acessado em 09/07/2023.Google Scholar
- 2023. Stack Overflow - Detox. https://stackoverflow.com/questions/tagged/detox. Acessado em 09/07/2023.Google Scholar
- 2023. Stack Overflow - Detox. https://stackoverflow.com/questions/tagged/calabash. Acessado em 09/07/2023.Google Scholar
- 2023. Stack Overflow - Maestro. https://stackoverflow.com/questions/tagged/maestro. Acessado em 09/07/2023.Google Scholar
- 2023. What is ubiquitous language. https://tanzu.vmware.com/developer/blog/ubiquitous-language/. Acessado em 09/07/2023.Google Scholar
- Samah WG AbuSalim, Rosziati Ibrahim, and Jahari Abdul Wahab. 2021. Comparative analysis of software testing techniques for mobile applications. In Journal of Physics: Conference Series, Vol. 1793. IOP Publishing, 012036.Google Scholar
- Domenico Amalfitano, Nicola Amatucci, Atif M Memon, Porfirio Tramontana, and Anna Rita Fasolino. 2017. A general framework for comparing automatic testing techniques of Android mobile apps. Journal of Systems and Software 125 (2017), 322–343.Google ScholarCross Ref
- Khawaja Sarmad Arif and Usman Ali. 2019. Mobile Application testing tools and their challenges: A comparative study. In 2019 2nd International Conference on Computing, Mathematics and Engineering Technologies (iCoMET). 1–6. https://doi.org/10.1109/ICOMET.2019.8673505Google ScholarCross Ref
- Gustavo Costa. 2023. Repositório do projeto. https://github.com/gustavos60/TG-2023. Acessado em 29/06/2023.Google Scholar
- Luis Cruz, Rui Abreu, and David Lo. 2019. To the attention of mobile software developers: guess what, test your app!Empirical Software Engineering 24 (2019), 2438–2468.Google Scholar
- Paul M Duvall, Steve Matyas, and Andrew Glover. 2007. Continuous integration: improving software quality and reducing risk. Pearson Education.Google Scholar
- Duaa R Mohammad, Sajedah Al-Momani, Yahya M Tashtoush, and Mohammad Alsmirat. 2019. A comparative analysis of quality assurance automated testing tools for windows mobile applications. In 2019 IEEE 9th Annual Computing and Communication Workshop and Conference (CCWC). IEEE, 0414–0419.Google ScholarCross Ref
- Maleknaz Nayebi, Bram Adams, and Guenther Ruhe. 2016. Release Practices for Mobile Apps–What do Users and Developers Think?. In 2016 ieee 23rd international conference on software analysis, evolution, and reengineering (saner), Vol. 1. IEEE, 552–562.Google Scholar
- F Okezie, Isaac Odun-Ayo, and S Bogle. 2019. A critical analysis of software testing tools. In Journal of Physics: Conference Series, Vol. 1378. IOP Publishing, 042030.Google Scholar
- Gustavo Pinto, Breno Miranda, Supun Dissanayake, Marcelo d’Amorim, Christoph Treude, and Antonia Bertolino. 2020. What is the Vocabulary of Flaky Tests?. In Proceedings of the 17th International Conference on Mining Software Repositories (Seoul, Republic of Korea) (MSR ’20). Association for Computing Machinery, New York, NY, USA, 492–502. https://doi.org/10.1145/3379597.3387482Google ScholarDigital Library
- Matt Wynne, Aslak Hellesoy, and Steve Tooke. 2017. The cucumber book: behaviour-driven development for testers and developers. Pragmatic Bookshelf.Google Scholar
Index Terms
- A Comparative Analysis of Mobile UI Testing Frameworks in Continuous Integration Environments
Recommendations
Measuring the energy footprint of mobile testing frameworks
ICSE '18: Proceedings of the 40th International Conference on Software Engineering: Companion ProceeedingsThis paper evaluates eight popular mobile UI automation frameworks. We have discovered that there are automation frameworks that increase energy consumption up to 7500%. While limited in the interactions one can do, Espresso is the most energy efficient ...
Cloud-Based Mobile App Testing Framework: Architecture, Implementation and Execution
SAST '16: Proceedings of the 1st Brazilian Symposium on Systematic and Automated Software TestingThe growth in the use of mobile devices is notorious due to the multiple functionalities they offer. The time between the release of new device models and mobile platform updates is very short, and this has a direct influence on the quality of mobile ...
Mobile Compatibility Testing Using Multi-objective Genetic Algorithm
SOSE '15: Proceedings of the 2015 IEEE Symposium on Service-Oriented System EngineeringMobile compatibility testing has been identified as one urgent and challenging issue. Mobile apps are expected to work on thousand kinds of mobile devices with diverse device features and mobile platforms. So mobile compatibility testing is complex and ...
Comments