Iterative Deepening URL-Based Search: Enhancing GUI Testing for Web Applications
Resumo
Automated GUI testing has become prevalent in web applications due to its efficiency in detecting visible failures. In this context, scriptless testing can systematically explore the application GUI. To achieve this, a GUI tree can be employed to generate test cases. Algorithms such as IDS can iteratively discover the GUI tree of an application while generating the test suite. However, the resulting suite in such scenarios is often redundant, leading to long execution times. This paper introduces the IDUBS algorithm, an optimized version of IDS that aims to reduce redundancies in state access by identifying URL changes during system exploration. It utilizes this information to streamline path discovery for automatic GUI testing. By employing IDUBS, repetitive actions can be replaced with direct URL visits, resulting in faster retrieval of previous GUI states in subsequent iterations and consequently reducing test costs for test suite execution while maintaining performance. We evaluated the performance of IDUBS in two empirical studies involving twenty industrial and four open-source web applications, comparing it with the baseline strategy (IDS). Our results showed that IDUBS achieved a general reduction in execution time and test case redundancy by 43.41% and 49.30%, respectively, while maintaining code coverage. Additionally, IDUBS suites detected more faults, demonstrating improved performance.
Palavras-chave:
GUI testing, web applications, search algorithms, fault localization
Referências
Pekka Aho, Teemu Kanstren, Tomi Räty, and Juha Röning. 2014. Automated extraction of GUI models for testing. In Advances in Computers. Vol. 95. Elsevier, 49–112.
Pekka Aho, Matias Suarez, Teemu Kanstrén, and Atif M Memon. 2014. Murphy tools: Utilizing extracted gui models for industrial software testing. In 2014 IEEE Seventh International Conference on Software Testing, Verification and Validation Workshops. IEEE, 343–348.
Pekka Aho, Tanja EJ Vos, Sami Ahonen, Tomi Piirainen, Perttu Moilanen, and Fernando Pastor Ricos. 2019. Continuous piloting of an open source test automation tool in an industrial environment. Jornadas de Ingeniería del Software y Bases de Datos (JISBD) (2019), 1–4.
Andrea Arcuri and Lionel Briand. 2014. A hitchhiker’s guide to statistical tests for assessing randomized algorithms in software engineering. Software Testing, Verification and Reliability 24, 3 (2014), 219–250.
Tanzirul Azim and Iulian Neamtiu. 2013. Targeted and depth-first exploration for systematic testing of android apps. In Proceedings of the 2013 ACM SIGPLAN international conference on Object oriented programming systems languages & applications. Association for Computing Machinery, New York, NY, USA, 641–660. DOI: 10.1145/2509136.2509549
Alexander Bainczyk, Alexander Schieweck, Bernhard Steffen, and Falk Howar. 2017. Model-based testing without models: the TodoMVC case study. ModelEd, TestEd, TrustEd: Essays Dedicated to Ed Brinksma on the Occasion of His 60th Birthday (2017), 125–144.
Ishan Banerjee, Bao Nguyen, Vahid Garousi, and Atif Memon. 2013. Graphical user interface (GUI) testing: Systematic mapping and repository. Information and Software Technology 55, 10 (2013), 1679–1694.
Axel Bons, Beatriz Marín, Pekka Aho, and Tanja EJ Vos. 2023. Scripted and scriptless GUI testing for web applications: An industrial case. Information and Software Technology 158 (2023), 107172.
Dmitry Davidov and Shaul Markovitch. 2002. Multiple-goal search algorithms and their application to Web crawling. In AAAI/IAAI. 713–718.
Dmitry Davidov and Shaul Markovitch. 2006. Multiple-goal heuristic search. Journal of Artificial Intelligence Research 26 (2006), 417–451.
Mark Grechanik, Qing Xie, and Chen Fu. 2009. Creating GUI testing tools using accessibility technologies. In 2009 International Conference on Software Testing, Verification, and Validation Workshops. IEEE, 243–250.
Shuai Hao, Bin Liu, Suman Nath, William GJ Halfond, and Ramesh Govindan. 2014. Puma: Programmable ui-automation for large-scale dynamic analysis of mobile apps. In Proceedings of the 12th annual international conference on Mobile systems, applications, and services. 204–217.
Liu Hongyun, Jiang Xiao, and Ju Hehua. 2013. Multi-goal path planning algorithm for mobile robots in grid space. In 2013 25th Chinese Control and Decision Conference (CCDC). IEEE, 2872–2876.
Md Hossain, Hyunsook Do, and Ravi Eda. 2014. Regression testing for web applications using reusable constraint values. In 2014 IEEE Seventh International Conference on Software Testing, Verification and Validation Workshops. IEEE, 312–321.
Xiaoming Hu and Yibo Huang. 2021. Research and Application of Software Automated Testing Based on Directed Graph. In 2021 IEEE 3rd International Conference on Frontiers Technology of Information and Computer (ICFTIC). IEEE, 661–664.
Taufan Fadhilah Iskandar, Muharman Lubis, Tien Fabrianti Kusumasari, and Arif Ridho Lubis. 2020. Comparison between client-side and server-side rendering in the web development. In IOP Conference Series: Materials Science and Engineering, Vol. 801. IOP Publishing, 012136.
Bo Jiang, Yaoyue Zhang, Wing Kwong Chan, and Zhenyu Zhang. 2019. A systematic study on factors impacting gui traversal-based test case generation techniques for android applications. IEEE Transactions on Reliability 68, 3 (2019), 913–926.
Imran Akhtar Khan and Roopa Singh. 2012. Quality Assurance And Integration Testing Aspects In Web Based Applications. ArXiv abs/1207.3213 (2012). DOI: 10.5121/ijcsea.2012.2310
Inessa V Krasnokutska and Oleksandr S Krasnokutskyi. 2024. Implementing E2E tests with Cypress and Page Object Model: evolution of approaches. In CEUR Workshop Proceedings. 101–110.
Maurizio Leotta, Diego Clerissi, Filippo Ricca, and Paolo Tonella. 2013. Capturereplay vs. programmable web testing: An empirical assessment during test case evolution. In 2013 20th Working Conference on Reverse Engineering (WCRE). IEEE, 272–281.
Kai Li Lim, Kah Phooi Seng, LS Yeong, SI Ch’ng, and K Ang Li-minn. 2013. The boundary iterative-deepening depth-first search algorithm. In Second International Conference on Advances in Computer and Information Technology: ACIT 2013. Institute of Research Engineers and Doctors, LLC, 119–124.
Kai Li Lim, Kah Phooi Seng, Lee Seng Yeong, Li-Minn Ang, and Sue Inn Ch’ng. 2016. Pathfinding for the navigation of visually impaired people. International Journal of Computational Complexity and Intelligent Algorithms 1, 1 (2016), 99–114.
Kai Li Lim, Kah Phooi Seng, Lee Seng Yeong, Li-Minn Ang, and Sue Inn Ch’ng. 2015. Uninformed pathfinding: A new approach. Expert systems with applications 42, 5 (2015), 2722–2730.
Atif Memon, Ishan Banerjee, and Adithya Nagarajan. 2003. GUI ripping: Reverse engineering of graphical user interfaces for testing. In 10th Working Conference on Reverse Engineering, 2003. WCRE 2003. Proceedings. IEEE, 260–269.
Atif M Memon. 2002. GUI testing: Pitfalls and process. Computer 35, 08 (2002), 87–88.
Fatini Mobaraya, Shahid Ali, et al. 2019. Technical Analysis of Selenium and Cypress as functional automation framework for modern web application testing. In 9th International Conference on Computer Science.
Thiago Santos de Moura, Everton L. G. Alves, Hugo Feitosa de Figueirêdo, and Cláudio de Souza Baptista. 2023. Cytestion: Automated GUI Testing for Web Applications. In Proceedings of the XXXVII Brazilian Symposium on Software Engineering. 388–397.
Fernando Pastor Ricós, Pekka Aho, Tanja Vos, Ismael Torres Boigues, Ernesto Calás Blasco, and Héctor Martínez Martínez. 2020. Deploying TESTAR to enable remote testing in an industrial CI pipeline: a case-based evaluation. In Leveraging Applications of Formal Methods, Verification and Validation: Verification Principles: 9th International Symposium on Leveraging Applications of Formal Methods, ISoLA 2020, Rhodes, Greece, October 20–30, 2020, Proceedings, Part I 9. Springer, 543–557.
Olivia Rodríguez-Valdés, Tanja EJ Vos, Pekka Aho, and Beatriz Marín. 2021. 30 years of automated GUI testing: A bibliometric analysis. In Quality of Information and Communications Technology: 14th International Conference, QUATIC 2021, Algarve, Portugal, September 8–11, 2021, Proceedings 14. Springer, 473–488.
Urko Rueda, Tanja EJ Vos, Francisco Almenar, MO Martınez, and Anna I Esparcia-Alcázar. 2015. TESTAR: from academic prototype towards an industry-ready tool for automated testing at the user interface level. Actas de las XX Jornadas de Ingenierıa del Software y Bases de Datos (JISBD 2015) (2015), 236–245.
Stuart J Russell and Peter Norvig. 2016. Artificial intelligence: a modern approach. Pearson.
Nema Salem, Hala Haneya, Hanin Balbaid, and Manal Asrar. 2024. Exploring the Maze: A Comparative Study of Path Finding Algorithms for PAC-Man Game. (2024).
Shahaf S Shperberg, Steven Danishevski, Ariel Felner, and Nathan R Sturtevant. 2021. Iterative-deepening bidirectional heuristic search with restricted memory. In Proceedings of the International Conference on Automated Planning and Scheduling, Vol. 31. 331–339.
Arie Van Deursen. 2015. Testing web applications with state objects. Commun. ACM 58, 8 (2015), 36–43.
Tanja EJ Vos, Pekka Aho, Fernando Pastor Ricos, Olivia Rodriguez-Valdes, and Ad Mulders. 2021. TESTAR–scriptless testing through graphical user interface. Software Testing, Verification and Reliability 31, 3 (2021), e1771.
Yan Wang, Jianguo Lu, and Jessica Chen. 2014. Ts-ids algorithm for query selection in the deep web crawling. In Web Technologies and Applications: 16th Asia-Pacific Web Conference, APWeb 2014, Changsha, China, September 5-7, 2014. Proceedings 16. Springer, 189–200.
Thomas Weise, Steffen Bleul, Diana Comes, and Kurt Geihs. 2008. Different approaches to semantic web service composition. In 2008 Third International Conference on Internet and Web Applications and Services. IEEE, 90–96.
Robert B Wen. 2001. URL-driven automated testing. In Proceedings Second Asia-Pacific Conference on Quality Software. IEEE, 268–272.
Dacong Yan, Shengqian Yang, and Atanas Rountev. 2013. Systematic testing for resource leaks in Android applications. In 2013 IEEE 24th International Symposium on Software Reliability Engineering (ISSRE). IEEE, 411–420. DOI: 10.1109/ISSRE.2013.6698894
Xun Yuan and Atif M Memon. 2010. Iterative execution-feedback model-directed GUI testing. Information and Software Technology 52, 5 (2010), 559–575.
Pekka Aho, Matias Suarez, Teemu Kanstrén, and Atif M Memon. 2014. Murphy tools: Utilizing extracted gui models for industrial software testing. In 2014 IEEE Seventh International Conference on Software Testing, Verification and Validation Workshops. IEEE, 343–348.
Pekka Aho, Tanja EJ Vos, Sami Ahonen, Tomi Piirainen, Perttu Moilanen, and Fernando Pastor Ricos. 2019. Continuous piloting of an open source test automation tool in an industrial environment. Jornadas de Ingeniería del Software y Bases de Datos (JISBD) (2019), 1–4.
Andrea Arcuri and Lionel Briand. 2014. A hitchhiker’s guide to statistical tests for assessing randomized algorithms in software engineering. Software Testing, Verification and Reliability 24, 3 (2014), 219–250.
Tanzirul Azim and Iulian Neamtiu. 2013. Targeted and depth-first exploration for systematic testing of android apps. In Proceedings of the 2013 ACM SIGPLAN international conference on Object oriented programming systems languages & applications. Association for Computing Machinery, New York, NY, USA, 641–660. DOI: 10.1145/2509136.2509549
Alexander Bainczyk, Alexander Schieweck, Bernhard Steffen, and Falk Howar. 2017. Model-based testing without models: the TodoMVC case study. ModelEd, TestEd, TrustEd: Essays Dedicated to Ed Brinksma on the Occasion of His 60th Birthday (2017), 125–144.
Ishan Banerjee, Bao Nguyen, Vahid Garousi, and Atif Memon. 2013. Graphical user interface (GUI) testing: Systematic mapping and repository. Information and Software Technology 55, 10 (2013), 1679–1694.
Axel Bons, Beatriz Marín, Pekka Aho, and Tanja EJ Vos. 2023. Scripted and scriptless GUI testing for web applications: An industrial case. Information and Software Technology 158 (2023), 107172.
Dmitry Davidov and Shaul Markovitch. 2002. Multiple-goal search algorithms and their application to Web crawling. In AAAI/IAAI. 713–718.
Dmitry Davidov and Shaul Markovitch. 2006. Multiple-goal heuristic search. Journal of Artificial Intelligence Research 26 (2006), 417–451.
Mark Grechanik, Qing Xie, and Chen Fu. 2009. Creating GUI testing tools using accessibility technologies. In 2009 International Conference on Software Testing, Verification, and Validation Workshops. IEEE, 243–250.
Shuai Hao, Bin Liu, Suman Nath, William GJ Halfond, and Ramesh Govindan. 2014. Puma: Programmable ui-automation for large-scale dynamic analysis of mobile apps. In Proceedings of the 12th annual international conference on Mobile systems, applications, and services. 204–217.
Liu Hongyun, Jiang Xiao, and Ju Hehua. 2013. Multi-goal path planning algorithm for mobile robots in grid space. In 2013 25th Chinese Control and Decision Conference (CCDC). IEEE, 2872–2876.
Md Hossain, Hyunsook Do, and Ravi Eda. 2014. Regression testing for web applications using reusable constraint values. In 2014 IEEE Seventh International Conference on Software Testing, Verification and Validation Workshops. IEEE, 312–321.
Xiaoming Hu and Yibo Huang. 2021. Research and Application of Software Automated Testing Based on Directed Graph. In 2021 IEEE 3rd International Conference on Frontiers Technology of Information and Computer (ICFTIC). IEEE, 661–664.
Taufan Fadhilah Iskandar, Muharman Lubis, Tien Fabrianti Kusumasari, and Arif Ridho Lubis. 2020. Comparison between client-side and server-side rendering in the web development. In IOP Conference Series: Materials Science and Engineering, Vol. 801. IOP Publishing, 012136.
Bo Jiang, Yaoyue Zhang, Wing Kwong Chan, and Zhenyu Zhang. 2019. A systematic study on factors impacting gui traversal-based test case generation techniques for android applications. IEEE Transactions on Reliability 68, 3 (2019), 913–926.
Imran Akhtar Khan and Roopa Singh. 2012. Quality Assurance And Integration Testing Aspects In Web Based Applications. ArXiv abs/1207.3213 (2012). DOI: 10.5121/ijcsea.2012.2310
Inessa V Krasnokutska and Oleksandr S Krasnokutskyi. 2024. Implementing E2E tests with Cypress and Page Object Model: evolution of approaches. In CEUR Workshop Proceedings. 101–110.
Maurizio Leotta, Diego Clerissi, Filippo Ricca, and Paolo Tonella. 2013. Capturereplay vs. programmable web testing: An empirical assessment during test case evolution. In 2013 20th Working Conference on Reverse Engineering (WCRE). IEEE, 272–281.
Kai Li Lim, Kah Phooi Seng, LS Yeong, SI Ch’ng, and K Ang Li-minn. 2013. The boundary iterative-deepening depth-first search algorithm. In Second International Conference on Advances in Computer and Information Technology: ACIT 2013. Institute of Research Engineers and Doctors, LLC, 119–124.
Kai Li Lim, Kah Phooi Seng, Lee Seng Yeong, Li-Minn Ang, and Sue Inn Ch’ng. 2016. Pathfinding for the navigation of visually impaired people. International Journal of Computational Complexity and Intelligent Algorithms 1, 1 (2016), 99–114.
Kai Li Lim, Kah Phooi Seng, Lee Seng Yeong, Li-Minn Ang, and Sue Inn Ch’ng. 2015. Uninformed pathfinding: A new approach. Expert systems with applications 42, 5 (2015), 2722–2730.
Atif Memon, Ishan Banerjee, and Adithya Nagarajan. 2003. GUI ripping: Reverse engineering of graphical user interfaces for testing. In 10th Working Conference on Reverse Engineering, 2003. WCRE 2003. Proceedings. IEEE, 260–269.
Atif M Memon. 2002. GUI testing: Pitfalls and process. Computer 35, 08 (2002), 87–88.
Fatini Mobaraya, Shahid Ali, et al. 2019. Technical Analysis of Selenium and Cypress as functional automation framework for modern web application testing. In 9th International Conference on Computer Science.
Thiago Santos de Moura, Everton L. G. Alves, Hugo Feitosa de Figueirêdo, and Cláudio de Souza Baptista. 2023. Cytestion: Automated GUI Testing for Web Applications. In Proceedings of the XXXVII Brazilian Symposium on Software Engineering. 388–397.
Fernando Pastor Ricós, Pekka Aho, Tanja Vos, Ismael Torres Boigues, Ernesto Calás Blasco, and Héctor Martínez Martínez. 2020. Deploying TESTAR to enable remote testing in an industrial CI pipeline: a case-based evaluation. In Leveraging Applications of Formal Methods, Verification and Validation: Verification Principles: 9th International Symposium on Leveraging Applications of Formal Methods, ISoLA 2020, Rhodes, Greece, October 20–30, 2020, Proceedings, Part I 9. Springer, 543–557.
Olivia Rodríguez-Valdés, Tanja EJ Vos, Pekka Aho, and Beatriz Marín. 2021. 30 years of automated GUI testing: A bibliometric analysis. In Quality of Information and Communications Technology: 14th International Conference, QUATIC 2021, Algarve, Portugal, September 8–11, 2021, Proceedings 14. Springer, 473–488.
Urko Rueda, Tanja EJ Vos, Francisco Almenar, MO Martınez, and Anna I Esparcia-Alcázar. 2015. TESTAR: from academic prototype towards an industry-ready tool for automated testing at the user interface level. Actas de las XX Jornadas de Ingenierıa del Software y Bases de Datos (JISBD 2015) (2015), 236–245.
Stuart J Russell and Peter Norvig. 2016. Artificial intelligence: a modern approach. Pearson.
Nema Salem, Hala Haneya, Hanin Balbaid, and Manal Asrar. 2024. Exploring the Maze: A Comparative Study of Path Finding Algorithms for PAC-Man Game. (2024).
Shahaf S Shperberg, Steven Danishevski, Ariel Felner, and Nathan R Sturtevant. 2021. Iterative-deepening bidirectional heuristic search with restricted memory. In Proceedings of the International Conference on Automated Planning and Scheduling, Vol. 31. 331–339.
Arie Van Deursen. 2015. Testing web applications with state objects. Commun. ACM 58, 8 (2015), 36–43.
Tanja EJ Vos, Pekka Aho, Fernando Pastor Ricos, Olivia Rodriguez-Valdes, and Ad Mulders. 2021. TESTAR–scriptless testing through graphical user interface. Software Testing, Verification and Reliability 31, 3 (2021), e1771.
Yan Wang, Jianguo Lu, and Jessica Chen. 2014. Ts-ids algorithm for query selection in the deep web crawling. In Web Technologies and Applications: 16th Asia-Pacific Web Conference, APWeb 2014, Changsha, China, September 5-7, 2014. Proceedings 16. Springer, 189–200.
Thomas Weise, Steffen Bleul, Diana Comes, and Kurt Geihs. 2008. Different approaches to semantic web service composition. In 2008 Third International Conference on Internet and Web Applications and Services. IEEE, 90–96.
Robert B Wen. 2001. URL-driven automated testing. In Proceedings Second Asia-Pacific Conference on Quality Software. IEEE, 268–272.
Dacong Yan, Shengqian Yang, and Atanas Rountev. 2013. Systematic testing for resource leaks in Android applications. In 2013 IEEE 24th International Symposium on Software Reliability Engineering (ISSRE). IEEE, 411–420. DOI: 10.1109/ISSRE.2013.6698894
Xun Yuan and Atif M Memon. 2010. Iterative execution-feedback model-directed GUI testing. Information and Software Technology 52, 5 (2010), 559–575.
Publicado
30/09/2024
Como Citar
MOURA, Thiago Santos de; FELIPE, Regina Letícia Santos; ALVES, Everton L. G.; GREGÓRIO, Pedro Henrique S. C.; BAPTISTA, Cláudio de Souza; FIGUEIRÊDO, Hugo Feitosa de.
Iterative Deepening URL-Based Search: Enhancing GUI Testing for Web Applications. In: SIMPÓSIO BRASILEIRO DE ENGENHARIA DE SOFTWARE (SBES), 38. , 2024, Curitiba/PR.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2024
.
p. 201-211.
DOI: https://doi.org/10.5753/sbes.2024.3363.
