Identificação de Smells em Testes Fim-a-Fim Implementados em Cypress
Resumo
Os testes de software são peças fundamentais para ajudar a manter a integridade do sistema. Dentre as modalidades existentes, os testes fim-a-fim se destacam por verificar o comportamento integral dos requisitos do sistema. Por outro lado, pouco se tem estudado sobre as más práticas adotadas na implementação deste tipo de teste automatizado. Esta pesquisa investiga essas más práticas e a viabilidade de identificá-las automaticamente. Para isso, foi realizada uma Revisão Sistemática da Literatura e uma Revisão da Literatura Cinza com objetivo de catalogar os smells mais comuns em testes fim-a-fim implementados em Cypress, um popular framework de teste fim-a-fim, resultando em um catálogo de 12 smells. Em seguida, foi conduzido um estudo exploratório onde avaliou-se o desempenho do ChatGPT na detecção desses smells em três sistemas open source. A precisão e cobertura do ChatGPT na detecção dos smells variou entre 15%–31% e 31%–62%, respectivamente. Os resultados mostram variações significativas no desempenho do modelo, destacando a necessidade de estudos mais aprofundados.Referências
Barbosa, L. and Hora, A. (2022). How and Why Developers Migrate Python Tests. In International Conference on Software Analysis, Evolution and Reengineering (SANER), pages 538–548.
Borges, H. S. and Valente, M. T. (2022). Github proxy server: A tool for supporting massive data collection on github. In Proceedings of the XXXVI Brazilian Symposium on Software Engineering, (SBES), page 370–375, New York, NY, USA. Association for Computing Machinery.
Chen, Y., Hu, Z., Zhi, C., Han, J., Deng, S., and Yin, J. (2024). ChatUniTest: A Framework for LLM-Based Test Generation. In ACM International Conference on the Foundations of Software Engineering (FSE).
Fard, A. M. and Mesbah, A. (2013). Jsnose: Detecting javascript code smells. In 2013 IEEE 13th international working conference on Source Code Analysis and Manipulation (SCAM), pages 116–125. IEEE.
Ferreira, F. and Valente, M. T. (2023). Detecting code smells in React-based web apps. Information and Software Technology, 155:1–16.
Garousi, V., Felderer, M., and Mäntylä, M. V. (2016). The need for multivocal literature reviews in software engineering: complementing systematic literature reviews with grey literature. In 20th International Conference on Evaluation and Assessment in Software Engineering (EASE), pages 1–6.
Habchi, S., Hecht, G., Rouvoy, R., and Moha, N. (2017). Code smells in ios apps: How do they compare to android? In 2017 IEEE/ACM 4th International Conference on Mobile Software Engineering and Systems (MOBILESoft), pages 110–121. IEEE.
Hecht, G., Benomar, O., Rouvoy, R., Moha, N., and Duchien, L. (2015). Tracking the Software Quality of Android Applications Along Their Evolution (T). In 2015 30th IEEE/ACM International Conference on Automated Software Engineering (ASE), pages 236–247.
ISTQB (2016). Certified Tester Syllabus Test Automation Engineer. [link]. Acesso em: 09 dez. 2023.
Kamei, F., Wiese, I., Lima, C., Polato, I., Nepomuceno, V., Ferreira, W., Ribeiro, M., Pena, C., Cartaxo, B., Pinto, G., and Soares, S. (2021). Grey literature in software engineering: A critical review. Information and Software Technology, 138:106609.
Leotta, M., Stocco, A., Ricca, F., and Tonella, P. (2016). Robula+: An algorithm for generating robust xpath locators for web testing. Journal of Software: Evolution and Process, 28(3):177–204.
Liu, H., Jin, J., Xu, Z., Bu, Y., Zou, Y., and Zhang, L. (2019). Deep learning based code smell detection. IEEE Transactions on software Engineering, pages 1–28.
Moha, N., Guéhéneuc, Y.-G., Duchien, L., and Le Meur, A.-F. (2009). Decor: A method for the specification and detection of code and design smells. IEEE Transactions on Software Engineering, 36(1):20–36.
Nguyen, H. V., Nguyen, H. A., Nguyen, T. T., Nguyen, A. T., and Nguyen, T. N. (2012). Detection of embedded code smells in dynamic web applications. In 27th IEEE/ACM International Conference on Automated Software Engineering (ASE), pages 282–285.
Palomba, F., Bavota, G., Penta, M. D., Oliveto, R., Lucia, A. D., and Poshyvanyk, D. (2013). Detecting bad smells in source code using change history information. In 28th IEEE/ACM International Conference on Automated Software Engineering (ASE), pages 268–278.
Punt, L., Visscher, S., and Zaytsev, V. (2016). The a?b*a pattern: Undoing style in css and refactoring opportunities it presents. In 2016 IEEE International Conference on Software Maintenance and Evolution (ICSME), pages 67–77.
Ricca, F. and Stocco, A. (2021). Web test automation: Insights from the grey literature. In Bureš, T., Dondi, R., Gamper, J., Guerrini, G., Jurdziński, T., Pahl, C., Sikora, F., and Wong, P. W., editors, SOFSEM 2021: Theory and Practice of Computer Science, pages 472–485, Cham. Springer International Publishing.
Rwemalika, R., Habchi, S., Papadakis, M., Traon, Y. L., and Brasseur, M.-C. (2021). Smells in system user interactive tests. arXiv preprint arXiv:2111.02317.
Schäfer, M., Nadi, S., Eghbali, A., and Tip, F. (2024). An Empirical Evaluation of Using Large Language Models for Automated Unit Test Generation. IEEE Transactions on Software Engineering, 50(1).
Sharma, T., Fragkoulis, M., and Spinellis, D. (2016). Does your configuration code smell? In 2016 IEEE/ACM 13th Working Conference on Mining Software Repositories (MSR), pages 189–200. IEEE.
Silva, L. L., Silva, J., Montandon, J. E., and Valente, M. T. (2024). Detecting code smells using chatgpt: Initial insights. In International Symposium on Empirical Software Engineering and Measurement - Emerging Results, Vision and Reflection (ESEIW-ERVR), pages 1–7.
Soares, E., Ribeiro, M., Gheyi, R., Amaral, G., and Santos, A. M. (2022). Refactoring test smells with junit 5: Why should developers keep up-to-date. IEEE Transactions on Software Engineering.
Sobrinho, E. V. d. P., De Lucia, A., and Maia, M. d. A. (2021). A systematic literature review on bad smells–5 w’s: Which, when, what, who, where. IEEE Transactions on Software Engineering, 47(1):17–66.
Sommerville, I. (2011). Engenharia de software. Pearson Prentice Hall.
Valente, M. T. (2020). Engenharia de Software Moderna: Princípios e Práticas para Desenvolvimento de Software com Produtividade. Editora: Independente.
Vegi, L. and Valente, M. T. (2022). Code smells in Elixir: Early results from a grey literature review. In 30th International Conference on Program Comprehension (ICPC), pages 1–5.
Zhang, H., Zhou, X., Huang, X., Huang, H., and Babar, M. A. (2020). An evidence-based inquiry into the use of grey literature in software engineering. In 42nd International Conference on Software Engineering (ICSE), pages 1422–1434.
Borges, H. S. and Valente, M. T. (2022). Github proxy server: A tool for supporting massive data collection on github. In Proceedings of the XXXVI Brazilian Symposium on Software Engineering, (SBES), page 370–375, New York, NY, USA. Association for Computing Machinery.
Chen, Y., Hu, Z., Zhi, C., Han, J., Deng, S., and Yin, J. (2024). ChatUniTest: A Framework for LLM-Based Test Generation. In ACM International Conference on the Foundations of Software Engineering (FSE).
Fard, A. M. and Mesbah, A. (2013). Jsnose: Detecting javascript code smells. In 2013 IEEE 13th international working conference on Source Code Analysis and Manipulation (SCAM), pages 116–125. IEEE.
Ferreira, F. and Valente, M. T. (2023). Detecting code smells in React-based web apps. Information and Software Technology, 155:1–16.
Garousi, V., Felderer, M., and Mäntylä, M. V. (2016). The need for multivocal literature reviews in software engineering: complementing systematic literature reviews with grey literature. In 20th International Conference on Evaluation and Assessment in Software Engineering (EASE), pages 1–6.
Habchi, S., Hecht, G., Rouvoy, R., and Moha, N. (2017). Code smells in ios apps: How do they compare to android? In 2017 IEEE/ACM 4th International Conference on Mobile Software Engineering and Systems (MOBILESoft), pages 110–121. IEEE.
Hecht, G., Benomar, O., Rouvoy, R., Moha, N., and Duchien, L. (2015). Tracking the Software Quality of Android Applications Along Their Evolution (T). In 2015 30th IEEE/ACM International Conference on Automated Software Engineering (ASE), pages 236–247.
ISTQB (2016). Certified Tester Syllabus Test Automation Engineer. [link]. Acesso em: 09 dez. 2023.
Kamei, F., Wiese, I., Lima, C., Polato, I., Nepomuceno, V., Ferreira, W., Ribeiro, M., Pena, C., Cartaxo, B., Pinto, G., and Soares, S. (2021). Grey literature in software engineering: A critical review. Information and Software Technology, 138:106609.
Leotta, M., Stocco, A., Ricca, F., and Tonella, P. (2016). Robula+: An algorithm for generating robust xpath locators for web testing. Journal of Software: Evolution and Process, 28(3):177–204.
Liu, H., Jin, J., Xu, Z., Bu, Y., Zou, Y., and Zhang, L. (2019). Deep learning based code smell detection. IEEE Transactions on software Engineering, pages 1–28.
Moha, N., Guéhéneuc, Y.-G., Duchien, L., and Le Meur, A.-F. (2009). Decor: A method for the specification and detection of code and design smells. IEEE Transactions on Software Engineering, 36(1):20–36.
Nguyen, H. V., Nguyen, H. A., Nguyen, T. T., Nguyen, A. T., and Nguyen, T. N. (2012). Detection of embedded code smells in dynamic web applications. In 27th IEEE/ACM International Conference on Automated Software Engineering (ASE), pages 282–285.
Palomba, F., Bavota, G., Penta, M. D., Oliveto, R., Lucia, A. D., and Poshyvanyk, D. (2013). Detecting bad smells in source code using change history information. In 28th IEEE/ACM International Conference on Automated Software Engineering (ASE), pages 268–278.
Punt, L., Visscher, S., and Zaytsev, V. (2016). The a?b*a pattern: Undoing style in css and refactoring opportunities it presents. In 2016 IEEE International Conference on Software Maintenance and Evolution (ICSME), pages 67–77.
Ricca, F. and Stocco, A. (2021). Web test automation: Insights from the grey literature. In Bureš, T., Dondi, R., Gamper, J., Guerrini, G., Jurdziński, T., Pahl, C., Sikora, F., and Wong, P. W., editors, SOFSEM 2021: Theory and Practice of Computer Science, pages 472–485, Cham. Springer International Publishing.
Rwemalika, R., Habchi, S., Papadakis, M., Traon, Y. L., and Brasseur, M.-C. (2021). Smells in system user interactive tests. arXiv preprint arXiv:2111.02317.
Schäfer, M., Nadi, S., Eghbali, A., and Tip, F. (2024). An Empirical Evaluation of Using Large Language Models for Automated Unit Test Generation. IEEE Transactions on Software Engineering, 50(1).
Sharma, T., Fragkoulis, M., and Spinellis, D. (2016). Does your configuration code smell? In 2016 IEEE/ACM 13th Working Conference on Mining Software Repositories (MSR), pages 189–200. IEEE.
Silva, L. L., Silva, J., Montandon, J. E., and Valente, M. T. (2024). Detecting code smells using chatgpt: Initial insights. In International Symposium on Empirical Software Engineering and Measurement - Emerging Results, Vision and Reflection (ESEIW-ERVR), pages 1–7.
Soares, E., Ribeiro, M., Gheyi, R., Amaral, G., and Santos, A. M. (2022). Refactoring test smells with junit 5: Why should developers keep up-to-date. IEEE Transactions on Software Engineering.
Sobrinho, E. V. d. P., De Lucia, A., and Maia, M. d. A. (2021). A systematic literature review on bad smells–5 w’s: Which, when, what, who, where. IEEE Transactions on Software Engineering, 47(1):17–66.
Sommerville, I. (2011). Engenharia de software. Pearson Prentice Hall.
Valente, M. T. (2020). Engenharia de Software Moderna: Princípios e Práticas para Desenvolvimento de Software com Produtividade. Editora: Independente.
Vegi, L. and Valente, M. T. (2022). Code smells in Elixir: Early results from a grey literature review. In 30th International Conference on Program Comprehension (ICPC), pages 1–5.
Zhang, H., Zhou, X., Huang, X., Huang, H., and Babar, M. A. (2020). An evidence-based inquiry into the use of grey literature in software engineering. In 42nd International Conference on Software Engineering (ICSE), pages 1422–1434.
Publicado
30/09/2024
Como Citar
BICALHO, Larissa; MONTANDON, João Eduardo; VALENTE, Marco Túlio.
Identificação de Smells em Testes Fim-a-Fim Implementados em Cypress. In: WORKSHOP DE VISUALIZAÇÃO, EVOLUÇÃO E MANUTENÇÃO DE SOFTWARE (VEM), 12. , 2024, Curitiba/PR.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2024
.
p. 1-12.
DOI: https://doi.org/10.5753/vem.2024.3785.
