AgNose: A Framework-Agnostic Test Smell Detection Tool
Resumo
Test smells indicate problems in automated tests that hinder the understanding, effectiveness, and maintenance of the test code. Due to their relevance, many tools have been created to detect them in xUnit frameworks, which has led to the emergence of test framework-agnostic strategies for detecting smells. However, these strategies are still little explored in the literature, and the available tools depend on obsolete components with considerable limitations. This study proposes the AgNose tool, which extends the strategy of agnostic test smell detection. The strategy is based on creating, from the test code in different frameworks, a simplified structure aimed solely at test smell detection. To implement the proposal, we (i) identify test smells analyzed simultaneously by different detection tools, as well as their identification steps; (ii) establish the minimum XML structure that represents the important information for detecting smells; (iii) implement the AgNose tool; (iv) validate the tool using real open-source projects from public repositories. As results, we map 8 test smells and their minimal XML structure. Also, the tool validation achieved a precision of 96.9%, a recall of 87.5%, and an f-measure of 92%. The presented strategy and tool extend current strategies for agnostic test smell detection, eliminating the perceived limitations of available tools.
Referências
Collard, M. L., Decker, M. J., and Maletic, J. I. (2013). srcml: An infrastructure for the exploration, analysis, and manipulation of source code: A tool demonstration. In 2013 IEEE International conference on software maintenance, ICSME ’13’, pages 516–519.
Lambiase, S., Cupito, A., Pecorelli, F., De Lucia, A., and Palomba, F. (2020). Just-in-time test smell detection and refactoring: The DARTS project. In Proceedings of the 28th international conference on program comprehension, ICPC, pages 441–445.
Lopes, G., Romão, D., Soares, E., Ribeiro, M., Amaral, G., Gheyi, R., and Machado, I. (2024). A road to find them all: Towards an agnostic strategy for test smell detection. In Proceedings of the XXIII Brazilian Symposium on Software Quality, SBQS’24, pages 231–241.
Peruma, A., Almalki, K., Newman, C. D., Mkaouer, M. W., Ouni, A., and Palomba, F. (2019). On the distribution of test smells in open-source Android applications: An exploratory study. In 29th Annual International Conference on Computer Science and Software Engineering, CASCON, pages 193–202.
Silva, P., Bezerra, C., and Machado, I. (2024). Toward a language-agnostic approach to detect test smells. In Anais do XXXVIII Simpósio Brasileiro de Engenharia de Software, SBES ’24, pages 686–692.
Soares, E., Aranda, M., Oliveira, N., Ribeiro, M., Gheyi, R., Souza, E., Machado, I., Santos, A., Fonseca, B., and Bonifácio, R. (2023a). Manual tests do smell! cataloging and identifying natural language test smells. In 2023 ACM/IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM), pages 1–11.
Soares, E., Ribeiro, M., Amaral, G., Gheyi, R., Fernandes, L., Garcia, A., Fonseca, B., and Santos, A. (2020). Refactoring test smells: A perspective from open-source developers. In Proceedings of the 5th Brazilian Symposium on Systematic and Automated Software Testing, SAST ’20, pages 50–59.
Soares, E., Ribeiro, M., Gheyi, R., Amaral, G., and Santos, A. (2023b). Refactoring test smells with JUnit 5: Why should developers keep up-to-date? IEEE Transactions on Software Engineering, 49(3):1152–1170.
Soares, E., Ribeiro, M., and Santos, A. (2024). A multimethod study of test smells: Cataloging removal and new types. In Proceedings of the XXIII Brazilian Symposium on Software Quality, SBQS ’24, pages 676–686.
Virgínio, T., Martins, L., Santana, R., Cruz, A., Rocha, L., Costa, H., and Machado, I. (2021). On the test smells detection: an empirical study on the jnose test accuracy. Journal of Software Engineering Research and Development, 9:8–1.
Wang, T., Golubev, Y., Smirnov, O., Li, J., Bryksin, T., and Ahmed, I. (2021). Pynose: A test smell detector for python. In 2021 36th IEEE/ACM international conference on automated software engineering (ASE), ASE, pages 593–605.
