Smart Prediction for Test Smell Refactorings
Abstract
Software plays a critical role in modern society, making software testing essential for ensuring quality. Poor testing practices, known as test smells, reduce test code quality and maintainability. Although refactoring is a widely used technique to address these issues, little is known about when developers choose to refactor test smells and whether these actions actually improve test code quality. This study introduces a machine learning approach to guide test smell refactoring. First, we aim to mine refactorings performed by developers to derive a catalog of test-specific refactorings. Findings show testing framework evolution helps address test smells. Second, we aim to understand whether developers target low-quality test codes to perform refactorings and their effects on code quality. Our findings reveal that developers tend to refactor structurally low-quality test code more often. Third, we aim to learn whether developers would perform refactorings and which refactorings they would apply to improve the test code quality. Results show that Support Vector Machine models predicted refactoring decisions with 30–100% accuracy. This paper is a summary of the full paper version [8] of the doctoral thesis [7].
References
Gabriele Bavota, Abdallah Qusef, Rocco Oliveto, Andrea Lucia, and Dave Binkley. 2015. Are Test Smells Really Harmful? An Empirical Study. Empirical Software Engineering 20, 4 (Aug. 2015), 1052–1094.
Denivan Campos, Larissa Rocha, and Ivan Machado. 2021. Developers perception on the severity of test smells: an empirical study. In Iberoamerican Conference on Software Engineering (Virtual Event). arxiv, Costa Rica, 1–14. arXiv:2107.13902
Arie Deursen, Leon M. Moonen, A. Bergh, and Gerard Kok. 2001. Refactoring Test Code. Technical Report. Centre for Mathematics and Computer Science, NLD.
Martin Fowler. 1999. Refactoring - Improving the Design of Existing Code. Addison-Wesley, Upper Saddle River, NJ.
Vahid Garousi and Baris Küçük. 2018. Smells in software test code: A survey of knowledge in industry and academia. Journal of Systems and Software 138 (2018), 52–81.
Luana Martins. 2024. Smart prediction for test smell refactorings. Ph.D. Dissertation. Universidade Federal da Bahia. [link]
Luana Martins, Heitor Costa, Fabio Palomba, and Ivan Machado. 2025. Smart prediction for test smell refactorings. In Anais estendidos do XVI Congresso Brasileiro de Software: Teoria e Prática - Concurso de Teses e Dissertações em Engenharia de Software (Recife/PE). SBC, Porto Alegre, RS, Brasil, 15.
Luana Martins, Taher A Ghaleb, Heitor Costa, and Ivan Machado. 2024. A comprehensive catalog of refactoring strategies to handle test smells in Javabased systems. Software Quality Journal 32, 2 (2024), 641–679.
Luana Martins, Valeria Pontillo, Heitor Costa, Filomena Ferrucci, Fabio Palomba, and Ivan Machado. 2025. Test code refactoring unveiled: where and how does it affect test code quality and effectiveness? Empirical Software Engineering 30, 1 (2025), 1–39.
Fabio Palomba, Dario Di Nucci, Annibale Panichella, Rocco Oliveto, and Andrea De Lucia. 2016. On the Diffusion of Test Smells in Automatically Generated Test Code: An Empirical Study. In Proceedings of the 9th International Workshop on Search-Based Software Testing (Austin, Texas). ACM, New York, NY, USA, 5–14.
Anthony Peruma, Steven Simmons, Eman Abdullah AlOmar, Christian D Newman, Mohamed Wiem Mkaouer, and Ali Ouni. 2022. How do i refactor this? An empirical study on refactoring trends and topics in Stack Overflow. Empirical Software Engineering 27, 1 (2022), 1–43.
Nildo Silva Junior, Larissa Rocha Soares, Luana Martins, and Ivan Machado. 2020. A survey on test practitioners’ awareness of test smells. CoRR abs/2003.05613 (2020), 1–14. arXiv:2003.05613 [link]
Davide Spadini, Martin Schvarcbacher, Ana-Maria Oprescu, Magiel Bruntink, and Alberto Bacchelli. 2020. Investigating Severity Thresholds for Test Smells. In Proceedings of the 17th International Conference on Mining Software Repositories (Seoul, Republic of Korea). ACM, NY, USA, 311–321.
Michele Tufano, Fabio Palomba, Gabriele Bavota, Massimiliano Di Penta, Rocco Oliveto, Andrea De Lucia, and Denys Poshyvanyk. 2016. An Empirical Investigation into the Nature of Test Smells. In Proceedings of the 31st IEEE/ACM International Conference on Automated Software Engineering (Singapore, Singapore). ACM, New York, NY, USA, 4–15.
