Can LLMs Detect and Refactor Test Smells in Performance Tests?

• Adelson Ornellas UFAL
• Manoel Aranda UFAL
• Márcio Ribeiro UFAL
• Samira Silva UFLA
• Rohit Gheyi UFCG
• Baldoino Fonseca UFAL
• Ivan Machado UFBA

DOI: https://doi.org/10.5753/sast.2026.31279

Resumo

Performance testing is essential to ensure that software meets its responsiveness and scalability requirements, and tools such as k6 are used to write performance test scripts that exercise these aspects in a structured and optimized way. Like any code, regardless of the programming language, k6 scripts are subject to recurring bad practices, known as performance test smells, that can yield misleading results or waste resources. While Large Language Models (LLMs) have shown promise in detecting and refactoring conventional test smells, their effectiveness on performance test smells remains largely unexplored. This paper evaluates whether three widely used LLMs (Claude, GPT, and Gemini) can detect and refactor four performance test smells in real-world k6 scripts, and whether adding illustrative code examples to the prompt improves their performance. Using a dataset of 228 k6 scripts collected from 58 GitHub repositories and labeled by a static linter, we submitted 1,368 detection-and-refactoring queries across two prompt variants and manually reviewed 120 of the resulting refactorings. We find that detection performance is moderate and comparable across the three models (F1 micro between 0.49 and 0.54, and between 0.53 and 0.54 under the definition-only prompt), that adding code examples does not consistently improve detection and can even degrade it, and most notably that detection difficulty and refactoring difficulty are largely decoupled: the smells that are hardest to detect are often refactored almost flawlessly once found, whereas one of the most reliably detected smells accounts for nearly all incorrect and breaking refactorings. We attribute this asymmetry to a distinction between declarative and architectural fixes, and discuss its implications for trusting LLM-generated refactorings in performance testing.

Palavras-chave: Performance Test Smells, Large Language Models, k6

Referências

Wajdi Aljedaani, Anthony Peruma, Ahmed Aljohani, Mazen Alotaibi, Mohamed Wiem Mkaouer, Ali Ouni, Christian D. Newman, Abdullatif Ghallab, and Stephanie Ludi. 2021. Test Smell Detection Tools: A Systematic Mapping Study. In Evaluation and Assessment in Software Engineering. ACM, 170–180.

Manoel Aranda, Naelson Oliveira, Elvys Soares, Márcio Ribeiro, Davi Romão, Ullyanne Patriota, Rohit Gheyi, Emerson Souza, and Ivan Machado. 2024. A Catalog of Transformations to Remove Smells from Natural Language Tests. In Evaluation and Assessment in Software Engineering. ACM, 7–16.

Samuel Barbosa. 2025. Detecção de Test Smells em Testes de Carga Implementados no Framework K6. Bachelor’s Thesis. UFAL. Artifacts available at: [link].

Dave Bergmann. 2025. A List of Large Language Models (LLMs). Retrieved June 13, 2026 from [link]

Jonathan Cordeiro, Shayan Noei, and Ying Zou. 2026. An Empirical Study on the Code Refactoring Capability of Large Language Models. ACM Transactions on Software Engineering and Methodology (2026). DOI: 10.1145/3801158

Grafana Labs. 2025. k6 Documentation. Retrieved June 20, 2026 from [link]

Zhen Ming Jiang and Ahmed E. Hassan. 2015. A Survey on Load Testing of Large-Scale Software Systems. IEEE Trans. on Soft. Eng. 41, 11 (2015), 1091–1118.

Gustavo Lopes, Davi Romão, Elvys Soares, Márcio Ribeiro, Guilherme Amaral, Rohit Gheyi, and Ivan Machado. 2024. A Road to Find Them All: Towards an Agnostic Strategy for Test Smell Detection. In Brazilian Symposium on Software Quality. ACM, 231–241.

Keila Lucas, Rohit Gheyi, Márcio Ribeiro, Fabio Palomba, Luana Martins, and Elvys Soares. 2025. Investigating the Performance of Small Language Models in Detecting Test Smells in Manual Test Cases. In Brazilian Symposium on Software Engineering. SBC, 783–789.

Keila Lucas, Rohit Gheyi, Márcio Ribeiro, Fabio Palomba, Luana Martins, and Elvys Soares. 2026. An Empirical Study of Gemini 3 for Detecting Natural Language Test Smells in Manual Test Cases. Journal of Software Engineering Research and Development (2026).

Keila Lucas, Rohit Gheyi, Elvys Soares, Márcio Ribeiro, and Ivan Machado. 2024. Evaluating Large Language Models in Detecting Test Smells. In Brazilian Symposium on Software Engineering. SBC, 672–678.

Rian Melo, Pedro Simões, Rohit Gheyi, Marcelo d’Amorim, Márcio Ribeiro, Gustavo Soares, Eduardo Almeida, and Elvys Soares. 2026. Agentic LMs: Hunting Down Test Smells. IEEE Software 43, 1 (2026), 32–40.

Annibale Panichella, Sebastiano Panichella, Gordon Fraser, Anand Ashok Sawant, and Vincent J. Hellendoorn. 2022. Test Smells 20 Years Later: Detectability, Validity, and Reliability. Empirical Software Engineering 27, 7 (2022), 170.

Valeria Pontillo, Dario Amoroso d’Aragona, Fabiano Pecorelli, Dario Di Nucci, Filomena Ferrucci, and Fabio Palomba. 2024. Machine Learning-Based Test Smell Detection. Empirical Software Engineering 29, 2 (2024), 55.

Daniel Bruno Ramos, Railana Santana, and Ivan Machado. 2025. DANTES: Automating Detection and Refactoring of Test Smells. In Brazilian Symposium on Software Engineering (SBES’25) – Tools Track. SBC, 1003–1009.

Pranab Sahoo, Ayush Kumar Singh, Sriparna Saha, Vinija Jain, Samrat Mondal, and Aman Chadha. 2024. A Systematic Survey of Prompt Engineering in Large Language Models: Techniques and Applications. arXiv:2402.07927

Elvys Soares, Manoel Aranda, Naelson Oliveira, Márcio Ribeiro, Rohit Gheyi, Emerson Souza, Ivan Machado, André Santos, Baldoino Fonseca, and Rodrigo Bonifácio. 2023. Manual Tests Do Smell! Cataloging and Identifying Natural Language Test Smells. In Emp. Soft. Engineering and Measurement. IEEE, 1–11.

Elvys Soares, Márcio Ribeiro, Guilherme Amaral, Rohit Gheyi, Leo Fernandes, Alessandro Garcia, Baldoino Fonseca, and André L. M. Santos. 2020. Refactoring Test Smells: A Perspective from Open-Source Developers. In Brazilian Symposium on Systematic and Automated Software. ACM, 50–59.

Elvys Soares, Márcio Ribeiro, and André Santos. 2024. A Multimethod Study of Test Smells: Cataloging Removal and New Types. In Brazilian Symposium on Software Quality. SBC.

Gabriela Soares, Vanessa Santos, Márcio Ribeiro, Luana Almeida Martins, Valeria Pontillo, Manoel Aranda III, Rohit Gheyi, Ivan Machado, and Fabio Palomba. 2025. On the Harmfulness of Test Smells in Manual System Testing: A Controlled Experiment. In Empirical Software Engineering and Measurement. IEEE, 185–195.

Arie van Deursen, Leon Moonen, Alex van den Bergh, and Gerard Kok. 2001. Refactoring Test Code. In Extreme Programming and Flexible Processes in Software Engineering. 92–95.