AromaLIA: A Language-Independent Approach to Detect Test Smells
Resumo
Test smells are indicators of poor practices or suboptimal decisions made during test code development, potentially undermining maintainability and evolution. Existing solutions for detecting test smells typically support only a limited set of programming languages (often just one), thereby requiring the development of new tools for each additional language. To address this limitation, this work proposes a language-independent approach for test smell detection.We conducted a Systematic Mapping Study (SMS) comprising 117 studies published up to August 2025, through which we identified the most frequently discussed and critical test smells across multiple programming languages. Building on these findings, we designed and evaluated our approach in C#, Java, JavaScript, TypeScript, and Python, covering 10 distinct test smells. Our evaluation achieved 97% precision, 96% recall, and a 97% F1-score. Furthermore, we implemented and released a test smell detection tool based on the proposed approach.
Palavras-chave:
Test Smell, Language-Independent Approach, Test Smell Detection
Referências
Wajdi Aljedaani, Anthony Peruma, Ahmed Aljohani, Mazen Alotaibi, Mohamed Wiem Mkaouer, Ali Ouni, Christian D. Newman, Abdullatif Ghallab, and Stephanie Ludi. 2021. Test Smell Detection Tools: A Systematic Mapping Study. In Proceedings of the 25th International Conference on Evaluation and Assessment in Software Engineering (Trondheim, Norway) (EASE ’21). Association for Computing Machinery, New York, NY, USA, 170–180. DOI: 10.1145/3463274.3463335
Luca Ardito, Luca Barbato, Marco Castelluccio, Riccardo Coppola, Calixte Denizet, Sylvestre Ledru, and Michele Valsesia. 2020. rust-code-analysis: A Rust Library to Analyze and Extract Maintainability Information from Source Codes. SoftwareX 12 (2020), 100635. DOI: 10.1016/j.softx.2020.100635
Alexandru Bodea. 2022. Pytest-Smell: A Smell Detection Tool for Python Unit Tests. In Proceedings of the 31st ACM SIGSOFT International Symposium on Software Testing and Analysis (Virtual, South Korea) (ISSTA 2022). Association for Computing Machinery, New York, NY, USA, 793–796. DOI: 10.1145/3533767.3543290
Denivan Campos, Luana Martins, and Ivan Machado. 2023. An Empirical Study on the Influence of Developers’ Experience on Software Test Code Quality. In Proceedings of the 21st Brazilian Symposium on Software Quality (Curitiba, Brazil) (SBQS ’23). Association for Computing Machinery, New York, NY, USA, Article 3, 10 pages. DOI: 10.1145/3571473.3571481
Zhongyan Chen, Suzanne M. Embury, and Markel Vigo. 2023. Who Is Afraid of Test Smells? Assessing Technical Debt from Developer Actions. In Testing Software and Systems. Springer, Cham, Switzerland, 160–175.
Arie Deursen, Leon M.F. Moonen, A. Bergh, and Gerard Kok. 2001. Refactoring Test Code. Technical Report. CWI (Centre for Mathematics and Computer Science), Amsterdam, The Netherlands.
Anna Rita Fasolino and Porfirio Tramontana. 2024. Test Smells Learning by a Gamification Approach. In Proceedings of the 3rd ACM International Workshop on Gamification in Software Development, Verification, and Validation (Vienna, Austria) (Gamify 2024). Association for Computing Machinery, New York, NY, USA, 30–33. DOI: 10.1145/3678869.3685687
Vahid Garousi and Barış Küçük. 2018. Smells in Software Test Code: A Survey of Knowledge in Industry and Academia. Journal of Systems and Software 138 (2018), 52–81. DOI: 10.1016/j.jss.2017.12.013
Ayesha Kiran, Wasi Haider Butt, Muhammad Waseem Anwar, Farooque Azam, and Bilal Maqbool. 2019. A Comprehensive Investigation of Modern Test Suite Optimization Trends, Tools and Techniques. IEEE Access 7 (2019), 89093–89117. DOI: 10.1109/ACCESS.2019.2926384
Xiangjun Liu and Ping Yu. 2022. Randoop-TSR: Random-based Test Generator with Test Suite Reduction. In Proceedings of the 13th Asia-Pacific Symposium on Internetware (Hohhot, China) (Internetware ’22). Association for Computing Machinery, New York, NY, USA, 221–230. DOI: 10.1145/3545258.3545280
Gustavo Lopes, Davi Romão, Elvys Soares, Márcio Ribeiro, Guilherme Amaral, Rohit Gheyi, and Ivan Machado. 2024. A Road to Find Them All: Towards an Agnostic Strategy for Test Smell Detection. In Proceedings of the 23rd Brazilian Symposium on Software Quality (Brasília, Brazil) (SBQS ’24). Association for Computing Machinery, New York, NY, USA, 231–241. DOI: 10.1145/3701625.3701662
Keila Lucas, Rohit Gheyi, Elvys Soares, Márcio Ribeiro, and Ivan Machado. 2024. Evaluating Large Language Models in Detecting Test Smells. In Anais do XXXVIII Simpósio Brasileiro de Engenharia de Software. SBC, Porto Alegre, RS, Brasil, 672–678. DOI: 10.5753/sbes.2024.3642
Luana Martins, Taher A. Ghaleb, Heitor Costa, and Ivan Machado. 2024. A Comprehensive Catalog of Refactoring Strategies to Handle Test Smells in Javabased Systems. Software Quality Journal 32, 2 (2024), 641–679. DOI: 10.1007/s11219-024-09663-7
Gerard Meszaros. 2007. xUnit Test Patterns: Refactoring Test Code. Addison-Wesley, Boston, MA, USA.
Partha P. Paul, Md Tonoy Akanda, M. Raihan Ullah, Dipto Mondal, Nazia S. Chowdhury, and Fazle M. Tawsif. [n. d.]. xNose: A Test Smell Detector for C#.
Anthony Peruma, Khalid Almalki, Christian D. Newman, Mohamed Wiem Mkaouer, Ali Ouni, and Fabio Palomba. 2020. tsDetect: An Open Source Test Smells Detection Tool. In Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering (Virtual Event, USA) (ESEC/FSE 2020). Association for Computing Machinery, New York, NY, USA, 1650–1654. DOI: 10.1145/3368089.3417921
Anthony Peruma and Christian D. Newman. 2021. On the Distribution of “Simple Stupid Bugs” in Unit Test Files: An Exploratory Study. In Proceedings of the 18th International Conference on Mining Software Repositories (MSR ’21). IEEE, 525–529. DOI: 10.1109/MSR52588.2021.00067
Anthony Peruma, Christian D. Newman, Mohamed Wiem Mkaouer, Ali Ouni, and Fabio Palomba. 2020. An Exploratory Study on the Refactoring of Unit Test Files in Android Applications. In Proceedings of the IEEE/ACM 42nd International Conference on Software Engineering Workshops (Seoul, Republic of Korea) (ICSEW ’20). Association for Computing Machinery, New York, NY, USA, 350–357. DOI: 10.1145/3387940.3392189
Railana Santana, Luana Martins, Tássio Virgínio, Larissa Rocha, Heitor Costa, and Ivan Machado. 2022. Refactoring Assertion Roulette and Duplicate Assert Test Smells: A Controlled Experiment. In Anais do XXV Congresso Ibero-Americano em Engenharia de Software (CIBSE). SBC, Porto Alegre, RS, Brasil, 263–277. DOI: 10.5753/cibse.2022.20977
Railana Santana, Luana Martins, Tássio Virgínio, Larissa Rocha, Heitor Costa, and Ivan Machado. 2024. An Empirical Evaluation of RAIDE: A Semi-Automated Approach for Test Smells Detection and Refactoring. Science of Computer Programming 231 (Jan. 2024). DOI: 10.1016/j.scico.2023.103013
E. G. Santana Jr, Jander Pereira Santos Junior, Erlon P. Almeida, Iftekhar Ahmed, Paulo Anselmo da Mota Silveira Neto, and Eduardo Santana de Almeida. 2025. Evaluating LLMs Effectiveness in Detecting and Correcting Test Smells: An Empirical Study. arXiv:2506.07594 [cs.SE] [link]
Max Schäfer, Sarah Nadi, Aryaz Eghbali, and Frank Tip. 2024. An Empirical Evaluation of Using Large Language Models for Automated Unit Test Generation. IEEE Transactions on Software Engineering 50, 1 (2024), 85–105. DOI: 10.1109/TSE.2023.3334955
Micah Schiewe, Jacob Curtis, Vincent Bushong, and Tomas Cerny. 2022. Advancing Static Code Analysis With Language-Agnostic Component Identification. IEEE Access 10 (2022), 30743–30761. DOI: 10.1109/ACCESS.2022.3160485
Nildo Silva Junior, Luana Martins, Larissa Rocha, Heitor Costa, and Ivan Machado. 2021. How Are Test Smells Treated in the Wild? A Tale of Two Empirical Studies. Journal of Software Engineering Research and Development 9, 1 (Sept. 2021), 9:1–9:16. DOI: 10.5753/jserd.2021.1802
Elvys Soares, Manoel Aranda III, Davi Romão, and Márcio Ribeiro. 2023. The Open Catalog of Test Smells. Available at [link].
Elvys Soares, Márcio Ribeiro, Rohit Gheyi, Guilherme Amaral, and André Santos. 2023. Refactoring Test Smells With JUnit 5: Why Should Developers Keep Up-to-Date? IEEE Transactions on Software Engineering 49, 3 (2023), 1152–1170. DOI: 10.1109/TSE.2022.3172654
Huynh Khanh Vi Tran, Michael Unterkalmsteiner, Jürgen Börstler, and Nauman bin Ali. 2021. Assessing Test Artifact Quality—A Tertiary Study. Information and Software Technology 139 (2021), 106620. DOI: 10.1016/j.infsof.2021.106620
Luca Ardito, Luca Barbato, Marco Castelluccio, Riccardo Coppola, Calixte Denizet, Sylvestre Ledru, and Michele Valsesia. 2020. rust-code-analysis: A Rust Library to Analyze and Extract Maintainability Information from Source Codes. SoftwareX 12 (2020), 100635. DOI: 10.1016/j.softx.2020.100635
Alexandru Bodea. 2022. Pytest-Smell: A Smell Detection Tool for Python Unit Tests. In Proceedings of the 31st ACM SIGSOFT International Symposium on Software Testing and Analysis (Virtual, South Korea) (ISSTA 2022). Association for Computing Machinery, New York, NY, USA, 793–796. DOI: 10.1145/3533767.3543290
Denivan Campos, Luana Martins, and Ivan Machado. 2023. An Empirical Study on the Influence of Developers’ Experience on Software Test Code Quality. In Proceedings of the 21st Brazilian Symposium on Software Quality (Curitiba, Brazil) (SBQS ’23). Association for Computing Machinery, New York, NY, USA, Article 3, 10 pages. DOI: 10.1145/3571473.3571481
Zhongyan Chen, Suzanne M. Embury, and Markel Vigo. 2023. Who Is Afraid of Test Smells? Assessing Technical Debt from Developer Actions. In Testing Software and Systems. Springer, Cham, Switzerland, 160–175.
Arie Deursen, Leon M.F. Moonen, A. Bergh, and Gerard Kok. 2001. Refactoring Test Code. Technical Report. CWI (Centre for Mathematics and Computer Science), Amsterdam, The Netherlands.
Anna Rita Fasolino and Porfirio Tramontana. 2024. Test Smells Learning by a Gamification Approach. In Proceedings of the 3rd ACM International Workshop on Gamification in Software Development, Verification, and Validation (Vienna, Austria) (Gamify 2024). Association for Computing Machinery, New York, NY, USA, 30–33. DOI: 10.1145/3678869.3685687
Vahid Garousi and Barış Küçük. 2018. Smells in Software Test Code: A Survey of Knowledge in Industry and Academia. Journal of Systems and Software 138 (2018), 52–81. DOI: 10.1016/j.jss.2017.12.013
Ayesha Kiran, Wasi Haider Butt, Muhammad Waseem Anwar, Farooque Azam, and Bilal Maqbool. 2019. A Comprehensive Investigation of Modern Test Suite Optimization Trends, Tools and Techniques. IEEE Access 7 (2019), 89093–89117. DOI: 10.1109/ACCESS.2019.2926384
Xiangjun Liu and Ping Yu. 2022. Randoop-TSR: Random-based Test Generator with Test Suite Reduction. In Proceedings of the 13th Asia-Pacific Symposium on Internetware (Hohhot, China) (Internetware ’22). Association for Computing Machinery, New York, NY, USA, 221–230. DOI: 10.1145/3545258.3545280
Gustavo Lopes, Davi Romão, Elvys Soares, Márcio Ribeiro, Guilherme Amaral, Rohit Gheyi, and Ivan Machado. 2024. A Road to Find Them All: Towards an Agnostic Strategy for Test Smell Detection. In Proceedings of the 23rd Brazilian Symposium on Software Quality (Brasília, Brazil) (SBQS ’24). Association for Computing Machinery, New York, NY, USA, 231–241. DOI: 10.1145/3701625.3701662
Keila Lucas, Rohit Gheyi, Elvys Soares, Márcio Ribeiro, and Ivan Machado. 2024. Evaluating Large Language Models in Detecting Test Smells. In Anais do XXXVIII Simpósio Brasileiro de Engenharia de Software. SBC, Porto Alegre, RS, Brasil, 672–678. DOI: 10.5753/sbes.2024.3642
Luana Martins, Taher A. Ghaleb, Heitor Costa, and Ivan Machado. 2024. A Comprehensive Catalog of Refactoring Strategies to Handle Test Smells in Javabased Systems. Software Quality Journal 32, 2 (2024), 641–679. DOI: 10.1007/s11219-024-09663-7
Gerard Meszaros. 2007. xUnit Test Patterns: Refactoring Test Code. Addison-Wesley, Boston, MA, USA.
Partha P. Paul, Md Tonoy Akanda, M. Raihan Ullah, Dipto Mondal, Nazia S. Chowdhury, and Fazle M. Tawsif. [n. d.]. xNose: A Test Smell Detector for C#.
Anthony Peruma, Khalid Almalki, Christian D. Newman, Mohamed Wiem Mkaouer, Ali Ouni, and Fabio Palomba. 2020. tsDetect: An Open Source Test Smells Detection Tool. In Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering (Virtual Event, USA) (ESEC/FSE 2020). Association for Computing Machinery, New York, NY, USA, 1650–1654. DOI: 10.1145/3368089.3417921
Anthony Peruma and Christian D. Newman. 2021. On the Distribution of “Simple Stupid Bugs” in Unit Test Files: An Exploratory Study. In Proceedings of the 18th International Conference on Mining Software Repositories (MSR ’21). IEEE, 525–529. DOI: 10.1109/MSR52588.2021.00067
Anthony Peruma, Christian D. Newman, Mohamed Wiem Mkaouer, Ali Ouni, and Fabio Palomba. 2020. An Exploratory Study on the Refactoring of Unit Test Files in Android Applications. In Proceedings of the IEEE/ACM 42nd International Conference on Software Engineering Workshops (Seoul, Republic of Korea) (ICSEW ’20). Association for Computing Machinery, New York, NY, USA, 350–357. DOI: 10.1145/3387940.3392189
Railana Santana, Luana Martins, Tássio Virgínio, Larissa Rocha, Heitor Costa, and Ivan Machado. 2022. Refactoring Assertion Roulette and Duplicate Assert Test Smells: A Controlled Experiment. In Anais do XXV Congresso Ibero-Americano em Engenharia de Software (CIBSE). SBC, Porto Alegre, RS, Brasil, 263–277. DOI: 10.5753/cibse.2022.20977
Railana Santana, Luana Martins, Tássio Virgínio, Larissa Rocha, Heitor Costa, and Ivan Machado. 2024. An Empirical Evaluation of RAIDE: A Semi-Automated Approach for Test Smells Detection and Refactoring. Science of Computer Programming 231 (Jan. 2024). DOI: 10.1016/j.scico.2023.103013
E. G. Santana Jr, Jander Pereira Santos Junior, Erlon P. Almeida, Iftekhar Ahmed, Paulo Anselmo da Mota Silveira Neto, and Eduardo Santana de Almeida. 2025. Evaluating LLMs Effectiveness in Detecting and Correcting Test Smells: An Empirical Study. arXiv:2506.07594 [cs.SE] [link]
Max Schäfer, Sarah Nadi, Aryaz Eghbali, and Frank Tip. 2024. An Empirical Evaluation of Using Large Language Models for Automated Unit Test Generation. IEEE Transactions on Software Engineering 50, 1 (2024), 85–105. DOI: 10.1109/TSE.2023.3334955
Micah Schiewe, Jacob Curtis, Vincent Bushong, and Tomas Cerny. 2022. Advancing Static Code Analysis With Language-Agnostic Component Identification. IEEE Access 10 (2022), 30743–30761. DOI: 10.1109/ACCESS.2022.3160485
Nildo Silva Junior, Luana Martins, Larissa Rocha, Heitor Costa, and Ivan Machado. 2021. How Are Test Smells Treated in the Wild? A Tale of Two Empirical Studies. Journal of Software Engineering Research and Development 9, 1 (Sept. 2021), 9:1–9:16. DOI: 10.5753/jserd.2021.1802
Elvys Soares, Manoel Aranda III, Davi Romão, and Márcio Ribeiro. 2023. The Open Catalog of Test Smells. Available at [link].
Elvys Soares, Márcio Ribeiro, Rohit Gheyi, Guilherme Amaral, and André Santos. 2023. Refactoring Test Smells With JUnit 5: Why Should Developers Keep Up-to-Date? IEEE Transactions on Software Engineering 49, 3 (2023), 1152–1170. DOI: 10.1109/TSE.2022.3172654
Huynh Khanh Vi Tran, Michael Unterkalmsteiner, Jürgen Börstler, and Nauman bin Ali. 2021. Assessing Test Artifact Quality—A Tertiary Study. Information and Software Technology 139 (2021), 106620. DOI: 10.1016/j.infsof.2021.106620
Publicado
08/09/2026
Como Citar
SILVA, Publio; BEZERRA, Carla; MACHADO, Ivan.
AromaLIA: A Language-Independent Approach to Detect Test Smells. In: CONCURSO DE TESES E DISSERTAÇÕES EM ENGENHARIA DE SOFTWARE (MESTRADO) - CONGRESSO BRASILEIRO DE SOFTWARE: TEORIA E PRÁTICA (CBSOFT), 17. , 2026, Cuiabá/MT.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2026
.
p. 3-11.
DOI: https://doi.org/10.5753/cbsoft_estendido.2026.29714.
