Identifying and Addressing Test Smells in JavaScript: A Developer-Centric Study

• Jhonatan Oliveira UEFS
• Luigi Mateus UNEB
• Gabriel Amaral UEFS
• Tássio Virgínio UFBA
• Carla Bezerra UFC
• Ivan Machado UFBA
• Larissa Rocha UNEB / UEFS

DOI: https://doi.org/10.5753/sbes.2025.9901

Abstract

Test smells are poor practices in test code that can compromise maintainability, reliability, and clarity. While the concept has been widely studied in languages such as Java and Python, research on test smells in JavaScript remains limited—despite its prominence in modern development. To address this gap, we conducted a focus group study with JavaScript developers of varying experience levels to explore their perceptions of seven test smells. These smells—Anonymous Test, Comments Only Test, Overcommented Test, General Fixture, Test Without Description, Transcripting Test, and Sensitive Equality—are particularly relevant to the JavaScript ecosystem and had not been systematically examined in this context prior to our study. We applied thematic analysis to transcribed discussions, uncovering developers’ concerns, recognition patterns, and proposed mitigation strategies. Our results show that experience level strongly influences the ability to detect and refactor test smells, with junior developers often struggling to identify more subtle patterns. To the best of our knowledge, this is the first study to investigate JavaScript developers’ perceptions of test smells using a qualitative approach. Our findings reveal key challenges, offer practical insights for test improvement, and support the development of better training and tooling for JavaScript test quality.

Keywords: Test Smells, Focus Group, JavaScript

References

Wajdi Aljedaani, Anthony Peruma, Ahmed Aljohani, Mazen Alotaibi, Mohamed Wiem Mkaouer, Ali Ouni, Christian D. Newman, Abdullatif Ghallab, and Stephanie Ludi. 2021. Test Smell Detection Tools: A Systematic Mapping Study. In Proceedings of the 25th International Conference on Evaluation and Assessment in Software Engineering (Trondheim, Norway) (EASE ’21). Association for Computing Machinery, New York, NY, USA, 170–180. DOI: 10.1145/3463274.3463335

Manoel Aranda, Naelson Oliveira, Elvys Soares, Márcio Ribeiro, Davi Romão, Ullyanne Patriota, Rohit Gheyi, Emerson Souza, and Ivan Machado. 2024. A Catalog of Transformations to Remove Smells From Natural Language Tests. In Proceedings of the 28th International Conference on Evaluation and Assessment in Software Engineering (Salerno, Italy) (EASE ’24). Association for Computing Machinery, New York, NY, USA, 7–16. DOI: 10.1145/3661167.3661225

Sharon L. Baker. 1991. Improving Business Services through the Use of Focus Groups. RQ 30, 3 (1991), 377–385. [link]

Gabriele Bavota, Abdallah Qusef, Rocco Oliveto, Andrea De Lucia, and Dave Binkley. 2015. Are test smells really harmful? an empirical study. Empirical Software Engineering 20 (2015), 1052–1094.

Raymond P. L. Buse and Westley R. Weimer. 2008. A Metric for Software Readability. In Proceedings of the 2008 International Symposium on Software Testing and Analysis (ISSTA). Association for Computing Machinery, 121–130. DOI: 10.1145/1390630.1390647

Denivan Campos, Larissa Rocha, and Ivan Machado. 2021. Developers perception on the severity of test smells: an empirical study. arXiv preprint arXiv:2107.13902 (2021).

Maya Daneva and Niv Ahituv. 2012. What agile ERP consultants think of requirements engineering for inter-organizational ERP Systems: Insights from a Focus Group in BeNeLux. In 16th International Conference on Evaluation and Assessment in Software Engineering (EASE 2012). 284–288. DOI: 10.1049/ic.2012.0037

Maya Daneva, Andrea Herrmann, Nelly Condori-Fernandez, and Chong Wang. 2019. Understanding the Most In-demand Soft Skills in Requirements Engineering Practice: Insights from Two Focus Groups. In Proceedings of the 23rd International Conference on Evaluation and Assessment in Software Engineering (Copenhagen, Denmark) (EASE ’19). Association for Computing Machinery, New York, NY, USA, 284–290. DOI: 10.1145/3319008.3319352

Jonas De Bleser, Dario Di Nucci, and Coen De Roover. 2019. SoCRATES: Scala radar for test smells. In Proceedings of the Tenth ACM SIGPLAN Symposium on Scala (London, United Kingdom) (Scala ’19). Association for Computing Machinery, New York, NY, USA, 22–26. DOI: 10.1145/3337932.3338815

Norman E. Fenton and Shari Lawrence Pfleeger. 1998. Software Metrics: A Rigorous and Practical Approach (2nd ed.). PWS Publishing Co., USA.

Daniel Fernandes, Ivan Machado, and Rita Maciel. 2022. TEMPY: Test Smell Detector for Python. In Proceedings of the XXXVI Brazilian Symposium on Software Engineering (, Virtual Event, Brazil, ) (SBES ’22). Association for Computing Machinery, New York, NY, USA, 214–219. DOI: 10.1145/3555228.3555280

Vahid Garousi and Barış Küçük. 2018. Smells in software test code: A survey of knowledge in industry and academia. Journal of Systems and Software 138 (2018), 52–81. DOI: 10.1016/j.jss.2017.12.013

Ronald E. Goldsmith. 2000. The Focus Group Research Handbook. The Service Industries Journal 20, 3 (07 2000), 214. [link] Copyright - Copyright Frank Cass and Co. Ltd Jul 2000.

Judith Green and Nicki Thorogood. 2018. Qualitative methods for health research. (2018).

International Organization for Standardization. 2001. ISO/IEC 9126: Software engineering — Product quality. Technical Report ISO/IEC 9126. International Organization for Standardization. [link]

Dalton Jorge, Patricia Machado, and Wilkerson Andrade. 2021. Investigating Test Smells in JavaScript Test Code. In Proceedings of the 6th Brazilian Symposium on Systematic and Automated Software Testing (Joinville, Brazil) (SAST ’21). Association for Computing Machinery, New York, NY, USA, 36–45. DOI: 10.1145/3482909.3482915

Rick Kazman, Phil Bianco, James Ivers, and John Klein. 2020. Maintainability. Technical Report CMU/SEI-2020-TR-006. Software Engineering Institute, Carnegie Mellon University. DOI: 10.1184/R1/12954908

Vladimir Khorikov. 2020. Unit Testing Principles, Practices, and Patterns. Simon and Schuster.

J. Kontio, L. Lehtola, and J. Bragge. 2004. Using the focus group method in software engineering: obtaining practitioner and user experiences. In Proceedings. 2004 International Symposium on Empirical Software Engineering, 2004. ISESE ’04. 271–280. DOI: 10.1109/ISESE.2004.1334914

J. Kontio, L. Lehtola, and J. Bragge. 2004. Using the focus group method in software engineering: obtaining practitioner and user experiences. In Proceedings. 2004 International Symposium on Empirical Software Engineering, 2004. ISESE ’04. 271–280. DOI: 10.1109/ISESE.2004.1334914

Oliver T. Massey. 2011. A proposed model for the analysis and interpretation of focus groups in evaluation research. Evaluation and Program Planning 34, 1 (2011), 21–28. DOI: 10.1016/j.evalprogplan.2010.06.003

Gerard Meszaros, Shaun M. Smith, and Jennitta Andrea. 2003. The Test Automation Manifesto. In Extreme Programming and Agile Methods - XP/Agile Universe 2003, Frank Maurer and Don Wells (Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg.

Jhonatan Oliveira, Luigi Mateus, Tássio Virgínio, and Larissa Rocha. 2024. SNUTS.js: Sniffing Nasty Unit Test Smells in Javascript. In Anais do XXXVIII Simpósio Brasileiro de Engenharia de Software (Curitiba/PR). SBC, Porto Alegre, RS, Brasil, 720–726. DOI: 10.5753/sbes.2024.3563

Diana Papaioannou, Anthea Sutton, and Andrew Booth. 2016. Systematic approaches to a successful literature review. Systematic approaches to a successful literature review (2016), 1–336.

Anthony Peruma, Khalid Almalki, Christian D. Newman, Mohamed Wiem Mkaouer, Ali Ouni, and Fabio Palomba. 2020. tsDetect: an open source test smells detection tool. In Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering (Virtual Event, USA) (ESEC/FSE 2020). Association for Computing Machinery, New York, NY, USA, 1650–1654. DOI: 10.1145/3368089.3417921

Valeria Pontillo. 2024. Insights Into Test Code Quality Prediction: Managing Machine Learning Techniques. In Proceedings of the 28th International Conference on Evaluation and Assessment in Software Engineering (Salerno, Italy) (EASE ’24). Association for Computing Machinery, New York, NY, USA, 2. DOI: 10.1145/3661167.3661268

Dudekula Mohammad Rafi, Katam Reddy Kiran Moses, Kai Petersen, and Mika V Mäntylä. 2012. Benefits and limitations of automated software testing: Systematic literature review and practitioner survey. In 2012 7th international workshop on automation of software test (AST). IEEE, 36–42.

Railana Santana, Luana Martins, Larissa Rocha, Tássio Virgínio, Adriana Cruz, Heitor Costa, and Ivan Machado. 2020. RAIDE: a tool for Assertion Roulette and Duplicate Assert identification and refactoring. In Proceedings of the XXXIV Brazilian Symposium on Software Engineering (, Natal, Brazil, ) (SBES ’20). Association for Computing Machinery, New York, NY, USA, 374–379. DOI: 10.1145/3422392.3422510

Gabriel Crespo de Souza. [n. d.]. Test Smells: An Industry Developer Perspective. [link]

J.F. Templeton. 1994. The Focus Group: A Strategic Guide to Organizing, Conducting and Analyzing the Focus Group Interview. Probus Publishing Company. [link]

Arie van Deursen, Leon Moonen, Alex van den Bergh, and Gerard Kok. 2001. In Refactoring Test Code, M. Marchesi and G. Succi (Eds.). Proceedings 2nd International Conference on Extreme Programming and Flexible Processes in Software Engineering (XP2001).

Tássio Virgínio, Luana Almeida Martins, Larissa Rocha Soares, Railana Santana, Heitor Costa, and Ivan Machado. 2020. An empirical study of automaticallygenerated tests from the perspective of test smells. In Proceedings of the XXXIV Brazilian Symposium on Software Engineering (, Natal, Brazil, ) (SBES ’20). Association for Computing Machinery, New York, NY, USA, 92–96. DOI: 10.1145/3422392.3422412

Tássio Virgínio, Railana Santana, Luana Almeida Martins, Larissa Rocha Soares, Heitor Costa, and Ivan Machado. 2019. On the influence of Test Smells on Test Coverage. In Proceedings of the XXXIII Brazilian Symposium on Software Engineering (Salvador, Brazil) (SBES ’19). Association for Computing Machinery, New York, NY, USA, 467–471. DOI: 10.1145/3350768.3350775

Tássio Virgínio, Luana Martins, Railana Santana, Adriana Cruz, Larissa Rocha, Heitor Costa, and Ivan Machado. 2021. On the test smells detection: an empirical study on the JNose Test accuracy. Journal of Software Engineering Research and Development 9, 1 (Sep. 2021), 8:1 – 8:14. DOI: 10.5753/jserd.2021.1893

Tongjie Wang, Yaroslav Golubev, Oleg Smirnov, Jiawei Li, Timofey Bryksin, and Iftekhar Ahmed. 2022. PyNose: a test smell detector for python. In Proceedings of the 36th IEEE/ACM International Conference on Automated Software Engineering (Melbourne, Australia) (ASE ’21). IEEE Press, 593–605. DOI: 10.1109/ASE51524.2021.9678615

Richard Widdows, Tia A. Hensler, and Marlaya H. Wyncott. 1991. The Focus Group Interview: A Method for Assessing Users’ Evaluation of Library Service. College and Research Libraries 52, 4 (1991), 352–359. DOI: 10.5860/crl_52_04_352