SNUTS.js: Sniffing Nasty Unit Test Smells in Javascript
Resumo
Test smells indicate potential issues or weaknesses within the test code, which can compromise its effectiveness and maintainability. They highlight areas where improvements can enhance the overall quality of the test suite or testing practices. For instance, an example of a test smell is the Anonymous Test, where the test’s name lacks descriptive information about its function or purpose. Addressing these test smells can result in more robust and maintainable test suites, thus improving the reliability of the testing process. Despite significant research on these issues, tools are scarce for automatically detecting them, particularly in certain programming languages such as JavaScript. In the current landscape, existing test smell detection tools for JavaScript lack intuitiveness and graphical interfaces, and require extensive configuration, which may lead to low adoption within the developer community. To address this gap, we propose SNUTS.js, a tool designed to streamline the detection of test smells in JavaScript. Designed as an API, SNUTS.js offers versatility, allowing integration with various tools and environments. This tool goes beyond existing solutions by identifying previously undetected test smells, including the Anonymous Test, Comments Only Test, Overcommented, General Fixture, Transcripting Test, and Sensitive Equality. We also introduce a new test smell termed Test Without Description, which denotes a test case lacking descriptive text. In a preliminary evaluation, we constructed a dataset of tests sourced from real-world projects on GitHub. Through manual analysis, we identified 285 instances of test smells. SNUTS.js demonstrated a detection accuracy of 100% for three specific types of test smells, Anonymous Test, Overcommented, and General Fixture, all tailored to the JavaScript environment. Link to the video: https://youtu.be/89z0jy4Nu0s
Referências
Maurício Aniche. 2022. Effective Software Testing: A developer’s guide. Simon and Schuster.
Gabriele Bavota, Abdallah Qusef, Rocco Oliveto, Andrea De Lucia, and Dave Binkley. 2015. Are test smells really harmful? an empirical study. Empirical Software Engineering 20 (2015), 1052–1094.
Arie Deursen, Leon M.F. Moonen, A. Bergh, and Gerard Kok. 2001. Refactoring Test Code. In Refactoring Test Code. CWI (Centre for Mathematics and Computer Science), Amsterdam, The Netherlands, The Netherlands.
Daniel Fernandes, Ivan Machado, and Rita Maciel. 2022. TEMPY: Test Smell Detector for Python. In Proceedings of the XXXVI Brazilian Symposium on Software Engineering (<conf-loc>, <city>Virtual Event</city>, <country>Brazil</country>, </conf-loc>) (SBES ’22). Association for Computing Machinery, New York, NY, USA, 214–219. DOI: 10.1145/3555228.3555280
Vahid Garousi and Barış Küçük. 2018. Smells in software test code: A survey of knowledge in industry and academia. Journal of Systems and Software 138 (2018), 52–81. DOI: 10.1016/j.jss.2017.12.013
Rahul Gopinath, Carlos Jensen, and Alex Groce. 2014. Code Coverage for Suite Evaluation by Developers. In Proceedings of the 36th International Conference on Software Engineering (Hyderabad, India) (ICSE 2014). ACM, New York, NY, USA, 72–82. DOI: 10.1145/2568225.2568278
Giovanni Grano, Fabio Palomba, Dario Di Nucci, Andrea De Lucia, and Harald C Gall. 2019. Scented since the beginning: On the diffuseness of test smells in automatically generated test code. Journal of Systems and Software 156 (2019), 312–327.
Giovanni Grano, Fabio Palomba, Dario Di Nucci, Andrea De Lucia, and Harald C Gall. 2019. Scented since the beginning: On the diffuseness of test smells in automatically generated test code. Journal of Systems and Software 156 (2019), 312–327.
Dayne Guerra Calle, Julien Delplanque, and Stéphane Ducasse. 2019. Exposing Test Analysis Results with DrTests. In International Workshop on Smalltalk Technologies. Cologne, Germany. [link]
Dalton Jorge, Patricia Machado, and Wilkerson Andrade. 2021. Investigating Test Smells in JavaScript Test Code. In Proceedings of the 6th Brazilian Symposium on Systematic and Automated Software Testing (Joinville, Brazil) (SAST ’21). Association for Computing Machinery, New York, NY, USA, 36–45. DOI: 10.1145/3482909.3482915
Gerard Meszaros, Shaun M. Smith, and Jennitta Andrea. 2003. The Test Automation Manifesto. In Extreme Programming and Agile Methods - XP/Agile Universe 2003, Frank Maurer and Don Wells (Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg.
F. Palomba, D. Di Nucci, A. Panichella, R. Oliveto, and A. De Lucia. 2016. On the Diffusion of Test Smells in Automatically Generated Test Code: An Empirical Study. In 2016 IEEE/ACM 9th International Workshop on Search-Based Software Testing (SBST). IEEE, Austin, TX, United States.
Anthony Peruma, Khalid Almalki, Christian D. Newman, Mohamed Wiem Mkaouer, Ali Ouni, and Fabio Palomba. 2020. tsDetect: an open source test smells detection tool. In Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering (Virtual Event, USA) (ESEC/FSE 2020). Association for Computing Machinery, New York, NY, USA, 1650–1654. DOI: 10.1145/3368089.3417921
Roger Pressman. 2016. Software Engineering: A Practitioner’s Approach (8 ed.). McGraw-Hill, Inc., USA.
Railana Santana, Luana Martins, Larissa Rocha, Tássio Virgínio, Adriana Cruz, Heitor Costa, and Ivan Machado. 2020. RAIDE: a tool for Assertion Roulette and Duplicate Assert identification and refactoring. In Proceedings of the XXXIV Brazilian Symposium on Software Engineering (<conf-loc>, <city>Natal</city>, <country> Brazil</country>, </conf-loc>) (SBES ’20). Association for Computing Machinery, New York, NY, USA, 374–379. DOI: 10.1145/3422392.3422510
Andrew Costa Silva. 2022. Identificação e Caracterização de Test Smells em JavaScript. Instituto de Ciencias Exatas e Informática - Pontifícia Universidade 138 (2022), 52–81. [link]
D. Spadini, F. Palomba, A. Zaidman, M. Bruntink, and A. Bacchelli. 2018. On the Relation of Test Smells to Software Code Quality. In 2018 IEEE International Conference on Software Maintenance and Evolution (ICSME). 1–12. DOI: 10.1109/ICSME.2018.00010
Tássio Virgínio, Luana Almeida Martins, Larissa Rocha Soares, Railana Santana, Heitor Costa, and Ivan Machado. 2020. An empirical study of automaticallygenerated tests from the perspective of test smells. In Proceedings of the XXXIV Brazilian Symposium on Software Engineering (<conf-loc>, <city>Natal</city>, <country>Brazil</country>, </conf-loc>) (SBES ’20). Association for Computing Machinery, New York, NY, USA, 92–96. DOI: 10.1145/3422392.3422412
Tássio Virgínio, Luana Martins, Railana Santana, Adriana Cruz, Larissa Rocha, Heitor Costa, and Ivan Machado. 2021. On the test smells detection: an empirical study on the JNose Test accuracy. Journal of Software Engineering Research and Development 9, 1 (Sep. 2021), 8:1 – 8:14. DOI: 10.5753/jserd.2021.1893
Tongjie Wang, Yaroslav Golubev, Oleg Smirnov, Jiawei Li, Timofey Bryksin, and Iftekhar Ahmed. 2022. PyNose: a test smell detector for python. In Proceedings of the 36th IEEE/ACM International Conference on Automated Software Engineering (Melbourne, Australia) (ASE ’21). IEEE Press, 593–605. DOI: 10.1109/ASE51524.2021.9678615
Vahid Garousi Yusifoğlu, Yasaman Amannejad, and Aysu Betin Can. 2015. Software test-code engineering: A systematic mapping. Information and Software Technology 58 (2015), 123 – 147. DOI: 10.1016/j.infsof.2014.06.009
