Toward a Language-Agnostic Approach to Detect Test Smells
Resumo
Tests play a crucial role in software development by ensuring code quality. However, test code can suffer from “smells” — poor implementation choices that hinder maintainability and evolution. Numerous studies have addressed test smells in various programming languages, proposing tools for detecting them in Java, C++, Scala, and others. These tools employ techniques such as information retrieval, metrics analysis, and abstract syntax tree (AST) parsing. However, their focus on specific languages limits their generalizability and applicability to other languages and test frameworks. This challenge is similar to issues found in code smell detection and static code analysis. Therefore, this work proposes a language-agnostic approach to detect test smells. Our approach leverages AST parsing to extract relevant information from the test code, followed by test smell detection based on this extracted data. This method aims to facilitate the detection of test smells across various programming languages and test frameworks, enhancing the tool’s generalizability and usability. To check the viability of our approach, we created a proof of concept using two test smells and two different languages.
Palavras-chave:
Test smell detection, language-agnostic
Referências
Wajdi Aljedaani, Anthony Peruma, Ahmed Aljohani, Mazen Alotaibi, Mohamed Wiem Mkaouer, Ali Ouni, Christian D. Newman, Abdullatif Ghallab, and Stephanie Ludi. 2021. Test Smell Detection Tools: A Systematic Mapping Study. In Proceedings of the 25th International Conference on Evaluation and Assessment in Software Engineering (EASE ’21). Association for Computing Machinery, New York, NY, USA, 170–180. DOI: 10.1145/3463274.3463335
Luca Ardito, Luca Barbato, Marco Castelluccio, Riccardo Coppola, Calixte Denizet, Sylvestre Ledru, and Michele Valsesia. 2020. rust-code-analysis: A Rust library to analyze and extract maintainability information from source codes. SoftwareX 12 (2020), 100635. DOI: 10.1016/j.softx.2020.100635
P. Baker, D. Evans, J. Grabowski, H. Neukirchen, and B. Zeiss. 2006. TRex - The Refactoring and Metrics Tool for TTCN-3 Test Specifications. In Testing: Academic Industrial Conference - Practice And Research Techniques (TAIC PART’06). 90–94. DOI: 10.1109/TAIC-PART.2006.35
Jonathan Bell, Gail Kaiser, Eric Melski, and Mohan Dattatreya. 2015. Efficient dependency detection for safe Java test acceleration. In Proceedings of the 2015 10th Joint Meeting on Foundations of Software Engineering (ESEC/FSE 2015). Association for Computing Machinery, New York, NY, USA, 770–781. DOI: 10.1145/2786805.2786823
Moritz Beller, Georgios Gousios, Annibale Panichella, and Andy Zaidman. 2015. When, how, and why developers (do not) test in their IDEs. In Proceedings of the 2015 10th Joint Meeting on Foundations of Software Engineering (ESEC/FSE 2015). Association for Computing Machinery, New York, NY, USA, 179–190. DOI: 10.1145/2786805.2786843
Jonas De Bleser, Dario Di Nucci, and Coen De Roover. 2019. SoCRATES: Scala radar for test smells. In Proceedings of the Tenth ACM SIGPLAN Symposium on Scala (Scala ’19). Association for Computing Machinery, New York, NY, USA, 22–26. DOI: 10.1145/3337932.3338815
Julien Delplanque, Stéphane Ducasse, Guillermo Polito, Andrew P. Black, and Anne Etien. 2019. Rotten Green Tests. In 2019 IEEE/ACM 41st International Conference on Software Engineering (ICSE). 500–511. DOI: 10.1109/ICSE.2019.00062
Arie Deursen, Leon M.F. Moonen, A. Bergh, and Gerard Kok. 2001. Refactoring test code. Technical Report. NLD.
S. Ducasse, M. Rieger, and S. Demeyer. 1999. A language independent approach for detecting duplicated code. In Proceedings IEEE International Conference on Software Maintenance - 1999 (ICSM’99). ’Software Maintenance for Business Change’ (Cat. No.99CB36360). 109–118. DOI: 10.1109/ICSM.1999.792593
Vahid Garousi and Barış Küçük. 2018. Smells in software test code: A survey of knowledge in industry and academia. Journal of Systems and Software 138 (2018), 52–81. DOI: 10.1016/j.jss.2017.12.013
Michaela Greiler, Arie van Deursen, and Margaret-Anne Storey. 2013. Automated Detection of Test Fixture Strategies and Smells. In 2013 IEEE Sixth International Conference on Software Testing, Verification and Validation. 322–331. DOI: 10.1109/ICST.2013.45
Michaela Greiler, Andy Zaidman, Arie van Deursen, and Margaret-Anne Storey. 2013. Strategies for avoiding text fixture smells during software evolution. In 2013 10th Working Conference on Mining Software Repositories (MSR). 387–396. DOI: 10.1109/MSR.2013.6624053
Chen Huo and James Clause. 2014. Improving oracle quality by detecting brittle assertions and unused inputs in tests. In Proceedings of the 22nd ACM SIGSOFT International Symposium on Foundations of Software Engineering (FSE 2014). Association for Computing Machinery, New York, NY, USA, 621–631. DOI: 10.1145/2635868.2635917
Negar Koochakzadeh and Vahid Garousi. 2010. A tester-assisted methodology for test redundancy detection. Adv. Soft. Eng. 2010, Article 6 (jan 2010), 13 pages. DOI: 10.1155/2010/932686
Stefano Lambiase, Andrea Cupito, Fabiano Pecorelli, Andrea De Lucia, and Fabio Palomba. 2020. Just-In-Time Test Smell Detection and Refactoring: The DARTS Project. In Proceedings of the 28th International Conference on Program Comprehension (ICPC ’20). Association for Computing Machinery, New York, NY, USA, 441–445. DOI: 10.1145/3387904.3389296
Fabio Palomba, Andy Zaidman, and Andrea De Lucia. 2018. Automatic Test Smell Detection Using Information Retrieval Techniques. In 2018 IEEE International Conference on Software Maintenance and Evolution (ICSME). 311–322. DOI: 10.1109/ICSME.2018.00040
Annibale Panichella, Sebastiano Panichella, Gordon Fraser, Anand Ashok Sawant, and Vincent J. Hellendoorn. 2022. Test smells 20 years later: detectability, validity, and reliability. Empirical Software Engineering 27, 7 (20 Sep 2022), 170. DOI: 10.1007/s10664-022-10207-5
Anthony Peruma, Khalid Almalki, Christian D. Newman, Mohamed Wiem Mkaouer, Ali Ouni, and Fabio Palomba. 2020. tsDetect: an open source test smells detection tool. In Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE 2020). Association for Computing Machinery, New York, NY, USA, 1650–1654. DOI: 10.1145/3368089.3417921
Gordana Rakić, Zoran Budimac, and Miloš Savić. 2013. Language independent framework for static code analysis. In Proceedings of the 6th Balkan Conference in Informatics (BCI ’13). Association for Computing Machinery, New York, NY, USA, 236–243. DOI: 10.1145/2490257.2490273
Railana Santana, Luana Martins, Larissa Rocha, Tássio Virgínio, Adriana Cruz, Heitor Costa, and Ivan Machado. 2020. RAIDE: a tool for Assertion Roulette and Duplicate Assert identification and refactoring. In Proceedings of the XXXIV Brazilian Symposium on Software Engineering (SBES ’20). Association for Computing Machinery, New York, NY, USA, 374–379. DOI: 10.1145/3422392.3422510
Micah Schiewe, Jacob Curtis, Vincent Bushong, and Tomas Cerny. 2022. Advancing Static Code Analysis With Language-Agnostic Component Identification. IEEE Access 10 (2022), 30743–30761. DOI: 10.1109/ACCESS.2022.3160485
Masayuki Taniguchi, Shinsuke Matsumoto, and Shinji Kusumoto. 2021. JTDog: a Gradle Plugin for Dynamic Test Smell Detection. In 2021 36th IEEE/ACM International Conference on Automated Software Engineering (ASE). 1271–1275. DOI: 10.1109/ASE51524.2021.9678529
Huynh Khanh Vi Tran, Michael Unterkalmsteiner, Jürgen Börstler, and Nauman bin Ali. 2021. Assessing test artifact quality—A tertiary study. Information and Software Technology 139 (2021), 106620. DOI: 10.1016/j.infsof.2021.106620
Bart Van Rompaey, Bart Du Bois, Serge Demeyer, and Matthias Rieger. 2007. On The Detection of Test Smells: A Metrics-Based Approach for General Fixture and Eager Test. IEEE Transactions on Software Engineering 33, 12 (2007), 800–817. DOI: 10.1109/TSE.2007.70745
Tássio Virgínio, Luana Martins, Larissa Rocha, Railana Santana, Adriana Cruz, Heitor Costa, and Ivan Machado. 2020. JNose: Java Test Smell Detector. In Proceedings of the XXXIV Brazilian Symposium on Software Engineering (SBES ’20). Association for Computing Machinery, New York, NY, USA, 564–569. DOI: 10.1145/3422392.3422499
Tongjie Wang, Yaroslav Golubev, Oleg Smirnov, Jiawei Li, Timofey Bryksin, and Iftekhar Ahmed. 2021. PyNose: A Test Smell Detector For Python. In 2021 36th IEEE/ACM International Conference on Automated Software Engineering (ASE). 593–605. DOI: 10.1109/ASE51524.2021.9678615
Claes Wohlin, Per Runeson, Martin Höst, Magnus C Ohlsson, Björn Regnell, and Anders Wesslén. 2012. Experimentation in software engineering. Springer Science & Business Media.
Sai Zhang, Darioush Jalali, Jochen Wuttke, Kıvanç Muşlu, Wing Lam, Michael D. Ernst, and David Notkin. 2014. Empirically revisiting the test independence assumption. In Proceedings of the 2014 International Symposium on Software Testing and Analysis (ISSTA 2014). Association for Computing Machinery, New York, NY, USA, 385–396. DOI: 10.1145/2610384.2610404
Luca Ardito, Luca Barbato, Marco Castelluccio, Riccardo Coppola, Calixte Denizet, Sylvestre Ledru, and Michele Valsesia. 2020. rust-code-analysis: A Rust library to analyze and extract maintainability information from source codes. SoftwareX 12 (2020), 100635. DOI: 10.1016/j.softx.2020.100635
P. Baker, D. Evans, J. Grabowski, H. Neukirchen, and B. Zeiss. 2006. TRex - The Refactoring and Metrics Tool for TTCN-3 Test Specifications. In Testing: Academic Industrial Conference - Practice And Research Techniques (TAIC PART’06). 90–94. DOI: 10.1109/TAIC-PART.2006.35
Jonathan Bell, Gail Kaiser, Eric Melski, and Mohan Dattatreya. 2015. Efficient dependency detection for safe Java test acceleration. In Proceedings of the 2015 10th Joint Meeting on Foundations of Software Engineering (ESEC/FSE 2015). Association for Computing Machinery, New York, NY, USA, 770–781. DOI: 10.1145/2786805.2786823
Moritz Beller, Georgios Gousios, Annibale Panichella, and Andy Zaidman. 2015. When, how, and why developers (do not) test in their IDEs. In Proceedings of the 2015 10th Joint Meeting on Foundations of Software Engineering (ESEC/FSE 2015). Association for Computing Machinery, New York, NY, USA, 179–190. DOI: 10.1145/2786805.2786843
Jonas De Bleser, Dario Di Nucci, and Coen De Roover. 2019. SoCRATES: Scala radar for test smells. In Proceedings of the Tenth ACM SIGPLAN Symposium on Scala (Scala ’19). Association for Computing Machinery, New York, NY, USA, 22–26. DOI: 10.1145/3337932.3338815
Julien Delplanque, Stéphane Ducasse, Guillermo Polito, Andrew P. Black, and Anne Etien. 2019. Rotten Green Tests. In 2019 IEEE/ACM 41st International Conference on Software Engineering (ICSE). 500–511. DOI: 10.1109/ICSE.2019.00062
Arie Deursen, Leon M.F. Moonen, A. Bergh, and Gerard Kok. 2001. Refactoring test code. Technical Report. NLD.
S. Ducasse, M. Rieger, and S. Demeyer. 1999. A language independent approach for detecting duplicated code. In Proceedings IEEE International Conference on Software Maintenance - 1999 (ICSM’99). ’Software Maintenance for Business Change’ (Cat. No.99CB36360). 109–118. DOI: 10.1109/ICSM.1999.792593
Vahid Garousi and Barış Küçük. 2018. Smells in software test code: A survey of knowledge in industry and academia. Journal of Systems and Software 138 (2018), 52–81. DOI: 10.1016/j.jss.2017.12.013
Michaela Greiler, Arie van Deursen, and Margaret-Anne Storey. 2013. Automated Detection of Test Fixture Strategies and Smells. In 2013 IEEE Sixth International Conference on Software Testing, Verification and Validation. 322–331. DOI: 10.1109/ICST.2013.45
Michaela Greiler, Andy Zaidman, Arie van Deursen, and Margaret-Anne Storey. 2013. Strategies for avoiding text fixture smells during software evolution. In 2013 10th Working Conference on Mining Software Repositories (MSR). 387–396. DOI: 10.1109/MSR.2013.6624053
Chen Huo and James Clause. 2014. Improving oracle quality by detecting brittle assertions and unused inputs in tests. In Proceedings of the 22nd ACM SIGSOFT International Symposium on Foundations of Software Engineering (FSE 2014). Association for Computing Machinery, New York, NY, USA, 621–631. DOI: 10.1145/2635868.2635917
Negar Koochakzadeh and Vahid Garousi. 2010. A tester-assisted methodology for test redundancy detection. Adv. Soft. Eng. 2010, Article 6 (jan 2010), 13 pages. DOI: 10.1155/2010/932686
Stefano Lambiase, Andrea Cupito, Fabiano Pecorelli, Andrea De Lucia, and Fabio Palomba. 2020. Just-In-Time Test Smell Detection and Refactoring: The DARTS Project. In Proceedings of the 28th International Conference on Program Comprehension (ICPC ’20). Association for Computing Machinery, New York, NY, USA, 441–445. DOI: 10.1145/3387904.3389296
Fabio Palomba, Andy Zaidman, and Andrea De Lucia. 2018. Automatic Test Smell Detection Using Information Retrieval Techniques. In 2018 IEEE International Conference on Software Maintenance and Evolution (ICSME). 311–322. DOI: 10.1109/ICSME.2018.00040
Annibale Panichella, Sebastiano Panichella, Gordon Fraser, Anand Ashok Sawant, and Vincent J. Hellendoorn. 2022. Test smells 20 years later: detectability, validity, and reliability. Empirical Software Engineering 27, 7 (20 Sep 2022), 170. DOI: 10.1007/s10664-022-10207-5
Anthony Peruma, Khalid Almalki, Christian D. Newman, Mohamed Wiem Mkaouer, Ali Ouni, and Fabio Palomba. 2020. tsDetect: an open source test smells detection tool. In Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE 2020). Association for Computing Machinery, New York, NY, USA, 1650–1654. DOI: 10.1145/3368089.3417921
Gordana Rakić, Zoran Budimac, and Miloš Savić. 2013. Language independent framework for static code analysis. In Proceedings of the 6th Balkan Conference in Informatics (BCI ’13). Association for Computing Machinery, New York, NY, USA, 236–243. DOI: 10.1145/2490257.2490273
Railana Santana, Luana Martins, Larissa Rocha, Tássio Virgínio, Adriana Cruz, Heitor Costa, and Ivan Machado. 2020. RAIDE: a tool for Assertion Roulette and Duplicate Assert identification and refactoring. In Proceedings of the XXXIV Brazilian Symposium on Software Engineering (SBES ’20). Association for Computing Machinery, New York, NY, USA, 374–379. DOI: 10.1145/3422392.3422510
Micah Schiewe, Jacob Curtis, Vincent Bushong, and Tomas Cerny. 2022. Advancing Static Code Analysis With Language-Agnostic Component Identification. IEEE Access 10 (2022), 30743–30761. DOI: 10.1109/ACCESS.2022.3160485
Masayuki Taniguchi, Shinsuke Matsumoto, and Shinji Kusumoto. 2021. JTDog: a Gradle Plugin for Dynamic Test Smell Detection. In 2021 36th IEEE/ACM International Conference on Automated Software Engineering (ASE). 1271–1275. DOI: 10.1109/ASE51524.2021.9678529
Huynh Khanh Vi Tran, Michael Unterkalmsteiner, Jürgen Börstler, and Nauman bin Ali. 2021. Assessing test artifact quality—A tertiary study. Information and Software Technology 139 (2021), 106620. DOI: 10.1016/j.infsof.2021.106620
Bart Van Rompaey, Bart Du Bois, Serge Demeyer, and Matthias Rieger. 2007. On The Detection of Test Smells: A Metrics-Based Approach for General Fixture and Eager Test. IEEE Transactions on Software Engineering 33, 12 (2007), 800–817. DOI: 10.1109/TSE.2007.70745
Tássio Virgínio, Luana Martins, Larissa Rocha, Railana Santana, Adriana Cruz, Heitor Costa, and Ivan Machado. 2020. JNose: Java Test Smell Detector. In Proceedings of the XXXIV Brazilian Symposium on Software Engineering (SBES ’20). Association for Computing Machinery, New York, NY, USA, 564–569. DOI: 10.1145/3422392.3422499
Tongjie Wang, Yaroslav Golubev, Oleg Smirnov, Jiawei Li, Timofey Bryksin, and Iftekhar Ahmed. 2021. PyNose: A Test Smell Detector For Python. In 2021 36th IEEE/ACM International Conference on Automated Software Engineering (ASE). 593–605. DOI: 10.1109/ASE51524.2021.9678615
Claes Wohlin, Per Runeson, Martin Höst, Magnus C Ohlsson, Björn Regnell, and Anders Wesslén. 2012. Experimentation in software engineering. Springer Science & Business Media.
Sai Zhang, Darioush Jalali, Jochen Wuttke, Kıvanç Muşlu, Wing Lam, Michael D. Ernst, and David Notkin. 2014. Empirically revisiting the test independence assumption. In Proceedings of the 2014 International Symposium on Software Testing and Analysis (ISSTA 2014). Association for Computing Machinery, New York, NY, USA, 385–396. DOI: 10.1145/2610384.2610404
Publicado
30/09/2024
Como Citar
SILVA, Publio Blenilio Tavares; BEZERRA, Carla; MACHADO, Ivan.
Toward a Language-Agnostic Approach to Detect Test Smells. In: SIMPÓSIO BRASILEIRO DE ENGENHARIA DE SOFTWARE (SBES), 38. , 2024, Curitiba/PR.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2024
.
p. 683-689.
DOI: https://doi.org/10.5753/sbes.2024.3647.
