AriesLinter: Sniffing Test Smells Before They Happen
Abstract
Software quality depends not only on production code but also on the quality of test code. Among the factors that compromise this quality, test smells stand out, poor implementation patterns in tests that negatively affect the readability, maintainability, and reliability of the test suite. Although tools such as tsDetect and JNose support test smell detection, they operate post-hoc, requiring the code to be complete and analyzed outside the development environment. Other tools, like RAIDE, offer detection within the IDE via plugins; however, the analysis is not instantaneous and still occurs after the code is written. These reactive approaches result in delayed feedback and significantly increase the refactoring effort. In this context, we propose AriesLinter, a static analysis tool that can be integrated into major development environments and is capable of detecting test smells in real time, as the test code is being written. Built on top of the Checkstyle infrastructure, AriesLinter automatically detects 10 of the most recurrent test smells reported in the literature and commonly found in Java projects. The tool is cross-platform and compatible with multiple IDEs. By anticipating the detection of poor patterns and embedding it into the natural workflow of developers, AriesLinter acts proactively, reducing the likelihood of test smells being introduced into the codebase and fostering a culture of continuous attention to test quality. Demo video: https://youtu.be/Cj6mQGGHlvk
References
Frans Kunst. 1988. Lint, a C Program Checker. (1988).
Nathaniel Ayewah, William Pugh, David Hovemeyer, J David Morgenthaler, and John Penix. 2008. Using static analysis to find bugs. IEEE software 25, 5 (2008), 22–29.
Moritz Beller, Georgios Gousios, Annibale Panichella, Sebastian Proksch, Sven Amann, and Andy Zaidman. 2017. Developer testing in the ide: Patterns, beliefs, and behavior. IEEE Transactions on Software Engineering 45, 3 (2017), 261–284.
Alexandru Bodea. 2022. Pytest-Smell: a smell detection tool for Python unit tests. In Proceedings of the 31st ACM SIGSOFT International Symposium on Software Testing and Analysis. 793–796.
Checkstyle Development Team. [n. d.]. Checkstyle - A development tool to help programmers write Java code that adheres to a coding standard. [link]. Accessed: 2025-05-10.
Humberto Damasceno, Carla Bezerra, Denivan Campos, Ivan Machado, and Emanuel Coutinho. 2023. Test smell refactoring revisited: What can internal quality attributes and developers’ experience tell us? Journal of Software Engineering Research and Development 11, 1 (Oct. 2023), 13:1 – 13:16. DOI: 10.5753/jserd.2023.3195
Jonas De Bleser, Dario Di Nucci, and Coen De Roover. 2019. Socrates: Scala radar for test smells. In Proceedings of the Tenth ACM SIGPLAN Symposium on Scala. 22–26.
Vahid Garousi, Baris Kucuk, and Michael Felderer. 2019. What We Know About Smells in Software Test Code. IEEE Software 36, 3 (2019), 61–73. DOI: 10.1109/MS.2018.2875843
Vahid Garousi and Barış Küçük. 2018. Smells in software test code: A survey of knowledge in industry and academia. Journal of Systems and Software 138 (2018), 52–81. DOI: 10.1016/j.jss.2017.12.013
Michaela Greiler, Arie van Deursen, and Margaret-Anne Storey. 2012. Test confessions: A study of testing practices for plug-in systems. In 2012 34th International Conference on Software Engineering (ICSE). IEEE, 244–254.
P. Louridas. 2006. Static code analysis. IEEE Software 23, 4 (2006), 58–61. DOI: 10.1109/MS.2006.114
Luana Martins, Taher A. Ghaleb, Heitor Costa, and Ivan Machado. 2024. A comprehensive catalog of refactoring strategies to handle test smells in Javabased systems. Software Quality Journal 32, 2 (March 2024), 641–679. DOI: 10. 1007/s11219-024-09663-7
Gerard Meszaros. 2007. xUnit test patterns: Refactoring test code. Pearson Education.
Jhonatan Oliveira, Luigi Mateus, Tássio Virgínio, and Larissa Rocha. 2024. SNUTS. js: Sniffing Nasty Unit Test Smells in Javascript. In Simpósio Brasileiro de Engenharia de Software (SBES). SBC, 720–726.
Anthony Peruma, Khalid Almalki, Christian D. Newman, Mohamed Wiem Mkaouer, Ali Ouni, and Fabio Palomba. 2020. tsDetect: an open source test smells detection tool. In Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering (Virtual Event, USA) (ESEC/FSE 2020). Association for Computing Machinery, New York, NY, USA, 1650–1654. DOI: 10.1145/3368089.3417921
Valeria Pontillo, Luana Martins, Ivan Machado, Fabio Palomba, and Filomena Ferrucci. 2025. An empirical investigation into the capabilities of anomaly detection approaches for test smell detection. Journal of Systems and Software 222 (2025), 112320. DOI: 10.1016/j.jss.2024.112320
Railana Santana, Luana Martins, Tássio Virgínio, Larissa Soares, Heitor Costa, and Ivan Machado. 2022. Refactoring assertion roulette and duplicate assert test smells: a controlled experiment. arXiv preprint arXiv:2207.05539 (2022).
Railana Santana, Luana Martins, Tássio Virgínio, Larissa Rocha, Heitor Costa, and Ivan Machado. 2024. An empirical evaluation of RAIDE: A semi-automated approach for test smells detection and refactoring. Science of Computer Programming 231 (2024), 103013. DOI: 10.1016/j.scico.2023.103013
Nildo Silva Junior, Luana Martins, Larissa Rocha, Heitor Costa, and Ivan Machado. 2021. How are test smells treated in the wild? A tale of two empirical studies. Journal of Software Engineering Research and Development 9, 1 (Sep. 2021), 9:1 – 9:16. DOI: 10.5753/jserd.2021.1802
Michele Tufano, Fabio Palomba, Gabriele Bavota, Massimiliano Di Penta, Rocco Oliveto, Andrea De Lucia, and Denys Poshyvanyk. 2016. An empirical investigation into the nature of test smells. In Proceedings of the 31st IEEE/ACM International Conference on Automated Software Engineering (Singapore, Singapore) (ASE ’16). Association for Computing Machinery, New York, NY, USA, 4–15. DOI: 10.1145/2970276.2970340
Tássio Virgínio, Luana Martins, Railana Santana, Adriana Cruz, Larissa Rocha, Heitor Costa, and Ivan Machado. 2021. On the test smells detection: an empirical study on the JNose Test accuracy. Journal of Software Engineering Research and Development 9, 1 (Sep. 2021), 8:1 – 8:14. DOI: 10.5753/jserd.2021.1893
