DNose: Dart Test Smell Detector
Abstract
Test smells, anti-patterns in test code, have emerged as key indicators of test quality, attracting significant attention in software engineering research. Automated detection of test smells is crucial for maintaining effective and readable test suites, yet existing tools often lack comprehensive language support. At the same time, sentiment analysis is gaining prominence as a technique for understanding developers’ emotional states during coding activities, providing deeper insights into software quality. Motivated by these research gaps, we present DNose, the first dedicated tool for detecting test smells in Dart-based projects. DNose identifies 14 types of test smells, correlates their introduction with developer sentiment derived from commit messages, and provides integrated insights into test code quality and developer emotions. Our approach contributes to enhancing test maintainability while illuminating emotional aspects of software development practices. Link to the video: https://youtu.be/fCYdDZA8Hkg.
Keywords:
Quality of Tests, Test Suite Evolution, Test Smells, Code Coverage
References
Gabriele Bavota, Abdallah Qusef, Rocco Oliveto, Andrea De Lucia, and Dave Binkley. 2015. Are test smells really harmful? An empirical study. Empirical Software Engineering 20, 4 (2015), 1052–1094.
Gabriele Bavota, Abdallah Qusef, Rocco Oliveto, Andrea De Lucia, and Dave Binkley. 2015. Are test smells really harmful? an empirical study. Empirical Software Engineering 20 (2015), 1052–1094.
CISQ. 2019. The Cost of Poor Quality Software in the US 2018 Report. [link] Last access: May 2025.
CISQ. 2020. The Cost of Poor Quality Software in the US 2018 Report. [link] Last access: May 2025.
Jonas De Bleser, Dario Di Nucci, and Coen De Roover. 2019. SoCRATES: Scala radar for test smells. In Proceedings of the Tenth ACM SIGPLAN Symposium on Scala (London, United Kingdom) (Scala ’19). Association for Computing Machinery, New York, NY, USA, 22–26. DOI: 10.1145/3337932.3338815
Arie Deursen, Leon M.F. Moonen, A. Bergh, and Gerard Kok. 2001. Refactoring Test Code. In Refactoring Test Code. CWI (Centre for Mathematics and Computer Science), Amsterdam, The Netherlands, The Netherlands.
Rashmi Dhakad and Luigi Benedicenti. 2023. Analyzing Emotional Contagion in Commit Messages of Open-Source Software Repositories. (05 2023), 113–133.
José Pereira dos Reis, Fernando Brito e Abreu, Glauco de Figueiredo Carneiro, and Craig Anslow. 2022. Code Smells Detection and Visualization: A Systematic Literature Review. Archives of Computational Methods in Engineering 29, 1 (2022), 47–94. DOI: 10.1007/s11831-021-09566-x
Daniel Fernandes, Ivan Machado, and Rita Maciel. 2022. TEMPY: Test Smell Detector for Python. In Proceedings of the XXXVI Brazilian Symposium on Software Engineering (Virtual Event, Brazil) (SBES ’22). Association for Computing Machinery, New York, NY, USA, 214–219. DOI: 10.1145/3555228.3555280
Vahid Garousi and Barış Küçük. 2018. Smells in software test code: A survey of knowledge in industry and academia. Journal of Systems and Software 138 (2018), 52 – 81.
O Globo. 2024. Raio-X da telefonia no Brasil. [link]. Accessed: 2024-07-15.
Giovanni Grano, Fabio Palomba, Dario Di Nucci, Andrea De Lucia, and Harald C Gall. 2019. Scented since the beginning: On the diffuseness of test smells in automatically generated test code. Journal of Systems and Software 156 (2019), 312–327.
Dayne Guerra Calle, Julien Delplanque, and Stéphane Ducasse. 2019. Exposing Test Analysis Results with DrTests. In International Workshop on Smalltalk Technologies. HAL, Cologne, Germany, 1–5. [link]
Emitza Guzman, David Azócar, and Yang Li. 2014. Sentiment analysis of commit comments in GitHub: an empirical study. In Proceedings of the 11th Working Conference on Mining Software Repositories (Hyderabad, India) (MSR 2014). Association for Computing Machinery, New York, NY, USA, 352–355. DOI: 10.1145/2597073.2597118
Nildo Silva Junior, Luana Almeida Martins, Larissa Rocha, Heitor A. X. Costa, and Ivan Machado. 2021. How are test smells treated in the wild? A tale of two empirical studies. J. Softw. Eng. Res. Dev. 9 (2021), 9:1–9:16. DOI: 10.5753/JSERD.2021.1802
Luana Almeida Martins, Heitor A. X. Costa, and Ivan Machado. 2024. On the diffusion of test smells and their relationship with test code quality of Java projects. J. Softw. Evol. Process. 36, 4 (2024). DOI: 10.1002/smr.2532
Luana Almeida Martins, Valeria Pontillo, Heitor A. X. Costa, Filomena Ferrucci, Fabio Palomba, and Ivan Machado. 2025. Test code refactoring unveiled: where and how does it affect test code quality and effectiveness? Empir. Softw. Eng. 30, 1 (2025), 27.
Gerard Meszaros, Shaun M. Smith, and Jennitta Andrea. 2003. The Test Automation Manifesto. In Extreme Programming and Agile Methods - XP/Agile Universe 2003, Frank Maurer and Don Wells (Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg.
F. Å. Nielsen. 2011. AFINN. [link]
Jhonatan Oliveira, Luigi Mateus, Tássio Virgínio, and Larissa Rocha. 2024. SNUTS.js: Sniffing Nasty Unit Test Smells in Javascript. In Anais do XXXVIII Simpósio Brasileiro de Engenharia de Software (Curitiba/PR). SBC, Porto Alegre, RS, Brasil, 720–726. DOI: 10.5753/sbes.2024.3563
Fabio Palomba, Andy Zaidman, and Andrea De Lucia. 2018. Automatic Test Smell Detection Using Information Retrieval Techniques. In IEEE International Conference on Software Maintenance and Evolution (ICSME). IEEE, Madrid, Spain, 311–322.
Annibale Panichella, Sebastiano Panichella, Gordon Fraser, Anand Ashok Sawant, and Vincent J. Hellendoorn. 2022. Test smells 20 years later: detectability, validity, and reliability. Empirical Software Engineering 27, 7 (2022), 170. DOI: 10.1007/s10664-022-10207-5
Anthony Peruma, Khalid Almalki, Christian D. Newman, Mohamed Wiem Mkaouer, Ali Ouni, and Fabio Palomba. 2020. tsDetect: an open source test smells detection tool. In Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering (Virtual Event, USA) (ESEC/FSE 2020). Association for Computing Machinery, New York, NY, USA, 1650–1654. DOI: 10.1145/3368089.3417921
Valeria Pontillo, Luana Almeida Martins, Ivan do Carmo Machado, Fabio Palomba, and Filomena Ferrucci. 2025. An empirical investigation into the capabilities of anomaly detection approaches for test smell detection. J. Syst. Softw. 222 (2025), 112320. DOI: 10.1016/J.JSS.2024.112320
Davide Quaglione, Nicola Matteucci, Donatella Furia, Alessandro Marra, and Cesare Pozzi. 2020. Are mobile and fixed broadband substitutes or complements? New empirical evidence from Italy and implications for the digital divide policies. Socio-Economic Planning Sciences 71 (2020), 100823. DOI: 10.1016/j.seps.2020.100823
Railana Santana, Luana Martins, Larissa Rocha, Tássio Virgínio, Adriana Cruz, Heitor Costa, and Ivan Machado. 2020. RAIDE: a tool for Assertion Roulette and Duplicate Assert identification and refactoring. In Proceedings of the XXXIV Brazilian Symposium on Software Engineering (Natal, Brazil) (SBES ’20). Association for Computing Machinery, New York, NY, USA, 374–379. DOI: 10.1145/3422392.3422510
Railana Santana, Luana Martins, Tássio Virgínio, Larissa Soares, Heitor Costa, and Ivan Machado. 2022. Refactoring Assertion Roulette and Duplicate Assert test smells: a controlled experiment. In Anais do XXV Congresso Ibero-Americano em Engenharia de Software (Córdoba). SBC, Porto Alegre, RS, Brasil, 263–277. DOI: 10.5753/cibse.2022.20977
Shahriar Shirvani Moghaddam. 2024. The Past, Present, and Future of the Internet: A Statistical, Technical, and Functional Comparison of Wired/Wireless Fixed/Mobile Internet. Electronics 13, 10 (2024). DOI: 10.3390/electronics13101986
Vinayak Sinha, Alina Lazar, and Bonita Sharif. 2016. Analyzing developer sentiment in commit logs. In Proceedings of the 13th International Conference on Mining Software Repositories (Austin, Texas) (MSR ’16). Association for Computing Machinery, New York, NY, USA, 520–523. DOI: 10.1145/2901739.2903501
Elvys Soares, Manoel Aranda III, Davi Romão, and Márcio Ribeiro. 2023. The Open Catalog of Test Smells. Available at [link].
Davide Spadini, Fabio Palomba, Andy Zaidman, Magiel Bruntink, and Alberto Bacchelli. 2018. On the Relation of Test Smells to Software Code Quality. In International Conference on Software Maintenance and Evolution (ICSME). IEEE, Madrid, Spain, 1–12.
Statista Research Department. 2024. Most used frameworks among developers worldwide as of 2024. [link]. Accessed: 2025-05-06.
Tricentis. 2018. Software Fail Watch: 5th Edition. [link] Last access: May 2025.
Tássio Virgínio, Luana Martins, Larissa Rocha, Railana Santana, Adriana Cruz, Heitor Costa, and Ivan Machado. 2020. JNose: Java Test Smell Detector. In Proceedings of the XXXIV Brazilian Symposium on Software Engineering (Natal, Brazil) (SBES ’20). Association for Computing Machinery, New York, NY, USA, 564–569. DOI: 10.1145/3422392.3422499
Tássio Virgínio, Luana Almeida Martins, Railana Santana, Adriana Cruz, Larissa Rocha, Heitor A. X. Costa, and Ivan Machado. 2021. On the test smells detection: an empirical study on the JNose Test accuracy. J. Softw. Eng. Res. Dev. 9 (2021), 8:1–8:14. DOI: 10.5753/JSERD.2021.1893
Tássio Virgínio, Luana Martins, Larissa Rocha Soares, Santana Railana, Heitor Costa, and Ivan Machado. 2020. An empirical study of automatically-generated tests from the perspective of test smells. In Proceedings of the XXXIV Brazilian Symposium on Software Engineering (Virtual Conference) (SBES 2020). ACM, New York, NY, USA, 5 pages.
Tongjie Wang, Yaroslav Golubev, Oleg Smirnov, Jiawei Li, Timofey Bryksin, and Iftekhar Ahmed. 2021. PyNose: A Test Smell Detector For Python. In 2021 36th IEEE/ACM International Conference on Automated Software Engineering (ASE). 593–605. DOI: 10.1109/ASE51524.2021.9678615
Pravin Singh Yadav, Rajwant Singh Rao, Alok Mishra, and Manjari Gupta. 2024. Machine Learning-Based Methods for Code Smell Detection: A Survey. Applied Sciences 14, 14 (2024). DOI: 10.3390/app14146149
Vahid Garousi Yusifoğlu, Yasaman Amannejad, and Aysu Betin Can. 2015. Software test-code engineering: A systematic mapping. Information and Software Technology 58 (2015), 123 – 147.
Fengji Zhang, Zexian Zhang, Jacky Wai Keung, Xiangru Tang, Zhen Yang, Xiao Yu, andWenhua Hu. 2024. Data preparation for Deep Learning based Code Smell Detection: A systematic literature review. Journal of Systems and Software 216 (2024), 112131. DOI: 10.1016/j.jss.2024.112131
Gabriele Bavota, Abdallah Qusef, Rocco Oliveto, Andrea De Lucia, and Dave Binkley. 2015. Are test smells really harmful? an empirical study. Empirical Software Engineering 20 (2015), 1052–1094.
CISQ. 2019. The Cost of Poor Quality Software in the US 2018 Report. [link] Last access: May 2025.
CISQ. 2020. The Cost of Poor Quality Software in the US 2018 Report. [link] Last access: May 2025.
Jonas De Bleser, Dario Di Nucci, and Coen De Roover. 2019. SoCRATES: Scala radar for test smells. In Proceedings of the Tenth ACM SIGPLAN Symposium on Scala (London, United Kingdom) (Scala ’19). Association for Computing Machinery, New York, NY, USA, 22–26. DOI: 10.1145/3337932.3338815
Arie Deursen, Leon M.F. Moonen, A. Bergh, and Gerard Kok. 2001. Refactoring Test Code. In Refactoring Test Code. CWI (Centre for Mathematics and Computer Science), Amsterdam, The Netherlands, The Netherlands.
Rashmi Dhakad and Luigi Benedicenti. 2023. Analyzing Emotional Contagion in Commit Messages of Open-Source Software Repositories. (05 2023), 113–133.
José Pereira dos Reis, Fernando Brito e Abreu, Glauco de Figueiredo Carneiro, and Craig Anslow. 2022. Code Smells Detection and Visualization: A Systematic Literature Review. Archives of Computational Methods in Engineering 29, 1 (2022), 47–94. DOI: 10.1007/s11831-021-09566-x
Daniel Fernandes, Ivan Machado, and Rita Maciel. 2022. TEMPY: Test Smell Detector for Python. In Proceedings of the XXXVI Brazilian Symposium on Software Engineering (Virtual Event, Brazil) (SBES ’22). Association for Computing Machinery, New York, NY, USA, 214–219. DOI: 10.1145/3555228.3555280
Vahid Garousi and Barış Küçük. 2018. Smells in software test code: A survey of knowledge in industry and academia. Journal of Systems and Software 138 (2018), 52 – 81.
O Globo. 2024. Raio-X da telefonia no Brasil. [link]. Accessed: 2024-07-15.
Giovanni Grano, Fabio Palomba, Dario Di Nucci, Andrea De Lucia, and Harald C Gall. 2019. Scented since the beginning: On the diffuseness of test smells in automatically generated test code. Journal of Systems and Software 156 (2019), 312–327.
Dayne Guerra Calle, Julien Delplanque, and Stéphane Ducasse. 2019. Exposing Test Analysis Results with DrTests. In International Workshop on Smalltalk Technologies. HAL, Cologne, Germany, 1–5. [link]
Emitza Guzman, David Azócar, and Yang Li. 2014. Sentiment analysis of commit comments in GitHub: an empirical study. In Proceedings of the 11th Working Conference on Mining Software Repositories (Hyderabad, India) (MSR 2014). Association for Computing Machinery, New York, NY, USA, 352–355. DOI: 10.1145/2597073.2597118
Nildo Silva Junior, Luana Almeida Martins, Larissa Rocha, Heitor A. X. Costa, and Ivan Machado. 2021. How are test smells treated in the wild? A tale of two empirical studies. J. Softw. Eng. Res. Dev. 9 (2021), 9:1–9:16. DOI: 10.5753/JSERD.2021.1802
Luana Almeida Martins, Heitor A. X. Costa, and Ivan Machado. 2024. On the diffusion of test smells and their relationship with test code quality of Java projects. J. Softw. Evol. Process. 36, 4 (2024). DOI: 10.1002/smr.2532
Luana Almeida Martins, Valeria Pontillo, Heitor A. X. Costa, Filomena Ferrucci, Fabio Palomba, and Ivan Machado. 2025. Test code refactoring unveiled: where and how does it affect test code quality and effectiveness? Empir. Softw. Eng. 30, 1 (2025), 27.
Gerard Meszaros, Shaun M. Smith, and Jennitta Andrea. 2003. The Test Automation Manifesto. In Extreme Programming and Agile Methods - XP/Agile Universe 2003, Frank Maurer and Don Wells (Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg.
F. Å. Nielsen. 2011. AFINN. [link]
Jhonatan Oliveira, Luigi Mateus, Tássio Virgínio, and Larissa Rocha. 2024. SNUTS.js: Sniffing Nasty Unit Test Smells in Javascript. In Anais do XXXVIII Simpósio Brasileiro de Engenharia de Software (Curitiba/PR). SBC, Porto Alegre, RS, Brasil, 720–726. DOI: 10.5753/sbes.2024.3563
Fabio Palomba, Andy Zaidman, and Andrea De Lucia. 2018. Automatic Test Smell Detection Using Information Retrieval Techniques. In IEEE International Conference on Software Maintenance and Evolution (ICSME). IEEE, Madrid, Spain, 311–322.
Annibale Panichella, Sebastiano Panichella, Gordon Fraser, Anand Ashok Sawant, and Vincent J. Hellendoorn. 2022. Test smells 20 years later: detectability, validity, and reliability. Empirical Software Engineering 27, 7 (2022), 170. DOI: 10.1007/s10664-022-10207-5
Anthony Peruma, Khalid Almalki, Christian D. Newman, Mohamed Wiem Mkaouer, Ali Ouni, and Fabio Palomba. 2020. tsDetect: an open source test smells detection tool. In Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering (Virtual Event, USA) (ESEC/FSE 2020). Association for Computing Machinery, New York, NY, USA, 1650–1654. DOI: 10.1145/3368089.3417921
Valeria Pontillo, Luana Almeida Martins, Ivan do Carmo Machado, Fabio Palomba, and Filomena Ferrucci. 2025. An empirical investigation into the capabilities of anomaly detection approaches for test smell detection. J. Syst. Softw. 222 (2025), 112320. DOI: 10.1016/J.JSS.2024.112320
Davide Quaglione, Nicola Matteucci, Donatella Furia, Alessandro Marra, and Cesare Pozzi. 2020. Are mobile and fixed broadband substitutes or complements? New empirical evidence from Italy and implications for the digital divide policies. Socio-Economic Planning Sciences 71 (2020), 100823. DOI: 10.1016/j.seps.2020.100823
Railana Santana, Luana Martins, Larissa Rocha, Tássio Virgínio, Adriana Cruz, Heitor Costa, and Ivan Machado. 2020. RAIDE: a tool for Assertion Roulette and Duplicate Assert identification and refactoring. In Proceedings of the XXXIV Brazilian Symposium on Software Engineering (Natal, Brazil) (SBES ’20). Association for Computing Machinery, New York, NY, USA, 374–379. DOI: 10.1145/3422392.3422510
Railana Santana, Luana Martins, Tássio Virgínio, Larissa Soares, Heitor Costa, and Ivan Machado. 2022. Refactoring Assertion Roulette and Duplicate Assert test smells: a controlled experiment. In Anais do XXV Congresso Ibero-Americano em Engenharia de Software (Córdoba). SBC, Porto Alegre, RS, Brasil, 263–277. DOI: 10.5753/cibse.2022.20977
Shahriar Shirvani Moghaddam. 2024. The Past, Present, and Future of the Internet: A Statistical, Technical, and Functional Comparison of Wired/Wireless Fixed/Mobile Internet. Electronics 13, 10 (2024). DOI: 10.3390/electronics13101986
Vinayak Sinha, Alina Lazar, and Bonita Sharif. 2016. Analyzing developer sentiment in commit logs. In Proceedings of the 13th International Conference on Mining Software Repositories (Austin, Texas) (MSR ’16). Association for Computing Machinery, New York, NY, USA, 520–523. DOI: 10.1145/2901739.2903501
Elvys Soares, Manoel Aranda III, Davi Romão, and Márcio Ribeiro. 2023. The Open Catalog of Test Smells. Available at [link].
Davide Spadini, Fabio Palomba, Andy Zaidman, Magiel Bruntink, and Alberto Bacchelli. 2018. On the Relation of Test Smells to Software Code Quality. In International Conference on Software Maintenance and Evolution (ICSME). IEEE, Madrid, Spain, 1–12.
Statista Research Department. 2024. Most used frameworks among developers worldwide as of 2024. [link]. Accessed: 2025-05-06.
Tricentis. 2018. Software Fail Watch: 5th Edition. [link] Last access: May 2025.
Tássio Virgínio, Luana Martins, Larissa Rocha, Railana Santana, Adriana Cruz, Heitor Costa, and Ivan Machado. 2020. JNose: Java Test Smell Detector. In Proceedings of the XXXIV Brazilian Symposium on Software Engineering (Natal, Brazil) (SBES ’20). Association for Computing Machinery, New York, NY, USA, 564–569. DOI: 10.1145/3422392.3422499
Tássio Virgínio, Luana Almeida Martins, Railana Santana, Adriana Cruz, Larissa Rocha, Heitor A. X. Costa, and Ivan Machado. 2021. On the test smells detection: an empirical study on the JNose Test accuracy. J. Softw. Eng. Res. Dev. 9 (2021), 8:1–8:14. DOI: 10.5753/JSERD.2021.1893
Tássio Virgínio, Luana Martins, Larissa Rocha Soares, Santana Railana, Heitor Costa, and Ivan Machado. 2020. An empirical study of automatically-generated tests from the perspective of test smells. In Proceedings of the XXXIV Brazilian Symposium on Software Engineering (Virtual Conference) (SBES 2020). ACM, New York, NY, USA, 5 pages.
Tongjie Wang, Yaroslav Golubev, Oleg Smirnov, Jiawei Li, Timofey Bryksin, and Iftekhar Ahmed. 2021. PyNose: A Test Smell Detector For Python. In 2021 36th IEEE/ACM International Conference on Automated Software Engineering (ASE). 593–605. DOI: 10.1109/ASE51524.2021.9678615
Pravin Singh Yadav, Rajwant Singh Rao, Alok Mishra, and Manjari Gupta. 2024. Machine Learning-Based Methods for Code Smell Detection: A Survey. Applied Sciences 14, 14 (2024). DOI: 10.3390/app14146149
Vahid Garousi Yusifoğlu, Yasaman Amannejad, and Aysu Betin Can. 2015. Software test-code engineering: A systematic mapping. Information and Software Technology 58 (2015), 123 – 147.
Fengji Zhang, Zexian Zhang, Jacky Wai Keung, Xiangru Tang, Zhen Yang, Xiao Yu, andWenhua Hu. 2024. Data preparation for Deep Learning based Code Smell Detection: A systematic literature review. Journal of Systems and Software 216 (2024), 112131. DOI: 10.1016/j.jss.2024.112131
Published
2025-09-22
How to Cite
VIRGÍNIO, Tássio; RIBEIRO, Marcio; MACHADO, Ivan.
DNose: Dart Test Smell Detector. In: BRAZILIAN SYMPOSIUM ON SOFTWARE ENGINEERING (SBES), 39. , 2025, Recife/PE.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2025
.
p. 976-982.
ISSN 2833-0633.
DOI: https://doi.org/10.5753/sbes.2025.11564.
