METEOR: A Tool for Monitoring Behavior Preservation in Test Code Refactorings
Resumo
The success of test refactoring, particularly in the removal of test smells, relies on a post-refactoring evaluation to ensure that the refactored test maintains its behavior, meaning its defect detection capability. Test smells are indicative of issues within test code. While various approaches in the literature propose strategies for evaluating test behavior, they lack tool support for practical industry application. Therefore, this study introduces MeteoR, a tool model that employs a mutation testing approach to assess refactored test behavior. MeteoR is implemented as a plugin for the Eclipse IDE and is integrated with PITclipse, streamlining the collection and analysis of mutation test data for effective evaluation of test refactoring. This paper provides a detailed description of the MeteoR implementation, discusses encountered challenges and findings, and includes illustrative examples demonstrating its efficacy through a preliminary evaluation, which shows promising initial results. The video presentation of MeteoR is available at https://doi.org/10.6084/m9.figshare.25954045.
Palavras-chave:
software engineering, test code refactoring, test smells, test behavior, mutation testing
Referências
Stevão A. Andrade, Claudinei Brito, Misael Júnior, Ana Claudia Marciel, Gabriel Abdalla, and Márcio E. Delamaro. 2019. Analyzing the effectiveness of One-Op Mutation against the minimal set of mutants. In Proceedings of the IV Brazilian Symposium on Systematic and Automated Software Testing (Salvador, Brazil) (SAST ’19). Association for Computing Machinery, New York, NY, USA, 22–31. DOI: 10.1145/3356317.3356321
M. Aniche. 2022. Effective Software Testing: A developer’s guide. Manning, NY. [link]
Brent van Bladel and Serge Demeyer. 2018. Test Behaviour Detection as a Test Refactoring Safety. In Proceedings of the 2nd International Workshop on Refactoring (Montpellier, France) (IWoR 2018). Association for Computing Machinery, New York, NY, USA, 22–25. DOI: 10.1145/3242163.3242168
Henry Coles. [n. d.]. How to get killing tests against each mutant or a complete killing matrix with PIT? [link]
Henry Coles, Thomas Laurent, Christopher Henard, Mike Papadakis, and Anthony Ventresque. 2016. PIT: a practical mutation testing tool for Java (demo). In Proceedings of the 25th International Symposium on Software Testing and Analysis (Saarbrücken, Germany) (ISSTA 2016). Association for Computing Machinery, New York, NY, USA, 449–452. DOI: 10.1145/2931037.2948707
M. Fowler. 2019. Refactoring: Improving the Design of Existing Code. Addison-Wesley, USA.
Eduardo Martins Guerra and Clovis Torres Fernandes. 2007. Refactoring Test Code Safely. In International Conference on Software Engineering Advances (ICSEA 2007). 44–44. DOI: 10.1109/ICSEA.2007.57
Rogério Marinke, Eduardo Martins Guerra, Fábio Fagundes Silveira, Rafael Monico Azevedo, Wagner Nascimento, Rodrigo Simões de Almeida, Bruno Rodrigues Demboscki, and Tiago Silva da Silva. 2019. Towards an Extensible Architecture for Refactoring Test Code. In Computational Science and Its Applications – ICCSA 2019, Sanjay Misra, Osvaldo Gervasi, Beniamino Murgante, Elena Stankova, Vladimir Korkhov, Carmelo Torre, Ana Maria A.C. Rocha, David Taniar, Bernady O. Apduhan, and Eufemia Tarantino (Eds.). Springer International Publishing, Cham, 456–471.
Luana Martins, Taher Ghaleb, Heitor Costa, and Ivan Machado. 2024. A comprehensive catalog of refactoring strategies to handle test smells in Java-based systems. Software Quality Journal (03 2024), 1–39. DOI: 10.1007/s11219-024-09663-7
Gerard Meszaros. 2007. xUnit Test Patterns: Refactoring Test Code. Pearson Education.
Ricardo Monteiro, Vinicius Humberto Serapilha Durelli, Marcelo Eler, and Andre Endo. 2022. An Empirical Analysis of Two Mutation Testing Tools for Java. In Proceedings of the 7th Brazilian Symposium on Systematic and Automated Software Testing (Uberlandia, Brazil) (SAST ’22). Association for Computing Machinery, New York, NY, USA, 49–58. DOI: 10.1145/3559744.3559751
A. Jefferson Offutt and Roland H. Untch. 2001. Mutation 2000: Uniting the Orthogonal. Springer US, Boston, MA. 34–44 pages. DOI: 10.1007/978-1-4757-5939-6_7
Ali Parsai, Alessandro Murgia, Quinten David Soetens, and Serge Demeyer. 2015. Mutation Testing as a Safety Net for Test Code Refactoring. In ScientificWorkshop Proceedings of the XP2015 (Helsinki, Finland) (XP ’15 workshops). Association for Computing Machinery, New York, NY, USA, Article 8, 7 pages. DOI: 10.1145/2764979.2764987
Adriano Pizzini. 2022. Behavior-based test smells refactoring : Toward an automatic approach to refactoring Eager Test and Lazy Test smells. In 2022 IEEE/ACM 44th International Conference on Software Engineering: Companion Proceedings (ICSE-Companion). 261–263. DOI: 10.1145/3510454.3517059
Adriano Pizzini, Sheila Reinehr, and Andreia Malucelli. 2023. Sentinel: A process for automatic removing of Test Smells. In Proceedings of the XXII Brazilian Symposium on Software Quality (, Brasília, Brazil,) (SBQS ’23). Association for Computing Machinery, New York, NY, USA, 80–89. DOI: 10.1145/3629479.3630019
Tiago Samuel Rodrigues Teixeira, Fábio Fagundes Silveira, and Eduardo Martins Guerra. 2023. Moving towards a Mutant-Based Testing Tool for Verifying Behavior Maintenance in Test Code Refactorings. Computers 12, 11 (2023). DOI: 10.3390/computers12110230
Brent van Bladel and Serge Demeyer. 2017. Test Refactoring: a Research Agenda. In Proceedings SATToSE.
Arie Van Deursen, Leon Moonen, Alex Van Den Bergh, and Gerard Kok. 2001. Refactoring test code. In Proc. Int’l Conf. eXtreme Programming and Flexible Processes in Software Engineering (XP), M. Marchesi (Ed.).
Jifeng Xuan, Benoit Cornu, Matias Martinez, Benoit Baudry, Lionel Seinturier, and Martin Monperrus. 2016. B-Refactoring: Automatic test code refactoring to improve dynamic analysis. Information and Software Technology 76 (2016), 65–80. DOI: 10.1016/j.infsof.2016.04.016
M. Aniche. 2022. Effective Software Testing: A developer’s guide. Manning, NY. [link]
Brent van Bladel and Serge Demeyer. 2018. Test Behaviour Detection as a Test Refactoring Safety. In Proceedings of the 2nd International Workshop on Refactoring (Montpellier, France) (IWoR 2018). Association for Computing Machinery, New York, NY, USA, 22–25. DOI: 10.1145/3242163.3242168
Henry Coles. [n. d.]. How to get killing tests against each mutant or a complete killing matrix with PIT? [link]
Henry Coles, Thomas Laurent, Christopher Henard, Mike Papadakis, and Anthony Ventresque. 2016. PIT: a practical mutation testing tool for Java (demo). In Proceedings of the 25th International Symposium on Software Testing and Analysis (Saarbrücken, Germany) (ISSTA 2016). Association for Computing Machinery, New York, NY, USA, 449–452. DOI: 10.1145/2931037.2948707
M. Fowler. 2019. Refactoring: Improving the Design of Existing Code. Addison-Wesley, USA.
Eduardo Martins Guerra and Clovis Torres Fernandes. 2007. Refactoring Test Code Safely. In International Conference on Software Engineering Advances (ICSEA 2007). 44–44. DOI: 10.1109/ICSEA.2007.57
Rogério Marinke, Eduardo Martins Guerra, Fábio Fagundes Silveira, Rafael Monico Azevedo, Wagner Nascimento, Rodrigo Simões de Almeida, Bruno Rodrigues Demboscki, and Tiago Silva da Silva. 2019. Towards an Extensible Architecture for Refactoring Test Code. In Computational Science and Its Applications – ICCSA 2019, Sanjay Misra, Osvaldo Gervasi, Beniamino Murgante, Elena Stankova, Vladimir Korkhov, Carmelo Torre, Ana Maria A.C. Rocha, David Taniar, Bernady O. Apduhan, and Eufemia Tarantino (Eds.). Springer International Publishing, Cham, 456–471.
Luana Martins, Taher Ghaleb, Heitor Costa, and Ivan Machado. 2024. A comprehensive catalog of refactoring strategies to handle test smells in Java-based systems. Software Quality Journal (03 2024), 1–39. DOI: 10.1007/s11219-024-09663-7
Gerard Meszaros. 2007. xUnit Test Patterns: Refactoring Test Code. Pearson Education.
Ricardo Monteiro, Vinicius Humberto Serapilha Durelli, Marcelo Eler, and Andre Endo. 2022. An Empirical Analysis of Two Mutation Testing Tools for Java. In Proceedings of the 7th Brazilian Symposium on Systematic and Automated Software Testing (Uberlandia, Brazil) (SAST ’22). Association for Computing Machinery, New York, NY, USA, 49–58. DOI: 10.1145/3559744.3559751
A. Jefferson Offutt and Roland H. Untch. 2001. Mutation 2000: Uniting the Orthogonal. Springer US, Boston, MA. 34–44 pages. DOI: 10.1007/978-1-4757-5939-6_7
Ali Parsai, Alessandro Murgia, Quinten David Soetens, and Serge Demeyer. 2015. Mutation Testing as a Safety Net for Test Code Refactoring. In ScientificWorkshop Proceedings of the XP2015 (Helsinki, Finland) (XP ’15 workshops). Association for Computing Machinery, New York, NY, USA, Article 8, 7 pages. DOI: 10.1145/2764979.2764987
Adriano Pizzini. 2022. Behavior-based test smells refactoring : Toward an automatic approach to refactoring Eager Test and Lazy Test smells. In 2022 IEEE/ACM 44th International Conference on Software Engineering: Companion Proceedings (ICSE-Companion). 261–263. DOI: 10.1145/3510454.3517059
Adriano Pizzini, Sheila Reinehr, and Andreia Malucelli. 2023. Sentinel: A process for automatic removing of Test Smells. In Proceedings of the XXII Brazilian Symposium on Software Quality (, Brasília, Brazil,) (SBQS ’23). Association for Computing Machinery, New York, NY, USA, 80–89. DOI: 10.1145/3629479.3630019
Tiago Samuel Rodrigues Teixeira, Fábio Fagundes Silveira, and Eduardo Martins Guerra. 2023. Moving towards a Mutant-Based Testing Tool for Verifying Behavior Maintenance in Test Code Refactorings. Computers 12, 11 (2023). DOI: 10.3390/computers12110230
Brent van Bladel and Serge Demeyer. 2017. Test Refactoring: a Research Agenda. In Proceedings SATToSE.
Arie Van Deursen, Leon Moonen, Alex Van Den Bergh, and Gerard Kok. 2001. Refactoring test code. In Proc. Int’l Conf. eXtreme Programming and Flexible Processes in Software Engineering (XP), M. Marchesi (Ed.).
Jifeng Xuan, Benoit Cornu, Matias Martinez, Benoit Baudry, Lionel Seinturier, and Martin Monperrus. 2016. B-Refactoring: Automatic test code refactoring to improve dynamic analysis. Information and Software Technology 76 (2016), 65–80. DOI: 10.1016/j.infsof.2016.04.016
Publicado
30/09/2024
Como Citar
TEIXEIRA, Tiago Samuel Rodrigues; SILVEIRA, Fábio Fagundes; GUERRA, Eduardo Martins.
METEOR: A Tool for Monitoring Behavior Preservation in Test Code Refactorings. In: SIMPÓSIO BRASILEIRO DE ENGENHARIA DE SOFTWARE (SBES), 38. , 2024, Curitiba/PR.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2024
.
p. 755-761.
DOI: https://doi.org/10.5753/sbes.2024.3580.
