DiffMutAnalyze: A tool to assist the analysis of equivalent mutants in mutation test teaching

• Juliana Botelho Federal University of Lavras http://orcid.org/0000-0002-4675-2043
• Maurício Souza Federal University of Lavras
• Vinicius H. S. Durelli Federal University of São João del Rei
• Rafael S. Durelli Federal University of Lavras

DOI: https://doi.org/10.5753/vem.2020.14529

Abstract

Mutation testing is a technique that aims to evaluate a test cases and, consequently, to help improve their quality. Mutation testing is a promising technique to assist students in developing skills in software testing. Thus, this article presents the use of the DiffMutAnalyze tool in the academic field. It provides an environment that: (i) promotes the generation of mutants, (ii) the execution of test cases in the mutants generated, (iii) the identification of the detected mutants, and (iv) the availability of the analysis of possible equivalent mutants. DiffMutAnalyze was evaluated with 11 Graduate Students in Computer Science, to support the teaching of mutant testing and to show the practical application of mutation and analysis of mutants. A study was performed with the students to verify the cost of the equivalent mutant analysis, being compared in two ways: (i) by using DiffMutAnalyze; and (ii) through manual analysis, without using DiffMutAnalyze. The results showed a significant time reduction in comparison to the manual analysis.

Keywords: Software testing, Mutation Test, Mutation analysis, DiffMutAnalyze, Software testing for education

References

Aniche, M., Hermans, F., and van Deursen, A. (2019). Pragmatic software testing education. In 50th Proceedings of the ACM Technical Symposium on Computer Science Education (SIGCSE), pages 414–420.

Botelho, J., Pereira, C. H., Durelli, V. H. S., and Durelli, R. S. (2018). Diffmutanalyze: Uma abordagem para auxiliar a identificação de mutantes equivalentes. In VIWorkshop on Software Visualization, Evolution and Maintenance (VEM).

Cerf, V. G. (2014). Responsible programming. Communications of the ACM, 57(7):7–7.

Delamaro, M., Maldonado, J., and Jino, M. (2016). Introdução ao teste de software. Elsevier.

Grün, B. J., Schuler, D., and Zeller, A. (2009). The impact of equivalent mutants. In 2nd International Conference on Software Testing, Verification and Validation Workshops (ICSTW), pages 192–199.

Just, R. (2014). The major mutation framework: Efficient and scalable mutation analysis for Java. In Proceedings of the International Symposium on Software Testing and Analysis (ISSTA), pages 433–436.

Papadakis, M., Kintis, M., Zhang, J., Jia, Y., Traon, Y. L., and Harman, M. (2019). Mutation testing advances: an analysis and survey. In Advances in Computers, pages 275–378.

Radhakrishnan, P., Kanmani, S., and Nandhini, M. (2015). Xsoft: A generic software teaching and learning model. Computer Applications in Engineering Education, 23(3):432–442.

Spacco, J. and Pugh, W. (2006). Helping students appreciate test-driven development (tdd). In 21st Symposium on Object-oriented Programming Systems, Languages, and Applications (ACM SIGPLAN), pages 907–913.

Zheng, W., Bai, Y., and Che, H. (2018). A computer-assisted instructional method based on machine learning in software testing class. Computer Applications in Engineering Education, 26(5):1150–1158.