Search-based Test Data Generation for Mutation Testing: a tool for Python programs
Resumo
Test data generation for mutation testing consists of identifying a set of inputs that maximizes the number of mutants killed. Mutation Testing is an excellent test criterion for detecting faults and measuring the effectiveness of test data sets. However, it is not widely used in practice due to the cost and complexity to perform some activities as generating test data. Although test suites can be produced and selected manually by a tester this practice is susceptible to errors and tools are needed to facilitate it. Several tools have been developed to automate mutation testing, but, only a few address the test data generation. The present paper proposes an automated test data generation tool based on weak mutation for Python programming language using the Hill Climbing algorithm. For evaluation, we performed an experiment concerning the effectiveness and cost computational of the tool in a database composed of 348 mutants and we compare it with random generation. Overall, the experiment achieved an average mutation score of 86% for our proposed tool and random testing 64% on average.
Palavras-chave:
mutation testing, test data generation, hill climbing algorithm
Referências
Basili, V. and Weiss, D. (1986). A methodology for collecting valid software engineering data. IEEE Transactions on Software Engineering, 10(6):728–738.
Bottaci, L. (2001). A genetic algorithm fitness function for mutation testing. In Proceedings of the First International Workshop on Software Engineering using Metaheuristic Innovative Algorithms, pages 3–7, Toronto, Ontario, Canada. IEEE Computer Society.
DeMillo, R. A. and Offutt, A. J. (1991). Constraint-based automatic test data generation. IEEE Trans. Softw. Eng., 17:900–910.
Derezinska, A. and Halas, K. (2014). Experimental evaluation of mutation testing approaches to python programs. In 2014 IEEE Seventh International Conference on Software Testing, Verification and Validation Workshops, pages 156–164.
Derezinska A., Hałas K. (2015) Improving Mutation Testing Process of Python Programs. In: Silhavy R., Senkerik R., Oplatkova Z., Prokopova Z., Silhavy P. Software Engineering in Intelligent Systems, vol 349.
Fraser, G. and Arcuri, A. (2015). Achieving scalable mutation-based generation of whole test suites. Empirical Software Engineering, 20(3):783–812.
Jatana, N., Suri, B., Misra, S., Kumar, P., and Choudhury, A. R. (2016). Particle swarm based evolution and generation of test data using mutation testing. In Computational Science and Its Applications – ICCSA 2016, pages 585–594. Springer International Publishing.
Korel, B. (1990). Automated software test data generation. IEEE Transactions on Software Engineering, 16(8):870–879.
McMinn, P. and Holcombe, M. (2006). Evolutionary testing using an extended chaining approach. Evolutionary Computation, 14(1):41–64.
Papadakis, M. and Malevris, N. (2010). Automatic mutation test case generation via dynamic symbolic execution. In 2010 IEEE 21st International Symposium on Software Reliability Engineering, pages 121–130.
Papadakis, M. and Malevris, N. (2013). Searching and generating test inputs for mutation testing. Springer Plus, 2(1):1–12.
Papadakis, M., Malevris, N., and Kallia, M. (2010). Towards automating the generation of mutation tests. In Proceedings of the 5th Workshop on Automation of Software Test (AST), pages 111–118, Cape Town, South Africa. ACM.
Rani, S., Dhawan, H., Nagpal, G., and Suri, B. (2019). Implementing time-bounded automatic test data generation approach based on search-based mutation testing. In Progress in Advanced Computing and Intelligent Engineering, pages 113–122. Springer Singapore.
Russell, S. and Norvig, P. (2009). Artificial Intelligence: A Modern Approach.
Souza, F. C. M., Papadakis, M., Le Traon, Y., and Delamaro, M. E. (2016). Strong mutation-based test data generation using hill climbing. In Proceedings of the 9th International Workshop on Search-Based Software Testing, pages 45–54. ACM.
Wegener, J., Baresel, A., and Harmen, S. (2001). Evolutionary test environment for automatic structural testing. Information and Software Technology, 43(14):841–854.
Wohlin, C., Runeson, P., Host, M., Ohlsson, M. C., Regnell, B., and Wesslen, A. (2012). Experimentation in Software Engineering: An Introduction. Springer-Verlag Berlin Heidelberg, 1st. edition.
Bottaci, L. (2001). A genetic algorithm fitness function for mutation testing. In Proceedings of the First International Workshop on Software Engineering using Metaheuristic Innovative Algorithms, pages 3–7, Toronto, Ontario, Canada. IEEE Computer Society.
DeMillo, R. A. and Offutt, A. J. (1991). Constraint-based automatic test data generation. IEEE Trans. Softw. Eng., 17:900–910.
Derezinska, A. and Halas, K. (2014). Experimental evaluation of mutation testing approaches to python programs. In 2014 IEEE Seventh International Conference on Software Testing, Verification and Validation Workshops, pages 156–164.
Derezinska A., Hałas K. (2015) Improving Mutation Testing Process of Python Programs. In: Silhavy R., Senkerik R., Oplatkova Z., Prokopova Z., Silhavy P. Software Engineering in Intelligent Systems, vol 349.
Fraser, G. and Arcuri, A. (2015). Achieving scalable mutation-based generation of whole test suites. Empirical Software Engineering, 20(3):783–812.
Jatana, N., Suri, B., Misra, S., Kumar, P., and Choudhury, A. R. (2016). Particle swarm based evolution and generation of test data using mutation testing. In Computational Science and Its Applications – ICCSA 2016, pages 585–594. Springer International Publishing.
Korel, B. (1990). Automated software test data generation. IEEE Transactions on Software Engineering, 16(8):870–879.
McMinn, P. and Holcombe, M. (2006). Evolutionary testing using an extended chaining approach. Evolutionary Computation, 14(1):41–64.
Papadakis, M. and Malevris, N. (2010). Automatic mutation test case generation via dynamic symbolic execution. In 2010 IEEE 21st International Symposium on Software Reliability Engineering, pages 121–130.
Papadakis, M. and Malevris, N. (2013). Searching and generating test inputs for mutation testing. Springer Plus, 2(1):1–12.
Papadakis, M., Malevris, N., and Kallia, M. (2010). Towards automating the generation of mutation tests. In Proceedings of the 5th Workshop on Automation of Software Test (AST), pages 111–118, Cape Town, South Africa. ACM.
Rani, S., Dhawan, H., Nagpal, G., and Suri, B. (2019). Implementing time-bounded automatic test data generation approach based on search-based mutation testing. In Progress in Advanced Computing and Intelligent Engineering, pages 113–122. Springer Singapore.
Russell, S. and Norvig, P. (2009). Artificial Intelligence: A Modern Approach.
Souza, F. C. M., Papadakis, M., Le Traon, Y., and Delamaro, M. E. (2016). Strong mutation-based test data generation using hill climbing. In Proceedings of the 9th International Workshop on Search-Based Software Testing, pages 45–54. ACM.
Wegener, J., Baresel, A., and Harmen, S. (2001). Evolutionary test environment for automatic structural testing. Information and Software Technology, 43(14):841–854.
Wohlin, C., Runeson, P., Host, M., Ohlsson, M. C., Regnell, B., and Wesslen, A. (2012). Experimentation in Software Engineering: An Introduction. Springer-Verlag Berlin Heidelberg, 1st. edition.
Publicado
11/11/2020
Como Citar
FERREIRA, Matheus; COSTA, Lincoln; SOUZA, Francisco Carlos.
Search-based Test Data Generation for Mutation Testing: a tool for Python programs. In: ESCOLA REGIONAL DE ENGENHARIA DE SOFTWARE (ERES), 4. , 2020, Evento Online.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2020
.
p. 116-125.
DOI: https://doi.org/10.5753/eres.2020.13722.
