Avaliando Diferentes Estratégias de Redução do Custo de Teste de Mutação
Abstract
Mutation based testing has been applied in diverse contexts and has been considered by many authors as the most efficacious criterion in terms of revealed faults. However, it presents high computational costs. Because of this, different strategies for reducing mutation test costs have been proposed in the literature. The evaluation of such strategies is fundamental to allow the mutation test application in large systems, however, they have been evaluated only with small programs, with few structures. To provide information about the effects of those strategies in complex programs, this work presents results from an experiment conducted with Proteum tool and four strategies: Random Mutation, Selective Mutation, Constraint Mutation and the Essential Strategy. The strategies are evaluated according to the mutation score and cost, given by the number of generated mutants.
References
Acree, A., Budd, T., DeMillo, R., Lipton, R., and Sayward, F. (1979). Mutation analysis. School of Information and Computer Science, Georgia Institute of Technology, Atlanta.
Barbosa, E. (1998). Uma contribuição para determinação de um conjunto essencial de operadores de mutação no teste de programas C. Biblioteca Digital de Teses e Dissertações da USP.
Barbosa, E., Maldonado, J., and Vincenzi, A. (2001). Towards the determination of sufficient mutant operators for C. Software Testing Verification and Reliability, 11:113–136.
Choi, B., DeMillo, R., Krauser, E., Martin, R., Mathur, A., Offutt, A., Pan, H., and Spafford, E. (1989). The Mothra tool set (software testing). In System Sciences, 1989. Vol. II: Software Track, Proceedings of the Twenty-Second Annual Hawaii International Conference on, volume 2.
Delamaro, M. and Maldonado, J. (1996). Proteum–a tool for the assessment of test adequacy for C programs. In Proceedings of the Conference on Performability in Computing Systems (PCS 96), pages 79–95.
DeMillo, R., Guindi, D., McCracken, W., Offutt, A., and King, K. (1988). An extended overview of the Mothra software testing environment. In Software Testing, Verification, and Analysis, 1988., Proceedings of the Second Workshop on, pages 142–151.
DeMillo, R., Lipton, R., and Sayward, F. (1978). Hints on test data selection: Help for the practicing programmer. IEEE Software, 11:34–41.
Franzotte, L. and Vergilio, S. (2006). Applying Mutation Testing to XML Schemas. In 18th International Conference on Software Engineering and Knowledge Engineering (SEKE’06).
Hutchins, M., Foster, H., Goradia, T., and Ostrand, T. (1994). Experiments of the effectiveness of data data flow and control flow-based w-test adequacy criteria. In 16th International Conference on Software Engineering (ICSE 1994), pages 191–200.
Lee, S. and Offutt, A. (2001). Generating Test Cases for XML-based Web Component Interaction Using Mutation Analysis. In 12th International Symposium o Software Reliability Engineering, pages 200–209.
Ma, Y.-S., Offutt, A., and Kwon, Y. (2005). MuJava : An automated class mutation system. Software Testing Verification and Reliability, 2(15):97–133.
Mathur, A. (1991). Performance, effectiveness, and reliability issues in software testing. In the Fifteenth Annual International Computer Software and Applications Conference, COMPSAC’91, pages 604–605.
Namin, A. S., Andrews, J. H., and Murdoch, D. (2008). Sufficient Mutation Operators for Measuring Test Effectiveness. In International Conference on Software Engineering (ICSE’2008), pages 351–360.
Offutt, A., Lee, A., Rothermel, G., Untch, R., and Zapf, C. (1996). An experimental determination of sufficient mutant operators. ACM Transactions on Software Engineering and Methodology, 2(5):99–118.
Offutt, A., Rothermel, G., and Zapf, C. (1993). An experimental evaluation of selective mutation. In Proceedings of the 15th international conference on Software Engineering, pages 100–107. IEEE Computer Society Press Los Alamitos, CA, USA.
Souza, S. (1996). Avaliação do custo e eficácia do critério análise de mutantes na atividade de teste de software. PhD thesis, Dissertação de Mestrado, ICMC/USP, São Carlos, SP.
Souza, S. and et al (2007). Estudos Teóricos e Experimentais. In Introdução ao Teste de Software. Editora Campus-Elsevier.
Tuya, J., Suárez-Cabal, M., and Riva, C. d. l. (2007). Mutating database queries. Information and Software Technology, 4(49):398–417.
Vincenzi, A., Maldonado, J., Barbosa, E., and Delamaro, M. (1999). Operadores Essenciais de Interface: Um Estudo de Caso. In 13th Simposio Brasileiro de Engenharia de Software, Florianopolis, SC, pages 373–391.
Wong, W., Delamaro, M., Maldonado, J., and Mathur, A. (1994). Constrained mutation in C programs. In Proceedings of the 8th Brazilian Symposium on Software Engineering, pages 439–452.
Wong, W., Maldonado, J., Delamaro, M., and Souza, S. (1997). A comparison of selective mutation in C and fortran. In Workshop do Projeto Validação e Teste de Sistemas de Operação, pages 71–84.
Wong, W. and Mathur, A. (1995). Reducing the cost of mutation testing: An empirical study. The Journal of Systems & Software, 31(3):185–196.
Barbosa, E. (1998). Uma contribuição para determinação de um conjunto essencial de operadores de mutação no teste de programas C. Biblioteca Digital de Teses e Dissertações da USP.
Barbosa, E., Maldonado, J., and Vincenzi, A. (2001). Towards the determination of sufficient mutant operators for C. Software Testing Verification and Reliability, 11:113–136.
Choi, B., DeMillo, R., Krauser, E., Martin, R., Mathur, A., Offutt, A., Pan, H., and Spafford, E. (1989). The Mothra tool set (software testing). In System Sciences, 1989. Vol. II: Software Track, Proceedings of the Twenty-Second Annual Hawaii International Conference on, volume 2.
Delamaro, M. and Maldonado, J. (1996). Proteum–a tool for the assessment of test adequacy for C programs. In Proceedings of the Conference on Performability in Computing Systems (PCS 96), pages 79–95.
DeMillo, R., Guindi, D., McCracken, W., Offutt, A., and King, K. (1988). An extended overview of the Mothra software testing environment. In Software Testing, Verification, and Analysis, 1988., Proceedings of the Second Workshop on, pages 142–151.
DeMillo, R., Lipton, R., and Sayward, F. (1978). Hints on test data selection: Help for the practicing programmer. IEEE Software, 11:34–41.
Franzotte, L. and Vergilio, S. (2006). Applying Mutation Testing to XML Schemas. In 18th International Conference on Software Engineering and Knowledge Engineering (SEKE’06).
Hutchins, M., Foster, H., Goradia, T., and Ostrand, T. (1994). Experiments of the effectiveness of data data flow and control flow-based w-test adequacy criteria. In 16th International Conference on Software Engineering (ICSE 1994), pages 191–200.
Lee, S. and Offutt, A. (2001). Generating Test Cases for XML-based Web Component Interaction Using Mutation Analysis. In 12th International Symposium o Software Reliability Engineering, pages 200–209.
Ma, Y.-S., Offutt, A., and Kwon, Y. (2005). MuJava : An automated class mutation system. Software Testing Verification and Reliability, 2(15):97–133.
Mathur, A. (1991). Performance, effectiveness, and reliability issues in software testing. In the Fifteenth Annual International Computer Software and Applications Conference, COMPSAC’91, pages 604–605.
Namin, A. S., Andrews, J. H., and Murdoch, D. (2008). Sufficient Mutation Operators for Measuring Test Effectiveness. In International Conference on Software Engineering (ICSE’2008), pages 351–360.
Offutt, A., Lee, A., Rothermel, G., Untch, R., and Zapf, C. (1996). An experimental determination of sufficient mutant operators. ACM Transactions on Software Engineering and Methodology, 2(5):99–118.
Offutt, A., Rothermel, G., and Zapf, C. (1993). An experimental evaluation of selective mutation. In Proceedings of the 15th international conference on Software Engineering, pages 100–107. IEEE Computer Society Press Los Alamitos, CA, USA.
Souza, S. (1996). Avaliação do custo e eficácia do critério análise de mutantes na atividade de teste de software. PhD thesis, Dissertação de Mestrado, ICMC/USP, São Carlos, SP.
Souza, S. and et al (2007). Estudos Teóricos e Experimentais. In Introdução ao Teste de Software. Editora Campus-Elsevier.
Tuya, J., Suárez-Cabal, M., and Riva, C. d. l. (2007). Mutating database queries. Information and Software Technology, 4(49):398–417.
Vincenzi, A., Maldonado, J., Barbosa, E., and Delamaro, M. (1999). Operadores Essenciais de Interface: Um Estudo de Caso. In 13th Simposio Brasileiro de Engenharia de Software, Florianopolis, SC, pages 373–391.
Wong, W., Delamaro, M., Maldonado, J., and Mathur, A. (1994). Constrained mutation in C programs. In Proceedings of the 8th Brazilian Symposium on Software Engineering, pages 439–452.
Wong, W., Maldonado, J., Delamaro, M., and Souza, S. (1997). A comparison of selective mutation in C and fortran. In Workshop do Projeto Validação e Teste de Sistemas de Operação, pages 71–84.
Wong, W. and Mathur, A. (1995). Reducing the cost of mutation testing: An empirical study. The Journal of Systems & Software, 31(3):185–196.
Published
2009-08-31
How to Cite
BANZI, Adam et al.
Avaliando Diferentes Estratégias de Redução do Custo de Teste de Mutação. In: FAULT TOLERANCE WORKSHOP (WTF), 10. , 2009, João Pessoa/PB.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2009
.
p. 90-100.
ISSN 2595-2684.
DOI: https://doi.org/10.5753/wtf.2009.23136.
