Avaliando Diferentes Estratégias de Redução do Custo de Teste de Mutação
Resumo
O teste baseado em mutação tem sido aplicado em diferentes contextos nos quais tem se mostrado um dos mais eficazes em revelar defeitos. Entretanto, uma disvantagem desse critério é o alto custo computacional. Por isso diferentes estratégias têm sido propostas na literatura com o objetivo de reduzir esse custo. A utilização de tais estratégias é fundamental para permitir a aplicação do teste de mutação em aplicações reais, entretanto, elas têm sido avaliadas somente para um conjunto de programas pequenos e com poucas estruturas de dados. Para fornecer indícios sobre o efeito dessas estratégias em programas mais complexos, este trabalho apresenta resultados de um experimento conduzido com a ferramenta Proteum e com quatro estratégias: Mutação Aleatória, Mutação Seletiva, Mutação Restrita e a Estratégia Essencial. As estratégias são avaliadas segundo o escore de mutação e custo, dado pela redução no número de mutantes gerados.
Referências
Acree, A., Budd, T., DeMillo, R., Lipton, R., and Sayward, F. (1979). Mutation analysis. School of Information and Computer Science, Georgia Institute of Technology, Atlanta.
Barbosa, E. (1998). Uma contribuição para determinação de um conjunto essencial de operadores de mutação no teste de programas C. Biblioteca Digital de Teses e Dissertações da USP.
Barbosa, E., Maldonado, J., and Vincenzi, A. (2001). Towards the determination of sufficient mutant operators for C. Software Testing Verification and Reliability, 11:113–136.
Choi, B., DeMillo, R., Krauser, E., Martin, R., Mathur, A., Offutt, A., Pan, H., and Spafford, E. (1989). The Mothra tool set (software testing). In System Sciences, 1989. Vol. II: Software Track, Proceedings of the Twenty-Second Annual Hawaii International Conference on, volume 2.
Delamaro, M. and Maldonado, J. (1996). Proteum–a tool for the assessment of test adequacy for C programs. In Proceedings of the Conference on Performability in Computing Systems (PCS 96), pages 79–95.
DeMillo, R., Guindi, D., McCracken, W., Offutt, A., and King, K. (1988). An extended overview of the Mothra software testing environment. In Software Testing, Verification, and Analysis, 1988., Proceedings of the Second Workshop on, pages 142–151.
DeMillo, R., Lipton, R., and Sayward, F. (1978). Hints on test data selection: Help for the practicing programmer. IEEE Software, 11:34–41.
Franzotte, L. and Vergilio, S. (2006). Applying Mutation Testing to XML Schemas. In 18th International Conference on Software Engineering and Knowledge Engineering (SEKE’06).
Hutchins, M., Foster, H., Goradia, T., and Ostrand, T. (1994). Experiments of the effectiveness of data data flow and control flow-based w-test adequacy criteria. In 16th International Conference on Software Engineering (ICSE 1994), pages 191–200.
Lee, S. and Offutt, A. (2001). Generating Test Cases for XML-based Web Component Interaction Using Mutation Analysis. In 12th International Symposium o Software Reliability Engineering, pages 200–209.
Ma, Y.-S., Offutt, A., and Kwon, Y. (2005). MuJava : An automated class mutation system. Software Testing Verification and Reliability, 2(15):97–133.
Mathur, A. (1991). Performance, effectiveness, and reliability issues in software testing. In the Fifteenth Annual International Computer Software and Applications Conference, COMPSAC’91, pages 604–605.
Namin, A. S., Andrews, J. H., and Murdoch, D. (2008). Sufficient Mutation Operators for Measuring Test Effectiveness. In International Conference on Software Engineering (ICSE’2008), pages 351–360.
Offutt, A., Lee, A., Rothermel, G., Untch, R., and Zapf, C. (1996). An experimental determination of sufficient mutant operators. ACM Transactions on Software Engineering and Methodology, 2(5):99–118.
Offutt, A., Rothermel, G., and Zapf, C. (1993). An experimental evaluation of selective mutation. In Proceedings of the 15th international conference on Software Engineering, pages 100–107. IEEE Computer Society Press Los Alamitos, CA, USA.
Souza, S. (1996). Avaliação do custo e eficácia do critério análise de mutantes na atividade de teste de software. PhD thesis, Dissertação de Mestrado, ICMC/USP, São Carlos, SP.
Souza, S. and et al (2007). Estudos Teóricos e Experimentais. In Introdução ao Teste de Software. Editora Campus-Elsevier.
Tuya, J., Suárez-Cabal, M., and Riva, C. d. l. (2007). Mutating database queries. Information and Software Technology, 4(49):398–417.
Vincenzi, A., Maldonado, J., Barbosa, E., and Delamaro, M. (1999). Operadores Essenciais de Interface: Um Estudo de Caso. In 13th Simposio Brasileiro de Engenharia de Software, Florianopolis, SC, pages 373–391.
Wong, W., Delamaro, M., Maldonado, J., and Mathur, A. (1994). Constrained mutation in C programs. In Proceedings of the 8th Brazilian Symposium on Software Engineering, pages 439–452.
Wong, W., Maldonado, J., Delamaro, M., and Souza, S. (1997). A comparison of selective mutation in C and fortran. In Workshop do Projeto Validação e Teste de Sistemas de Operação, pages 71–84.
Wong, W. and Mathur, A. (1995). Reducing the cost of mutation testing: An empirical study. The Journal of Systems & Software, 31(3):185–196.
Barbosa, E. (1998). Uma contribuição para determinação de um conjunto essencial de operadores de mutação no teste de programas C. Biblioteca Digital de Teses e Dissertações da USP.
Barbosa, E., Maldonado, J., and Vincenzi, A. (2001). Towards the determination of sufficient mutant operators for C. Software Testing Verification and Reliability, 11:113–136.
Choi, B., DeMillo, R., Krauser, E., Martin, R., Mathur, A., Offutt, A., Pan, H., and Spafford, E. (1989). The Mothra tool set (software testing). In System Sciences, 1989. Vol. II: Software Track, Proceedings of the Twenty-Second Annual Hawaii International Conference on, volume 2.
Delamaro, M. and Maldonado, J. (1996). Proteum–a tool for the assessment of test adequacy for C programs. In Proceedings of the Conference on Performability in Computing Systems (PCS 96), pages 79–95.
DeMillo, R., Guindi, D., McCracken, W., Offutt, A., and King, K. (1988). An extended overview of the Mothra software testing environment. In Software Testing, Verification, and Analysis, 1988., Proceedings of the Second Workshop on, pages 142–151.
DeMillo, R., Lipton, R., and Sayward, F. (1978). Hints on test data selection: Help for the practicing programmer. IEEE Software, 11:34–41.
Franzotte, L. and Vergilio, S. (2006). Applying Mutation Testing to XML Schemas. In 18th International Conference on Software Engineering and Knowledge Engineering (SEKE’06).
Hutchins, M., Foster, H., Goradia, T., and Ostrand, T. (1994). Experiments of the effectiveness of data data flow and control flow-based w-test adequacy criteria. In 16th International Conference on Software Engineering (ICSE 1994), pages 191–200.
Lee, S. and Offutt, A. (2001). Generating Test Cases for XML-based Web Component Interaction Using Mutation Analysis. In 12th International Symposium o Software Reliability Engineering, pages 200–209.
Ma, Y.-S., Offutt, A., and Kwon, Y. (2005). MuJava : An automated class mutation system. Software Testing Verification and Reliability, 2(15):97–133.
Mathur, A. (1991). Performance, effectiveness, and reliability issues in software testing. In the Fifteenth Annual International Computer Software and Applications Conference, COMPSAC’91, pages 604–605.
Namin, A. S., Andrews, J. H., and Murdoch, D. (2008). Sufficient Mutation Operators for Measuring Test Effectiveness. In International Conference on Software Engineering (ICSE’2008), pages 351–360.
Offutt, A., Lee, A., Rothermel, G., Untch, R., and Zapf, C. (1996). An experimental determination of sufficient mutant operators. ACM Transactions on Software Engineering and Methodology, 2(5):99–118.
Offutt, A., Rothermel, G., and Zapf, C. (1993). An experimental evaluation of selective mutation. In Proceedings of the 15th international conference on Software Engineering, pages 100–107. IEEE Computer Society Press Los Alamitos, CA, USA.
Souza, S. (1996). Avaliação do custo e eficácia do critério análise de mutantes na atividade de teste de software. PhD thesis, Dissertação de Mestrado, ICMC/USP, São Carlos, SP.
Souza, S. and et al (2007). Estudos Teóricos e Experimentais. In Introdução ao Teste de Software. Editora Campus-Elsevier.
Tuya, J., Suárez-Cabal, M., and Riva, C. d. l. (2007). Mutating database queries. Information and Software Technology, 4(49):398–417.
Vincenzi, A., Maldonado, J., Barbosa, E., and Delamaro, M. (1999). Operadores Essenciais de Interface: Um Estudo de Caso. In 13th Simposio Brasileiro de Engenharia de Software, Florianopolis, SC, pages 373–391.
Wong, W., Delamaro, M., Maldonado, J., and Mathur, A. (1994). Constrained mutation in C programs. In Proceedings of the 8th Brazilian Symposium on Software Engineering, pages 439–452.
Wong, W., Maldonado, J., Delamaro, M., and Souza, S. (1997). A comparison of selective mutation in C and fortran. In Workshop do Projeto Validação e Teste de Sistemas de Operação, pages 71–84.
Wong, W. and Mathur, A. (1995). Reducing the cost of mutation testing: An empirical study. The Journal of Systems & Software, 31(3):185–196.
Publicado
31/08/2009
Como Citar
BANZI, Adam et al.
Avaliando Diferentes Estratégias de Redução do Custo de Teste de Mutação. In: WORKSHOP DE TESTES E TOLERÂNCIA A FALHAS (WTF), 10. , 2009, João Pessoa/PB.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2009
.
p. 90-100.
ISSN 2595-2684.
DOI: https://doi.org/10.5753/wtf.2009.23136.
