Constrained Mutation in C programs
Resumo
Software development is always under the pressure of time and budget constraints before release. A good testing strategy should not only be effective and economical but also incremental. Although mutation testing has been empirically found to be effective in detecting faults, it remains unused for reasons of economics. A major obstacle to the use of mutation testing is its high computational cost. In this paper we report results from experiments designed to investigate six different constrained mutation mechanisms. Our data indicate that these alternatives not only reduce the cost of mutation significantly in terms of the number of test cases required and the number of mutants to be examined, but also maintain very good fault detection effectiveness. Effects of incremental mutation testing examining different sets of mutants are also discussed. Furthermore, our experiments are unique in that constrained mutation was performed directly on C programs. This eliminates the possible bias experienced by earlier mutation studies because of the programming language translation between the Fortran, Pascal, and C.
Referências
H. Agrawal, R. A. DeMillo. R. Hathaway. Wm. Hsu, W. Hsu, E. W. Krauser. R. J. Martin. A. P. Mathur, and E. H. Spafford. "Design of mutant operators for the C programming language." Technical Report SERC-TR-41-P, Software Engineering Research Center. Purdue University, W. Lafayette, IN, March 1989.
R.S. Boyer. B. Elspas. and K. N. Levitt. "SELECT - a formal system for testing and debugging programs by symbolic execution." Sigplan Notices. 10(6):234-245. June 1975.
T. A. Budd. "Mutation Analysis of Program Test Data. PhD thesis. Yale University. New Haven, CT. May 1980.
T. A. Budd. "Mutation analysis: Ideas, examples, problems and prospect." in Computer Program Testing. B. Chandrasekaran and S. Radicchi. Eds. Amsterdam. North Holland. July 1981.
B. J. Choi. R. A. DeMillo, E. W. Krauser, A. P. Mathur. R. J. Martin. A. J. Offutt. H. Pan. and E. H. Spafford. "The Mothra toolset," in Proceedings of the Twenty-Second Annual Hawaii International Conference on System Sciences, HI, January 1989.
L. A. Clarke, "A system to generate test data and symbolically execute programs." IEEE Trans. on Software Engineering, SE-2(3):215-222, September 1976.
M. E. Delamaro, J. C, Maldonado. M. Jino, and M. Chaim. "PROTEUM: A testing tool based on mutation analysis," in Proceedings of the Seventh Brazilian Symposium on Software Engineering, Rio de Janeiro, Brazil. October 1993.
R. A. DeMillo. D. S. Guindi. K. N. King, W. M. MeCracken. and A. J. Offutt, "An extended overview of the Mothra software testing environment." in Proceedings of the Second Workshop on Software Testing. Verification and Analysis, pp 142-151. Banff, Alberta. Canada, July 1988.
R. A. DeMillo, R. J. Lipton, and F. G. Sayward, "Hints on test data selection: Help for the practicing programmer." IEEE Computer. 11(4):34-41, April 1978.
W. E. Howden. "Reliability of the path analysis testing strategy," IEEE Trans. on Software Engineering. SE-2(3):208-214. September 1976.
E. W. Krauser. A. P. Mathur, and V. J. Rego, "High performance software testing on SIMD machines." IEEE Trans. on Software Engineering, 17(5):403-423, May 1991.
A. P. Mathur. "Performance, effectiveness, and reliability issues in software testing," in Proceedings of the Fifteenth Annual International Computer Software and Applications Conference, pp 604-605, Tokyo, Japan, September 1991.
A. P. Mathur. "Mutation testing," Encyclopedia of Software Engineering, pp 707-713. 1994.
A. P. Mathur and W. E. Wong, "Evaluation of the cost of alternate mutation testing strategies," in Proceedings of the Seventh Brasilian Symposium on Software Engineering, pp 320-335. Rio de Janeiro. Brazil. October 1993.
A. P. Mathur and W. E. Wong, "An empitical comparison of data flow and mutation hased test adequacy criteria." The Journal of Software Testing. Verification. and Reliability. 4(1):9-31, March 1994.
A. J. Offutt. G. Rothermel. and C. Zapf, "An experimental evaluation of selective mutation:" in Proceedings of the 15th International Conference on Software Engineering, pp 100-107, Baltimore. MD, May 1993.
S. N. Weiss and V. N. Fleyshgakker. "Improved serial algorithms for mutation analysis." in Proceedings of International Symposium on Software Testing and Analysis, pp 149-158, Cambridge, MA. June 1993.
E. J. Weyuker and B. Jeng. "Analyzing partition testing strategies." IEEE Trans. on Software Engineering, 17(7):703-711. July 1991.
W. E. Wong, "On Mutation and Data Flow, " PhD thesis, Department of Computer Science, Purdue University, W. Lafayette, IN. December 1993.
W. E. Wong and A. P. Mathur, "Fault detection effectiveness of mutation and data fow testing," Software Quality Journal, 1995. (To appear)
W. E. Wong and A. P. Mathur, "Reducing the cost of mutation testing: An empirical study," The Journal of Systems and Software, 1995. (To appear)
