Fault model-based variability testing

• Ivan do Carmo Machado UFBA
• Eduardo Santana de Almeida UFBA

DOI: https://doi.org/10.5753/sbqs.2015.15233

Resumo

Software Product Lines (SPL) testing techniques are commonly focused on handling variability from a high level abstraction perspective, despite the importance of understanding the nature of issues emerging from source code that could affect the overall quality of products. In this investigation, we present a framework aimed to handle such a neglected issue by augmenting an SPL testing process with fault modeling support. Fault modeling is an strategy employed to capture the behaviour of the system against faults. By understanding the nature of faults before developing the tests might improve the likelihood of finding particular classes of errors. The proposed framework encompasses test assessment, to evaluate the effectiveness of existing test suites, and test design, by focusing on fault-prone elements. We carried out a controlled experiment to assess the test effectiveness of the proposed framework. Software engineers from an industrial partner acted as subjects. The assessment has shown promising results that confirm the hypothesis that combining fault models in an SPL testing process performs significantly better on improving the quality of test inputs.

Palavras-chave: Fault, variability, testing

Referências

Ammann, P. and Offutt, J. (2008). Introduction to software testing. Cambridge University Press.

Briand, L., Emam, K., and Morasca, S. (1996). On the application of measurement theory in software engineering. Empirical Software Engineering, 1(1):61–88.

Catal, C. and Mishra, D. (2012). Test case prioritization: a systematic mapping study. Software Quality Journal, pages 1–34.

Clements, P. and Northrop, L. (2001). Software Product Lines: Practices and Patterns. Addison-Wesley, Boston, MA, USA.

Fraser, G. and Zeller, A. (2012). Mutation-driven generation of unit tests and oracles. IEEE Transactions on Software Engineering, 38(2):278 –292.

Gamma, E., Helm, R., Johnson, R., and Vlissides, J. (1995). Design Patterns: Elements of Reusable Object-oriented Software. Addison-Wesley Longman Publishing Co., Inc., Boston, MA, USA.

Jia, Y. and Harman, M. (2011). An analysis and survey of the development of mutation testing. IEEE Transactions on Software Engineering, 37(5):649 –678.

Machado, I. C., Almeida, E. S., Gomes, G. S. S., Neto, P. A. M. S., Novais, R. L., and Neto, M. G. M. (2012a). A preliminary study on the effects of working with a testing Machado, I. C., Almeida, E. S., Gomes, G. S. S., Neto, P. A. M. S., Novais, R. L., and Neto, M. G. M. (2012a). A preliminary study on the effects of working with a testing

Machado, I. C., McGregor, J. D., Cavalcanti, Y. C., and Almeida, E. S. (2014a). On strategies for testing software product lines: A systematic literature review. Information & Software Technology, 56(10):1183–1199.

Machado, I. C., Neto, P. A. M. S., and Almeida, E. S. (2012b). Towards an integration testing approach for software product lines. In IEEE 13th International Conference on Information Reuse and Integration (IRI), pages 616–623.

Machado, I. C., Neto, P. A. M. S., Almeida, E. S., and Meira, S. R. L. (2011). RiPLE-TE: A Process for Testing Software Product Lines. In Proceedings of the 23rd International Conference on Software Engineering & Knowledge Engineering (SEKE).

Machado, I. C., Santos, A. R., Cavalcanti, Y. C., Trzan, E. G., Souza, M. M., and Almeida, E. S. (2014b). Low-level variability support for web-based software product lines. In Proceedings of the Eighth International Workshop on Variability Modelling of Software-Intensive Systems (VaMoS). ACM.

Martin, E. and Xie, T. (2007). A fault model and mutation testing of access control policies. In Proceedings of the 16th international conference on World Wide Web, WWW, pages 667–676, Banff, Alberta, Canada. ACM.

McGregor, J. D. (2008). Toward a fault model for software product lines. In Proceedings of the 12th International Conference on Software Product Lines, SPLC, pages 157– 162, Limerick, Ireland. IEEE Computer Society.

Nakagawa, E. Y., Antonino, P. O., and Becker, M. (2011). Reference architecture and product line architecture: A subtle but critical difference. In Proceedings of the 5th European Conference on Software Architecture, ECSA, pages 207–211.

Neto, P. A. M. S., Machado, I. C., McGregor, J. D., Almeida, E. S., and Meira, S. R. L. (2011). A systematic mapping study of software product lines testing. Information & Software Technology, 53(5):407–423.

Offutt, J., Alexander, R., Wu, Y., Xiao, Q., and Hutchinson, C. (2001). A fault model for subtype inheritance and polymorphism. In 12th International Symposium on Software Reliability Engineering (ISSRE).

Pohl, K., Bockle, G., and van der Linden, F. J. (2005). ̈ Software Product Line Engineer-ing: Foundations, Principles and Techniques. Springer-Verlag.

Rothermel, G., Untch, R. H., Chu, C., and Harrold, M. J. (2001). Prioritizing test cases for regression testing. IEEE Transactions on Software Engineering, 27(10):929–948.

Souza, I. S., Gomes, G. S. S., Neto, P. A. M. S., Machado, I. C., Almeida, E. S., and Meira, S. R. L. (2013). Evidence of software inspection on feature specification for software product lines. Journal of Systems and Software, 86(5):1172 – 1190.

Svahnberg, M., van Gurp, J., and Bosch, J. (2005). A taxonomy of variability realization techniques: Research articles. Software Practice & Experience, 35(8):705–754.

Xiao, X., Thummalapenta, S., and Xie, T. (2012). Advances on improving automation in developer testing. Advances in Computers, 85:165–212.