Testing Techniques Selection: A Systematic Mapping Study
Resumo
[Context] Software projects must consider the selection of testing techniques and criteria during their life cycles. This practice increases the chances of testing activity to be appropriately performed. In a previous work, an infrastructure to support the selection of testing techniques was proposed for the context of concurrent software. This infrastructure considers information (attributes) of the project to make the selection closer to the testers need. [Objective] This paper extends the previous work by identifying new studies concerning testing techniques selection, project attributes that can be used for this selection and which approaches can be employed to support the combined selection. [Method] A mapping study was conducted and a total of 15 primary studies, published in the last 20 years were selected. Information about approaches to testing techniques selection was analyzed and classified. [Results] The following results were obtained: (i) existing approaches for selection of testing techniques; (ii) proposition of a taxonomy of selection approaches; (iii) characterization of attributes to offer the support for the selection of a testing technique; and (iv) identification of approaches that perform combined selection of testing techniques. [Conclusion] Combining testing techniques is essential to improve the testing activity quality by finding different failure categories and supplementing other techniques limitations. This paper describes an initiative to offer support for the construction of new or combined strategies for the selection of testing techniques capable of being used in practice.
Referências
Norma Barrett, Simon Martin, and Chryssa Dislis. 1999. Test process optimization: Closing the gap in the defect spectrum. In Test Conference, 1999. Proceedings. International. IEEE, 124--129.
Domenico Cotroneo, Roberto Pietrantuono, and Stefano Russo. 2013. A learning-based method for combining testing techniques. In Proceedings of the 2013 International Conference on Software Engineering. IEEE Press, 142--151.
Domenico Cotroneo, Roberto Pietrantuono, and Stefano Russo. 2013. Testing techniques selection based on ODC fault types and software metrics. Journal of Systems and Software 86, 6 (2013), 1613--1637.
Aurélio da Silva Grande, Arilo Claudio Dias Neto, and Rosiane de Freitas Rodrigues. 2012. Providing trade-off techniques subsets to improve software testing effectiveness: using evolutionary algorithm to support software testing techniques selection by a web tool. In Advances in Artificial Intelligence-SBIA 2012. Springer, 222--232.
Diego Dermeval, Jéssyka Vilela, Ig Ibert Bittencourt, Jaelson Castro, Seiji Isotani, and Patrick Brito. 2014. A systematic review on the use of ontologies in requirements engineering. In Software Engineering (SBES), 2014 Brazilian Symposium on. IEEE, 1--10.
Arilo Claudio Dias-Neto and Guilherme Horta Travassos. 2009. Evaluation of {model-based} testing techniques selection approaches: An external replication. In Empirical Software Engineering and Measurement, 2009. ESEM 2009. 3rd International Symposium on. IEEE, 269--278.
Arilo Claudio Dias-Neto and Guilherme Horta Travassos. 2014. Supporting the combined selection of model-based testing techniques. IEEE Transactions on Software Engineering 40, 10 (2014), 1025--1041.
Wei Ding, Peng Liang, Antony Tang, and Hans Van Vliet. 2014. Knowledge-based approaches in software documentation: A systematic literature review. Information and Software Technology 56, 6 (2014), 545--567.
Tore Dybå and Torgeir Dingsøyr. 2008. Empirical studies of agile software development: A systematic review. Information and software technology 50, 9 (2008), 833--859.
Sheikh Umar Farooq and SMK Quadri. 2013. Empirical evaluation of software testing techniques-need, issues and mitigation. Software engineering: an international Journal 3, 3 (2013), 41--51.
Barbara Kitchenham. 2004. Procedures for performing systematic reviews. Keele, UK, Keele University 33, 2004 (2004), 1--26.
Sara Mahdavi-Hezavehi, Matthias Galster, and Paris Avgeriou. 2013. Variability in quality attributes of service-based software systems: A systematic literature review. Information and Software Technology 55, 2 (2013), 320--343.
Silvana Morita Melo. 2018. A Framework for systematic testing techniques evaluation applied to concurrent programming. Ph.D. Dissertation. Universidade de São Paulo.
Silvana Morita Melo, Simone Rocio Senger de Souza, Paulo Sergio Lopes de Souza, and Jeffrey C Carver. 2017. How to test your concurrent software: an approach for the selection of testing techniques. In Proceedings of the 4th ACM SIGPLAN International Workshop on Software Engineering for Parallel Systems. ACM, 42--43.
Felipe Moreira Moura, Silvana Morita Melo, Simone do Rocio Senger de Souza, et al. 2018. SeleCTT: ferramenta para automatização do processo de seleção de técnicas de teste de software concorrente. (2018).
Glenford J Myers, Corey Sandler, and Tom Badgett. 2011. The art of software testing. John Wiley & Sons.
Arilo Claudio Dias Neto, Rosiane de Freitas Rodrigues, and Guilherme Horta Travassos.2011. Porantim-opt: Optimizing the combined selection of model-based testing techniques. In Software Testing, Verification and Validation Workshops (ICSTW), 2011 IEEE Fourth International Conference on. IEEE, 174--183.
Arilo Claudio Dias Neto and Guilherme Horta Travassos. 2009. Porantim: An approach to support the combination and selection of Model-based Testing techniques. In Automation of Software Test, 2009. AST'09. ICSE Workshop on. IEEE, 1--9.
Gregg Rothermel and Mary Jean Harrold. 1996. Analyzing regression test selection techniques. IEEE Transactions on software engineering 22, 8 (1996), 529--551.
Silvia Santa Isabel and Guilherme H Travassos. 2011. Features of software testing techniques for use in projects web. 14th Ibero-American Conference on Software Engineering and 14th Workshop on Requirements Engineering, CIbSE 2011 (2011), 129--142.
Muhammad Usman, Emilia Mendes, Francila Weidt, and Ricardo Britto. 2014. Effort estimation in agile software development: a systematic literature review. In Proceedings of the 10th international conference on predictive models in software engineering. ACM, 82--91.
Sira Vegas. 2004. Identifying the relevant information for software testing technique selection. In Empirical Software Engineering, 2004. ISESE'04. Proceedings. 2004 International Symposium on. IEEE, 39--48.
Sira Vegas and Victor Basili. 2005. A characterisation schema for software testing techniques. Empirical Software Engineering 10, 4 (2005), 437--466.Google Scholar
Sira Vegas, Natalia Juristo, and Victor Basili. 2006. Packaging experiences for improving testing technique selection. Journal of Systems and Software 79, 11 (2006), 1606--1618.
Monisha Victor and Nitin Upadhyay. 2011. Selection of software testing technique: A multi criteria decision making approach. In Trends in Computer Science, Engineering and Information Technology. Springer, 453--462.
Margaret A Wojcicki and Paul Strooper. 2007. An iterative empirical strategy for the systematic selection of a combination of verification and validation technologies. In Proceedings of the 5th International Workshop on Software Quality. IEEE Computer Society, 9.
