Selection of Test Cases Using Concepts of Variability: A Systematic Review
Abstract
In order to increase the quality, efficiency and effectiveness in the software testing process, the selection of test cases that explore in depth the features of the system for discovering defects is essential. In this context, this article presents a systematic review of test case selection using concepts of variability, whose goal is to verify the state of the art in this field. The results, such as confirming the importance of experiments, evaluation of results and use of qualitative and quantitative analysis of papers related to the selection of test cases are also shown in this paper.
Keywords:
Test Cases, Variability, Systematic Review
References
Biolchini, J., Mian, P.G., Natali, A.C.C., and Travassos, G.H. (2005). Sytematic review in software engineering. Technical report, RT–ES679/05. System Engineering and Computer Science Dept., COOPE/UFRJ.
Chen, T. Y., Kuo, F. and Liu, H. (2008). Distributing test cases more evenly in adaptive random testing. Journal of Systems and Software, 81(12), 2146-2162. www.scopus.com.
Chen, T. Y. (2010). "Fundamentals of test case selection: Diversity, diversity, diversity". Software Engineering and Data Mining (SEDM), 2010 2nd International Conference on , vol., no., pp.723-724, 23-25 June 2010
Delamaro, M. E., Nunes, F. d. L. d. S. and de Oliveira, R. A. P. (2011). Using concepts of content-based image retrieval to implement graphical testing oracles. Software Testing, Verification and Reliability. doi: 10.1002/stvr.463
Engstrom, Emelie; Skoglund, Mats; Runeson, Per. (2008). Empirical evaluations of regression test selection techniques: a systematic review. In Proceedings of the Second ACM-IEEE international symposium on Empirical software engineering and measurement (ESEM '08). ACM, New York, NY, USA, 22-31. DOI=10.1145/1414004.1414011 http://doi.acm.org/10.1145/1414004.1414011
Fanping Zeng; Ling Li; Juan Li; Xufa Wang. (2009). Research on Test Suite Reduction Using Attribute Relevance Analysis. Computer and Information Science, 2009. ICIS 2009. Eighth IEEE/ACIS International Conference no, pp.961-966, 1-3 June 2009. http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=5223197&isnumber=5222 522
Gleirscher, Mario (2011). Hazard-based selection of test cases. Proceeding of the 6th international workshop on Automation of software test. AST '11. doi: http://doi.acm.org/10.1145/1982595.1982609.
Kasurinen, Jussi; Taipale, Ossi; Smolander, Kari. (2010). Test case selection and prioritization: risk-based or design-based? In Proceedings of the 2010 ACM-IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM '10). ACM, New York, NY, USA, Article 10, 10 pages. DOI=10.1145/1852786.1852800 http://doi.acm.org/10.1145/1852786.1852800.
Kitchenham, B. (2004). Procedures for performing systematic reviews. Technical report, Keele University and NICTA.
Kosindrdecha, N., & Daengdej, J. (2010). A test case generation process and technique. Journal of Software Engineering, 4(4), 265-287. www.scopus.com.
Levenshtein, V. I. (2006). An evaluation of combination strategies for test case selection. In Empirical Software Engineering, volume 11, 2006.
Mirarab, S.; Akhlaghi Esfahani, S.; Tahvildari, L. (2011). "Size-Constrained Regression Test Case Selection Using Multi-Criteria Optimization," Software Engineering, IEEE Transactions on, vol.PP, no.99, pp.1, 0. doi: 10.1109/TSE.2011.56. http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=5928351&isnumber=4359 464.
Sapna, P. G.; Mohanty, H. (2010). Clustering test cases to achieve effective test selection. In: Proceedings of the 1st Amrita ACM-W Celebration on Women in Computing in India. New York, NY, USA: ACM, 2010. (A2CWiC'10), p.15:1-15:8. ISBN 978-1-4503-0194-7. http://doi.acm.org/10.1145/1858378.1858393.
Simons, A.J.H.; Wenwen Zhao. (2009). "Dynamic Analysis of Algebraic Structure to Optimize Test Generation and Test Case Selection," Testing: Academic and Industrial Conference - Practice and Research Techniques, 2009. TAIC PART '09. , vol., no., pp.33-42, 4-6 Sept. 2009. doi: 10.1109/TAICPART.2009.28.
Sommerville, Ian. (2007). Engenharia de Software. Oitava Edição. São Paulo, Pearson Addison Wesley: 2007.
Tsai, W.T.; Xinyu Zhou; Paul, R.A.; Yinong Chen; Xiaoying Bai. (2007). "A Coverage Relationship Model for Test Case Selection and Ranking for Multi-version Software," High Assurance Systems Engineering Symposium, 2007. HASE '07. 10th IEEE, vol., no., pp.105-112, 14-16 Nov. 2007. doi: 10.1109/HASE.2007.64.http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=4404732&isnumber=4404 712.
Vilkomir, S.A.; Swain, W.T.; Poore, J.H. (2008). "Combinatorial Test Case Selection with Markovian Usage Models," Information Technology: New Generations, 2008. ITNG 2008. Fifth International Conference on, vol., no., pp.3-8, 7-9 April 2008. http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=4492446&isnumber=4492 438.
Yu, L., Xu, L., & Tsai, W. (2010). Time-constrained test selection for regression testing. www.scopus.com.
Zhu Xiaochun; Zhou Bo; Li Juefeng; Gao Qiu; (2008). "A test automation solution on GUI functional test". Industrial Informatics, 2008, 6th IEEE International Conference on , vol., no., pp.1413-1418, 13-16 July 2008. doi: 10.1109/INDIN.2008.4618325.
Chen, T. Y., Kuo, F. and Liu, H. (2008). Distributing test cases more evenly in adaptive random testing. Journal of Systems and Software, 81(12), 2146-2162. www.scopus.com.
Chen, T. Y. (2010). "Fundamentals of test case selection: Diversity, diversity, diversity". Software Engineering and Data Mining (SEDM), 2010 2nd International Conference on , vol., no., pp.723-724, 23-25 June 2010
Delamaro, M. E., Nunes, F. d. L. d. S. and de Oliveira, R. A. P. (2011). Using concepts of content-based image retrieval to implement graphical testing oracles. Software Testing, Verification and Reliability. doi: 10.1002/stvr.463
Engstrom, Emelie; Skoglund, Mats; Runeson, Per. (2008). Empirical evaluations of regression test selection techniques: a systematic review. In Proceedings of the Second ACM-IEEE international symposium on Empirical software engineering and measurement (ESEM '08). ACM, New York, NY, USA, 22-31. DOI=10.1145/1414004.1414011 http://doi.acm.org/10.1145/1414004.1414011
Fanping Zeng; Ling Li; Juan Li; Xufa Wang. (2009). Research on Test Suite Reduction Using Attribute Relevance Analysis. Computer and Information Science, 2009. ICIS 2009. Eighth IEEE/ACIS International Conference no, pp.961-966, 1-3 June 2009. http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=5223197&isnumber=5222 522
Gleirscher, Mario (2011). Hazard-based selection of test cases. Proceeding of the 6th international workshop on Automation of software test. AST '11. doi: http://doi.acm.org/10.1145/1982595.1982609.
Kasurinen, Jussi; Taipale, Ossi; Smolander, Kari. (2010). Test case selection and prioritization: risk-based or design-based? In Proceedings of the 2010 ACM-IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM '10). ACM, New York, NY, USA, Article 10, 10 pages. DOI=10.1145/1852786.1852800 http://doi.acm.org/10.1145/1852786.1852800.
Kitchenham, B. (2004). Procedures for performing systematic reviews. Technical report, Keele University and NICTA.
Kosindrdecha, N., & Daengdej, J. (2010). A test case generation process and technique. Journal of Software Engineering, 4(4), 265-287. www.scopus.com.
Levenshtein, V. I. (2006). An evaluation of combination strategies for test case selection. In Empirical Software Engineering, volume 11, 2006.
Mirarab, S.; Akhlaghi Esfahani, S.; Tahvildari, L. (2011). "Size-Constrained Regression Test Case Selection Using Multi-Criteria Optimization," Software Engineering, IEEE Transactions on, vol.PP, no.99, pp.1, 0. doi: 10.1109/TSE.2011.56. http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=5928351&isnumber=4359 464.
Sapna, P. G.; Mohanty, H. (2010). Clustering test cases to achieve effective test selection. In: Proceedings of the 1st Amrita ACM-W Celebration on Women in Computing in India. New York, NY, USA: ACM, 2010. (A2CWiC'10), p.15:1-15:8. ISBN 978-1-4503-0194-7. http://doi.acm.org/10.1145/1858378.1858393.
Simons, A.J.H.; Wenwen Zhao. (2009). "Dynamic Analysis of Algebraic Structure to Optimize Test Generation and Test Case Selection," Testing: Academic and Industrial Conference - Practice and Research Techniques, 2009. TAIC PART '09. , vol., no., pp.33-42, 4-6 Sept. 2009. doi: 10.1109/TAICPART.2009.28.
Sommerville, Ian. (2007). Engenharia de Software. Oitava Edição. São Paulo, Pearson Addison Wesley: 2007.
Tsai, W.T.; Xinyu Zhou; Paul, R.A.; Yinong Chen; Xiaoying Bai. (2007). "A Coverage Relationship Model for Test Case Selection and Ranking for Multi-version Software," High Assurance Systems Engineering Symposium, 2007. HASE '07. 10th IEEE, vol., no., pp.105-112, 14-16 Nov. 2007. doi: 10.1109/HASE.2007.64.http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=4404732&isnumber=4404 712.
Vilkomir, S.A.; Swain, W.T.; Poore, J.H. (2008). "Combinatorial Test Case Selection with Markovian Usage Models," Information Technology: New Generations, 2008. ITNG 2008. Fifth International Conference on, vol., no., pp.3-8, 7-9 April 2008. http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=4492446&isnumber=4492 438.
Yu, L., Xu, L., & Tsai, W. (2010). Time-constrained test selection for regression testing. www.scopus.com.
Zhu Xiaochun; Zhou Bo; Li Juefeng; Gao Qiu; (2008). "A test automation solution on GUI functional test". Industrial Informatics, 2008, 6th IEEE International Conference on , vol., no., pp.1413-1418, 13-16 July 2008. doi: 10.1109/INDIN.2008.4618325.
Published
2012-05-16
How to Cite
NARCISO, Everton N.; NUNES, Fátima L. S.; DELAMARO, Márcio E..
Selection of Test Cases Using Concepts of Variability: A Systematic Review. In: BRAZILIAN SYMPOSIUM ON INFORMATION SYSTEMS (SBSI), 8. , 2012, São Paulo.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2012
.
p. 193-203.
DOI: https://doi.org/10.5753/sbsi.2012.14405.
