Relevance and Impact of Critical Factors of Success in Software Test Automation lifecycle: A Survey

• Anderson Rodrigues UFAM
• Arilo Dias-Neto UFAM

Resumo

Context:Software has increased in size and complexity. It has increased the amount of time and money required for testing. Many organizations have invested in software test automation (STA) waiting for reducing costs and improving the testing process. However, less than 50% of them reach the expected objectives related to software test automation mainly due to the lack of a clear understanding of what is involved. Objective: There is a documented gap between the academic and practitioners' points of views about software test automation. This paper has two main objectives. First, evaluating the relevance of 12 critical factors of success (CFS) in software test automation collected from the technical literature according to researchers views. Second, evaluating the impact of each of them on a basic software test automation lifecycle (BSTAL). Method: To archive the above objectives, we have performed a survey with software test practitioners. Each participant was invited by e-mail to answer an electronic survey to evaluate the relevance of CFSs. A cutoff value was defined to classify factors according to their relevant levels. Factors with relevant levels under 50% are not considered relevant. Relevant level was calculated in two ways: based on the entire population and based on subsets of participants grouped by their years of working in software testing. Results: We observed differences between academic and practitioner evaluations about the relevance level as well between the evaluations given by participants with different level of experience in software testing. We also observed that each factor impacts the BSTAL phases differently and they can be more or less relevant according to the phase under observation.

Palavras-chave: critical factors of success, lifecyle, test automation