Characterisation of Challenges for Testing of Adaptive Systems

• Bento Rafael Siqueira UFSCar
• Fabiano Cutigi Ferrari UFSCar
• Marcel Akira Serikawa UFSCar
• Ricardo Menotti UFSCar
• Valter Vieira de Camargo UFSCar

Resumo

Context: Exercising the implementation of an adaptive system (AS) effectively, in order to detect faults, is not a trivial task. This is due to characteristics of this type of systems, such as the high number of configurations and runtime adaptations. In this context, the characterisation of challenges for testing ASs may support the definition of suitable testing strategies. However, such characterisation is currently spread over several studies in the literature. Objective: analysing testing-related challenges for ASs which are described in the literature, and establishing a generic classification. Method: we extended a Systematic Literature Review (SLR) of studies that address challenges for AS testing. By analysing the combined results (original and extended), we mapped descriptions found in individual studies, based on their similarities, in order to create a list of generic challenges. Results: a list of 12 generic challenges based on 34 specific ones described by several authors. Additionally, we analysed the relationship between the challenges and inherent properties of ASs. Conclusion: different terminology, varied levels of details and particular contexts (e.g. specific AS implementations) are recurring problems related to the characterisation of challenges for testing ASs. Generalisations such as the one proposed in this work may help researchers and practitioners do devise customised testing strategies that focus in recurring, inherent properties of ASs.

Palavras-chave: Adaptive systems, software testing, testing challenges