ARRESTT: A framework to create reproducible experiments to evaluate software testing techniques

• Iaron da Costa Araújo UFCG
• Wesley Oliveira da Silva UFCG
• José B. de Sousa Nunes UFCG
• Francisco Oliveira Neto UFCG

Resumo

Researchers have reported that software testing techniques (STT) in general lack empirical evidence, and yet empirical studies are still maturing in our field. Furthermore, validating existing experiments is often neglected by researchers in software engineering. Both executing and reproducing experiments are important to validate current scientific discoveries reported in literature. However, there is a lack of tools and frameworks to support these tasks. We propose a framework named ARRESTT that aids experimenters in creating and reproducing experiments. We validate ARRESTT through reproduction of a known experiment with test case selection techniques, and we are able to achieve results very similar to the original experiment. Based on an evaluation of reproducibility attributes, we conclude that ARRESTT enhances reproducibility of an experiment and does not demand a lot of effort to configure and execute an experiment.

Palavras-chave: Empirical Software Engineering, Reproducible Research, Software Testing Techniques