Obtaining Trustworthy Test Results in Multi-threaded Systems - Dantas, Ayla; Gaudencio, Matheus; Brasileiro, Francisco; Cirne, Walfredo (Anais do Simpósio Brasileiro de Engenharia de Software (SBES), 2008)
Testing multi-threaded systems is quite a challenge. The non-determinism of these systems makes their implementation and their test implementation far more susceptible to error. It is common to have tests of these systems that may not pass sometimes and whose failures are not caused by application faults (bugs). For instance, this can happen when test verifications (assertions) are performed at inappropriate times. Unreliable tests make developers waste their time trying to find non-existing bugs, or else make them search for bugs in the wrong place. Another problem is that developers may cease to believe that certain test failures are caused by software bugs even when this is the case. In this paper, we propose an approach to avoid test failures that are caused, not by application bugs, but by test assertions performed either too early or too late. Our basic idea is to use thread monitoring to support test development. Moreover, we put forward a testing framework that follows the proposed approach, and we also evaluate the effectiveness of this framework for testing a multi-threaded system.