Manually Written or Generated Tests? A Study with Developers and Maintenance Tasks

• Wesley B. R. Herculano UFCG
• Melina Mongiovi UFCG
• Everton L. G. Alves UFCG

Resumo

Software testing is an important part of software development. In addition to finding faults in the code, good tests should help developers to correct faults and be easily updated in case of modifications due to code changes. Automatically generated tests can save time and may lead to higher code coverage. However, these tests may be less readable and not based on realistic scenarios. Few pieces of research have been done to evaluate whether automatically generated tests are maintainable and support developers when maintaining code. To further investigate this issue, we perform an empirical study with 20 real developers to compare how they perform maintenance tasks with automatically generated (Evosuite or Randoop) and manually-written test cases. Our results indicate that automatically generated tests can be a great help for identifying faults during maintenance. Also, we found that all strategies were similar at helping to produce correct bug fixes and with similar efficiency. Therefore, we can say that developers may integrate generated test suites into the project at any stage.

Palavras-chave: randoop, generated tests, evosuite, maintenance