What is the adoption level of automated support for testing in open-source ecosystems?

• Rômulo Martins da Silva UFF
• Cafer Cruz UFF
• Heleno de S. Campos UFF
• Leonardo G. P. Murta UFF
• Vânia de Oliveira Neves UFF

Resumo

In the last decades, different kinds of automated support for testing have emerged in the open-source community. However, we still have limited evidence about the adoption level of such automated support in practice, considering different programming language ecosystems. In this paper, we investigate the adoption of automated support for testing among 184 popular open-source projects. Besides, we also investigate test coverage and metrics correlations on 571 open-source projects. As results, we found that projects written in Go, PHP, and JavaScript are the ones that most adopt automated support and that JavaScript and Python projects have the largest test coverage, with, on average, 84% and 81%, respectively. Moreover, we also found overall negligible correlations between projects' amount of stars, commits and source lines of code and coverage. Knowing that an open-source project has a high test coverage may enhance users' confidence in using this project. Besides that, we also listed the testing tools, libraries or frameworks that are most adopted for each programming language ecosystem. It may help developers in choosing appropriate automated support. Finally, we established a research agenda on this topic that motivates deeper studies as future work.

Palavras-chave: White-Box Coverage, Testing Tools, Software Testing, Repository Mining, Open Source