Predicting Software Defects with Explainable Machine Learning

• Geanderson Santos UFMG https://orcid.org/0000-0002-7571-6578
• Eduardo Figueiredo UFMG https://orcid.org/0000-0002-6004-2718
• Adriano Veloso UFMG
• Markos Viggiato University of Alberta
• Nivio Ziviani UFMG

Resumo

Most software systems must evolve to cope with stakeholders’ requirements and fix existing defects. Hence, software defect prediction represents an area of interest in both academia and the software industry. As a result, predicting software defects can help the development team to maintain substantial levels of software quality. For this reason, machine learning models have increased in popularity for software defect prediction and have demonstrated effectiveness in many scenarios. In this paper, we evaluate a machine learning approach for selecting features to predict software module defects. We use a tree boosting algorithm that receives as input a training set comprising records of software features encoding characteristics of each module and outputs whether the corresponding module is defective prone. For nine projects within the widely known NASA data program, we build prediction models from a set of easy-to-compute module features. We then sample this sizable model space by randomly selecting software features to compose each model. This significant number of models allows us to structure our work along model understandability and predictive accuracy. We argue that explaining model predictions is meaningful to provide information to developers on features related to each module defective-prone. We show that (i) features that contribute most to finding the best models may vary depending on the project, and (ii) effective models are highly understandable based on a survey with 40 developers.

Palavras-chave: software defects, explainable models, NASA datasets, SHAP values