Causality Assessment between Internal Quality Metrics and Defects
Abstract
Despite the interest and the common recommendation for the use of metrics to improve the internal quality of a system, there is still no consensus about the real impacts of metrics in external quality in software systems. Therefore, this paper reports a study carried out to evaluate whether there is a causal relationship between source code metrics and central measure of the external quality of a system, number of bugs. The study aimed to evaluate the cause effect relationship between CK and OO metrics and the number of bugs of five systems belonging to a public benchmark. As a result, we have concluded that in the presence of all classes of a system, there is no evidence of causality between metrics is bugs. However, when we have restricted to classes of a system that had at least one bug in its history, there is evidence of causality for some metrics.
Keywords:
Causality Assessment, Quality Metrics, Defects
References
Giuliano Antoniol, Kamel Ayari, Massimiliano Di Penta, Foutse Khomh, and Yann-Gaël Guèhèneuc. Is it a bug or an enhancement? In 18th Conference of the Center for Advanced Studies on Collaborative Research: Meeting of Minds (CASCON), pages 304–318, 2008.
Gerardo Canfora, Michele Ceccarelli, Massimiliano Di Penta, and Luigi Cerulo. Using multivariate time series and association rules to detect logical change coupling: an empirical study. In 26th International Conference on Software Maintenance (ICSM), pages 1–10, 2010.
S. R. Chidamber and C. F. Kemerer. A metrics suite for object oriented design. IEEE Transactions on Software Engineering, 20(6):476–493, 1994.
Marco D’Ambros, Alberto Bacchelli, and Michele Lanza. On the impact of design flaws on software defects. In 10th International Conference on Quality Software (QSIC), pages 23–31, 2010.
Marco D’Ambros, Michele Lanza, and Romain Robbes. An extensive comparison of bug prediction appro aches. In 7th Working Conference on Mining Software Repositories (MSR), pages 31–41, 2010.
C. W. J. Granger. Investigating causal relations by econometric models and cross-spectral methods. Econometrica, 37(3):424–438, 19
Ahmed E. Hassan. Predicting faults using the complexity of code changes. In 31st International Conference on Software Engineering (ICSE), pages 78–88, 2009.
Tilman Holschuh, Markus Pauser, Kim Herzig, Thomas Zimmermann, Rahul Premraj, and Andreas Zel ler. Predicting defects in sap java code: An experience report. In 31st International Conference on Software Engineering (ICSE), pages 172–181, 2009.
Sunghun Kim, Thomas Zimmermann, E. James Whitehead Jr., and Andreas Zeller. Predicting faults from cached history. In 29th International Conference on Software Engineering (ICSE), pages 489–498, 2007.
Raimund Moser, Witold Pedrycz, and Giancarlo Succi. A comparative analysis of the efficiency of change metrics and static code attributes for defect prediction. In 30th International Conference on Software Engineering (ICSE), pages 181–190, 2008.
Nachiappan Nagappan and Thomas Ball. Static analysis tools as early indicators of pre-release defect density. In 27th International Conference on Software Engineering (ICSE), pages 580–586, 2005.
Nachiappan Nagappan and Thomas Ball. Use of relative code churn measures to predict system defect density. In 27th International Conference on Software Engineering (ICSE), pages 284–292, 2005.
Nachiappan Nagappan, Thomas Ball, and Andreas Zeller. Mining metrics to predict component failures. In 28th International Conference on Software Engineering (ICSE), pages 452–461, 2006.
Dewayne E. Perry, Adam A. Porter, and Lawrence G. Votta. A primer on empirical studies (tutorial). In Tutorial presented at 19th International Conference on Software Engineering (ICSE), pages 657–658, 1997.
Ramanath Subramanyam and M. S. Krishnan. Empirical analysis of CK metrics for object-oriented design complexity: Implications for software defects. IEEE Transaction on Software Engineering, 29(4):297–310, 2003.
Claes Wohlin, Per Runeson, Martin Höst, Magnus C. Ohlsson, Bjöörn Regnell, and Anders Wesslén. Experimentation in software engineering: an introduction. Kluwer Academic Publishers, 2000.
Thomas Zimmermann, Nachiappan Nagappan, and Andreas Zeller. Predicting Bugs from History, chapter 4, pages 69–88. Springer, 2008.
Gerardo Canfora, Michele Ceccarelli, Massimiliano Di Penta, and Luigi Cerulo. Using multivariate time series and association rules to detect logical change coupling: an empirical study. In 26th International Conference on Software Maintenance (ICSM), pages 1–10, 2010.
S. R. Chidamber and C. F. Kemerer. A metrics suite for object oriented design. IEEE Transactions on Software Engineering, 20(6):476–493, 1994.
Marco D’Ambros, Alberto Bacchelli, and Michele Lanza. On the impact of design flaws on software defects. In 10th International Conference on Quality Software (QSIC), pages 23–31, 2010.
Marco D’Ambros, Michele Lanza, and Romain Robbes. An extensive comparison of bug prediction appro aches. In 7th Working Conference on Mining Software Repositories (MSR), pages 31–41, 2010.
C. W. J. Granger. Investigating causal relations by econometric models and cross-spectral methods. Econometrica, 37(3):424–438, 19
Ahmed E. Hassan. Predicting faults using the complexity of code changes. In 31st International Conference on Software Engineering (ICSE), pages 78–88, 2009.
Tilman Holschuh, Markus Pauser, Kim Herzig, Thomas Zimmermann, Rahul Premraj, and Andreas Zel ler. Predicting defects in sap java code: An experience report. In 31st International Conference on Software Engineering (ICSE), pages 172–181, 2009.
Sunghun Kim, Thomas Zimmermann, E. James Whitehead Jr., and Andreas Zeller. Predicting faults from cached history. In 29th International Conference on Software Engineering (ICSE), pages 489–498, 2007.
Raimund Moser, Witold Pedrycz, and Giancarlo Succi. A comparative analysis of the efficiency of change metrics and static code attributes for defect prediction. In 30th International Conference on Software Engineering (ICSE), pages 181–190, 2008.
Nachiappan Nagappan and Thomas Ball. Static analysis tools as early indicators of pre-release defect density. In 27th International Conference on Software Engineering (ICSE), pages 580–586, 2005.
Nachiappan Nagappan and Thomas Ball. Use of relative code churn measures to predict system defect density. In 27th International Conference on Software Engineering (ICSE), pages 284–292, 2005.
Nachiappan Nagappan, Thomas Ball, and Andreas Zeller. Mining metrics to predict component failures. In 28th International Conference on Software Engineering (ICSE), pages 452–461, 2006.
Dewayne E. Perry, Adam A. Porter, and Lawrence G. Votta. A primer on empirical studies (tutorial). In Tutorial presented at 19th International Conference on Software Engineering (ICSE), pages 657–658, 1997.
Ramanath Subramanyam and M. S. Krishnan. Empirical analysis of CK metrics for object-oriented design complexity: Implications for software defects. IEEE Transaction on Software Engineering, 29(4):297–310, 2003.
Claes Wohlin, Per Runeson, Martin Höst, Magnus C. Ohlsson, Bjöörn Regnell, and Anders Wesslén. Experimentation in software engineering: an introduction. Kluwer Academic Publishers, 2000.
Thomas Zimmermann, Nachiappan Nagappan, and Andreas Zeller. Predicting Bugs from History, chapter 4, pages 69–88. Springer, 2008.
Published
2011-06-06
How to Cite
COUTO, César; VALENTE, Marco Túlio; BIGONHA, Roberto da Silva.
Causality Assessment between Internal Quality Metrics and Defects. In: BRAZILIAN SOFTWARE QUALITY SYMPOSIUM (SBQS), 10. , 2011, Curitiba.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2011
.
p. 279-293.
DOI: https://doi.org/10.5753/sbqs.2011.15401.
