Guidance for Efficiently Implementing Defect Causal Analysis

  • Marcos Kalinowski UFRJ / Bennett
  • Guilherme H. Travassos UFRJ
  • David N. Card Det Norske Veritas


Defect causal analysis has shown itself to be a cheap and high return means of product-focused software process improvement. However, despite its advantages and wide industry adoption little academic research is being done in this area. Thus, professionals face several questions when implementing it in software organizations. Aiming to provide unbiased and evidence-based answers to those questions, a systematic review has been conducted. Based on the results of the systematic review, better guidance for implementing defect causal analysis efficiently in software organizations can be elaborated.
Palavras-chave: Guidance, Efficiently Implementing, Defect Causal Analysis


Al-Shehab, A. J., Hughes, R. T., Winstanley, G., Facilitating Organisational Learning through Causal Mapping Techniques in IS/IT Project Risk Management, LNCS, 3782 NAI, 145 - 154, 2005.

Bhandari, I., Halliday, M., Tarver, E., Brown, D., Chaar, J., Chillarege, R., A Case Study of Software Process Improvement During Development, IEEE Trans. on Soft. Eng., 19 (12), 1157 – 1170, 1993.

Bhandari, I., Roth, N., Post-process Feedback with and without Attribute Focusing: a Comparative Evaluation, in 'ICSE '93: Proceedings of the 15th Intl’ Conf. on Soft. Eng.', IEEE Computer Society Press, Los Alamitos, CA, USA, pp. 89-98, 1993.

Biolchini, J.; Mian, P.G.; Natali, A.C.; & Travassos, G.H. (2005), “Systematic Review in Software Engineering: Relevance and Utility”, Technical Report ES-679/05, PESC-COPPE/UFRJ. Available at .

Boehm, B., A view of 20th and 21st century software engineering, in 'ICSE '06: Proceeding of the 28th international conf. on Soft. Eng.', ACM Press, New York, NY, USA, pp. 12-29, 2006.

Brereton, P., Kitchenham, B. A., Budgen, D., Turner, M., Khalil, M., Lessons from applying the systematic literature review process within the software engineering domain. Journal of Systems and Software. Volume 80, Issue 4. Pages 571-583, 2007.

Bush, M., Getting started on metrics - JPL productivity and quality, in 'ICSE '90: Proceedings of the 12th international conf. on soft. eng.', IEEE Comp. Soc. Press, Los Alamitos, CA, USA, pp. 133-142, 1990.

Card, D., “Defect Analysis: Basic Techniques for Management and Learning”, Advances in Computers, vol. 65, chapter 7, pp. 259-295, 2005.

Card, D., Learning from our mistakes with defect causal analysis, IEEE Software, 15(1), pp. 56-63, 1998.

Card, D., Defect Causal Analysis Drives Down Error Rates, IEEE Software 1/1993, Volume 10, Issue 4, 7/1993, p.98-99, 1993.

Chang, C., Chu, C., "Defect Prevention in Software Processes: An Action-Based Approach", The Journal of Systems and Software, Vol. 80, issue 4, April 2007, pp. 559-570, 2007.

Chernak, Y., A statistical Approach to the Inspection Checklist Formal Synthesis and Improvement, IEEE Transactions on Software Engineering, 22(12), 866-874, 1996.

Chillarege, R., Bhandari, I., Chaar, J., Halliday, M., Moebus, D., Ray, B., Wong, M.Y., Ortogonal Defect Classification – A Concept for In-Process Measurement, IEEE Transactions on Software Engineering, vol. 18, pp. 943-956, 1992.

Collofello, J., Gosalla, B., Application of Causal Analysis to the Software Modification Process, Software Practice and Experience, 23(10), pp. 1095–1105, 1993.

Dalal, S. R., Horgan, J. R., Kettenring, J. R., Reliable Software and Communication: Software Quality, Reliability, and Safety, in 'ICSE '93: Proceedings of the 15th International Conference on Software Engineering', IEEE Computer Society Press, Los Alamitos, CA, USA, pp. 425-435, 1993.

Damele, G., Bazzana, G., Andreis, F., Aquilio, S., Process Improvement through Root Cause Analysis, in ‘Proceedings of Third International Conference on Achieving Quality in Software’, pp. 35--47, 1996.

Damm, L., Lundberg, L., Company-wide Implementation of Metrics for Early Software Fault Detection, Proc. of the 29th International Conference on Software Engineering (ICSE’07), Minneapolis, 2007.

Dangerfield, O., Ambardekar, P., Paluzzi, P., Card, D., Giblin, D., Defect Causal Analysis: A Report from the Field, Proceedings of International Conference of Software Quality, American Society for Quality Control, 1992.

Eckes, G., The Six Sigma Revolution: How General Electric and Others Turned Process Into Profits, John Wiley and Sons, 2000.

Endres, A., “An Analysis of Errors and Their Causes in Systems Programs", IEEE Transactions on Software Engineering, SE-1, 2, June 1975, pp. 140-149, 1975.

Fairley, R. E., Managing by the Numbers: a tutorial on quantitative measurement and control of software projects, in 'Proceedings of the 21st International Conference on Software Engineering, ICSE 99', IEEE Computer Society Press, Los Alamitos, CA, USA, pp. 677- 678, 1999.

Florac, A.W., Carleton A.D., Measuring the Software Process: Statistical Process Control for Software Process Improvement, Pearson Education, 1999.

Grady, R. B., Software Failure Analysis for High-Return Process Improvement Decisions, Hewlett-Packard Journal, 47 (4), 15 - 24, 1996.

Gras, J.J., End-to-End Defect Modeling, IEEE Software, 21(5), 98-100, 2004.

Hayes, J.H., Raphael, I., Holbrook, E.A., Pruett, D.M., A case history of International Space Station requirement faults, Proceedings of the11th IEEE International Conference on Engineering of Complex Computer Systems, Stanford University, California, 2006.

Hong, G., Xie, M., Shanmugan, P., A Statistical Method for Controlling Software Defect Detection Process, Computers and Industrial Engineering 37 (1-2), pp. 137-140, 1999.

IEEE, 1990, IEEE Standard Glossary of Software Engineering Terminology, Standard 610, IEEE Press.

ISO/IEC, ISO/IEC 12207: Information technology - Software life-cycle processes – Amendment 2, Geneve: ISO, Int. Org. for Standardization and the Int. Electrotechnical Commission, 2004.

Jacobs, J.C., van Moll, J.H., Krause, P.J., Kusters, R.J., Trienekens, J.J.M., Brombacher, A., Exploring Defect Causes in Products Developed by Virtual Teams, Journal on Information and Software Technology, 47( 6), 399 – 410, 2005.

Jacobs, J.C., van Moll, J.H., Krause, P.J., Kusters, R.J., Trienekens, J.J.M., Effects of Virtual Development on Product Quality: Exploring Defect Causes, Proceedings of the 11th Annual International Workshop on Software Technology and Engineering Practice (STEP’04), 2004.

Jalote, P., Agrawal, N., “Using Defect Analysis Feedback for Improving Quality and Productivity in Iterative Software Development”, (Invited paper, 3rd International Conference on Information and Communication Technology, ICICT, 2005.), pp. 701 – 713, Cairo, 2005.

Jantti, M., Toroi, T., Eerola, A., Difficulties in establishing a defect management process: A case study, Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) 4034 NCS, pp. 142-150, 2006.

Jones, C.L., A process-integrated approach to defect prevention, IBM Systems Journal, 24(2), 150-67, 1985.

Kalinowski, M., Travassos, G.H., Uma revisão sistemática a respeito de análise causal de defeitos de software, technical report available at , COPPE/UFRJ, 2008.

Kalinowski, M., Travassos, G.H., Card, D.N., Towards a Defect Prevention Based Process Improvement Approach, 34th EUROMICRO Conference, IEEE Computer Society, 2008.

Kalinowski, M., Defect Causal Analysis: An Opportunity for Product Focused Software Process Improvement, Invited paper at the V CICIS (Congreso Internacional de Computación y Ingenieria de Sistemas), Moquegua, Peru, 2007.

Keene, J., Managing Software Reliability and Support Costs, in 'Proceedings of the Leesburg Workshop on Reliability and Maintainability Computer-Aided Engineering in Concurrent Engineering', pp. 201-205, 1991.

Kelsey, R.B., Integrating a Defect Typology with Containment Metrics, ACM SIGSOFT Software Engineering Notes, 22(2), 64-67, 1997.

Kitchenham, B.A., Procedures for Performing Systematic Reviews, Joint Technical Report Keele University and National ICT Australia Ltd., 2004.

Krause, P., Freimut, B., Suryn, W., New Directions in Measurement for Software Quality Control, in 'Proc. of the 10th Int. Workshop on Soft. Tech. and Eng. Practice, STEP 2002', pp. 129-143, 2002.

Lederer, A.L., Prasad, J., A Causal Model for Software Cost Estimating Error, IEEE Transactions on Software Engineering, 24 (2), pp. 137-148, 1998.

Leszak, M., Perry, D., Stoll, D., A Case Study in Root Cause Defect Analysis, in 'International Conference on Software Engineering, ICSE 2000', pp. 428-437, 2000.

Leszak, M., Perry, D. E., Stoll, D., Classification and evaluation of defects in a project retrospective, Journal of Systems and Software, 61( 3), 173 - 187, 2002.

Li, Z., Tan, L., Wang, X., Lu, S., Zhou, Y., Zhai, C., Have things changed now? An empirical study of bug characteristics in modern open source software, Proceedings of ASID'06: 1st Workshop on Architectural and System Support for Improving Software Dependability, pp. 25-33, 2006.

Linger, R. C., Cleanroom Software Engineering for Zero-Defect Software, in 'ICSE '93: Proceedings of the 15th International Conference on Software Engineering', IEEE Computer Society Press, Los Alamitos, CA, USA, pp. 2-13, 1993.

Mays, R.G., Applications of Defect Prevention in Software Development, IEEE Journal on Selected Areas in Communications, Vol.8, No.2, pp. 164-168, February 1990.

Mays, R.G., Jones, C.L., Holloway, G.J., Studinski, D.P., Experiences with Defect Prevention, IBM Systems Journal, 29(1), pp. 4-32, 1990.

van Moll, J., Jacobs, J., Freimut, B., Trienekens, J., The Importance of Life Cycle Modeling to Defect Detection and Prevention, in '10th International Workshop on Software Technology and Engineering Practice, 2002. STEP 2002', pp. 144-155, 2002.

Nakajo, T. & Kume, H. (1991), A Case History Analysis of Software Error Cause-Effect Relationships, IEEE Transactions on Software Engineering, 17(8), 830-838, 1991.

Nakajo, T., Foolproofing and Quality Feedback: Keys of Process-Based Management, Proceedings of the Fifteenth Annual International Computer Software and Applications Conference, COMPSAC'91, IEEE Computer Society Press, 390-391, 1991.

Nakamura, T., Hochstein, L., Basili, V.R., Identifying Domain-Specific Defect Classes Using Inspections and Change History, Proceedings of the 2006 International Symposium on Empirical Software Engineering (ISESE), Sept. 21-22, Rio de Janeiro, Brazil. p. 346-355, 2006.

Oshana, R., Coyle, F.P., Implementing Cleanroom Software Engineering into a Mature CMM-Based Software Organization, Proceedings of the 19th International Conference on Software Engineering, ICSE’97, ACM Press, New York, NY, USA, pp. 572-573, 1997.

Paul, R. A., Bastani, F.; Yen, I., Challagulla, V. U., Defect-based Reliability Analysis for Mission-Critical Software, Proceedings of the IEEE Computer Society's International Computer Software and Applications Conference, pp. 439 - 444, 2000.

Philips, R.T., An approach to software causal analysis and defect extinction, in ‘Proceedings of the IEEE Globecom Conference, pp. 412-416, 1986.

Poppendieck, M., Poppendieck, T., Lean Software Development: An Agile Toolkit, Addison Wesley Professional, 2003.

Pratt, W. M., Experiences in the Application of Customer-Based Metrics in Improving Software Service Quality, Conf. Record of the International Conference on Communications, pp. 1459 - 1462, 1991.

Robitaille, D., Root Cause Analysis – Basic Tools and Techniques, Paton Press, 2004.

Rombach, D., Endres, A., A Handbook of Software and Systems Engineering – Empirical Observations, Laws and Theories, Pearson Addison Wesley, 2003.

Rosen, C., PLUNGE DA: A Case Study, ACM SIGSOFT Soft. Eng. Notes, 22(2), pp. 82-83, 1997. SEI, CMMI for Development (CMMI-DEV), Version 1.2, Technical report CMU/SEI-2006-TR-008. Pittsburgh, PA: Software Engineering Institute, Carnegie Mellon University, 2006.

Shull, F., Developing Techniques for Using Software Documents: A Series of Empirical Studies, Ph.D. thesis, University of Maryland, College Park, 1998.

SOFTEX, MPS.BR – Melhoria de Processo do Software Brasileiro – Guia Geral, version 1.2, June 2007, available at , 2007a.

SOFTEX (Rocha, A.R.C., Montoni, M.A., Souza, G.S., Kalinowski, M., Scalet, D.), MPS.BR – Melhoria de Processo do Software Brasileiro – Guia de Implementação do Nível A, version 1.0, June 2007, available at , 2007b.

Travassos, G.H., Shull, F., Carver, J., “Working with UML: A Software Design Process Based on Inspections for the Unified Modeling Language”, in: Advances in Computers, vol. 54, Academic Press, 2001.

Troster, J., Henshaw, J., Buss, E., Filtering for Quality, in 'Proceedings of the 1993 Conf. of the Centre for Advanced Studies on Collaborative Research, CASCON 93', IBM Press, pp. 429-449, 1993.

Vantine, W., Benfield, K., Pritts, D., Ballard, K., Evaluating and Incorporating New Age Software Technology for Identifying Systemic Root Causes, in: ‘Proceedings of the Joint ESA-NASA Space Flight Safety Conference, ESTEC’, ESA SP-486, Noordwijk (NL), pp. 369 - 376, 2002.

Walia, G., Carver, J., Philip, T., Requirements Error Abstraction and Classification: An Empirical Study, Proc. Int. Symposium on Empirical Soft. Eng. (ISESE), Sept. 21-22, Rio de Janeiro, Brazil, 2006.

Wang, Q., Jiang, N., Gou, L., Liu, X., Li, M., Wang, Y., BSR: A Statistic-based Approach for Establishing and Refining Software Process Performance Baseline, in 'ICSE '06: Proceeding of the 28th Int. Conf. on Software Engineering', ACM Press, New York, NY, USA, pp. 585-594., 2006.

Wheeler, D.J., Chambers, D.S., Understanding Statistical Process Control, 2ed, Knoxville, Tenn., SPC Press, 1992.

Wigglesworth, J., Surveys as a Method for Improving the Development Process, in 'CASCON '93: Proc. conf. of the Centre for Advanced Studies on Collaborative research', IBM Press, pp. 337-355, 1993.

Wohlin, C., Host, M., Ohlsson, M., Understanding the Sources of Software Defects: A Filtering Approach, in '8th Int. Workshop on Program Comprehension, 2000, IWPC 2000', pp. 9-17, 2000.
Como Citar

Selecione um Formato
KALINOWSKI, Marcos; TRAVASSOS, Guilherme H.; CARD, David N.. Guidance for Efficiently Implementing Defect Causal Analysis. In: SIMPÓSIO BRASILEIRO DE QUALIDADE DE SOFTWARE (SBQS), 7. , 2008, Florianópolis. Anais [...]. Porto Alegre: Sociedade Brasileira de Computação, 2008 . p. 139-155. DOI: