Tool Support for Spectrum-based Fault Localization during Continuous Integration
Abstract
Program debugging is one of the most time consuming activities carried out by developers because, in large extent, it is performed in ad hoc fashion. Significant research efforts have been directed to the development of new debugging techniques, in special, fault localization techniques. Spectrum-based Fault Localization (SBFL) is a debugging technique that has been experimented on programs similar to those developed in industry with promising results. To support the automated application of SBFL, tools have been developed, yet in the academic realm. Few of them, though, aim to support SBFL in the context of the Continuous Integration (CI) practice. We present the Jaguar Portal platform, designed and developed to receive, store, and provide access to SBFL information in CI environments. The platform encampsulates the Jaguar 2 tool, which is responsible for executing the SBFL technique during the build process in the CI pipeline on GitHub. Furthermore, it provides web interfaces that display SBFL analyses with code snippets and suspicious line markings.We believe Jaguar Portal is a step towards the adoption of SBFL techniques in industrial settings. Demo video: https://www.youtube.com/@JaguarPortalSBFL.
Keywords:
Debugging, Fault localization, Spectra, Continuous Integration
References
Rui Abreu, Peter Zoeteweij, Rob Golsteijn, and Arjan J.C. van Gemund. 2009. A practical evaluation of spectrum-based fault localization. Journal of Systems and Software 82, 11 (2009), 1780–1792. SI: TAIC PART 2007 and MUTATION 2007.
Rui Abreu, Peter Zoeteweij, and Arjan JC Van Gemund. 2007. On the accuracy of spectrum-based fault localization. In Testing: Academic and industrial conference practice and research techniques-MUTATION (TAICPART-MUTATION 2007). IEEE, 89–98.
Roberto Paulo Andrioli de Araujo. 2014. Scalable data-flow testing. Master’s thesis. Escola de Artes, Ciências e Humanidades, Universidade de São Paulo.
José Campos, André Riboira, Alexandre Perez, and Rui Abreu. 2012. GZoltar: an eclipse plug-in for testing and debugging. In 2012 Proceedings of the 27th IEEE/ACM International Conference on Automated Software Engineering. 378–381.
Marcos L Chaim, José C Maldonado, and Mario Jino. 2004. A debugging strategy based on the requirements of testing. Journal of Software Maintenance and Evolution: Research and Practice 16, 4-5 (2004), 277–308.
Eclipse Contributors. 2023. JaCoCo. [link]
Fred D Davis. 1989. Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS quarterly (1989), 319–340.
Fred D Davis. 1993. User acceptance of information technology: system characteristics, user perceptions and behavioral impacts. International journal of man-machine studies 38, 3 (1993), 475–487.
Higor Amario de Souza. 2018. Assessment of spectrum-based fault localization for practical use. Ph.D. Dissertation. Instituto de Matemática e Estatística (IME), Universidade de São Paulo (USP).
Paul M Duvall, Steve Matyas, and Andrew Glover. 2007. Continuous integration: improving software quality and reducing risk. Pearson Education.
Eclipse Foundation. 2023. Eclipse. [link]
.NET Foundation. 2023. NUnit. [link]
.NET Foundation. 2023. xUnit. [link]
Martin Fowler and Matthew Foemmel. 2006. Continuous integration. [link]
Mary Jean Harrold, Gregg Rothermel, Rui Wu, and Liu Yi. 1998. An empirical investigation of program spectra. In Proceedings of the 1998 ACM SIGPLANSIGSOFT workshop on Program analysis for software tools and engineering. 83–90.
Q. Idrees Sarhan, A. Szatmari, R. Toth, and A. Beszedes. 2021. CharmFL: A Fault Localization Tool for Python. Proceedings - IEEE 21st International Working Conference on Source Code Analysis and Manipulation, SCAM 2021 (2021), 114–119.
GitHub Inc. 2023. GitHub. [link]
Tom Janssen, Rui Abreu, and Arjan JC Van Gemund. 2009. Zoltar: a spectrumbased fault localization tool. In Proceedings of the 2009 ESEC/FSE workshop on Software integration and evolution@ runtime. 23–30.
JetBrains. 2023. PyCharm. [link]
James A Jones, Mary Jean Harrold, and John Stasko. 2002. Visualization of test information to assist fault localization. In Proceedings of the 24th International Conference on Software Engineering. ICSE 2002. IEEE, 467–477.
René Just, Darioush Jalali, and Michael D. Ernst. 2014. Defects4J: A Database of Existing Faults to Enable Controlled Testing Studies for Java Programs. In Proceedings of the 2014 International Symposium on Software Testing and Analysis (San Jose, CA, USA) (ISSTA 2014). Association for Computing Machinery, New York, NY, USA, 437–440.
Microsoft. 2023. Visual Studio Code. [link]
Lee Naish, Hua Jie Lee, and Kotagiri Ramamohanarao. 2009. Spectral Debugging with Weights and Incremental Ranking. In 2009 16th Asia-Pacific Software Engineering Conference. 168–175.
Lee Naish, Hua Jie Lee, and Kotagiri Ramamohanarao. 2011. A model for spectrabased software diagnosis. ACM Transactions on software engineering and methodology (TOSEM) 20, 3 (2011), 1–32.
Ali Ouni, Islem Saidani, Eman Alomar, and Mohamed Wiem Mkaouer. 2023. An Empirical Study on Continuous Integration Trends, Topics and Challenges in Stack Overflow. In Proceedings of the 27th International Conference on Evaluation and Assessment in Software Engineering (Oulu, Finland) (EASE ’23). Association for Computing Machinery, New York, NY, USA, 141–151.
Hugo Paiva, Jose Campos, and Rui Abreu. 2025. GZoltarAction: A Fault Localization Bot for GitHub Repositories. In 2025 IEEE/ACM International Workshop on Bots in Software Engineering (BotSE). IEEE Computer Society, 38–42.
PostgresSQL. 2023. PostgresSQL. [link]
Apache Maven Project. 2023. Apache Maven is a software project management and comprehension tool. [link]
Sandra Rapps and Elaine J. Weyuker. 1985. Selecting Software Test Data Using Data Flow Information. IEEE Trans. Softw. Eng. 11, 4 (April 1985), 367–375. DOI: 10.1109/TSE.1985.232226
Henrique L. Ribeiro, Roberto P. A. de Araujo, Marcos L. Chaim, Higor A. de Souza, and Fabio Kon. 2018. Jaguar: A Spectrum-Based Fault Localization Tool for Real-World Software. In 2018 IEEE 11th International Conference on Software Testing, Verification and Validation (ICST). 404–409.
Software Analysis & Experimentation Group SAEG. 2023. BA-DUA. [link]
Software Analysis & Experimentation Group SAEG. 2023. Jaguar 2. [link]
André Silva, Matias Martinez, Benjamin Danglot, Davide Ginelli, and Martin Monperrus. 2021. FLACOCO: Fault Localization for Java based on Industry-grade Coverage. arXiv preprint arXiv:2111.12513 (2021).
Erickson Lima Barbosa Silva. 2024. Localização de Defeitos Baseada em Espectro e a Prática de Integração Contínua. Master’s thesis. Escola de Artes, Ciências e Humanidades, Universidade de São Paulo.
The JUnit Team. 2023. JUnit. [link]
Nan Wang, Zheng Zheng, Zhenyu Zhang, and Cheng Chen. 2015. FLAVS: A Fault Localization Add-In for Visual Studio. In 2015 IEEE/ACM 1st International Workshop on Complex Faults and Failures in Large Software Systems (COUFLESS). 1–6.
W Eric Wong, Vidroha Debroy, and Byoungju Choi. 2010. A family of code coverage-based heuristics for effective fault localization. Journal of Systems and Software 83, 2 (2010), 188–208.
Jian Xu, Zhenyu Zhang, Wing Kwong Chan, TH Tse, and Shanping Li. 2013. A general noise-reduction framework for fault localization of Java programs. Information and Software Technology 55, 5 (2013), 880–896.
Abubakar Zakari, Sai Peck Lee, Khubaib Amjad Alam, and Rodina Ahmad. 2019. Software fault localisation: a systematic mapping study. IET Software 13, 1 (2019), 60–74.
Rui Abreu, Peter Zoeteweij, and Arjan JC Van Gemund. 2007. On the accuracy of spectrum-based fault localization. In Testing: Academic and industrial conference practice and research techniques-MUTATION (TAICPART-MUTATION 2007). IEEE, 89–98.
Roberto Paulo Andrioli de Araujo. 2014. Scalable data-flow testing. Master’s thesis. Escola de Artes, Ciências e Humanidades, Universidade de São Paulo.
José Campos, André Riboira, Alexandre Perez, and Rui Abreu. 2012. GZoltar: an eclipse plug-in for testing and debugging. In 2012 Proceedings of the 27th IEEE/ACM International Conference on Automated Software Engineering. 378–381.
Marcos L Chaim, José C Maldonado, and Mario Jino. 2004. A debugging strategy based on the requirements of testing. Journal of Software Maintenance and Evolution: Research and Practice 16, 4-5 (2004), 277–308.
Eclipse Contributors. 2023. JaCoCo. [link]
Fred D Davis. 1989. Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS quarterly (1989), 319–340.
Fred D Davis. 1993. User acceptance of information technology: system characteristics, user perceptions and behavioral impacts. International journal of man-machine studies 38, 3 (1993), 475–487.
Higor Amario de Souza. 2018. Assessment of spectrum-based fault localization for practical use. Ph.D. Dissertation. Instituto de Matemática e Estatística (IME), Universidade de São Paulo (USP).
Paul M Duvall, Steve Matyas, and Andrew Glover. 2007. Continuous integration: improving software quality and reducing risk. Pearson Education.
Eclipse Foundation. 2023. Eclipse. [link]
.NET Foundation. 2023. NUnit. [link]
.NET Foundation. 2023. xUnit. [link]
Martin Fowler and Matthew Foemmel. 2006. Continuous integration. [link]
Mary Jean Harrold, Gregg Rothermel, Rui Wu, and Liu Yi. 1998. An empirical investigation of program spectra. In Proceedings of the 1998 ACM SIGPLANSIGSOFT workshop on Program analysis for software tools and engineering. 83–90.
Q. Idrees Sarhan, A. Szatmari, R. Toth, and A. Beszedes. 2021. CharmFL: A Fault Localization Tool for Python. Proceedings - IEEE 21st International Working Conference on Source Code Analysis and Manipulation, SCAM 2021 (2021), 114–119.
GitHub Inc. 2023. GitHub. [link]
Tom Janssen, Rui Abreu, and Arjan JC Van Gemund. 2009. Zoltar: a spectrumbased fault localization tool. In Proceedings of the 2009 ESEC/FSE workshop on Software integration and evolution@ runtime. 23–30.
JetBrains. 2023. PyCharm. [link]
James A Jones, Mary Jean Harrold, and John Stasko. 2002. Visualization of test information to assist fault localization. In Proceedings of the 24th International Conference on Software Engineering. ICSE 2002. IEEE, 467–477.
René Just, Darioush Jalali, and Michael D. Ernst. 2014. Defects4J: A Database of Existing Faults to Enable Controlled Testing Studies for Java Programs. In Proceedings of the 2014 International Symposium on Software Testing and Analysis (San Jose, CA, USA) (ISSTA 2014). Association for Computing Machinery, New York, NY, USA, 437–440.
Microsoft. 2023. Visual Studio Code. [link]
Lee Naish, Hua Jie Lee, and Kotagiri Ramamohanarao. 2009. Spectral Debugging with Weights and Incremental Ranking. In 2009 16th Asia-Pacific Software Engineering Conference. 168–175.
Lee Naish, Hua Jie Lee, and Kotagiri Ramamohanarao. 2011. A model for spectrabased software diagnosis. ACM Transactions on software engineering and methodology (TOSEM) 20, 3 (2011), 1–32.
Ali Ouni, Islem Saidani, Eman Alomar, and Mohamed Wiem Mkaouer. 2023. An Empirical Study on Continuous Integration Trends, Topics and Challenges in Stack Overflow. In Proceedings of the 27th International Conference on Evaluation and Assessment in Software Engineering (Oulu, Finland) (EASE ’23). Association for Computing Machinery, New York, NY, USA, 141–151.
Hugo Paiva, Jose Campos, and Rui Abreu. 2025. GZoltarAction: A Fault Localization Bot for GitHub Repositories. In 2025 IEEE/ACM International Workshop on Bots in Software Engineering (BotSE). IEEE Computer Society, 38–42.
PostgresSQL. 2023. PostgresSQL. [link]
Apache Maven Project. 2023. Apache Maven is a software project management and comprehension tool. [link]
Sandra Rapps and Elaine J. Weyuker. 1985. Selecting Software Test Data Using Data Flow Information. IEEE Trans. Softw. Eng. 11, 4 (April 1985), 367–375. DOI: 10.1109/TSE.1985.232226
Henrique L. Ribeiro, Roberto P. A. de Araujo, Marcos L. Chaim, Higor A. de Souza, and Fabio Kon. 2018. Jaguar: A Spectrum-Based Fault Localization Tool for Real-World Software. In 2018 IEEE 11th International Conference on Software Testing, Verification and Validation (ICST). 404–409.
Software Analysis & Experimentation Group SAEG. 2023. BA-DUA. [link]
Software Analysis & Experimentation Group SAEG. 2023. Jaguar 2. [link]
André Silva, Matias Martinez, Benjamin Danglot, Davide Ginelli, and Martin Monperrus. 2021. FLACOCO: Fault Localization for Java based on Industry-grade Coverage. arXiv preprint arXiv:2111.12513 (2021).
Erickson Lima Barbosa Silva. 2024. Localização de Defeitos Baseada em Espectro e a Prática de Integração Contínua. Master’s thesis. Escola de Artes, Ciências e Humanidades, Universidade de São Paulo.
The JUnit Team. 2023. JUnit. [link]
Nan Wang, Zheng Zheng, Zhenyu Zhang, and Cheng Chen. 2015. FLAVS: A Fault Localization Add-In for Visual Studio. In 2015 IEEE/ACM 1st International Workshop on Complex Faults and Failures in Large Software Systems (COUFLESS). 1–6.
W Eric Wong, Vidroha Debroy, and Byoungju Choi. 2010. A family of code coverage-based heuristics for effective fault localization. Journal of Systems and Software 83, 2 (2010), 188–208.
Jian Xu, Zhenyu Zhang, Wing Kwong Chan, TH Tse, and Shanping Li. 2013. A general noise-reduction framework for fault localization of Java programs. Information and Software Technology 55, 5 (2013), 880–896.
Abubakar Zakari, Sai Peck Lee, Khubaib Amjad Alam, and Rodina Ahmad. 2019. Software fault localisation: a systematic mapping study. IET Software 13, 1 (2019), 60–74.
Published
2025-09-22
How to Cite
SILVA, Erickson Lima Barbosa da; ARAUJO, Roberto Paulo Andrioli; CHAIM, Marcos Lordello.
Tool Support for Spectrum-based Fault Localization during Continuous Integration. In: BRAZILIAN SYMPOSIUM ON SOFTWARE ENGINEERING (SBES), 39. , 2025, Recife/PE.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2025
.
p. 942-948.
ISSN 2833-0633.
DOI: https://doi.org/10.5753/sbes.2025.11252.
