Empirical investigation of the influence of continuous integration bad practices on software quality
Resumo
Continuous Integration (CI) is a powerful tool to leverage software development in a safe, reliable, and efficient way. However, in day-to-day of software development, bad habits can arise in working with CI, which can make CI not reach its full potential in projects. These bad habits we call CI bad practices. This paper presented an exploratory study on closed-source projects to investigate how CI bad practices can affect software quality. We observe (1) the impact on the quality internal attributes after the implantation of CI, (2) the evolution of software quality indicators over time, and (3) the bad practices considered most harmful to the software quality by the development teams. Our results mean that projects affected by CI bad practices will not necessarily have their quality impaired. However, this does not mean that quality will be maintained or improved over time. Our analysis of the quality indicators has allowed us to observe that, generally, they have remained stable, and the number of quality issues reported has not decreased considerably either. Finally, the quality indicator most affected by the CI bad practices was maintainability, followed by reliability and security. In addition, the vast majority of the bad practices analyzed were classified as having a medium level of effort for resolution. In this sense, we also classify bad practices according to the level of effort/impact on the quality that can help prioritize their resolution.
Referências
K.K. Aggarwal, Y. Singh, and J.K. Chhabra. 2002. An integrated measure of software maintainability. In Annual Reliability and Maintainability Symposium. 2002 Proceedings (Cat. No.02CH37318). 235-241. https://doi.org/10.1109/RAMS.2002.981648
Moritz Beller, Georgios Gousios, and Andy Zaidman. 2017. Oops, my tests broke the build: An explorative analysis of Travis CI with GitHub. In 2017 IEEE/ACM 14th International Conference on Mining Software Repositories (MSR). IEEE.
FP Brooks. 1978. The Mythical Man-Month: Essays on Softw. 1st.
Shyam R Chidamber and Chris F Kemerer. 1994. A metrics suite for object oriented design. IEEE Trans. Softw. Eng. 20, 6 (1994), 476-493.
Giuseppe Destefanis, Steve Counsell, Giulio Concas, and Roberto Tonelli. 2014. Software metrics in agile software: An empirical study. In International Conference on Agile Software Development. Springer, 157-170.
John Downs, John Hosking, and Beryl Plimmer. 2010. Status communication in agile software teams: A case study. In 2010 Fifth International Conference on Software Engineering Advances. IEEE.
PM Duvall. 2018. Continuous Delivery Patterns and AntiPatterns in the Software LifeCycle. WWW], Available (accessed on 25.7. 2022): https://dzone.com/refcardz/continuous-delivery-patterns (2018).
Paul M Duvall. 2010. Continuous Integration: Patterns and Anti-Patterns. DZone, Incorporated.
Omar Elazhary, Colin Werner, Ze Shi Li, Derek Lowlind, Neil A. Ernst, and Margaret-Anne Storey. 2021. Uncovering the Benefits and Challenges of Continuous Integration Practices. IEEE Transactions on Software Engineering (2021), 1-1. https://doi.org/10.1109/TSE.2021.3064953
Wagner Felidré, Leonardo Furtado, Daniel A da Costa, Bruno Cartaxo, and Gustavo Pinto. 2019. Continuous Integration Theater. In 2019 ACM/IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM).
David Goodman and Michael Elbaz. 2008. "It's Not the Pants, it's the People in the Pants" Learnings from the Gap Agile Transformation What Worked, How We Did it, and What Still Puzzles Us. In Agile 2008 Conference. IEEE.
Mark Lorenz and Jeff Kidd. 1994. Object-oriented software metrics: a practical guide. Prentice-Hall, Inc.
Thomas J McCabe. 1976. A complexity measure. IEEE Trans. Softw. Eng. 4 (1976), 308-320.
G. McGraw. 2004. Software security. IEEE Security Privacy 2, 2 (2004), 80-83. https://doi.org/10.1109/MSECP.2004.1281254
Ade Miller. 2008. A hundred days of continuous integration. In Agile 2008 conference. IEEE.
Hoang Pham. 2000. Software reliability. Springer Science & Business Media.
Mojtaba Shahin, Muhammad Ali Babar, and Liming Zhu. 2017. Continuous integration, delivery and deployment: a systematic review on approaches, tools, challenges and practices. IEEE Access (2017).
Ruben Blenicio Tavares Silva and Carla I. M. Bezerra. 2020. Analyzing Continuous Integration Bad Practices in Closed-Source Projects: An Initial Study. In Proceedings of the 34th Brazilian Symposium on Software Engineering (Natal, Brazil) (SBES '20). Association for Computing Machinery, New York, NY, USA, 642-647. https://doi.org/10.1145/3422392.3422474
Eliezio Soares, Gustavo Sizilio, Jadson Santos, Daniel Alencar da Costa, and Uirá Kulesza. 2022. The effects of continuous integration on software development: a systematic literature review. Empirical Software Engineering 27, 3 (2022), 1-61.
Daniel Ståhl, Torvald Mårtensson, and Jan Bosch. 2017. The continuity of continuous integration: Correlations and consequences. Journal of Systems and Software (2017).
Bogdan Vasilescu, Yue Yu, Huaimin Wang, Premkumar Devanbu, and Vladimir Filkov. 2015. Quality and productivity outcomes relating to continuous integration in GitHub. In Proceedings of the 2015 10th Joint Meeting on Foundations of Software Engineering.
Carmine Vassallo, Fabio Palomba, Alberto Bacchelli, and Harald C Gall. 2018. Continuous code quality: are we (really) doing that?. In Proceedings of the 33rd ACM/IEEE International Conference on Automated Software Engineering.
Fiorella Zampetti, Carmine Vassallo, Sebastiano Panichella, Gerardo Canfora, Harald Gall, and Massimiliano Di Penta. 2020. An empirical characterization of bad practices in continuous integration. Empirical Software Engineering (2020).
