skip to main content
10.1145/3613372.3613390acmotherconferencesArticle/Chapter ViewAbstractPublication PagessbesConference Proceedingsconference-collections
research-article

Empirical evidence on technical challenges when adopting continuous practices

Published:25 September 2023Publication History

ABSTRACT

Context: Continuous Software Engineering offers several benefits like fast iterations, increased reliability, and shorter lead times to react to business needs through practices like Continuous Integration, Delivery, and Deployment, but these practices come with many adoption challenges, which may appear as social (hard to identify and overcome) or technical challenges (TCs). Since anticipating even a small amount of possible challenges can be overwhelming, there is value in detecting them as early as possible to make it cheaper, faster, and more feasible to adopt this set of practices. Objective: We summarize evidence about identifying TCs to adopt continuous deployment and related practices. Method: We performed a Rapid Review that analyzed 14 studies and performed a narrative summary to provide an evidence briefing to practitioners. Results: We identified 121 TCs and organized them into nine categories: architecture, building, code, deployment, development practices, operations, pipeline, security, and testing. TCs related to code, pipeline, and testing are the most prevalent in the reviewed literature, while operations and deployment are the least. Static analysis strategies are the most used for detecting TCs, accounting for 94 detection strategies out of 100. However, most of these static analyses resort to some parsing technique. Specifically, Abstract Syntax Tree is the most used, detecting 44 TCs. Conclusion: Most studies have mixed evidence about identifying TCs to adopt continuous practices - some focus on evaluating the efficiency of detection strategies. In contrast, others are more focused on their characterization. We also identified varying levels of efficiency when detecting TCs. Finally, we advise attention to interpreting the results based on the overall quality score.

References

  1. 2020. Guidelines for creating a Narrative Summary. Technical Report. Vanderbilt-Meharry Community Engaged Research Core. https://victr.vumc.org/wp-content/uploads/2020/01/Narrative-Summary-Guidelines_1.23.2020.pdfGoogle ScholarGoogle Scholar
  2. 2023. Replication Package. Zenodo. https://doi.org/10.5281/zenodo.7942306Google ScholarGoogle ScholarCross RefCross Ref
  3. Amine Barrak, Ellis E Eghan, Bram Adams, and Foutse Khomh. 2021. Why do builds fail?—A conceptual replication study. J. Syst. Softw. 177, 110939 (July 2021), 110939. https://doi.org/10.1016/j.jss.2021.110939Google ScholarGoogle ScholarCross RefCross Ref
  4. Bruno Cartaxo, Gustavo Pinto, and Sergio Soares. 2020. Rapid Reviews in Software Engineering. In Contemporary Empirical Methods in Software Engineering. Springer International Publishing, Cham, 357–384. https://doi.org/10.1007/978-3-030-32489-6_13Google ScholarGoogle ScholarCross RefCross Ref
  5. Lianping Chen. 2015. Continuous delivery: Huge benefits, but challenges too. IEEE software 32, 2 (2015), 50–54.Google ScholarGoogle Scholar
  6. Brian Fitzgerald and Klaas-Jan Stol. 2017. Continuous software engineering: A roadmap and agenda. Journal of Systems and Software 123 (2017), 176–189.Google ScholarGoogle ScholarCross RefCross Ref
  7. Keheliya Gallaba and Shane McIntosh. 2020. Use and misuse of continuous integration features: An empirical study of projects that (mis)use Travis CI. IEEE Trans. Softw. Eng. 46, 1 (Jan. 2020), 33–50. https://doi.org/10.1109/TSE.2018.2838131Google ScholarGoogle ScholarCross RefCross Ref
  8. Jordan Henkel, Christian Bird, Shuvendu K Lahiri, and Thomas Reps. 2020. Learning from, understanding, and supporting DevOps artifacts for docker. In Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering (Seoul South Korea). ACM, New York, NY, USA, 38–49. https://doi.org/10.1145/3377811.3380406Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. Adha Hrusto, Per Runeson, and Emelie Engström. 2021. Closing the feedback loop in DevOps through autonomous monitors in operations. SN Comput. Sci. 2, 6 (Nov. 2021), 447–. https://doi.org/10.1007/s42979-021-00826-yGoogle ScholarGoogle ScholarDigital LibraryDigital Library
  10. Jez Humble and David Farley. 2011. Continuous delivery reliable software releases through build, test, and Deployment Automation. Addison-Wesley. https://www.oreilly.com/library/view/continuous-delivery-reliable/9780321670250/Google ScholarGoogle Scholar
  11. Eero Laukkanen, Juha Itkonen, and Casper Lassenius. 2017. Problems, causes and solutions when adopting continuous delivery—A systematic literature review. Information and Software Technology 82 (2017), 55–79. https://doi.org/10.1016/j.infsof.2016.10.001Google ScholarGoogle ScholarCross RefCross Ref
  12. Evangelos Ntentos, Uwe Zdun, Konstantinos Plakidas, and Sebastian Geiger. 2021. Evaluating and improving microservice architecture conformance to architectural design decisions. In Service-Oriented Computing. Springer International Publishing, Cham, 188–203. https://doi.org/10.1007/978-3-030-91431-8_12Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. Akond Rahman, Effat Farhana, and Laurie Williams. 2020. The ‘as code’ activities: development anti-patterns for infrastructure as code. Empir. Softw. Eng. 25, 5 (Sept. 2020), 3430–3467. https://doi.org/10.1007/s10664-020-09841-8Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. Akond Rahman, Chris Parnin, and Laurie Williams. 2019. The seven sins: Security smells in infrastructure as code scripts. In 2019 IEEE/ACM 41st International Conference on Software Engineering (ICSE) (Montreal, QC, Canada). IEEE. https://doi.org/10.1109/ICSE.2019.00033Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. Akond Rahman, Md Rayhanur Rahman, Chris Parnin, and Laurie Williams. 2021. Security smells in Ansible and Chef scripts: A Replication Study. ACM Trans. Softw. Eng. Methodol. 30, 1 (Jan. 2021), 1–31. https://doi.org/10.1145/3408897Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. Zhongshan Ren, Wei Wang, Guoquan Wu, Chushu Gao, Wei Chen, Jun Wei, and Tao Huang. 2018. Migrating web applications from monolithic structure to microservices architecture. In Proceedings of the Tenth Asia-Pacific Symposium on Internetware (Beijing China). ACM, New York, NY, USA. https://doi.org/10.1145/3275219.3275230Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. Victor Vidigal Ribeiro, Daniela Soares Cruzes, and Guilherme Horta Travassos. 2021. Replication package - Moderator Factors of Software Security and Performance Verification. https://doi.org/10.5281/zenodo.5093800Google ScholarGoogle ScholarCross RefCross Ref
  18. Pilar Rodríguez, Alireza Haghighatkhah, Lucy Ellen Lwakatare, Susanna Teppola, Tanja Suomalainen, Juho Eskeli, Teemu Karvonen, Pasi Kuvaja, June M. Verner, and Markku Oivo. 2017. Continuous deployment of software intensive products and services: A systematic mapping study. Journal of Systems and Software 123 (1 2017), 263–291. https://doi.org/10.1016/j.jss.2015.12.015Google ScholarGoogle ScholarCross RefCross Ref
  19. Islem Saidani and Ali Ouni. 2021. Toward a smell-aware prediction model for CI build failures. In 2021 36th IEEE/ACM International Conference on Automated Software Engineering Workshops (ASEW) (Melbourne, Australia). IEEE. https://doi.org/10.1109/ASEW52652.2021.00017Google ScholarGoogle ScholarCross RefCross Ref
  20. Julian Schwarz, Andreas Steffens, and Horst Lichter. 2018. Code Smells in Infrastructure as Code. In 2018 11th International Conference on the Quality of Information and Communications Technology (QUATIC) (Coimbra). IEEE. https://doi.org/10.1109/QUATIC.2018.00040Google ScholarGoogle ScholarCross RefCross Ref
  21. Mojtaba Shahin, Muhammad Ali Babar, and Liming Zhu. 2017. Continuous integration, delivery and deployment: A systematic review on approaches, tools, challenges and practices. IEEE Access 5 (2017), 3909–3943. https://doi.org/10.1109/ACCESS.2017.2685629Google ScholarGoogle ScholarCross RefCross Ref
  22. Carmine Vassallo, Sebastian Proksch, Harald C Gall, and Massimiliano Di Penta. 2019. Automated reporting of anti-patterns and decay in continuous integration. In 2019 IEEE/ACM 41st International Conference on Software Engineering (ICSE) (Montreal, QC, Canada). IEEE. https://doi.org/10.1109/ICSE.2019.00028Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. Carmine Vassallo, Sebastian Proksch, Anna Jancso, Harald C Gall, and Massimiliano Di Penta. 2020. Configuration smells in continuous delivery pipelines: a linter and a six-month study on GitLab. In Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering (Virtual Event USA). ACM, New York, NY, USA. https://doi.org/10.1145/3368089.3409709Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. Fiorella Zampetti, Carmine Vassallo, Sebastiano Panichella, Gerardo Canfora, Harald Gall, and Massimiliano Di Penta. 2020. An empirical characterization of bad practices in continuous integration. Empir. Softw. Eng. 25, 2 (March 2020), 1095–1135. https://doi.org/10.1007/s10664-019-09785-8Google ScholarGoogle ScholarCross RefCross Ref

Index Terms

  1. Empirical evidence on technical challenges when adopting continuous practices

        Recommendations

        Comments

        Login options

        Check if you have access through your login credentials or your institution to get full access on this article.

        Sign in
        • Published in

          cover image ACM Other conferences
          SBES '23: Proceedings of the XXXVII Brazilian Symposium on Software Engineering
          September 2023
          570 pages
          ISBN:9798400707872
          DOI:10.1145/3613372

          Copyright © 2023 ACM

          Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

          Publisher

          Association for Computing Machinery

          New York, NY, United States

          Publication History

          • Published: 25 September 2023

          Permissions

          Request permissions about this article.

          Request Permissions

          Check for updates

          Qualifiers

          • research-article
          • Research
          • Refereed limited

          Acceptance Rates

          Overall Acceptance Rate147of427submissions,34%
        • Article Metrics

          • Downloads (Last 12 months)49
          • Downloads (Last 6 weeks)5

          Other Metrics

        PDF Format

        View or Download as a PDF file.

        PDF

        eReader

        View online with eReader.

        eReader

        HTML Format

        View this article in HTML Format .

        View HTML Format