How do Agile Organizations Manage Risks: An Analysis of the State of Practice in Brazil
Resumo
The use of agile methods tends to keep risks under control in software projects, due to their inherent characteristics of small increments, work visibility and expectation management. Thus, explicit risk management in agile projects has often been neglected, as the use of agile methods, with a focus on rapid value delivery, tends to lead to implicit risk management. However, software projects that use agile methods also can fail and implicit risk management may often not be sufficient for certain contexts. This has sparked research interest in the possible need for explicit risk management in software organizations that use agile methods. Motivated by the lack of information about risk management in agile software development contexts, in this work a comprehensive survey is carried out to understand how software development organizations that use agile methods are managing risk. We conducted an online survey with a statistically significant sample of 273 agile professionals in Brazil. Our findings indicate that although most organizations engage in some form of explicit risk management, a significant proportion consider agile methods insufficient for comprehensive risk mitigation. We also observed a set of 15 explicit risk management practices in agile contexts, with some agile ceremonies, notably daily and sprint planning meetings, emerging as conducive to the integration of explicit risk management practices. We also verified a statistically relevant association between these agile ceremonies and risk management processes.
Palavras-chave:
risk management, agile, practices, survey
Referências
Aalaa Albadarneh, Israa Albadarneh, and Abdallah Qusef. 2015. Risk management in Agile software development: A comparative study. In 2015 IEEE Jordan Conference on Applied Electrical Engineering and Computing Technologies (AEECT). 1–6. DOI: 10.1109/AEECT.2015.7360573
Associação Brasileira das Empresas de Software. 2021. Mercado Brasileiro de Software – Panorama e Tendências 2021. [link]
Andrea E Berndt. 2020. Sampling methods. Journal of Human Lactation 36, 2 (2020), 224–226.
Barry Boehm. 1989. Software risk management. In ESEC ’89, C. Ghezzi and J. A. McDermid (Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg, 1–19.
Katarína Buganová and Jana Šimíčková. 2019. Risk management in traditional and agile project management. Transportation Research Procedia 40 (2019), 986–993. DOI: 10.1016/j.trpro.2019.07.138 TRANSCOM 2019 13th International Scientific Conference on Sustainable, Modern and Safe Transport.
Juliette Michelle Parada Carvallo, Hanna Oktaba, and Elsa Ramírez Hernández. 2018. Risk Assessment Forum. In 2018 6th International Conference in Software Engineering Research and Innovation (CONISOFT). 160–164. DOI: 10.1109/CONISOFT.2018.8645949
Mauricio Concha, Marcello Visconti, and Hernán Astudillo. 2007. Agile Commitments: Enhancing Business Risk Management in Agile Development Projects. In Agile Processes in Software Engineering and Extreme Programming, Giulio Concas, Ernesto Damiani, Marco Scotto, and Giancarlo Succi (Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg, 149–152.
Lee J Cronbach. 1951. Coefficient alpha and the internal structure of tests. psychometrika 16, 3 (1951), 297–334.
Portia Crowe, Ali Mostashari, Mo Mansouri, and Robert Cloutier. 2009. 9.2.1 Reference Framework and Model for Integration of Risk Management in Agile Systems Engineering Lifecycle of the Defense Acquisition Management Framework. INCOSE International Symposium 19, 1 (2009), 1391–1405. DOI: 10.1002/j.2334-5837.2009.tb01022.x
Saru Dhir, Deepak Kumar, and V. Singh. 2019. Success and Failure Factors that Impact on Project Implementation Using Agile Software Development Methodology. 647–654. DOI: 10.1007/978-981-10-8848-3_62
Digital.ai. 2021. 15th annual state of agile report. Digital.ai.
Vasile Dorca, Radu Munteanu, Sorin Popescu, Adrian Chioreanu, and Claudius Peleskei. 2016. Agile approach with Kanban in information security risk management. In 2016 IEEE International Conference on Automation, Quality and Testing, Robotics (AQTR). 1–6. DOI: 10.1109/AQTR.2016.7501278
Abdelrafe Elzamly, Burairah Hussin, and Norhaziah Md Salleh. 2016. Top Fifty Software Risk Factors and the Best Thirty Risk Management Techniques in Software Development Lifecycle for Successful Software Projects. International Journal of Hybrid Information Technology 9 (06 2016), 11–32. DOI: 10.14257/ijhit.2016.9.6.02
Priscila Ferreira de Araújo Lima, Maria Crema, and Chiara Verbano. 2020. Risk management in SMEs: A systematic literature review and future directions. European Management Journal 38, 1 (2020), 78–94. DOI: 10.1016/j.emj.2019.06.005
Fernando Vedoin Garcia, Jean Hauck, and Fernanda Narloch Rizzo Hahn. 2022. Managing Risks in Agile Methods: a Systematic Literature Mapping. In Proceedings of the 34th International Conference on Software Engineering and Knowledge Engineering. DOI: 10.18293/SEKE2022-123
Muhammad Hammad and Irum Inayat. 2018. Integrating Risk Management in Scrum Framework. In 2018 International Conference on Frontiers of Information Technology (FIT). 158–163. DOI: 10.1109/FIT.2018.00035
Muhammad Hammad, Irum Inayat, and Maryam Zahid. 2019. Risk Management in Agile Software Development: A Survey. In 2019 International Conference on Frontiers of Information Technology (FIT). 162–1624. DOI: 10.1109/FIT47737.2019.00039
Faisal Hayat, Ammar Ur Rehman, Khawaja Sarmad Arif, Kanwal Wahab, and Muhammad Abbas. 2019. The Influence of Agile Methodology (Scrum) on Software Project Management. In 2019 20th IEEE/ACIS International Conference on Software Engineering, Artificial Intelligence, Networking and Parallel/Distributed Computing (SNPD). 145–149. DOI: 10.1109/SNPD.2019.8935813
Juhani Iivari and Netta Iivari. 2011. The relationship between organizational culture and the deployment of agile methods. Information and Software Technology 53 (05 2011), 509–520. DOI: 10.1016/j.infsof.2010.10.008
Shareeful Islam, Haris Mouratidis, and EdgarWeippl. 2014. An empirical study on the implementation and evaluation of a goal-driven software development risk management model. Information and Software Technology 56 (02 2014), 117–133. DOI: 10.1016/j.infsof.2013.06.003
ISO 14971:2019 2019. Medical devices – Application of risk management to medical devices. Standard. International Organization for Standardization.
ISO 31000:2009 2009. Risk Management – Principles and guidelines. Standard. International Organization for Standardization.
ISO 62304:2006 2006. Medical device software – Software life cycle processes. Standard. International Organization for Standardization.
ISO 80001:2010 2010. Application of risk management for IT-networks incorporating medical devices. Standard. International Organization for Standardization.
Shahedul Huq Khandkar. 2009. Open coding. University of Calgary 23 (2009), 2009.
Diana Kirk and Ewan Tempero. 2006. Identifying Risks in XP Projects through Process Modelling. In Proceedings of the Australian Software Engineering Conference (ASWEC ’06). IEEE Computer Society, USA, 411–420. DOI: 10.1109/ASWEC.2006.31
A. Koutsoyiannis. 1977. Theory of Econometrics. Open Journal of Statistics (1977).
Jefferson Seide Molléri, Kai Petersen, and Emilia Mendes. 2020. An empirically evaluated checklist for surveys in software engineering. Information and Software Technology 119 (2020), 106240. DOI: 10.1016/j.infsof.2019.106240
Jaana Nyfjord and Mira Kajko-Mattsson. 2007. Commonalities in Risk Management and Agile Process Models. International Conference on Software Engineering Advances (ICSEA 2007) (2007), 18–18. [link]
Edzreena Edza Odzaly, Des Greer, and Darryl Stewart. 2018. Agile risk management using software agents. Journal of Ambient Intelligence and Humanized Computing 9 (2018), 823–841.
Alain Pinsonneault and Kenneth Kraemer. 1993. Survey Research Methodology in Management Information Systems: An Assessment. Journal of Management Information Systems 10, 2 (1993), 75–105. DOI: 10.1080/07421222.1993.11518001 arXiv: [link]
Project Management Institute PMI. 2017. A Guide to the Project Management Body of Knowledge (PMBOK® Guide) (6 ed.). Project Management Institute.
T. Punter, M. Ciolkowski, B. Freimut, and I. John. 2003. Conducting on-line surveys in software engineering. In 2003 International Symposium on Empirical Software Engineering, 2003. ISESE 2003. Proceedings. 80–88. DOI: 10.1109/ISESE.2003.1237967
Sandra L. Ramírez-Mora and Hanna Oktaba. 2017. Productivity in Agile Software Development: A Systematic Mapping Study. In 2017 5th International Conference in Software Engineering Research and Innovation (CONISOFT). 44–53. DOI: 10.1109/CONISOFT.2017.00013
Lucio Ribeiro, Cristine Gusmao, Wilmar Feijo, and Vicente Bezerra. 2009. A case study for the implementation of an agile risk management process in multiple projects environments. In PICMET ’09 - 2009 Portland International Conference on Management of Engineering Technology. 1396–1404. DOI: 10.1109/PICMET.2009.5262002
Rogério Salgado. 2021. Brasil já é o 4º maior mercado do LinkedIn no planeta. [link] Acessado em 14/09/2023.
Eva-Maria Schön, Dirk Radtke, and Christian Jordan. 2020. Improving Risk Management in a Scaled Agile Environment. In Agile Processes in Software Engineering and Extreme Programming. Springer International Publishing, Cham, 132–141.
André Sousa, Joao Pascoal Faria, and Joao Mendes-Moreira. 2021. An Analysis of the State of the Art of Machine Learning for Risk Assessment in Software Projects. In Proceedings of the 33rd International Conference on Software Engineering and Knowledge Engineering, SEKE. 1–10.
Anselm L Strauss and Juliet Corbin. 2004. Open coding. Social research methods: A reader (2004), 303–306.
Kevin Suryaatmaja, Dermawan Wibisono, Achmad Ghazali, and Rachma Fitriati. 2020. Uncovering the failure of Agile framework implementation using SSMbased action research. Palgrave Communications 6 (01 2020), 8. DOI: 10.1057/s41599-019-0384-9
Keith S. Taber. 2017. The Use of Cronbach’s Alpha When Developing and Reporting Research Instruments in Science Education. Research in Science Education 48, 6 (June 2017), 1273–1296. DOI: 10.1007/s11165-016-9602-2
Maureen Tanner and Ulrich von Willingh. 2014. Factors Leading to the Success and Failure of Agile Projects Implemented in Traditionally Waterfall Environments. In Human Capital without Borders: Knowledge and Learning for the Quality of Life. Portoroz, Slovenia, 693–701. [link]
Breno Gontijo Tavares, Carlos Eduardo Sanches da Silva, and Adler Diniz de Souza. 2019. Practices to Improve Risk Management in Agile Projects. International Journal of Software Engineering and Knowledge Engineering 29, 03 (2019), 381–399. DOI: 10.1142/S0218194019500165 arXiv: [link]
Breno Gontijo Tavares, Mark Keil, Carlos Eduardo Sanches da Silva, and Adler Diniz de Souza. 2020. A risk management tool for agile software development. Journal of Computer Information Systems 61, 6 (2020), 561–570.
Fernando Vedoin Garcia, Jean Hauck, and Adriano Borgatto. 2024. An Analysis of the State of Practice of Risk Management in Agile Organizations in Brazil - Supplementary Material. Mendeley Data, V2. DOI: 10.17632/rdy3s9vzsb.2
Marcel Vieira, Jean C. R. Hauck, and Santiago Matalonga. 2020. How Explicit Risk Management is Being Integrated Into Agile Methods: Results From a Systematic Literature Mapping. In 19th Brazilian Symposium on Software Quality (São Luís, Brazil) (SBQS’20). Article 15, 10 pages. DOI: 10.1145/3439961.3439976
Zuzana Virglerova, Muhammad Asif Khan, Raimonda Martinkute-Kauliene, and Sandor Kovacs. 2020. The internationalization of smes in Central Europe and its impact on their methods of Risk Management. www.amfiteatrueconomic.ro 22, 55 (2020), 792. DOI: 10.24818/ea/2020/55/792
Andreas Westfeld and Andreas Pfitzmann. 2000. Attacks on Steganographic Systems. In Information Hiding, Andreas Pfitzmann (Ed.). Springer Berlin Heidelberg, Berlin, Heidelberg, 61–76.
Li Xiaosong, Liu Shushi, Cai Wenjun, and Feng Songjiang. 2009. The Application of Risk Matrix to Software Project Risk Management. In 2009 International Forum on Information Technology and Applications, Vol. 2. 480–483. DOI: 10.1109/IFITA.2009.542
Associação Brasileira das Empresas de Software. 2021. Mercado Brasileiro de Software – Panorama e Tendências 2021. [link]
Andrea E Berndt. 2020. Sampling methods. Journal of Human Lactation 36, 2 (2020), 224–226.
Barry Boehm. 1989. Software risk management. In ESEC ’89, C. Ghezzi and J. A. McDermid (Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg, 1–19.
Katarína Buganová and Jana Šimíčková. 2019. Risk management in traditional and agile project management. Transportation Research Procedia 40 (2019), 986–993. DOI: 10.1016/j.trpro.2019.07.138 TRANSCOM 2019 13th International Scientific Conference on Sustainable, Modern and Safe Transport.
Juliette Michelle Parada Carvallo, Hanna Oktaba, and Elsa Ramírez Hernández. 2018. Risk Assessment Forum. In 2018 6th International Conference in Software Engineering Research and Innovation (CONISOFT). 160–164. DOI: 10.1109/CONISOFT.2018.8645949
Mauricio Concha, Marcello Visconti, and Hernán Astudillo. 2007. Agile Commitments: Enhancing Business Risk Management in Agile Development Projects. In Agile Processes in Software Engineering and Extreme Programming, Giulio Concas, Ernesto Damiani, Marco Scotto, and Giancarlo Succi (Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg, 149–152.
Lee J Cronbach. 1951. Coefficient alpha and the internal structure of tests. psychometrika 16, 3 (1951), 297–334.
Portia Crowe, Ali Mostashari, Mo Mansouri, and Robert Cloutier. 2009. 9.2.1 Reference Framework and Model for Integration of Risk Management in Agile Systems Engineering Lifecycle of the Defense Acquisition Management Framework. INCOSE International Symposium 19, 1 (2009), 1391–1405. DOI: 10.1002/j.2334-5837.2009.tb01022.x
Saru Dhir, Deepak Kumar, and V. Singh. 2019. Success and Failure Factors that Impact on Project Implementation Using Agile Software Development Methodology. 647–654. DOI: 10.1007/978-981-10-8848-3_62
Digital.ai. 2021. 15th annual state of agile report. Digital.ai.
Vasile Dorca, Radu Munteanu, Sorin Popescu, Adrian Chioreanu, and Claudius Peleskei. 2016. Agile approach with Kanban in information security risk management. In 2016 IEEE International Conference on Automation, Quality and Testing, Robotics (AQTR). 1–6. DOI: 10.1109/AQTR.2016.7501278
Abdelrafe Elzamly, Burairah Hussin, and Norhaziah Md Salleh. 2016. Top Fifty Software Risk Factors and the Best Thirty Risk Management Techniques in Software Development Lifecycle for Successful Software Projects. International Journal of Hybrid Information Technology 9 (06 2016), 11–32. DOI: 10.14257/ijhit.2016.9.6.02
Priscila Ferreira de Araújo Lima, Maria Crema, and Chiara Verbano. 2020. Risk management in SMEs: A systematic literature review and future directions. European Management Journal 38, 1 (2020), 78–94. DOI: 10.1016/j.emj.2019.06.005
Fernando Vedoin Garcia, Jean Hauck, and Fernanda Narloch Rizzo Hahn. 2022. Managing Risks in Agile Methods: a Systematic Literature Mapping. In Proceedings of the 34th International Conference on Software Engineering and Knowledge Engineering. DOI: 10.18293/SEKE2022-123
Muhammad Hammad and Irum Inayat. 2018. Integrating Risk Management in Scrum Framework. In 2018 International Conference on Frontiers of Information Technology (FIT). 158–163. DOI: 10.1109/FIT.2018.00035
Muhammad Hammad, Irum Inayat, and Maryam Zahid. 2019. Risk Management in Agile Software Development: A Survey. In 2019 International Conference on Frontiers of Information Technology (FIT). 162–1624. DOI: 10.1109/FIT47737.2019.00039
Faisal Hayat, Ammar Ur Rehman, Khawaja Sarmad Arif, Kanwal Wahab, and Muhammad Abbas. 2019. The Influence of Agile Methodology (Scrum) on Software Project Management. In 2019 20th IEEE/ACIS International Conference on Software Engineering, Artificial Intelligence, Networking and Parallel/Distributed Computing (SNPD). 145–149. DOI: 10.1109/SNPD.2019.8935813
Juhani Iivari and Netta Iivari. 2011. The relationship between organizational culture and the deployment of agile methods. Information and Software Technology 53 (05 2011), 509–520. DOI: 10.1016/j.infsof.2010.10.008
Shareeful Islam, Haris Mouratidis, and EdgarWeippl. 2014. An empirical study on the implementation and evaluation of a goal-driven software development risk management model. Information and Software Technology 56 (02 2014), 117–133. DOI: 10.1016/j.infsof.2013.06.003
ISO 14971:2019 2019. Medical devices – Application of risk management to medical devices. Standard. International Organization for Standardization.
ISO 31000:2009 2009. Risk Management – Principles and guidelines. Standard. International Organization for Standardization.
ISO 62304:2006 2006. Medical device software – Software life cycle processes. Standard. International Organization for Standardization.
ISO 80001:2010 2010. Application of risk management for IT-networks incorporating medical devices. Standard. International Organization for Standardization.
Shahedul Huq Khandkar. 2009. Open coding. University of Calgary 23 (2009), 2009.
Diana Kirk and Ewan Tempero. 2006. Identifying Risks in XP Projects through Process Modelling. In Proceedings of the Australian Software Engineering Conference (ASWEC ’06). IEEE Computer Society, USA, 411–420. DOI: 10.1109/ASWEC.2006.31
A. Koutsoyiannis. 1977. Theory of Econometrics. Open Journal of Statistics (1977).
Jefferson Seide Molléri, Kai Petersen, and Emilia Mendes. 2020. An empirically evaluated checklist for surveys in software engineering. Information and Software Technology 119 (2020), 106240. DOI: 10.1016/j.infsof.2019.106240
Jaana Nyfjord and Mira Kajko-Mattsson. 2007. Commonalities in Risk Management and Agile Process Models. International Conference on Software Engineering Advances (ICSEA 2007) (2007), 18–18. [link]
Edzreena Edza Odzaly, Des Greer, and Darryl Stewart. 2018. Agile risk management using software agents. Journal of Ambient Intelligence and Humanized Computing 9 (2018), 823–841.
Alain Pinsonneault and Kenneth Kraemer. 1993. Survey Research Methodology in Management Information Systems: An Assessment. Journal of Management Information Systems 10, 2 (1993), 75–105. DOI: 10.1080/07421222.1993.11518001 arXiv: [link]
Project Management Institute PMI. 2017. A Guide to the Project Management Body of Knowledge (PMBOK® Guide) (6 ed.). Project Management Institute.
T. Punter, M. Ciolkowski, B. Freimut, and I. John. 2003. Conducting on-line surveys in software engineering. In 2003 International Symposium on Empirical Software Engineering, 2003. ISESE 2003. Proceedings. 80–88. DOI: 10.1109/ISESE.2003.1237967
Sandra L. Ramírez-Mora and Hanna Oktaba. 2017. Productivity in Agile Software Development: A Systematic Mapping Study. In 2017 5th International Conference in Software Engineering Research and Innovation (CONISOFT). 44–53. DOI: 10.1109/CONISOFT.2017.00013
Lucio Ribeiro, Cristine Gusmao, Wilmar Feijo, and Vicente Bezerra. 2009. A case study for the implementation of an agile risk management process in multiple projects environments. In PICMET ’09 - 2009 Portland International Conference on Management of Engineering Technology. 1396–1404. DOI: 10.1109/PICMET.2009.5262002
Rogério Salgado. 2021. Brasil já é o 4º maior mercado do LinkedIn no planeta. [link] Acessado em 14/09/2023.
Eva-Maria Schön, Dirk Radtke, and Christian Jordan. 2020. Improving Risk Management in a Scaled Agile Environment. In Agile Processes in Software Engineering and Extreme Programming. Springer International Publishing, Cham, 132–141.
André Sousa, Joao Pascoal Faria, and Joao Mendes-Moreira. 2021. An Analysis of the State of the Art of Machine Learning for Risk Assessment in Software Projects. In Proceedings of the 33rd International Conference on Software Engineering and Knowledge Engineering, SEKE. 1–10.
Anselm L Strauss and Juliet Corbin. 2004. Open coding. Social research methods: A reader (2004), 303–306.
Kevin Suryaatmaja, Dermawan Wibisono, Achmad Ghazali, and Rachma Fitriati. 2020. Uncovering the failure of Agile framework implementation using SSMbased action research. Palgrave Communications 6 (01 2020), 8. DOI: 10.1057/s41599-019-0384-9
Keith S. Taber. 2017. The Use of Cronbach’s Alpha When Developing and Reporting Research Instruments in Science Education. Research in Science Education 48, 6 (June 2017), 1273–1296. DOI: 10.1007/s11165-016-9602-2
Maureen Tanner and Ulrich von Willingh. 2014. Factors Leading to the Success and Failure of Agile Projects Implemented in Traditionally Waterfall Environments. In Human Capital without Borders: Knowledge and Learning for the Quality of Life. Portoroz, Slovenia, 693–701. [link]
Breno Gontijo Tavares, Carlos Eduardo Sanches da Silva, and Adler Diniz de Souza. 2019. Practices to Improve Risk Management in Agile Projects. International Journal of Software Engineering and Knowledge Engineering 29, 03 (2019), 381–399. DOI: 10.1142/S0218194019500165 arXiv: [link]
Breno Gontijo Tavares, Mark Keil, Carlos Eduardo Sanches da Silva, and Adler Diniz de Souza. 2020. A risk management tool for agile software development. Journal of Computer Information Systems 61, 6 (2020), 561–570.
Fernando Vedoin Garcia, Jean Hauck, and Adriano Borgatto. 2024. An Analysis of the State of Practice of Risk Management in Agile Organizations in Brazil - Supplementary Material. Mendeley Data, V2. DOI: 10.17632/rdy3s9vzsb.2
Marcel Vieira, Jean C. R. Hauck, and Santiago Matalonga. 2020. How Explicit Risk Management is Being Integrated Into Agile Methods: Results From a Systematic Literature Mapping. In 19th Brazilian Symposium on Software Quality (São Luís, Brazil) (SBQS’20). Article 15, 10 pages. DOI: 10.1145/3439961.3439976
Zuzana Virglerova, Muhammad Asif Khan, Raimonda Martinkute-Kauliene, and Sandor Kovacs. 2020. The internationalization of smes in Central Europe and its impact on their methods of Risk Management. www.amfiteatrueconomic.ro 22, 55 (2020), 792. DOI: 10.24818/ea/2020/55/792
Andreas Westfeld and Andreas Pfitzmann. 2000. Attacks on Steganographic Systems. In Information Hiding, Andreas Pfitzmann (Ed.). Springer Berlin Heidelberg, Berlin, Heidelberg, 61–76.
Li Xiaosong, Liu Shushi, Cai Wenjun, and Feng Songjiang. 2009. The Application of Risk Matrix to Software Project Risk Management. In 2009 International Forum on Information Technology and Applications, Vol. 2. 480–483. DOI: 10.1109/IFITA.2009.542
Publicado
30/09/2024
Como Citar
GARCIA, Fernando Vedoin; HAUCK, Jean Carlo Rossa; BORGATTO, Adriano.
How do Agile Organizations Manage Risks: An Analysis of the State of Practice in Brazil. In: SIMPÓSIO BRASILEIRO DE ENGENHARIA DE SOFTWARE (SBES), 38. , 2024, Curitiba/PR.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2024
.
p. 80-91.
DOI: https://doi.org/10.5753/sbes.2024.3292.
