How has process assessment been automated by organizations? A systematic literature mapping
Resumo
Organizational process improvement initiatives typically apply some kind of process assessment. Even partial automation of these process assessments can be beneficial. In this sense, the aim of this study is to investigate the state of the art in automated process assessment. A systematic literature mapping is performed in order to answer the research question: What are the approaches to automated process assessment? Thus, the state of the art is analyzed following the steps of protocol definition, search execution, study selection and data extraction and synthesis. We found 16 studies that directly or indirectly use some kind of automated process assessment support. The approaches reported in these studies cover most of the steps expected in typical process assessments and used nine different software tools to support these steps. The main observed results of process assessment were reduction of effort, time and complexity and automated assessment feedback. There are two major contributions of this research: a comprehensive and structured overview of the state of the art in automated process assessment, contributing to the knowledge organization of this field; and an initial guide to organizations that want to apply efforts to automate their process assessments in choosing the tools, techniques and approaches, and knowing the expected outcomes of this kind of initiative.
Palavras-chave:
Process Assessment, Literature Mapping, Automated Assessment
Referências
Davenport T, Short JE. 1990. The new industrial engineering: information technology and business process redesign. Sloan Manage Rev 31, 4, 11–27.
Kohlbacher, M. 2010. The Effects of Process Orientation: A Literature Review. Business process management journal 16, 1, 135-152.
de Bruin, T. 2009. Business process management: theory on progression and maturity, working paper, School of Information Technology, Queensland University of Technology, Brisbane.
OMG. 2008. Business Process Maturity Model (BPMM). Available at: http://www.omg.org/spec/BPMM/.
de Boer, F.G., Müller, C.J. and ten Caten, C.S. 2015. Assessment model for organizational business process maturity with a focus on BPM governance practices. Journal of Business Process Management, 2, 4, 908-927.
Tarhan, A., Turetken, O. and Ilisulu, F. 2015. Business process maturity assessment: state of the art and key characteristics. In 2015 41st Euromicro Conference on Software Engineering and Advanced Applications, 430-437.
ISO/IEC. 2015. ISO/IEC 33003:2015 - Information technology - Process assessment - Requirements for process measurement frameworks.
Ashrafi, N. (2003), The impact of software process improvement on quality: in theory and practice, Journal of Information & Management, 40, 7, 677-690.
Conradi, H., and Fuggetta, A. 2002. Improving software process improvement. IEEE software, 19, 4, 92-99.
Jarvinen, J. 2000. Measurement based continuous assessment of software engineering processes. In Espoo 2000, Technical Research Centre of Finland, VTT PUBLICATIONS, 426.
Tarhan, A., Turetken, O., and Reijers, H. A. 2016. Business process maturity models: A systematic literature review, Information and Software Technology, 75, 122-134.
Wangenheim, C. G. von, Hauck, J. C. R., Salviano, C. F., and von Wangenheim, A. 2010. Systematic literature review of software process capability/maturity models. In Proceedings of SPICE 2010, Pisa, Italy, 1-9.
ISO/IEC. 2015. ISO/IEC 33020:2015 - Information technology - Process assessment - Process measurement framework for assessment of process capability.
Shrestha, A., Cater-Steel, A., Tan, W. G., & Toleman, M. 2014. Software-mediated process assessment for IT service capability management. In European Conference on Information Systems (ECIS), Tel Aviv, Israel.
Petersen, K., Feldt, R., Mujtaba, S., & Mattsson, M. 2008. Systematic mapping studies in software engineering. Ease, 8, 68-77.
Kitchenham, B., Charters S. 2007. Guidelines for performing systematic literature reviews in software engineering. Version 2.3, EBSE Technical Report, EBSE-2007-01 – Department of Computer Science, Keele University.
Calabro, A., Lonetti, F. and Marchetti, E. 2015. Monitoring of business process execution based on performance indicators. In 41st Euromicro, 255-258. DOI: https://doi.org/10.1109/seaa.2015.73
Khokhar, M. N., Mansoor, A., Khokhar, M. N., Rehman, S.U. and Rauf, A. 2010. MECA: Software process improvement for small organizations. In 2010 International Conference on Information and Emerging Technologies, 1-6. DOI: https://doi.org/10.1109/iciet.2010.5625678
Garcia, I. and Pacheco, C. 2009. Toward automated support for software process improvement initiatives in small and medium size enterprises. In Software engineering research, management and applications, Springer, Berlin, Heidelberg, pp. 51-58. DOI: https://doi.org/10.1007/978-3-642-05441-9_5
Choi, S., Kim, D. K. and Park, S. 2012. ReMo: A recommendation model for software process improvement. In Proceedings of the International Conference on Software and System Process, 135-139. DOI: https://doi.org/10.1109/icssp.2012.6225957
Patel, C. and Ramachandran, M. 2009. Agile maturity model (AMM): A Software Process Improvement framework for agile software development practices. International Journal of Software Engineering, IJSE, 2, 1, 3-28. Available at: [link].
Pino, F. J., Garcia, F. and Piattini, M. 2007. A support tool for rapid software process assessment. IEEE Latin America Transactions, 5, 4, 218-223. DOI: https://doi.org/10.1109/tla.2007.4378509
Astorga-Vargas, A., Johanna, M. B., Brenda, F. R. and Jorge, I. E. 2014. Determining software process capability in conformity to the process assessment model NMX-I-15504 applied to the reference model NMX-I-059 supported by the AURAP tool. In 2014 9th Iberian Conference on Information Systems and Technologies (CISTI), 1-6.DOI: https://doi.org/10.1109/cisti.2014.6876898
Sabato, S., Filangieri, C. 2002. ASSO: a Web-portal application for operational process assessment and organizational improvement. In NOMS 2002 IEEE/IFIP Networks Operations and Management Symposium, 843-853.DOI: https://doi.org/10.1109/noms.2002.1015628
Adali, O. E., Top, Ö. Ö. and Demirörs, O. 2017. Assessment of agility in software organizations with a web-based agility assessment tool. In 43rd Euromicro Conference on Software Engineering and Advanced Applications (SEAA 2007), 88-95.DOI: https://doi.org/10.1109/seaa.2017.61
Homchuenchom, D., Piyabunditkul, C., Lichter, H. and Anwar, T. 2011. SPIALS: A light-weight software process improvement self-assessment tool. In 2011 Malaysian Conference in Software Engineering, 195-199.DOI: https://doi.org/10.1109/mysec.2011.6140668
Dai, G., Bai, X., Zhao, C. 2007. A framework for model checking web service compositions based on bpel4ws. In IEEE International Conference on e-Business Engineering (ICEBE'07). 165-172. DOI: https://doi.org/10.1109/icebe.2007.11
Pino, F. J., Pedreira, O., García, F., Luaces, M.R. and Piattini, M. 2010. Using Scrum to guide the execution of software process improvement in small organizations. Journal of Systems and Software, 83, 10, 1662-1677. DOI: https://doi.org/10.1016/j.jss.2010.03.077
Zaouali, S. and Ghannouchi, S.A. 2016. Proposition of an approach based on BPM to manage agile development processes. In 2016 Third International Conference on Systems of Collaboration (SysCo), 1-6. DOI: https://doi.org/10.1109/sysco.2016.7831331
Varkoi, T. 2010. Process assessment in very small entities-An ISO/IEC 29110 based method. In 2010 Seventh International Conference on the Quality of Information and Communications Technology, 436-440. DOI: https://doi.org/10.1109/quatic.2010.80
Suteeca, K., Ramingwong, S. 2016. A framework to apply ISO/IEC29110 on SCRUM. In 2016 International Computer Science and Engineering Conference (ICSEC), 1-5.DOI: https://doi.org/10.1109/icsec.2016.7859884
Montenegro, C., & Arévalo, R. 2018. Software development governance for VSE-SCRUM teams: Model and evaluation in a developing country. In Proceedings of the 2018 International Conference on Software Engineering and Information Management, ACM, 1-5.DOI: https://doi.org/10.1145/3178461.3178474
Buglione, L., von Wangenheim, C. G., McCaffery, F., & Hauck, J. C. R. 2013. The LEGO strategy: guidelines for a profitable deployment. Computer Standards & Interfaces, 36, 1, 10-20.
Hauck, J. C. R., Von Wangenheim, C. G., Mc Caffery, F., & Buglione, L. 2011. Proposing an ISO/IEC 15504-2 compliant method for process capability/maturity models customization. In International Conference on Product Focused Software Process Improvement. Springer, Berlin, Heidelberg, 44-58.
Zemguliene, J.; Valukonis, M. 2018. Structured literature review on business process performance analysis and evaluation. Entrepreneurship and Sustainability Issues, 6, 1, 226-252.
Sarkis, J.; Meade, L.; Presley, A. 2006. An activity based management methodology for evaluating business processes for environmental sustainability. Business Process Management Journal, 12, 6, 751-769.
Cho, M., Song, M., Comuzzi, M., & Yoo, S. 2017. Evaluating the effect of best practices for business process redesign: An evidence-based approach based on process mining techniques. Decision Support Systems, 104, 92-103.
Claes, J., Vanderfeesten, I., Gailly, F., Grefen, P., & Poels, G. 2017. The Structured Process Modeling Method (SPMM) what is the best way for me to construct a process model? Decision Support Systems, 100, 57-76.
Yilmaz, G., Akcamete, A., & Demirors, O. 2019. A reference model for BIM capability assessments. Automation in Construction, 101, 245-263.
Delgado, A., Weber, B., Ruiz, F., De Guzmán, I. G. R., & Piattini, M. 2014. An integrated approach based on execution measures for the continuous improvement of business processes realized by services. Information and Software Technology, 56, 2, 134-162.
Abeygunasekera, A. W. J. C., Bandara, W., Wynn, M. T., & Yigitbasioglu, O. 2019. Process performance measurement system characteristics: An empirically validated framework. In International Conference on Business Process Management, 211-227.
Van Looy, A. and Shafagatova, A. 2016, Business process performance measurement: a structured literature review of indicators, measures and metrics. Springerplus. 18, 5, 1797.
Zhou, X., Jin, Y., Zhang, H., Li, S., & Huang, X. 2016. A map of threats to validity of systematic literature reviews in software engineering. In 23rd Asia-Pacific Software Engineering Conference (APSEC 2016), 153-160.
Kohlbacher, M. 2010. The Effects of Process Orientation: A Literature Review. Business process management journal 16, 1, 135-152.
de Bruin, T. 2009. Business process management: theory on progression and maturity, working paper, School of Information Technology, Queensland University of Technology, Brisbane.
OMG. 2008. Business Process Maturity Model (BPMM). Available at: http://www.omg.org/spec/BPMM/.
de Boer, F.G., Müller, C.J. and ten Caten, C.S. 2015. Assessment model for organizational business process maturity with a focus on BPM governance practices. Journal of Business Process Management, 2, 4, 908-927.
Tarhan, A., Turetken, O. and Ilisulu, F. 2015. Business process maturity assessment: state of the art and key characteristics. In 2015 41st Euromicro Conference on Software Engineering and Advanced Applications, 430-437.
ISO/IEC. 2015. ISO/IEC 33003:2015 - Information technology - Process assessment - Requirements for process measurement frameworks.
Ashrafi, N. (2003), The impact of software process improvement on quality: in theory and practice, Journal of Information & Management, 40, 7, 677-690.
Conradi, H., and Fuggetta, A. 2002. Improving software process improvement. IEEE software, 19, 4, 92-99.
Jarvinen, J. 2000. Measurement based continuous assessment of software engineering processes. In Espoo 2000, Technical Research Centre of Finland, VTT PUBLICATIONS, 426.
Tarhan, A., Turetken, O., and Reijers, H. A. 2016. Business process maturity models: A systematic literature review, Information and Software Technology, 75, 122-134.
Wangenheim, C. G. von, Hauck, J. C. R., Salviano, C. F., and von Wangenheim, A. 2010. Systematic literature review of software process capability/maturity models. In Proceedings of SPICE 2010, Pisa, Italy, 1-9.
ISO/IEC. 2015. ISO/IEC 33020:2015 - Information technology - Process assessment - Process measurement framework for assessment of process capability.
Shrestha, A., Cater-Steel, A., Tan, W. G., & Toleman, M. 2014. Software-mediated process assessment for IT service capability management. In European Conference on Information Systems (ECIS), Tel Aviv, Israel.
Petersen, K., Feldt, R., Mujtaba, S., & Mattsson, M. 2008. Systematic mapping studies in software engineering. Ease, 8, 68-77.
Kitchenham, B., Charters S. 2007. Guidelines for performing systematic literature reviews in software engineering. Version 2.3, EBSE Technical Report, EBSE-2007-01 – Department of Computer Science, Keele University.
Calabro, A., Lonetti, F. and Marchetti, E. 2015. Monitoring of business process execution based on performance indicators. In 41st Euromicro, 255-258. DOI: https://doi.org/10.1109/seaa.2015.73
Khokhar, M. N., Mansoor, A., Khokhar, M. N., Rehman, S.U. and Rauf, A. 2010. MECA: Software process improvement for small organizations. In 2010 International Conference on Information and Emerging Technologies, 1-6. DOI: https://doi.org/10.1109/iciet.2010.5625678
Garcia, I. and Pacheco, C. 2009. Toward automated support for software process improvement initiatives in small and medium size enterprises. In Software engineering research, management and applications, Springer, Berlin, Heidelberg, pp. 51-58. DOI: https://doi.org/10.1007/978-3-642-05441-9_5
Choi, S., Kim, D. K. and Park, S. 2012. ReMo: A recommendation model for software process improvement. In Proceedings of the International Conference on Software and System Process, 135-139. DOI: https://doi.org/10.1109/icssp.2012.6225957
Patel, C. and Ramachandran, M. 2009. Agile maturity model (AMM): A Software Process Improvement framework for agile software development practices. International Journal of Software Engineering, IJSE, 2, 1, 3-28. Available at: [link].
Pino, F. J., Garcia, F. and Piattini, M. 2007. A support tool for rapid software process assessment. IEEE Latin America Transactions, 5, 4, 218-223. DOI: https://doi.org/10.1109/tla.2007.4378509
Astorga-Vargas, A., Johanna, M. B., Brenda, F. R. and Jorge, I. E. 2014. Determining software process capability in conformity to the process assessment model NMX-I-15504 applied to the reference model NMX-I-059 supported by the AURAP tool. In 2014 9th Iberian Conference on Information Systems and Technologies (CISTI), 1-6.DOI: https://doi.org/10.1109/cisti.2014.6876898
Sabato, S., Filangieri, C. 2002. ASSO: a Web-portal application for operational process assessment and organizational improvement. In NOMS 2002 IEEE/IFIP Networks Operations and Management Symposium, 843-853.DOI: https://doi.org/10.1109/noms.2002.1015628
Adali, O. E., Top, Ö. Ö. and Demirörs, O. 2017. Assessment of agility in software organizations with a web-based agility assessment tool. In 43rd Euromicro Conference on Software Engineering and Advanced Applications (SEAA 2007), 88-95.DOI: https://doi.org/10.1109/seaa.2017.61
Homchuenchom, D., Piyabunditkul, C., Lichter, H. and Anwar, T. 2011. SPIALS: A light-weight software process improvement self-assessment tool. In 2011 Malaysian Conference in Software Engineering, 195-199.DOI: https://doi.org/10.1109/mysec.2011.6140668
Dai, G., Bai, X., Zhao, C. 2007. A framework for model checking web service compositions based on bpel4ws. In IEEE International Conference on e-Business Engineering (ICEBE'07). 165-172. DOI: https://doi.org/10.1109/icebe.2007.11
Pino, F. J., Pedreira, O., García, F., Luaces, M.R. and Piattini, M. 2010. Using Scrum to guide the execution of software process improvement in small organizations. Journal of Systems and Software, 83, 10, 1662-1677. DOI: https://doi.org/10.1016/j.jss.2010.03.077
Zaouali, S. and Ghannouchi, S.A. 2016. Proposition of an approach based on BPM to manage agile development processes. In 2016 Third International Conference on Systems of Collaboration (SysCo), 1-6. DOI: https://doi.org/10.1109/sysco.2016.7831331
Varkoi, T. 2010. Process assessment in very small entities-An ISO/IEC 29110 based method. In 2010 Seventh International Conference on the Quality of Information and Communications Technology, 436-440. DOI: https://doi.org/10.1109/quatic.2010.80
Suteeca, K., Ramingwong, S. 2016. A framework to apply ISO/IEC29110 on SCRUM. In 2016 International Computer Science and Engineering Conference (ICSEC), 1-5.DOI: https://doi.org/10.1109/icsec.2016.7859884
Montenegro, C., & Arévalo, R. 2018. Software development governance for VSE-SCRUM teams: Model and evaluation in a developing country. In Proceedings of the 2018 International Conference on Software Engineering and Information Management, ACM, 1-5.DOI: https://doi.org/10.1145/3178461.3178474
Buglione, L., von Wangenheim, C. G., McCaffery, F., & Hauck, J. C. R. 2013. The LEGO strategy: guidelines for a profitable deployment. Computer Standards & Interfaces, 36, 1, 10-20.
Hauck, J. C. R., Von Wangenheim, C. G., Mc Caffery, F., & Buglione, L. 2011. Proposing an ISO/IEC 15504-2 compliant method for process capability/maturity models customization. In International Conference on Product Focused Software Process Improvement. Springer, Berlin, Heidelberg, 44-58.
Zemguliene, J.; Valukonis, M. 2018. Structured literature review on business process performance analysis and evaluation. Entrepreneurship and Sustainability Issues, 6, 1, 226-252.
Sarkis, J.; Meade, L.; Presley, A. 2006. An activity based management methodology for evaluating business processes for environmental sustainability. Business Process Management Journal, 12, 6, 751-769.
Cho, M., Song, M., Comuzzi, M., & Yoo, S. 2017. Evaluating the effect of best practices for business process redesign: An evidence-based approach based on process mining techniques. Decision Support Systems, 104, 92-103.
Claes, J., Vanderfeesten, I., Gailly, F., Grefen, P., & Poels, G. 2017. The Structured Process Modeling Method (SPMM) what is the best way for me to construct a process model? Decision Support Systems, 100, 57-76.
Yilmaz, G., Akcamete, A., & Demirors, O. 2019. A reference model for BIM capability assessments. Automation in Construction, 101, 245-263.
Delgado, A., Weber, B., Ruiz, F., De Guzmán, I. G. R., & Piattini, M. 2014. An integrated approach based on execution measures for the continuous improvement of business processes realized by services. Information and Software Technology, 56, 2, 134-162.
Abeygunasekera, A. W. J. C., Bandara, W., Wynn, M. T., & Yigitbasioglu, O. 2019. Process performance measurement system characteristics: An empirically validated framework. In International Conference on Business Process Management, 211-227.
Van Looy, A. and Shafagatova, A. 2016, Business process performance measurement: a structured literature review of indicators, measures and metrics. Springerplus. 18, 5, 1797.
Zhou, X., Jin, Y., Zhang, H., Li, S., & Huang, X. 2016. A map of threats to validity of systematic literature reviews in software engineering. In 23rd Asia-Pacific Software Engineering Conference (APSEC 2016), 153-160.
Publicado
07/06/2021
Como Citar
HAUCK, Jean Carlo Rossa; GALIMBERTI, Maurício Floriano; ZWIRTES, Augusto Guarda Lara; BOSSE, Jacyara.
How has process assessment been automated by organizations? A systematic literature mapping. In: SIMPÓSIO BRASILEIRO DE SISTEMAS DE INFORMAÇÃO (SBSI), 17. , 2021, Uberlândia.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2021
.
