Quality Management in the Public Sector: Definition and Prioritization of Indicators for Assessment and Decision-Making
Abstract
The use of historical data in software projects, when aligned with quality goals and business objectives, has increasingly been recognized as a strategic approach to enhancing both management and system development. This practice relies on the analysis of collected data, which enables the identification of opportunities and the anticipation of risks. As quality processes are implemented, these data are transformed into measures and metrics used to develop indicators, allowing for continuous performance monitoring and driving organizational improvement. In this context, this report presents the process used for the definition, validation, and prioritization of quality measures and indicators applied to the management and development of software projects in a public organization. Furthermore, in this study we discuss the main challenges faced, as well as the lessons learned and the benefits observed at the current stage of the process of implementing this practice within the institution.
Keywords:
Software Quality Management, Software Process, Quality Indicators and Measures
References
M. Alsulami. 2021. A Systematic Literature Review on Software Metrics. International Transaction Journal of Engineering, Management, & Applied Sciences & Technologies 12, 12 (2021), 12A12G, 1–13. DOI: 10.14456/ITJEMAST.2021.238
Victor R. Basili, Gianluigi Caldiera, and H. Dieter Rombach. 1994. The Goal Question Metric Approach. In Encyclopedia of Software Engineering. John Wiley & Sons, 528–532.
Victor R. Basili, Gianluigi Caldiera, and H. D. Rombach. 1994. The Goal Question Metric Approach. Encyclopedia of Software Engineering 2 (1994), 528–532.
Rainara Carvalho, Anna Marques, Rossana Andrade, Amanda Sousa, Raquel Dias, and Guilherme Campagnoli. 2022. An Experience of Using the GQM Approach in a Remote Environment to Define Requirements Metrics. In Anais do XXI Simpósio Brasileiro de Qualidade de Software (Curitiba/PR). SBC, Porto Alegre, RS, Brasil, 219–228. [link]
Dave Chaffey. 2015. Digital Business and E-commerce Management: Strategy, Implementation and Practice (6 ed.). Pearson Education, Hoboken, NJ.
CMMI Institute. 2010. CMMI for Development, Version 1.3: Improving processes for better products. Software Engineering Institute, Carnegie Mellon University, Pittsburgh, PA. Available at: [link].
Alex Costa, Leidiana Freitas, Dafne Cavalcante, Victória Oliveira, Valéria Lelli, Ismayle Santos, Pedro Oliveira, Tales Nogueira, and Rossana Andrade. 2024. Especificação de Requisitos em um projeto de Big Data no Setor Público. In Anais do XXVII Congresso Ibero-Americano em Engenharia de Software (Curitiba/PR). SBC, Porto Alegre, RS, Brasil, 417–420. DOI: 10.5753/cibse.2024.28528
Alistair Croll and Benjamin Yoskovitz. 2013. Lean Analytics: Use Data to Build a Better Startup Faster. O’Reilly Media, Sebastopol, CA.
Thomas H. Davenport. 2013. Analytics 3.0. Harvard Business Review (Dec. 2013).
Wayne W. Eckerson. 2011. Performance Dashboards: Measuring, Monitoring, and Managing Your Business (2 ed.). John Wiley & Sons, Hoboken, NJ.
Norman E. Fenton and James M. Bieman. 2014. Software Metrics: A Rigorous and Practical Approach (3rd ed.). CRC Press, Boca Raton, FL.
José Finocchio Júnior. 2013. Project Model Canvas: Gerenciamento de projetos sem burocracia. Elsevier, Rio de Janeiro.
Xavier Franch. 2021. Data-Driven Requirements Engineering: A Guided Tour. In Evaluation of Novel Approaches to Software Engineering, Raian Ali, Hermann Kaindl, and Leszek A. Maciaszek (Eds.). Springer International Publishing, Cham, 83–105.
R. B. Grady. 1992. Practical Software Metrics for Project Management and Process Improvement. Prentice-Hall, Inc., Englewood Cliffs, NJ.
Heli Helenius. 2024. Metrics Contributing to the Implementation of Successful Projects. (2024). Unpublished manuscript.
International Organization for Standardization. 2011. ISO/IEC 25010:2011 Systems and software engineering – Systems and software Quality Requirements and Evaluation (SQuaRE) – System and software quality models.
International Organization for Standardization. 2015. ISO/IEC 25024:2015 — Systems and software engineering – Systems and software Quality Requirements and Evaluation (SQuaRE) – Measurement of data quality.
International Organization for Standardization. 2016. ISO/IEC 25022:2016 — Systems and software engineering – Systems and software Quality Requirements and Evaluation (SQuaRE) – Measurement of quality in use.
International Organization for Standardization. 2016. ISO/IEC 25023:2016 — Systems and software engineering – Systems and software Quality Requirements and Evaluation (SQuaRE) – Measurement of system and software product quality.
International Organization for Standardization. 2017. ISO/IEC 15939:2017 — Systems and software engineering — Measurement process. [link]. Accessed: 2025-05-10.
ISACA. 2012. COBIT 5: A Business Framework for the Governance and Management of Enterprise IT. ISACA, Rolling Meadows, IL.
Newton Krüger Tallens Junior, Emerson Gomes dos Santos, Camila Bertini Martins, and Rogerio Scabim Morano. 2025. Construção de indicadores de desempenho organizacional para o setor público: atributos e critérios de validação em proposta inovadora. Revista do TCU 155, 1 (2025), 270–299. DOI: 10.69518/rtcu.155.270-299 Acesso em: 5 out. 2025.
Liza Kayser, Roland M. Mueller, and Tizian Kronsbein. 2019. Data Collection Map: A Canvas for Shared Data Awareness in Data-Driven Innovation Projects. In Proceedings of the 2019 Pre-ICIS SIGDSA Symposium. 18. [link]
Barry Kitchenham, Emilia Mendes, and Gerson H. Travassos. 2007. A Systematic Review of Cross-Versus Within-Company Cost Estimation Studies. IEEE Transactions on Software Engineering 33, 5 (2007), 316–329. DOI: 10.1109/TSE.2007.1001
OECD. 2020. The OECD Digital Government Policy Framework: Six dimensions of a Digital Government. OECD Public Governance Policy Papers 2. OECD Publishing, Paris. DOI: 10.1787/f64fed2a-en
Alexander Osterwalder and Yves Pigneur. 2010. Business Model Generation: A Handbook for Visionaries, Game Changers, and Challengers. John Wiley & Sons, Hoboken, NJ.
US Poornima and V Suma. 2012. Significance of Quality Metrics in Software Development Process. arXiv preprint arXiv:1207.0592 (2012).
Roger S. Pressman and Bruce R. Maxim. 2016. Engenharia de Software: uma abordagem profissional (8 ed.). McGraw-Hill Brasil.
Foster Provost and Tom Fawcett. 2013. Data Science for Business: What You Need to Know about Data Mining and Data-Analytic Thinking. O’Reilly Media, Sebastopol, CA.
K. Pulford, A. Kuntzmann-Combelles, and S. Shirlaw. 1995. A Quantitative Approach to Software Management: The AMI Handbook. Addison-Wesley Longman Publishing Co., Inc., Boston, MA.
Junaid Rashid, Toqeer Mahmood, and Muhamad Wasif Nisar. 2019. A study on software metrics and its impact on software quality. arXiv preprint arXiv:1905.12922 (2019).
Laura Cozzi Ribeiro, Matheus Pereira Libório, Hasheem Mannan, Sandro Laudares, Petr Iakovlevich Ekel, Douglas Alexandre Gomes Vieira, and Cristiane Neri Nobre. 2024. Software for building and measuring the quality of composite indicators using ordered weighted averaging: So-called S-CI-OWA. SoftwareX 26 (2024), 101660. DOI: 10.1016/j.softx.2024.101660
Serviço Brasileiro de Apoio às Micro e Pequenas Empresas (SEBRAE-PR). [n. d.]. Sebrae Canvas: ferramenta para criação de modelos de negócios digitais. [link]. Acesso em: 11 jul. 2025.
Ian Sommerville. 2011. Software Engineering (9 ed.). Addison-Wesley.
Touseef Tahir, Ghulam Rasool, and Cigdem Gencel. 2016. A systematic literature review on software measurement programs. Information and Software Technology 73 (2016), 101–121. DOI: 10.1016/j.infsof.2016.01.014
Rini Van Solingen and EgonW. Berghout. 1999. The Goal/Question/Metric Method: A Practical Guide for Quality Improvement of Software Development. McGraw-Hill.
Richard Y. Wang and Diane M. Strong. 1996. Beyond Accuracy: What Data Quality Means to Data Consumers. Journal of Management Information Systems 12, 4 (1996), 5–33. DOI: 10.1080/07421222.1996.11518099
Yichuan Wang and Nick Hajli. 2017. Exploring the Path to Big Data Analytics Success in Healthcare. Journal of Business Research 70 (2017), 287–299. DOI: 10.1016/j.jbusres.2016.08.002
Élcio Batista, Rossana Andrade, Ismayle Santos, Tales Nogueira, Pedro Oliveira, Valéria Lelli, and Victória Oliveira. 2024. Fortaleza City Hall Strategic Planning based on Data Analysis and Forecasting. In Anais do XXVII Congresso Ibero-Americano em Engenharia de Software (Curitiba/PR). SBC, Porto Alegre, RS, Brasil, 433–436. DOI: 10.5753/cibse.2024.28532
Victor R. Basili, Gianluigi Caldiera, and H. Dieter Rombach. 1994. The Goal Question Metric Approach. In Encyclopedia of Software Engineering. John Wiley & Sons, 528–532.
Victor R. Basili, Gianluigi Caldiera, and H. D. Rombach. 1994. The Goal Question Metric Approach. Encyclopedia of Software Engineering 2 (1994), 528–532.
Rainara Carvalho, Anna Marques, Rossana Andrade, Amanda Sousa, Raquel Dias, and Guilherme Campagnoli. 2022. An Experience of Using the GQM Approach in a Remote Environment to Define Requirements Metrics. In Anais do XXI Simpósio Brasileiro de Qualidade de Software (Curitiba/PR). SBC, Porto Alegre, RS, Brasil, 219–228. [link]
Dave Chaffey. 2015. Digital Business and E-commerce Management: Strategy, Implementation and Practice (6 ed.). Pearson Education, Hoboken, NJ.
CMMI Institute. 2010. CMMI for Development, Version 1.3: Improving processes for better products. Software Engineering Institute, Carnegie Mellon University, Pittsburgh, PA. Available at: [link].
Alex Costa, Leidiana Freitas, Dafne Cavalcante, Victória Oliveira, Valéria Lelli, Ismayle Santos, Pedro Oliveira, Tales Nogueira, and Rossana Andrade. 2024. Especificação de Requisitos em um projeto de Big Data no Setor Público. In Anais do XXVII Congresso Ibero-Americano em Engenharia de Software (Curitiba/PR). SBC, Porto Alegre, RS, Brasil, 417–420. DOI: 10.5753/cibse.2024.28528
Alistair Croll and Benjamin Yoskovitz. 2013. Lean Analytics: Use Data to Build a Better Startup Faster. O’Reilly Media, Sebastopol, CA.
Thomas H. Davenport. 2013. Analytics 3.0. Harvard Business Review (Dec. 2013).
Wayne W. Eckerson. 2011. Performance Dashboards: Measuring, Monitoring, and Managing Your Business (2 ed.). John Wiley & Sons, Hoboken, NJ.
Norman E. Fenton and James M. Bieman. 2014. Software Metrics: A Rigorous and Practical Approach (3rd ed.). CRC Press, Boca Raton, FL.
José Finocchio Júnior. 2013. Project Model Canvas: Gerenciamento de projetos sem burocracia. Elsevier, Rio de Janeiro.
Xavier Franch. 2021. Data-Driven Requirements Engineering: A Guided Tour. In Evaluation of Novel Approaches to Software Engineering, Raian Ali, Hermann Kaindl, and Leszek A. Maciaszek (Eds.). Springer International Publishing, Cham, 83–105.
R. B. Grady. 1992. Practical Software Metrics for Project Management and Process Improvement. Prentice-Hall, Inc., Englewood Cliffs, NJ.
Heli Helenius. 2024. Metrics Contributing to the Implementation of Successful Projects. (2024). Unpublished manuscript.
International Organization for Standardization. 2011. ISO/IEC 25010:2011 Systems and software engineering – Systems and software Quality Requirements and Evaluation (SQuaRE) – System and software quality models.
International Organization for Standardization. 2015. ISO/IEC 25024:2015 — Systems and software engineering – Systems and software Quality Requirements and Evaluation (SQuaRE) – Measurement of data quality.
International Organization for Standardization. 2016. ISO/IEC 25022:2016 — Systems and software engineering – Systems and software Quality Requirements and Evaluation (SQuaRE) – Measurement of quality in use.
International Organization for Standardization. 2016. ISO/IEC 25023:2016 — Systems and software engineering – Systems and software Quality Requirements and Evaluation (SQuaRE) – Measurement of system and software product quality.
International Organization for Standardization. 2017. ISO/IEC 15939:2017 — Systems and software engineering — Measurement process. [link]. Accessed: 2025-05-10.
ISACA. 2012. COBIT 5: A Business Framework for the Governance and Management of Enterprise IT. ISACA, Rolling Meadows, IL.
Newton Krüger Tallens Junior, Emerson Gomes dos Santos, Camila Bertini Martins, and Rogerio Scabim Morano. 2025. Construção de indicadores de desempenho organizacional para o setor público: atributos e critérios de validação em proposta inovadora. Revista do TCU 155, 1 (2025), 270–299. DOI: 10.69518/rtcu.155.270-299 Acesso em: 5 out. 2025.
Liza Kayser, Roland M. Mueller, and Tizian Kronsbein. 2019. Data Collection Map: A Canvas for Shared Data Awareness in Data-Driven Innovation Projects. In Proceedings of the 2019 Pre-ICIS SIGDSA Symposium. 18. [link]
Barry Kitchenham, Emilia Mendes, and Gerson H. Travassos. 2007. A Systematic Review of Cross-Versus Within-Company Cost Estimation Studies. IEEE Transactions on Software Engineering 33, 5 (2007), 316–329. DOI: 10.1109/TSE.2007.1001
OECD. 2020. The OECD Digital Government Policy Framework: Six dimensions of a Digital Government. OECD Public Governance Policy Papers 2. OECD Publishing, Paris. DOI: 10.1787/f64fed2a-en
Alexander Osterwalder and Yves Pigneur. 2010. Business Model Generation: A Handbook for Visionaries, Game Changers, and Challengers. John Wiley & Sons, Hoboken, NJ.
US Poornima and V Suma. 2012. Significance of Quality Metrics in Software Development Process. arXiv preprint arXiv:1207.0592 (2012).
Roger S. Pressman and Bruce R. Maxim. 2016. Engenharia de Software: uma abordagem profissional (8 ed.). McGraw-Hill Brasil.
Foster Provost and Tom Fawcett. 2013. Data Science for Business: What You Need to Know about Data Mining and Data-Analytic Thinking. O’Reilly Media, Sebastopol, CA.
K. Pulford, A. Kuntzmann-Combelles, and S. Shirlaw. 1995. A Quantitative Approach to Software Management: The AMI Handbook. Addison-Wesley Longman Publishing Co., Inc., Boston, MA.
Junaid Rashid, Toqeer Mahmood, and Muhamad Wasif Nisar. 2019. A study on software metrics and its impact on software quality. arXiv preprint arXiv:1905.12922 (2019).
Laura Cozzi Ribeiro, Matheus Pereira Libório, Hasheem Mannan, Sandro Laudares, Petr Iakovlevich Ekel, Douglas Alexandre Gomes Vieira, and Cristiane Neri Nobre. 2024. Software for building and measuring the quality of composite indicators using ordered weighted averaging: So-called S-CI-OWA. SoftwareX 26 (2024), 101660. DOI: 10.1016/j.softx.2024.101660
Serviço Brasileiro de Apoio às Micro e Pequenas Empresas (SEBRAE-PR). [n. d.]. Sebrae Canvas: ferramenta para criação de modelos de negócios digitais. [link]. Acesso em: 11 jul. 2025.
Ian Sommerville. 2011. Software Engineering (9 ed.). Addison-Wesley.
Touseef Tahir, Ghulam Rasool, and Cigdem Gencel. 2016. A systematic literature review on software measurement programs. Information and Software Technology 73 (2016), 101–121. DOI: 10.1016/j.infsof.2016.01.014
Rini Van Solingen and EgonW. Berghout. 1999. The Goal/Question/Metric Method: A Practical Guide for Quality Improvement of Software Development. McGraw-Hill.
Richard Y. Wang and Diane M. Strong. 1996. Beyond Accuracy: What Data Quality Means to Data Consumers. Journal of Management Information Systems 12, 4 (1996), 5–33. DOI: 10.1080/07421222.1996.11518099
Yichuan Wang and Nick Hajli. 2017. Exploring the Path to Big Data Analytics Success in Healthcare. Journal of Business Research 70 (2017), 287–299. DOI: 10.1016/j.jbusres.2016.08.002
Élcio Batista, Rossana Andrade, Ismayle Santos, Tales Nogueira, Pedro Oliveira, Valéria Lelli, and Victória Oliveira. 2024. Fortaleza City Hall Strategic Planning based on Data Analysis and Forecasting. In Anais do XXVII Congresso Ibero-Americano em Engenharia de Software (Curitiba/PR). SBC, Porto Alegre, RS, Brasil, 433–436. DOI: 10.5753/cibse.2024.28532
Published
2025-11-04
How to Cite
LELLI, Valéria et al.
Quality Management in the Public Sector: Definition and Prioritization of Indicators for Assessment and Decision-Making. In: BRAZILIAN SOFTWARE QUALITY SYMPOSIUM (SBQS), 24. , 2025, São José dos Campos/SP.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2025
.
p. 346-355.
DOI: https://doi.org/10.5753/sbqs.2025.15017.
