A Heuristics-Based Framework for User Satisfaction Evaluation Across the Lifecycle of Information Systems Projects
Resumo
Research Context: User satisfaction is recognized as a key success criterion in Information Systems (IS) projects. In the IS project context, continuously monitoring this criterion throughout the project lifecycle can reduce training and rework costs, enhance user experience, and prevent client loss. Scientific and/or Practical Problem: Despite its relevance, there is a lack of methods in the IS project management literature to evaluate user satisfaction during the development cycle. Existing approaches focus mainly on end-stage evaluations, limiting timely adjustments and evidence-based decision-making during project execution. Proposed Solution and/or Analysis: This paper proposes a framework for assessing user satisfaction in IS projects through three stages: expectation elicitation, expert inspection, and post-use verification. All stages are supported by questionnaires derived from usability heuristics to guide structured evaluation and comparison. Related IS Theory: The study is grounded in the Information Systems Success Model, which identifies user satisfaction as an essential dimension of project success, and integrates principles from Human–Computer Interaction (HCI). Research Method: An exploratory review of relevant literature was conducted, followed by the design of the proposed framework and its validation through three instrumental case studies across different organizational domains. Summary of Results: The results indicate that the proposed framework enables the evaluation of user satisfaction during system development, allowing early identification of usability issues and improvement opportunities. Contributions and Impact to the IS Area: This work contributes to IS project success management by operationalizing user satisfaction as a continuous and measurable criterion, ensuring its systematic verification throughout the project lifecycle and providing managers with a practical tool for evidence-based decision support.
Referências
Albert, W. and Tullis, T. (2023). Measuring the User Experience. Elsevier. Barbosa, S. D. J., Silva, B. S. d., Silveira, M. S., Gasparini, I., Darin, T., and Barbosa, G. D. J. (2021). Interação Humano-Computador e Experiência do Usuário. Autopublicação.
Benyon, D. (2011). Interação Humano-Computador. Pearson Prentice Hall, São Paulo, 2 edition.
Bevan, N. (2010). Extending the concept of satisfaction in iso standards. In Proceedings of the KEER 2010 International Conference on Kansei Engineering and Emotion Research.
Carvalho, J. (1998). Using the viable system model to describe the role of computer-based systems in organizations. In Callaos, N., L. Holmes & R. Osers, Proceedings of the World Multiconference on Systems, Cybernetics and Informatics, Orlando, Florida, USA, volume 4, pages 497–502.
Carvalho, J. A. (2000). Information System? Which One Do You Mean?, pages 259–277. Springer US, Boston, MA.
Chang, S.-I., Wu, H.-C., Chang, I.-C., and Wang, Y.-K. (2011). Hci-based erp auditing system usability assessment framework. Cited by: 1.
Choma, J., Quintale, D., Zaina, L. A. M., and Beraldo, D. (2015). A perspective-based usability inspection for erp systems. In Proceedings of the 17th International Conference on Enterprise Information Systems - Volume 3: ICEIS,, pages 57–64. INSTICC, SciTePress.
DeLone, W. H. and McLean, E. R. (1992). Information systems success: The quest for the dependent variable. Information systems research, 3(1):60–95.
Dresch, A., Lacerda, D. P., and Junior, J. A. V. A. (2015). Design science research: método de pesquisa para avanço da ciência e tecnologia. Bookman Editora.
Faisal, C. M. N., Shakeel Faridi, M., and Javed, Z. (2011). Usability evaluation of in-housed developed erp system. volume 8285. Cited by: 6.
Faisal, C. M. N., Tariq, S., Ahtram, T., Abbasi, M. S., Sarwer, S., and Selamat, A. (2012). A novel usability matrix for erp systems using heuristic approach. page 291 – 296. Cited by: 4.
Hevner, A. R., March, S. T., Park, J., and Ram, S. (2004). Design science in information systems research. MIS quarterly, pages 75–105.
ISO (2019). ISO 9241-210:2019 - Ergonomics of human-system interaction – Part 210: Human-centred design for interactive systems. [link]. Accessed: 2025-05-01.
Kumar, A., Tadayoni, R., and Sorensen, L. T. (2016). Metric based efficiency analysis of educational erp system usability-using fuzzy model. page 382 – 386. Institute of Electrical and Electronics Engineers Inc. Cited by: 7.
Kurniawan, I., Ardianto, Y. T., and Hidayatullah, S. (2021). The effect of the information system quality, service quality, and user satisfaction on academic information system user loyalty. International Journal of Scientific and Technology Research, 10(5):350–355.
Lobato, B. S. d. A. L. (2021). Modelo de critérios para avaliação do sucesso em projetos de sistemas de informação. Master’s thesis, Universidade do Minho (Portugal).
Mir, F. A. and Pinnington, A. H. (2014). Exploring the value of project management: linking project management performance and project success. International journal of project management, 32(2):202–217.
Nielsen, J. (1994). Usability engineering. Morgan Kaufmann. Nielsen, J. (2024). 10 usability heuristics for user interface design. Acesso em: 7 fev. 2026.
Omar, K., Rapp, B., and Gomez, J. M. (2016). Heuristic evaluation checklist for mobile erp user interfaces. page 180 – 185. Cited by: 10.
Pereira, R. and Baranauskas, M. C. C. (2017). Systemic and socially aware perspective for information systems. I GranDSI-BR, page 148.
PMI (2023). Process Groups: A Practice Guide. Project Management Institute, Inc., Newtown Square, PA.
Polancos, R. V. (2019). A usability study of an enterprise resource planning system: A case study on sap business one. Advances in Intelligent Systems and Computing, 824:1203 – 1223. Cited by: 5.
Rocha, H. V. and Baranauskas, M. C. C. (2003). Design e avaliação de interfaces humano-computador. Campinas: Unicamp.
Scholtz, B., Cilliers, C., and Calitz, A. (2010). Qualitative techniques for evaluating enterprise resource planning (erp) user interfaces. page 284 – 293. Cited by: 20.
Shneiderman, B. and Plaisant, C. (2005). Designing the User Interface. Pearson Education, Inc., 4 edition.
Singh, A. and Wesson, J. (2009). Evaluation criteria for assessing the usability of erp systems. In Proceedings of the 2009 Annual Research Conference of the South African Institute of Computer Scientists and Information Technologists, SAICSIT ’09, page 87–95, New York, NY, USA. Association for Computing Machinery.
Sonali, S. and Thamada, S. (2024). FR NFR dataset. Takagi, N. and Ventura, T. (2024). Artificial intelligence insights on success evaluation in information systems government-to-government projects. In MCIS 2024 Proceedings, volume 19.
Varajão, J., Magalhães, L., Freitas, L., and Rocha, P. (2022). Success management–from theory to practice. International Journal of Project Management, 40(5):481–498.
Varajão, J. (2016). Success management as a pm knowledge area – work-in-progress. Procedia Computer Science, 100:1095–1102. International Conference on ENTERprise Information Systems/International Conference on Project MANagement/International Conference on Health and Social Care Information Systems and Technologies, CENTERIS/ProjMAN / HCist 2016.
