Using Real-world Scenarios to Bridge Training and Practice in Evidence-Based Software Engineering
Resumo
Context: Training is crucial for the adoption of Evidence-Based Software Engineering (EBSE), but there is a gap in its integration with professional practice. Objective: To investigate the creation and use of scenarios that enhance EBSE training by simulating the application of evidence in real-world situations. Method: A mixed methods study was conducted, starting with consultations with three Uruguayan industry and government practitioners to create a practical scenario, followed by its use to improve an EBSE course. Results: Scenario development is challenging, primarily due to limitations of evidence, but their use enhances students’ ability to apply evidence. Conclusions: Using scenarios to align EBSE training with practice seems to be feasible & effective.
Palavras-chave:
Evidence-based Software Engineering, Mixed Methods Study, Training, Real-World Scenarios, Use of Evidence
Referências
Atkinson, P. and Hammersley, M. (1994). Ethnography and participant observation. In Denzin, N. and Lincoln, Y., editors, Handbook of Qualitative Research. Thousand Oaks, CA.
Bano, M. and Zowghi, D. (2015). A systematic review on the relationship between user involvement and system success. Information and Software Technology, 58.
Bloom, B. (1956). Taxonomy of educational objectives, the classification of educational Goals, handbook I: Cognitive domain. D. McKay.
Braun, V. and Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3(2).
Budgen, D., Brereton, P., Williams, N., and Drummond, S. (2020). What support do systematic reviews provide for evidence-informed teaching about software engineering practice? E-Informatica Software Engineering Journal, 14(1).
Cartaxo, B., Pinto, G., and Soares, S. (2018). The role of rapid reviews in supporting decision-making in software engineering practice. In International Conference on Evaluation and Assessment in Software Engineering.
Coomarasamy, A. and Khan, K. (2004). What is the evidence that postgraduate teaching in evidence based medicine changes anything? a systematic review. BMJ (Clinical research ed.), 329.
Da Silva, F., Santos, A., Soares, S., Frana, A., Monteiro, C., and Maciel, F. (2011). Six years of systematic literature reviews in software engineering: An updated tertiary study. Information and Software Technology, 53(9).
Davison, R., Martinsons, M. G., and Kock, N. (2004). Principles of canonical action research. Information Systems Journal, 14(1).
Hassler, E., Carver, J., Kraft, N., and Hale, D. (2014). Outcomes of a community workshop to identify and rank barriers to the systematic literature review process. In International Conference on Evaluation and Assessment in Software Engineering.
Ilic, D. and Maloney, S. (2014). Methods of teaching medical trainees evidence-based medicine: A systematic review. Medical Education, 48(2).
Jørgensen, M., Dybå, T., and Kitchenham, B. (2005). Teaching evidence-based software engineering to university students. In International Software Metrics Symposium.
Kamei, F., Wiese, I., Lima, C., Polato, I., Nepomuceno, V., Ferreira, W., Ribeiro, M., Pena, C., Cartaxo, B., Pinto, G., and Soares, S. (2021). Grey Literature in Software Engineering: A critical review. Information and Software Technology, 138.
Kamei, F., Wiese, I., Neto, C., Polato, I., Nepomuceno, V., Ferreira, W., Ribeiro, M., Pena, C., Cartaxo, B., Pinto, G., and Soares, S. (2022). Grey Literature in Software Engineering: A Critical Review.
Kasoju, A., Petersen, K., and Mäntylä, M. (2013). Analyzing an automotive testing process with evidence-based software engineering. Information and Software Technology, 55(7).
Kennedy, D., Hyl, Á., Ryan, N., Hyland, Á., and Ryan, N. (2007). Writing and using learning outcomes: A practical guide, chapter C 3.4-1.
Kitchenham, B., Budgen, D., and Brereton, P. (2015). Evidence-based software engineering and systematic reviews. Chapman & Hall/CRC Innovations in Software Engineering and Software Development Series. CRC Press, London.
Kitchenham, B., Dybå, T., and Jørgensen, M. (2004). Evidence-based software engineering. In International Conference on Software Engineering.
Kitson, A., Harvey, G., and Mccormack, B. (1998). Enabling the implementation of evidence based practice: a conceptual framework. Quality in Health Care, 7.
Larsen, C. M., Terkelsen, A. S., Carlsen, A. M. F., and Kristensen, H. K. (2019). Methods for teaching evidence-based practice: A scoping review. BMC Medical Education, 19(1).
Manns, P. and Darrah, J. (2012). A structured process to develop scenarios for use in evaluation of an evidence-based approach in clinical decision making. Advances in Medical Education and Practice, 3.
Milewicz, R. M., Bisila, J. D., Mundt, M. R., and Teves, J. B. (2023). Seeking enlightenment: Incorporating evidence-based practice techniques in a research software engineering team. In Conference of the United States Research Software Engineer Association.
Pacheco, C. and Garcia, I. (2012). A systematic literature review of stakeholder identification methods in requirements elicitation. Journal of Systems and Software, 85(9).
Pizard, S., Acerenza, F., Otegui, X., Moreno, S., Vallespir, D., and Kitchenham, B. (2021). Training students in evidence-based software engineering and systematic reviews: a systematic review and empirical study. Empirical Software Engineering, 26(3).
Pizard, S., Acerenza, F., Vallespir, D., and Kitchenham, B. (2023). Assessing attitudes towards evidence-based software engineering in a government agency. Information and Software Technology, 154.
Pizard, S., Lezama, J., García, R., Vallespir, D., and Kitchenham, B. (2025). Using rapid reviews to support software engineering practice: a systematic review and a replication study. Empirical Software Engineering, 30(1).
Pizard, S. and Marenco, L. (2025). Data & Codification from Stage 2 of the study “Using Real-world Scenarios to Bridge Training and Practice in Evidence-Based Software Engineering”.
Pizard, S., Vallespir, D., and Kitchenham, B. (2022). A longitudinal case study on the effects of an evidence-based software engineering training. In IEEE/ACM International Conference on Software Engineering: Software Engineering Education and Training (ICSE-SEET).
Rainer, A. and Beecham, S. (2008). A follow-up empirical evaluation of evidence based software engineering by undergraduate students. In International Conference on Evaluation and Assessment in Software Engineering.
Rainer, A., Hall, T., and Baddoo, N. (2006). A preliminary empirical investigation of the use of evidence based software engineering by undergraduate students. In International Conference on Evaluation and Assessment in Software Engineering.
Sadeghi-Bazargani, H., Tabrizi, J. S., and Azami-Aghdash, S. (2014). Barriers to evidence-based medicine: A systematic review. Journal of Evaluation in Clinical Practice, 20(6).
Scurlock-Evans, L. and Upton, D. (2015). The role and nature of evidence: A systematic review of social workers’ evidence-based practice orientation, attitudes, and implementation. Journal of Evidence-Informed Social Work, 12(4).
Stol, K.-J. and Fitzgerald, B. (2018). The ABC of Software Engineering Research. ACM Transactions on Software Engineering and Methodology, 27(3).
Swanberg, S. M., Dennison, C. C., Farrell, A., Machel, V., Marton, C., O’Brien, K. K., Pannabecker, V., Thuna, M., and Holyoke, A. N. (2016). Instructional methods used by health sciences librarians to teach evidence-based practice (EBP): a systematic review. J. Med. Libr. Assoc., 104(3).
Upton, D., Stephens, D., Williams, B., and Scurlock-Evans, L. (2014). Occupational therapists’ attitudes, knowledge, and implementation of evidence-based practice: A systematic review of published research. British Journal of Occupational Therapy, 77(1).
Venkatesh, V., Brown, S. A., and Bala, H. (2013). Bridging the qualitative-quantitative divide: guidelines for conducting mixed methods research in information systems. MIS Q., 37(1).
Wilson, G., Aranda, J., Hoye, M., and Johnson, B. (2024). Experience Report: It Will Never Work in Theory. IEEE Software, 41(03).
Zwolsman, S., Te Pas, E., Hooft, L., Wieringa-De Waard, M., and Van Dijk, N. (2012). Barriers to gps’ use of evidence-based medicine: A systematic review. British Journal of General Practice, 62(600).
Bano, M. and Zowghi, D. (2015). A systematic review on the relationship between user involvement and system success. Information and Software Technology, 58.
Bloom, B. (1956). Taxonomy of educational objectives, the classification of educational Goals, handbook I: Cognitive domain. D. McKay.
Braun, V. and Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3(2).
Budgen, D., Brereton, P., Williams, N., and Drummond, S. (2020). What support do systematic reviews provide for evidence-informed teaching about software engineering practice? E-Informatica Software Engineering Journal, 14(1).
Cartaxo, B., Pinto, G., and Soares, S. (2018). The role of rapid reviews in supporting decision-making in software engineering practice. In International Conference on Evaluation and Assessment in Software Engineering.
Coomarasamy, A. and Khan, K. (2004). What is the evidence that postgraduate teaching in evidence based medicine changes anything? a systematic review. BMJ (Clinical research ed.), 329.
Da Silva, F., Santos, A., Soares, S., Frana, A., Monteiro, C., and Maciel, F. (2011). Six years of systematic literature reviews in software engineering: An updated tertiary study. Information and Software Technology, 53(9).
Davison, R., Martinsons, M. G., and Kock, N. (2004). Principles of canonical action research. Information Systems Journal, 14(1).
Hassler, E., Carver, J., Kraft, N., and Hale, D. (2014). Outcomes of a community workshop to identify and rank barriers to the systematic literature review process. In International Conference on Evaluation and Assessment in Software Engineering.
Ilic, D. and Maloney, S. (2014). Methods of teaching medical trainees evidence-based medicine: A systematic review. Medical Education, 48(2).
Jørgensen, M., Dybå, T., and Kitchenham, B. (2005). Teaching evidence-based software engineering to university students. In International Software Metrics Symposium.
Kamei, F., Wiese, I., Lima, C., Polato, I., Nepomuceno, V., Ferreira, W., Ribeiro, M., Pena, C., Cartaxo, B., Pinto, G., and Soares, S. (2021). Grey Literature in Software Engineering: A critical review. Information and Software Technology, 138.
Kamei, F., Wiese, I., Neto, C., Polato, I., Nepomuceno, V., Ferreira, W., Ribeiro, M., Pena, C., Cartaxo, B., Pinto, G., and Soares, S. (2022). Grey Literature in Software Engineering: A Critical Review.
Kasoju, A., Petersen, K., and Mäntylä, M. (2013). Analyzing an automotive testing process with evidence-based software engineering. Information and Software Technology, 55(7).
Kennedy, D., Hyl, Á., Ryan, N., Hyland, Á., and Ryan, N. (2007). Writing and using learning outcomes: A practical guide, chapter C 3.4-1.
Kitchenham, B., Budgen, D., and Brereton, P. (2015). Evidence-based software engineering and systematic reviews. Chapman & Hall/CRC Innovations in Software Engineering and Software Development Series. CRC Press, London.
Kitchenham, B., Dybå, T., and Jørgensen, M. (2004). Evidence-based software engineering. In International Conference on Software Engineering.
Kitson, A., Harvey, G., and Mccormack, B. (1998). Enabling the implementation of evidence based practice: a conceptual framework. Quality in Health Care, 7.
Larsen, C. M., Terkelsen, A. S., Carlsen, A. M. F., and Kristensen, H. K. (2019). Methods for teaching evidence-based practice: A scoping review. BMC Medical Education, 19(1).
Manns, P. and Darrah, J. (2012). A structured process to develop scenarios for use in evaluation of an evidence-based approach in clinical decision making. Advances in Medical Education and Practice, 3.
Milewicz, R. M., Bisila, J. D., Mundt, M. R., and Teves, J. B. (2023). Seeking enlightenment: Incorporating evidence-based practice techniques in a research software engineering team. In Conference of the United States Research Software Engineer Association.
Pacheco, C. and Garcia, I. (2012). A systematic literature review of stakeholder identification methods in requirements elicitation. Journal of Systems and Software, 85(9).
Pizard, S., Acerenza, F., Otegui, X., Moreno, S., Vallespir, D., and Kitchenham, B. (2021). Training students in evidence-based software engineering and systematic reviews: a systematic review and empirical study. Empirical Software Engineering, 26(3).
Pizard, S., Acerenza, F., Vallespir, D., and Kitchenham, B. (2023). Assessing attitudes towards evidence-based software engineering in a government agency. Information and Software Technology, 154.
Pizard, S., Lezama, J., García, R., Vallespir, D., and Kitchenham, B. (2025). Using rapid reviews to support software engineering practice: a systematic review and a replication study. Empirical Software Engineering, 30(1).
Pizard, S. and Marenco, L. (2025). Data & Codification from Stage 2 of the study “Using Real-world Scenarios to Bridge Training and Practice in Evidence-Based Software Engineering”.
Pizard, S., Vallespir, D., and Kitchenham, B. (2022). A longitudinal case study on the effects of an evidence-based software engineering training. In IEEE/ACM International Conference on Software Engineering: Software Engineering Education and Training (ICSE-SEET).
Rainer, A. and Beecham, S. (2008). A follow-up empirical evaluation of evidence based software engineering by undergraduate students. In International Conference on Evaluation and Assessment in Software Engineering.
Rainer, A., Hall, T., and Baddoo, N. (2006). A preliminary empirical investigation of the use of evidence based software engineering by undergraduate students. In International Conference on Evaluation and Assessment in Software Engineering.
Sadeghi-Bazargani, H., Tabrizi, J. S., and Azami-Aghdash, S. (2014). Barriers to evidence-based medicine: A systematic review. Journal of Evaluation in Clinical Practice, 20(6).
Scurlock-Evans, L. and Upton, D. (2015). The role and nature of evidence: A systematic review of social workers’ evidence-based practice orientation, attitudes, and implementation. Journal of Evidence-Informed Social Work, 12(4).
Stol, K.-J. and Fitzgerald, B. (2018). The ABC of Software Engineering Research. ACM Transactions on Software Engineering and Methodology, 27(3).
Swanberg, S. M., Dennison, C. C., Farrell, A., Machel, V., Marton, C., O’Brien, K. K., Pannabecker, V., Thuna, M., and Holyoke, A. N. (2016). Instructional methods used by health sciences librarians to teach evidence-based practice (EBP): a systematic review. J. Med. Libr. Assoc., 104(3).
Upton, D., Stephens, D., Williams, B., and Scurlock-Evans, L. (2014). Occupational therapists’ attitudes, knowledge, and implementation of evidence-based practice: A systematic review of published research. British Journal of Occupational Therapy, 77(1).
Venkatesh, V., Brown, S. A., and Bala, H. (2013). Bridging the qualitative-quantitative divide: guidelines for conducting mixed methods research in information systems. MIS Q., 37(1).
Wilson, G., Aranda, J., Hoye, M., and Johnson, B. (2024). Experience Report: It Will Never Work in Theory. IEEE Software, 41(03).
Zwolsman, S., Te Pas, E., Hooft, L., Wieringa-De Waard, M., and Van Dijk, N. (2012). Barriers to gps’ use of evidence-based medicine: A systematic review. British Journal of General Practice, 62(600).
Publicado
12/05/2025
Como Citar
PIZARD, Sebastián; MARENCO, Leticia.
Using Real-world Scenarios to Bridge Training and Practice in Evidence-Based Software Engineering. In: CONGRESSO IBERO-AMERICANO EM ENGENHARIA DE SOFTWARE (CIBSE), 28. , 2025, Ciudad Real/Espanha.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2025
.
p. 1-15.
DOI: https://doi.org/10.5753/cibse.2025.35288.
