Learning through Practice: Teaching Empirical Software Engineering for Undergraduate Students

• Ranya Duran Greco UFC
• Sofia Bento Desidério UFC
• Caio César Sousa Bandeira UFC
• Anna Beatriz dos Santos Marques UFC

DOI: https://doi.org/10.5753/sbes.2025.11268

Abstract

Empirical Software Engineering (ESE) is essential for understanding the benefits and limitations of existing software engineering solutions. The evidence obtained through experiments guides the evolution of this area and optimizes the use of effort in relevant research that solves real problems or gaps. In ESE teaching, the challenge arises of preparing future software engineers to plan, conduct, and analyze experiments. This article reports on an experience of teaching ESE to 54 undergraduate students in an elective course, using Active Learning. To this end, we describe the activities carried out during the course, and to understand the effect of Active Learning, we conducted a survey using an online questionnaire. Twenty-seven students from the class consented to take part in the survey. The results show that 77.7% of the respondents had little or no experience of ESE before the course. At the end of the course, around 88.2% reported greater confidence in carrying out research in the area. The hypothesis test showed a significant increase in knowledge of the course’s content, emphasizing the topics of surveys and empirical studies. Correlation analysis revealed that the perceived contribution of the course is positively associated with student motivation. Despite some challenges, the students highlighted the methodology applied as being of great value to their training and entry into the job market. The experience of using Active Learning to teach ESE showed that, as well as contributing to the students’ education, the methodology gave them more confidence and motivation, which suggests the importance of pedagogical approaches that are closer to practice.

Keywords: Empirical Software Engineering, Teaching, Education, Active Learning, Undergraduate Student

References

Victor R Basili. 1996. The role of experimentation in software engineering: past, current, and future. In Proceedings of IEEE 18th International Conference on Software Engineering. IEEE, 442–449.

Victor R Basili, Forrest Shull, and Filippo Lanubile. 1999. Building knowledge through families of experiments. IEEE Transactions on Software Engineering 25, 4 (1999), 456–473.

John W Creswell. 2007. An introduction to mixed methods research. Lincoln, Nebraska, USA: University of Nebraska 23 (2007).

Elias José de Rezende Freitas and Laís Sergiane Fortes. 2020. Aprendizagem ativa aplicada à Engenharia: um estudo sobre a percepção do aprendizado. Revista de Ensino de Engenharia 39 (2020).

Sofia Bento Desidério, Maria Rebecca Lopes Lelis, Maria Elanne Rodrigues, and Anna Beatriz Marques. 2024. How ready for hci? a qualitative analysis of the practice of soft skills related to hci by women involved in the digital girls program partners projects. Journal on Interactive Systems 15, 1 (2024), 504–516.

Michael Felderer and Marco Kuhrmann. 2019. Using Mini-Projects to Teach Empirical Software Engineering.. In SEUH. 75–86.

Thaís Ferreira, Davi Viana, Juliana Fernandes, and Rodrigo Santos. 2018. Identifying emerging topics and difficulties in software engineering education in Brazil. In Proceedings of the XXXII Brazilian Symposium on Software Engineering. 230–239.

Clara Lima Fonseca, Luciana AM Zaina, and Anna Beatriz Marques. 2024. Avaliação de Métodos para Elicitação e Especificação de Requisitos de Usabilidade com Histórias de Usuário: Um Experimento Controlado. In Simpósio Brasileiro de Engenharia de Software (SBES). SBC, 257–268.

Vahid Garousi, Gorkem Giray, Eray Tuzun, Cagatay Catal, and Michael Felderer. 2019. Closing the gap between software engineering education and industrial needs. IEEE software 37, 2 (2019), 68–77.

Lucas Gren. 2020. A flipped classroom approach to teaching empirical software engineering. IEEE Transactions on Education 63, 3 (2020), 155–163.

Marcela Hernández-de Menéndez, Antonio Vallejo Guevara, Juan Carlos Tudón Martínez, Diana Hernández Alcántara, and Ruben Morales-Menendez. 2019. Active learning in engineering education. A review of fundamentals, best practices and experiences. International Journal on Interactive Design and Manufacturing (IJIDeM) 13 (2019), 909–922.

Barbara Kitchenham. 2004. Procedures for performing systematic reviews. Keele, UK, Keele University 33, 2004 (2004), 1–26.

Barbara A Kitchenham and Shari L Pfleeger. 2008. Personal opinion surveys. In Guide to advanced empirical software engineering. Springer, 63–92.

Carlos Danilo Luz, Edson OliveiraJr, and Igor F Steinmacher. 2022. How Experimentation in Software Engineering has been taught? Survey and Research Agenda. In Proceedings of the 26th International Conference on Evaluation and Assessment in Software Engineering. 299–304.

Maíra R Marques, Alcides Quispe, and Sergio F Ochoa. 2014. A systematic mapping study on practical approaches to teaching software engineering. In 2014 IEEE Frontiers in education conference (FIE) proceedings. IEEE, 1–8.

Suéllen Martinelli, Joelma Choma, Jullia Saad, and Luciana Zaina. 2023. Análise Qualitativa em IHC: da codificação à criação de visualizações. In Simpósio Brasileiro sobre Fatores Humanos em Sistemas Computacionais (IHC). SBC, 9–10.

Maria Alcimar Costa Meireles, Sabrina Rocha, Jose Carlos Maldonado, and Tayana Conte. 2024. An experience report on the use of Active Learning in Empirical Software Engineering Education: Understanding the pros and cons from the student’s perspective. In Proceedings of the 46th International Conference on Software Engineering: Software Engineering Education and Training. 380–390.

Duncan D Nulty. 2008. The adequacy of response rates to online and paper surveys: what can be done? Assessment & evaluation in higher education 33, 3 (2008), 301–314.

Victor R Basili Dieter Rombach, Kurt Schneider Barbara Kitchenham, and Dietmar Pfahl Richard W Selby. 2007. Empirical Software Engineering Issues. Springer.

Johnny Saldaña. 2021. The coding manual for qualitative researchers. (2021).

Cynthia Pinheiro Santiago, José Wally Mendonça Menezes, and Francisco José Alves de Aquino. 2023. Proposta e Avaliação de uma Metodologia de Aprendizagem Baseada em Projetos em Disciplinas de Engenharia de Software através de uma Sequência Didática. Revista Brasileira de Informática na Educação 31 (2023), 31–59.

Adrian Santos, Omar Gómez, and Natalia Juristo. 2018. Analyzing families of experiments in SE: A systematic mapping study. IEEE Transactions on Software Engineering 46, 5 (2018), 566–583.

Sindiany Santos, Maria Silva, Caroline Caputo, Elaine dos Santos, Anézia Barbosa, and Jaqson Santos. 2023. Aprendizagem Ativa Experiências teórico-práticas no ensino superior. DOI: 10.47247/SSCS/88471.87.6

Samuel S. Shapiro and Martin B. Wilk. 1965. An analysis of variance test for normality (complete samples). Biometrika 52, 3/4 (1965), 591–611.

Forrest Shull, Janice Singer, and Dag IK Sjøberg. 2008. Guide to advanced empirical software engineering. Vol. 93. Springer.

Charles Spearman. 1904. The proof and measurement of association between two things. The American Journal of Psychology 15, 1 (1904), 72–101.

Student. 1908. The probable error of a mean. Biometrika 6, 1 (1908), 1–25.

Antonio Karlos Araújo Valença. 2023. Metodologias ativas no ensino de engenharia: uma revisão bibliométrica. Revista Produção Online 23, 2 (2023), 4982–4982.

Jéssyka Vilela and Carla Silva. 2024. Empowering Undergraduates in Empirical Research Methods: an Experience Report. In Simpósio Brasileiro de Engenharia de Software (SBES). SBC, 543–553.

Frank Wilcoxon. 1945. Individual Comparisons by Ranking Methods. Biometrics Bulletin 1, 6 (1945), 80–83.

Claes Wohlin. 2007. Empirical software engineering: Teaching methods and conducting studies. In Empirical Software Engineering Issues. Critical Assessment and Future Directions: International Workshop, Dagstuhl Castle, Germany, June 26-30, 2006. Revised Papers. Springer, 135–142.

Claes Wohlin, Per Runeson, Martin Höst, Magnus C Ohlsson, Björn Regnell, Anders Wesslén, et al. 2012. Experimentation in software engineering. Vol. 236. Springer.

He Zhang, Muhammad Ali Babar, and Paolo Tell. 2011. Identifying relevant studies in software engineering. Information and Software Technology 53, 6 (2011), 625–637.