Enhancing the Mutation Testing Education through a Gamified Environment

• Dário Darienço UENP
• Alessandreia Marta de Oliveira UFJF
• Ricardo Ferreira Vilela UNICAMP
• Williamson Silva UFCA
• Maurício Massaru Arimoto UENP
• Pedro Henrique Valle USP

DOI: https://doi.org/10.5753/educomp.2026.18661

Resumo

Software testing plays a central role in quality assurance. However, challenges remain in teaching advanced concepts, such as mutation testing. The literature suggests that traditional methods do not always effectively support student engagement, motivating the adoption of active learning approaches. We present GAMutation, a gamified environment developed with the Construct 2 to support the teaching of mutation testing. To evaluate its usability, a heuristic evaluation was conducted with undergraduate students. The results indicated positive perceptions and highlighted aspects of the interface and interaction that may guide future improvements. The study reinforces the potential of gamification as a complementary strategy in software testing education.

Referências

Alzahrani, F. K. J. and Alhalafawy, W. S. (2022). Benefits and challenges of using gamification across distance learning platforms at higher education: a systematic review of research studies published during the covid-19 pandemic. Journal of Positive School Psychology, 6(10):1948–1977.

Autor, N. (2024). Mutant codebraker: Um jogo para apoiar o ensino de teste de mutação. Trabalho de Conclusão de Curso. Disponível em: repositório institucional.

Blanco, R., Trinidad, M., Suárez-Cabal, M. J., Calderón, A., Ruiz, M., and Tuya, J. (2023). Can gamification help in software testing education? findings from an empirical study. Journal of Systems and Software, 200:111647.

de Jesus, G. M., Ferrari, F. C., de Paula Porto, D., and Fabbri, S. C. P. F. (2018). Gamification in software testing: A characterization study. In Proceedings of the III Brazilian Symposium on Systematic and Automated Software Testing, pages 39–48.

de Souza, D. M., Maldonado, J. C., and Barbosa, E. F. (2012). Aspectos de desenvolvimento e evolução de um ambiente de apoio ao ensino de programação e teste de software. In Brazilian Symposium on Computers in Education (Simpósio Brasileiro de Informática na Educação-SBIE), volume 23.

Delamaro, M., Maldonado, J., and Jino, M. (2016). Introdução ao teste de software. Elsevier, Rio de Janeiro, 2 edição edition.

DeMillo, R. A., Lipton, R. J., and Perlis, A. J. (1979). Social processes and proofs of theorems and programs. Communications of the ACM, 22(5):271–280.

Deterding, S., Dixon, D., Khaled, R., and Nacke, L. (2011). From game design elements to gamefulness: defining" gamification". In Proceedings of the 15th international academic MindTrek conference: Envisioning future media environments, pages 9–15.

Elgrably, I. S. and de Oliveira, S. R. B. (2021). A diagnosis on software testing education in the brazilian universities. In 2021 IEEE Frontiers in Education Conference (FIE), pages 1–8. IEEE.

Farias, L., Santos, R., and Barbosa, D. (2021). Gamut: A game-based learning approach for teaching unit testing. In Proceedings of the Brazilian Symposium on Software Quality (SBQS), pages 120–131.

Florea, R. and Stray, V. (2019). The skills that employers look for in software testers. Software Quality Journal, 27(4):1449–1479.

Fraser, G. (2017). Gamification of software testing. In 2017 IEEE/ACM 12th International Workshop on Automation of Software Testing (AST), pages 2–7, Buenos Aires, Argentina. IEEE.

Fraser, G. and Arcuri, A. (2012). Whole test suite generation. IEEE Transactions on Software Engineering, 99.

Jalil, S., Rafi, S., LaToza, T. D., Moran, K., and Lam, W. (2023). Chatgpt and software testing education: Promises & perils. In 2023 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW), pages 4130–4137. IEEE.

Jia, Y. and Harman, M. (2011). An analysis and survey of the development of mutation testing. IEEE Transactions on Software Engineering, 37(5):649–678.

Lauvås Jr, P. and Arcuri, A. (2018). Recent trends in software testing education: A systematic literature review. In Norsk IKT-konferanse for forskning og utdanning.

Lima, J., Silva, M., and Oliveira, P. (2022). Protesters: A board game for teaching the software testing process. In Anais do Simpósio Brasileiro de Qualidade de Software (SBQS), pages 1–12.

Mäntylä, M. V. and Smolander, K. (2016). Gamification of software testing-an mlr. In Product-Focused Software Process Improvement: 17th International Conference, PROFES 2016, Trondheim, Norway, November 22-24, 2016, Proceedings 17, pages 611–614. Springer.

Martins, L., Brito, V., Feitosa, D., Rocha, L., Costa, H., and Machado, I. (2021). From blackboard to the office: A look into how practitioners perceive software testing education. In Evaluation and Assessment in Software Engineering, pages 211–220.

Nadi-Ravandi, S. and Batooli, Z. (2022). Gamification in education: A scientometric, content and co-occurrence analysis of systematic review and meta-analysis articles. Education and Information Technologies, 27(7):10207–10238.

Offutt, A. J. (1996). A practical system for mutation testing. Journal of Software Testing, Verification and Reliability, 6(1):3–23.

Rojas, J. M. and Fraser, G. (2016). Code defenders: A mutation testing game. In 2016 IEEE Ninth International Conference on Software Testing, Verification and Validation Workshops (ICSTW), pages 1–6, Chicago. IEEE.

Rojas, J. P. and Fraser, G. (2017). Code defenders: A mutation testing game. In Proceedings of the IEEE International Conference on Software Testing, Verification and Validation (ICST), pages 404–409.

Silva, A., Pereira, C., and Souza, B. (2023). An open educational resource supporting mutation testing teaching. In Anais do Simpósio Brasileiro de Qualidade de Software (SBQS), pages 1–10.

Silva, W., Gadelha, B., Steinmacher, I., and Conte, T. (2020). Towards an open repository for teaching software modeling applying active learning strategies. In 2020 IEEE/ACM 42nd International Conference on Software Engineering: Software Engineering Education and Training (ICSE-SEET), pages 162–172. IEEE.

Silva, W. A. F., Steinmacher, I. F., and Conte, T. U. (2017). Is it better to learn from problems or erroneous examples? In 2017 IEEE 30th Conference on Software Engineering Education and Training (CSEE&T), pages 222–231. IEEE.

Sommerville, I. (2011). Engenharia de software, 9a. São Palo, SP, Brasil, page 63.

Souza, É. R. and Souto, E. (2015). Utilizacao de heuristicas de jogos para avaliacao de um aplicativo gamificado. Proceedings of SBGames, pages 666–673.

Straubinger, P. and Fraser, G. (2022). Gamekins: Gamifying software testing in jenkins. In Proceedings of the ACM/IEEE 44th International Conference on Software Engineering: Companion Proceedings, pages 85–89.

Toda, A. M., Klock, A. C., Oliveira, W., Palomino, P. T., Rodrigues, L., Shi, L., Bittencourt, I., Gasparini, I., Isotani, S., and Cristea, A. I. (2019). Analysing gamification elements in educational environments using an existing gamification taxonomy. Smart Learning Environments, 6(1):1–14.

Valente, M. T. (2020). Engenharia de software moderna. Princípios e Práticas para Desenvolvimento de Software com Produtividade, 1.

Valle, P., Barbosa, E. F., and Maldonado, J. (2015). Um mapeamento sistemático sobre ensino de teste de software. In Brazilian Symposium on Computers in Education (Simpósio Brasileiro de Informática na Educação-SBIE), volume 26, page 71.

Valle, P. H. D. (2016). Jogos educacionais: uma contribuição para o ensino de teste de software. PhD thesis, Universidade de São Paulo.

Valle, P. H. D., Toda, A. M., Barbosa, E. F., and Maldonado, J. C. (2017). Educational games: A contribution to software testing education. In 2017 IEEE Frontiers in education Conference (FIE), pages 1–8. IEEE.

Wohlin, C., Runeson, P., Höst, M., Ohlsson, M. C., Regnell, B., and Wesslén, A. (2012). Experimentation in software engineering. Springer Science & Business Media.