Definição e Avaliação de uma Abordagem Gamificada para o Ensino de Teste de Software
Resumo
Testing is one of the most adopted practices in the software industry to reveal failures early in the development process and thus improve the quality of software in production. Despite their relevance, activities related to software testing are usually not taught and practiced enough in undergrad courses, and they are commonly considered demotivating. In this scenario, gamification has been a widely used approach, consisting of using common game elements in non-game contexts, which can make teaching activities and software testing practices more interesting. Therefore, the present work discusses how gamification was applied to support the Software Testing Discipline offered to undergraduate courses on Software Engineering, Computer Science, and Information Technology. A case study was conducted to evaluate the effects of the proposed gamified approach on students’ motivation and learning. In this study, the students had to create JUnit test cases applying domain partitioning criteria using both traditional and gamified approaches. We used Latin Square to perform the study. The results obtained from quantitative and qualitative analyses indicate that (i) there was no significant difference in test coverage when comparing both approaches, (ii) the gamified approach led to higher mutation scores, (iii) and gamification contributed positively to student motivation.
Palavras-chave:
Teste de Software, Ensino, Gamificação, Design Thinking, Relato de Experiência
Referências
Paola Accioly, Paulo Borba, and Rodrigo Bonifacio. 2012. Comparing two blackbox testing strategies for software product lines. In 2012 Sixth Brazilian Symposium on Software Components, Architectures and Reuse. IEEE, 1–10.
Paul Ammann and Jeff Offutt. 2016. Introduction to software testing. Cambridge University Press.
Tara Astigarraga, Eli M Dow, Christina Lara, Richard Prewitt, and Maria R Ward. 2010. The emerging role of software testing in curricula. In 2010 IEEE Transforming Engineering Education: Creating Interdisciplinary Skills for Complex Global Environments. IEEE, 1–26.
Raquel Blanco, Manuel Trinidad, María José Suárez-Cabal, Alejandro Calderón, Mercedes Ruiz, and Javier Tuya. 2023. Can gamification help in software testing education? Findings from an empirical study. Journal of Systems and Software 200 (2023), 111647.
Rodrigo Bonifácio, Paulo Borba, Cristiano Ferraz, and Paola Accioly. 2017. Empirical assessment of two approaches for specifying software product line use case scenarios. Software & Systems Modeling 16 (2017), 97–123.
George EP Box, J Stuart Hunter, and William G Hunter. 2005. Statistics for experimenters: design, discovery and innovation. Hoboken: Wiley-Interscience 639 (2005).
André Luiz de Souza Brito. 2017. Level Up: uma proposta de processo gamificado para a educação. Master’s thesis. Brasil.
Tim Brown. 2020. Design Thinking: uma metodologia poderosa para decretar o fim das velhas ideias. Alta Books.
Renée Bryce, Quentin Mayo, Aaron Andrews, Daniel Bokser, Michael Burton, Chelynn Day, Jessica Gonzolez, and Tara Noble. 2013. Bug catcher: A system for software testing competitions. In Proceeding of the 44th ACM technical symposium on Computer science education. 513–518.
Ilaria Caponetto, Jeffrey Earp, and Michela Ott. 2014. Gamification and education: A literature review. In European conference on games based learning, Vol. 1. Academic Conferences International Limited, 50.
Yu-kai Chou. 2019. Actionable gamification: Beyond points, badges, and leaderboards. Packt Publishing Ltd.
Peter J Clarke, Debra L Davis, Ingrid A Buckley, Geoff Potvin, Mandayam Thirunarayanan, and Edward L Jones. 2021. Combining learning and engagement strategies in a software testing learning environment. ACM Transactions on Computing Education (TOCE) 22, 2 (2021), 1–25.
Benjamin S Clegg, José Miguel Rojas, and Gordon Fraser. 2017. Teaching software testing concepts using a mutation testing game. In 2017 IEEE/ACM 39th International Conference on Software Engineering: Software Engineering Education and Training Track (ICSE-SEET). IEEE, 33–36.
Gabriela Martins de Jesus, Fabiano Cutigi Ferrari, Leo Natan Paschoal, Simone do Rocio Senger de Souza, Daniel de Paula Porto, and Vinicius Humberto Serapilha Durelli. 2020. Is it worth using gamification on software testing education? An extended experience report in the context of undergraduate students. Journal of Software Engineering Research and Development 8 (2020), 6–1.
EL Deci and RM Ryan. 2010. Intrinsic motivation. The corsini encyclopedia of psychology (pp. 1–2). Online Library: Wiley (2010).
Marcio Delamaro, Mario Jino, and Jose Maldonado. 2013. Introdução ao teste de software. Elsevier Brasil.
Sebastian Deterding. 2012. Gamification: designing for motivation. interactions 19, 4 (2012), 14–17.
Sebastian Deterding, Dan Dixon, Rilla Khaled, and Lennart Nacke. 2011. From game design elements to gamefulness: defining" gamification". In Proceedings of the 15th international academic MindTrek conference: Envisioning future media environments. 9–15.
Gordon Fraser. 2017. Gamification of software testing. In 2017 IEEE/ACM 12th International Workshop on Automation of Software Testing (AST). IEEE, 2–7.
Laura Inozemtseva and Reid Holmes. 2014. Coverage is not strongly correlated with test suite effectiveness. In Proceedings of the 36th international conference on software engineering. 435–445.
Karl M Kapp. 2012. The gamification of learning and instruction: game-based methods and strategies for training and education. John Wiley & Sons.
Mark R Lepper, David Greene, and Richard E Nisbett. 1973. Undermining children’s intrinsic interest with extrinsic reward: A test of the" overjustification" hypothesis. Journal of Personality and social Psychology 28, 1 (1973), 129.
ThomasWMalone. 1981. Toward a theory of intrinsically motivating instruction. Cognitive science 5, 4 (1981), 333–369.
Jane McGonigal. 2011. Reality is broken: Why games make us better and how they can change the world. Penguin.
Cristina Ioana Muntean. 2011. Raising engagement in e-learning through gamification. In Proc. 6th international conference on virtual learning ICVL, Vol. 1. 323–329.
Glenford J Myers, Corey Sandler, and Tom Badgett. 2011. The art of software testing. John Wiley & Sons.
Bret Pettichord, James Bach, and Cem Kaner. 2013. Lessons Learned in Software Testing: a Context-Driven Approach. Wiley.
Márcio Ribeiro, Paulo Borba, and Christian Kästner. 2014. Feature maintenance with emergent interfaces. In Proceedings of the 36th International Conference on Software Engineering. 989–1000.
José Miguel Rojas and Gordon Fraser. 2016. Code defenders: a mutation testing game. In 2016 IEEE Ninth International Conference on Software Testing, Verification and Validation Workshops (ICSTW). IEEE, 162–167.
Lilian Passos Scatalon, Jeffrey C Carver, Rogério Eduardo Garcia, and Ellen Francine Barbosa. 2019. Software testing in introductory programming courses: A systematic mapping study. In Proceedings of the 50th ACM Technical Symposium on Computer Science Education. 421–427.
Jorge Simões, Rebeca Díaz Redondo, and Ana Fernández Vilas. 2013. A social gamification framework for a K-6 learning platform. Computers in human behavior 29, 2 (2013), 345–353.
Kurt D Squire. 2008. Video games and education: Designing learning systems for an interactive age. Educational technology (2008), 17–26.
Steffen P Walz and Sebastian Deterding. 2015. The gameful world: Approaches, issues, applications. Mit Press.
Kevin Werbach and Dan Hunter. 2015. The gamification toolkit: dynamics, mechanics, and components for the win. University of Pennsylvania Press.
Paul Ammann and Jeff Offutt. 2016. Introduction to software testing. Cambridge University Press.
Tara Astigarraga, Eli M Dow, Christina Lara, Richard Prewitt, and Maria R Ward. 2010. The emerging role of software testing in curricula. In 2010 IEEE Transforming Engineering Education: Creating Interdisciplinary Skills for Complex Global Environments. IEEE, 1–26.
Raquel Blanco, Manuel Trinidad, María José Suárez-Cabal, Alejandro Calderón, Mercedes Ruiz, and Javier Tuya. 2023. Can gamification help in software testing education? Findings from an empirical study. Journal of Systems and Software 200 (2023), 111647.
Rodrigo Bonifácio, Paulo Borba, Cristiano Ferraz, and Paola Accioly. 2017. Empirical assessment of two approaches for specifying software product line use case scenarios. Software & Systems Modeling 16 (2017), 97–123.
George EP Box, J Stuart Hunter, and William G Hunter. 2005. Statistics for experimenters: design, discovery and innovation. Hoboken: Wiley-Interscience 639 (2005).
André Luiz de Souza Brito. 2017. Level Up: uma proposta de processo gamificado para a educação. Master’s thesis. Brasil.
Tim Brown. 2020. Design Thinking: uma metodologia poderosa para decretar o fim das velhas ideias. Alta Books.
Renée Bryce, Quentin Mayo, Aaron Andrews, Daniel Bokser, Michael Burton, Chelynn Day, Jessica Gonzolez, and Tara Noble. 2013. Bug catcher: A system for software testing competitions. In Proceeding of the 44th ACM technical symposium on Computer science education. 513–518.
Ilaria Caponetto, Jeffrey Earp, and Michela Ott. 2014. Gamification and education: A literature review. In European conference on games based learning, Vol. 1. Academic Conferences International Limited, 50.
Yu-kai Chou. 2019. Actionable gamification: Beyond points, badges, and leaderboards. Packt Publishing Ltd.
Peter J Clarke, Debra L Davis, Ingrid A Buckley, Geoff Potvin, Mandayam Thirunarayanan, and Edward L Jones. 2021. Combining learning and engagement strategies in a software testing learning environment. ACM Transactions on Computing Education (TOCE) 22, 2 (2021), 1–25.
Benjamin S Clegg, José Miguel Rojas, and Gordon Fraser. 2017. Teaching software testing concepts using a mutation testing game. In 2017 IEEE/ACM 39th International Conference on Software Engineering: Software Engineering Education and Training Track (ICSE-SEET). IEEE, 33–36.
Gabriela Martins de Jesus, Fabiano Cutigi Ferrari, Leo Natan Paschoal, Simone do Rocio Senger de Souza, Daniel de Paula Porto, and Vinicius Humberto Serapilha Durelli. 2020. Is it worth using gamification on software testing education? An extended experience report in the context of undergraduate students. Journal of Software Engineering Research and Development 8 (2020), 6–1.
EL Deci and RM Ryan. 2010. Intrinsic motivation. The corsini encyclopedia of psychology (pp. 1–2). Online Library: Wiley (2010).
Marcio Delamaro, Mario Jino, and Jose Maldonado. 2013. Introdução ao teste de software. Elsevier Brasil.
Sebastian Deterding. 2012. Gamification: designing for motivation. interactions 19, 4 (2012), 14–17.
Sebastian Deterding, Dan Dixon, Rilla Khaled, and Lennart Nacke. 2011. From game design elements to gamefulness: defining" gamification". In Proceedings of the 15th international academic MindTrek conference: Envisioning future media environments. 9–15.
Gordon Fraser. 2017. Gamification of software testing. In 2017 IEEE/ACM 12th International Workshop on Automation of Software Testing (AST). IEEE, 2–7.
Laura Inozemtseva and Reid Holmes. 2014. Coverage is not strongly correlated with test suite effectiveness. In Proceedings of the 36th international conference on software engineering. 435–445.
Karl M Kapp. 2012. The gamification of learning and instruction: game-based methods and strategies for training and education. John Wiley & Sons.
Mark R Lepper, David Greene, and Richard E Nisbett. 1973. Undermining children’s intrinsic interest with extrinsic reward: A test of the" overjustification" hypothesis. Journal of Personality and social Psychology 28, 1 (1973), 129.
ThomasWMalone. 1981. Toward a theory of intrinsically motivating instruction. Cognitive science 5, 4 (1981), 333–369.
Jane McGonigal. 2011. Reality is broken: Why games make us better and how they can change the world. Penguin.
Cristina Ioana Muntean. 2011. Raising engagement in e-learning through gamification. In Proc. 6th international conference on virtual learning ICVL, Vol. 1. 323–329.
Glenford J Myers, Corey Sandler, and Tom Badgett. 2011. The art of software testing. John Wiley & Sons.
Bret Pettichord, James Bach, and Cem Kaner. 2013. Lessons Learned in Software Testing: a Context-Driven Approach. Wiley.
Márcio Ribeiro, Paulo Borba, and Christian Kästner. 2014. Feature maintenance with emergent interfaces. In Proceedings of the 36th International Conference on Software Engineering. 989–1000.
José Miguel Rojas and Gordon Fraser. 2016. Code defenders: a mutation testing game. In 2016 IEEE Ninth International Conference on Software Testing, Verification and Validation Workshops (ICSTW). IEEE, 162–167.
Lilian Passos Scatalon, Jeffrey C Carver, Rogério Eduardo Garcia, and Ellen Francine Barbosa. 2019. Software testing in introductory programming courses: A systematic mapping study. In Proceedings of the 50th ACM Technical Symposium on Computer Science Education. 421–427.
Jorge Simões, Rebeca Díaz Redondo, and Ana Fernández Vilas. 2013. A social gamification framework for a K-6 learning platform. Computers in human behavior 29, 2 (2013), 345–353.
Kurt D Squire. 2008. Video games and education: Designing learning systems for an interactive age. Educational technology (2008), 17–26.
Steffen P Walz and Sebastian Deterding. 2015. The gameful world: Approaches, issues, applications. Mit Press.
Kevin Werbach and Dan Hunter. 2015. The gamification toolkit: dynamics, mechanics, and components for the win. University of Pennsylvania Press.
Publicado
30/09/2024
Como Citar
NASCIMENTO, Eduardo Henrique; COELHO, Roberta; MADEIRA, Charles; MEDEIROS, Kláudio; SILVA, Lucas; OLIVEIRA NETO, Carlos.
Definição e Avaliação de uma Abordagem Gamificada para o Ensino de Teste de Software. In: SIMPÓSIO BRASILEIRO DE ENGENHARIA DE SOFTWARE (SBES), 38. , 2024, Curitiba/PR.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2024
.
p. 455-465.
DOI: https://doi.org/10.5753/sbes.2024.3523.
