Avaliação da Qualidade de Testes Elaborados por Estudantes na Resolução de Problemas de Programação
Resumo
Resolver problemas de programação exige compreender enunciados e validar soluções por meio de testes. No entanto, muitos estudantes apresentam dificuldades em elaborar testes que reflitam corretamente o comportamento esperado. Este estudo investiga a qualidade dos testes produzidos durante a resolução de problemas, analisando sua capacidade de identificar soluções incorretas. A análise considera métricas de desempenho e aspectos estruturais, como classes de equivalência e taxa de redundância. Os resultados indicam boa capacidade de detecção de falhas; no entanto, uma parcela significativa dos testes é redundante ou apresenta baixa contribuição para a identificação de comportamentos incorretos.
Palavras-chave:
Ensino de Programação, Teste de Software, Qualidade de Testes
Referências
Almeida, A., Guerrero, D., and Andrade, W. (2025). Avaliando o efeito da criação de testes na aprendizagem e no desempenho de estudantes de programação. In Anais do XXXVI Simpósio Brasileiro de Informática na Educação, pages 140–152, Porto Alegre, RS, Brasil. SBC.
Ammann, P. and Offutt, J. (2017). Introduction to software testing. Cambridge University Press.
Andleeb, S., Mendoza, T., Cordova, L. P., Walia, G., and Carver, J. (2025). Enhancing software testing education: Understanding where students struggle. In Proceedings of the 26th ACM Annual Conference on Cybersecurity & Information Technology Education, pages 154–160.
Denny, P., Kumar, V., MacNeil, S., Prather, J., and Leinonen, J. (2025). Probing the unknown: Exploring student interactions with probeable problems at scale in introductory programming. In Proceedings of the 30th ACM Conference on Innovation and Technology in Computer Science Education V. 1, pages 618–624.
Edwards, S. H. (2004). Using software testing to move students from trial-and-error to reflection-in-action. In Proceedings of the 35th SIGCSE technical symposium on Computer science education, pages 26–30.
Marijan, D. and Sen, S. (2017). Detecting and reducing redundancy in software testing for highly configurable systems. In 2017 IEEE 18th International Symposium on High Assurance Systems Engineering (HASE), pages 96–99. IEEE.
Mohammadian, M. and Javed, Z. (2021). Intelligent evaluation of test suites for developing efficient and reliable software. International Journal of Parallel, Emergent and Distributed Systems, 36(2):159–188.
Neto, F. G. D. O., Feldt, R., Erlenhov, L., and Nunes, J. B. D. S. (2018). Visualizing test diversity to support test optimisation. In 2018 25th Asia-Pacific Software Engineering Conference (APSEC), pages 149–158. IEEE.
Perretta, J., DeOrio, A., Guha, A., and Bell, J. (2022). On the use of mutation analysis for evaluating student test suite quality. In Proceedings of the 31st ACM SIGSOFT International Symposium on Software Testing and Analysis, pages 263–275.
Potter, H., Paiva, A. C., Amalfitano, D., Fasolino, A. R., Tramontana, P., and Just, R. (2025). Evaluating the impact of scaffolding and visualizations for mutation testing exercises in software engineering education. In Proceedings of the 33rd ACM International Conference on the Foundations of Software Engineering, pages 823–834.
Robins, A., Rountree, J., and Rountree, N. (2003). Learning and teaching programming: A review and discussion. Computer science education, 13(2):137–172.
Shams, Z. and Edwards, S. H. (2013). Toward practical mutation analysis for evaluating the quality of student-written software tests. In Proceedings of the ninth annual international ACM conference on International computing education research, pages 53–58.
Showler, A., Miljanovic, M. A., and Bradbury, J. S. (2025). How effective and efficient are student-written software tests? In Proceedings of the 56th ACM Technical Symposium on Computer Science Education V. 1, pages 1057–1063.
Vasconcelos, P. (2025). Exploring mutation testing for teaching introductory programming. In 6th International Computer Programming Education Conference (ICPEC 2025), pages 1–1. Schloss Dagstuhl–Leibniz-Zentrum für Informatik.
Watson, C. and Li, F. W. (2014). Failure rates in introductory programming revisited. In Proceedings of the 2014 conference on Innovation & technology in computer science education, pages 39–44.
Wrenn, J. and Krishnamurthi, S. (2019). Executable examples for programming problem comprehension. In Proceedings of the 2019 ACM conference on international computing education research, pages 131–139.
Ammann, P. and Offutt, J. (2017). Introduction to software testing. Cambridge University Press.
Andleeb, S., Mendoza, T., Cordova, L. P., Walia, G., and Carver, J. (2025). Enhancing software testing education: Understanding where students struggle. In Proceedings of the 26th ACM Annual Conference on Cybersecurity & Information Technology Education, pages 154–160.
Denny, P., Kumar, V., MacNeil, S., Prather, J., and Leinonen, J. (2025). Probing the unknown: Exploring student interactions with probeable problems at scale in introductory programming. In Proceedings of the 30th ACM Conference on Innovation and Technology in Computer Science Education V. 1, pages 618–624.
Edwards, S. H. (2004). Using software testing to move students from trial-and-error to reflection-in-action. In Proceedings of the 35th SIGCSE technical symposium on Computer science education, pages 26–30.
Marijan, D. and Sen, S. (2017). Detecting and reducing redundancy in software testing for highly configurable systems. In 2017 IEEE 18th International Symposium on High Assurance Systems Engineering (HASE), pages 96–99. IEEE.
Mohammadian, M. and Javed, Z. (2021). Intelligent evaluation of test suites for developing efficient and reliable software. International Journal of Parallel, Emergent and Distributed Systems, 36(2):159–188.
Neto, F. G. D. O., Feldt, R., Erlenhov, L., and Nunes, J. B. D. S. (2018). Visualizing test diversity to support test optimisation. In 2018 25th Asia-Pacific Software Engineering Conference (APSEC), pages 149–158. IEEE.
Perretta, J., DeOrio, A., Guha, A., and Bell, J. (2022). On the use of mutation analysis for evaluating student test suite quality. In Proceedings of the 31st ACM SIGSOFT International Symposium on Software Testing and Analysis, pages 263–275.
Potter, H., Paiva, A. C., Amalfitano, D., Fasolino, A. R., Tramontana, P., and Just, R. (2025). Evaluating the impact of scaffolding and visualizations for mutation testing exercises in software engineering education. In Proceedings of the 33rd ACM International Conference on the Foundations of Software Engineering, pages 823–834.
Robins, A., Rountree, J., and Rountree, N. (2003). Learning and teaching programming: A review and discussion. Computer science education, 13(2):137–172.
Shams, Z. and Edwards, S. H. (2013). Toward practical mutation analysis for evaluating the quality of student-written software tests. In Proceedings of the ninth annual international ACM conference on International computing education research, pages 53–58.
Showler, A., Miljanovic, M. A., and Bradbury, J. S. (2025). How effective and efficient are student-written software tests? In Proceedings of the 56th ACM Technical Symposium on Computer Science Education V. 1, pages 1057–1063.
Vasconcelos, P. (2025). Exploring mutation testing for teaching introductory programming. In 6th International Computer Programming Education Conference (ICPEC 2025), pages 1–1. Schloss Dagstuhl–Leibniz-Zentrum für Informatik.
Watson, C. and Li, F. W. (2014). Failure rates in introductory programming revisited. In Proceedings of the 2014 conference on Innovation & technology in computer science education, pages 39–44.
Wrenn, J. and Krishnamurthi, S. (2019). Executable examples for programming problem comprehension. In Proceedings of the 2019 ACM conference on international computing education research, pages 131–139.
Publicado
05/10/2026
Como Citar
ALMEIDA, André; GUERRERO, Dalton D. S.; ANDRADE, Wilkerson L..
Avaliação da Qualidade de Testes Elaborados por Estudantes na Resolução de Problemas de Programação. In: SIMPÓSIO BRASILEIRO DE INFORMÁTICA NA EDUCAÇÃO (SBIE), 37. , 2026, Goiânia/GO.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2026
.
p. 304-317.
DOI: https://doi.org/10.5753/sbie.2026.27055.
