Do students read the extended feedback messages displayed by online judges?

• Joseph de Oliveira Federal University of Amazonas
• Felipe Salem Universidade Federal do Amazonas
• Elaine Harada Teixeira de Oliveira Federal University of Amazonas https://orcid.org/0000-0003-2884-9359
• David Braga Fernandes Oliveira Federal University of Amazonas
• Leandro Silva Galvão de Carvalho Federal University of Amazonas https://orcid.org/0000-0003-2970-2084
• Filipe Dwan Pereira Federal University of Roraima

DOI: https://doi.org/10.5753/cbie.sbie.2020.1723

Abstract

Despite the benefits of online judges over the automatic code correction process, the literature states that feedback from these systems needs to be improved to generate more effective learning for the student. This article deepens an earlier study, intervention-control type, and aims to identify whether the extended feedback presented to the students of introductory programming was visualized, and whether there was a statistically significant difference between the interest of different groups in the content of the feedback. A difference was observed between students in the test and control groups in relation to reading time, reading time per word, occurrences of some types of errors and in relation to the frequency of access to extended feedback.

Keywords: Enhanced Compiler Error Messages, introduction to programming, feedback messages

References

Alves, F. P. and Jaques, P. (2014). Um ambiente virtual com feedback personalizado para apoio a disciplinas de programação. In Anais dos Workshops do Congresso Brasileiro de Informática na Educação, volume 3, page 51.


Becker, B. A. (2016). An effective approach to enhancing compiler error messages. In Proceedings of the 47th ACM Technical Symposium on Computing Science Education, pages 126–131.


Becker, B. A., Glanville, G., Iwashima, R., McDonnell, C., Goslin, K., and Mooney, C. (2016). Effective compiler error message enhancement for novice programming students. Computer Science Education, 26(2-3):148–175.


de Oliveira, J., Oliveira, E., de Carvalho, L. S. G., and Fernandes, D. (2019). Mensagensestendidas de feedback em um juiz online para alunos de introdução à computação: resultados preliminares. In Brazilian Symposium on Computers in Education (SimpósioBrasileiro de Informática na Educação-SBIE), volume 30, page 329.


Denny, P., Prather, J., and Becker, B. A. (2020). Error message readability and novice debugging performance. In Proceedings of the 2020 ACM Conference on Innovation and Technology in Computer Science Education, pages 480–486.


Jesus, G. S., Santos, K., Conceição, J., Ribeiro, E., and Neto, A. C. (2018). Avaliação de uma abordagem para auxiliar a correção de erros de aprendizes de programação. In Simpósio Brasileiro de Informática na Educação (SBIE), volume 29, pages 1–10.


Karvelas, I., Li, A., and Becker, B. A. (2020). The effects of compilation mechanisms and error message presentation on novice programmer behavior. In Proceedings of the51st ACM Technical Symposium on Computer Science Education, pages 759–765.


Keuning, H., Jeuring, J., and Heeren, B. (2018). A systematic literature review of automated feedback generation for programming exercises.ACM Transactions on Computing Education (TOCE), 19(1):1–43.


Narciss, S. (2008). Feedback strategies for interactive learning tasks.Handbook of research on educational communications and technology, 3:125–144.


Nienaltowski, M.-H., Pedroni, M., and Meyer, B. (2008). Compiler error messages: What can help novices? InProceedings of the 39th SIGCSE technical symposium on Computer science education, pages 168–172.


Prather, J., Pettit, R., McMurry, K. H., Peters, A., Homer, J., Simone, N., and Cohen, M. (2017). On novices’ interaction with compiler error messages: A human factors approach. In Proceedings of the 2017 ACM Conference on International ComputingEducation Research, pages 74–82.