A data-driven analysis to assess the impact of an online judge's gamification on student performance

• Daniel Lopes Zordan Filho Universidade Federal do Amazonas
• Elaine Harada Teixeira de Oliveira Federal University of Amazonas https://orcid.org/0000-0003-2884-9359
• Leandro Silva Galvão de Carvalho Federal University of Amazonas https://orcid.org/0000-0003-2970-2084
• Marcela Pessoa State University of Amazonas https://orcid.org/0000-0002-3064-5585
• Filipe Dwan Pereira Federal University of Roraima https://orcid.org/0000-0003-4914-3347
• David Braga Fernandes de Oliveira Federal University of Amazonas

DOI: https://doi.org/10.5753/cbie.sbie.2020.491

Abstract

The use of gamification in educational settings can have positive, neutral or even negative effects. Therefore, it is essential to assess the impact of gamification on student performance. Recent studies point to the use of analysis directed at student data to assess gamification instead of questionnaires that are poorly scalable and may present biases that depend on many factors. In fact, this article presents a study on the effects of a structural gamification platform integrated to an online judge with its own IDE (Integrated Development Environment), for classes in the Introduction to Computer Programming (IPC) discipline. The study carried out on 9 classes analyzes the correlations between the time that students spend in the programming environment and on the gamification platform, and their respective performances based on their final grades. As a result, a positive correlation was observed between practice time and performance, which was shown to be positively correlated with gamification.

Keywords: gamification, online judge, introduction to computer programming

References

Cohen, J. (2013).Statistical power analysis for the behavioral sciences. Academic press.


Dichev, C. and Dicheva, D. (2017). Gamifying education: what is known, what is believedand what remains uncertain: a critical review. International journal of educationaltechnology in higher education, 14(1):9.


Dwan, F., Oliveira, E., and Fernandes, D. (2017). Predição de zona de aprendizagem de alunos de introdução à programação em ambientes de correção automática de código. In Brazilian Symposium on Computers in Education (Simpósio Brasileiro de Informática na Educação-SBIE), volume 28, page 1507.


Klock, A. C. T., Ogawa, A. N., Gasparini, I., and Pimenta, M. S. (2018). Does gamification matter?: a systematic mapping about the evaluation of gamification in educational environments. In Proceedings of the 33rd Annual ACM Symposium on Applied Computing, pages 2006–2012. ACM.


Meder, M., Plumbaum, T., and Albayrak, S. (2017). A primer on data-driven gamification design. In DDGD@ MindTrek, pages 12–17.


Ogawa, A., Klock, A. C. T., and Gasparini, I. (2016). Avaliação da gamificação na área educacional: um mapeamento sistemático. In Brazilian Symposium on Computers in Education (Simpósio Brasileiro de Informática na Educação-SBIE), volume 27, page 440.


Pereira, F. D., Oliveira, E., Cristea, A., Fernandes, D., Silva, L., Aguiar, G., Alamri, A.,and Alshehri, M. (2019). Early dropout prediction for programming courses supported by online judges. In International Conference on Artificial Intelligence in Education, pages 67–72. Springer.


Pereira, F. D., Toda, A., Oliveira, E. H., Cristea, A. I., Isotani, S., Laranjeira, D., Al-meida, A., and Mendonça, J. (2020). Can we use gamification to predict student's performance? a case study supported by an online judge. In International Conference on Intelligent Tutoring Systems, pages 259–269. Springer.


Pessoa, M., Fernandes, D., de Carvalho, L. S. G., Oliveira, E., Nakamura, W., and Conte,T. (2019). Codeplay: Uma plataforma de gamificação baseada em jogos de rpg multiplayer. In Brazilian Symposium on Computers in Education (Simpósio Brasileiro de Informática na Educação-SBIE), volume 30, page 843.


Sailer, M. and Homner, L. (2020). The gamification of learning: A meta-analysis. Educational Psychology Review, pages 77–112.


Stuart, H., Lavoué, E., and Serna, A. (2020). To tailor or not to tailor gamification? an analysis of the impact of tailored game elements on learner's behaviours and motivation. In 21th International Conference on Artificial Intelligence in Education.


Toda, A. M., Valle, P. H., and Isotani, S. (2017). The dark side of gamification: An overview of negative effects of gamification in education. In Researcher Links Workshop: Higher Education for All, pages 143–156. Springer.