Performance prediction in computational environments to support programming classes: a systematic literature mapping

• Filipe Dwan Pereira Federal University of Roraima https://orcid.org/0000-0003-4914-3347
• Linnik Maciel de Souza Universidade Federal do Amazonas
• Elaine Harada Teixeira de Oliveira Federal University of Amazonas https://orcid.org/0000-0003-2884-9359
• David Braga Fernandes de Oliveira Federal University of Amazonas
• Leandro Silva Galvão de Carvalho Federal University of Amazonas https://orcid.org/0000-0003-2970-2084

DOI: https://doi.org/10.5753/cbie.sbie.2020.1673

Abstract

Programming classes have a high failure rate and, as a result, many studies has been conducted to predict student performance, to help in decision making. The present work carried out a systematic mapping of studies from 2009 to 2019 to address six research questions. In total, we analysed 911 publications and filtered 70 works related to performance prediction in virtual environments, especially online judges. In general, the publications accepted explore Machine Learning and Data Mining techniques to make inferences through the modelling of students' programming profiles. Results point for research gaps and open questions for this field.

Keywords: machine learning, data mining, programming, systematic mapping, systematic review

References

Azcona, D., Hsiao, I. H., & Smeaton, A. F. (2018). Personalizing computer science education by leveraging multimodal learning analytics. IEEE Frontiers in Education Conference (FIE) (pp. 1-9).


Ahadi, A., Vihavainen, A. e Lister, R. (2016). On the number of attempts students made on some online programming exercises during semester and their subsequent performance on final exam questions. ACM Conference on Innovation and Technology in Computer Science Education, p. 218–223.


Auvinen, T. (2015). Harmful study habits in online learning environments with automatic assessment. 2015 International Conference on Learning and Teaching in Computing and Engineering, p. 50–57.


Dwan, F., Oliveira, E., & Fernandes, D. (2017). Predição de zona de aprendizagem de alunos de introdução à programação em ambientes de correção automática de código. Simpósio Brasileiro de Informática na Educação-SBIE).


Edwards, S.H., Snyder, J., Pérez-Quiñones, M.A., Allevato, A., Kim, D., Tretola, B., 2009. Comparing effective and ineffective behaviors of student programmers. Workshop on Computing education research, ACM. pp. 3– 14.


Fonseca, S., Oliveira, E., Pereira, F., Fernandes, D., de Carvalho, L. S. G. (2019). Adaptação de um método preditivo para inferir o desempenho de alunos de programação. Simpósio Brasileiro de Informática na Educação (SBIE).


Jadud, M. C. (2006). Methods and tools for exploring novice compilation behaviour. International Workshop on Computing Education Research, p. 73–84.


Leinonen, J., Longi, K., Klami, A. e Vihavainen, A. (2016). Automatic inference of programming performance and experience from typing patterns. ACM Technical Symposium on Computing Science Education, p. 132–137.


Otero, J., Junco, L., Suarez, R., Palacios, A., Couso, I. e Sanchez, L. (2016). Finding informative code metrics under uncertainty for predicting the pass rate of online courses. Information Sciences, 373:42–56.


Pereira, F.D., Oliveira, E. H. T., Fernandes, D., Cristea, A. (2019a). Early performance prediction for CS1 course students using a combination of machine learning and an evolutionary algorithm. in: 2019 IEEE 19th International Conference on Advanced Learning Technologies (ICALT), IEEE. pp. 183–184.


Pereira, F. D., Oliveira, E., Cristea, A., Fernandes, D., Silva, L., Aguiar, G., Ahmed, A., Alshehri, M. (2019b). Early dropout prediction for programming courses supported by online judges. International Conference on Artificial Intelligence in Education (pp. 67-72). Springer, Cham.


Pereira, F., Oliveira, E., Fernandes, D., de Carvalho, L. S. G., & Junior, H. (2019c). Otimização e automação da predição precoce do desempenho de alunos que utilizam juízes online: uma abordagem com algoritmo genético. Simpósio Brasileiro de Informática na Educação (SBIE).


Pereira, F. D., Oliveira, E. H., Oliveira, D. B., Cristea, A. I., Carvalho, L. S., Fonseca, S. C., Toda, A., Isotani, S. (2020). Using learning analytics in the Amazonas: understanding students’ behaviour in introductory programming. British Journal of Educational Technology.


Quille, K., Bergin, S. (2019). CS1: how will they do? How can we help? A decade of research and practice. Computer Science Education, 29(2-3), 254-282.