Use of Rubrics in Introductory Programming Courses: A Systematic Literature Review
Abstract
The use of rubrics in the assessment of introductory programming courses is an expanding practice in higher education institutions. Despite their popularity, little is known about the effect of these tools on promoting self-regulation, co-regulation, and shared regulation among students. This systematic review, conducted according to the PRISMA method, aims to evaluate the effectiveness of rubrics in these regulatory dimensions. Studies published from 2018 to 2024 were analyzed, with a total of 231 articles identified through the search string and 7 articles selected for final analysis. The results reveal that rubrics are rarely used to support regulatory aspects in programming learning. This analysis highlights the need for more structured assessment practices that effectively integrate self-regulation, co-regulation, and shared regulation. This study contributes a synthesis of current evidence and suggests directions for future research in the field.
Keywords:
Rubrics, Assessment, Introductory Programming, Self-Regulation, Co-Regulation, Shared Regulation, Systematic Review, Higher Education
References
Adesina, A., Stone, R., Batmaz, F., and Jones, I. (2015). A semi-automatic computer-aided assessment approach for marking and providing feedback comments. In Proceedings of the 7th International Conference on Computer Supported Education, volume 1, pages 93–100, Lisbon, Portugal.
Alkhalifa, A. K. (2022). Designing A Prototype Model of Peer Assessment for Introductory Computer Programming Courses. PhD thesis, Newcastle University, Newcastle upon Tyne, UK.
Brookhart, S. M. (2013). How to Use and Create Rubrics. Association for Supervision and Curriculum Development, USA.
Calderon, I., Silva, W., and Feitosa, E. (2021). Um mapeamento sistemático da literatura sobre o uso de metodologias ativas durante o ensino de programação no Brasil. Anais do XXXII Simpósio Brasileiro de Informática na Educação, pages 1152–1161.
Campos, D. and Ferreira, D. (2023). Um estudo exploratório do uso de rubricas pedagógicas para avaliar a aprendizagem corregulada e a regulação compartilhada de estudantes de disciplinas introdutórias de programação. In Anais do VIII Congresso sobre Tecnologias na Educação, pages 370–379, Porto Alegre, RS, Brasil. SBC.
Carrera-Rivera, A., Ochoa, W., Larrinaga, F., and Lasa, G. (2022). How to conduct a systematic literature review: A quick guide for computer science research. MethodsX, 9:101895.
de Campos, D. S. and Ferreira, D. J. (2021). Uso de rubricas em pesquisas de informática e educação - uma revisão sistemática da literatura. Anais do Simpósio Brasileiro de Educação em Computação, pages 83–92.
Hadwin, A., Järvelä, S., and Miller, M. (2018). Self-regulation, co-regulation, and shared regulation in collaborative learning environments. In Schunk, D. and Greene, J., editors, Handbook of self-regulation of learning and performance. Routledge, New York, NY, 2nd edition.
Hinnant-Crawford, B., Faison, M. Z., and Chang, M.-L. (2016). Culture as mediator: Co-regulation, self-regulation, and middle school mathematics achievement. Journal for Multicultural Education, 10:274–293.
Järvelä, S., Järvenoja, H., Malmberg, J., Isohätälä, J., and Sobocinski, M. (2016). How do types of interaction and phases of self-regulated learning set a stage for collaborative engagement? Learning and Instruction, 43:39–51.
Lee, J., Kazerouni, A. M., Siu, C., and Migler, T. (2023). Exploring the impact of cognitive awareness scaffolding for debugging in an introductory programming class. In Proceedings of the 54th ACM Technical Symposium on Computer Science Education V. 1, SIGCSE 2023, pages 1007–1013, New York, NY, USA. Association for Computing Machinery.
Lima, A. F. O. and Pimentel, E. P. (2013). Mecanismos para suporte à auto-regulação da aprendizagem do estudante. In Brazilian Symposium on Computers in Education (Simpósio Brasileiro de Informática na Educação-SBIE), volume 24, page 296.
Lima, G., Araújo, R. D., and Dorça, F. (2020). Uma análise dos recursos tecnológicos utilizados na estimulação da aprendizagem autorregulada em ambientes educacionais na última década. In Anais do XXXI Simpósio Brasileiro de Informática na Educação, pages 732–741. SBC.
Lishinski, A. and Yadav, A. (2021). Self-evaluation interventions: Impact on self-efficacy and performance in introductory programming. ACM Trans. Comput. Educ., 21(3).
Lishinski, A. and Yadav, A. (2021). Self-evaluation interventions: Impact on self-efficacy and performance in introductory programming. ACM Trans. Comput. Educ., 21(3).
Margulieux, L., Morrison, B., and Decker, A. (2020). Reducing withdrawal and failure rates in introductory programming with subgoal labeled worked examples. IJ STEM Ed, 7:19.
Modi, B. A., Cain, A., Wood-Bradley, G., Tubino, L., and Renzella, J. (2023). Improving computing education through a holistic learning framework: A focus group study. In 2023 IEEE International Conference on Teaching, Assessment and Learning for Engineering (TALE), pages 1–8.
Page, M., McKenzie, J., Bossuyt, P., Boutron, I., Hoffmann, T., Mulrow, C., Shamseer, L., Tetzlaff, J., Akl, E., Brennan, S., Chou, R., Glanville, J. M., Grimshaw, J., Hrðobjartsson, A., Lalu, M., Li, T., Loder, E., Mayo-Wilson, E., McDonald, S., McGuinness, L., Stewart, L., Thomas, J., Tricco, A., Welch, V., Whiting, P., and Moher, D. (2021). The PRISMA 2020 statement: An updated guideline for reporting systematic reviews. Journal of Clinical Epidemiology.
Panadero, E. and Romero, M. (2014). To rubric or not to rubric? The effects of self-assessment on self-regulation, performance, and self-efficacy. Assessment in Education: Principles, Policy & Practice, 21(2):133–148.
Pintrich, P. (2000). The Role of Goal Orientation in Self-Regulated Learning. PhD thesis, The University of Michigan, Ann Arbor, Michigan.
Siddaway, A. P., Wood, A., and Hedges, L. (2019). How to do a systematic review: A best practice guide for conducting and reporting narrative reviews, meta-analyses, and meta-syntheses. Annual Review of Psychology, 70:747–770.
Stegeman, M., Barendsen, E., and Smetsers, S. (2016). Designing a rubric for feedback on code quality in programming courses. In Proceedings of the 16th Koli Calling International Conference on Computing Education Research, Koli Calling ’16, pages 160–164, New York, NY, USA. Association for Computing Machinery.
Tsai, C., Lee, L., Cheng, Y., et al. (2024). Integrating online meta-cognitive learning strategy and team regulation to develop students’ programming skills, academic motivation, and refusal self-efficacy of internet use in a cloud classroom. Universal Access in the Information Society, 23:395–410.
Tubino, L., Cain, A., Schneider, J. G., Thiruvady, D., and Fernando, N. (2020). Authentic individual assessment for team-based software engineering projects. In Proceedings - International Conference on Software Engineering, volume 1, pages 71–81.
Alkhalifa, A. K. (2022). Designing A Prototype Model of Peer Assessment for Introductory Computer Programming Courses. PhD thesis, Newcastle University, Newcastle upon Tyne, UK.
Brookhart, S. M. (2013). How to Use and Create Rubrics. Association for Supervision and Curriculum Development, USA.
Calderon, I., Silva, W., and Feitosa, E. (2021). Um mapeamento sistemático da literatura sobre o uso de metodologias ativas durante o ensino de programação no Brasil. Anais do XXXII Simpósio Brasileiro de Informática na Educação, pages 1152–1161.
Campos, D. and Ferreira, D. (2023). Um estudo exploratório do uso de rubricas pedagógicas para avaliar a aprendizagem corregulada e a regulação compartilhada de estudantes de disciplinas introdutórias de programação. In Anais do VIII Congresso sobre Tecnologias na Educação, pages 370–379, Porto Alegre, RS, Brasil. SBC.
Carrera-Rivera, A., Ochoa, W., Larrinaga, F., and Lasa, G. (2022). How to conduct a systematic literature review: A quick guide for computer science research. MethodsX, 9:101895.
de Campos, D. S. and Ferreira, D. J. (2021). Uso de rubricas em pesquisas de informática e educação - uma revisão sistemática da literatura. Anais do Simpósio Brasileiro de Educação em Computação, pages 83–92.
Hadwin, A., Järvelä, S., and Miller, M. (2018). Self-regulation, co-regulation, and shared regulation in collaborative learning environments. In Schunk, D. and Greene, J., editors, Handbook of self-regulation of learning and performance. Routledge, New York, NY, 2nd edition.
Hinnant-Crawford, B., Faison, M. Z., and Chang, M.-L. (2016). Culture as mediator: Co-regulation, self-regulation, and middle school mathematics achievement. Journal for Multicultural Education, 10:274–293.
Järvelä, S., Järvenoja, H., Malmberg, J., Isohätälä, J., and Sobocinski, M. (2016). How do types of interaction and phases of self-regulated learning set a stage for collaborative engagement? Learning and Instruction, 43:39–51.
Lee, J., Kazerouni, A. M., Siu, C., and Migler, T. (2023). Exploring the impact of cognitive awareness scaffolding for debugging in an introductory programming class. In Proceedings of the 54th ACM Technical Symposium on Computer Science Education V. 1, SIGCSE 2023, pages 1007–1013, New York, NY, USA. Association for Computing Machinery.
Lima, A. F. O. and Pimentel, E. P. (2013). Mecanismos para suporte à auto-regulação da aprendizagem do estudante. In Brazilian Symposium on Computers in Education (Simpósio Brasileiro de Informática na Educação-SBIE), volume 24, page 296.
Lima, G., Araújo, R. D., and Dorça, F. (2020). Uma análise dos recursos tecnológicos utilizados na estimulação da aprendizagem autorregulada em ambientes educacionais na última década. In Anais do XXXI Simpósio Brasileiro de Informática na Educação, pages 732–741. SBC.
Lishinski, A. and Yadav, A. (2021). Self-evaluation interventions: Impact on self-efficacy and performance in introductory programming. ACM Trans. Comput. Educ., 21(3).
Lishinski, A. and Yadav, A. (2021). Self-evaluation interventions: Impact on self-efficacy and performance in introductory programming. ACM Trans. Comput. Educ., 21(3).
Margulieux, L., Morrison, B., and Decker, A. (2020). Reducing withdrawal and failure rates in introductory programming with subgoal labeled worked examples. IJ STEM Ed, 7:19.
Modi, B. A., Cain, A., Wood-Bradley, G., Tubino, L., and Renzella, J. (2023). Improving computing education through a holistic learning framework: A focus group study. In 2023 IEEE International Conference on Teaching, Assessment and Learning for Engineering (TALE), pages 1–8.
Page, M., McKenzie, J., Bossuyt, P., Boutron, I., Hoffmann, T., Mulrow, C., Shamseer, L., Tetzlaff, J., Akl, E., Brennan, S., Chou, R., Glanville, J. M., Grimshaw, J., Hrðobjartsson, A., Lalu, M., Li, T., Loder, E., Mayo-Wilson, E., McDonald, S., McGuinness, L., Stewart, L., Thomas, J., Tricco, A., Welch, V., Whiting, P., and Moher, D. (2021). The PRISMA 2020 statement: An updated guideline for reporting systematic reviews. Journal of Clinical Epidemiology.
Panadero, E. and Romero, M. (2014). To rubric or not to rubric? The effects of self-assessment on self-regulation, performance, and self-efficacy. Assessment in Education: Principles, Policy & Practice, 21(2):133–148.
Pintrich, P. (2000). The Role of Goal Orientation in Self-Regulated Learning. PhD thesis, The University of Michigan, Ann Arbor, Michigan.
Siddaway, A. P., Wood, A., and Hedges, L. (2019). How to do a systematic review: A best practice guide for conducting and reporting narrative reviews, meta-analyses, and meta-syntheses. Annual Review of Psychology, 70:747–770.
Stegeman, M., Barendsen, E., and Smetsers, S. (2016). Designing a rubric for feedback on code quality in programming courses. In Proceedings of the 16th Koli Calling International Conference on Computing Education Research, Koli Calling ’16, pages 160–164, New York, NY, USA. Association for Computing Machinery.
Tsai, C., Lee, L., Cheng, Y., et al. (2024). Integrating online meta-cognitive learning strategy and team regulation to develop students’ programming skills, academic motivation, and refusal self-efficacy of internet use in a cloud classroom. Universal Access in the Information Society, 23:395–410.
Tubino, L., Cain, A., Schneider, J. G., Thiruvady, D., and Fernando, N. (2020). Authentic individual assessment for team-based software engineering projects. In Proceedings - International Conference on Software Engineering, volume 1, pages 71–81.
Published
2024-11-04
How to Cite
LIMA, Maurício Rodrigues; FERREIRA, Deller James; DIAS, Elisângela Silva.
Use of Rubrics in Introductory Programming Courses: A Systematic Literature Review. In: BRAZILIAN SYMPOSIUM ON COMPUTERS IN EDUCATION (SBIE), 35. , 2024, Rio de Janeiro/RJ.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2024
.
p. 1-14.
DOI: https://doi.org/10.5753/sbie.2024.240991.
