Evaluating Knowledge Building in Programming Through SOLO Taxonomy

• André Almeida Federal University of Campina Grande
• Eliane Araújo Federal University of Campina Grande http://orcid.org/0000-0002-6837-6173
• Jorge Figueiredo Federal University of Campina Grande https://orcid.org/0000-0003-1197-6183

DOI: https://doi.org/10.5753/cbie.sbie.2020.1813

Abstract

A great challenge in the teaching of programming is evaluating the process of building this knowledge. Automated assessment tools evaluate the final produced code. However, the development process that involves choosing adequate structures to implement the problem solution can reveal a clear picture of the beginner’s knowledge of programming. In this work, we seek to evaluate the students’ progress considering the code complexity of their programming exercises using a framework that extends the SOLO taxonomy through the use of predefined programming building blocks. In the empirical study we have carried out, we evaluated the correlation between the mastery of basic design skills and the automated program assessment.

Keywords: Beginner programmers, SOLO, Code Design

References

Ala-Mutka, K. M. (2005). A survey of automated assessment approaches for program-ming assignments.Computer science education, 15(2):83–102.


Araújo, E., Gaudencio, M., Menezes, A., Ferreira, I., Ribeiro, I., Fagner, A., Ponciano, L., Morais, F., Guerrero, D., and Figueiredo, J. (2013). O papel do hábito de estudo no desempenho do aluno de programação.


Biggs, J., Collis, K., and Edward, A. (2014). Evaluating the Quality of Learning: The SOLO Taxonomy (Structure of the Observed Learning Outcome). Elsevier Science.


Cardell-Oliver, R. (2011). How can software metrics help novice programmers? In Proceedings of the Thirteenth Australasian Computing Education Conference - Volume 114, ACE ’11, page 55–62, AUS. Australian Computer Society, Inc


Costantini, U., Lonati, V., and Morpurgo, A. (2020). How plans occur in novices’ programs: A method to evaluate program-writing skills. In Proceedings of the 51st ACM Technical Symposium on Computer Science Education, SIGCSE ’20, page 852–858, New York, NY, USA. Association for Computing Machinery.


Douce, C., Livingstone, D., and Orwell, J. (2005). Automatic test-based assessment of programming: A review. J. Educ. Resour. Comput., 5(3):4–es.


Ihantola, P., Ahoniemi, T., Karavirta, V., and Seppal¨ a, O. (2010). Review of recent systems for automatic assessment of programming assignments. In Proceedings of the 10th Koli Calling International Conference on Computing Education Research, Koli Calling ’10, page 86–93, New York, NY, USA. Association for Computing Machinery.


Izu, C., Weerasinghe, A., and Pope, C. (2016). A study of code design skills in novice programmers using the solo taxonomy. In Proceedings of the 2016 ACM Conference


on International Computing Education Research, ICER ’16, page 251–259, New York, NY, USA. Association for Computing Machinery.


Jimoyiannis, A. (2011). Using solo taxonomy to explore students’ mental models of the programming variable and the assignment statement. Themes in Science and Technology Education, pages 53–74.


Lopez, M., Whalley, J., Robbins, P., and Lister, R. (2008). Relationships between reading, tracing and writing skills in introductory programming. ICER ’08, page 101–112, New York, NY, USA. Association for Computing Machinery.


Mengel, S. A. and Yerramilli, V. (1999). A case study of the static analysis of the quality of novice student programs. SIGCSE Bull., 31(1):78–82.


Polya, G. and Conway, J. (2004). How to Solve It: A New Aspect of Mathematical Method. Penguin mathematics. Princeton University Press.


Sheard, J., Carbone, A., Lister, R., Simon, B., Thompson, E., and Whalley, J. L. (2008). Going solo to assess novice programmers. In Proceedings of the 13th Annual Conference on Innovation and Technology in Computer Science Education, ITiCSE ’08, page 209–213, New York, NY, USA. Association for Computing Machinery.


Shuhidan, S., Hamilton, M., and D’Souza, D. (2009). A taxonomic study of novice programming summative assessment. In Proceedings of the Eleventh Australasian


Conference on Computing Education - Volume 95, ACE ’09, page 147–156, AUS. Australian Computer Society, Inc.


Singh, R., Gulwani, S., and Solar-Lezama, A. (2013). Automated feedback generation for introductory programming assignments. SIGPLAN Not., 48(6):15–26.


Whalley, J., Clear, T., Robbins, P., and Thompson, E. (2011). Salient elements in novice solutions to code writing problems. In Proceedings of the Thirteenth Australasian Computing Education Conference - Volume 114, ACE ’11, page 37–46, AUS. Australian Computer Society, Inc.