Investigating Difficulties with Code.org: Applying the Cognitive Walkthrough Method to the Hour of Code Artist Tutorial

• Andrea Schwertner Charão Federal University of Santa Maria https://orcid.org/0000-0003-3695-8547
• Francisco Ritter Federal University of Santa Maria

DOI: https://doi.org/10.5753/cbie.sbie.2020.1483

Abstract

Tutorial games on the Code.org platform have become popular aslarge-scale activities to promote Computational Thinking and to develop programming skills. Previous research works evaluate aspects of these games based on data collected from users. In this work, we propose another approach, exploring a method for usability evaluation – the cognitive walkthrough. This method proposes a detailed inspection of the user interface, conducted by specialists. We applied the method to the “Artist” tutorial from the Hour of Code, on the Code.org platform, and identified tasks where the game presents cognitive obstacles to users. Such result can be useful to all initiatives that rely on Code.org to develop Computational Thinking.

Keywords: cognitive walkthrough, computational thinking, Code.org

References

Bruner, J. (1984). Vygotsky’s zone of proximal development: The hidden agenda. New Directions for Child Development.


Cavalcante, A., Costa, L., e Araujo, A. (2016). Um estudo de caso sobre competências do pensamento computacional desenvolvidas na programação em blocos no Code.Org. Anais dos Workshops do Congresso Brasileiro de Informática na Educação, 5(1).


Du, J., Wimmer, H., e Rada, R. (2016). “Hour of Code”: Can it change students’ attitudes toward programming? Journal of Information Technology Education: Innovations in Practice, 15:53–73.


Farrell, D. e Moffat, D. C. (2014). Adapting cognitive walkthrough to support game based learning design. Int. J. Game-Based Learn., 4(3):23–34.


Freitas, M. e Morais, P. (2019). Possibilidade de desenvolvimento do pensamento computacional por meio do Code.Org: aplicado ao ensino fundamental (anos iniciais). Anais do Workshop de Informática na Escola, 25(1).


GhasemAghaei, R., Arya, A., e Biddle, R. (2017). Affective walkthroughs and heuristics: Evaluating Minecraft Hour of Code. In Zaphiris, P. e Ioannou, A., editors, Learning


and Collaboration Technologies. Technology in Education, pages 22–40.


Kalelioğlu, F. (2015). A new way of teaching programming skills to K-12 students: Code.org. Computers in Human Behavior, 52:200 – 210.


Lambić, D., Dorić, B., e Ivakić, S. (2020). Investigating the effect of the use of Code.org on younger elementary school students’ attitudes towards programming. Behaviour & Information Technology, 0(0):1–12.


Martins, R., Reis, R., e Marques, A. B. (2016). Inserção da programação no ensino fundamental uma análise do jogo Labirinto Clássico da Code.org através de um modelo de avaliação de jogos educacionais. In XXII Workshop de Informática na Escola (WIE).


Piaget, J. (1972). A evolução intelectual da adolescência à vida adulta. Development, 15:1–12.


Polson, P. G., Lewis, C., Rieman, J., e Wharton, C. (1992). Cognitive walkthroughs: A method for theory-based evaluation of user interfaces. Int. J. Man-Mach. Stud.,


36(5):741–773.


Popat, S. e Starkey, L. (2019). Learning to code or coding to learn? A systematic review. Computers & Education, 128:365 – 376.


Qian, Y. e Lehman, J. (2017). Students’ misconceptions and other difficulties in introductory programming: A literature review. ACM Trans. Comput. Educ., 18(1).


Raabe, A., Zanchett, G., e Vahldick, A. (2015). Jogos de programar como uma abordagem para os primeiros contatos dos estudantes com à programação. Anais dos Workshops do Congresso Brasileiro de Informática na Educação, 4(1).


Wilson, C. (2013). What’s up next for Code.org? Computer, 46(8):95–97. Yáñez Gómez, R., Cascado-Caballero, D., e Sevillano, J.-L. (2017). Academic methods for usability evaluation of serious games: A systematic review. Multimedia Tools Appl., 76(4):5755–5784.