An Item Response Theory Analysis of Algorithms and Programming Concepts in App Inventor Projects
Resumo
Computing education is often introduced in K-12 focusing on algorithms and programming concepts using block-based programming environments, such as App Inventor. Yet, learning programming is a complex process and novices struggle with several difficulties. Thus, to be effective, instructional units need to be designed regarding not only the content but also its sequencing taking into consideration difficulties related to the concepts and the idiosyncrasies of programming environments. Such systematic sequencing can be based on large-scale project analyses by regarding the volition, incentive, and opportunity of students to apply the relevant program constructs as latent psychometric constructs using Item Response Theory to obtain quantitative ?difficulty? estimates for each concept. Therefore, this article presents the results of a large-scale data-driven analysis of the demonstrated use in practice of algorithms and programming concepts in App Inventor. Based on a dataset of more than 88,000 App Inventor projects assessed automatically with the ANON rubric, we perform an analysis using Item Response Theory. The results demonstrate that the easiness of some concepts can be explained by their inherent characteristics, but also due to the characteristics of App Inventor as a programming environment. These results can help teachers, instructional and curriculum designers in the sequencing, scaffolding and assessment design of programming education in K-12.Referências
S. Grover and R. Pea. 2013. Computational thinking in K–1: A review of the state of the field. Educational Researcher, 42, 1, 38–43. DOI:https://doi.org/10.3102/0013189X12463051
P. Hubwieser, M. N. Giannakos, M. Berges, T. Brinda, I. Diethelm, J. Magenheim, Y. Pal, J. Jackova, and E. Jasute. 2015. A Global Snapshot of Computer Science Education in K-12 Schools. In Proceedings of the ITiCSE on Working Group Reports. Association for Computing Machinery, New York, NY, USA, 65–83. DOI:https://doi.org/10.1145/2858796.2858799
S. Y. Lye and J. H. L. Koh. 2014. Review on teaching and learning of computational thinking through programming: What is next for K-12? Computers in Human Behavior, 41, C, 51–61. DOI:https://doi.org/10.1016/j.chb.2014.09.012
M. Webb, N. Davis, and T. Bell. 2017. Computer science in K-12 school curricula of the 2lst century: Why, what and when?. Education and Information Technologies, 22, 445–468. DOI:https://doi.org/10.1007/s10639-016-9493-x
CSTA. 2016. K-12 Computer Science Framework. Retrieved September 2, 2020 from https://k12cs.org/
CAS. 2015. Computing at School. Retrieved September 1, 2020, from https://www.computingatschool.org.uk/
SBC. 2018. Brazilian Computer Society Guidelines for Computing Education in K-12. Retrieved September 3, 2020, from https://www.sbc.org.br/educacao/diretoria-de-educacao-basica
S. Grover, S. Basu, and P. Schank. 2018. What We Can Learn About Student Learning From Open-Ended Programming Projects in Middle School Computer Science. In Proceedings of the 49th ACM Technical Symposium on Computer Science Education. Association for Computing Machinery, NY, USA, 999-1004. DOI:https://doi.org/10.1145/3159450.3159522
F. Turbak, M. Sherman, F. Martin, D. Wolber, and S. C. Pokress. 2014. Events First Programming in App Inventor. Journal of Computing Sciences in Colleges, 29, 6, 81-89.
S. Papadakis, M. Kalogiannakis, V. Orfanakis, and N. Zaranis. 2017. The appropriateness of Scratch and App Inventor as educational environments for teaching introductory programming in primary and secondary education. International Journal of Web-Based Learning and Teaching Technologies, 12, 4, 58-77. DOI:https://doi.org/10.4018/IJWLTT.2017100106
D. Weintrop. 2019. Block-based Programming in Computer Science Education. Communications of the ACM, 62, 8, 22-25. DOI: http://doi.org/10.1145/3341221
MIT. 2020. MIT App Inventor. About us. Retrieved September 1, 2020 from http://appinventor.mit.edu/explore/about-us.html
D. Wolber, H. Abelson, and M. Friedman. 2014. Democratizing Computing with App Inventor. GetMobile: Mobile Computing and Communications. 18, 4, 53–58. DOI:https://doi.org/10.1145/2721914.2721935
E.W. Patton, M. Tissenbaum, and F. Harunani. 2019. MIT App Inventor: Objectives, Design, and Development. In Kong SC., Abelson H. (eds), Computational Thinking Education, Springer. DOI:https://doi.org/10.1007/978-981-13-6528-7_3
S. B. Fee and A. M. Holland-Minkley. 2010. Teaching computer science through problems, not solutions. Computer Science Education, 20, 2, 129-144. DOI:https://doi.org/10.1080/08993408.2010.486271
M. Tissenbaum, J. Sheldon, and H. Abelson. 2019. From Computational Thinking to Computational Action. Communications of the ACM, 62, 3, 34-36. DOI:https://doi.org/10.1145/3265747
H. Khosravi, S. Sadiq, and D. Gasevic. 2020. Development and Adoption of an Adaptive Learning System. In Proceedings of the 51st ACM Technical Symposium on Computer Science Education, Association for Computing Machinery, New York, NY, USA, 58–64. DOI:https://doi.org/10.1145/3328778.3366900
J. Bennedsen and M. E. Caspersen. 2007. Failure Rates in Introductory Programming, ACM SIGCSE Bulletin, 39, 2, 32-36. DOI:https://doi.org/10.1145/1272848.1272879
J. Bennedsen and M. E. Caspersen. 2019. Failure rates in introductory programming: 12 years later. ACM Inroads 10, 2, 30–36. DOI:https://doi.org/10.1145/3324888
J. Rogalski and R. Samurçay. 1990. Acquisition of programming knowledge and skills. In J.M.Hoc, T.R.G. Green, R. Samurçay, & D.J. Gillmore (eds.), Psychology of programming, Academic Press. DOI:https://doi.org/10.1016/B978-0-12-350772-3.50015-X
K. Brennan and M. Resnick. 2012. New frameworks for studying and assessing the development of computational thinking. In Proceedings of the Annual Meeting of the American Educational Research Association, Vancouver, Canada.
R. D. Pea and D. M. Kurland. 1984. On the cognitive effects of learning computer programming. New Ideas in Psychology, 2, 2, 137–168. DOI:https://doi.org/10.1016/0732-118X(84)90018-7
M. Resnick, J. Maloney, A. Monroy-Hernández, N. Rusk, E. Eastmond, K. Brennan, A. Millner, E. Rosenbaum, J. Silver, B. Silverman, and Y. Kafai. 2009. Scratch: programming for all. Communications of the ACM 52, 11, 60–67. DOI:https://doi.org/10.1145/1592761.1592779
J. van Patten, C. I. Chao, and C. M. Reigeluth. 1986. A review of strategies for sequencing and synthesizing instruction. Review of Educational Research, 56, 4, 437-471. DOI:https://doi.org/10.3102/00346543056004437
J. S. Bruner. 1966. Toward a theory of instruction. Harvard University Press.
G. R. Morrison, S. M. Ross, and J. E. Kemp. 2010. Designing Effective Instruction, 6th ed. John Wiley & Sons.
J. Sweller, J. J. G. van Merrienboer, and F. G. W. C. Paas, 1998. Cognitive Architecture and Instructional Design. Educational Psychology Review, 10, 251–296. DOI:https://doi.org/10.1023/A:1022193728205
C. Dede. 1986. A review and synthesis of recent research in intelligent computer-assisted instruction. International Journal on Man-Machine Studies, 24, 329-353. DOI:https://doi.org/10.1016/S0020-7373(86)80050-5
O. Vainas, Y. Ben-David, R. Gilad-Bachrach, M. Ronen, O. Bar-Ilan, R. Shillo, G. Lukin, D. Sitton, D. 2019. Staying in The Zone: Sequencing Content in Classrooms Based on The Zone of Proximal Development. In Proceedings of the 12th International Conference on Educational Data Mining, Montreal, Canada, 659 – 662.
C. M. Reigeluth. 1999. The Elaboration theory: Guidance for scope and sequence decision. In C. Reigeluth (ed.) Instructional-Design Theories and Models (vol.II), Erlbaum Associates.
I. Li, F. Turbak, and E. Mustafaraj. 2017. Calls of the Wild:Exploring Procedural Abstraction in App Inventor. In Proceedings of the IEEE Blocks and Beyond Workshop. Raleigh, NC, USA, 79-86. DOI:http://doi.org/10.1109/BLOCKS.2017.8120417
B. Xie, I. Shabir, and H. Abelson. 2015. Measuring the programmatic sophistication of app inventor projects grouped by functionality. Retrieved September 2, 2020 from http://web.mit.edu/bxie/www/thesis.pdf
Y. Park and Y. Shin. 2019. Comparing the Effectiveness of Scratch and App Inventor with Regard to Learning Computational Thinking Concepts. Electronics, 8, 1269. DOI:http://doi.org/10.3390/electronics8111269
B. Xie and H. Abelson. 2016. Skill progression in MIT app inventor. In Proceedings of the IEEE Symposium on Visual Languages and Human-Centric Computing, Cambridge, GB, 213-217. DOI:http://doi.org/10.1109/VLHCC.2016.7739687.
C. Piech, M. Sahami, D. Koller, S. Cooper, and P. Blikstein. 2012. Modeling how students learn to program. In Proceedings of the 43rd ACM Technical Symposium on Computer Science Education. Association for Computing Machinery, New York, NY, USA, 153–160. DOI:https://doi.org/10.1145/2157136.2157182
S. Grover and S. Basu. 2017. Measuring student learning in introductory block-based programming: Examining misconceptions of loops, variables, and Boolean Logic. In Proceedings of the ACM SIGCSE Technical Symposium on Computer Science Education, Association for Computing Machinery. New York, NY, USA, 267–272. DOI:https://doi.org/10.1145/3017680.3017723
J. Moreno-León, G. Robles, and M. Román-González. 2020. Towards data-driven learning paths to develop computational thinking with Scratch. IEEE Transactions on Emerging Topics in Computing, 8, 1, 193-205. DOI:https://doi.org/10.1109/TETC.2017.2734818.
K. M. Rich, C. T. Strickland, T. A. Binkowski, T. A. Moran and D. Franklin. 2017. K-8 learning trajectories derived from research literature: Sequence, repetition, conditionals. In Proceedings of the ACM Conference on International Computing Education Research, Association for Computing Machinery, New York, NY, USA, 182-190. DOI:https://doi.org/10.1145/3105726.3106166
K. M. Rich, C. T. Strickland, T. A. Binkowski and D. Franklin. 2018. Decomposition: A K-8 computational thinking learning trajectory. In Proceedings of the 2018 ACM Conference on International Computing Education Research, Association for Computing Machinery, New York, NY, USA, 124-132. DOI:https://doi.org/10.1145/3230977.3230979
K. M. Rich, C. T. Strickland, T. A. Binkowski and D. Franklin. 2019. A K-8 debugging learning trajectory derived from research literature. In Proceedings of the 50th ACM Technical Symposium on Computer Science Education, Association for Computing Machinery, New York, NY, USA, 745-751. DOI:https://doi.org/10.1145/3287324.3287396
K. Seiter and B. Foreman. 2013. Modeling the learning progressions of computational thinking of primary grade students. In Proceedings of the 9th Annual International ACM Conference on International Computing Education Research, Association for Computing Machinery, New York, NY, USA, 59–66. DOI: https://doi.org/10.1145/2493394.2493403
D. Franklin, G. Skifstad, R. Rolock, I. Mehrotra, V. Ding, A. Hansen, D. Weintrop, and D. Harlow. 2017. Using Upper-Elementary Student Performance to Understand Conceptual Sequencing in a Blocks-based Curriculum. In Proceedings of the ACM SIGCSE Technical Symposium on Computer Science Education. Association for Computing Machinery, New York, NY, USA, 231–236. DOI:https://doi.org/10.1145/3017680.3017760
N. Lytle, V. Cateté, D. Boulden, Y. Dong, J. Houchins, A. Milliken, A. Isvik, D. Bounajim, E. Wiebe, and T. Barnes. 2019. Use, Modify, Create: Comparing Computational Thinking Lesson Progressions for STEM Classes. In Proceedings of the ACM Conference on Innovation and Technology in Computer Science Education. Association for Computing Machinery, New York, NY, USA, 395–401. DOI:https://doi.org/10.1145/3304221.3319786
J. Krugel et al. 2020. Automated Measurement of Competencies and Generation of Feedback in Object-Oriented Programming Courses. In Proceedings of the IEEE Global Engineering Education Conference (EDUCON), Porto, Portugal, 329-338. DOI:https://doi.org/10.1109/EDUCON45650.2020.9125323.
R. J. De Ayala. 2009. The theory and practice of item response theory. Guilford Press.
J. E. Carlson and M. van Davier. 2017. Item Response Theory. In Advancing Human Assessment, eds. Bennet & van Davier, Springer.
M. Berges and P. Hubwieser. 2015. Evaluation of Source Code with Item Response Theory. In Proceedings of the 2015 ACM Conference on Innovation and Technology in Computer Science Education, Association for Computing Machinery, New York, NY, USA, 51–56. DOI:https://doi.org/10.1145/2729094.2742619
M. Kramer, D. A. Tobinski, and T. Brinda. 2016. On the way to a test instrument for object-oriented programming competencies. In Proceedings of the 16th Koli Calling International Conference on Computing Education Research. Association for Computing Machinery, New York, NY, USA, 145–149. DOI:https://doi.org/10.1145/2999541.2999544
N. da C. Alves, C. Gresse von Wangenheim, J. C. R. Hauck and A. F. Borgatto. 2020. A large-scale evaluation of a rubric for the automatic assessment of algorithms and programming concepts. In Proceedings of the 51st ACM Technical Symposium on Computer Science Education. Association for Computing Machinery, New York, NY, USA, 556–562. DOI:https://doi.org/10.1145/3328778.3366840
C. Gresse von Wangenheim, J. C. R. Hauck, M. F. Demetrio, R. Pelle, N. da C. Alves, H. Barbosa, L. F. Azevedo. 2018. CodeMaster – Automatic Assessment and Grading of App Inventor and Snap! Programs. Informatics in Education, 17, 1, 117-150. DOI:https://doi.org/ 10.15388/infedu.2018.08
J. Ko, R. Abraham, L. Beckwith, A. Blackwell, M. Burnett, M. Erwig, C. Scaffidi, J. Lawrance, H. Lieberman, B. Myers, M. B. Rosson, G. Rothermel, M. Shaw, and S. Wiedenbeck. 2011. The state of the art in end-user software engineering. ACM Computing Surveys, 43, 3, Article 21. DOI:https://doi.org/10.1145/1922649.1922658
E. Mustafaraj, F. Turbak, and M. Svanberg. 2017. Identifying Original Projects in App Inventor. In Proceedings of the 30th International Florida Artificial Intelligence Research Society Conference, Marco Island, FL, USA. 567-572.
N. da C. Alves, C. Gresse von Wangenheim, and J. C. R. Hauck. 2019. Approaches to assess computational thinking competences based on code analysis in K-12 education: A systematic mapping study. Informatics in Education, 18, 1, 17-39. DOI: https://doi.org/ 10.15388/infedu.2019.02
R. M. Branch. 2010. Instructional Design: The ADDIE Approach. Springer.
H. Goodrich. 1996. Understanding Rubrics. Educational Leadership, 54, 4, 14–18.
M. Sherman and F. Martin. 2015. The assessment of mobile computational thinking. Journal of Computing Sciences in Colleges, 30, 6, 53–59.
F. A. Samejima. 1969. Estimation of latent ability using a response pattern of graded scores. Psychometric Monograph, 34, 4, 2-17.
V. R. Basili, G. Caldiera, and H. D. Rombach. 1994. The goal question metric approach. In Encyclopedia of Software Engineering, John Wiley & Sons.
R. P. Chalmers. 2012. Mirt: A multidimensional item response theory package for the R Environment. Journal of Statistical Software, 48, 6, 1–29.
M. D. Reckase. 1979. Unifactor latent trait models applied to multifactor tests: Results and implications. Journal of educational statistics, Sage Publications CA: Thousand Oaks, 4, 3, 207–230.
LEGO. 2018. Lego Education Documentation. Retrieved September 2, 2020 from: https://makecode.mindstorms.com/types/string
J. S. Santos, W. L. Andrade, J. Brunet, and M. R. Araujo Melo. 2020. A Systematic Literature Review of Methodology of Learning Evaluation Based on Item Response Theory in the Context of Programming Teaching. In Proceedings of the IEEE Frontiers in Education Conference (FIE), Uppsala, Sweden, 1-9. DOI:http://doi.org/10.1109/FIE44824.2020.9274068.
P. Hubwieser, M. N. Giannakos, M. Berges, T. Brinda, I. Diethelm, J. Magenheim, Y. Pal, J. Jackova, and E. Jasute. 2015. A Global Snapshot of Computer Science Education in K-12 Schools. In Proceedings of the ITiCSE on Working Group Reports. Association for Computing Machinery, New York, NY, USA, 65–83. DOI:https://doi.org/10.1145/2858796.2858799
S. Y. Lye and J. H. L. Koh. 2014. Review on teaching and learning of computational thinking through programming: What is next for K-12? Computers in Human Behavior, 41, C, 51–61. DOI:https://doi.org/10.1016/j.chb.2014.09.012
M. Webb, N. Davis, and T. Bell. 2017. Computer science in K-12 school curricula of the 2lst century: Why, what and when?. Education and Information Technologies, 22, 445–468. DOI:https://doi.org/10.1007/s10639-016-9493-x
CSTA. 2016. K-12 Computer Science Framework. Retrieved September 2, 2020 from https://k12cs.org/
CAS. 2015. Computing at School. Retrieved September 1, 2020, from https://www.computingatschool.org.uk/
SBC. 2018. Brazilian Computer Society Guidelines for Computing Education in K-12. Retrieved September 3, 2020, from https://www.sbc.org.br/educacao/diretoria-de-educacao-basica
S. Grover, S. Basu, and P. Schank. 2018. What We Can Learn About Student Learning From Open-Ended Programming Projects in Middle School Computer Science. In Proceedings of the 49th ACM Technical Symposium on Computer Science Education. Association for Computing Machinery, NY, USA, 999-1004. DOI:https://doi.org/10.1145/3159450.3159522
F. Turbak, M. Sherman, F. Martin, D. Wolber, and S. C. Pokress. 2014. Events First Programming in App Inventor. Journal of Computing Sciences in Colleges, 29, 6, 81-89.
S. Papadakis, M. Kalogiannakis, V. Orfanakis, and N. Zaranis. 2017. The appropriateness of Scratch and App Inventor as educational environments for teaching introductory programming in primary and secondary education. International Journal of Web-Based Learning and Teaching Technologies, 12, 4, 58-77. DOI:https://doi.org/10.4018/IJWLTT.2017100106
D. Weintrop. 2019. Block-based Programming in Computer Science Education. Communications of the ACM, 62, 8, 22-25. DOI: http://doi.org/10.1145/3341221
MIT. 2020. MIT App Inventor. About us. Retrieved September 1, 2020 from http://appinventor.mit.edu/explore/about-us.html
D. Wolber, H. Abelson, and M. Friedman. 2014. Democratizing Computing with App Inventor. GetMobile: Mobile Computing and Communications. 18, 4, 53–58. DOI:https://doi.org/10.1145/2721914.2721935
E.W. Patton, M. Tissenbaum, and F. Harunani. 2019. MIT App Inventor: Objectives, Design, and Development. In Kong SC., Abelson H. (eds), Computational Thinking Education, Springer. DOI:https://doi.org/10.1007/978-981-13-6528-7_3
S. B. Fee and A. M. Holland-Minkley. 2010. Teaching computer science through problems, not solutions. Computer Science Education, 20, 2, 129-144. DOI:https://doi.org/10.1080/08993408.2010.486271
M. Tissenbaum, J. Sheldon, and H. Abelson. 2019. From Computational Thinking to Computational Action. Communications of the ACM, 62, 3, 34-36. DOI:https://doi.org/10.1145/3265747
H. Khosravi, S. Sadiq, and D. Gasevic. 2020. Development and Adoption of an Adaptive Learning System. In Proceedings of the 51st ACM Technical Symposium on Computer Science Education, Association for Computing Machinery, New York, NY, USA, 58–64. DOI:https://doi.org/10.1145/3328778.3366900
J. Bennedsen and M. E. Caspersen. 2007. Failure Rates in Introductory Programming, ACM SIGCSE Bulletin, 39, 2, 32-36. DOI:https://doi.org/10.1145/1272848.1272879
J. Bennedsen and M. E. Caspersen. 2019. Failure rates in introductory programming: 12 years later. ACM Inroads 10, 2, 30–36. DOI:https://doi.org/10.1145/3324888
J. Rogalski and R. Samurçay. 1990. Acquisition of programming knowledge and skills. In J.M.Hoc, T.R.G. Green, R. Samurçay, & D.J. Gillmore (eds.), Psychology of programming, Academic Press. DOI:https://doi.org/10.1016/B978-0-12-350772-3.50015-X
K. Brennan and M. Resnick. 2012. New frameworks for studying and assessing the development of computational thinking. In Proceedings of the Annual Meeting of the American Educational Research Association, Vancouver, Canada.
R. D. Pea and D. M. Kurland. 1984. On the cognitive effects of learning computer programming. New Ideas in Psychology, 2, 2, 137–168. DOI:https://doi.org/10.1016/0732-118X(84)90018-7
M. Resnick, J. Maloney, A. Monroy-Hernández, N. Rusk, E. Eastmond, K. Brennan, A. Millner, E. Rosenbaum, J. Silver, B. Silverman, and Y. Kafai. 2009. Scratch: programming for all. Communications of the ACM 52, 11, 60–67. DOI:https://doi.org/10.1145/1592761.1592779
J. van Patten, C. I. Chao, and C. M. Reigeluth. 1986. A review of strategies for sequencing and synthesizing instruction. Review of Educational Research, 56, 4, 437-471. DOI:https://doi.org/10.3102/00346543056004437
J. S. Bruner. 1966. Toward a theory of instruction. Harvard University Press.
G. R. Morrison, S. M. Ross, and J. E. Kemp. 2010. Designing Effective Instruction, 6th ed. John Wiley & Sons.
J. Sweller, J. J. G. van Merrienboer, and F. G. W. C. Paas, 1998. Cognitive Architecture and Instructional Design. Educational Psychology Review, 10, 251–296. DOI:https://doi.org/10.1023/A:1022193728205
C. Dede. 1986. A review and synthesis of recent research in intelligent computer-assisted instruction. International Journal on Man-Machine Studies, 24, 329-353. DOI:https://doi.org/10.1016/S0020-7373(86)80050-5
O. Vainas, Y. Ben-David, R. Gilad-Bachrach, M. Ronen, O. Bar-Ilan, R. Shillo, G. Lukin, D. Sitton, D. 2019. Staying in The Zone: Sequencing Content in Classrooms Based on The Zone of Proximal Development. In Proceedings of the 12th International Conference on Educational Data Mining, Montreal, Canada, 659 – 662.
C. M. Reigeluth. 1999. The Elaboration theory: Guidance for scope and sequence decision. In C. Reigeluth (ed.) Instructional-Design Theories and Models (vol.II), Erlbaum Associates.
I. Li, F. Turbak, and E. Mustafaraj. 2017. Calls of the Wild:Exploring Procedural Abstraction in App Inventor. In Proceedings of the IEEE Blocks and Beyond Workshop. Raleigh, NC, USA, 79-86. DOI:http://doi.org/10.1109/BLOCKS.2017.8120417
B. Xie, I. Shabir, and H. Abelson. 2015. Measuring the programmatic sophistication of app inventor projects grouped by functionality. Retrieved September 2, 2020 from http://web.mit.edu/bxie/www/thesis.pdf
Y. Park and Y. Shin. 2019. Comparing the Effectiveness of Scratch and App Inventor with Regard to Learning Computational Thinking Concepts. Electronics, 8, 1269. DOI:http://doi.org/10.3390/electronics8111269
B. Xie and H. Abelson. 2016. Skill progression in MIT app inventor. In Proceedings of the IEEE Symposium on Visual Languages and Human-Centric Computing, Cambridge, GB, 213-217. DOI:http://doi.org/10.1109/VLHCC.2016.7739687.
C. Piech, M. Sahami, D. Koller, S. Cooper, and P. Blikstein. 2012. Modeling how students learn to program. In Proceedings of the 43rd ACM Technical Symposium on Computer Science Education. Association for Computing Machinery, New York, NY, USA, 153–160. DOI:https://doi.org/10.1145/2157136.2157182
S. Grover and S. Basu. 2017. Measuring student learning in introductory block-based programming: Examining misconceptions of loops, variables, and Boolean Logic. In Proceedings of the ACM SIGCSE Technical Symposium on Computer Science Education, Association for Computing Machinery. New York, NY, USA, 267–272. DOI:https://doi.org/10.1145/3017680.3017723
J. Moreno-León, G. Robles, and M. Román-González. 2020. Towards data-driven learning paths to develop computational thinking with Scratch. IEEE Transactions on Emerging Topics in Computing, 8, 1, 193-205. DOI:https://doi.org/10.1109/TETC.2017.2734818.
K. M. Rich, C. T. Strickland, T. A. Binkowski, T. A. Moran and D. Franklin. 2017. K-8 learning trajectories derived from research literature: Sequence, repetition, conditionals. In Proceedings of the ACM Conference on International Computing Education Research, Association for Computing Machinery, New York, NY, USA, 182-190. DOI:https://doi.org/10.1145/3105726.3106166
K. M. Rich, C. T. Strickland, T. A. Binkowski and D. Franklin. 2018. Decomposition: A K-8 computational thinking learning trajectory. In Proceedings of the 2018 ACM Conference on International Computing Education Research, Association for Computing Machinery, New York, NY, USA, 124-132. DOI:https://doi.org/10.1145/3230977.3230979
K. M. Rich, C. T. Strickland, T. A. Binkowski and D. Franklin. 2019. A K-8 debugging learning trajectory derived from research literature. In Proceedings of the 50th ACM Technical Symposium on Computer Science Education, Association for Computing Machinery, New York, NY, USA, 745-751. DOI:https://doi.org/10.1145/3287324.3287396
K. Seiter and B. Foreman. 2013. Modeling the learning progressions of computational thinking of primary grade students. In Proceedings of the 9th Annual International ACM Conference on International Computing Education Research, Association for Computing Machinery, New York, NY, USA, 59–66. DOI: https://doi.org/10.1145/2493394.2493403
D. Franklin, G. Skifstad, R. Rolock, I. Mehrotra, V. Ding, A. Hansen, D. Weintrop, and D. Harlow. 2017. Using Upper-Elementary Student Performance to Understand Conceptual Sequencing in a Blocks-based Curriculum. In Proceedings of the ACM SIGCSE Technical Symposium on Computer Science Education. Association for Computing Machinery, New York, NY, USA, 231–236. DOI:https://doi.org/10.1145/3017680.3017760
N. Lytle, V. Cateté, D. Boulden, Y. Dong, J. Houchins, A. Milliken, A. Isvik, D. Bounajim, E. Wiebe, and T. Barnes. 2019. Use, Modify, Create: Comparing Computational Thinking Lesson Progressions for STEM Classes. In Proceedings of the ACM Conference on Innovation and Technology in Computer Science Education. Association for Computing Machinery, New York, NY, USA, 395–401. DOI:https://doi.org/10.1145/3304221.3319786
J. Krugel et al. 2020. Automated Measurement of Competencies and Generation of Feedback in Object-Oriented Programming Courses. In Proceedings of the IEEE Global Engineering Education Conference (EDUCON), Porto, Portugal, 329-338. DOI:https://doi.org/10.1109/EDUCON45650.2020.9125323.
R. J. De Ayala. 2009. The theory and practice of item response theory. Guilford Press.
J. E. Carlson and M. van Davier. 2017. Item Response Theory. In Advancing Human Assessment, eds. Bennet & van Davier, Springer.
M. Berges and P. Hubwieser. 2015. Evaluation of Source Code with Item Response Theory. In Proceedings of the 2015 ACM Conference on Innovation and Technology in Computer Science Education, Association for Computing Machinery, New York, NY, USA, 51–56. DOI:https://doi.org/10.1145/2729094.2742619
M. Kramer, D. A. Tobinski, and T. Brinda. 2016. On the way to a test instrument for object-oriented programming competencies. In Proceedings of the 16th Koli Calling International Conference on Computing Education Research. Association for Computing Machinery, New York, NY, USA, 145–149. DOI:https://doi.org/10.1145/2999541.2999544
N. da C. Alves, C. Gresse von Wangenheim, J. C. R. Hauck and A. F. Borgatto. 2020. A large-scale evaluation of a rubric for the automatic assessment of algorithms and programming concepts. In Proceedings of the 51st ACM Technical Symposium on Computer Science Education. Association for Computing Machinery, New York, NY, USA, 556–562. DOI:https://doi.org/10.1145/3328778.3366840
C. Gresse von Wangenheim, J. C. R. Hauck, M. F. Demetrio, R. Pelle, N. da C. Alves, H. Barbosa, L. F. Azevedo. 2018. CodeMaster – Automatic Assessment and Grading of App Inventor and Snap! Programs. Informatics in Education, 17, 1, 117-150. DOI:https://doi.org/ 10.15388/infedu.2018.08
J. Ko, R. Abraham, L. Beckwith, A. Blackwell, M. Burnett, M. Erwig, C. Scaffidi, J. Lawrance, H. Lieberman, B. Myers, M. B. Rosson, G. Rothermel, M. Shaw, and S. Wiedenbeck. 2011. The state of the art in end-user software engineering. ACM Computing Surveys, 43, 3, Article 21. DOI:https://doi.org/10.1145/1922649.1922658
E. Mustafaraj, F. Turbak, and M. Svanberg. 2017. Identifying Original Projects in App Inventor. In Proceedings of the 30th International Florida Artificial Intelligence Research Society Conference, Marco Island, FL, USA. 567-572.
N. da C. Alves, C. Gresse von Wangenheim, and J. C. R. Hauck. 2019. Approaches to assess computational thinking competences based on code analysis in K-12 education: A systematic mapping study. Informatics in Education, 18, 1, 17-39. DOI: https://doi.org/ 10.15388/infedu.2019.02
R. M. Branch. 2010. Instructional Design: The ADDIE Approach. Springer.
H. Goodrich. 1996. Understanding Rubrics. Educational Leadership, 54, 4, 14–18.
M. Sherman and F. Martin. 2015. The assessment of mobile computational thinking. Journal of Computing Sciences in Colleges, 30, 6, 53–59.
F. A. Samejima. 1969. Estimation of latent ability using a response pattern of graded scores. Psychometric Monograph, 34, 4, 2-17.
V. R. Basili, G. Caldiera, and H. D. Rombach. 1994. The goal question metric approach. In Encyclopedia of Software Engineering, John Wiley & Sons.
R. P. Chalmers. 2012. Mirt: A multidimensional item response theory package for the R Environment. Journal of Statistical Software, 48, 6, 1–29.
M. D. Reckase. 1979. Unifactor latent trait models applied to multifactor tests: Results and implications. Journal of educational statistics, Sage Publications CA: Thousand Oaks, 4, 3, 207–230.
LEGO. 2018. Lego Education Documentation. Retrieved September 2, 2020 from: https://makecode.mindstorms.com/types/string
J. S. Santos, W. L. Andrade, J. Brunet, and M. R. Araujo Melo. 2020. A Systematic Literature Review of Methodology of Learning Evaluation Based on Item Response Theory in the Context of Programming Teaching. In Proceedings of the IEEE Frontiers in Education Conference (FIE), Uppsala, Sweden, 1-9. DOI:http://doi.org/10.1109/FIE44824.2020.9274068.
Publicado
26/04/2021
Como Citar
DA CRUZ ALVES, Nathalia; VON WANGENHEIM, Christiane Gresse; ROSSA HAUCK, Jean Carlo; FERRETI BORGATTO, Adriano.
An Item Response Theory Analysis of Algorithms and Programming Concepts in App Inventor Projects. In: SIMPÓSIO BRASILEIRO DE EDUCAÇÃO EM COMPUTAÇÃO (EDUCOMP), 1. , 2021, On-line.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2021
.
p. 01-11.
DOI: https://doi.org/10.5753/educomp.2021.14466.
