Analisando a efetividade da formação de clusters na avaliação de exercícios de programação
Resumo
As plataformas online de ensino de programação têm o potencial de escalar o acesso de estudantes a uma educação de qualidade. No entanto, um passo importante do processo de aprendizagem ainda é um gargalo nesses sistemas: o feedback. Este artigo considera a formação automática de clusters de códigos como uma alternativa para reduzir o trabalho docente de produção de feedback para as atividades de prática em programação. Apresentamos uma análise das abordagens possíveis e o processo de escolha de uma ferramenta para uma avaliação mais profunda. Uma vez escolhido o Overcode, o atualizamos para funcionar com o Python 3 e avaliamos a sua capacidade de formar clusters em um conjunto de dados com mais de 100 questões e dezenas de milhares de soluções. Esses dados foram extraídos do Machine Teaching, plataforma utilizada para o ensino de programação em cursos de graduação da Universidade Federal do Rio de Janeiro.
Palavras-chave:
Ensino de programação, Análise de dados educacionais, Ambiente educacional
Referências
Uri Alon, Meital Zilberstein, Omer Levy, and Eran Yahav. 2019. Code2vec: Learning Distributed Representations of Code. Proc. ACM Program. Lang. 3, POPL, Article 40, 29 pages.
S.A. Ambrose, M.W. Bridges, M. DiPietro, M.C. Lovett, M.K. Norman, and R.E. Mayer. 2010. How Learning Works: Seven Research-Based Principles for Smart Teaching. Wiley. [link]
Matthew Butler and Michael Morgan. 2007. Learning challenges faced by novice programming students studying high level and low feedback concepts. In Proceedings of ascilite Singapore 2007 ICT: Providing Choices for Learners and Learning, Roger Atkinson, Clare McBeath, Alan Soong Swee Kit, and Chris Cheers (Eds.). Nanyang Technological University, 99 – 107. [link] Annual Conference of the Australasian Society for Computers in Learning in Tertiary Education 2007, ASCILITE 2007 ; Conference date: 02-12-2007 Through 05-12-2007.
Git. 2023. Git 2.41.0 - git-diff Documentation. [link] [Online; acessado em 10 de Agosto de 2023].
Elena L. Glassman, Jeremy Scott, Rishabh Singh, Philip J. Guo, and Robert C. Miller. 2015. OverCode: Visualizing Variation in Student Solutions to Programming Problems at Scale. ACM Trans. Comput.-Hum. Interact. 22, 2, Article 7, 35 pages.
Andrew Head, Elena Glassman, Gustavo Soares, Ryo Suzuki, Lucas Figueredo, Loris D’Antoni, and Björn Hartmann. 2017. Writing Reusable Code Feedback at Scale with Mixed-Initiative Program Synthesis. In Proceedings of the Fourth (2017) ACM Conference on Learning @ Scale (Cambridge, Massachusetts, USA) (L@S ’17). Association for Computing Machinery, New York, NY, USA, 89–98.
Jonathan Huang, Chris Piech, An Thanh Nguyen, and Leonidas J. Guibas. 2013. Syntactic and Functional Variability of a Million Code Submissions in a Machine Learning MOOC. In International Conference on Artificial Intelligence in Education.
J. W. Hunt and M. D. Mcilroy. 1976. An algorithm for differential file comparison. Computer Science. [link]
Oscar Karnalim and Simon. 2021. Explanation in Code Similarity Investigation. IEEE Access 9, 59935–59948.
Laura Moraes, Carla Delgado, João Freire, and Carlos Pedreira. 2022. Machine Teaching: uma ferramenta didática e de análise de dados para suporte a cursos introdutórios de programação. In Anais do II Simpósio Brasileiro de Educação em Computação (Online). SBC, Porto Alegre, RS, Brasil, 213–223.
Andy Nguyen, Christopher Piech, Jonathan Huang, and Leonidas Guibas. 2014. Codewebs: Scalable Homework Search for Massive Open Online Programming Courses. In Proceedings of the 23rd International Conference on World Wide Web (Seoul, Korea) (WWW ’14). Association for Computing Machinery, New York, NY, USA, 491–502.
James Prather, Raymond Pettit, Kayla McMurry, Alani Peters, John Homer, and Maxine Cohen. 2018. Metacognitive Difficulties Faced by Novice Programmers in Automated Assessment Tools. In Proceedings of the 2018 ACM Conference on International Computing Education Research (Espoo, Finland) (ICER ’18). Association for Computing Machinery, New York, NY, USA, 41–50.
Blaine Price and Marian Petre. 1997. Teaching Programming through Paperless Assignments: An Empirical Evaluation of Instructor Feedback. SIGCSE Bull. 29, 3, 94–99.
Kelly Rivers and K. Koedinger. 2013. Automatic Generation of Programming Feedback; A Data-Driven Approach. In International Conference on Artificial Intelligence in Education.
Saul Schleimer, Daniel S. Wilkerson, and Alex Aiken. 2003. Winnowing: Local Algorithms for Document Fingerprinting. In Proceedings of the 2003 ACM SIGMOD International Conference on Management of Data (San Diego, California) (SIGMOD ’03). Association for Computing Machinery, New York, NY, USA, 76–85.
Rishabh Singh, Sumit Gulwani, and Armando Solar-Lezama. 2013. Automated Feedback Generation for Introductory Programming Assignments. SIGPLAN Not. 48, 6, 15–26.
Armando Solar-Lezama. 2008. Program Synthesis by Sketching. Ph.D. Dissertation. USA. Advisor(s) Bodik, Rastislav. AAI3353225.
Joaquin Vanschoren. 2018. Meta-Learning: A Survey. arXiv:1810.03548 [cs.LG]
Kai Willadsen, Stephen Kennedy, and Vincent Legoll. 2022. Meld - Visual Diff and Merge Tool (Version 3.22.0). [link] [Online; acessado em 10 de Agosto de 2023].
Mike Wu, Noah Goodman, Chris Piech, and Chelsea Finn. 2021. ProtoTransformer: A Meta-Learning Approach to Providing Student Feedback.
Mike Wu, Milan Mosse, Noah Goodman, and Chris Piech. 2019. Zero Shot Learning for Code Education: Rubric Sampling with Deep Learning Inference. In Proceedings of the Thirty-Third AAAI Conference on Artificial Intelligence and Thirty-First Innovative Applications of Artificial Intelligence Conference and Ninth AAAI Symposium on Educational Advances in Artificial Intelligence (Honolulu, Hawaii, USA) (AAAI’19/IAAI’19/EAAI’19). AAAI Press, Article 97, 9 pages.
S.A. Ambrose, M.W. Bridges, M. DiPietro, M.C. Lovett, M.K. Norman, and R.E. Mayer. 2010. How Learning Works: Seven Research-Based Principles for Smart Teaching. Wiley. [link]
Matthew Butler and Michael Morgan. 2007. Learning challenges faced by novice programming students studying high level and low feedback concepts. In Proceedings of ascilite Singapore 2007 ICT: Providing Choices for Learners and Learning, Roger Atkinson, Clare McBeath, Alan Soong Swee Kit, and Chris Cheers (Eds.). Nanyang Technological University, 99 – 107. [link] Annual Conference of the Australasian Society for Computers in Learning in Tertiary Education 2007, ASCILITE 2007 ; Conference date: 02-12-2007 Through 05-12-2007.
Git. 2023. Git 2.41.0 - git-diff Documentation. [link] [Online; acessado em 10 de Agosto de 2023].
Elena L. Glassman, Jeremy Scott, Rishabh Singh, Philip J. Guo, and Robert C. Miller. 2015. OverCode: Visualizing Variation in Student Solutions to Programming Problems at Scale. ACM Trans. Comput.-Hum. Interact. 22, 2, Article 7, 35 pages.
Andrew Head, Elena Glassman, Gustavo Soares, Ryo Suzuki, Lucas Figueredo, Loris D’Antoni, and Björn Hartmann. 2017. Writing Reusable Code Feedback at Scale with Mixed-Initiative Program Synthesis. In Proceedings of the Fourth (2017) ACM Conference on Learning @ Scale (Cambridge, Massachusetts, USA) (L@S ’17). Association for Computing Machinery, New York, NY, USA, 89–98.
Jonathan Huang, Chris Piech, An Thanh Nguyen, and Leonidas J. Guibas. 2013. Syntactic and Functional Variability of a Million Code Submissions in a Machine Learning MOOC. In International Conference on Artificial Intelligence in Education.
J. W. Hunt and M. D. Mcilroy. 1976. An algorithm for differential file comparison. Computer Science. [link]
Oscar Karnalim and Simon. 2021. Explanation in Code Similarity Investigation. IEEE Access 9, 59935–59948.
Laura Moraes, Carla Delgado, João Freire, and Carlos Pedreira. 2022. Machine Teaching: uma ferramenta didática e de análise de dados para suporte a cursos introdutórios de programação. In Anais do II Simpósio Brasileiro de Educação em Computação (Online). SBC, Porto Alegre, RS, Brasil, 213–223.
Andy Nguyen, Christopher Piech, Jonathan Huang, and Leonidas Guibas. 2014. Codewebs: Scalable Homework Search for Massive Open Online Programming Courses. In Proceedings of the 23rd International Conference on World Wide Web (Seoul, Korea) (WWW ’14). Association for Computing Machinery, New York, NY, USA, 491–502.
James Prather, Raymond Pettit, Kayla McMurry, Alani Peters, John Homer, and Maxine Cohen. 2018. Metacognitive Difficulties Faced by Novice Programmers in Automated Assessment Tools. In Proceedings of the 2018 ACM Conference on International Computing Education Research (Espoo, Finland) (ICER ’18). Association for Computing Machinery, New York, NY, USA, 41–50.
Blaine Price and Marian Petre. 1997. Teaching Programming through Paperless Assignments: An Empirical Evaluation of Instructor Feedback. SIGCSE Bull. 29, 3, 94–99.
Kelly Rivers and K. Koedinger. 2013. Automatic Generation of Programming Feedback; A Data-Driven Approach. In International Conference on Artificial Intelligence in Education.
Saul Schleimer, Daniel S. Wilkerson, and Alex Aiken. 2003. Winnowing: Local Algorithms for Document Fingerprinting. In Proceedings of the 2003 ACM SIGMOD International Conference on Management of Data (San Diego, California) (SIGMOD ’03). Association for Computing Machinery, New York, NY, USA, 76–85.
Rishabh Singh, Sumit Gulwani, and Armando Solar-Lezama. 2013. Automated Feedback Generation for Introductory Programming Assignments. SIGPLAN Not. 48, 6, 15–26.
Armando Solar-Lezama. 2008. Program Synthesis by Sketching. Ph.D. Dissertation. USA. Advisor(s) Bodik, Rastislav. AAI3353225.
Joaquin Vanschoren. 2018. Meta-Learning: A Survey. arXiv:1810.03548 [cs.LG]
Kai Willadsen, Stephen Kennedy, and Vincent Legoll. 2022. Meld - Visual Diff and Merge Tool (Version 3.22.0). [link] [Online; acessado em 10 de Agosto de 2023].
Mike Wu, Noah Goodman, Chris Piech, and Chelsea Finn. 2021. ProtoTransformer: A Meta-Learning Approach to Providing Student Feedback.
Mike Wu, Milan Mosse, Noah Goodman, and Chris Piech. 2019. Zero Shot Learning for Code Education: Rubric Sampling with Deep Learning Inference. In Proceedings of the Thirty-Third AAAI Conference on Artificial Intelligence and Thirty-First Innovative Applications of Artificial Intelligence Conference and Ninth AAAI Symposium on Educational Advances in Artificial Intelligence (Honolulu, Hawaii, USA) (AAAI’19/IAAI’19/EAAI’19). AAAI Press, Article 97, 9 pages.
Publicado
22/04/2024
Como Citar
SASSE, Arthur M.; MORAES, Laura O.; DELGADO, Carla A. D. M.; PEDREIRA, Carlos E..
Analisando a efetividade da formação de clusters na avaliação de exercícios de programação. In: SIMPÓSIO BRASILEIRO DE EDUCAÇÃO EM COMPUTAÇÃO (EDUCOMP), 4. , 2024, Evento Online.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2024
.
p. 325-335.
DOI: https://doi.org/10.5753/educomp.2024.237380.
