Scripted Reflective Prompts vs. Free Exploration in AI-Assisted Programming Learning: A Quasi-Experimental Study

• Ana Paula Siqueira Universidade Federal do Paraná (UFPR)
• Talita de Paula Cypriano de Souza Instituto Federal de São Paulo (IFSP) / Universidade de São Paulo (USP)
• Rachel C. D. Reis Universidade Federal do Paraná (UFPR)
• Patricia A. Jaques Universidade Federal do Paraná (UFPR)

DOI: https://doi.org/10.5753/sbie.2026.28107

Resumo

Large Language Models (LLMs) are increasingly integrated into programming education, raising questions about how much structure such tools require to support learning. This quasi-experimental study investigated whether scripted reflective prompting enhances learning over unstructured exploration in LLM-assisted programming. Twenty-four Brazilian high school students learned Python loops using Google Gemini under guided or free exploration. ANCOVA revealed no significant differences between conditions F(1,21)=1.097, p=.307, although both groups showed gains. Results suggest scripted scaffolding may not outperform free exploration in brief interventions, and instructional sequencing may influence learning trajectories in AI-assisted contexts.

Palavras-chave: AI-Assisted Programming, Scaffolding, Programming Education

Referências

Brender, J., El-Hamamsy, L., Mondada, F., and Bumbacher, E. (2024). Who's helping who? when students use ChatGPT to engage in practice lab sessions. In Proceedings of the International Conference on Artificial Intelligence in Education (AIED), pages 235–249, Cham, Switzerland. Springer Nature.

Brender, J., El-Hamamsy, L., Uittenhove, K., Perez, A., Jermann, P., Mondada, F., and Bumbacher, E. (2026). Reflective dialogue or prompt refinement? effects of tutor scaffolding on students' independent llm use for programming. In Blanchard, E. G., Chen, G., Chi, M., and Isotani, S., editors, Artificial Intelligence in Education. AIED 2026, volume 16584 of Lecture Notes in Computer Science. Springer, Cham.

Cheng, G. et al. (2025). Integrating a scaffolding-based, LLM-driven chatbot into programming education: A university case study. In Proceedings of the 2025 International Symposium on Educational Technology (ISET), pages 1–5. IEEE.

Chi, M. T., Bassok, M., Lewis, M. W., Reimann, P., and Glaser, R. (1989). Self-explanations: How students study and use examples in learning to solve problems. Cognitive Science, 13(2):145–182.

Hou, X., Wu, Z., Wang, X., and Ericson, B. J. (2024). Codetailor: Llm-powered personalized parsons puzzles for engaging support while learning programming. In Proceedings of the Eleventh ACM Conference on Learning@ Scale, pages 51–62.

Jin, H. et al. (2024). Teach AI how to code: Using large language models as teachable agents for programming education. In Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems (CHI), pages 1–28.

Kazemitabaar, M. et al. (2023). Studying the effect of AI code generators on supporting novice learners in introductory programming. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems (CHI), pages 1–23.

Kirschner, P. A., Sweller, J., and Clark, R. E. (2006). Why minimal guidance during instruction does not work: An analysis of the failure of constructivist, discovery, problem-based, experiential, and inquiry-based teaching. Educational Psychologist, 41(2):75–86.

Mavundla, K., Abayomi, A., Adetiba, E., Olaitan, O., and Thakur, S. (2025). The transformative influence of generative AI on teaching and learning. In González Vallejo, R., Moukhliss, G., Schaeffer, E., and Paliktzoglou, V., editors, The Second International Symposium on Generative AI and Education (ISGAIE 2025), volume 262 of Lecture Notes on Data Engineering and Communications Technologies, Cham, Switzerland. Springer.

Paludo, G. and Montresor, A. (2024). Fostering metacognitive skills in programming: Leveraging AI to reflect on code. In Proceedings of the 2nd International Workshop on Artificial Intelligence Systems in Education (AIxEDU), volume 3879 of CEUR Workshop Proceedings, Bolzano, Italy.

Phung, T., Choi, H., Wu, M., Singla, A., and Brooks, C. (2025). Plan more, debug less: Applying metacognitive theory to AI-assisted programming education. In Proceedings of the International Conference on Artificial Intelligence in Education (AIED), pages 3–17, Cham, Switzerland. Springer Nature.

Prather, J. et al. (2023). The robots are here: Navigating the generative AI revolution in computing education. In Proceedings of the 2023 Working Group Reports on Innovation and Technology in Computer Science Education (ITiCSE-WGR), pages 108–159.

Sok, S. and Heng, K. (2023). ChatGPT for education and research: A review of benefits and risks. SSRN Electronic Journal.

Wang, Z., Zou, D., Zhang, R., Lee, L.-K., Xie, H., and Wang, F. L. (2025). ChatGPT-enhanced self-regulated learning in programming education: Impacts on motivation, self-efficacy, and learning outcomes. Interactive Learning Environments, 34(5):3041–3066.

Wood, D., Bruner, J. S., and Ross, G. (1976). The role of tutoring in problem solving. Journal of Child Psychology and Psychiatry, 17(2):89–100.

Zhang, L. et al. (2025). Effects of ChatGPT-based human–computer dialogic interaction programming activities on student engagement. Journal of Educational Computing Research, 63(4):988–1023.