Reflections on the Use of LLMs in Programming Education

• Yasmim Silva Rosa UFMG http://orcid.org/0009-0004-0093-4513
• Pedro Garcia UFOP https://orcid.org/0009-0005-7744-697X
• Kattiana Constantino UFVJM https://orcid.org/0000-0003-4511-7504
• Eduardo Figueiredo UFMG https://orcid.org/0000-0002-6004-2718

DOI: https://doi.org/10.5753/educomp.2025.5366

Abstract

The objective of this article is to understand how Large Language Models (LLMs) impact programming teaching. Currently, many beginners in programming languages, in terms of creating, correcting and explaining code, have been resorting to Large Language Models due to their ease in obtaining ready-made codes and complete explanations. But do LLMs have a negative or positive impact on programming language teaching? This is what this article will discuss.

References

Carbonell, J. G., Michalski, R. S., and Mitchell, T. M. (1983). An overview of machine learning. Machine learning, pages 3–23.

da Silva Junior, S. M., de Freitas, R. A. B., de Morais, M. A. C., and Costa, D. L. V. (2023). Chatgpt no auxílio da aprendizagem de programação: Um estudo de caso. In Anais do XXXIV Simpósio Brasileiro de Informática na Educação, pages 1375–1384. SBC.

de Araujo Costa, D. B., Knupp, D. R., de Souza Alves, C., Júnior, W. d. S. A., Correa, M. E. L., and Cupti, M. d. C. G. Inteligência artificial na educação.

Geng, C., Yihan, Z., Pientka, B., and Si, X. (2023). Can chatgpt pass an introductory level functional language programming course? arXiv preprint arXiv:2305.02230.

Jošt, G., Taneski, V., and Karakatič, S. (2024). The impact of large language models on programming education and student learning outcomes. Applied Sciences, 14(10):4115.

Keele, S. et al. (2007). Guidelines for performing systematic literature reviews in software engineering. Technical report, Technical report, ver. 2.3 ebse technical report. ebse.

Kiesler, N. and Schiffner, D. (2023). Large language models in introductory programming education: Chatgpt’s performance and implications for assessments. arXiv preprint arXiv:2308.08572.

Kitchenham, B. (2006). Evidence-based software engineering and systematic literature reviews. In Product-Focused Software Process Improvement: 7th International Conference, PROFES 2006, Amsterdam, The Netherlands, June 12-14, 2006. Proceedings 7, pages 3–3. Springer.

Kumar, N. A. and Lan, A. (2024). Using large language models for student-code guided test case generation in computer science education. arXiv preprint arXiv:2402.07081.

Leinonen, J., Denny, P., MacNeil, S., Sarsa, S., Bernstein, S., Kim, J., Tran, A., and Hellas, A. (2023). Comparing code explanations created by students and large language models. In Proceedings of the 2023 Conference on Innovation and Technology in Computer Science Education V. 1, pages 124–130.

Lyu, W., Wang, Y., Chung, T., Sun, Y., and Zhang, Y. (2024). Evaluating the effectiveness of llms in introductory computer science education: A semester-long field study. In Proceedings of the Eleventh ACM Conference on Learning@ Scale, pages 63–74.

Nunes, R., Figueiredo, E., Soares, L., Nadi, S., Ferreira, F., and Santos, G. (2025). Evaluating the effectiveness of llms in fixing maintainability issues in real-world projects. In IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER).

Pereira Filho, L. C., de Souza, T. d. P., and de Paula, L. B. (2023). Análise das respostas do chatgpt em relaçao ao conteudo de programaçao para iniciantes. In Anais do XXXIV Simpósio Brasileiro de Informática na Educação, pages 1738–1748. SBC.

Rodrigues, P., Souza, M., and Figueiredo, E. (2018). Games and gamification in software engineering education: A survey with educators. In IEEE Frontiers in Education Conference (FIE), pages 1–9.

Santos, A., Souza, M., Dayrell, M., and Figueiredo, E. (2019). A systematic mapping study on game elements and serious games for learning programming. Communications in Computer and Information Science (CCIS), 1022.

Silva, C. A. G. d., Ramos, F. N., de Moraes, R. V., and Santos, E. L. d. (2024). Chatgpt: Challenges and benefits in software programming for higher education. Sustainability, 16(3):1245.

Souza, M., Veado, L., Moreira, R. T., Figueiredo, E., and Costa, H. (2018). A systematic mapping study on game-related methods for software engineering education. Information and Software Technology, 95:201–218.

Xue, Y., Chen, H., Bai, G. R., Tairas, R., and Huang, Y. (2024). Does chatgpt help with introductory programming? an experiment of students using chatgpt in cs1. In Proceedings of the 46th International Conference on Software Engineering: Software Engineering Education and Training, pages 331–341.

Yepes, I. and Fiorin, A. (2023). Chatgpt como assistente de ensino na disciplina de estruturas de dados. Anais do Encontro Anual de Tecnologia da Informação, 12(1):36–36.