Geração de questões de programação baseada em templates e IA generativa
Resumo
Ensinar programação é uma tarefa complexa devido à própria natureza do assunto. Há uma necessidade constante de que os alunos pratiquem por meio de questões para absorver o conteúdo, criando uma alta demanda sobre os professores. Este artigo visa automatizar esse processo utilizando Modelos de Linguagem de Grande Escala (LLMs), aprimorando o aprendizado de maneira personalizada. Estudos fundamentais sobre geração automática de questões foram baseados em sistemas baseados em regras e correspondência de padrões; avanços recentes empregam técnicas de processamento de linguagem natural e aprendizado profundo para produzir questões mais diversificadas e sensíveis ao contexto, sendo que os LLMs tornaram-se uma abordagem amplamente adotada e relevante para este campo. O sistema busca fornecer questões coerentes e contextualizadas com base em tópicos específicos, respeitando restrições pedagógicas que garantem o desenvolvimento gradual dos conceitos abordados. As questões foram geradas seguindo um modelo baseado em templates, permitindo maior controle tanto sobre a qualidade quanto sobre a variedade dos exercícios. Técnicas de *Prompt Engineering* foram utilizadas para garantir que cada questão seguisse um formato fixo (tópico, dificuldade, contexto, problema e variáveis). Entre os principais problemas identificados, destacam-se: (i) a geração de questões incompletas ou ambíguas; (ii) a introdução prematura de conceitos avançados em tópicos mais básicos, o que vai contra a progressão de aprendizagem esperada; e (iii) a dificuldade do modelo em manter uma diversidade de problemas dentro do mesmo contexto sem reformulações excessivamente semelhantes. Especialistas avaliaram uma amostra de questões com base em fatores como relevância para os objetivos de aprendizagem, clareza e nível de dificuldade, considerando o modelo apropriado para o ensino de programação.Referências
Ai, R., Krause, S., Kasper, W., Xu, F., and Uszkoreit, H. (2015). Semi-automatic generation of multiple-choice tests from mentions of semantic relations. In 2nd Workshop on Natural Language Processing Techniques for Educational Applications, pages 26–33.
Chan, W., An, A., and Davoudi, H. (2023). A case study on chatgpt question generation. In 2023 IEEE International Conference on Big Data (BigData), pages 1647–1656.
Chinkina, M., Ruiz, S., and Meurers, D. (2017). Automatically generating questions to support the acquisition of particle verbs: Evaluating via crowdsourcing. In CALL in a Climate of Change: Adapting to Turbulent Global Conditions, pages 73–78.
Doughty, J., Wan, Z., Bompelli, A., Qayum, J., Wang, T., Zhang, J., Zheng, Y., Doyle, A., Sridhar, P., Agarwal, A., Bogart, C., Keylor, E., Kultur, C., Savelka, J., and Sakr, M. (2024). A comparative study of ai-generated (gpt-4) and human-crafted mcqs in programming education. In 26th Australasian Computing Education Conference (ACE ’24), pages 114–123, New York, NY, USA. Association for Computing Machinery.
Fatih, M. Z. A. and Romadhony, A. (2023). Automatic true/false question generation using template-based framework. In 2023 International Conference on Data Science and Its Applications (ICoDSA), pages 403–407, Bandung, Indonesia.
Huang, Y. T. and Mostow, J. (2015). Evaluating human and automated generation of distractors for diagnostic multiple-choice cloze questions to assess children’s reading comprehension. In Conati, C., Heffernan, N., Mitrovic, A., and Verdejo, M., editors, Artificial Intelligence in Education, pages 155–164. Springer International Publishing, Cham.
Kumar, G., Banchs, R., and D’Haro, L. F. (2015). Automatic fill-the-blank question generator for student self-assessment. In IEEE Frontiers in Education Conference (FIE), pages 1–3.
Kurdi, G., Leo, J., and Parsia, B. e. a. (2020). A systematic review of automatic question generation for educational purposes. International Journal of Artificial Intelligence in Education, 30:121–204.
Luen William, C. W. and Lim, T. M. (2024). Comparative studies: Leveraging large language model in theoretical and practical assessment sample question-answer bank on programming related subjects. In IEEE 4th International Conference on Electronic Communications, Internet of Things and Big Data (ICEIB), pages 331–335, Taipei, Taiwan.
Meißner, N., Speth, S., and Becker, S. (2024). Automated programming exercise generation in the era of large language models. In 36th International Conference on Software Engineering Education and Training (CSEET), pages 1–5, Würzburg, Germany.
Mostow, J., Huang, Y. T., Jang, H., Weinstein, A., Valeri, J., and Gates, D. (2017). Developing, evaluating, and refining an automatic generator of diagnostic multiple choice cloze questions to assess children’s comprehension while reading. Natural Language Engineering, 23(2):245–294.
Naeiji, A., An, A., Davoudi, H., Delpisheh, M., and Alzghool, M. (2023). Question generation using sequence-to-sequence model with semantic role labels. In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 2830–2842, Dubrovnik, Croatia. Association for Computational Linguistics.
Niu, Y. and Xue, H. (2023). Exercise generation and student cognitive ability research based on chatgpt and rasch model. IEEE Access, 11:116695–116705.
Odilinye, L., Popowich, F., Zhang, E., Nesbit, J., and Winne, P. H. (2015). Aligning automatically generated questions to instructor goals and learner behaviour. In IEEE 9th International Conference on Semantic Computing (ICS), pages 216–223.
Santana, A., Silva, F., Dantas, J., Souza, J., and Aranha, E. (2025). Geração automática de questões de programação usando llm: Um relato de experiência. In Anais do XXXIII Workshop sobre Educação em Computação, pages 1415–1425, Porto Alegre, RS, Brasil. SBC.
Thalheimer, W. (2003). The learning benefits of questions. Technical report, Work Learning Research.
Chan, W., An, A., and Davoudi, H. (2023). A case study on chatgpt question generation. In 2023 IEEE International Conference on Big Data (BigData), pages 1647–1656.
Chinkina, M., Ruiz, S., and Meurers, D. (2017). Automatically generating questions to support the acquisition of particle verbs: Evaluating via crowdsourcing. In CALL in a Climate of Change: Adapting to Turbulent Global Conditions, pages 73–78.
Doughty, J., Wan, Z., Bompelli, A., Qayum, J., Wang, T., Zhang, J., Zheng, Y., Doyle, A., Sridhar, P., Agarwal, A., Bogart, C., Keylor, E., Kultur, C., Savelka, J., and Sakr, M. (2024). A comparative study of ai-generated (gpt-4) and human-crafted mcqs in programming education. In 26th Australasian Computing Education Conference (ACE ’24), pages 114–123, New York, NY, USA. Association for Computing Machinery.
Fatih, M. Z. A. and Romadhony, A. (2023). Automatic true/false question generation using template-based framework. In 2023 International Conference on Data Science and Its Applications (ICoDSA), pages 403–407, Bandung, Indonesia.
Huang, Y. T. and Mostow, J. (2015). Evaluating human and automated generation of distractors for diagnostic multiple-choice cloze questions to assess children’s reading comprehension. In Conati, C., Heffernan, N., Mitrovic, A., and Verdejo, M., editors, Artificial Intelligence in Education, pages 155–164. Springer International Publishing, Cham.
Kumar, G., Banchs, R., and D’Haro, L. F. (2015). Automatic fill-the-blank question generator for student self-assessment. In IEEE Frontiers in Education Conference (FIE), pages 1–3.
Kurdi, G., Leo, J., and Parsia, B. e. a. (2020). A systematic review of automatic question generation for educational purposes. International Journal of Artificial Intelligence in Education, 30:121–204.
Luen William, C. W. and Lim, T. M. (2024). Comparative studies: Leveraging large language model in theoretical and practical assessment sample question-answer bank on programming related subjects. In IEEE 4th International Conference on Electronic Communications, Internet of Things and Big Data (ICEIB), pages 331–335, Taipei, Taiwan.
Meißner, N., Speth, S., and Becker, S. (2024). Automated programming exercise generation in the era of large language models. In 36th International Conference on Software Engineering Education and Training (CSEET), pages 1–5, Würzburg, Germany.
Mostow, J., Huang, Y. T., Jang, H., Weinstein, A., Valeri, J., and Gates, D. (2017). Developing, evaluating, and refining an automatic generator of diagnostic multiple choice cloze questions to assess children’s comprehension while reading. Natural Language Engineering, 23(2):245–294.
Naeiji, A., An, A., Davoudi, H., Delpisheh, M., and Alzghool, M. (2023). Question generation using sequence-to-sequence model with semantic role labels. In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 2830–2842, Dubrovnik, Croatia. Association for Computational Linguistics.
Niu, Y. and Xue, H. (2023). Exercise generation and student cognitive ability research based on chatgpt and rasch model. IEEE Access, 11:116695–116705.
Odilinye, L., Popowich, F., Zhang, E., Nesbit, J., and Winne, P. H. (2015). Aligning automatically generated questions to instructor goals and learner behaviour. In IEEE 9th International Conference on Semantic Computing (ICS), pages 216–223.
Santana, A., Silva, F., Dantas, J., Souza, J., and Aranha, E. (2025). Geração automática de questões de programação usando llm: Um relato de experiência. In Anais do XXXIII Workshop sobre Educação em Computação, pages 1415–1425, Porto Alegre, RS, Brasil. SBC.
Thalheimer, W. (2003). The learning benefits of questions. Technical report, Work Learning Research.
Publicado
24/11/2025
Como Citar
SANTANA, Abner; SILVA, Francisco Genivan; SOUZA, Jadson Lucas Gomes; DANTAS, Júlio César da S.; ARANHA, Eduardo Henrique da Silva.
Geração de questões de programação baseada em templates e IA generativa. In: WORKSHOP DE INFORMÁTICA NA ESCOLA (WIE), 31. , 2025, Curitiba/PR.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2025
.
p. 437-448.
DOI: https://doi.org/10.5753/wie.2025.13388.
