Análise das Respostas do ChatGPT em Relação ao Conteúdo de Programação para Iniciantes
Resumo
O ensino de programação de computadores é considerado importante e, ao mesmo tempo, desafiador. Atualmente, ferramentas de IA abriram um leque de possibilidades para uso na educação. O exemplo mais conhecido é o ChatGPT, que possui interação em linguagem natural e, em relação ao conteúdo de programação básica, demonstra habilidades de criação, correção e explicação de códigos. O objetivo deste artigo foi analisar respostas do ChatGPT em relação ao conteúdo inicial de programação, no contexto de estudantes iniciantes. Foram realizados testes qualitativos, nos os quais o ChatGPT mostrou potencial para dar respostas corretas e explicações consistentes, e testes quantitativos, com aproveitamento superior a 80%.Referências
Aljanabi, M., Ghazi, M., Ali, A. H., Abed, S. A., and ChatGpt (2023). Chatgpt: Open possibilities. Iraqi Journal For Computer Science and Mathematics, 4(1):62–64.
Cámara, J., Troya, J., Burgueño, L., and Vallecillo, A. (2023). On the assessment of generative ai in modeling tasks: an experience report with chatgpt and uml. Software and Systems Modeling, 22(3):781–793.
du Boulay, J. B. H. (1986). Some difficulties of learning to program. Journal of Educational Computing Research, 2(1):57–73.
Finnie-Ansley, J., Denny, P., Becker, B. A., Luxton-Reilly, A., and Prather, J. (2022). The robots are coming: Exploring the implications of openai codex on introductory programming. ACE ’22, page 10–19, New York, NY, USA. ACM.
Finnie-Ansley, J., Denny, P., Luxton-Reilly, A., Santos, E. A., Prather, J., and Becker, B. A. (2023). My ai wants to know if this will be on the exam: Testing openai’s codex on cs2 programming exercises. In Proceedings of the 25th Australasian Computing Education Conference, ACE ’23, page 97–104, New York, NY, USA. ACM.
Lee, M. (2023). A mathematical investigation of hallucination and creativity in gpt models. Mathematics, 11(10).
Li, Y., Choi, D., Chung, J., Kushman, N., Schrittwieser, J., Leblond, R., Eccles, T., Keeling, J., Gimeno, F., Lago, A. D., Hubert, T., Choy, P., de Masson d’Autume, C., Babuschkin, I., Chen, X., Huang, P.-S., Welbl, J., Gowal, S., Cherepanov, A., Molloy, J., Mankowitz, D. J., Robson, E. S., Kohli, P., de Freitas, N., Kavukcuoglu, K., and Vinyals, O. (2022). Competition-level code generation with alphacode. Science, 378(6624):1092–1097.
Lo, C. K. (2023). What is the impact of chatgpt on education? a rapid review of the literature. Education Sciences, 13(4).
MacNeil, S., Tran, A., Mogil, D., Bernstein, S., Ross, E., and Huang, Z. (2022). Generating diverse code explanations using the gpt-3 large language model. In Proceedings of the 2022 ACM Conference on International Computing Education Research - Volume 2, ICER ’22, page 37–39, New York, NY, USA. Association for Computing Machinery.
Mizrahi, V. V. (2008). Treinamento em Linguagem C, volume 1. Person Prentice Hall.
OpenAI (2023). Chatgpt. https://openai.com/chatgpt. Acessado em 10/07/2023.
Sarsa, S., Denny, P., Hellas, A., and Leinonen, J. (2022). Automatic generation of programming exercises and code explanations using large language models. In Proc. of the 2022 ACM Conf. on International Computing Education Research V.1. ACM.
Tsai, M.-L., Ong, C.W., and Chen, C.-L. (2023). Exploring the use of large language models (llms) in chemical engineering education: Building core course problem models with chat-gpt. Education for Chemical Engineers, 44:71–95.
Cámara, J., Troya, J., Burgueño, L., and Vallecillo, A. (2023). On the assessment of generative ai in modeling tasks: an experience report with chatgpt and uml. Software and Systems Modeling, 22(3):781–793.
du Boulay, J. B. H. (1986). Some difficulties of learning to program. Journal of Educational Computing Research, 2(1):57–73.
Finnie-Ansley, J., Denny, P., Becker, B. A., Luxton-Reilly, A., and Prather, J. (2022). The robots are coming: Exploring the implications of openai codex on introductory programming. ACE ’22, page 10–19, New York, NY, USA. ACM.
Finnie-Ansley, J., Denny, P., Luxton-Reilly, A., Santos, E. A., Prather, J., and Becker, B. A. (2023). My ai wants to know if this will be on the exam: Testing openai’s codex on cs2 programming exercises. In Proceedings of the 25th Australasian Computing Education Conference, ACE ’23, page 97–104, New York, NY, USA. ACM.
Lee, M. (2023). A mathematical investigation of hallucination and creativity in gpt models. Mathematics, 11(10).
Li, Y., Choi, D., Chung, J., Kushman, N., Schrittwieser, J., Leblond, R., Eccles, T., Keeling, J., Gimeno, F., Lago, A. D., Hubert, T., Choy, P., de Masson d’Autume, C., Babuschkin, I., Chen, X., Huang, P.-S., Welbl, J., Gowal, S., Cherepanov, A., Molloy, J., Mankowitz, D. J., Robson, E. S., Kohli, P., de Freitas, N., Kavukcuoglu, K., and Vinyals, O. (2022). Competition-level code generation with alphacode. Science, 378(6624):1092–1097.
Lo, C. K. (2023). What is the impact of chatgpt on education? a rapid review of the literature. Education Sciences, 13(4).
MacNeil, S., Tran, A., Mogil, D., Bernstein, S., Ross, E., and Huang, Z. (2022). Generating diverse code explanations using the gpt-3 large language model. In Proceedings of the 2022 ACM Conference on International Computing Education Research - Volume 2, ICER ’22, page 37–39, New York, NY, USA. Association for Computing Machinery.
Mizrahi, V. V. (2008). Treinamento em Linguagem C, volume 1. Person Prentice Hall.
OpenAI (2023). Chatgpt. https://openai.com/chatgpt. Acessado em 10/07/2023.
Sarsa, S., Denny, P., Hellas, A., and Leinonen, J. (2022). Automatic generation of programming exercises and code explanations using large language models. In Proc. of the 2022 ACM Conf. on International Computing Education Research V.1. ACM.
Tsai, M.-L., Ong, C.W., and Chen, C.-L. (2023). Exploring the use of large language models (llms) in chemical engineering education: Building core course problem models with chat-gpt. Education for Chemical Engineers, 44:71–95.
Publicado
06/11/2023
Como Citar
PEREIRA FILHO, Luiz C.; SOUZA, Talita de P. C. de; PAULA, Luciano Bernardes de.
Análise das Respostas do ChatGPT em Relação ao Conteúdo de Programação para Iniciantes. In: SIMPÓSIO BRASILEIRO DE INFORMÁTICA NA EDUCAÇÃO (SBIE), 34. , 2023, Passo Fundo/RS.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2023
.
p. 1738-1748.
DOI: https://doi.org/10.5753/sbie.2023.234870.
