Análise das Respostas do ChatGPT em Relação ao Conteúdo de Programação para Iniciantes

• Luiz C. Pereira Filho IFSP
• Talita de P. C. de Souza IFSP
• Luciano Bernardes de Paula IFSP

DOI: https://doi.org/10.5753/sbie.2023.234870

Abstract

Teaching computer programming is considered important and, at the same time, challenging. Currently, AI tools have opened up a range of possibilities for use in education. The best known example is ChatGPT, which has natural language interaction and, in relation to basic programming content, demonstrates skills in creating, correcting and explaining codes. The aim of this paper was to analyze ChatGPT responses regarding initial programming content, in the context of beginning students. Qualitative tests were performed, in which the ChatGPT showed the potential to give correct answers and consistent explanations, and quantitative tests, with a success rate greater than 80%.

References

Aljanabi, M., Ghazi, M., Ali, A. H., Abed, S. A., and ChatGpt (2023). Chatgpt: Open possibilities. Iraqi Journal For Computer Science and Mathematics, 4(1):62–64.

Cámara, J., Troya, J., Burgueño, L., and Vallecillo, A. (2023). On the assessment of generative ai in modeling tasks: an experience report with chatgpt and uml. Software and Systems Modeling, 22(3):781–793.

du Boulay, J. B. H. (1986). Some difficulties of learning to program. Journal of Educational Computing Research, 2(1):57–73.

Finnie-Ansley, J., Denny, P., Becker, B. A., Luxton-Reilly, A., and Prather, J. (2022). The robots are coming: Exploring the implications of openai codex on introductory programming. ACE ’22, page 10–19, New York, NY, USA. ACM.

Finnie-Ansley, J., Denny, P., Luxton-Reilly, A., Santos, E. A., Prather, J., and Becker, B. A. (2023). My ai wants to know if this will be on the exam: Testing openai’s codex on cs2 programming exercises. In Proceedings of the 25th Australasian Computing Education Conference, ACE ’23, page 97–104, New York, NY, USA. ACM.

Lee, M. (2023). A mathematical investigation of hallucination and creativity in gpt models. Mathematics, 11(10).

Li, Y., Choi, D., Chung, J., Kushman, N., Schrittwieser, J., Leblond, R., Eccles, T., Keeling, J., Gimeno, F., Lago, A. D., Hubert, T., Choy, P., de Masson d’Autume, C., Babuschkin, I., Chen, X., Huang, P.-S., Welbl, J., Gowal, S., Cherepanov, A., Molloy, J., Mankowitz, D. J., Robson, E. S., Kohli, P., de Freitas, N., Kavukcuoglu, K., and Vinyals, O. (2022). Competition-level code generation with alphacode. Science, 378(6624):1092–1097.

Lo, C. K. (2023). What is the impact of chatgpt on education? a rapid review of the literature. Education Sciences, 13(4).

MacNeil, S., Tran, A., Mogil, D., Bernstein, S., Ross, E., and Huang, Z. (2022). Generating diverse code explanations using the gpt-3 large language model. In Proceedings of the 2022 ACM Conference on International Computing Education Research - Volume 2, ICER ’22, page 37–39, New York, NY, USA. Association for Computing Machinery.

Mizrahi, V. V. (2008). Treinamento em Linguagem C, volume 1. Person Prentice Hall.

OpenAI (2023). Chatgpt. https://openai.com/chatgpt. Acessado em 10/07/2023.

Sarsa, S., Denny, P., Hellas, A., and Leinonen, J. (2022). Automatic generation of programming exercises and code explanations using large language models. In Proc. of the 2022 ACM Conf. on International Computing Education Research V.1. ACM.

Tsai, M.-L., Ong, C.W., and Chen, C.-L. (2023). Exploring the use of large language models (llms) in chemical engineering education: Building core course problem models with chat-gpt. Education for Chemical Engineers, 44:71–95.