Evidências sobre o uso do ChatGPT no ensino de modelagem de software: um experimento controlado
Resumo
Chatbots baseados em LLM têm sido explorados em diversos domínios e com diferentes propósitos, incluindo como mecanismos de apoio ao ensino. Quando utilizados como recursos educacionais, é essencial compreender os efeitos de seu uso. Este trabalho descreve um experimento controlado que analisou os efeitos da utilização do ChatGPT 3.5 no apoio ao ensino de modelagem de software, especificamente na construção de diagramas UML. O experimento foi elaborado com o objetivo de avaliar a eficácia e a eficiência dos estudantes na elaboração de diagramas de casos de uso, classes e atividades, verificando também o ganho de aprendizagem proporcionado pelo uso desse recurso. Os resultados indicaram que os estudantes que utilizaram o ChatGPT demonstraram em média uma maior eficácia e eficiência na produção dos modelos. Além disso, esses estudantes apresentaram um ganho de aprendizagem superior em comparação àqueles que realizaram a modelagem sem o suporte do ChatGPT.Referências
Abukhalaf, S., Hamdaqa, M., and Khomh, F. (2023). On codex prompt engineering for ocl generation: An empirical study. In 2023 IEEE/ACM 20th International Conference on Mining Software Repositories (MSR), pages 148–157.
Baidoo-Anu, D. and Ansah, L. O. (2023). Education in the era of generative artificial intelligence (ai): Understanding the potential benefits of chatgpt in promoting teaching and learning. Journal of AI, 7(1):52–62.
Basili, V. R., Caldiera, G., and Rombach, D. H. (1994). The Goal Question Metric Approach, volume I. John Wiley & Sons.
Cámara, J., Troya, J., Burgueño, L., and Vallecillo, A. (2023). On the assessment of generative ai in modeling tasks: an experience report with chatgpt and uml. Softw. Syst. Model., 22(3):781–793.
Chaaben, M. B., Burgueño, L., and Sahraoui, H. (2023). Towards using few-shot prompt learning for automating model completion. In Proceedings of the 45th International Conference on Software Engineering: New Ideas and Emerging Results, ICSE-NIER ’23, page 7–12. IEEE Press.
Choudhuri, R., Liu, D., Steinmacher, I., Gerosa, M., and Sarma, A. (2024). How far are we? the triumphs and trials of generative ai in learning software engineering. In 2024 IEEE/ACM 46th International Conference on Software Engineering (ICSE), ICSE ’24, pages 2270–2282, Los Alamitos, CA, USA. IEEE Computer Society.
Eysenbach, G. et al. (2023). The role of chatgpt, generative language models, and artificial intelligence in medical education: a conversation with chatgpt and a call for papers. JMIR medical education, 9(1):e46885.
Golgiyaz, S. (2023). Chatgpt in computer software education. ICHEAS, 4th International Conference On Health, Engineering And Applied Sciences., pages 115–126.
Jalil, S., Rafi, S., LaToza, T. D., Moran, K., and Lam, W. (2023). Chatgpt and software testing education: Promises perils. In 2023 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW), pages 4130–4137.
Khojah, R., Mohamad, M., Leitner, P., and de Oliveira Neto, F. G. (2024). Beyond code generation: An observational study of chatgpt usage in software engineering practice. Proc. ACM Softw. Eng., 1(FSE).
Kosar, T., Ostojić, D., Liu, Y. D., and Mernik, M. (2024). Computer science education in chatgpt era: Experiences from an experiment in a programming course for novice programmers. Mathematics, 12(5).
Li, Y., Xu, J., Zhu, Y., Liu, H., and Liu, P. (2023). The impact of chatgpt on software engineering education: A quick peek. In 2023 10th International Conference on Dependable Systems and Their Applications (DSA), pages 595–596.
Paschoal, L. N. (2024). Um framework para o planejamento de experimentos controlados na pesquisa de chatbots educacionais. Tese de doutorado, Instituto de Ciências Matemáticas e de Computação, University of São Paulo, São Carlos.
Rodriguez-Echeverría, R., Gutiérrez, J. D., Conejero, J. M., and Prieto, A. E. (2024). Analysis of chatgpt performance in computer engineering exams. IEEE Revista Ibero-americana de Tecnologias del Aprendizaje, 19:71–80.
Ronanki, K., Berger, C., and Horkoff, J. (2023). Investigating chatgpt’s potential to assist in requirements elicitation processes. In 2023 49th Euromicro Conference on Software Engineering and Advanced Applications (SEAA), pages 354–361.
Ronanki, K., Cabrero-Daniel, B., Horkoff, J., and Berger, C. (2024). Requirements Engineering Using Generative AI: Prompts and Prompting Patterns, pages 109–127. Springer Nature Switzerland, Cham.
Russo, D. (2024). Navigating the complexity of generative ai adoption in software engineering. ACM Trans. Softw. Eng. Methodol., 33(5).
Sami, M. A., Rasheed, Z., Waseem, M., Zhang, Z., Herda, T., and Abrahamsson, P. (2024). Prioritizing software requirements using large language models. CoRR, abs/2405.01564.
Schäfer, M., Nadi, S., Eghbali, A., and Tip, F. (2024). An empirical evaluation of using large language models for automated unit test generation. IEEE Transactions on Software Engineering, 50(1):85–105.
Wohlin, C., Runeson, P., Hst, M., Ohlsson, M. C., Regnell, B., and Wessln, A. (2012). Experimentation in Software Engineering. Springer Publishing Company, Incorporated.
Xue, Y., Chen, H., Bai, G. R., Tairas, R., and Huang, Y. (2024). Does chatgpt help with introductory programming?an experiment of students using chatgpt in cs1. In Proceedings of the 46th International Conference on Software Engineering: Software Engineering Education and Training, ICSE-SEET ’24, page 331–341, New York, NY, USA. Association for Computing Machinery.
Baidoo-Anu, D. and Ansah, L. O. (2023). Education in the era of generative artificial intelligence (ai): Understanding the potential benefits of chatgpt in promoting teaching and learning. Journal of AI, 7(1):52–62.
Basili, V. R., Caldiera, G., and Rombach, D. H. (1994). The Goal Question Metric Approach, volume I. John Wiley & Sons.
Cámara, J., Troya, J., Burgueño, L., and Vallecillo, A. (2023). On the assessment of generative ai in modeling tasks: an experience report with chatgpt and uml. Softw. Syst. Model., 22(3):781–793.
Chaaben, M. B., Burgueño, L., and Sahraoui, H. (2023). Towards using few-shot prompt learning for automating model completion. In Proceedings of the 45th International Conference on Software Engineering: New Ideas and Emerging Results, ICSE-NIER ’23, page 7–12. IEEE Press.
Choudhuri, R., Liu, D., Steinmacher, I., Gerosa, M., and Sarma, A. (2024). How far are we? the triumphs and trials of generative ai in learning software engineering. In 2024 IEEE/ACM 46th International Conference on Software Engineering (ICSE), ICSE ’24, pages 2270–2282, Los Alamitos, CA, USA. IEEE Computer Society.
Eysenbach, G. et al. (2023). The role of chatgpt, generative language models, and artificial intelligence in medical education: a conversation with chatgpt and a call for papers. JMIR medical education, 9(1):e46885.
Golgiyaz, S. (2023). Chatgpt in computer software education. ICHEAS, 4th International Conference On Health, Engineering And Applied Sciences., pages 115–126.
Jalil, S., Rafi, S., LaToza, T. D., Moran, K., and Lam, W. (2023). Chatgpt and software testing education: Promises perils. In 2023 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW), pages 4130–4137.
Khojah, R., Mohamad, M., Leitner, P., and de Oliveira Neto, F. G. (2024). Beyond code generation: An observational study of chatgpt usage in software engineering practice. Proc. ACM Softw. Eng., 1(FSE).
Kosar, T., Ostojić, D., Liu, Y. D., and Mernik, M. (2024). Computer science education in chatgpt era: Experiences from an experiment in a programming course for novice programmers. Mathematics, 12(5).
Li, Y., Xu, J., Zhu, Y., Liu, H., and Liu, P. (2023). The impact of chatgpt on software engineering education: A quick peek. In 2023 10th International Conference on Dependable Systems and Their Applications (DSA), pages 595–596.
Paschoal, L. N. (2024). Um framework para o planejamento de experimentos controlados na pesquisa de chatbots educacionais. Tese de doutorado, Instituto de Ciências Matemáticas e de Computação, University of São Paulo, São Carlos.
Rodriguez-Echeverría, R., Gutiérrez, J. D., Conejero, J. M., and Prieto, A. E. (2024). Analysis of chatgpt performance in computer engineering exams. IEEE Revista Ibero-americana de Tecnologias del Aprendizaje, 19:71–80.
Ronanki, K., Berger, C., and Horkoff, J. (2023). Investigating chatgpt’s potential to assist in requirements elicitation processes. In 2023 49th Euromicro Conference on Software Engineering and Advanced Applications (SEAA), pages 354–361.
Ronanki, K., Cabrero-Daniel, B., Horkoff, J., and Berger, C. (2024). Requirements Engineering Using Generative AI: Prompts and Prompting Patterns, pages 109–127. Springer Nature Switzerland, Cham.
Russo, D. (2024). Navigating the complexity of generative ai adoption in software engineering. ACM Trans. Softw. Eng. Methodol., 33(5).
Sami, M. A., Rasheed, Z., Waseem, M., Zhang, Z., Herda, T., and Abrahamsson, P. (2024). Prioritizing software requirements using large language models. CoRR, abs/2405.01564.
Schäfer, M., Nadi, S., Eghbali, A., and Tip, F. (2024). An empirical evaluation of using large language models for automated unit test generation. IEEE Transactions on Software Engineering, 50(1):85–105.
Wohlin, C., Runeson, P., Hst, M., Ohlsson, M. C., Regnell, B., and Wessln, A. (2012). Experimentation in Software Engineering. Springer Publishing Company, Incorporated.
Xue, Y., Chen, H., Bai, G. R., Tairas, R., and Huang, Y. (2024). Does chatgpt help with introductory programming?an experiment of students using chatgpt in cs1. In Proceedings of the 46th International Conference on Software Engineering: Software Engineering Education and Training, ICSE-SEET ’24, page 331–341, New York, NY, USA. Association for Computing Machinery.
Publicado
20/07/2025
Como Citar
MURAD, Samir B.; LEMOS, Fabricio F. S.; MELO, Silvana M.; PASCHOAL, Leo Natan; PRATES, Jorge M..
Evidências sobre o uso do ChatGPT no ensino de modelagem de software: um experimento controlado. In: WORKSHOP SOBRE EDUCAÇÃO EM COMPUTAÇÃO (WEI), 33. , 2025, Maceió/AL.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2025
.
p. 1289-1300.
ISSN 2595-6175.
DOI: https://doi.org/10.5753/wei.2025.9261.
