Utilizando Question Answering no Auxílio ao Processo de Ensino e Aprendizagem de Programação: Um Estudo de Caso com BERT e ChatGPT
Resumo
Natural Language Processing is an area of Artificial Intelligence that has brought benefits to the most diverse processes of human activity. In this context, this work uses a Question Answering (QA) approach to help in the process of teaching and learning programming. For this purpose, it analyzed two case studies with different QA models and a database with 87 questions and answers related to teaching programming. Thus, the models achieved 62% and 85% accuracy in the first and second scenarios, respectively. The present work also discusses (i) some limitations of the approach, (ii) some qualitative pedagogical results, and, finally, indicates (iii) recommendations for future work.
Referências
Bispo Jr., E. L., Raabe, A., Matos, E. d. S., Maschio, E., Barbosa, E. F., Carvalho, L. G., Bittencourt, R. A., Duran, R. S., e Falcão, T. P. d. R. (2020). Tecnologias na Educaçao em Computaçao: Primeiros referenciais. RBIE, 28:509–527.
Brown, T. et al. (2020). Language models are few-shot learners. In Advances in Neural Information Processing Systems, volume 33, pages 1877–1901.
Farias, A. M. (2019). How to create your own question-answering system easily with Python. Disponível em: [link]. Acesso em: 27 de maio de 2023.
Gil, A. C. et al. (2017). Como elaborar projetos de pesquisa, volume 6. Atlas São Paulo.
Jurafsky, D. e Martin, J. H. (2023). Speech and Language Processing. Prentice Hall, 3 edition. Disponível em [link].
Lende, S. P. e Raghuwanshi, M. (2016). Question answering system on education acts using NLP techniques. In 2016 world conference on futuristic trends in research and innovation for social welfare (Startup Conclave), pages 1–6. IEEE.
Mohler, M., Bunescu, R., e Mihalcea, R. (2011). Learning to grade short answer questions using semantic similarity measures and dependency graph alignments. In Proceedings of the 49th annual meeting of the association for computational linguistics: Human language technologies, pages 752–762.
Rajpurkar, P., Zhang, J., Lopyrev, K., e Liang, P. (2016). Squad: 100,000+ questions for machine comprehension of text. arXiv preprint arXiv:1606.05250.
Roberts, A., Raffel, C., e Shazeer, N. (2020). How much knowledge can you pack into the parameters of a language model? arXiv preprint arXiv:2002.08910.
Soares, M. A. C. e Parreiras, F. S. (2020). A literature review on question answering techniques, paradigms and systems. Journal of King Saud University-Computer and Information Sciences, 32(6):635–646.
Swathilakshmi, V. et al. (2021). Student suite+, a closed domain question answering system for educational domain. Turkish Journal of Computer and Mathematics Education (TURCOMAT), 12(10):3168–3172.
Brown, T. et al. (2020). Language models are few-shot learners. In Advances in Neural Information Processing Systems, volume 33, pages 1877–1901.
Farias, A. M. (2019). How to create your own question-answering system easily with Python. Disponível em: [link]. Acesso em: 27 de maio de 2023.
Gil, A. C. et al. (2017). Como elaborar projetos de pesquisa, volume 6. Atlas São Paulo.
Jurafsky, D. e Martin, J. H. (2023). Speech and Language Processing. Prentice Hall, 3 edition. Disponível em [link].
Lende, S. P. e Raghuwanshi, M. (2016). Question answering system on education acts using NLP techniques. In 2016 world conference on futuristic trends in research and innovation for social welfare (Startup Conclave), pages 1–6. IEEE.
Mohler, M., Bunescu, R., e Mihalcea, R. (2011). Learning to grade short answer questions using semantic similarity measures and dependency graph alignments. In Proceedings of the 49th annual meeting of the association for computational linguistics: Human language technologies, pages 752–762.
Rajpurkar, P., Zhang, J., Lopyrev, K., e Liang, P. (2016). Squad: 100,000+ questions for machine comprehension of text. arXiv preprint arXiv:1606.05250.
Roberts, A., Raffel, C., e Shazeer, N. (2020). How much knowledge can you pack into the parameters of a language model? arXiv preprint arXiv:2002.08910.
Soares, M. A. C. e Parreiras, F. S. (2020). A literature review on question answering techniques, paradigms and systems. Journal of King Saud University-Computer and Information Sciences, 32(6):635–646.
Swathilakshmi, V. et al. (2021). Student suite+, a closed domain question answering system for educational domain. Turkish Journal of Computer and Mathematics Education (TURCOMAT), 12(10):3168–3172.
Publicado
06/08/2023
Como Citar
FREIRE, Marcelo de L.; FEITOSA, Robson G. Fechine; SANTOS, Yuri D.; MENEZES, Hanna; ESMERALDO, Guilherme Á. R. M.; MELLO, Harley M. de; BISPO JR., Esdras L.; CAMPOS, Gustavo A. L. de.
Utilizando Question Answering no Auxílio ao Processo de Ensino e Aprendizagem de Programação: Um Estudo de Caso com BERT e ChatGPT. In: ENCONTRO NACIONAL DE COMPUTAÇÃO DOS INSTITUTOS FEDERAIS (ENCOMPIF), 10. , 2023, João Pessoa/PB.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2023
.
p. 77-84.
ISSN 2763-8766.
DOI: https://doi.org/10.5753/encompif.2023.230661.
