Análise Automática de Feedback em Ambientes de Aprendizagem Online
Resumo
O feedback é um componente muito importante no processo de ensino-aprendizagem, pois ajuda aos alunos a identificar as lacunas no aprendizado e avaliar o seu progresso no aprendizado. Em cursos a distância o feedback se torna ainda mais importante, pois é um dos recursos mais utilizados na interação entre professor e aluno, já que ambos estão separados fisicamente. No entanto, devido ao crescimento significativo da quantidade de alunos em cursos a distância, é difícil para os instrutores fornecer um feedback de alta qualidade. Nesse contexto, este artigo tem como objetivo propor uma abordagem para analisar automaticamente os feedback fornecidos por professores em cursos online. Para isso, foram utilizados recursos linguísticos para extrair as características dos textos e os algoritmos de aprendizagem de máquina AdaBoost e XGBoost para classificação. Os resultados encontrados mostram uma grande melhoria na acurácia e Kappa quando comparado com um estudo anterior.
Palavras-chave:
Feedback Educacional, Mineração de Texto, Ambientes Virtuais de Aprendizagem
Referências
[Akçapınar 2015] Akçapınar, G. (2015). How automated feedback through text mining changes plagiaristic behavior in online assignments. Computers & Education, 87:123–130.
[Balage Filho et al. 2013] Balage Filho, P. P., Pardo, T. A. S., and Aluísio, S. M. (2013). An evaluation of the brazilian portuguese liwc dictionary for sentiment analysis. In Proceedings of the 9th Brazilian Symposium in Information and Human Language Technology, pages 215–219. Sociedade Brasileira de Computação.
[Belcadhi 2016] Belcadhi, L. C. (2016). Personalized feedback for self assessment in lifelong learning environments based on semantic web. Computers in Human Behavior, 55:562–570.
[Breiman 2001] Breiman, L. (2001). Random forests. Machine learning, 45(1):5–32.
[Burke 2009] Burke, D. (2009). Strategies for using feedback students bring to higher education. Assessment & Evaluation in Higher Education, 34(1):41–50.
[Carvalho and Silva 2015] Carvalho, P. and Silva, M. J. (2015). Sentilex-pt: Principais características e potencialidades. Oslo Studies in Language, 7(1).
[Cavalcanti et al. 2020] Cavalcanti, A. P., Diego, A., Mello, R. F., Mangaroska, K., Nascimento, A., Freitas, F., and Gašević, D. (2020). How good is my feedback? a content analysis of written feedback. In Proceedings of the Tenth International Conference on Learning Analytics & Knowledge, pages 428–437.
[Cavalcanti et al. 2019] Cavalcanti, A. P., Mello, R. F., Rolim, V., André, M., Freitas, F., and Gašević, D. (2019). An analysis of the use of good feedback practices in online learning courses. In 2019 IEEE 19th International Conference on Advanced Learning Technologies (ICALT), volume 2161, pages 153–157. IEEE.
[Chawla et al. 2002] Chawla, N. V., Bowyer, K. W., Hall, L. O., and Kegelmeyer, W. P. (2002). Smote: synthetic minority over-sampling technique. Journal of artificial intelligence research, 16:321–357.
[Chen and Guestrin 2016] Chen, T. and Guestrin, C. (2016). Xgboost: A scalable tree boosting system. In Proceedings of the 22nd acm sigkdd international conference on knowledge discovery and data mining, pages 785–794.
[Coates et al. 2005] Coates, H., James, R., and Baldwin, G. (2005). A critical examination of the effects of learning management systems on university teaching and learning. Tertiary education and management, 11:19–36.
[Correia and dos Santos 2013] Correia, R. L. and dos Santos, J. G. (2013). A importância da tecnologia da informação e comunicação (tic) na educação a distância (ead) do ensino superior (ies). Revista Aprendizagem em EAD, 2(1).
[Dietterich 2000] Dietterich, T. G. (2000). Ensemble methods in machine learning. In International workshop on multiple classifier systems, pages 1–15. Springer.
[Espasa and Meneses 2010] Espasa, A. and Meneses, J. (2010). Analysing feedback processes in an online teaching and learning environment: an exploratory study. Higher education, 59(3):277–292.
[Ferreira-Mello et al. 2019] Ferreira-Mello, R., André, M., Pinheiro, A., Costa, E., and Romero, C. (2019). Text mining in education. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, 9(6):e1332.
[Friedman et al. 2001] Friedman, J., Hastie, T., and Tibshirani, R. (2001). The elements of statistical learning, volume 1. Springer series in statistics New York, NY, USA.
[Hattie and Timperley 2007] Hattie, J. and Timperley, H. (2007). The power of feedback. Review of educational research, 77(1):81–112.
[Jeria and Villalon 2017] Jeria, H. and Villalon, J. (2017). Incorporating open education resources into computer supported marking tool to enhance formative feedback creation. In Advanced Learning Technologies (ICALT), 2017 IEEE 17th International Conference on, pages 256–260. IEEE.
[Langer 2011] Langer, P. (2011). The use of feedback in education: a complex instructional strategy. Psychological reports, 109(3):775–784.
[Maitra et al. 2018] Maitra, S., Madan, S., Kandwal, R., and Mahajan, P. (2018). Mining authentic student feedback for faculty using naıve bayes classifier. Procedia computer science, 132:1171–1183.
[Marin et al. 2017] Marin, V. J., Pereira, T., Sridharan, S., and Rivero, C. R. (2017). Automated personalized feedback in introductory java programming moocs. In 2017 IEEE 33rd International Conference on Data Engineering (ICDE), pages 1259–1270. IEEE.
[McNamara et al. 2014] McNamara, D. S., Graesser, A. C., McCarthy, P. M., and Cai, Z. (2014). Automated evaluation of text and discourse with Coh-Metrix. Cambridge University Press.
[Nicol and Macfarlane-Dick 2006] Nicol, D. J. and Macfarlane-Dick, D. (2006). Formative assessment and self-regulated learning: A model and seven principles of good feedback practice. Studies in higher education, 31(2):199–218.
[Sadler 1989] Sadler, D. R. (1989). Formative assessment and the design of instructional systems. Instructional science, 18(2):119–144.
[Sirotheau et al. 2019] Sirotheau, S., Santos, J., Favero, E., and de Freitas, S. N. (2019). Avaliação automática de respostas discursivas curtas baseado em três dimensões linguísticas. In Brazilian Symposium on Computers in Education (Simpósio Brasileiro de Informática na Educação- SBIE), volume 30, page 1551.
[Tausczik and Pennebaker 2010] Tausczik, Y. R. and Pennebaker, J. W. (2010). The psychological meaning of words: Liwc and computerized text analysis methods. Journal of language and social psychology, 29(1):24–54.
[Balage Filho et al. 2013] Balage Filho, P. P., Pardo, T. A. S., and Aluísio, S. M. (2013). An evaluation of the brazilian portuguese liwc dictionary for sentiment analysis. In Proceedings of the 9th Brazilian Symposium in Information and Human Language Technology, pages 215–219. Sociedade Brasileira de Computação.
[Belcadhi 2016] Belcadhi, L. C. (2016). Personalized feedback for self assessment in lifelong learning environments based on semantic web. Computers in Human Behavior, 55:562–570.
[Breiman 2001] Breiman, L. (2001). Random forests. Machine learning, 45(1):5–32.
[Burke 2009] Burke, D. (2009). Strategies for using feedback students bring to higher education. Assessment & Evaluation in Higher Education, 34(1):41–50.
[Carvalho and Silva 2015] Carvalho, P. and Silva, M. J. (2015). Sentilex-pt: Principais características e potencialidades. Oslo Studies in Language, 7(1).
[Cavalcanti et al. 2020] Cavalcanti, A. P., Diego, A., Mello, R. F., Mangaroska, K., Nascimento, A., Freitas, F., and Gašević, D. (2020). How good is my feedback? a content analysis of written feedback. In Proceedings of the Tenth International Conference on Learning Analytics & Knowledge, pages 428–437.
[Cavalcanti et al. 2019] Cavalcanti, A. P., Mello, R. F., Rolim, V., André, M., Freitas, F., and Gašević, D. (2019). An analysis of the use of good feedback practices in online learning courses. In 2019 IEEE 19th International Conference on Advanced Learning Technologies (ICALT), volume 2161, pages 153–157. IEEE.
[Chawla et al. 2002] Chawla, N. V., Bowyer, K. W., Hall, L. O., and Kegelmeyer, W. P. (2002). Smote: synthetic minority over-sampling technique. Journal of artificial intelligence research, 16:321–357.
[Chen and Guestrin 2016] Chen, T. and Guestrin, C. (2016). Xgboost: A scalable tree boosting system. In Proceedings of the 22nd acm sigkdd international conference on knowledge discovery and data mining, pages 785–794.
[Coates et al. 2005] Coates, H., James, R., and Baldwin, G. (2005). A critical examination of the effects of learning management systems on university teaching and learning. Tertiary education and management, 11:19–36.
[Correia and dos Santos 2013] Correia, R. L. and dos Santos, J. G. (2013). A importância da tecnologia da informação e comunicação (tic) na educação a distância (ead) do ensino superior (ies). Revista Aprendizagem em EAD, 2(1).
[Dietterich 2000] Dietterich, T. G. (2000). Ensemble methods in machine learning. In International workshop on multiple classifier systems, pages 1–15. Springer.
[Espasa and Meneses 2010] Espasa, A. and Meneses, J. (2010). Analysing feedback processes in an online teaching and learning environment: an exploratory study. Higher education, 59(3):277–292.
[Ferreira-Mello et al. 2019] Ferreira-Mello, R., André, M., Pinheiro, A., Costa, E., and Romero, C. (2019). Text mining in education. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, 9(6):e1332.
[Friedman et al. 2001] Friedman, J., Hastie, T., and Tibshirani, R. (2001). The elements of statistical learning, volume 1. Springer series in statistics New York, NY, USA.
[Hattie and Timperley 2007] Hattie, J. and Timperley, H. (2007). The power of feedback. Review of educational research, 77(1):81–112.
[Jeria and Villalon 2017] Jeria, H. and Villalon, J. (2017). Incorporating open education resources into computer supported marking tool to enhance formative feedback creation. In Advanced Learning Technologies (ICALT), 2017 IEEE 17th International Conference on, pages 256–260. IEEE.
[Langer 2011] Langer, P. (2011). The use of feedback in education: a complex instructional strategy. Psychological reports, 109(3):775–784.
[Maitra et al. 2018] Maitra, S., Madan, S., Kandwal, R., and Mahajan, P. (2018). Mining authentic student feedback for faculty using naıve bayes classifier. Procedia computer science, 132:1171–1183.
[Marin et al. 2017] Marin, V. J., Pereira, T., Sridharan, S., and Rivero, C. R. (2017). Automated personalized feedback in introductory java programming moocs. In 2017 IEEE 33rd International Conference on Data Engineering (ICDE), pages 1259–1270. IEEE.
[McNamara et al. 2014] McNamara, D. S., Graesser, A. C., McCarthy, P. M., and Cai, Z. (2014). Automated evaluation of text and discourse with Coh-Metrix. Cambridge University Press.
[Nicol and Macfarlane-Dick 2006] Nicol, D. J. and Macfarlane-Dick, D. (2006). Formative assessment and self-regulated learning: A model and seven principles of good feedback practice. Studies in higher education, 31(2):199–218.
[Sadler 1989] Sadler, D. R. (1989). Formative assessment and the design of instructional systems. Instructional science, 18(2):119–144.
[Sirotheau et al. 2019] Sirotheau, S., Santos, J., Favero, E., and de Freitas, S. N. (2019). Avaliação automática de respostas discursivas curtas baseado em três dimensões linguísticas. In Brazilian Symposium on Computers in Education (Simpósio Brasileiro de Informática na Educação- SBIE), volume 30, page 1551.
[Tausczik and Pennebaker 2010] Tausczik, Y. R. and Pennebaker, J. W. (2010). The psychological meaning of words: Liwc and computerized text analysis methods. Journal of language and social psychology, 29(1):24–54.
Publicado
24/11/2020
Como Citar
CAVALCANTI, Anderson Pinheiro; MELLO, Rafael Ferreira Leite de; MIRANDA, Péricles Barbosa Cunha de; FREITAS, Frederico Luiz Gonçalves de.
Análise Automática de Feedback em Ambientes de Aprendizagem Online. In: SIMPÓSIO BRASILEIRO DE INFORMÁTICA NA EDUCAÇÃO (SBIE), 31. , 2020, Online.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2020
.
p. 892-901.
DOI: https://doi.org/10.5753/cbie.sbie.2020.892.
