ATHOS: Ferramenta computacional para avaliação automatizada da escrita de projetos de pesquisa
Resumo
Este trabalho apresenta o desenvolvimento e avaliação da ferramenta ATHOS, que integra Processamento de Linguagem Natural e Analítica da Aprendizagem para auxiliar professores universitários na avaliação de projetos de pesquisa. A ferramenta combina análises de complexidade textual, estrutural e IA generativa para extrair elementos chave dos projetos, gerar visualizações comparativas e sugerir feedbacks personalizáveis. Um estudo de casos múltiplos com professores de metodologia da pesquisa revelou que a ATHOS otimiza parcialmente o processo avaliativo, permitindo identificação rápida de pontos críticos e liberando tempo para comentários substantivos. Os resultados indicam que a abordagem centrada no professor, mantendo-o no controle do feedback final, é promissora para integrar tecnologias de avaliação automatizada na prática pedagógica.Referências
BASTIANI, Ederson; ZARTH, Antonio Miguel Faustini; REATEGUI, Eliseo. MTA: uma ferramenta apoiada em IA para aprimoramento da escrita de projetos de pesquisa acadêmica. Renote, v. 19, n. 1, p. 1-10, 2025. DOI: 10.22456/1679-1916.141555.
BIRD, S., LOPER, E. e KLEIN, E. Natural Language Processing with Python. O'Reilly Media, Inc. 2009
CARLESS, D.; BOUD, D. The development of student feedback literacy: enabling uptake of feedback. Assessment & Evaluation in Higher Education, v. 43, n. 8, p. 1315-1325, 2018.
CROSSLEY, S. A., MCNAMARA, D. S. e MCCARTHY, P. M. Learning analytics and writing instruction. Computers and Composition, 27(1), 34-47. 2010.
CROSSLEY, S. A., ALLEN, D. B. e MCNAMARA, D. S. Text readability and intuitive simplification: A comparison of readability formulas and Coh-Metrix. Reading in a Foreign Language, 23(1), 84-101. 2011.
COPE, B. e KALANTZIS, M. Big Data Comes to School: Implications for Learning, Assessment, and Research. AERA Open, 2(2). 2016. DOI: 10.1177/2332858416641907
EVERS, Aline; FINATTO, Maria José Bocorny. Linguística de Corpus, Léxico-Estatística Textual e Processamento de Linguagem Natural: perspectiva para estudos de vocabulário em produções textuais. Revista GTLex, v. 1, n. 2, p. 271, 2016. DOI: 10.14393/lex2-v1n2a2016-3.
GALLIEN, Tara; EARLY, Jody Oomen. Personalized versus collective instructor feedback in the online courseroom: does type of feedback affect student satisfaction, academic performance and perceived connectedness with the instructor? International Journal on E-Learning, v. 7, n. 3, p. 463-476, 2005.
HATTIE, John; TIMPERLEY, Helen. The Power of Feedback. Review of Educational Research, v. 77, n. 1, p. 81-112, 2007. DOI: 10.3102/003465430298487.
HIGGINS, R.; HARTLEY, P.; SKELTON, A. The conscientious consumer: reconsidering the role of assessment feedback in student learning. Studies in Higher Education, v. 27, n. 1, p. 53-64, 2002. DOI: 10.1080/03075070120099368.
JURAFSKY, D. e MARTIN, J. H. "Speech and Language Processing" (3rd ed.). 2018. Disponível em: [link]
LEAL, S.E., DURAN, M.S., SCARTON, C., HARTMANN, N.S., e ALUÍSIO, S.M. NILC-Metrix: Assessing the complexity of written and spoken language in Brazilian Portuguese. Language Resources and Evaluation, p. 1-38, 2023. DOI: 10.1007/s10579-023-09693-w
LILLIS, T. e CURRY, M. J. Academic writing in a global context: The politics and practices of publishing in English. Routledge. J Bus Tech Commun, v. 22, p. 179-198, 2010.
LIU, Bing. Sentiment analysis and opinion mining. Cham: Springer, 2012. 167 p. (Synthesis Lectures on Human Language Technologies). DOI: 10.1007/978-3-031-02145-9.
MCNAMARA, Danielle S.; GRAESSER, Arthur C. Coh-Metrix: an automated tool for theoretical and applied natural language processing. In: MCCARTHY, Philip; BOONTHUM-DENECKE, Chutima (Org.). Applied natural language processing: identification, investigation and resolution. Hershey: IGI Global, 2012. p. 18. DOI: 10.4018/978-1-60960-741-8.ch011.
MEIRA, R. R.; WEIAND, A.; REATEGUI, E.; BIGOLIN, M.; MOTZ, R. A Analítica da Escrita para Identificação de Indicadores de Qualidade Textual. RENOTE, Porto Alegre, v. 21, n. 2, p. 342–351, 2023. DOI: 10.22456/1679-1916.137756
MEIRA, R. R.; REATEGUI, E.; MOTZ, R. Análise de trabalhos de conclusão de curso utilizando técnicas de processamento de linguagem natural. RENOTE, Porto Alegre, v. 22, n. 1, p. 456–465, 2024. DOI: 10.22456/1679-1916.141571.
NADEAU, David; SEKINE, Satoshi. A survey of named entity recognition and classification. Lingvisticæ Investigationes, v. 30, n. 1, p. 3-26, jan. 2007. DOI: 10.1075/li.30.1.03nad
NICOL, D. J. e MACFARLANE-DICK, D. Formative assessment and self-regulated learning: A model and seven principles of good feedback practice. Studies in Higher Education, v. 31, n. 2, p. 199-218. 2006. DOI: 10.1080/03075070600572090
SALDAÑA, J. The coding manual for qualitative researchers. London: Sage. 2013.
SEO, K., TANG, J., ROLL, I. The impact of artificial intelligence on learner–instructor interaction in online learning. Int J Educ Technol High Educ, v. 18, n. 54. 2021. DOI: 10.1186/s41239-021-00292-9
SOUSA, Luciano Dias de, ALMEIDA, Flávio Aparecido de, BARD, Lucimere Aleixo, CANCELA, Lucas Borcard. Os desafios enfrentados pelos professores no processo de avaliação no ensino superior. Revista de Gestão e Avaliação Educacional, v. 7, n. 16, p. 59-66, 2018.
TASKIRAN, Ayşe e GÖKSEL, Nil. Automated feedback and teacher feedback: Writing achievement in learning English as a foreign language at a distance Turkish Online Journal of Distance Education, v. 23, n. 2, p. 120-139, 2022. DOI: 10.17718/tojde.1096260
WANG, Z.; HAN, F. The Effects of Teacher Feedback and Automated Feedback on Cognitive and Psychological Aspects of Foreign Language Writing: A Mixed-Methods Research. Frontiers in Psychology, v. 13, 2022. DOI: 10.3389/fpsyg.2022.909802
WARSCHAUER, M. e WARE, P. Automated writing evaluation: defining the classroom research agenda. Language Teaching Research, v. 10, n. 2, p. 157-180. 2006. DOI: 10.1191/1362168806lr190oa
WILSON, Joshua; OLINGHOUSE, Natalie G.; ANDRADA, Gilbert N. Does Automated Feedback Improve Writing Quality? Learning Disabilities: A Contemporary Journal, v.12, n.1, p. 93-118. Mar 2014.
BIRD, S., LOPER, E. e KLEIN, E. Natural Language Processing with Python. O'Reilly Media, Inc. 2009
CARLESS, D.; BOUD, D. The development of student feedback literacy: enabling uptake of feedback. Assessment & Evaluation in Higher Education, v. 43, n. 8, p. 1315-1325, 2018.
CROSSLEY, S. A., MCNAMARA, D. S. e MCCARTHY, P. M. Learning analytics and writing instruction. Computers and Composition, 27(1), 34-47. 2010.
CROSSLEY, S. A., ALLEN, D. B. e MCNAMARA, D. S. Text readability and intuitive simplification: A comparison of readability formulas and Coh-Metrix. Reading in a Foreign Language, 23(1), 84-101. 2011.
COPE, B. e KALANTZIS, M. Big Data Comes to School: Implications for Learning, Assessment, and Research. AERA Open, 2(2). 2016. DOI: 10.1177/2332858416641907
EVERS, Aline; FINATTO, Maria José Bocorny. Linguística de Corpus, Léxico-Estatística Textual e Processamento de Linguagem Natural: perspectiva para estudos de vocabulário em produções textuais. Revista GTLex, v. 1, n. 2, p. 271, 2016. DOI: 10.14393/lex2-v1n2a2016-3.
GALLIEN, Tara; EARLY, Jody Oomen. Personalized versus collective instructor feedback in the online courseroom: does type of feedback affect student satisfaction, academic performance and perceived connectedness with the instructor? International Journal on E-Learning, v. 7, n. 3, p. 463-476, 2005.
HATTIE, John; TIMPERLEY, Helen. The Power of Feedback. Review of Educational Research, v. 77, n. 1, p. 81-112, 2007. DOI: 10.3102/003465430298487.
HIGGINS, R.; HARTLEY, P.; SKELTON, A. The conscientious consumer: reconsidering the role of assessment feedback in student learning. Studies in Higher Education, v. 27, n. 1, p. 53-64, 2002. DOI: 10.1080/03075070120099368.
JURAFSKY, D. e MARTIN, J. H. "Speech and Language Processing" (3rd ed.). 2018. Disponível em: [link]
LEAL, S.E., DURAN, M.S., SCARTON, C., HARTMANN, N.S., e ALUÍSIO, S.M. NILC-Metrix: Assessing the complexity of written and spoken language in Brazilian Portuguese. Language Resources and Evaluation, p. 1-38, 2023. DOI: 10.1007/s10579-023-09693-w
LILLIS, T. e CURRY, M. J. Academic writing in a global context: The politics and practices of publishing in English. Routledge. J Bus Tech Commun, v. 22, p. 179-198, 2010.
LIU, Bing. Sentiment analysis and opinion mining. Cham: Springer, 2012. 167 p. (Synthesis Lectures on Human Language Technologies). DOI: 10.1007/978-3-031-02145-9.
MCNAMARA, Danielle S.; GRAESSER, Arthur C. Coh-Metrix: an automated tool for theoretical and applied natural language processing. In: MCCARTHY, Philip; BOONTHUM-DENECKE, Chutima (Org.). Applied natural language processing: identification, investigation and resolution. Hershey: IGI Global, 2012. p. 18. DOI: 10.4018/978-1-60960-741-8.ch011.
MEIRA, R. R.; WEIAND, A.; REATEGUI, E.; BIGOLIN, M.; MOTZ, R. A Analítica da Escrita para Identificação de Indicadores de Qualidade Textual. RENOTE, Porto Alegre, v. 21, n. 2, p. 342–351, 2023. DOI: 10.22456/1679-1916.137756
MEIRA, R. R.; REATEGUI, E.; MOTZ, R. Análise de trabalhos de conclusão de curso utilizando técnicas de processamento de linguagem natural. RENOTE, Porto Alegre, v. 22, n. 1, p. 456–465, 2024. DOI: 10.22456/1679-1916.141571.
NADEAU, David; SEKINE, Satoshi. A survey of named entity recognition and classification. Lingvisticæ Investigationes, v. 30, n. 1, p. 3-26, jan. 2007. DOI: 10.1075/li.30.1.03nad
NICOL, D. J. e MACFARLANE-DICK, D. Formative assessment and self-regulated learning: A model and seven principles of good feedback practice. Studies in Higher Education, v. 31, n. 2, p. 199-218. 2006. DOI: 10.1080/03075070600572090
SALDAÑA, J. The coding manual for qualitative researchers. London: Sage. 2013.
SEO, K., TANG, J., ROLL, I. The impact of artificial intelligence on learner–instructor interaction in online learning. Int J Educ Technol High Educ, v. 18, n. 54. 2021. DOI: 10.1186/s41239-021-00292-9
SOUSA, Luciano Dias de, ALMEIDA, Flávio Aparecido de, BARD, Lucimere Aleixo, CANCELA, Lucas Borcard. Os desafios enfrentados pelos professores no processo de avaliação no ensino superior. Revista de Gestão e Avaliação Educacional, v. 7, n. 16, p. 59-66, 2018.
TASKIRAN, Ayşe e GÖKSEL, Nil. Automated feedback and teacher feedback: Writing achievement in learning English as a foreign language at a distance Turkish Online Journal of Distance Education, v. 23, n. 2, p. 120-139, 2022. DOI: 10.17718/tojde.1096260
WANG, Z.; HAN, F. The Effects of Teacher Feedback and Automated Feedback on Cognitive and Psychological Aspects of Foreign Language Writing: A Mixed-Methods Research. Frontiers in Psychology, v. 13, 2022. DOI: 10.3389/fpsyg.2022.909802
WARSCHAUER, M. e WARE, P. Automated writing evaluation: defining the classroom research agenda. Language Teaching Research, v. 10, n. 2, p. 157-180. 2006. DOI: 10.1191/1362168806lr190oa
WILSON, Joshua; OLINGHOUSE, Natalie G.; ANDRADA, Gilbert N. Does Automated Feedback Improve Writing Quality? Learning Disabilities: A Contemporary Journal, v.12, n.1, p. 93-118. Mar 2014.
Publicado
24/11/2025
Como Citar
MEIRA, Ricardo R.; REATEGUI, Eliseo.
ATHOS: Ferramenta computacional para avaliação automatizada da escrita de projetos de pesquisa. In: CONCURSO ALEXANDRE DIRENE (CTD-IE) - TESES DE DOUTORADO - CONGRESSO BRASILEIRO DE INFORMÁTICA NA EDUCAÇÃO (CBIE), 14. , 2025, Curitiba/PR.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2025
.
p. 61-74.
DOI: https://doi.org/10.5753/cbie_estendido.2025.13779.
