Evaluating an AI-Powered Learning Analytics Tool for Automated Essay Scoring in Real Classroom Contexts: An Experience Report from a Brazilian Public Secondary School
Resumo
This experience report, drawn from firsthand classroom practice, presents teachers’ perceptions, opinions, and experiences regarding the adoption of the Escreva Mais, an Artificial Intelligence (AI)-powered Learning Analytics (LA) tool designed for low-resource educational settings and essay assessment. The proposed experience report was conducted in a Brazilian secondary school and involved an essay teacher and a pedagogical coordinator. Both stakeholders adopted the solution through a complete essay assessment and feedback cycle involving thirty-five students. The qualitative findings highlight the Escreva Mais tool pedagogical benefits, perceived usefulness, reliability, performance, and potential contributions to student learning.
Palavras-chave:
AI-powered Learning Analytics, Automated Essay Scoring, Experience Report
Referências
Braun, V. and Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3(2):77–101.
Braun, V. and Clarke, V. (2012). Thematic analysis, pages 57–71.
Chan, S., Lo, N., and Wong, A. (2024). Generative ai and essay writing: Impacts of automated feedback on revision performance and engagement. Reflections, 31(3):1249–1284.
Clarke, V. and Braun, V. (2016). Thematic analysis. The Journal of Positive Psychology, 12:1–2.
Dawson, P., Carless, D., and Lee, P. P. W. (2021). Authentic feedback: supporting learners to engage in disciplinary feedback practices. Assessment & Evaluation in Higher Education, 46(2):286–296.
Ferman, B., Lima, L., and Riva, F. L. R. (2023). The impact of artificial intelligence on learning in brazil.
Fleckenstein, J., Liebenow, L. W., and Meyer, J. (2023). Automated feedback and writing: A multi-level meta-analysis of effects on students' performance. Frontiers in Artificial Intelligence, 6:1162454.
Florentino, B., Sestito, C., Cruz, W., de Carvalho, A., and Bonidia, R. (2026). Artificial intelligence for all? brazilian teachers on ethics, equity, and the everyday challenges of ai in education.
Graham, S., Cao, Y., Kim, Y.-S. G., Lee, J., Tate, T., Collins, P., Cho, M., Moon, Y., Chung, H. Q., and Olson, C. B. (2024). Effective writing instruction for students in grades 6 to 12: a best evidence meta-analysis. Reading and Writing, pages 1–46.
Hooshyar, D., Xiaojing, W., Sillat, P. J., Tammets, K., Wang, M., and Hämäläinen, R. (2024). The effectiveness of personalized technology-enhanced learning in higher education: A meta-analysis with association rule mining. Computers & Education, page 105169.
Isotani, S., Bittencourt, I. I., Challco, G. C., Dermeval, D., and Mello, R. F. (2023). Aied unplugged: Leapfrogging the digital divide to reach the underserved. In Wang, N., Rebolledo-Mendez, G., Dimitrova, V., Matsuda, N., and Santos, O. C., editors, Artificial Intelligence in Education. Posters and Late Breaking Results, Workshops and Tutorials, Industry and Innovation Tracks, Practitioners, Doctoral Consortium and Blue Sky, pages 772–779, Cham. Springer Nature Switzerland.
Mello, R. F., Anthony, L., Lobo, J., Ribeiro, F. G. C., Xavier, C., da Costa, N. T., Gasevic, D., and Rodrigues, L. (2026). Empowering equitable learning with llms: Enhancing writing skills in low-resource contexts. In Tammets, K., Sosnovsky, S., Ferreira Mello, R., Pishtari, G., and Nazaretsky, T., editors, Two Decades of TEL. From Lessons Learnt to Challenges Ahead. Springer Nature Switzerland.
Nazaretsky, T., Bar, C., Walter, M., and Alexandron, G. (2022). Empowering teachers with ai: Co-designing a learning analytics tool for personalized instruction in the science classroom. pages 1–12.
Pahi, K., Hawlader, S., Hicks, E., Zaman, A., and Phan, V. (2024). Enhancing active learning through collaboration between human teachers and generative ai. Computers and Education Open, 6:100183.
PISA (2023). Pisa 2022 results (volume i): The state of learning and equity in education. OECD Pub., pages 1–10.
Rodrigues, L., Guerino, G., da Veloso, T. E., Bianchini, L. P., Xavier, M. X., Vieira, T., Marinho, M., Macario, V., Dermeval, D., Bittencourt, I. I., et al. (2024a). Mathaide in the classroom: A qualitative analysis of teachers' perspectives of intelligent tutoring systems unplugged. In Simpósio Brasileiro de Informática na Educação (SBIE), pages 1515–1528. SBC.
Rodrigues, L., Guerino, G., Silva, T. E., Challco, G. C., Oliveira, L., da Penha, R. S., Melo, R. F., Vieira, T., Marinho, M., Macario, V., et al. (2024b). Mathaide: A qualitative study of teachers' perceptions of an its unplugged for underserved regions. International Journal of Artificial Intelligence in Education, pages 1–29.
Rodrigues, L., Guerino, G., Silva, T. E. V., Challco, G. C., Oliveira, L., da Penha, R. S., Melo, R. F., Vieira, T., Marinho, M., Macario, V., Bittencourt, I. I., Dermeval, D., and Isotani, S. (2025). Mathaide: A qualitative study of teachers' perceptions of an its unplugged for underserved regions. International Journal of Artificial Intelligence in Education, 35:2–30.
Seßler, K., Xiang, T., Bogenrieder, L., and Kasneci, E. (2023). PEER: Empowering Writing with Large Language Models, pages 755–761. Springer, Cham.
Shibani, A., Knight, S., and Buckingham Shum, S. (2019). Contextualizable learning analytics design: A generic model and writing analytics evaluations. In Proceedings of the 9th international conference on learning analytics & knowledge, pages 210–219.
Shoufan, A. (2023). Exploring students' perceptions of chatgpt: Thematic analysis and follow-up survey. IEEE Access.
Stamper, J., Xiao, R., and Hou, X. (2024). Enhancing llm-based feedback: Insights from intelligent tutoring systems and the learning sciences. In Proceedings of the International Conference on Intelligent Tutoring Systems (ITS).
Xavier, C., Rodrigues, L., Costa, N., Neto, R., Alves, G., Falcão, T. P., Gasevic, D., and Mello, R. F. (2025). Empowering instructors with ai: Evaluating the impact of an ai-driven feedback tool in learning analytics. IEEE Transactions on Learning Technologies, pages 1–15.
Braun, V. and Clarke, V. (2012). Thematic analysis, pages 57–71.
Chan, S., Lo, N., and Wong, A. (2024). Generative ai and essay writing: Impacts of automated feedback on revision performance and engagement. Reflections, 31(3):1249–1284.
Clarke, V. and Braun, V. (2016). Thematic analysis. The Journal of Positive Psychology, 12:1–2.
Dawson, P., Carless, D., and Lee, P. P. W. (2021). Authentic feedback: supporting learners to engage in disciplinary feedback practices. Assessment & Evaluation in Higher Education, 46(2):286–296.
Ferman, B., Lima, L., and Riva, F. L. R. (2023). The impact of artificial intelligence on learning in brazil.
Fleckenstein, J., Liebenow, L. W., and Meyer, J. (2023). Automated feedback and writing: A multi-level meta-analysis of effects on students' performance. Frontiers in Artificial Intelligence, 6:1162454.
Florentino, B., Sestito, C., Cruz, W., de Carvalho, A., and Bonidia, R. (2026). Artificial intelligence for all? brazilian teachers on ethics, equity, and the everyday challenges of ai in education.
Graham, S., Cao, Y., Kim, Y.-S. G., Lee, J., Tate, T., Collins, P., Cho, M., Moon, Y., Chung, H. Q., and Olson, C. B. (2024). Effective writing instruction for students in grades 6 to 12: a best evidence meta-analysis. Reading and Writing, pages 1–46.
Hooshyar, D., Xiaojing, W., Sillat, P. J., Tammets, K., Wang, M., and Hämäläinen, R. (2024). The effectiveness of personalized technology-enhanced learning in higher education: A meta-analysis with association rule mining. Computers & Education, page 105169.
Isotani, S., Bittencourt, I. I., Challco, G. C., Dermeval, D., and Mello, R. F. (2023). Aied unplugged: Leapfrogging the digital divide to reach the underserved. In Wang, N., Rebolledo-Mendez, G., Dimitrova, V., Matsuda, N., and Santos, O. C., editors, Artificial Intelligence in Education. Posters and Late Breaking Results, Workshops and Tutorials, Industry and Innovation Tracks, Practitioners, Doctoral Consortium and Blue Sky, pages 772–779, Cham. Springer Nature Switzerland.
Mello, R. F., Anthony, L., Lobo, J., Ribeiro, F. G. C., Xavier, C., da Costa, N. T., Gasevic, D., and Rodrigues, L. (2026). Empowering equitable learning with llms: Enhancing writing skills in low-resource contexts. In Tammets, K., Sosnovsky, S., Ferreira Mello, R., Pishtari, G., and Nazaretsky, T., editors, Two Decades of TEL. From Lessons Learnt to Challenges Ahead. Springer Nature Switzerland.
Nazaretsky, T., Bar, C., Walter, M., and Alexandron, G. (2022). Empowering teachers with ai: Co-designing a learning analytics tool for personalized instruction in the science classroom. pages 1–12.
Pahi, K., Hawlader, S., Hicks, E., Zaman, A., and Phan, V. (2024). Enhancing active learning through collaboration between human teachers and generative ai. Computers and Education Open, 6:100183.
PISA (2023). Pisa 2022 results (volume i): The state of learning and equity in education. OECD Pub., pages 1–10.
Rodrigues, L., Guerino, G., da Veloso, T. E., Bianchini, L. P., Xavier, M. X., Vieira, T., Marinho, M., Macario, V., Dermeval, D., Bittencourt, I. I., et al. (2024a). Mathaide in the classroom: A qualitative analysis of teachers' perspectives of intelligent tutoring systems unplugged. In Simpósio Brasileiro de Informática na Educação (SBIE), pages 1515–1528. SBC.
Rodrigues, L., Guerino, G., Silva, T. E., Challco, G. C., Oliveira, L., da Penha, R. S., Melo, R. F., Vieira, T., Marinho, M., Macario, V., et al. (2024b). Mathaide: A qualitative study of teachers' perceptions of an its unplugged for underserved regions. International Journal of Artificial Intelligence in Education, pages 1–29.
Rodrigues, L., Guerino, G., Silva, T. E. V., Challco, G. C., Oliveira, L., da Penha, R. S., Melo, R. F., Vieira, T., Marinho, M., Macario, V., Bittencourt, I. I., Dermeval, D., and Isotani, S. (2025). Mathaide: A qualitative study of teachers' perceptions of an its unplugged for underserved regions. International Journal of Artificial Intelligence in Education, 35:2–30.
Seßler, K., Xiang, T., Bogenrieder, L., and Kasneci, E. (2023). PEER: Empowering Writing with Large Language Models, pages 755–761. Springer, Cham.
Shibani, A., Knight, S., and Buckingham Shum, S. (2019). Contextualizable learning analytics design: A generic model and writing analytics evaluations. In Proceedings of the 9th international conference on learning analytics & knowledge, pages 210–219.
Shoufan, A. (2023). Exploring students' perceptions of chatgpt: Thematic analysis and follow-up survey. IEEE Access.
Stamper, J., Xiao, R., and Hou, X. (2024). Enhancing llm-based feedback: Insights from intelligent tutoring systems and the learning sciences. In Proceedings of the International Conference on Intelligent Tutoring Systems (ITS).
Xavier, C., Rodrigues, L., Costa, N., Neto, R., Alves, G., Falcão, T. P., Gasevic, D., and Mello, R. F. (2025). Empowering instructors with ai: Evaluating the impact of an ai-driven feedback tool in learning analytics. IEEE Transactions on Learning Technologies, pages 1–15.
Publicado
05/10/2026
Como Citar
RIBEIRO, Fabíola G. C. et al.
Evaluating an AI-Powered Learning Analytics Tool for Automated Essay Scoring in Real Classroom Contexts: An Experience Report from a Brazilian Public Secondary School. In: WORKSHOP DE INFORMÁTICA NA ESCOLA (WIE), 32. , 2026, Goiânia/GO.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2026
.
p. 440-452.
DOI: https://doi.org/10.5753/wie.2026.28044.
