MathAIde in the Classroom: A Qualitative Analysis of Teachers' Perspectives of Intelligent Tutoring Systems Unplugged

• Luiz Rodrigues Universidade Federal de Alagoas https://orcid.org/0000-0003-0343-3701
• Guilherme Guerino Universidade Estadual do Paraná
• Thomaz E. V. da Veloso Universidade Federal de Alagoas
• Luana P. Bianchini Universidade Federal de Alagoas
• Mariana X. M. S. Xavier Universidade Federal de Alagoas
• Thales Vieira Universidade Federal de Alagoas https://orcid.org/0000-0001-7775-5258
• Marcelo Marinho Universidade Federal Rural de Pernambuco
• Valmir Macario Universidade Federal Rural de Pernambuco
• Diego Dermeval Universidade Federal de Alagoas https://orcid.org/0000-0002-8415-6955
• Ig Ibert Bittencourt Universidade Federal de Alagoas / Harvard University https://orcid.org/0000-0001-5676-2280
• Seiji Isotani Universidade de São Paulo / Harvard University https://orcid.org/0000-0003-1574-0784

DOI: https://doi.org/10.5753/sbie.2024.242017

Resumo

Intelligent Tutoring Systems (ITS) have shown significant promise in enhancing math education by providing personalized and adaptive learning experiences. However, their adoption in resource-constrained environments is limited by the need for dedicated computing devices for each student. To address this issue, the concept of ITS unplugged has emerged, which helps deliver ITS benefits in resource-constrained settings, for instance, without the necessity for individual computers. However, past research has not investigated how ITS unplugged contributes to mathematics education when deployed in real classrooms. This paper presents a qualitative evaluation of MathAIde, an ITS unplugged designed to support numeracy education, where three teachers used MathAIde in 12 lessons, involving 49 students, and their experiences were captured through semi-structured interviews. Thematic analysis revealed that MathAIde facilitated lesson planning and execution, provided valuable feedback and learning analytics, but faced challenges such as technical issues and the need for more adaptive content. This study contributes empirical evidence on the impact of ITS unplugged in real classrooms, offering insights for future development and adoption of such technologies to promote equitable access to ITS benefits in education.

Referências

Barbosa, S. D. J., Silva, B. d., Silveira, M. S., Gasparini, I., Darin, T., & Barbosa, G. D. J. (2021). Interação humano-computador e experiência do usuário. Auto publicação.

Blandford, A., Furniss, D., & Makri, S. (2016). Qualitative HCI research: Going behind the scenes. Synthesis Lectures on Human-Centered Informatics, 9(1), 1–115.

Citaristi, I. (2022). International telecommunication union—ITU. In The Europa Directory of International Organizations 2022 (pp. 365–369). Routledge.

Clarke, V., & Braun, V. (2017). Thematic analysis. The Journal of Positive Psychology, 12(3), 297–298.

Davis, S. R., DeCapito, C., Nelson, E., Sharma, K., & Hand, E. M. (2020). Homework helper: Providing valuable feedback on math mistakes. In Advances in Visual Computing: 15th International Symposium, ISVC 2020, San Diego, CA, USA, October 5–7, 2020, Proceedings, Part II 15 (pp. 533–544). Springer.

Instituto de Pesquisa, I. D. (2022). Pesquisa nacional por amostra de domicílios contínua. Rio de Janeiro, 2021b. Disponível em: [link]. Acesso em: 22.

Fang, Y., Ren, Z., Hu, X., & Graesser, A. C. (2019). A meta-analysis of the effectiveness of ALEKS on learning. Educational Psychology, 39(10), 1278–1292.

Freitas, E., Batista, H. H., Barbosa, G. A., Wenceslau, M., Portela, C., Isotani, S., Cordeiro, T., Bittencourt, I. I., Yasojima, K., Sobrinho, Á., & others. (2022). Learning analytics desconectada: Um estudo de caso em análise de produções textuais. In Anais do I Workshop de Aplicações Práticas de Learning Analytics em Instituições de Ensino no Brasil (pp. 40–49). SBC.

Gasevic, D., Paul, P., Chen, B., Fan, Y., Rodrigo, M. M., Cobo, C., & Cecilia, A. (2018). Learning Analytics for the Global South. Foundation for Information Technology Education and Development.

Guo, L., Wang, D., Gu, F., Li, Y., Wang, Y., & Zhou, R. (2021). Evolution and trends in intelligent tutoring systems research: A multidisciplinary and scientometric view. Asia Pacific Education Review, 22(3), 441–461.

Hillmayr, D., Ziernwald, L., Reinhold, F., Hofer, S. I., & Reiss, K. M. (2020). The potential of digital tools to enhance mathematics and science learning in secondary schools: A context-specific meta-analysis. Computers & Education, 153, 103897.

Isotani, S., Bittencourt, I. I., Challco, G. C., Dermeval, D., & Mello, R. F. (2023). AIED unplugged: Leapfrogging the digital divide to reach the underserved. In N. Wang, G. Rebolledo-Mendez, V. Dimitrova, N. Matsuda, & O. C. Santos (Eds.), Artificial Intelligence in Education. Posters and Late Breaking Results, Workshops and Tutorials, Industry and Innovation Tracks, Practitioners, Doctoral Consortium and Blue Sky (pp. 772–779). Springer Nature Switzerland.

Kizilcec, R. F., Chen, M., Jasińska, K. K., Madaio, M., & Ogan, A. (2021). Mobile learning during school disruptions in Sub-Saharan Africa. AERA Open, 7, 23328584211014860.

Lin, C.-C., Huang, A. Y., & Lu, O. H. (2023). Artificial intelligence in intelligent tutoring systems toward sustainable education: A systematic review. Smart Learning Environments, 10(1), 41.

Modén, M. U., Tallvid, M., Lundin, J., & Lindström, B. (2021). Intelligent tutoring systems: Why teachers abandoned a technology aimed at automating teaching processes. In Hawaii International Conference on System Sciences.

Mousavinasab, E., Zarifsanaiey, N., Niakan Kalhori, S. R., Rakhshan, M., Keikha, L., & Ghazi Saeedi, M. (2021). Intelligent tutoring systems: A systematic review of characteristics, applications, and evaluation methods. Interactive Learning Environments, 29(1), 142–163.

Nkambou, R., Mizoguchi, R., & Bourdeau, J. (2010). Advances in Intelligent Tutoring Systems (Vol. 308). Springer Science & Business Media.

Patel, N., Thakkar, M., Rabadiya, B., Patel, D., Malvi, S., Sharma, A., & Lomas, D. (2022). Equitable access to intelligent tutoring systems through paper-digital integration. In Intelligent Tutoring Systems: 18th International Conference, ITS 2022, Bucharest, Romania, June 29–July 1, 2022, Proceedings (pp. 255–263). Springer.

Portela, C., Lisbôa, R., Yasojima, K., Cordeiro, T., Silva, A., Dermeval, D., Marques, L., Santos, J., Mello, R., Macário, V., & others. (2023). A case study on AIED unplugged applied to public policy for learning recovery post-pandemic in Brazil. In International Conference on Artificial Intelligence in Education (pp. 788–796). Springer.

Rodrigues, L., Guerino, G., Challco, G. C., Veloso, T. E., Oliveira, L., da Penha, R. S., Melo, R. F., Vieira, T., Marinho, M., Macario, V., & others. (2023a). Teacher-centered intelligent tutoring systems: Design considerations from Brazilian public school teachers. In Anais do XXXIV Simpósio Brasileiro de Informática na Educação (pp. 1419–1430). SBC.

Rodrigues, L., Guerino, G., Silva, T. E., Challco, G. C., Oliveira, L., da Penha, R. S., Melo, R. F., Vieira, T., Marinho, M., Macario, V., & others. (2024). MathAide: A qualitative study of teachers’ perceptions of an ITS unplugged for underserved regions. International Journal of Artificial Intelligence in Education, 1–29.

Rodrigues, L., Pereira, F. D., Marinho, M., Macario, V., Bittencourt, I. I., Isotani, S., Dermeval, D., & Mello, R. (2023b). Mathematics intelligent tutoring systems with handwritten input: A scoping review. Education and Information Technologies, 1–27.

Steenbergen-Hu, S., & Cooper, H. (2014). A meta-analysis of the effectiveness of intelligent tutoring systems on college students’ academic learning. Journal of Educational Psychology, 106(2), 331.

VanLehn, K. (2006). The behavior of tutoring systems. International Journal of Artificial Intelligence in Education, 16(3), 227–265.

Veloso, T. E., Chalco Challco, G., Rodrigues, L., Versuti, F. M., Sena da Penha, R., Silva Oliveira, L., Corredato Guerino, G., Cavalcanti de Amorim, L. F., Monteiro Marinho, M. L., Macario, V., Dermeval, D., Bittencourt, I. I., & Isotani, S. (2023). ITS unplugged: Leapfrogging the digital divide for teaching numeracy skills in underserved populations. In Towards the Future of AI-Augmented Human Tutoring in Math Learning 2023 - Proceedings of the Workshop on International Conference of Artificial Intelligence in Education (AIED 2023). Springer.

Wisniewski, B., Zierer, K., & Hattie, J. (2020). The power of feedback revisited: A meta-analysis of educational feedback research. Frontiers in Psychology, 10, 3087.

Xia, Q., Chiu, T. K., Zhou, X., Chai, C. S., & Cheng, M. (2022). Systematic literature review on opportunities, challenges, and future research recommendations of artificial intelligence in education. Computers and Education: Artificial Intelligence, 100118.

Xu, Z., Wijekumar, K., Ramirez, G., Hu, X., & Irey, R. (2019). The effectiveness of intelligent tutoring systems on K-12 students' reading comprehension: A meta-analysis. British Journal of Educational Technology, 50(6):3119--3137.