Design, Development and Evaluation of a Lightweight Knowledge-based System for Theoretically-grounded Math Error Classification

• Anderson P. Avila-Santos Universidade Federal de Alagoas / Universidade Estadual de Londrina https://orcid.org/0000-0002-6791-0768
• Luiz Rodrigues Universidade Federal de Alagoas https://orcid.org/0000-0003-0343-3701
• Thomaz E. V. da Veloso Universidade Federal de Alagoas
• Thales Vieira Universidade Federal de Alagoas https://orcid.org/0000-0001-7775-5258
• Marcelo Marinho Universidade Federal Rural de Pernambuco
• Valmir Macario Universidade Federal Rural de Pernambuco
• Diego Dermeval Universidade Federal de Alagoas / Harvard University https://orcid.org/0000-0002-8415-6955
• Ig Ibert Bittencourt Universidade Federal de Alagoas / Harvard University https://orcid.org/0000-0001-5676-2280
• Seiji Isotani Universidade de São Paulo / Harvard University https://orcid.org/0000-0003-1574-0784

DOI: https://doi.org/10.5753/sbie.2024.242500

Resumo

Mathematical problem solving is essential for developing analytical skills and critical thinking among learners. Traditional methods of detecting and correcting errors in mathematical problem-solving are manual and time-consuming, leading to delayed feedback, which can hinder effective learning. Intelligent Tutoring Systems (ITS) provide real-time, error-specific feedback but require direct interaction with computational devices, limiting their use in classrooms with insufficient technological infrastructure. ITS unplugged (ITS-U) have been proposed to address these technological barriers, but no research has investigated how to design a lightweight error classificatin systems aligned to the resource-constrained setting of ITS-U. Therefore, this paper presents the Mathematical Solutions Error Classification (MSEC) API, a lightweight, knowledge-based tool designed to automate the classification of errors in mathematical solutions. This API categorizes errors into theoretically-grounded types, such as syntactical mistakes, conceptual misunderstandings, and calculation errors. MSEC is particularly suited for low-tech educational environments, aligning with the principles of ITS-U. We implemented MSEC within an ITS-U and conducted a case study involving 49 students, demonstrating its practical applicability and positive contribution to teaching and learning. The study highlights MSECs efficiency, ability to provide real-time feedback, and adaptability to various educational contexts, offering a significant advancement in automated error detection for ITS-U.

Referências

Brasil. Ministério da Educação (2018). Base nacional comum curricular. Brasília: MEC.

Conati, C., & Merten, C. (2014). Impact of adaptive feedback on learning and behavior. User Modeling and User-Adapted Interaction, 24(5), 413–431.

Davis, S. R., DeCapito, C., Nelson, E., Sharma, K., & Hand, E. M. (2020). Homework helper: Providing valuable feedback on math mistakes. In Advances in Visual Computing: 15th International Symposium, ISVC 2020, San Diego, CA, USA, October 5-7, 2020, Proceedings, Part II 15 (pp. 533–544). Springer.

Gasevic, D., Paul, P., Chen, B., Fan, Y., Rodrigo, M. M., Cobo, C., & Cecilia, A. (2018). Learning Analytics for the Global South. Foundation for Information Technology Education and Development, Quezon City, Philippines.

Hattie, J., & Timperley, H. (2007). The power of feedback. Review of Educational Research, 77(1), 81–112.

Hevner, A. R., March, S. T., Park, J., & Ram, S. (2004). Design science in information systems research. MIS Quarterly, 28(1), 75–105.

Hillmayr, D., Ziernwald, L., Reinhold, F., Hofer, S. I., & Reiss, K. M. (2020). The potential of digital tools to enhance mathematics and science learning in secondary schools: A context-specific meta-analysis. Computers & Education, 153, 103897.

Isotani, S., Bittencourt, I. I., Challco, G. C., Dermeval, D., & Mello, R. F. (2023). AIED unplugged: Leapfrogging the digital divide to reach the underserved. In N. Wang, G. Rebolledo-Mendez, V. Dimitrova, N. Matsuda, & O. C. Santos (Eds.), Artificial Intelligence in Education. Posters and Late Breaking Results, Workshops and Tutorials, Industry and Innovation Tracks, Practitioners, Doctoral Consortium and Blue Sky (pp. 772–779). Springer Nature Switzerland.

Levin, T., & Levin, I. (2012). Fostering mathematical thinking through cognitive scaffolding. Educational Psychology Review, 24(1), 401–418.

Lin, C. C., Huang, A. Y., & Lu, O. H. (2023). Artificial intelligence in intelligent tutoring systems toward sustainable education: A systematic review. Smart Learning Environments, 10(1), 41.

Mousavinasab, E., Zarifsanaiey, N., Niakan Kalhori, S. R., Rakhshan, M., Keikha, L., & Ghazi Saeedi, M. (2021). Intelligent tutoring systems: A systematic review of characteristics, applications, and evaluation methods. Interactive Learning Environments, 29(1), 142–163.

Nkambou, R., Mizoguchi, R., & Bourdeau, J. (2010). Advances in Intelligent Tutoring Systems (Vol. 308). Springer Science & Business Media.

Patel, N., Thakkar, M., Rabadiya, B., Patel, D., Malvi, S., Sharma, A., & Lomas, D. (2022). Equitable access to intelligent tutoring systems through paper-digital integration. In Intelligent Tutoring Systems: 18th International Conference, ITS 2022, Bucharest, Romania, June 29–July 1, 2022, Proceedings (pp. 255–263). Springer.

Polya, G. (2014). How to Solve It: A New Aspect of Mathematical Method. Princeton University Press.

Rodrigues, L., Guerino, G., Silva, T. E., Challco, G. C., Oliveira, L., da Penha, R. S., Melo, R. F., Vieira, T., Marinho, M., Macario, V., et al. (2024). Mathaide: A qualitative study of teachers’ perceptions of an ITS unplugged for underserved regions. International Journal of Artificial Intelligence in Education, 1–29.

Rodrigues, L., Pereira, F. D., Marinho, M., Macario, V., Bittencourt, I. I., Isotani, S., Dermeval, D., & Mello, R. (2023). Mathematics intelligent tutoring systems with handwritten input: A scoping review. Education and Information Technologies, 1–27.

Santos, F. et al. (2018). National Curriculum Standards of Brazil: Challenges and Perspectives. Springer.

Shute, V. J. (2008). Focus on formative feedback. Review of Educational Research, 78(1), 153–189.

Steenbergen-Hu, S., & Cooper, H. (2014). A meta-analysis of the effectiveness of intelligent tutoring systems on college students’ academic learning. Journal of Educational Psychology, 106(2), 331.

VanLehn, K. (2006). The behavior of tutoring systems. International Journal of Artificial Intelligence in Education, 16(3), 227–265.

Veloso, T. E., Chalco Challco, G., Rogrigues, L., Versuti, F. M., Sena da Penha, R., Silva Oliveira, L., Corredato Guerino, G., Cavalcanti de Amorim, L. F., Monteiro Marinho, M. L., Macario, V., Dermeval, D., Bittencourt, I. I., & Isotani, S. (2023). ITS unplugged: Leapfrogging the digital divide for teaching numeracy skills in underserved populations. In Towards the Future of AI-augmented Human Tutoring in Math Learning 2023 - Proceedings of the Workshop on International Conference of Artificial Intelligence in Education co-located with The 24th International Conference on Artificial Intelligence in Education (AIED 2023). Springer.

Vinuesa, R., Azizpour, H., Leite, I., Balaam, M., Dignum, V., Domisch, S., Felländer, A., Langhans, S. D., Tegmark, M., & Fuso Nerini, F. (2020). The role of artificial intelligence in achieving the sustainable development goals. Nature Communications, 11(1), 233.