Mutation-Based Error Diagnosis in a Web Intelligent Tutoring System for First-Degree Equations: A Quasi-Experimental Study
Resumo
This paper investigates the use of mutation analysis as a mechanism for diagnosing student errors in a Web-based Intelligent Tutoring System for first-degree equations. The proposed module generates alternative solution trajectories by injecting atomic error patterns into correct solution steps and comparing the resulting answers with student responses. A quasi-experimental study involving 113 participants was conducted using a simulated test with ten procedurally generated problems. The mutation engine identified 70 of 387 incorrect answers (18.09%), while the guided step-by-step interaction achieved a 75.94% accuracy rate (101 correct responses out of 133). Results indicate that mutation-based reasoning reconstruction can provide interpretable diagnostic feedback and support guided problem solving. The paper discusses the potential, limitations, and future applications of mutation-based diagnosis in Intelligent Tutoring Systems.
Palavras-chave:
Intelligent Tutoring Systems, Mutation Analysis, Error Diagnosis
Referências
Alam, A. (2023). Harnessing the power of ai to create intelligent tutoring systems for enhanced classroom experience and improved learning outcomes. In Intelligent Communication Technologies and Virtual Mobile Networks, pages 571–591. Springer Nature Singapore, Singapore.
Anderson-Cook, C. M. (2005). Experimental and quasi-experimental designs for generalized causal inference.
Azevedo, R., Aleven, V., et al. (2013). International handbook of metacognition and learning technologies, volume 26. Springer.
Brown, J. S. and Burton, R. R. (1978). Diagnostic models for procedural bugs in basic mathematical skills. Cognitive science, 2(2):155–192.
Campbell, D. T. (1963). and jc stanley. Experimental and quasi-experimental designs for research.
Chrysafiadi, K. and Virvou, M. (2013). Student modeling approaches: A literature review for the last decade. Expert Systems with Applications, 40(11):4715–4729.
Costa Santos, L. d., Severo, L. R., and Correia, L. d. A. (2023). Desafios ao engajamento acadêmico no ensino superior: uma análise a partir de posicionamentos sobre satisfação de estudantes. Revista Internacional de Educação Superior, 9:e023027.
Crossley, S., Liu, R., and McNamara, D. (2017). Predicting math performance using natural language processing tools. In Proceedings of the Seventh International Learning Analytics & Knowledge Conference (LAK '17), pages 339–347, New York, NY, USA. ACM.
da Silva, T. L., Vidotto, K. N. S., Tarouco, L. M. R., and da Silva, P. F. (2024). Potencialidades do uso de inteligência artificial generativa como apoio ao ensino de programação. In Simpósio Brasileiro de Informática na Educação (SBIE), pages 1942–1956. SBC.
de Sousa, H. and Cruz, D. M. (2024). Capacitando educadores com ia generativa: Implicações na educação. In Simpósio Brasileiro de Informática na Educação (SBIE), pages 1931–1941. SBC.
Gamboa, H. and Fred, A. (2002). Designing intelligent tutoring systems: a bayesian approach. In Filipe, J., Sharp, B., and Miranda, P., editors, Enterprise Information Systems III, pages 146–152. Springer, New York.
Gong, Y. and Beck, J. E. (2015). Towards detecting wheel-spinning: Future failure in mastery learning. In Proceedings of the Second ACM Conference on Learning@Scale (L@S '15), pages 67–74.
Karpouzis, K. (2023). Explainable ai for intelligent tutoring systems. In International Conference on Frontiers of Artificial Intelligence, Ethics, and Multidisciplinary Applications, pages 59–70. Springer.
Kautzmann, T. R. and Jaques, P. A. (2019). Effects of adaptive training on metacognitive knowledge monitoring ability in computer-based learning. Computers & Education, 129:92–105.
Létourneau, A., Deslandes Martineau, M., Charland, P., Karran, J. A., Boasen, J., and Léger, P. M. (2025). A systematic review of ai-driven intelligent tutoring systems (its) in k-12 education. npj Science of Learning, 10(1):29.
Lin, C.-C., Huang, A. Y., and Lu, O. H. (2023). Artificial intelligence in intelligent tutoring systems toward sustainable education: a systematic review. Smart learning environments, 10(1):41.
Ruan, S. et al. (2020). Supporting children's math learning with feedback-augmented narrative technology. In Proceedings of the Interaction Design and Children Conference, pages 567–580.
Shen, Q. et al. (2021). A comprehensive study of deep learning compiler bugs. In Proceedings of the 29th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE '21), pages 968–980.
Smith, R. C. et al. (2023). A research agenda for computational empowerment for emerging technology education. International Journal of Child-Computer Interaction, 38:100616.
Türkmen, G. (2025). The review of studies on explainable artificial intelligence in educational research. Journal of Educational Computing Research, 63(2):277–310.
Untch, R. H., Offutt, A. J., and Harrold, M. J. (1993). Mutation analysis using mutant schemata. In Proceedings of the 1993 ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA '93), pages 139–148.
Wang, Z. et al. (2021). Prioritizing test inputs for deep neural networks via mutation analysis. In 2021 IEEE/ACM 43rd International Conference on Software Engineering (ICSE), pages 397–409.
Xing, W. and Goggins, S. (2015). Learning analytics in outer space: a hidden naïve bayes model for automatic student off-task behavior detection. In Proceedings of the Fifth International Conference on Learning Analytics and Knowledge (LAK '15), pages 176–183.
Zerkouk, M., Mihoubi, M., and Chikhaoui, B. (2025). A comprehensive review of ai-based intelligent tutoring systems: applications and challenges. arXiv preprint arXiv:2507.18882.
Anderson-Cook, C. M. (2005). Experimental and quasi-experimental designs for generalized causal inference.
Azevedo, R., Aleven, V., et al. (2013). International handbook of metacognition and learning technologies, volume 26. Springer.
Brown, J. S. and Burton, R. R. (1978). Diagnostic models for procedural bugs in basic mathematical skills. Cognitive science, 2(2):155–192.
Campbell, D. T. (1963). and jc stanley. Experimental and quasi-experimental designs for research.
Chrysafiadi, K. and Virvou, M. (2013). Student modeling approaches: A literature review for the last decade. Expert Systems with Applications, 40(11):4715–4729.
Costa Santos, L. d., Severo, L. R., and Correia, L. d. A. (2023). Desafios ao engajamento acadêmico no ensino superior: uma análise a partir de posicionamentos sobre satisfação de estudantes. Revista Internacional de Educação Superior, 9:e023027.
Crossley, S., Liu, R., and McNamara, D. (2017). Predicting math performance using natural language processing tools. In Proceedings of the Seventh International Learning Analytics & Knowledge Conference (LAK '17), pages 339–347, New York, NY, USA. ACM.
da Silva, T. L., Vidotto, K. N. S., Tarouco, L. M. R., and da Silva, P. F. (2024). Potencialidades do uso de inteligência artificial generativa como apoio ao ensino de programação. In Simpósio Brasileiro de Informática na Educação (SBIE), pages 1942–1956. SBC.
de Sousa, H. and Cruz, D. M. (2024). Capacitando educadores com ia generativa: Implicações na educação. In Simpósio Brasileiro de Informática na Educação (SBIE), pages 1931–1941. SBC.
Gamboa, H. and Fred, A. (2002). Designing intelligent tutoring systems: a bayesian approach. In Filipe, J., Sharp, B., and Miranda, P., editors, Enterprise Information Systems III, pages 146–152. Springer, New York.
Gong, Y. and Beck, J. E. (2015). Towards detecting wheel-spinning: Future failure in mastery learning. In Proceedings of the Second ACM Conference on Learning@Scale (L@S '15), pages 67–74.
Karpouzis, K. (2023). Explainable ai for intelligent tutoring systems. In International Conference on Frontiers of Artificial Intelligence, Ethics, and Multidisciplinary Applications, pages 59–70. Springer.
Kautzmann, T. R. and Jaques, P. A. (2019). Effects of adaptive training on metacognitive knowledge monitoring ability in computer-based learning. Computers & Education, 129:92–105.
Létourneau, A., Deslandes Martineau, M., Charland, P., Karran, J. A., Boasen, J., and Léger, P. M. (2025). A systematic review of ai-driven intelligent tutoring systems (its) in k-12 education. npj Science of Learning, 10(1):29.
Lin, C.-C., Huang, A. Y., and Lu, O. H. (2023). Artificial intelligence in intelligent tutoring systems toward sustainable education: a systematic review. Smart learning environments, 10(1):41.
Ruan, S. et al. (2020). Supporting children's math learning with feedback-augmented narrative technology. In Proceedings of the Interaction Design and Children Conference, pages 567–580.
Shen, Q. et al. (2021). A comprehensive study of deep learning compiler bugs. In Proceedings of the 29th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE '21), pages 968–980.
Smith, R. C. et al. (2023). A research agenda for computational empowerment for emerging technology education. International Journal of Child-Computer Interaction, 38:100616.
Türkmen, G. (2025). The review of studies on explainable artificial intelligence in educational research. Journal of Educational Computing Research, 63(2):277–310.
Untch, R. H., Offutt, A. J., and Harrold, M. J. (1993). Mutation analysis using mutant schemata. In Proceedings of the 1993 ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA '93), pages 139–148.
Wang, Z. et al. (2021). Prioritizing test inputs for deep neural networks via mutation analysis. In 2021 IEEE/ACM 43rd International Conference on Software Engineering (ICSE), pages 397–409.
Xing, W. and Goggins, S. (2015). Learning analytics in outer space: a hidden naïve bayes model for automatic student off-task behavior detection. In Proceedings of the Fifth International Conference on Learning Analytics and Knowledge (LAK '15), pages 176–183.
Zerkouk, M., Mihoubi, M., and Chikhaoui, B. (2025). A comprehensive review of ai-based intelligent tutoring systems: applications and challenges. arXiv preprint arXiv:2507.18882.
Publicado
05/10/2026
Como Citar
SANTANA, Alan de Oliveira; DA SILVA, Thiago Reis; ARANHA, Eduardo Henrique da Silva.
Mutation-Based Error Diagnosis in a Web Intelligent Tutoring System for First-Degree Equations: A Quasi-Experimental Study. In: SIMPÓSIO BRASILEIRO DE INFORMÁTICA NA EDUCAÇÃO (SBIE), 37. , 2026, Goiânia/GO.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2026
.
p. 1262-1275.
DOI: https://doi.org/10.5753/sbie.2026.27726.
