Model and Evaluation of an Intelligent Tutoring System with a Reinforcement Learning Agent

• Vitor Gouvêa UFRJ
• Amanda C. N. de Oliveira UFRJ
• Daniel R. Figueiredo UFRJ

DOI: https://doi.org/10.5753/eniac.2026.31404

Resumo

The use of Reinforcement Learning (RL) in completing human tasks has exploded in recent years, with RL models often beating human experts and achieving record performance. A particularly interesting application of RL is in the Intelligent Tutoring Systems (ITS) problem of sequential learning. In this problem the ITS has to decide which questions to give to a student during an exercise session. In this paper, we model the sequential learning problem as a Markov Decision Process (MDP) environment and estimate an optimal policy using a model-free Monte Carlo algorithm from simulations. We compare the optimal policy to other trivial and intuitive policies and show that such a policy can achieve good performance, however it is limited by state representation.

Referências

Arroyo, I., Woolf, B. P., Burleson, W., Muldner, K., Rai, D., and Tai, M. (2014). A multimedia adaptive tutoring system for mathematics that addresses cognition, metacognition and affect. International Journal of Artificial Intelligence in Education, 24(4):387–426.

Atkinson, R. C. (1972). Optimizing the learning of a second-language vocabulary. Journal of Experimental Psychology, 96:124–129.

Bertsekas, D. P. and Tsitsiklis, J. N. (2008). Introduction to Probability. Athena Scientific, Belmont, MA, 2 edition.

Chi, M., VanLehn, K., and Litman, D. (2010). Do micro-level tutorial decisions matter: Applying reinforcement learning to induce pedagogical tutorial tactics. In Intelligent Tutoring Systems, pages 224–234.

Doroudi, S., Aleven, V., and Brunskill, E. (2019). Where’s the reward?: A review of reinforcement learning for instructional sequencing. International Journal of Artificial Intelligence in Education, 29(4):568–620.

Guo, D., Yang, D., Zhang, H., et al. (2025). Deepseek-r1 incentivizes reasoning in llms through reinforcement learning. Nature, 645:633–638.

Kaelbling, L. P., Littman, M. L., and Moore, A. W. (1996). Reinforcement learning: A survey. Journal of Artificial Intelligence Research, 4:237–285.

Nie, A., Reuel, A. K., and Brunskill, E. (2023). Understanding the impact of reinforcement learning personalization on subgroups of students in math tutoring. In Artificial Intelligence in Education., volume 1831, pages 939–946. Springer.

Riedmann, A., S. P. . L. B. (2025). Reinforcement learning in education: A systematic literature review. Int J Artif Intell Educ 35, page 2669–2723.

Ruan, S., Nie, A., Steenbergen, W., He, J., Zhang, J. Q., Guo, M., Liu, Y., Nguyen, K. D., Wang, C. Y., Ying, R., Landay, J. A., and Brunskill, E. (2024). Reinforcement learning tutor better supported lower performers in a math task. Machine Learning, 113:3023–3048.

Sen, A., Patel, P., Rau, M. A., Mason, B., Nowak, R., Rogers, T. T., and 0001, X. Z. (2018). Machine beats human at sequencing visuals for perceptual-fluency practice. In Proceedings of the 11th International Conference on Educational Data Mining.

Sutton, R. S. and Barto, A. G. (2018). Reinforcement Learning: An Introduction. MIT Press, 2 edition.

Watkins, C. J. C. H. and Dayan, P. (1992). Q-learning. Machine Learning, 8(3):279–292.