Evaluating Educational Recommendation Systems: a systematic mapping
Resumo
Recommendation systems (RS) have been used in many scenarios, from entertainment to health. Inside the RS area, Educational Recommendation Systems (ERS) are becoming popular, been used for different types of recommendations such as recommending materials, exercises, and learning paths. As ERS works in a different scenario of classics RS, ERS requires specific evaluation metrics. However, the task of evaluating ERS is difficult once the educational field has its features to be analyzed. To help other researchers in this field, this work presents a systematic mapping on methods used for evaluating ERS. This study analyzed 91 papers of the last five years and provide an overview of the main methodologies, subject, metrics, and trends in the evaluation of ERS.Referências
Andjelkovic, I., Parra, D., and O’Donovan, J. (2019). Moodplay: Interactive music recommendation based on artists’ mood similarity. International Journal of Human-Computer Studies, 121:142–159.
Cui, L.-Z., Guo, F.-L., and Liang, Y.-j. (2018). Research overview of educational recommender systems. In Proceedings of the 2nd International Conference on Computer
Science and Application Engineering, pages 1–7.
de Almeida, M. A., Souza, J., and Barrere, E. (2019). Learning style identification and usage in academia: A systematic mapping. In Brazilian Symposium on Computers in Education (Simposio Brasileiro de Informática na Educação-SBIE), volume 30, page 1341.
De Medio, C., Limongelli, C., Sciarrone, F., and Temperini, M. (2020). Moodlerec: A recommendation system for creating courses using the moodle e-learning platform.
Computers in Human Behavior, 104:106168.
Drachsler, H., Verbert, K., Santos, O. C., and Manouselis, N. (2015). Panorama of Recommender Systems to Support Learning, pages 421–451. Springer US, Boston, MA.
Dwivedi, S. and Roshni, V. K. (2017). Recommender system for big data in education. In 2017 5th National Conference on E-Learning & E-Learning Technologies (ELELTECH), pages 1–4. IEEE.
Erdt, M., Fernandez, A., and Rensing, C. (2015). Evaluating recommender systems for technology enhanced learning: a quantitative survey. IEEE Transactions on Learning Technologies, 8(4):326–344.
Gyrard, A. and Sheth, A. (2020). Iamhappy: Towards an iot knowledge-based crossdomain well-being recommendation system for everyday happiness. Smart Health,
15:100083.
Khosravifar, B., Cuevas, C., and Bentahar, J. (2018). Dynamic peer-to-peer amplifier system used in agent-based intelligent tutoring system. In Proceedings on the International Conference on Artificial Intelligence (ICAI), pages 352–358. The Steering Committee of The World Congress in Computer Science, Computer . . . .
Knijnenburg, B. P. (2012). Conducting user experiments in recommender systems. In Proceedings of the sixth ACM conference on Recommender systems, pages 3–4.
Lv, P., Wang, X., Xu, J., and Wang, J. (2018). Utilizing knowledge graph and student testing behavior data for personalized exercise recommendation. In Proceedings of
ACM Turing Celebration Conference-China, pages 53–59.
Machado, M. d. O. C., Barrere, E., and Souza, J. (2019). Solving the adaptive curriculum sequencing problem with prey-predator algorithm. International Journal of Distance Education Technologies (IJDET), 17(4):71–93.
Machado, M. d. O. C., Bravo, N. F. S., Martins, A. F., Bernardino, H. S., Barrere, E., and de Souza, J. F. (2020). Metaheuristic-based adaptive curriculum sequencing approaches: a systematic review and mapping of the literature. Artificial Intelligence Review, pages 1–44.
Melville, P. and Sindhwani, V. (2010). Recommender Systems, pages 829–838. Springer US, Boston, MA.
Nabizadeh, A. H., Leal, J. P., Rafsanjani, H. N., and Shah, R. R. (2020). Learning path personalization and recommendation methods: A survey of the state-of-the-art. Expert Systems with Applications, page 113596.
Peralta, M., Alarcon, R., Pichara, K., Mery, T., Cano, F., and Bozo, J. (2018). Understanding learning resources metadata for primary and secondary education. IEEE
Transactions on Learning Technologies, 11(4):456–467.
Pu, P., Chen, L., and Hu, R. (2011). A user-centric evaluation framework for recommender systems. In Proceedings of the fifth ACM conference on Recommender systems, pages 157–164.
Rivera, A. C., Tapia-Leon, M., and Lujan-Mora, S. (2018). Recommendation systems in education: A systematic mapping study. In International Conference on Information Theoretic Security, pages 937–947. Springer.
Santos, O. C., Boticario, J. G., and Manjarres-Riesco, ´ A. (2014). An approach for an affective educational recommendation model. In Recommender Systems for Technology Enhanced Learning, pages 123–143. Springer.
Shani, G. and Gunawardana, A. (2011). Evaluating Recommendation Systems, pages 257–297. Springer US, Boston, MA.
Tarus, J. K., Niu, Z., and Mustafa, G. (2018). Knowledge-based recommendation: a review of ontology-based recommender systems for e-learning. Artificial Intelligence Review, 50(1):21–48.
Truong, H. M. (2016). Integrating learning styles and adaptive e-learning system: Current developments, problems and opportunities. Computers in Human Behavior, 55:1185 – 1193.
Yu, X., Wei, D., Chu, Q., and Wang, H. (2018). The personalized recommendation algorithms in educational application. In 2018 9th International Conference on Information Technology in Medicine and Education (ITME), pages 664–668. IEEE
Cui, L.-Z., Guo, F.-L., and Liang, Y.-j. (2018). Research overview of educational recommender systems. In Proceedings of the 2nd International Conference on Computer
Science and Application Engineering, pages 1–7.
de Almeida, M. A., Souza, J., and Barrere, E. (2019). Learning style identification and usage in academia: A systematic mapping. In Brazilian Symposium on Computers in Education (Simposio Brasileiro de Informática na Educação-SBIE), volume 30, page 1341.
De Medio, C., Limongelli, C., Sciarrone, F., and Temperini, M. (2020). Moodlerec: A recommendation system for creating courses using the moodle e-learning platform.
Computers in Human Behavior, 104:106168.
Drachsler, H., Verbert, K., Santos, O. C., and Manouselis, N. (2015). Panorama of Recommender Systems to Support Learning, pages 421–451. Springer US, Boston, MA.
Dwivedi, S. and Roshni, V. K. (2017). Recommender system for big data in education. In 2017 5th National Conference on E-Learning & E-Learning Technologies (ELELTECH), pages 1–4. IEEE.
Erdt, M., Fernandez, A., and Rensing, C. (2015). Evaluating recommender systems for technology enhanced learning: a quantitative survey. IEEE Transactions on Learning Technologies, 8(4):326–344.
Gyrard, A. and Sheth, A. (2020). Iamhappy: Towards an iot knowledge-based crossdomain well-being recommendation system for everyday happiness. Smart Health,
15:100083.
Khosravifar, B., Cuevas, C., and Bentahar, J. (2018). Dynamic peer-to-peer amplifier system used in agent-based intelligent tutoring system. In Proceedings on the International Conference on Artificial Intelligence (ICAI), pages 352–358. The Steering Committee of The World Congress in Computer Science, Computer . . . .
Knijnenburg, B. P. (2012). Conducting user experiments in recommender systems. In Proceedings of the sixth ACM conference on Recommender systems, pages 3–4.
Lv, P., Wang, X., Xu, J., and Wang, J. (2018). Utilizing knowledge graph and student testing behavior data for personalized exercise recommendation. In Proceedings of
ACM Turing Celebration Conference-China, pages 53–59.
Machado, M. d. O. C., Barrere, E., and Souza, J. (2019). Solving the adaptive curriculum sequencing problem with prey-predator algorithm. International Journal of Distance Education Technologies (IJDET), 17(4):71–93.
Machado, M. d. O. C., Bravo, N. F. S., Martins, A. F., Bernardino, H. S., Barrere, E., and de Souza, J. F. (2020). Metaheuristic-based adaptive curriculum sequencing approaches: a systematic review and mapping of the literature. Artificial Intelligence Review, pages 1–44.
Melville, P. and Sindhwani, V. (2010). Recommender Systems, pages 829–838. Springer US, Boston, MA.
Nabizadeh, A. H., Leal, J. P., Rafsanjani, H. N., and Shah, R. R. (2020). Learning path personalization and recommendation methods: A survey of the state-of-the-art. Expert Systems with Applications, page 113596.
Peralta, M., Alarcon, R., Pichara, K., Mery, T., Cano, F., and Bozo, J. (2018). Understanding learning resources metadata for primary and secondary education. IEEE
Transactions on Learning Technologies, 11(4):456–467.
Pu, P., Chen, L., and Hu, R. (2011). A user-centric evaluation framework for recommender systems. In Proceedings of the fifth ACM conference on Recommender systems, pages 157–164.
Rivera, A. C., Tapia-Leon, M., and Lujan-Mora, S. (2018). Recommendation systems in education: A systematic mapping study. In International Conference on Information Theoretic Security, pages 937–947. Springer.
Santos, O. C., Boticario, J. G., and Manjarres-Riesco, ´ A. (2014). An approach for an affective educational recommendation model. In Recommender Systems for Technology Enhanced Learning, pages 123–143. Springer.
Shani, G. and Gunawardana, A. (2011). Evaluating Recommendation Systems, pages 257–297. Springer US, Boston, MA.
Tarus, J. K., Niu, Z., and Mustafa, G. (2018). Knowledge-based recommendation: a review of ontology-based recommender systems for e-learning. Artificial Intelligence Review, 50(1):21–48.
Truong, H. M. (2016). Integrating learning styles and adaptive e-learning system: Current developments, problems and opportunities. Computers in Human Behavior, 55:1185 – 1193.
Yu, X., Wei, D., Chu, Q., and Wang, H. (2018). The personalized recommendation algorithms in educational application. In 2018 9th International Conference on Information Technology in Medicine and Education (ITME), pages 664–668. IEEE
Publicado
24/11/2020
Como Citar
MARANTE, Yelco; SILVA, Vinicius Alberto Alves da; GOMES JR., Jorão; VITOR, Marluce Aparecida; MARTINS, André Ferreira; DE SOUZA, Jairo Francisco.
Evaluating Educational Recommendation Systems: a systematic mapping. In: SIMPÓSIO BRASILEIRO DE INFORMÁTICA NA EDUCAÇÃO (SBIE), 31. , 2020, Online.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2020
.
p. 912-921.
DOI: https://doi.org/10.5753/cbie.sbie.2020.912.
