An Evaluation of the GamAnalytics Tool: Is the Gamification Analytics Model Ready for Teachers?
Resumo
In order to improve students' learning outcomes, researchers and practitioners have increasingly applied gamification in technology-enhanced learning environments. However, some studies in the literature have reported unexpected negative results with that. To avoid these unexpected outcomes in gamified learning systems, we proposed the ``gamification analytics model for teachers" in a previous study. This model allows teachers to monitor and adapt the gamification design in the run-time of the teaching-learning process. In this paper, we present the results obtained in an empirical study to assess teachers' perception and acceptance regarding the GamAnalytics tool, a tool developed based on our proposed model that allows teachers to monitor students' interaction with learning resources and game elements and adapt tailored missions for students in gamified educational systems. The results indicate that the teachers have significantly good behavioral intention to use our tool and good perception of its usefulness and ease of use. The teachers also indicated enjoyment, relevance, and self-efficacy, and that the tool does not cause anxiety.Referências
Andrade, F. R. H., Mizoguchi, R., and Isotani, S. (2016). The Bright and Dark Sides of Gamification. In Micarelli, A., Stamper, J., and Panourgia, K., editors, Intelligent Tutoring Systems, pages 176–186, Cham. Springer International Publishing.
Baker, R. S., Corbett, A. T., Koedinger, K. R., and Wagner, A. Z. (2004). Off-task Behavior in the Cognitive Tutor Classroom: When Students ”Game the System”. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI ’04, pages 383–390, New York, NY, USA. ACM.
Baker, R. S., D’Mello, S. K., Rodrigo, M. T., and Graesser, A. C. (2010). Better to be frustrated than bored: The incidence, persistence, and impact of learners‘ cognitive-affective states during interactions with three different computer-based learning environments. International Journal of Human-Computer Studies, 68(4):223–241.
Corbin, J. M. and Strauss, A. L. (2008). Basics of qualitative research: techniques and procedures for developing grounded theory. Sage Publications.
Corsi-Bunker, A. (2000). GUIDE TO THE EDUCATION SYSTEM IN THE UNITED STATES. Technical report, University of Minnesota, Minnesota.
Dermeval, D. and Bittencourt, I. I. (2020). Co-designing Gamified Intelligent Tutoring Systems with Teachers. Revista Brasileira de Informatica na Educação (RBIE), 28(0):73–91.
Dermeval, D., Leite, G., Almeida, J., Albuquerque, J., Bittencourt, I. I., Siqueira, S. W., Isotani, S., and Da Silva, A. P. (2017). An ontology-driven software product line architecture for developing gamified intelligent tutoring systems. International Journal of Knowledge and Learning, 12(1):27–48.
Deterding, S., Dixon, D., Khaled, R., and Nacke, L. (2011). From game design elements to gamefulness. In Proceedings of the 15th International Academic MindTrek Conference on Envisioning Future Media Environments - MindTrek ’11, pages 9–15, Tampere. ACM.
D’Mello, S. and Graesser, A. (2012). Dynamics of affective states during complex learning. Learning and Instruction, 22(2):145–157.
Domínguez, A., Saenz-de Navarrete, J., de Marcos, L., Fernández-Sanz, L., Pagés, C., and Martínez-Herráiz, J.-J. (2013). Gamifying learning experiences: Practical implications and outcomes. Computers & Education, 63:380–392.
Hamari, J., Koivisto, J., and Sarsa, H. (2014). Does Gamification Work? – A Literature Review of Empirical Studies on Gamification. In 2014 47th Hawaii International Conference on System Sciences, pages 3025–3034.
Heilbrunn, B., Herzig, P., and Schill, A. (2017). Gamification Analytics - Methods and Tools for Monitoring and Adapting Gamification Designs. In Stieglitz, S., Lattemann, C., Robra-Bissantz, S., Zarnekow, R., and Brockmann, T., editors, Gamification: Using Game Elements in Serious Contexts, pages 31–47. Springer International Publishing, Cham.
Holstein, K., McLaren, B. M., and Aleven, V. (2017). Intelligent tutors as teachers’ aides: Exploring teacher needs for real-time analytics in blended classrooms. In ACM International Conference Proceeding Series, pages 257–266. Association for Computing Machinery.
Jackson, G. T. and McNamara, D. S. (2013). Motivation and performance in a game-based intelligent tutoring system. Journal of Educational Psychology, 105(4):1036–1049.
Kapp, K. M. (2012). The Gamification of Learning and Instruction: Game-based Methods and Strategies for Training and Education. Pfeiffer & Company, 1st edition.
Macleod, H. and Sinclair, C. (2017). Digital Learning and the Changing Role of the Teacher. In Encyclopedia of Educational Philosophy and Theory, pages 566–571. Springer Singapore, Singapore.
Orhan Göksün, D. and Gürsoy, G. (2019). Comparing success and engagement in ¨ gamified learning experiences via Kahoot and Quizizz. Computers and Education, 135:15–29.
Paiva, R., Bittencourt, I. I., Tenrio, T., Jaques, P., and Isotani, S. (2016). What Do Students Do On-line? Modeling Students’ Interactions to Improve Their Learning Experience. Comput. Hum. Behav., 64(C):769–781.
Sharpe, D. (2015). Chi-Square Test is Statistically Significant: Now What? Practical Assessment, Research, and Evaluation, 20(1).
Snow, E. L., Allen, L. K., Jackson, G. T., and McNamara, D. S. (2015). Spendency: Students‘ Propensity to Use System Currency. International Journal of Artificial Intelligence in Education, 25(3):407–427.
Subhash, S. and Cudney, E. A. (2018). Gamified learning in higher education: A systematic review of the literature. Computers in Human Behavior, 87:192 – 206.
Tenório, K., Chalco Challco, G., Dermeval, D., Lemos, B., Nascimento, P., Santos, R., and Pedro da Silva, A. (2020a). Helping Teachers Assist Their Students in Gamified Adaptive Educational Systems: Towards a Gamification Analytics Tool. pages 312– 317. Springer, Cham.
Tenório, K., Dermeval, D., Monteiro, M., Peixoto, A., and Pedro, A. (2020b). Raising Teachers Empowerment in Gamification Design of Adaptive Learning Systems: A Qualitative Research. pages 524–536. Springer, Cham.
Venkatesh, V. and Bala, H. (2008). Technology Acceptance Model 3 and a Research Agenda on Interventions. Decision Sciences, 39(2):273–315.
Werbach, K. and Hunter, D. (2015). The Gamification Toolkit: Dynamics, Mechanics, and Components for the Win. Wharton Digital Press.
Yuwono, K. T. and Sujono, H. D. (2018). The Effectiveness of E-Learning: A MetaAnalysis. Journal of Physics: Conference Series, 1140(1):012024.
Baker, R. S., Corbett, A. T., Koedinger, K. R., and Wagner, A. Z. (2004). Off-task Behavior in the Cognitive Tutor Classroom: When Students ”Game the System”. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI ’04, pages 383–390, New York, NY, USA. ACM.
Baker, R. S., D’Mello, S. K., Rodrigo, M. T., and Graesser, A. C. (2010). Better to be frustrated than bored: The incidence, persistence, and impact of learners‘ cognitive-affective states during interactions with three different computer-based learning environments. International Journal of Human-Computer Studies, 68(4):223–241.
Corbin, J. M. and Strauss, A. L. (2008). Basics of qualitative research: techniques and procedures for developing grounded theory. Sage Publications.
Corsi-Bunker, A. (2000). GUIDE TO THE EDUCATION SYSTEM IN THE UNITED STATES. Technical report, University of Minnesota, Minnesota.
Dermeval, D. and Bittencourt, I. I. (2020). Co-designing Gamified Intelligent Tutoring Systems with Teachers. Revista Brasileira de Informatica na Educação (RBIE), 28(0):73–91.
Dermeval, D., Leite, G., Almeida, J., Albuquerque, J., Bittencourt, I. I., Siqueira, S. W., Isotani, S., and Da Silva, A. P. (2017). An ontology-driven software product line architecture for developing gamified intelligent tutoring systems. International Journal of Knowledge and Learning, 12(1):27–48.
Deterding, S., Dixon, D., Khaled, R., and Nacke, L. (2011). From game design elements to gamefulness. In Proceedings of the 15th International Academic MindTrek Conference on Envisioning Future Media Environments - MindTrek ’11, pages 9–15, Tampere. ACM.
D’Mello, S. and Graesser, A. (2012). Dynamics of affective states during complex learning. Learning and Instruction, 22(2):145–157.
Domínguez, A., Saenz-de Navarrete, J., de Marcos, L., Fernández-Sanz, L., Pagés, C., and Martínez-Herráiz, J.-J. (2013). Gamifying learning experiences: Practical implications and outcomes. Computers & Education, 63:380–392.
Hamari, J., Koivisto, J., and Sarsa, H. (2014). Does Gamification Work? – A Literature Review of Empirical Studies on Gamification. In 2014 47th Hawaii International Conference on System Sciences, pages 3025–3034.
Heilbrunn, B., Herzig, P., and Schill, A. (2017). Gamification Analytics - Methods and Tools for Monitoring and Adapting Gamification Designs. In Stieglitz, S., Lattemann, C., Robra-Bissantz, S., Zarnekow, R., and Brockmann, T., editors, Gamification: Using Game Elements in Serious Contexts, pages 31–47. Springer International Publishing, Cham.
Holstein, K., McLaren, B. M., and Aleven, V. (2017). Intelligent tutors as teachers’ aides: Exploring teacher needs for real-time analytics in blended classrooms. In ACM International Conference Proceeding Series, pages 257–266. Association for Computing Machinery.
Jackson, G. T. and McNamara, D. S. (2013). Motivation and performance in a game-based intelligent tutoring system. Journal of Educational Psychology, 105(4):1036–1049.
Kapp, K. M. (2012). The Gamification of Learning and Instruction: Game-based Methods and Strategies for Training and Education. Pfeiffer & Company, 1st edition.
Macleod, H. and Sinclair, C. (2017). Digital Learning and the Changing Role of the Teacher. In Encyclopedia of Educational Philosophy and Theory, pages 566–571. Springer Singapore, Singapore.
Orhan Göksün, D. and Gürsoy, G. (2019). Comparing success and engagement in ¨ gamified learning experiences via Kahoot and Quizizz. Computers and Education, 135:15–29.
Paiva, R., Bittencourt, I. I., Tenrio, T., Jaques, P., and Isotani, S. (2016). What Do Students Do On-line? Modeling Students’ Interactions to Improve Their Learning Experience. Comput. Hum. Behav., 64(C):769–781.
Sharpe, D. (2015). Chi-Square Test is Statistically Significant: Now What? Practical Assessment, Research, and Evaluation, 20(1).
Snow, E. L., Allen, L. K., Jackson, G. T., and McNamara, D. S. (2015). Spendency: Students‘ Propensity to Use System Currency. International Journal of Artificial Intelligence in Education, 25(3):407–427.
Subhash, S. and Cudney, E. A. (2018). Gamified learning in higher education: A systematic review of the literature. Computers in Human Behavior, 87:192 – 206.
Tenório, K., Chalco Challco, G., Dermeval, D., Lemos, B., Nascimento, P., Santos, R., and Pedro da Silva, A. (2020a). Helping Teachers Assist Their Students in Gamified Adaptive Educational Systems: Towards a Gamification Analytics Tool. pages 312– 317. Springer, Cham.
Tenório, K., Dermeval, D., Monteiro, M., Peixoto, A., and Pedro, A. (2020b). Raising Teachers Empowerment in Gamification Design of Adaptive Learning Systems: A Qualitative Research. pages 524–536. Springer, Cham.
Venkatesh, V. and Bala, H. (2008). Technology Acceptance Model 3 and a Research Agenda on Interventions. Decision Sciences, 39(2):273–315.
Werbach, K. and Hunter, D. (2015). The Gamification Toolkit: Dynamics, Mechanics, and Components for the Win. Wharton Digital Press.
Yuwono, K. T. and Sujono, H. D. (2018). The Effectiveness of E-Learning: A MetaAnalysis. Journal of Physics: Conference Series, 1140(1):012024.
Publicado
24/11/2020
Como Citar
TENÓRIO, Kamilla; DERMEVAL, Diego; CHALLCO, Geiser Chalco; LEMOS, Bruno; NASCIMENTO, Pedro; SANTOS, Rodrigo; SILVA, Alan Pedro da.
An Evaluation of the GamAnalytics Tool: Is the Gamification Analytics Model Ready for Teachers?. In: SIMPÓSIO BRASILEIRO DE INFORMÁTICA NA EDUCAÇÃO (SBIE), 31. , 2020, Online.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2020
.
p. 562-571.
DOI: https://doi.org/10.5753/cbie.sbie.2020.562.
