How Metacognition Shapes Higher Education Adoption of LLMs: A Structural Equation Modeling Approach
Abstract
Large Language Models (LLMs) are transforming education by facilitating adaptive learning experiences, yet their societal acceptance remains vital for successful integration. This study investigates the relationship between metacognition and LLM acceptance, with academic burnout as a mediating factor. Using Structural Equation Modeling (SEM), we analyzed validated psychometric data from 178 computer science students, drawn from a published dataset. Results show that metacognition significantly enhances LLM acceptance (β² = 0.220) and reduces academic burnout (β² = 0.038). While burnout positively predicts LLM acceptance (β² = 0.067), its indirect effect was negative, revealing a suppressor effect: students using LLMs metacognitively tend to experience less burnout, yet the beneficial impact on acceptance stems from proactive, strategic engagement—not from stress relief.
References
Breder, G., Brum, D., Dirk, L., & Ferro, M. (2024). O Paradoxo da IA para a Sustentabilidade e a Sustentabilidade da IA. In Anais do V Workshop sobre as Implicações da Computação na Sociedade, (pp. 105-116). Porto Alegre: SBC. DOI: 10.5753/wics.2024.2363
Carvalho, D., Ferro, M., Corrêa, F., Faria, V., Lima, L., Souza, A., & Gromato, M. (2024). Um Estudo Sobre a Percepção e Atitude dos Usuários de Sistemas Computacionais em Relação à Inteligência Artificial. In Anais do V Workshop sobre as Implicações da Computação na Sociedade, pp. 13-23. Porto Alegre: SBC. DOI: 10.5753/wics.2024.1919
Delello, J. A., Sung, W., Mokhtari, K., Hebert, J., Bronson, A., and DeGiuseppe, T .(2025). AI in the Classroom: Insights fromEducators onUsage,Challenges,andMental Health. Education Sciences, 15(2), 113. DOI: 10.3390/educsci1502011
Dempere, J. M., Modugu, K. P., Hesham, A., and Ramasamy, L. K. (2023). The impact of ChatGPT on higher education. Frontiers in Education, 8. DOI: 10.3389/feduc.2023.1206936
Gan, W., Qi, Z., Wu, J., & Lin, J. C. W. (2023). Large Language Models in Education: Vision and Opportunities. ArXiv Preprints. DOI: 10.48550/arXiv.2311.13160
Godoe, P., and Johansen, T. S. (2012). Understanding adoption of new technologies: Technology readiness and technology acceptance as an integrated concept. Journal of European Psychology Students, 3(1), 38-52. DOI: 10.5334/jeps.aq
Leite, P., Guarda, G., & Silveira, I. (2023). Building Awareness About Computational Thinking as a Research Field through a Graduate Course for Computer Scientists: a Metacognitive and Self-regulated proposal. In Anais do XXXI Workshop sobre Educação em Computação, 2023, 224-234. Porto Alegre: SBC. DOI: 10.5753/wei.2023.230615
Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V., Vanderplas, J., Passos, A., Cournapeau, D., Brucher, M., Perrot, M., & Duchesnay, É. (2011). Scikit-learn: Machine Learning in Python. Journal of Machine Learning Research, 12, 2825–2830. DOI: 10.5555/1953048.2078195
Pinto, P. H. R., Araujo, V. M. U. D., Ferreira Junior, C. D. S., Goulart, L. L., Aguiar, G. S., Beltrão, J. V. C., Lira, P. D. D., Mendes, S. J. F., Monteiro, F. D. L. V., & Avelino, E. L. (2023). Assessing the Psychological Impact of Generative AI on Computer and Data Science Education: An Exploratory Study. Preprints. DOI: 10.20944/preprints202312.0379.v2
Pinto, P. H. R. (2025). Psychometric Responses to LLMs [Data set]. Kaggle. DOI: 10.34740/KAGGLE/DS/6906431
Preacher, K. J., & Hayes, A. F. (2008). Asymptotic and resampling strategies for assessing and comparing indirect effects in multiple mediator models. Behavior Research Methods, 40(3), 879–891. DOI: 10.3758/BRM.40.3.879
Sardi, J., Darmansyah, Candra, O., Yuliana, D. F., Habibullah, Yanto, D. T. P., and Eliza, F. (2025). How Generative AI Influences Students’ Self-Regulated Learning and Critical Thinking Skills? A Systematic Review. International Journal of Engineering Pedagogy, 15(1), 94–108. DOI: 10.3991/ijep.v15i1.53379
Sapancı, A. (2023). From metacognition to academic burnout in university students: The mediating role of mindfulness. Journal of Pedagogical Research, 7(4), 356-368. DOI: 10.33902/JPR.202313708
Silva, M., Seixas, E., Ferro, M., Viterbo, J., Seixas, F., & Salgado, L. (2024). Ética e Responsabilidade na Era da Inteligência Artificial: Um Survey com Estudantes de Computação. In Anais do XXXII Workshop sobre Educação em Computação, (pp. 854-865). Porto Alegre: SBC. DOI: 10.5753/wei.2024.3148
Tu, X., Zou, J., Su, W.J., Zhang, L. (2023). What Should Data Science Education Do with Large Language Models? ArXiv. DOI: 10.1162/99608f92.bff007ab
Yin, J., Zhu, Y., Goh, T.-T., Wu, W., & Hu, Y. (2024). Using Educational Chatbots with Metacognitive Feedback to Improve Science Learning. Applied Sciences, 14(20), 9345. DOI: 10.3390/app14209345
Walter, Y. (2024) Embracing the future of Artificial Intelligence in the classroom: the relevance of AI literacy, prompt engineering, and critical thinking in modern education. International Journal of Educational Technology in Higher Education, 21(15). DOI: 10.1186/s41239-024-00448-3
Wolf, E. J., Harrington, K. M., Clark, S. L., & Miller, M. W. (2013). Sample size requirements for structural equation models: An evaluation of power, bias, and solution propriety. Educational and Psychological Measurement, 73(6), 913. DOI: 10.1177/0013164413495237
Wijaya, T. T., Yu, Q., Cao, Y., He, Y. & Leung, F. K. S. (2024). Latent Profile Analysis of AI Literacy and Trust in Mathematics Teachers and Their Relations with AI Dependency and 21st-Century Skills. Behavioral Sciences, 14(11). DOI: 10.3390/bs14111008
Xiao, J., Alibakhshi, G., Zamanpour, A., Zarei, M. A., Sherafat, S., & Behzadpoor, S.-F. (2024). How AI Literacy Affects Students’ Educational Attainment in Online Learning: Testing a Structural Equation Model in Higher Education Context. The International Review of Research in Open and Distributed Learning, 25(3), 179–198. DOI: 10.19173/irrodl.v25i3.7720
Zhou, X., Teng, D., & Hosam Al-Samarraie. (2024). The Mediating Role of Generative AI Self-Regulation on Students’ Critical Thinking and Problem-Solving. Education Sciences, 14(12), 1302–1302. DOI: 10.3390/educsci14121302.
Zhou, X., Fang, L., & Rajaram, K. (2025). Exploring the digital divide among students of diverse demographic backgrounds: A survey of UK undergraduates. Journal of Applied Learning & Teaching, 8(1). DOI: 10.37074/jalt.2025.8.1.22.
Carvalho, D., Ferro, M., Corrêa, F., Faria, V., Lima, L., Souza, A., & Gromato, M. (2024). Um Estudo Sobre a Percepção e Atitude dos Usuários de Sistemas Computacionais em Relação à Inteligência Artificial. In Anais do V Workshop sobre as Implicações da Computação na Sociedade, pp. 13-23. Porto Alegre: SBC. DOI: 10.5753/wics.2024.1919
Delello, J. A., Sung, W., Mokhtari, K., Hebert, J., Bronson, A., and DeGiuseppe, T .(2025). AI in the Classroom: Insights fromEducators onUsage,Challenges,andMental Health. Education Sciences, 15(2), 113. DOI: 10.3390/educsci1502011
Dempere, J. M., Modugu, K. P., Hesham, A., and Ramasamy, L. K. (2023). The impact of ChatGPT on higher education. Frontiers in Education, 8. DOI: 10.3389/feduc.2023.1206936
Gan, W., Qi, Z., Wu, J., & Lin, J. C. W. (2023). Large Language Models in Education: Vision and Opportunities. ArXiv Preprints. DOI: 10.48550/arXiv.2311.13160
Godoe, P., and Johansen, T. S. (2012). Understanding adoption of new technologies: Technology readiness and technology acceptance as an integrated concept. Journal of European Psychology Students, 3(1), 38-52. DOI: 10.5334/jeps.aq
Leite, P., Guarda, G., & Silveira, I. (2023). Building Awareness About Computational Thinking as a Research Field through a Graduate Course for Computer Scientists: a Metacognitive and Self-regulated proposal. In Anais do XXXI Workshop sobre Educação em Computação, 2023, 224-234. Porto Alegre: SBC. DOI: 10.5753/wei.2023.230615
Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V., Vanderplas, J., Passos, A., Cournapeau, D., Brucher, M., Perrot, M., & Duchesnay, É. (2011). Scikit-learn: Machine Learning in Python. Journal of Machine Learning Research, 12, 2825–2830. DOI: 10.5555/1953048.2078195
Pinto, P. H. R., Araujo, V. M. U. D., Ferreira Junior, C. D. S., Goulart, L. L., Aguiar, G. S., Beltrão, J. V. C., Lira, P. D. D., Mendes, S. J. F., Monteiro, F. D. L. V., & Avelino, E. L. (2023). Assessing the Psychological Impact of Generative AI on Computer and Data Science Education: An Exploratory Study. Preprints. DOI: 10.20944/preprints202312.0379.v2
Pinto, P. H. R. (2025). Psychometric Responses to LLMs [Data set]. Kaggle. DOI: 10.34740/KAGGLE/DS/6906431
Preacher, K. J., & Hayes, A. F. (2008). Asymptotic and resampling strategies for assessing and comparing indirect effects in multiple mediator models. Behavior Research Methods, 40(3), 879–891. DOI: 10.3758/BRM.40.3.879
Sardi, J., Darmansyah, Candra, O., Yuliana, D. F., Habibullah, Yanto, D. T. P., and Eliza, F. (2025). How Generative AI Influences Students’ Self-Regulated Learning and Critical Thinking Skills? A Systematic Review. International Journal of Engineering Pedagogy, 15(1), 94–108. DOI: 10.3991/ijep.v15i1.53379
Sapancı, A. (2023). From metacognition to academic burnout in university students: The mediating role of mindfulness. Journal of Pedagogical Research, 7(4), 356-368. DOI: 10.33902/JPR.202313708
Silva, M., Seixas, E., Ferro, M., Viterbo, J., Seixas, F., & Salgado, L. (2024). Ética e Responsabilidade na Era da Inteligência Artificial: Um Survey com Estudantes de Computação. In Anais do XXXII Workshop sobre Educação em Computação, (pp. 854-865). Porto Alegre: SBC. DOI: 10.5753/wei.2024.3148
Tu, X., Zou, J., Su, W.J., Zhang, L. (2023). What Should Data Science Education Do with Large Language Models? ArXiv. DOI: 10.1162/99608f92.bff007ab
Yin, J., Zhu, Y., Goh, T.-T., Wu, W., & Hu, Y. (2024). Using Educational Chatbots with Metacognitive Feedback to Improve Science Learning. Applied Sciences, 14(20), 9345. DOI: 10.3390/app14209345
Walter, Y. (2024) Embracing the future of Artificial Intelligence in the classroom: the relevance of AI literacy, prompt engineering, and critical thinking in modern education. International Journal of Educational Technology in Higher Education, 21(15). DOI: 10.1186/s41239-024-00448-3
Wolf, E. J., Harrington, K. M., Clark, S. L., & Miller, M. W. (2013). Sample size requirements for structural equation models: An evaluation of power, bias, and solution propriety. Educational and Psychological Measurement, 73(6), 913. DOI: 10.1177/0013164413495237
Wijaya, T. T., Yu, Q., Cao, Y., He, Y. & Leung, F. K. S. (2024). Latent Profile Analysis of AI Literacy and Trust in Mathematics Teachers and Their Relations with AI Dependency and 21st-Century Skills. Behavioral Sciences, 14(11). DOI: 10.3390/bs14111008
Xiao, J., Alibakhshi, G., Zamanpour, A., Zarei, M. A., Sherafat, S., & Behzadpoor, S.-F. (2024). How AI Literacy Affects Students’ Educational Attainment in Online Learning: Testing a Structural Equation Model in Higher Education Context. The International Review of Research in Open and Distributed Learning, 25(3), 179–198. DOI: 10.19173/irrodl.v25i3.7720
Zhou, X., Teng, D., & Hosam Al-Samarraie. (2024). The Mediating Role of Generative AI Self-Regulation on Students’ Critical Thinking and Problem-Solving. Education Sciences, 14(12), 1302–1302. DOI: 10.3390/educsci14121302.
Zhou, X., Fang, L., & Rajaram, K. (2025). Exploring the digital divide among students of diverse demographic backgrounds: A survey of UK undergraduates. Journal of Applied Learning & Teaching, 8(1). DOI: 10.37074/jalt.2025.8.1.22.
Published
2025-07-20
How to Cite
PINTO, Pedro Henrique Ramos; ARAÚJO, Vitor Meneghetti Ugulino de; LOPEZ, Luiz Carlos Serramo.
How Metacognition Shapes Higher Education Adoption of LLMs: A Structural Equation Modeling Approach. In: WORKSHOP ON THE IMPLICATIONS OF COMPUTING IN SOCIETY (WICS), 6. , 2025, Maceió/AL.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2025
.
p. 92-104.
ISSN 2763-8707.
DOI: https://doi.org/10.5753/wics.2025.8977.
