A Psychometric Re-evaluation of ChatGPT-Related Constructs Across Demographic Groups in Higher Education
Resumo
Generative artificial intelligence systems offer increasing opportunities to support learning and teaching in higher education. However, their effective, ethical, and responsible integration depends strongly on students' AI literacy. This study applies and psychometrically re-evaluates a validated AI Literacy Scale among 270 higher education students in a new empirical context, study design, and broader usage scenario. It further investigates students' factual knowledge, Actual Use, Use Value, and Attitude toward generative AI, as well as differences across gender and age groups. The findings support the use of the instrument in comparative AI literacy studies and provide evidence on demographic variability in students' engagement with generative AI.
Palavras-chave:
AI Literacy, ChatGPT, Higher Education
Referências
Ali, D., Fatemi, Y., Boskabadi, E., Nikfar, M., Ugwuoke, J., and Ali, H. (2024). Chatgpt in teaching and learning: A systematic review. Education Sciences, 14(6):643.
Annapureddy, R., Fornaroli, A., and Gatica-Perez, D. (2025). Generative ai literacy: Twelve defining competencies. Digit. Gov.: Res. Pract., 6(1).
Bauer, E., Greiff, S., Graesser, A. C., Scheiter, K., and Sailer, M. (2025). Looking beyond the hype: Understanding the effects of ai on learning. Educational Psychology Review, 37:45.
Chiu, T. K. (2024). The impact of generative ai (genai) on practices, policies and research direction in education: A case of chatgpt and midjourney. Interactive Learning Environments.
Chiu, T. K., Ahmad, Z., Sanusi, T., Sanusi, I., and Ismailov, M. (2024). What are artificial intelligence literacy and competency? a comprehensive framework to support them. Computers and Education Open, 6:100171.
Chiu, T. K. and Sanusi, I. (2024). Define, foster, and assess student and teacher ai literacy and competency for all: Current status and future research direction. Computers and Education Open.
Costa, A., Lima, N., Viegas, C., and Caldeira, A. (2024). Critical minds: Enhancing education with chatgpt. Cogent Education, 11(1).
Dempere, J., Modugu, K., Hesham, A., and Ramasamy, L. K. (2023). The impact of ChatGPT on higher education. Frontiers in Education, 8.
Eljack, N., Altahir, N., Mohammed, N., and Osman, S. (2025). Chatgpt as an ai assistant tool to enhance university students' teaching, assessment and academic achievements: A case study of an eap lecturer. Journal of Language Teaching and Research, 16:1813–1823.
Hornberger, M., Bewersdorff, A., and Nerdel, C. (2023). What do university students know about artificial intelligence? development and validation of an ai literacy test. Computers and Education: Artificial Intelligence, 5:100165.
Hornberger, M., Bewersdorff, A., Schiff, D., and Nerdel, C. (2025). A multinational assessment of ai literacy among university students in germany, the uk, and the us. Computers in Human Behavior: Artificial Humans, 4:100132.
Jalon, J. and Chua, G. (2024). Chatgpt as a learning assistant: Its impact on students' learning and experiences. International Journal of Education in Mathematics Science and Technology, 12:1603–1619.
Jin, Y., Martinez-Maldonado, R., Gašević, D., and Yan, L. (2025a). Glat: The generative ai literacy assessment test. Computers and Education: Artificial Intelligence, 9:100436.
Jin, Y., Yan, L., Echeverria, V., Gašević, D., and Martinez-Maldonado, R. (2025b). Generative ai in higher education: A global perspective of institutional adoption policies and guidelines. Computers and Education: Artificial Intelligence, 8:100348.
Khan, N., Khan, Z., Koubaa, A., Khan, M. K., and bin Salleh, R. (2024). Global insights and the impact of generative ai-chatgpt on multidisciplinary: a systematic review and bibliometric analysis. Connection Science, 36(1):2353630.
Köhler, C. and Hartig, J. (2024). Chatgpt in higher education: Measurement instruments to assess student knowledge, usage, and attitude. Contemporary Educational Technology, 16(1):ep528.
Long, D. and Magerko, B. (2020). What is ai literacy? competencies and design considerations. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, CHI '20, page 1–16, New York, NY, USA. Association for Computing Machinery.
Markus, A., Carolus, A., and Wienrich, C. (2025). Objective measurement of ai literacy: Development and validation of the ai competency objective scale (aicos). Computers and Education: Artificial Intelligence, 9:100485.
Marzano, D. (2025). Generative artificial intelligence (gai) in teaching and learning processes at the k-12 level: A systematic review. Tech Know Learn.
McDonald, N., Johri, A., and Collier, A. (2025). Generative artificial intelligence in higher education: Evidence from an analysis of institutional policies and guidelines. Computers in Human Behavior: Artificial Humans, 3:100121.
Molerov, D., Zlatkin-Troitschanskaia, O., Nagel, M.-T., Brückner, S., Schmidt, S., and Shavelson, R. (2020). Assessing university students' critical online reasoning ability: A conceptual and assessment framework with preliminary evidence. Frontiers in Education, 5.
Ogunleye, B., Zakariyyah, K. I., Ajao, O., Olayinka, O., and Sharma, H. (2024). A systematic review of generative ai for teaching and learning practice. Education Sciences, 14(6).
Sallam, M., Elsayed, W., Al-Shorbagy, M., Barakat, M., El Khatib, S., Ghach, W., Alwan, N., Hallit, S., and Malaeb, D. (2024). Chatgpt usage and attitudes are driven by perceptions of usefulness, ease of use, risks, and psycho-social impact: A study among university students in the uae.
Shi, Y., Yu, K., Dong, Y., and Chen, F. (2026). Large language models in education: a systematic review of empirical applications, benefits, and challenges. Computers and Education: Artificial Intelligence, 10:100529.
Sullivan, M., Kelly, A., and McLaughlan, P. (2023). Chatgpt in higher education: Considerations for academic integrity and student learning. Journal of Applied Learning & Teaching, 6(1):17–34.
Villar Onrubia, D., Cachia, R., Rietz, C., Feltrero, R., Niemi, H., Hallissy, M., and Reuter, R. (2025). Generative artificial intelligence in secondary education: Uses and perceptions from the perspective of early adopters across five eu member states.
Wu, F., Dang, Y., and Li, M. (2025). A systematic review of responses, attitudes, and utilization behaviors on generative ai for teaching and learning in higher education. Behavioral Sciences, 15(4).
Yusuf, A., Pervin, N., Román-González, M., and Noor, N. M. (2024). Generative ai in education and research: A systematic mapping review. Review of Education, 12(2):e3489. e3489 RoE-2024-01-1127.R3.
Zhang, P. and Tur, G. (2023). A systematic review of chatgpt use in k-12 education. European Journal of Education, pages 1–22.
Zirar, A. (2023). Exploring the impact of language models, such as ChatGPT, on student learning and assessment. Review of Education, 11.
Zlatkin-Troitschanskaia, O., Fischer, J., and Molerov, D. (2023). Critical online reasoning in higher education (core): Overview of the research unit.
Annapureddy, R., Fornaroli, A., and Gatica-Perez, D. (2025). Generative ai literacy: Twelve defining competencies. Digit. Gov.: Res. Pract., 6(1).
Bauer, E., Greiff, S., Graesser, A. C., Scheiter, K., and Sailer, M. (2025). Looking beyond the hype: Understanding the effects of ai on learning. Educational Psychology Review, 37:45.
Chiu, T. K. (2024). The impact of generative ai (genai) on practices, policies and research direction in education: A case of chatgpt and midjourney. Interactive Learning Environments.
Chiu, T. K., Ahmad, Z., Sanusi, T., Sanusi, I., and Ismailov, M. (2024). What are artificial intelligence literacy and competency? a comprehensive framework to support them. Computers and Education Open, 6:100171.
Chiu, T. K. and Sanusi, I. (2024). Define, foster, and assess student and teacher ai literacy and competency for all: Current status and future research direction. Computers and Education Open.
Costa, A., Lima, N., Viegas, C., and Caldeira, A. (2024). Critical minds: Enhancing education with chatgpt. Cogent Education, 11(1).
Dempere, J., Modugu, K., Hesham, A., and Ramasamy, L. K. (2023). The impact of ChatGPT on higher education. Frontiers in Education, 8.
Eljack, N., Altahir, N., Mohammed, N., and Osman, S. (2025). Chatgpt as an ai assistant tool to enhance university students' teaching, assessment and academic achievements: A case study of an eap lecturer. Journal of Language Teaching and Research, 16:1813–1823.
Hornberger, M., Bewersdorff, A., and Nerdel, C. (2023). What do university students know about artificial intelligence? development and validation of an ai literacy test. Computers and Education: Artificial Intelligence, 5:100165.
Hornberger, M., Bewersdorff, A., Schiff, D., and Nerdel, C. (2025). A multinational assessment of ai literacy among university students in germany, the uk, and the us. Computers in Human Behavior: Artificial Humans, 4:100132.
Jalon, J. and Chua, G. (2024). Chatgpt as a learning assistant: Its impact on students' learning and experiences. International Journal of Education in Mathematics Science and Technology, 12:1603–1619.
Jin, Y., Martinez-Maldonado, R., Gašević, D., and Yan, L. (2025a). Glat: The generative ai literacy assessment test. Computers and Education: Artificial Intelligence, 9:100436.
Jin, Y., Yan, L., Echeverria, V., Gašević, D., and Martinez-Maldonado, R. (2025b). Generative ai in higher education: A global perspective of institutional adoption policies and guidelines. Computers and Education: Artificial Intelligence, 8:100348.
Khan, N., Khan, Z., Koubaa, A., Khan, M. K., and bin Salleh, R. (2024). Global insights and the impact of generative ai-chatgpt on multidisciplinary: a systematic review and bibliometric analysis. Connection Science, 36(1):2353630.
Köhler, C. and Hartig, J. (2024). Chatgpt in higher education: Measurement instruments to assess student knowledge, usage, and attitude. Contemporary Educational Technology, 16(1):ep528.
Long, D. and Magerko, B. (2020). What is ai literacy? competencies and design considerations. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, CHI '20, page 1–16, New York, NY, USA. Association for Computing Machinery.
Markus, A., Carolus, A., and Wienrich, C. (2025). Objective measurement of ai literacy: Development and validation of the ai competency objective scale (aicos). Computers and Education: Artificial Intelligence, 9:100485.
Marzano, D. (2025). Generative artificial intelligence (gai) in teaching and learning processes at the k-12 level: A systematic review. Tech Know Learn.
McDonald, N., Johri, A., and Collier, A. (2025). Generative artificial intelligence in higher education: Evidence from an analysis of institutional policies and guidelines. Computers in Human Behavior: Artificial Humans, 3:100121.
Molerov, D., Zlatkin-Troitschanskaia, O., Nagel, M.-T., Brückner, S., Schmidt, S., and Shavelson, R. (2020). Assessing university students' critical online reasoning ability: A conceptual and assessment framework with preliminary evidence. Frontiers in Education, 5.
Ogunleye, B., Zakariyyah, K. I., Ajao, O., Olayinka, O., and Sharma, H. (2024). A systematic review of generative ai for teaching and learning practice. Education Sciences, 14(6).
Sallam, M., Elsayed, W., Al-Shorbagy, M., Barakat, M., El Khatib, S., Ghach, W., Alwan, N., Hallit, S., and Malaeb, D. (2024). Chatgpt usage and attitudes are driven by perceptions of usefulness, ease of use, risks, and psycho-social impact: A study among university students in the uae.
Shi, Y., Yu, K., Dong, Y., and Chen, F. (2026). Large language models in education: a systematic review of empirical applications, benefits, and challenges. Computers and Education: Artificial Intelligence, 10:100529.
Sullivan, M., Kelly, A., and McLaughlan, P. (2023). Chatgpt in higher education: Considerations for academic integrity and student learning. Journal of Applied Learning & Teaching, 6(1):17–34.
Villar Onrubia, D., Cachia, R., Rietz, C., Feltrero, R., Niemi, H., Hallissy, M., and Reuter, R. (2025). Generative artificial intelligence in secondary education: Uses and perceptions from the perspective of early adopters across five eu member states.
Wu, F., Dang, Y., and Li, M. (2025). A systematic review of responses, attitudes, and utilization behaviors on generative ai for teaching and learning in higher education. Behavioral Sciences, 15(4).
Yusuf, A., Pervin, N., Román-González, M., and Noor, N. M. (2024). Generative ai in education and research: A systematic mapping review. Review of Education, 12(2):e3489. e3489 RoE-2024-01-1127.R3.
Zhang, P. and Tur, G. (2023). A systematic review of chatgpt use in k-12 education. European Journal of Education, pages 1–22.
Zirar, A. (2023). Exploring the impact of language models, such as ChatGPT, on student learning and assessment. Review of Education, 11.
Zlatkin-Troitschanskaia, O., Fischer, J., and Molerov, D. (2023). Critical online reasoning in higher education (core): Overview of the research unit.
Publicado
05/10/2026
Como Citar
RIBEIRO, Fabíola G. C. et al.
A Psychometric Re-evaluation of ChatGPT-Related Constructs Across Demographic Groups in Higher Education. In: SIMPÓSIO BRASILEIRO DE INFORMÁTICA NA EDUCAÇÃO (SBIE), 37. , 2026, Goiânia/GO.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2026
.
p. 1717-1731.
DOI: https://doi.org/10.5753/sbie.2026.28061.
