Aprender ou Delegar? Um Estudo sobre Pensamento Crítico no Uso de IA Generativa em Avaliação Heurística
Resumo
Este trabalho investiga o uso de Inteligência Artificial Generativa (IAGen) por estudantes em atividades de Avaliação Heurística, analisando tanto a identificação de problemas de usabilidade quanto o pensamento crítico no uso das respostas geradas pela IA. Foi conduzido um experimento com 24 alunos de Engenharia de Software, divididos em grupo controle (sem IA) e experimental (com IA). Os resultados quantitativos não indicaram diferença estatisticamente significativa entre os grupos, embora o grupo com IA tenha apresentado menor média de acertos. A análise qualitativa e dos logs revelou discrepância entre a percepção dos estudantes e o uso real da IA, evidenciando predominância de uso passivo, reprodução acrítica de respostas, alucinações, sicofantia e dificuldade em identificar erros gerados pela ferramenta. Os resultados sugerem que o uso de IAGen sem mediação pedagógica adequada pode favorecer processos de delegação cognitiva em atividades conceituais de aprendizagem.
Palavras-chave:
Inteligência Artificial Generativa, Avaliação Heurística, Pensamento Crítico
Referências
Athey, S. and Imbens, G. W. (2017). The econometrics of randomized experiments. In Handbook of economic field experiments, volume 1, pages 73–140. Elsevier.
Bahroun, Z., Anane, C., Ahmed, V., and Zacca, A. (2023). Transforming education: A comprehensive review of generative artificial intelligence in educational settings through bibliometric and content analysis. Sustainability.
Baidoo-Anu, D. and Ansah, L. O. (2023). Education in the era of generative artificial intelligence (ai): Understanding the potential benefits of chatgpt in promoting teaching and learning. SSRN Electronic Journal.
Barbosa, S. D. J., Silva, B. d., Silveira, M. S., Gasparini, I., Darin, T., and Barbosa, G. D. J. (2021). Interação humano-computador e experiência do usuario. Auto publicação.
Barke, S., James, M. B., and Polikarpova, N. (2023). Grounded copilot: How programmers interact with code-generating models. Proceedings of the ACM on Programming Languages, 7(OOPSLA1):85–111.
Bauschard, S. and Quidwai, S. (2024). From insight to implementation: How to create your ai school guidance. Available at SSRN 4784207.
Becker, B. A., Craig, M., Denny, P., Keuning, H., Kiesler, N., Leinonen, J., Luxton-Reilly, A., Prather, J., and Quille, K. (2023). Generative ai in introductory programming. Computer Science Curricula, pages 438–439.
Branco, B. C., Neto, A. M., Avelino, G., and Machado, V. (2025). Avaliando o papel do gpt-4o na aprendizagem de engenharia de software: Um relato de experiência. In Anais do XXXIX Simpósio Brasileiro de Engenharia de Software, pages 470–481, Porto Alegre, RS, Brasil. SBC.
Chan, C. K. Y. and Hu, W. (2023). Students' voices on generative ai: Perceptions, benefits, and challenges in higher education. International journal of educational technology in higher education, 20(1):43.
De Cremer, D., Bianzino, N. M., and Falk, B. (2023). How generative ai could disrupt creative work. Harvard Business Review, 13:13.
Dibia, V., Fourney, A., Bansal, G., Poursabzi-Sangdeh, F., Liu, H., and Amershi, S. (2023). Aligning offline metrics and human judgments of value for code generation models. In Findings of the Association for Computational Linguistics: ACL 2023, pages 8516–8528.
Favero, L., Pérez-Ortiz, J.-A., Käser, T., and Oliver, N. (2025). Do ai tutors empower or enslave learners? toward a critical use of ai in education. arXiv preprint arXiv:2507.06878.
Gerlich, M. (2025). Ai tools in society: Impacts on cognitive offloading and the future of critical thinking. Societies, 15(1):6.
Guerino, G., Rodrigues, L., Capeleti, B., Mello, R., Freire, A., and Zaina, L. A. M. (2025). Can gpt-4o evaluate usability like human experts? a comparative study on issue identification in heuristic evaluation. pages 381–402.
Hong, H., Vate-U-Lan, P., and Viriyavejakul, C. (2025). Cognitive offload instruction with generative ai: A quasi-experimental study on critical thinking gains in english writing. In Forum for Linguistic Studies, volume 7, pages 325–334.
Lazar, J., Feng, J. H., and Hochheiser, H. (2017). Research methods in human-computer interaction. Morgan Kaufmann.
Mittal, U., Sai, S., Chamola, V., and Sangwan, D. (2024). A comprehensive review on generative ai for education. Ieee Access, 12:142733–142759.
Natella, R., Winter, S., Cotroneo, D., and Suri, N. (2018). Analyzing the effects of bugs on software interfaces. IEEE Transactions on Software Engineering, 46(3):280–301.
Nielsen, J. (1994a). Enhancing the explanatory power of usability heuristics. In Proceedings of the SIGCHI conference on Human Factors in Computing Systems, pages 152–158.
Nielsen, J. (1994b). Usability inspection methods. In Conference companion on Human factors in computing systems, pages 413–414.
Ravšelj, D., Keržič, D., Tomaževič, N., Umek, L., Brezovar, N., Iahad, N. A., Abdulla, A. A., Akopyan, A., Segura, M. W. A., AlHumaid, J., et al. (2025). Higher education students' perceptions of chatgpt: A global study of early reactions. PLoS One, 20(2):e0315011.
Rocha, H. and Baranauskas, M. C. C. (2003). Design e avaliação de interfaces humano-computador. Campinas: Unicamp.
Sajja, R., Sermet, Y., Fodale, B., and Demir, I. (2025). Evaluating ai-powered learning assistants in engineering higher education: Student engagement, ethical challenges, and policy implications. ArXiv, abs/2506.05699.
Silvestre, A. S. S., de Moura Amaral, E. L., Holanda, M. E., and Canedo, E. D. (2023). Students' perception about chatgpt's impact on their academic education. In Simpósio Brasileiro de Informática na Educação (SBIE), pages 1260–1270. SBC.
Sobrino-Duque, R., Martínez-Rojo, N., Carrillo-de Gea, J. M., López-Jiménez, J. J., Nicolás, J., and Fernández-Alemán, J. L. (2022). Evaluating a gamification proposal for learning usability heuristics: Heureka. International Journal of Human-Computer Studies, 161:102774.
Sousa, H. and Cruz, D. (2024). Capacitando educadores com ia generativa: Implicações na educação. In Anais do XXXV Simpósio Brasileiro de Informática na Educação, pages 1931–1941, Porto Alegre, RS, Brasil. SBC.
Triberti, S., Di Fuccio, R., Scuotto, C., Marsico, E., and Limone, P. (2024). "better than my professor?" how to develop artificial intelligence tools for higher education. Frontiers in Artificial Intelligence, 7:1329605.
Wang, S., Xu, T., Li, H., Zhang, C., Liang, J., Tang, J., Yu, P. S., and Wen, Q. (2025). Large language models for education: A survey and outlook. IEEE Signal Processing Magazine, 42(6):51–63.
Wohlin, C., Runeson, P., Höst, M., Ohlsson, M. C., Regnell, B., and Wesslén, A. (2012). Experimentation in Software Engineering. Springer, Berlin, Germany.
Xia, Q., Weng, X., Fan, O., Lin, T.-J., and Chiu, T. (2024). A scoping review on how generative artificial intelligence transforms assessment in higher education. International Journal of Educational Technology in Higher Education, 21:1–22.
Yan, L., Sha, L., Zhao, L., Li, Y., Martinez-Maldonado, R., Chen, G., Li, X., Jin, Y., and Gašević, D. (2024). Practical and ethical challenges of large language models in education: A systematic scoping review. British Journal of Educational Technology, 55(1):90–112.
Zhan, Z., Wu, Q., Lin, Z., and Cai, J. (2021). Smart classroom environments affect teacher-student interaction: Evidence from a behavioural sequence analysis. Australasian Journal of Educational Technology, 37(2):96–109.
Zhong, R., McDonald, D. W., and Hsieh, G. (2025). Synthetic heuristic evaluation: A comparison between ai- and human-powered usability evaluation. ArXiv, abs/2507.02306.
Bahroun, Z., Anane, C., Ahmed, V., and Zacca, A. (2023). Transforming education: A comprehensive review of generative artificial intelligence in educational settings through bibliometric and content analysis. Sustainability.
Baidoo-Anu, D. and Ansah, L. O. (2023). Education in the era of generative artificial intelligence (ai): Understanding the potential benefits of chatgpt in promoting teaching and learning. SSRN Electronic Journal.
Barbosa, S. D. J., Silva, B. d., Silveira, M. S., Gasparini, I., Darin, T., and Barbosa, G. D. J. (2021). Interação humano-computador e experiência do usuario. Auto publicação.
Barke, S., James, M. B., and Polikarpova, N. (2023). Grounded copilot: How programmers interact with code-generating models. Proceedings of the ACM on Programming Languages, 7(OOPSLA1):85–111.
Bauschard, S. and Quidwai, S. (2024). From insight to implementation: How to create your ai school guidance. Available at SSRN 4784207.
Becker, B. A., Craig, M., Denny, P., Keuning, H., Kiesler, N., Leinonen, J., Luxton-Reilly, A., Prather, J., and Quille, K. (2023). Generative ai in introductory programming. Computer Science Curricula, pages 438–439.
Branco, B. C., Neto, A. M., Avelino, G., and Machado, V. (2025). Avaliando o papel do gpt-4o na aprendizagem de engenharia de software: Um relato de experiência. In Anais do XXXIX Simpósio Brasileiro de Engenharia de Software, pages 470–481, Porto Alegre, RS, Brasil. SBC.
Chan, C. K. Y. and Hu, W. (2023). Students' voices on generative ai: Perceptions, benefits, and challenges in higher education. International journal of educational technology in higher education, 20(1):43.
De Cremer, D., Bianzino, N. M., and Falk, B. (2023). How generative ai could disrupt creative work. Harvard Business Review, 13:13.
Dibia, V., Fourney, A., Bansal, G., Poursabzi-Sangdeh, F., Liu, H., and Amershi, S. (2023). Aligning offline metrics and human judgments of value for code generation models. In Findings of the Association for Computational Linguistics: ACL 2023, pages 8516–8528.
Favero, L., Pérez-Ortiz, J.-A., Käser, T., and Oliver, N. (2025). Do ai tutors empower or enslave learners? toward a critical use of ai in education. arXiv preprint arXiv:2507.06878.
Gerlich, M. (2025). Ai tools in society: Impacts on cognitive offloading and the future of critical thinking. Societies, 15(1):6.
Guerino, G., Rodrigues, L., Capeleti, B., Mello, R., Freire, A., and Zaina, L. A. M. (2025). Can gpt-4o evaluate usability like human experts? a comparative study on issue identification in heuristic evaluation. pages 381–402.
Hong, H., Vate-U-Lan, P., and Viriyavejakul, C. (2025). Cognitive offload instruction with generative ai: A quasi-experimental study on critical thinking gains in english writing. In Forum for Linguistic Studies, volume 7, pages 325–334.
Lazar, J., Feng, J. H., and Hochheiser, H. (2017). Research methods in human-computer interaction. Morgan Kaufmann.
Mittal, U., Sai, S., Chamola, V., and Sangwan, D. (2024). A comprehensive review on generative ai for education. Ieee Access, 12:142733–142759.
Natella, R., Winter, S., Cotroneo, D., and Suri, N. (2018). Analyzing the effects of bugs on software interfaces. IEEE Transactions on Software Engineering, 46(3):280–301.
Nielsen, J. (1994a). Enhancing the explanatory power of usability heuristics. In Proceedings of the SIGCHI conference on Human Factors in Computing Systems, pages 152–158.
Nielsen, J. (1994b). Usability inspection methods. In Conference companion on Human factors in computing systems, pages 413–414.
Ravšelj, D., Keržič, D., Tomaževič, N., Umek, L., Brezovar, N., Iahad, N. A., Abdulla, A. A., Akopyan, A., Segura, M. W. A., AlHumaid, J., et al. (2025). Higher education students' perceptions of chatgpt: A global study of early reactions. PLoS One, 20(2):e0315011.
Rocha, H. and Baranauskas, M. C. C. (2003). Design e avaliação de interfaces humano-computador. Campinas: Unicamp.
Sajja, R., Sermet, Y., Fodale, B., and Demir, I. (2025). Evaluating ai-powered learning assistants in engineering higher education: Student engagement, ethical challenges, and policy implications. ArXiv, abs/2506.05699.
Silvestre, A. S. S., de Moura Amaral, E. L., Holanda, M. E., and Canedo, E. D. (2023). Students' perception about chatgpt's impact on their academic education. In Simpósio Brasileiro de Informática na Educação (SBIE), pages 1260–1270. SBC.
Sobrino-Duque, R., Martínez-Rojo, N., Carrillo-de Gea, J. M., López-Jiménez, J. J., Nicolás, J., and Fernández-Alemán, J. L. (2022). Evaluating a gamification proposal for learning usability heuristics: Heureka. International Journal of Human-Computer Studies, 161:102774.
Sousa, H. and Cruz, D. (2024). Capacitando educadores com ia generativa: Implicações na educação. In Anais do XXXV Simpósio Brasileiro de Informática na Educação, pages 1931–1941, Porto Alegre, RS, Brasil. SBC.
Triberti, S., Di Fuccio, R., Scuotto, C., Marsico, E., and Limone, P. (2024). "better than my professor?" how to develop artificial intelligence tools for higher education. Frontiers in Artificial Intelligence, 7:1329605.
Wang, S., Xu, T., Li, H., Zhang, C., Liang, J., Tang, J., Yu, P. S., and Wen, Q. (2025). Large language models for education: A survey and outlook. IEEE Signal Processing Magazine, 42(6):51–63.
Wohlin, C., Runeson, P., Höst, M., Ohlsson, M. C., Regnell, B., and Wesslén, A. (2012). Experimentation in Software Engineering. Springer, Berlin, Germany.
Xia, Q., Weng, X., Fan, O., Lin, T.-J., and Chiu, T. (2024). A scoping review on how generative artificial intelligence transforms assessment in higher education. International Journal of Educational Technology in Higher Education, 21:1–22.
Yan, L., Sha, L., Zhao, L., Li, Y., Martinez-Maldonado, R., Chen, G., Li, X., Jin, Y., and Gašević, D. (2024). Practical and ethical challenges of large language models in education: A systematic scoping review. British Journal of Educational Technology, 55(1):90–112.
Zhan, Z., Wu, Q., Lin, Z., and Cai, J. (2021). Smart classroom environments affect teacher-student interaction: Evidence from a behavioural sequence analysis. Australasian Journal of Educational Technology, 37(2):96–109.
Zhong, R., McDonald, D. W., and Hsieh, G. (2025). Synthetic heuristic evaluation: A comparison between ai- and human-powered usability evaluation. ArXiv, abs/2507.02306.
Publicado
05/10/2026
Como Citar
MATTHAUS, Fabiann; RODRIGUES, Luana; LIMA, Matheus; CONTE, Tayana.
Aprender ou Delegar? Um Estudo sobre Pensamento Crítico no Uso de IA Generativa em Avaliação Heurística. In: SIMPÓSIO BRASILEIRO DE INFORMÁTICA NA EDUCAÇÃO (SBIE), 37. , 2026, Goiânia/GO.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2026
.
p. 1077-1091.
DOI: https://doi.org/10.5753/sbie.2026.27476.
