Geração de Questões com LLMs Leves: Um Estudo Inicial sobre a Percepção de Educadores
Resumo
LLMs podem auxiliar na geração automática de questões, mas exigem internet ou infraestrutura robusta, um obstáculo em contextos com poucos recursos. LLMs leves (lightweight), que operam offline em smartphones comuns, surgem como alternativa, embora ainda faltem evidências sobre a qualidade dos conteúdos gerados. Este trabalho investiga o uso desses modelos para gerar questões de matemática. Foram produzidas 59 questões e avaliadas por uma pedagoga, com resultados mistos. A maioria mostrou-se utilizável, sobretudo em operações de soma e subtração, mas com limitações conceituais, gramaticais e semânticas. O estudo destaca o potencial dos LLMs leves e a necessidade de revisão humana para garantir qualidade pedagógica.
Referências
Creswell, J. W. and Creswell, J. D. (2017). Research design: Qualitative, quantitative, and mixed methods approaches. Sage publications.
Freitas, E., Batista, H. H., Barbosa, G. A., Wenceslau, M., Portela, C., Isotani, S., and Mello, R. F. (2022). Learning analytics desconectada: Um estudo de caso em análise de produções textuais. In Anais do I Workshop de Aplicações Práticas de Learning Analytics em Instituições de Ensino no Brasil, pages 40–49. SBC.
Gasevic, D., Paul, P., Chen, B., Fan, Y., Rodrigo, M., Cobo, C., and Cecilia, A. (2018). Learning analytics for the global south. In C.P. Lim & V.L. Tinio (Eds.), Foundation for Information Technology Education and Development. Foundation for Information Technology Education and Development, Quezon City, Philippines.
Gerganov, G. (2023). llama.cpp.
Graesser, A. C., VanLehn, K., and Rosé, C. P. e. a. (2004). Intelligent tutoring systems with conversational dialogue. AI Magazine, 22(4):39–51.
Guerino, G., Rodrigues, L., Oliveira, L., Marinho, M., Silva, T., Amorim, L., Dermeval, D., da Penha, R., Bittencourt, I., and Isotani, S. (2024). We see you: Understanding math teachers from brazilian public schools to design equitable educational technology. Revista Brasileira de Informática na Educação, 32:336–358.
Hillmayr, D., Ziernwald, L., Reinhold, F., Hofer, S. I., and Reiss, K. M. (2020). The potential of digital tools to enhance mathematics and science learning in secondary schools: A context-specific meta-analysis. Computers & Education, 153:103897.
Indran, I. R., Paranthaman, P., Gupta, N., and Mustafa, N. (2024). Twelve tips to leverage ai for efficient and effective medical question generation: a guide for educators using chat gpt. Medical Teacher, 46(8):1021–1026.
Instituto Nacional de Estudos e Pesquisas Educacionais Anísio Teixeira (Inep) (2022). Sistema de avaliação da educação básica 2019: Relatório de desempenho. Technical report, Inep, Brasília, DF.
Isotani, S., Bittencourt, I. I., and Challco, G. C. e. a. (2023). Aied unplugged: Leapfrogging the digital divide to reach the underserved. In International Conference on Artificial Intelligence in Education, pages 772–779. Springer.
Lee, U., Jung, H., Jeon, Y., Sohn, Y., Hwang, W., Moon, J., and Kim, H. (2024). Few-shot is enough: exploring chatgpt prompt engineering method for automatic question generation in english education. Education and Information Technologies, 29(9):11483–11515.
Li, K. and Zhang, Y. (2024). Planning first, question second: An LLM-guided method for controllable question generation. In Ku, L.-W., Martins, A., and Srikumar, V., editors, Findings of the Association for Computational Linguistics: ACL 2024, pages 4715–4729, Bangkok, Thailand. Association for Computational Linguistics.
Monteiro Santos, M., Barros, A., and Rodrigues, L. e. a. (2025). Near feasibility, distant practicality: Empirical analysis of deploying and using llms on resource-constrained smartphones. In Proceedings of the 13th International Conference on Information & Communication Technologies and Development, ICTD ’24, page 224–235, New York, NY, USA. Association for Computing Machinery.
Nikolovski, V., Trajanov, D., and Chorbev, I. (2025). Advancing ai in higher education: A comparative study of large language model-based agents for exam question generation, improvement, and evaluation. Algorithms, 18(3).
OECD (2023). Pisa 2022 results (volume i).
Portela, C., Lisbôa, R., Yasojima, K., Cordeiro, T., Silva, A., Dermeval, D., and Isotani, S. (2023). A case study on aied unplugged applied to public policy for learning recovery post-pandemic in brazil. In International Conference on Artificial Intelligence in Education, pages 788–796. Springer Nature Switzerland.
Rodrigues, L., Guerino, G., Challco, G. C., Veloso, T. E., Oliveira, L., da Penha, R. S., Melo, R. F., Vieira, T., Marinho, M., Macario, V., et al. (2023). Teacher-centered intelligent tutoring systems: Design considerations from brazilian, public school teachers. In Simpósio Brasileiro de Informática na Educação (SBIE), pages 1419–1430. SBC.
Rodrigues, L., Guerino, G., Silva, T. E., Challco, G. C., Oliveira, L., da Penha, R. S., and Isotani, S. (2024a). Mathaide: A qualitative study of teachers’ perceptions of an its unplugged for underserved regions. International Journal of Artificial Intelligence in Education, pages 1–29.
Rodrigues, L., Guerino, G., Veloso, T. E., Bianchini, L., Xavier, M., Vieira, T., Marinho, M., and Macario, V. (2024b). Mathaide in the classroom: A qualitative analysis of teachers’ perspectives of intelligent tutoring systems unplugged. In Proceedings of the Brazilian Symposium on Computers in Education (SBIE). to appear.
Scaria, N., Dharani Chenna, S., and Subramani, D. (2024). Automated educational question generation at different bloom’s skill levels using large language models: Strategies and evaluation. In Olney, A. M., Chounta, I.-A., Liu, Z., Santos, O. C., and Bittencourt, I. I., editors, Artificial Intelligence in Education, pages 165–179, Cham. Springer Nature Switzerland.
Sewunetie, W. and Kovács, L. (2023). Chatgpt in education: Opportunities and concerns for functional english sentence structure analysis for automatic question generation. In International Conference on Advances of Science and Technology, pages 79–90. Springer.
Soofi, A. A. and Ahmed, M. U. (2019). A systematic review of domains, techniques, delivery modes and validation methods for intelligent tutoring systems. International Journal of Advanced Computer Science and Applications, 10(3).
StatCounter (2024). Mobile vendor market share worldwide. Accessed: 2024-02-14.
Team, G. (2024). Gemma.
Tenório, K., Dermeval, D., Monteiro, M., Peixoto, A., and Silva, A. P. d. (2022). Exploring design concepts to enable teachers to monitor and adapt gamification in adaptive learning systems: a qualitative research approach. International Journal of Artificial Intelligence in Education, 32(4):867–891.
Vanlehn, K. (2006). The behavior of tutoring systems. Int. J. Artif. Intell. Ed., 16(3):227–265.
Veloso, T. E., Chalco Challco, G., Rogrigues, L., Versuti, F. M., Sena da Penha, R., Silva Oliveira, L., and Isotani, S. (2023). Its unplugged: Leapfrogging the digital divide for teaching numeracy skills in underserved populations. In Workshop on International Conference of Artificial Intelligence in Education co-located with The 24th International Conference on Artificial Intelligence in Education.
Venkatesh, V. and Davis, F. D. (2000). A theoretical extension of the technology acceptance model: Four longitudinal field studies. Management science, 46(2):186–204.
Xia, Q., Chiu, T. K., Zhou, X., Chai, C. S., and Cheng, M. (2022). Systematic literature review on opportunities, challenges, and future research recommendations of artificial intelligence in education. Computers and Education: Artificial Intelligence, page 100118.
Xin, C. (2023). Testing llm-based applications: Strategy and challenges. [link].
Xu, Z., Wijekumar, K., Ramirez, G., Hu, X., and Irey, R. (2019). The effectiveness of intelligent tutoring systems on k-12 students’ reading comprehension: A meta-analysis. British Journal of Educational Technology, 50(6):3119–3137.
Freitas, E., Batista, H. H., Barbosa, G. A., Wenceslau, M., Portela, C., Isotani, S., and Mello, R. F. (2022). Learning analytics desconectada: Um estudo de caso em análise de produções textuais. In Anais do I Workshop de Aplicações Práticas de Learning Analytics em Instituições de Ensino no Brasil, pages 40–49. SBC.
Gasevic, D., Paul, P., Chen, B., Fan, Y., Rodrigo, M., Cobo, C., and Cecilia, A. (2018). Learning analytics for the global south. In C.P. Lim & V.L. Tinio (Eds.), Foundation for Information Technology Education and Development. Foundation for Information Technology Education and Development, Quezon City, Philippines.
Gerganov, G. (2023). llama.cpp.
Graesser, A. C., VanLehn, K., and Rosé, C. P. e. a. (2004). Intelligent tutoring systems with conversational dialogue. AI Magazine, 22(4):39–51.
Guerino, G., Rodrigues, L., Oliveira, L., Marinho, M., Silva, T., Amorim, L., Dermeval, D., da Penha, R., Bittencourt, I., and Isotani, S. (2024). We see you: Understanding math teachers from brazilian public schools to design equitable educational technology. Revista Brasileira de Informática na Educação, 32:336–358.
Hillmayr, D., Ziernwald, L., Reinhold, F., Hofer, S. I., and Reiss, K. M. (2020). The potential of digital tools to enhance mathematics and science learning in secondary schools: A context-specific meta-analysis. Computers & Education, 153:103897.
Indran, I. R., Paranthaman, P., Gupta, N., and Mustafa, N. (2024). Twelve tips to leverage ai for efficient and effective medical question generation: a guide for educators using chat gpt. Medical Teacher, 46(8):1021–1026.
Instituto Nacional de Estudos e Pesquisas Educacionais Anísio Teixeira (Inep) (2022). Sistema de avaliação da educação básica 2019: Relatório de desempenho. Technical report, Inep, Brasília, DF.
Isotani, S., Bittencourt, I. I., and Challco, G. C. e. a. (2023). Aied unplugged: Leapfrogging the digital divide to reach the underserved. In International Conference on Artificial Intelligence in Education, pages 772–779. Springer.
Lee, U., Jung, H., Jeon, Y., Sohn, Y., Hwang, W., Moon, J., and Kim, H. (2024). Few-shot is enough: exploring chatgpt prompt engineering method for automatic question generation in english education. Education and Information Technologies, 29(9):11483–11515.
Li, K. and Zhang, Y. (2024). Planning first, question second: An LLM-guided method for controllable question generation. In Ku, L.-W., Martins, A., and Srikumar, V., editors, Findings of the Association for Computational Linguistics: ACL 2024, pages 4715–4729, Bangkok, Thailand. Association for Computational Linguistics.
Monteiro Santos, M., Barros, A., and Rodrigues, L. e. a. (2025). Near feasibility, distant practicality: Empirical analysis of deploying and using llms on resource-constrained smartphones. In Proceedings of the 13th International Conference on Information & Communication Technologies and Development, ICTD ’24, page 224–235, New York, NY, USA. Association for Computing Machinery.
Nikolovski, V., Trajanov, D., and Chorbev, I. (2025). Advancing ai in higher education: A comparative study of large language model-based agents for exam question generation, improvement, and evaluation. Algorithms, 18(3).
OECD (2023). Pisa 2022 results (volume i).
Portela, C., Lisbôa, R., Yasojima, K., Cordeiro, T., Silva, A., Dermeval, D., and Isotani, S. (2023). A case study on aied unplugged applied to public policy for learning recovery post-pandemic in brazil. In International Conference on Artificial Intelligence in Education, pages 788–796. Springer Nature Switzerland.
Rodrigues, L., Guerino, G., Challco, G. C., Veloso, T. E., Oliveira, L., da Penha, R. S., Melo, R. F., Vieira, T., Marinho, M., Macario, V., et al. (2023). Teacher-centered intelligent tutoring systems: Design considerations from brazilian, public school teachers. In Simpósio Brasileiro de Informática na Educação (SBIE), pages 1419–1430. SBC.
Rodrigues, L., Guerino, G., Silva, T. E., Challco, G. C., Oliveira, L., da Penha, R. S., and Isotani, S. (2024a). Mathaide: A qualitative study of teachers’ perceptions of an its unplugged for underserved regions. International Journal of Artificial Intelligence in Education, pages 1–29.
Rodrigues, L., Guerino, G., Veloso, T. E., Bianchini, L., Xavier, M., Vieira, T., Marinho, M., and Macario, V. (2024b). Mathaide in the classroom: A qualitative analysis of teachers’ perspectives of intelligent tutoring systems unplugged. In Proceedings of the Brazilian Symposium on Computers in Education (SBIE). to appear.
Scaria, N., Dharani Chenna, S., and Subramani, D. (2024). Automated educational question generation at different bloom’s skill levels using large language models: Strategies and evaluation. In Olney, A. M., Chounta, I.-A., Liu, Z., Santos, O. C., and Bittencourt, I. I., editors, Artificial Intelligence in Education, pages 165–179, Cham. Springer Nature Switzerland.
Sewunetie, W. and Kovács, L. (2023). Chatgpt in education: Opportunities and concerns for functional english sentence structure analysis for automatic question generation. In International Conference on Advances of Science and Technology, pages 79–90. Springer.
Soofi, A. A. and Ahmed, M. U. (2019). A systematic review of domains, techniques, delivery modes and validation methods for intelligent tutoring systems. International Journal of Advanced Computer Science and Applications, 10(3).
StatCounter (2024). Mobile vendor market share worldwide. Accessed: 2024-02-14.
Team, G. (2024). Gemma.
Tenório, K., Dermeval, D., Monteiro, M., Peixoto, A., and Silva, A. P. d. (2022). Exploring design concepts to enable teachers to monitor and adapt gamification in adaptive learning systems: a qualitative research approach. International Journal of Artificial Intelligence in Education, 32(4):867–891.
Vanlehn, K. (2006). The behavior of tutoring systems. Int. J. Artif. Intell. Ed., 16(3):227–265.
Veloso, T. E., Chalco Challco, G., Rogrigues, L., Versuti, F. M., Sena da Penha, R., Silva Oliveira, L., and Isotani, S. (2023). Its unplugged: Leapfrogging the digital divide for teaching numeracy skills in underserved populations. In Workshop on International Conference of Artificial Intelligence in Education co-located with The 24th International Conference on Artificial Intelligence in Education.
Venkatesh, V. and Davis, F. D. (2000). A theoretical extension of the technology acceptance model: Four longitudinal field studies. Management science, 46(2):186–204.
Xia, Q., Chiu, T. K., Zhou, X., Chai, C. S., and Cheng, M. (2022). Systematic literature review on opportunities, challenges, and future research recommendations of artificial intelligence in education. Computers and Education: Artificial Intelligence, page 100118.
Xin, C. (2023). Testing llm-based applications: Strategy and challenges. [link].
Xu, Z., Wijekumar, K., Ramirez, G., Hu, X., and Irey, R. (2019). The effectiveness of intelligent tutoring systems on k-12 students’ reading comprehension: A meta-analysis. British Journal of Educational Technology, 50(6):3119–3137.
Publicado
24/11/2025
Como Citar
SANTOS, Mateus Monteiro et al.
Geração de Questões com LLMs Leves: Um Estudo Inicial sobre a Percepção de Educadores. In: SIMPÓSIO BRASILEIRO DE INFORMÁTICA NA EDUCAÇÃO (SBIE), 36. , 2025, Curitiba/PR.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2025
.
p. 1635-1646.
DOI: https://doi.org/10.5753/sbie.2025.12574.
