Automatic Distractor Generation for Multiple-Choice Items in Word Reading Assessments

• João Augusto Pilato de Castro Universidade Federal de Juiz de Fora (UFJF)
• João Marcelo Amaral Universidade Federal de Juiz de Fora (UFJF)
• Lucas Larcher Universidade Federal de Juiz de Fora (UFJF)
• Lara Dias Universidade Federal de Juiz de Fora (UFJF)
• João Vítor de C. M. F. Nogueira Universidade Federal de Juiz de Fora (UFJF)
• Rosângela Veiga Universidade Federal de Juiz de Fora (UFJF)
• Begma Tavares Universidade Federal de Juiz de Fora (UFJF)
• Jairo Francisco de Souza Universidade Federal de Juiz de Fora (UFJF)

DOI: https://doi.org/10.5753/sbie.2026.27201

Resumo

Creating educational assessment items requires selecting plausible distractors that meet constraints on syllable count, syllabic pattern, and canonicity. This is particularly challenging in word-reading items, whose distractor quality depends on fine phonological and graphemic relations between words, and pedagogical criteria. This work proposes a rule-based linguistic approach for Brazilian Portuguese to support the development of word-reading items. The proposed solution uses a structured word database containing phonological, graphemic, and syllabic information and applies filtering and similarity-based ranking steps relative to the correct answer. According to expert analysis of literacy item creation, our approach outperforms the LLM-based approach.

Palavras-chave: Automatic Item Generation, Distractors, Word Reading Assessment

Referências

Alhazmi, E., Sheng, Q. Z., Zhang, W. E., Zaib, M., and Alhazmi, A. (2024). Distractor generation in multiple-choice tasks: A survey of methods, datasets, and evaluation. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics.

Awalurahman, R. and Budi, I. (2024). Automatic distractor generation in multiple-choice questions: a systematic literature review. PeerJ Computer Science, 10:e2441.

Bommasani, R., Hudson, D. A., Adeli, E., Altman, R., Arora, S., von Arx, S., Bernstein, M. S., Bohg, J., Bosselut, A., Brunskill, E., et al. (2021). On the opportunities and risks of foundation models. arXiv preprint arXiv:2108.07258.

Brasil. Ministério da Educação (2018). Base Nacional Comum Curricular. MEC, Brasília. Acesso em: 7 abr. 2026.

Chan, K. W., Ali, F., Park, J., Sham, K. S. B., Tan, E. Y. T., Chong, F. W. C., Qian, K., and Sze, G. K. (2025). Automatic item generation in various stem subjects using large language model prompting. Computers and Education: Artificial Intelligence, 8:100344.

Chico, V., Regino, A., Bonacin, R., and Reis, J. (2025). Llm-based automatic generation of multiple-choice questions with meaningful distractors. In Anais do XXXVI Simpósio Brasileiro de Informática na Educação, pages 814–828, Porto Alegre, RS, Brasil. SBC.

Clarke, M. and Luna-Bazaldua, D. (2021). Primer on large-scale assessments of educational achievement. World Bank Publications.

Dubeck, M. M. and Gove, A. (2015). The early grade reading assessment (egra): Its theoretical foundation, purpose, and limitations. International Journal of Educational Development, 40:315–322.

Feng, W., Lee, J., McNichols, H., Scarlatos, A., Smith, D., Woodhead, S., Ornelas, N., and Lan, A. (2024). Exploring automated distractor generation for math multiple-choice questions via large language models. In Findings of the Association for Computational Linguistics: NAACL 2024, pages 3067–3082.

Gierl, M. J. and Haladyna, T. M. (2012). Automatic item generation: Theory and practice. Routledge.

Gonçalo Oliveira, H., Caetano, I., Matos, R., and Amaro, H. (2023). Generating and ranking distractors for multiple-choice questions in portuguese. In 12th Symposium on Languages, Applications and Technologies (SLATE 2023), pages 4–1. Schloss Dagstuhl–Leibniz-Zentrum für Informatik.

Griffin, P., Burns, M. S., and Snow, C. E. (1998). Preventing reading difficulties in young children. National Academies Press.

Haladyna, T. (2004). Developing and Validating Multiple-choice Test Items. Lawrence Erlbaum Associates.

Kıyak, Y. S. and Kononowicz, A. A. (2025). Using a hybrid of ai and template-based method in automatic item generation to create multiple-choice questions in medical education: Hybrid aig. JMIR Formative Research, 9:e65726.

Lai, H., Gierl, M. J., Touchie, C., Pugh, D., Boulais, A.-P., and De Champlain, A. (2016). Using automatic item generation to improve the quality of mcq distractors. Teaching and Learning in Medicine, 28(2):166–173.

Lietz, P. and Tobin, M. (2016). The impact of large-scale assessments in education on education policy: Evidence from around the world.

Mello, R., Silva, G., Carvalho, P., Lopes, R., Carneiro, M., and Souza, J. (2025). An evolutionary approach for the automatic generation of word list fluency assessment items. In Anais do XXXVI Simpósio Brasileiro de Informática na Educação, pages 365–378, Porto Alegre, RS, Brasil. SBC.

Nogueira, J., Castro, J., Larcher, L., Ferreira, R., Barbosa, B., and Souza, J. (2025). Uma análise de qualidade do uso de grandes modelos de linguagem para geração automática de itens avaliativos em português. In Anais do XXXVI Simpósio Brasileiro de Informática na Educação, pages 235–248, Porto Alegre, RS, Brasil. SBC.

Odden, D. (2005). Introducing phonology. Cambridge university press.

Oka, H., Tan, Y., Ishioka, T., and Okada, K. (2025). Systematic control of multiple-choice item difficulty through llm-based distractor generation. In International Conference on Artificial Intelligence in Education, pages 147–157. Springer.

Rodriguez-Torrealba, R., Garcia-Lopez, E., and Garcia-Cabot, A. (2022). End-to-end generation of multiple-choice questions using text-to-text transfer transformer models. Expert Systems with Applications, 208:118258.

Song, Y., Du, J., and Zheng, Q. (2025). Automatic item generation for educational assessments: a systematic literature review. Interactive Learning Environments, 33(9):5386–5405.

Tan, B., Armoush, N., Mazzullo, E., Bulut, O., and Gierl, M. (2025). A review of automatic item generation techniques leveraging large language models. International Journal of Assessment Tools in Education, 12(2):317–340.

Tian, Y., Huynh, L., Christhilf, K., Chakraborty, S., Watanabe, M., Arner, T., and McNamara, D. (2026). Cognitively diverse multiple-choice question generation: A hybrid multi-agent framework with large language models. Electronics, 15(6).