Aplicação de verbos como proxy para identificação automática do nível cognitivo de questões: uma abordagem baseada na taxonomia de Bloom
Resumo
A utilização de questões fornece um importante mecanismo para promover o desenvolvimento da aprendizagem. Por sua vez, itens educacionais podem ser caracterizados através de métodos como os níveis cognitivos da taxonomia de Bloom e conjuntos de verbos associados a cada um deles. Diante disto, este trabalho propõe uma nova abordagem para a classificação automática de questões. Isso foi feito através da construção de classificadores treinados com features construídas com léxicos baseados nos verbos de ação da taxonomia, em contraste com as soluções existentes. Foi demonstrado a viabilidade desta solução, com os resultados indicando um F1 médio de 0,51 e 0,55 para as diferentes versões de léxicos produzidas.
Palavras-chave:
sistemas de apoio à educação, taxonomia de Bloom, classificação de questões, extração de features, processamento de linguagem natural, aprendizado de máquina
Referências
Bloom, B. S., Engelhart, M. B., Furst, E. J., Hill, W. H., and Krathwohl, D. R. (1956). Taxonomy of educational objectives. The classification of educational goals. Handbook 1: Cognitive domain. Longmans Green, New York.
Caruana, R. and Niculescu-Mizil, A. (2006). An empirical comparison of supervised learning algorithms. In Proceedings of the 23rd international conference on Machine learning, pages 161–168.
Chi, M. T., De Leeuw, N., Chiu, M.-H., and LaVancher, C. (1994). Eliciting self-explanations improves understanding. Cognitive science, 18(3):439–477.
Diab, S. and Sartawi, B. (2017). Classification of questions and learning outcome statements (LOS) into blooms taxonomy (BT) by similarity measurements towards extracting of learning outcome from learning material. CoRR, abs/1706.03191.
Fast, E., Chen, B., and Bernstein, M. S. (2016). Empath: Understanding topic signals in large-scale text. In Proceedings of the 2016 CHI conference on human factors in computing systems, pages 4647–4657.
Galhardi, A. C. and Azevedo, M. M. d. (2013). Avaliações de aprendizagem: o uso da taxonomia de bloom. In Anais do VII Workshop Pos-graduação e Pesquisa do Centro Paula Souza, São Paulo, volume 1, pages 237–247.
Jayakodi, K., Bandara, M., and Perera, I. (2015). An automatic classifier for exam questions in engineering: A process for bloom’s taxonomy. In 2015 IEEE International Conference on Teaching, Assessment, and Learning for Engineering (TALE), pages 195–202. IEEE.
Kusner, M., Sun, Y., Kolkin, N., and Weinberger, K. (2015). From word embeddings to document distances. In International conference on machine learning, pages 957–966. PMLR.
Kusuma, S. F., Siahaan, D., and Yuhana, U. L. (2015). Automatic indonesia’s questions classification based on bloom’s taxonomy using natural language processing a preliminary study. In 2015 International Conference on Information Technology Systems and Innovation (ICITSI), pages 1–6. IEEE.
Lane, H. C. and VanLehn, K. (2005). Teaching the tacit knowledge of programming to noviceswith natural language tutoring. Computer Science Education, 15(3):183–201.
Li, X. and Roth, D. (2002). Learning question classifiers. In COLING 2002: The 19th International Conference on Computational Linguistics.
LW, A., DR, K., PW, A., KA, C., Mayer, R., PR, P., Raths, J., and MC, W. (2001). A Taxonomy for Learning, Teaching, and Assessing: A Revision of Bloom’s Taxonomy of Educational Objectives.
Mikolov, T., Chen, K., Corrado, G. S., and Dean, J. A. (2015). Computing numeric representations of words in a high-dimensional space. US Patent 9,037,464.
Mohammed, M. and Omar, N. (2020). Question classification based on bloom’s taxonomy cognitive domain using modified tf-idf and word2vec. PloS one, 15(3):e0230442.
Omar, N., Haris, S. S., Hassan, R., Arshad, H., Rahmat, M., Zainal, N. F. A., and Zulkifli, R. (2012). Automated analysis of exam questions according to bloom’s taxonomy. Procedia-Social and Behavioral Sciences, 59:297–303.
Padó, U. (2017). Question difficulty–how to estimate without norming, how to use for automated grading. In Proceedings of the 12th workshop on innovative use of NLP for building educational applications, pages 1–10.
Stanny, C. J. (2016). Reevaluating bloom’s taxonomy: What measurable verbs can and cannot say about student learning. Education Sciences, 6(4):37.
Swart, A. J. (2009). Evaluation of final examination papers in engineering: A case study using bloom’s taxonomy. IEEE Transactions on Education, 53(2):257–264.
Tenenberg, J. and Murphy, L. (2005). Knowing what i know: An investigation of undergraduate knowledge and self-knowledge of data structures. Computer Science Education, 15(4):297–315.
Wei, J. and Zou, K. (2019). Eda: Easy data augmentation techniques for boosting performance on text classification tasks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 6383–6389.
Yahya, A. A. and Osman, A. (2011). Automatic classification of questions into bloom’s cognitive levels using support vector machines. In The International Arab Conference on Information Technology, Naif Arab University for Security Science (NAUSS), pages 1–6.
Yahya, A. A., Toukal, Z., and Osman, A. (2012). Bloom’s taxonomy–based classification for item bank questions using support vector machines. In Modern advances in intelligent systems and tools, pages 135–140. Springer.
Yu, F.-Y., Liu, Y.-H., and Chan, T.-W. (2005). A web-based learning system for question-posing and peer assessment. Innovations in education and teaching international, 42(4):337–348.
Caruana, R. and Niculescu-Mizil, A. (2006). An empirical comparison of supervised learning algorithms. In Proceedings of the 23rd international conference on Machine learning, pages 161–168.
Chi, M. T., De Leeuw, N., Chiu, M.-H., and LaVancher, C. (1994). Eliciting self-explanations improves understanding. Cognitive science, 18(3):439–477.
Diab, S. and Sartawi, B. (2017). Classification of questions and learning outcome statements (LOS) into blooms taxonomy (BT) by similarity measurements towards extracting of learning outcome from learning material. CoRR, abs/1706.03191.
Fast, E., Chen, B., and Bernstein, M. S. (2016). Empath: Understanding topic signals in large-scale text. In Proceedings of the 2016 CHI conference on human factors in computing systems, pages 4647–4657.
Galhardi, A. C. and Azevedo, M. M. d. (2013). Avaliações de aprendizagem: o uso da taxonomia de bloom. In Anais do VII Workshop Pos-graduação e Pesquisa do Centro Paula Souza, São Paulo, volume 1, pages 237–247.
Jayakodi, K., Bandara, M., and Perera, I. (2015). An automatic classifier for exam questions in engineering: A process for bloom’s taxonomy. In 2015 IEEE International Conference on Teaching, Assessment, and Learning for Engineering (TALE), pages 195–202. IEEE.
Kusner, M., Sun, Y., Kolkin, N., and Weinberger, K. (2015). From word embeddings to document distances. In International conference on machine learning, pages 957–966. PMLR.
Kusuma, S. F., Siahaan, D., and Yuhana, U. L. (2015). Automatic indonesia’s questions classification based on bloom’s taxonomy using natural language processing a preliminary study. In 2015 International Conference on Information Technology Systems and Innovation (ICITSI), pages 1–6. IEEE.
Lane, H. C. and VanLehn, K. (2005). Teaching the tacit knowledge of programming to noviceswith natural language tutoring. Computer Science Education, 15(3):183–201.
Li, X. and Roth, D. (2002). Learning question classifiers. In COLING 2002: The 19th International Conference on Computational Linguistics.
LW, A., DR, K., PW, A., KA, C., Mayer, R., PR, P., Raths, J., and MC, W. (2001). A Taxonomy for Learning, Teaching, and Assessing: A Revision of Bloom’s Taxonomy of Educational Objectives.
Mikolov, T., Chen, K., Corrado, G. S., and Dean, J. A. (2015). Computing numeric representations of words in a high-dimensional space. US Patent 9,037,464.
Mohammed, M. and Omar, N. (2020). Question classification based on bloom’s taxonomy cognitive domain using modified tf-idf and word2vec. PloS one, 15(3):e0230442.
Omar, N., Haris, S. S., Hassan, R., Arshad, H., Rahmat, M., Zainal, N. F. A., and Zulkifli, R. (2012). Automated analysis of exam questions according to bloom’s taxonomy. Procedia-Social and Behavioral Sciences, 59:297–303.
Padó, U. (2017). Question difficulty–how to estimate without norming, how to use for automated grading. In Proceedings of the 12th workshop on innovative use of NLP for building educational applications, pages 1–10.
Stanny, C. J. (2016). Reevaluating bloom’s taxonomy: What measurable verbs can and cannot say about student learning. Education Sciences, 6(4):37.
Swart, A. J. (2009). Evaluation of final examination papers in engineering: A case study using bloom’s taxonomy. IEEE Transactions on Education, 53(2):257–264.
Tenenberg, J. and Murphy, L. (2005). Knowing what i know: An investigation of undergraduate knowledge and self-knowledge of data structures. Computer Science Education, 15(4):297–315.
Wei, J. and Zou, K. (2019). Eda: Easy data augmentation techniques for boosting performance on text classification tasks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 6383–6389.
Yahya, A. A. and Osman, A. (2011). Automatic classification of questions into bloom’s cognitive levels using support vector machines. In The International Arab Conference on Information Technology, Naif Arab University for Security Science (NAUSS), pages 1–6.
Yahya, A. A., Toukal, Z., and Osman, A. (2012). Bloom’s taxonomy–based classification for item bank questions using support vector machines. In Modern advances in intelligent systems and tools, pages 135–140. Springer.
Yu, F.-Y., Liu, Y.-H., and Chan, T.-W. (2005). A web-based learning system for question-posing and peer assessment. Innovations in education and teaching international, 42(4):337–348.
Publicado
22/11/2021
Como Citar
ROCHA, Daniyel N. N.; CAMPELO, Cláudio E. C.; JERÔNIMO, Caio L. M..
Aplicação de verbos como proxy para identificação automática do nível cognitivo de questões: uma abordagem baseada na taxonomia de Bloom. In: SIMPÓSIO BRASILEIRO DE INFORMÁTICA NA EDUCAÇÃO (SBIE), 32. , 2021, Online.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2021
.
p. 897-908.
DOI: https://doi.org/10.5753/sbie.2021.218656.
