Empowering Instructors with Collective Intelligence: Learning Problem-Solving Paths to Facilitate Feedback Generation
Resumo
The feedback on the Intelligent Tutoring Systems (ITS) providing the necessary support and guidance to students successfully complete a given task, improving their learning. However, building proper feedback demands time, whereas instructors are often overloaded, and several interactions with experts of the domain knowledge to know the several paths students, which is often infeasible and increases the costs and the complexity to develop ITSs at scale. To address this problem, we proposed a novel approach to build feedback using students’ collective intelligence (CI), where our ITS might learn such paths incrementally building a knowledge graph with more detail than those predicted by instructors, optimizing the identification of problem-solving paths, and facilitating the iterative designing of meaningful and fine-grained feedback for the instructors by presenting domain model’s updated, meaningful visualizations. To evaluate our approach we developed an ITS in a domain of numerical expressions that was used by 99 students. As a result, we observed that our approach helps to create a knowledge graph with a quality equivalent to that built by specialists in less time and considerably reducing the instructor’s overload.
Referências
Baker, S. and Green, H. (2005). Blogs will change your business. Business week, 3931(5):56–65.
Landers, R. N. and Behrend, T. S. (2015). An inconvenient truth: Arbitrary distinctions between organizational, mechanical turk, and other convenience samples. Industrial and Organizational Psychology, 8(2):142–164.
Leimeister, J. M. (2010). Collective intelligence. Business & Information Systems Engineering, 2(4):245–248.
Malone, T. W., Laubacher, R., and Dellarocas, C. (2010). The collective intelligence genome. MIT Sloan management review, 51(3):21.
Meza, J., Jimenez, A., Mendoza, K., and Vaca-Cárdenas, L. (2018). Collective intelligence education, enhancing the collaborative learning. In 2018 International Conference on eDemocracy & eGovernment (ICEDEG), pages 24–30. IEEE.
Paolacci, G., Chandler, J., and Ipeirotis, P. G. (2010). Running experiments on amazon mechanical turk. Judgment and Decision making, 5(5):411–419.
Tenório, T., Isotani, S., and Bittencourt, I. I. (2022). Authoring inner loops of intelligent tutoring systems using collective intelligence. In International Conference on Artificial Intelligence in Education, pages 400–404. Springer.
Tenório, T., Isotani, S., Bittencourt, I. I., and Lu, Y. (2021). The state-of-the-art on collective intelligence in online educational technologies. IEEE Transactions on Learning Technologies, 14(2):257–271.
Tenório, T., Isotani, S., and Bittencourt, I. I. (2020). Inteligência coletiva como ferramenta de apoio na construção de loops internos em sistemas tutores inteligentes. In Anais do XXXI Simpósio Brasileiro de Informática na Educação, pages 1233–1242, Porto Alegre, RS, Brasil. SBC.
VanLehn, K. (2006). The behavior of tutoring systems. International journal of artificial intelligence in education, 16(3):227–265.
Von Ahn, L. (2013). Duolingo: learn a language for free while helping to translate the web. In Proceedings of the 2013 international conference on Intelligent user interfaces, pages 1–2.
Wisniewski, B., Zierer, K., and Hattie, J. (2020). The power of feedback revisited: A meta-analysis of educational feedback research. Frontiers in Psychology, 10:3087.
