From Text to Maps: Automated Concept Map Generation Using Fine-tuned Large Language Model

• Wagner de A. Perin UFES
• Davidsom Cury UFES
• Camila Zacché de Aguiar UFES
• Crediné S. de Menezes UFRGS

DOI: https://doi.org/10.5753/sbie.2023.234749

Resumo

Concept maps (CMs) are tools for visualizing relationships between ideas, facilitating more effective comprehension and learning. However, the automatic generation of CMs from unstructured text presents a challenge, often requiring semantic markup and subsequent complex processing. This paper introduces a novel approach to address this hurdle by harnessing the capabilities of fine-tuned Large Language Models (LLMs). Our innovative methodology uses these models to extract structured propositions from unstructured text, subsequently serving as the foundation for constructing a CM. This process reverses the transformation of CM relations into first-order logic propositions, a concept explored in our previous work. To achieve this, we train the LLM using fine-tuning techniques, leveraging the latest advancements in artificial intelligence and machine learning. We evaluate our proposed solution based on precision and recall metrics, comparing our outcomes against models crafted by experts. Notably, the results indicate that our method can contribute significantly to advancements in the automatic generation of CMs, illustrating another application bolstered by recent breakthroughs in artificial intelligence. As a stepping stone in this promising direction, future research should continue to refine the model and explore potential applications across diverse domains.

Referências

Acharya, A. and Sinha, D. (2017). An educational data mining approach to concept map construction for web based learning. Informatica Economica, 21(4):41–58.

Aguiar, C. and Cury, D. (2017). Mineração de mapas conceituais a partir de textos em português. In Brazilian Symposium on Computers in Education (Simpósio Brasileiro de Informática na Educação-SBIE), volume 28, page 1117.

Aguiar, C. Z., Cury, D., and Zouaq, A. (2016). Automatic construction of concept maps from texts. In Proceedings of the 7th International Conference on Concept Mapping, volume 2, pages 20–30, Tallinn, Estonia. CMC 2016.

Bai, S.-M. and Chen, S.-M. (2008). Automatically constructing concept maps based on fuzzy rules for adapting learning systems. Expert Systems with Applications, 35(1):41–49.

Baker, R. S. (2014). Educational data mining: An advance for intelligent systems in education. IEEE Intelligent Systems, 29(3):78–82.

Booth, D. E. (2007). Data mining methods and models. Technometrics, 49(4):500–500.

Chen, S.-M. and Sue, P.-J. (2013). Constructing concept maps for adaptive learning systems based on data mining techniques. Expert Systems with Applications, 40(7):2746–2755.

Cowan, A. M. (2002). Data mining in finance: Advances in relational and hybrid methods: Boris kovalerchuk and evgenii vityaev (eds.), kluwer academic publishers, norwell, massachusetts, 2000, hb us $120, isbn 0-7923-7804-0. International Journal of Forecasting, 18(1):155–156.

de Castro, R. N., Perin, W. A., and Cury, D. (2015). Layouts automáticos para mapas conceituais-um serviço integrado a uma plataforma de serviços web. In XX Congresso Internacional de Informática Educativa (TISE).

de la Villa, M., Aparicio, F., Maña, M. J., and de Buenaga, M. (2012). A learning support tool with clinical cases based on concept maps and medical entity recognition. In Proceedings of the 2012 ACM International Conference on Intelligent User Interfaces, IUI ’12, page 61–70, New York, NY, USA. Association for Computing Machinery.

Ecoffet, A. (2023). Gpt-4 technical report. arXiv preprint arXiv:2303.08774.

Hirschman, L., Park, J. C., Tsujii, J., Wong, L., and Wu, C. H. (2002). Accomplishments and challenges in literature data mining for biology. Bioinformatics, 18(12):1553–1561.

Lai, C.-F., Chen, S.-Y., Tsai, C.-W., Chang, Y.-C., and Su, Y.-S. (2017). Using information retrieval to construct an intelligent e-book with keyword concept map. Eurasia Journal of Mathematics, Science and Technology Education, 13(10):6737–6747.

Li, L., Tang, H., Wu, Z., Gong, J., Gruidl, M., Zou, J., Tockman, M., and Clark, R. A. (2004). Data mining techniques for cancer detection using serum proteomic profiling. Artificial Intelligence in Medicine, 32(2):71–83.

Novak, J. D. and Cañas, A. J. (2008). The theory underlying concept maps and how to construct and use them. IHMC.

Nugumanova, A., Mansurova, M., Alimzhanov, E., Zyryanov, D., and Apayev, K. (2015). Automatic generation of concept maps based on collection of teaching materials. In International conference on data management technologies and applications, volume 2, pages 248–254. SCITEPRESS.

Oppl, S. and Stary, C. (2011). Effects of a tabletop interface on the co-construction of concept maps. In Human-Computer Interaction–INTERACT 2011: 13th IFIP TC 13 International Conference, Lisbon, Portugal, September 5-9, 2011, Proceedings, Part III 13, pages 443–460. Springer.

Perin, W. and Cury, D. (2016). Uma plataforma de serviços para mapas conceituais. In Brazilian Symposium on Computers in Education (Simpósio Brasileiro de Informática na Educação-SBIE), volume 27, page 230.

Perin, W., Cury, D., and Menezes, C. d. (2015). imap & cmpaas-de ferramenta à plataforma de serviços para mapas conceituais. Revista Brasileira de Informática na Educação, 24(3).

Perin, W. d. A. (2014). imap: um mecanismo de inferência para mapas conceituais. Repositorio de Dissertacoes UFES (repositorio.ufes.br:10/1484).

Perin, W. d. A., Cury, D., and Menezes, C. (2014). Nlp-imap: Integrated solution based on question-answer model in natural language for an inference mechanism in concepts maps. In Proceedings of the 14th International Conference on Concept Mapping.

Qasim, I., Jeong, J.-W., Heu, J.-U., and Lee, D.-H. (2013). Concept map construction from text documents using affinity propagation. Journal of Information Science, 39(6):719–736.

Romero, C. and Ventura, S. (2007). Educational data mining: A survey from 1995 to 2005. Expert Systems with Applications, 33(1):135–146.

Shao, Z., Li, Y., Wang, X., Zhao, X., and Guo, Y. (2020). Research on a new automatic generation algorithm of concept map based on text analysis and association rules mining. Journal of ambient intelligence and humanized computing, 11:539–551.

Tseng, S.-S., Sue, P.-C., Su, J.-M., Weng, J.-F., and Tsai, W.-N. (2007). A new approach for constructing the concept map. Computers & Education, 49(3):691–707.

Vassoler, G. A., Perin, W. d. A., and Cury, D. (2014). Mergemaps–a computational tool for merging of concept maps. In Proceedings of the 14th International Conference on Concept Mapping.

Wang, M. and Hu, Y. (2016). I-raid: A novel redundant storage architecture for improving reliability, performance, and life-span of solid-state disk systems. In Proceedings of the 31st Annual ACM Symposium on Applied Computing, SAC ’16, page 1824–1831, New York, NY, USA. Association for Computing Machinery.

Wang, W., Cheung, C., Lee, W., and Kwok, S. (2008). Mining knowledge from natural language texts using fuzzy associated concept mapping. Information Processing & Management, 44(5):1707–1719.

Yang, J., Jin, H., Tang, R., Han, X., Feng, Q., Jiang, H., Yin, B., and Hu, X. (2023). Harnessing the power of llms in practice: A survey on chatgpt and beyond. arXiv preprint arXiv:2304.13712.

Zouaq, A. and Nkambou, R. (2009). Evaluating the generation of domain ontologies in the knowledge puzzle project. IEEE Transactions on Knowledge and Data Engineering, 21(11):1559–1572.

Žubrinić, K., Obradović, I., and Sjekavica, T. (2015). Implementation of method for generating concept map from unstructured text in the croatian language. In 2015 23rd International Conference on Software, Telecommunications and Computer Networks (SoftCOM), pages 220–223.