Augmenting, Not Replacing: The Role of Generative AI in Teaching Software Modeling
Resumo
Context: Software Modeling is a fundamental component of Computer Science curricula, addressing diverse themes including domain modeling. Despite its importance and years of research, a persistent challenge remains: students’ difficulty in comprehending and applying core modeling concepts. While numerous studies have identified these issues and proposed solutions, many involving collaborative and feedback-based environments, the recent advances of Large Language Models (LLMs) offer a new frontier. This study proposes and evaluates a pedagogical approach that leverages a generative LLM to improve students’ practical skills in modeling UML class diagrams. Method: In an empirical study with 30 undergraduate students, participants first created a class diagram for a given problem, then used an LLM for the same task, and finally compared both models. We analyzed quantitative pre- and post-study self-assessments and qualitative feedback on the process. Results: The study revealed a statistically significant improvement (p<0.05) in students’ self-assessed knowledge for class diagrams, an effect not observed in control topics. Qualitatively, students praised the LLM for accelerating ideation and identifying errors, but also noted significant inaccuracies in the generated models, particularly in class relationships. Students perceive the LLM not as an autonomous ’autopilot’ for generating solutions, but as a valuable yet flawed ’co-pilot’ that augments the learning process. This suggests LLMs are effective tools for support and validation rather than direct solution generation.
Referências
Nilufar Baghaei and Antonija Mitrovic. 2006. A Constraint-Based Collaborative Environment for Learning UML Class Diagrams. In Intelligent Tutoring Systems, Mitsuru Ikeda, Kevin D. Ashley, and Tak-Wai Chan (Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg, 176–186.
Lenz Belzner, Thomas Gabor, and Martin Wirsing. 2023. Large Language Model Assisted Software Engineering: Prospects, Challenges, andnbsp;anbsp;Case Study. In Bridging the Gap Between AI and Reality: First International Conference, AISoLA 2023, Crete, Greece, October 23–28, 2023, Proceedings (Crete, Greece). Springer-Verlag, Berlin, Heidelberg, 355–374. DOI: 10.1007/978-3-031-46002-9_23
Alberto. Cardoso, Gustavo R.. Alves, and Teresa. Restivo. 2020. Proceedings of the 2020 IEEE Global Engineering Education Conference (EDUCON) : date and venue, 27-30 April, 2020, Porto, Portugal. IEEE.
Jianguo Chen, Huijuan Lu, Lixin An, and Yongxia Zhou. 2009. Exploring teaching methods in software engineering education. In 2009 4th International Conference on Computer Science & Education. IEEE, 1733–1738.
Javier Cámara, Javier Troya, Lola Burgueño, and Antonio Vallecillo. 2023. On the assessment of generative AI in modeling tasks: an experience report with ChatGPT and UML. Software and Systems Modeling 22 (6 2023), 781–793. Issue 3. DOI: 10.1007/s10270-023-01105-5
Alessio Ferrari, Sallam Abualhaija, and Chetan Arora. 2024. Model generation with LLMs: From requirements to UML sequence diagrams. In 2024 IEEE 32nd International Requirements Engineering Conference Workshops (REW). IEEE, 291– 300.
Eleonora Grassucci, Gualtiero Grassucci, Aurelio Uncini, and Danilo Comminiello. 2025. Beyond answers: How llms can pursue strategic thinking in education. arXiv preprint arXiv:2504.04815 (2025).
Bilal Karasneh, Rodi Jolak, and Michel RV Chaudron. 2015. Using examples for teaching software design: An experiment using a repository of uml class diagrams. In 2015 Asia-Pacific Software Engineering Conference (APSEC). IEEE, 261–268.
Enkelejda Kasneci, Kathrin Seßler, Stefan Küchemann, Maria Bannert, Daryna Dementieva, Frank Fischer, Urs Gasser, Georg Groh, Stephan Günnemann, Eyke Hüllermeier, et al. 2023. ChatGPT for good? On opportunities and challenges of large language models for education. Learning and individual differences 103 (2023), 102274.
Maryam Khan, Muhammad Azeem Akbar, and Jussi Kasurinen. 2025. Integrating LLMs in Software Engineering Education: Motivators, Demotivators, and a Roadmap Towards a Framework for Finnish Higher Education Institutes. arXiv preprint arXiv:2503.22238 (2025).
Ranim Khojah, Mazen Mohamad, Philipp Leitner, and Francisco Gomes de Oliveira Neto. 2024. Beyond code generation: An observational study of chatgpt usage in software engineering practice. Proceedings of the ACM on Software Engineering 1, FSE (2024), 1819–1840.
H. B. Mann and D. R. Whitney. 1947. On a Test of Whether one of Two Random Variables is Stochastically Larger than the Other. The Annals of Mathematical Statistics 18, 1 (1947), 50 – 60. DOI: 10.1214/aoms/1177730491
Modeling-Languages.com. 2024. Textual UML Tools – The Complete List. [link].
PlantUML Team. 2025. PlantUML: Create diagrams from textual description. [link].
Nishat Raihan, Mohammed Latif Siddiq, Joanna CS Santos, and Marcos Zampieri. 2025. Large language models in computer science education: A systematic literature review. In Proceedings of the 56th ACM Technical Symposium on Computer Science Education V. 1. 938–944.
S. S. SHAPIRO and M. B. WILK. 1965. An analysis of variance test for normality (complete samples)†. Biometrika 52, 3-4 (12 1965), 591–611. DOI: 10.1093/biomet/52.3-4.591 arXiv: [link]
Ven Yu Sien. 2011. An investigation of difficulties experienced by students developing unified modelling language (UML) class and sequence diagrams. Computer Science Education 21 (12 2011), 317–342. Issue 4. DOI: 10.1080/08993408.2011.630127
Williamson Silva, Igor Steinmacher, and Tayana Conte. 2019. Students’ and instructors’ perceptions of five different active learning strategies used to teach software modeling. IEEE Access 7 (2019), 184063–184077.
Ann T S Taylor. 2011. Top 10 reasons students dislike working in small groups . . . and why i do it anyway. Biochemistry and Molecular Biology Education 39 (2011).
