Augmenting, Not Replacing: The Role of Generative AI in Teaching Software Modeling - Nascimento, Maria Vitória Costa do; Lima, Márcia Sampaio; Conte, Tayana Uchôa (Anais do Simpósio Brasileiro de Qualidade de Software (SBQS); 2025: Anais do XXIV Simpósio Brasileiro de Qualidade de Software, 2025)
Context: Software Modeling is a fundamental component of Computer Science curricula, addressing diverse themes including domain modeling. Despite its importance and years of research, a persistent challenge remains: students’ difficulty in comprehending and applying core modeling concepts. While numerous studies have identified these issues and proposed solutions, many involving collaborative and feedback-based environments, the recent advances of Large Language Models (LLMs) offer a new frontier. This study proposes and evaluates a pedagogical approach that leverages a generative LLM to improve students’ practical skills in modeling UML class diagrams. Method: In an empirical study with 30 undergraduate students, participants first created a class diagram for a given problem, then used an LLM for the same task, and finally compared both models. We analyzed quantitative pre- and post-study self-assessments and qualitative feedback on the process. Results: The study revealed a statistically significant improvement (p<0.05) in students’ self-assessed knowledge for class diagrams, an effect not observed in control topics. Qualitatively, students praised the LLM for accelerating ideation and identifying errors, but also noted significant inaccuracies in the generated models, particularly in class relationships. Students perceive the LLM not as an autonomous ’autopilot’ for generating solutions, but as a valuable yet flawed ’co-pilot’ that augments the learning process. This suggests LLMs are effective tools for support and validation rather than direct solution generation.