Prompt Engineering Applied to the Generation of Software Modeling Documents: A Systematic Mapping Study

  • Rafael Ribas de Lima UTFPR
  • Simone Nasser Matos UTFPR

Resumo


Automating software modeling from natural language specifications is a central challenge in Model-Driven Engineering. Large Language Models (LLMs) and Prompt Engineering techniques have emerged as resources to automate this task, but the literature on the topic remains fragmented across different diagram types, representation languages, and instruction strategies. This paper presents a Systematic Mapping Study on the use of LLMs and Prompt Engineering for the generation, modification, and validation of software modeling documents. From 475 studies identified in three digital libraries (Scopus, IEEE Xplore, and ScienceDirect), 52 primary studies published between 2023 and 2026 were included after applying inclusion and exclusion criteria and full-text reading. Results indicate that GPT-family models remain predominant as the generation engine (73.1% of the studies), that Class diagrams and Architecture and Component models concentrate most of the produced artifacts, and that Few-shot, Chain-of-Thought, and prompt chaining strategies coupled with deterministic syntactic validators constitute the most reported pattern to contain structural conformance errors. The mapping also identifies a persistent gap in semantic validation and explainability (XAI) mechanisms for the generated artifacts.

Palavras-chave: Prompt Engineering, Large Language Models, Software Modeling, UML, Systematic Mapping Study

Referências

Seif Abukhalaf, Mohammad Hamdaqa, and Foutse Khomh. 2023. On Codex Prompt Engineering for OCL Generation: An Empirical Study. In IEEE Xplore. 148–157. DOI: 10.1109/MSR59073.2023.00033

Seif Abukhalaf, Mohammad Hamdaqa, and Foutse Khomh. 2024. PathOCL: Path-Based Prompt Augmentation for OCL Generation with GPT-4. Proceedings - 2024 IEEE/ACM 1st International Conference on AI Foundation Models and Software Engineering, FORGE 2024 (2024), 108 – 118. DOI: 10.1145/3650105.3652290

Bilal Al-Ahmad, Anas Alsobeh, Omar Meqdadi, and Nazimuddin Shaikh. 2025. A Student-Centric Evaluation Survey to Explore the Impact of LLMs on UML Modeling. Information (Switzerland) 16 (2025). DOI: 10.3390/info16070565

Averi Bates, Ryan Vavricka, Shane Carleton, Ruosi Shao, and Chongle Pan. 2025. Unified modeling language code generation from diagram images using multimodal large language models. Machine Learning with Applications 20 (2025), 100660. DOI: 10.1016/j.mlwa.2025.100660

Marco Calamo, Massimo Mecella, and Monique Snoeck. 2025. Assessing the Suitability of Large Language Models in Generating UML Class Diagrams as Conceptual Models. Lecture Notes in Business Information Processing 558 LNBIP (2025), 211 – 226. DOI: 10.1007/978-3-031-95397-2_13

Sebastian Copei, Oliver Hohlfeld, Jens Kosiol, and Aleksandar Ristoski. 2025. LLMs Choose the Right Stack: From Patterns to Tools. In IEEE Xplore. 276–283. DOI: 10.1109/ASEW67777.2025.00057

Gabriele De Vito, Fabio Palomba, Carmine Gravino, Sergio Di Martino, and Filomena Ferrucci. 2023. ECHO: An Approach to Enhance Use Case Quality Exploiting Large LanguageModels. In IEEE Xplore. 53–60. DOI: 10.1109/SEAA60479. 2023.00017

David Delgado, Lola Burgueño, and Robert Clarisó. 2026. A framework for assessing the capabilities of code generation of constraint domain-specific languages with large language models. Journal of Systems and Software 238 (2026), 112871. DOI: 10.1016/j.jss.2026.112871

Damaris Naomi Dolha and Robert Andrei Buchmann. 2024. Experiments with natural language queries on RDF vs. XML-serialized BPMN diagrams. Procedia Computer Science 246 (2024), 3246–3255. DOI: 10.1016/j.procs.2024.09.315

Tobias Eisenreich, Nicholas Friedlaender, and Stefan Wagner. 2025. Leveraging Large Language Models for Use Case Model Generation from Software Requirements. Proceedings - 2025 40th IEEE/ACM International Conference on Automated Software Engineering Workshops, ASEW 2025 (2025), 221 – 227. DOI: 10.1109/ASEW67777.2025.00050

Zobia Erum and Issam Al-Azzoni. 2025. A GPT for Ecore Model Building. In IEEE Xplore. 54–57. DOI: 10.1109/FLLM67465.2025.11391086

Matteo Esposito, Xiaozhou Li, Sergio Moreschini, Noman Ahmad, Tomas Cerny, Karthik Vaidhyanathan, Valentina Lenarduzzi, and Davide Taibi. 2026. Generative AI for software architecture. Applications, challenges, and future directions. Journal of Systems and Software 231 (2026), 112607. DOI: 10.1016/j.jss.2025.112607

Alessio Ferrari, Sallam Abualhaijal, and Chetan Arora. 2024. Model Generation with LLMs: From Requirements to UML Sequence Diagrams. Proceedings - 32nd IEEE International Requirements Engineering Conference Workshops, REW 2024 (2024), 291 – 300. DOI: 10.1109/REW61692.2024.00044

Marios Fokaefs, Hashim Khan, and Borna Ahmadzadeh. 2025. Can AI Build Systems? an Exploratory Study on Generating Software Architecture With LLMS. Proceedings - 2025 IEEE International Conference on Collaborative Advances in Software and Computing, CASCON 2025 (2025), 399 – 408. DOI: 10.1109/CASCON66301.2025.00069

Carina Fresemann, Claudius Ellsel, Carl-Philipp Grunenwald, and Rainer Stark. 2025. Using Large Language Models for System Modeling: Current State and Future Implications. In IEEE Xplore. 1–8. DOI: 10.1109/ISSE65546.2025.11369978

Giacomo Garaccione, Diego Maria Calabrese, Riccardo Coppola, and Luca Ardito. 2025. A comparison of different Large LanguageModels for the generation of UML class diagrams. In IEEE Xplore. 541–545. DOI: 10.1109/MODELS-C68889.2025.00078

Stefana Gheorghita, Cosmin-Iulian Irimia, and Adrian Iftene. 2025. Automating Software Diagram Generation with Large Language Models. Procedia Computer Science 270 (2025), 713–722. DOI: 10.1016/j.procs.2025.09.191

Ahmad Hatahet, Christoph Knieke, and Andreas Rausch. 2025. Generating Software Architecture Description from Source Code using Reverse Engineering and Large Language Model. Proceedings - 2025 ACM/IEEE 28th International Conference on Model Driven Engineering Languages and Systems Companion, MODELS-C 2025 (2025), 566 – 575. DOI: 10.1109/MODELS-C68889.2025.00080

Dongming Jin, Zhi Jin, Xiaohong Chen, and Chunhui Wang. 2024. ChatModeler: A human-machine collaborative and iterative requirements elicitation and modeling approach via large language models; [ChatModeler]. Jisuanji Yanjiu yu Fazhan/Computer Research and Development 61 (2024), 338 – 350. DOI: 10.7544/issn1000-1239.202330746

Dae-Kyoo Kim. 2025. Quantitative Assessment of Generative Large Language Models on Design Pattern Application. Computers, Materials and Continua 82 (2025), 3843–3872. DOI: 10.32604/cmc.2025.062552

Dae-Kyoo Kim. 2026. Artifact validity under varying agent configurations in LLM-assisted software development: A comparative analysis. Information and Software Technology 192 (2026), 108022. DOI: 10.1016/j.infsof.2026.108022

Barbara Kitchenham and Stuart Charters. 2007. Guidelines for performing Systematic Literature Reviews in Software Engineering. Technical Report EBSE-2007-01. Keele University and University of Durham.

Yuan Kong, Nan Zhang, Zhenhua Duan, and Bin Yu. 2025. Collaboration with Generative AI to improve Requirements Change. Computer Standards & Interfaces 94 (2025), 104013. DOI: 10.1016/j.csi.2025.104013

Victor Lamas, Daniel Garcia-Gonzalez, Luca Sala, and Miguel R. Luaces. 2026. DSL-Xpert 2.0: Enhancing LLM-driven code generation for domain-specific languages. Information and Software Technology 190 (2026), 107954. DOI: 10.1016/j.infsof.2025.107954

Josip Tomo Licardo, Nikola Tanković, and Darko Etinger. 2024. A Method for Extracting BPMN Models from Textual Descriptions Using Natural Language Processing. Procedia Computer Science 239 (2024), 483–490. DOI: 10.1016/j.procs.2024.06.196

Xianhui Liu, Yueying Liu, Yiheng Zhuang, and Wenlong Hou. 2026. UCD-LLM: A use case diagram requirement modeling multi-agent framework with large language model. Information and Software Technology 190 (2026), 107955. DOI: 10.1016/j.infsof.2025.107955

Yue Liu, Sin Kit Lo, Qinghua Lu, Liming Zhu, Dehai Zhao, Xiwei Xu, Stefan Harrer, and Jon Whittle. 2025. Agent design pattern catalogue: A collection of architectural patterns for foundation model based agents. Journal of Systems and Software 220 (2025), 112278. DOI: 10.1016/j.jss.2024.112278

Nikolaos Malamas, Emmanouil Tsardoulias, Konstantinos Panayiotou, and Andreas L. Symeonidis. 2025. Toward efficient vibe coding: An LLM-based agent for low-code software development. Journal of Computer Languages 85 (2025), 101367. DOI: 10.1016/j.cola.2025.101367

João José Maranhão and Eduardo Martins Guerra. 2024. A Prompt Pattern Sequence Approach to Apply Generative AI in Assisting Software Architecture Decision-making. ACM International Conference Proceeding Series (2024). DOI: 10.1145/3698322.3698324

Christoph Mayr-Dorn, Anmol Bilal, Cosmina Ratiu, and Alexander Egyed. 2025. Generating Quality Assurance Constraints From Natural Language With LLMs. Journal of Software: Evolution and Process 37 (2025). DOI: 10.1002/smr.70062

Lukasz Mazur, Nenad Petrovic, James Pontes Miranda, Ansgar Radermacher, Robert Rasche, and Alois Knoll. 2025. Querying Large Automotive Software Models: Agentic vs. Direct LLM Approaches. 2025 2nd International Generative AI and Computational Language Modelling Conference, GACLM 2025 (2025), 221 – 228. DOI: 10.1109/GACLM67198.2025.11232128

Jonathan Silva Mercado, Qin Ma, Sybren de Kinderen, Karolin Winter, and Jordi Cabot. 2026. CLERK: A Companion Large Language Model Expert for modeling Regulatory Knowledge. Data & Knowledge Engineering 164 (2026), 102578. DOI: 10.1016/j.datak.2026.102578

Vittoriano Muttillo, Claudio Di Sipio, Riccardo Rubei, and Luca Berardinelli. 2025. Leveraging synthetic trace generation of modeling operations for intelligent modeling assistants using large language models. Information and Software Technology 186 (2025), 107806. DOI: 10.1016/j.infsof.2025.107806

Vittoriano Muttillo, Claudio Di Sipio, Riccardo Rubei, Luca Berardinelli, and MohammadHadi Dehghani. 2024. Towards Synthetic Trace Generation of Modeling Operations using In-Context Learning Approach. Proceedings - 2024 39th ACM/IEEE International Conference on Automated Software Engineering, ASE 2024 (2024), 619 – 630. DOI: 10.1145/3691620.3695058

Lahbib Naimi, El Mahi Bouziane, Abdeslam Jakimi, Rachid Saadane, and Abdellah Chehri. 2024. Automating Software Documentation: Employing LLMs for Precise Use Case Description. Procedia Computer Science 246 (2024), 1346 – 1354. DOI: 10.1016/j.procs.2024.09.568

Reshma P. Nair, M. G. Thushara, and Vijayan Sugumaran. 2025. Fine-Tuned LLMs Versus Rule-Based NLP for UML Diagram Generation: An Educational Evaluation. IEEE Access 13 (2025), 211605–211619. DOI: 10.1109/ACCESS.2025.3638372

Benjamin Nast, Leon Görgen, Eric Müller, Marcus Triller, and Kurt Sandkuhl. 2025. Exploring large language models in enterprise modeling. Discover Artificial Intelligence 5 (2025). DOI: 10.1007/s44163-025-00585-2

Van-Viet Nguyen, Huu-Khanh Nguyen, Kim-Son Nguyen, Thi Minh-Hue Luong, Duc-Quang Vu, Trung-Nghia Phung, and The-Vinh Nguyen. 2026. A Novel Unified Framework for Automated Generation and Multimodal Validation of UML Diagrams. CMES - Computer Modeling in Engineering and Sciences 146 (2026). DOI: 10.32604/cmes.2025.075442

Oluwafemi Odu, Alvine B. Belle, Song Wang, Segla Kpodjedo, Timothy C. Lethbridge, and Hadi Hemmati. 2025. Automatic instantiation of assurance cases from patterns using large language models. Journal of Systems and Software 222 (2025), 112353. DOI: 10.1016/j.jss.2025.112353

Matthew J. Page, Joanne E. McKenzie, Patrick M. Bossuyt, Isabelle Boutron, Tammy C. Hoffmann, Cynthia D. Mulrow, Larissa Shamseer, Jennifer M. Tetzlaff, and David Moher. 2021. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ 372 (2021), n71. DOI: 10.1136/bmj.n71

Krish Patel, Brian Sheehan, and Shamsnaz Bhada. 2025. Exploring Intelligent Document Processing Technology for Discovering Requirements Debt in Systems Engineering Management Plans. Procedia Computer Science 268 (2025), 437–446. DOI: 10.1016/j.procs.2025.08.223

Kai Petersen, Robert Feldt, Shahid Mujtaba, and Michael Mattsson. 2008. Systematic Mapping Studies in Software Engineering. In Proceedings of the 12th International Conference on Evaluation and Assessment in Software Engineering (EASE). 68–77.

Gokul Rejithkumar, Preethu Rose Anish, Jyoti Shukla, and Smita Ghaisas. 2024. Probing with Precision: Probing Question Generation for Architectural Information Elicitation. Proceedings - 2024 IEEE/ACM Workshop on Multi-disciplinary, Open, and RElevant Requirements Engineering, MO2RE 2024 (2024), 8 – 14. DOI: 10.1145/3643666.3648577

Stefano Rizzi, Matteo Francia, Enrico Gallinucci, and Matteo Golfarelli. 2025. Conceptual design of multidimensional cubes with LLMs: An investigation. Data & Knowledge Engineering 159 (2025), 102452. DOI: 10.1016/j.datak.2025.102452

Satrio Adi Rukmono, Lina Ochoa, and Michel R.V. Chaudron. 2023. Achieving High-Level Software Component Summarization via Hierarchical Chain-of-Thought Prompting and Static Code Analysis. In IEEE Xplore. 7–12. DOI: 10.1109/ICoDSE59534.2023.10292037

Satrio Adi Rukmono, Lina Ochoa, and Michel R.V. Chaudron. 2024. Deductive Software Architecture Recovery via Chain-of-thought Prompting. Proceedings - International Conference on Software Engineering (2024), 92 – 96. DOI: 10.1145/3639476.3639776

Hanan Abdulwahab Siala and Kevin Lano. 2026. Leveraging LLMs for abstracting UML and OCL representations from Java and Python programs. Journal of Systems and Software 238 (2026), 112874. DOI: 10.1016/j.jss.2026.112874

Ruoyu Su, Noman Ahmad, Matteo Esposito, Andrea Janes, Davide Taibi, and Valentina Lenarduzzi. 2026. Emerging trends in software architecture from the practitioner’s perspective: A five-year review. Journal of Systems and Software 236 (2026), 112820. DOI: 10.1016/j.jss.2026.112820

Mohammad Tanhaei. 2026. MedNegotiator: Automating requirements negotiation in healthcare; simulating clinical and technical perspectives using Generative AI agents. Array 30 (2026), 100783. DOI: 10.1016/j.array.2026.100783

Mircea Trifan, Bogdan Ionescu, and Dan Ionescu. 2025. Generative AI for Diagrams as Code and Code as Diagrams. In IEEE Xplore. 01–06. DOI: 10.1109/SACI66288.2025.11030105

Francesco Vitale, Francesco Flammini, Mauro Caporuscio, and Nicola Mazzocca. 2026. Architecting software monitors for control-flow anomaly detection through large language models and conformance checking. Information and Software Technology 195 (2026), 108133. DOI: 10.1016/j.infsof.2026.108133

Bangrui Wan, Zheng Wei, Shiyu Wang, Jiangping Huang, and Chunqiang Hu. 2026. Structure-guided function-level code generation with LLMs via UML activity diagrams. Neurocomputing 669 (2026), 132502. DOI: 10.1016/j.neucom.2025.132502

Chong Wang, Beian Wang, Peng Liang, and Jie Liang. 2026. Assessing UML diagrams by GPT: Implications for education. Journal of Systems and Software 234 (2026), 112709. DOI: 10.1016/j.jss.2025.112709

Claes Wohlin, Per Runeson, Martin Höst, Magnus C. Ohlsson, Björn Regnell, and Anders Wesslén. 2012. Experimentation in Software Engineering. Springer Science & Business Media.

Yongheng Zhang, Tingwen Du, Yunshan Ma, Xiang Wang, Yi Xie, Guozheng Yang, Yuliang Lu, and Ee-Chien Chang. 2025. AttacKG+: Boosting attack graph construction with Large Language Models. Computers & Security 150 (2025), 104220. DOI: 10.1016/j.cose.2024.104220

Zelong Zhao, Nan Zhang, Bin Yu, and Zhenhua Duan. 2024. Generating Java code pairing with ChatGPT. Theoretical Computer Science 1021 (2024), 114879. DOI: 10.1016/j.tcs.2024.114879

Rui Zhu, Jiapeng Chen, Tianrui Bai, Hua Yue, Jianglong Qin, and Xuan Zhang. 2025. MASFlow: Multi-Agent Based Service Workflow Generation. Proceedings - 2025 IEEE International Conference on Software Services Engineering, SSE 2025 (2025), 24 – 30. DOI: 10.1109/SSE67621.2025.00012

Zhixian Zhuang, Xiaodong Lee, Aiyao Zhang, Jiuqi Wei, Yufan Fu, and Botao Peng. 2026. Structured policy modeling and context-aware generation for multi-jurisdictional compliance in global software systems. Information and Software Technology 193 (2026), 108041. DOI: 10.1016/j.infsof.2026.108041

Yaxin Zou, Zhibin Yang, Hao Liu, Jiawei Liang, Yong Zhou, and Zonghua Gu. 2025. Automated AADL Architecture Modeling: Leveraging Large Language Models for Safety-Critical Software. In IEEE Xplore. 311–321. DOI: 10.1109/MODELS-C68889.2025.00050
Publicado
11/09/2026
LIMA, Rafael Ribas de; MATOS, Simone Nasser. Prompt Engineering Applied to the Generation of Software Modeling Documents: A Systematic Mapping Study. In: WORKSHOP BRASILEIRO DE ENGENHARIA DE SOFTWARE INTELIGENTE (ISE), 5. , 2026, São Paulo/SP. Anais [...]. Porto Alegre: Sociedade Brasileira de Computação, 2026 . p. 43-48. DOI: https://doi.org/10.5753/ise.2026.32863.