Do LLMs support generating code that is compliant with creational design patterns? - Machado, Caio S.; Viegas, Eduardo Kugler; Melo, Silvana Morita; Paschoal, Leo Natan (Anais do Workshop sobre Bots na Engenharia de Software (WBOTS); 2025: Anais do II Workshop sobre Bots na Engenharia de Software, 2025)
Large Language Models (LLMs) have been increasingly employed in Software Engineering; however, their reliability in applying established practices such as design patterns remains uncertain. This study investigates the ability of the LLMs ChatGPT, Gemini, and Copilot to generate code that adheres to creational design patterns. The analysis considers the correctness of the implementations, as well as the complexity and maintainability of the generated code. To this end, a comparative study was conducted involving 150 programs generated by the models and 50 reference programs developed by humans. The results reveal that, although LLMs are generally capable of applying design patterns, a critical failure rate of 22.67% was observed, with significant performance variation across models and design patterns. Regarding code complexity, no substantial differences were identified; however, in most cases, the artifacts generated by the LLMs exhibited higher maintainability than the human-written reference implementations.