Quality Evaluation of Software Functional Requirements Generated by LLMs: A Systematic Mapping Study - Lourenço, Lucas M.; Bezerra, Carla; Marques, Allan P. (Anais do Simpósio Brasileiro de Sistemas de Informação (SBSI); 2026: Anais do XXII Simpósio Brasileiro de Sistemas de Informação, 2026)
Research Context: Large Language Models (LLMs) have been increasingly applied in software engineering, especially in the generation of functional requirements for information systems. Scientific and/or Practical Problem: However, the quality of these requirements still raises concerns, such as ambiguities, incompleteness, and dependency on prompts. Proposed Solution and/or Analysis: This study conducts a systematic mapping study to analyze how the literature has evaluated requirements generated by LLMs. Related IS Theory: The work is aligned with the sociotechnical perspective, considering information systems requirements as both technical and social artifacts. Research Method: A systematic mapping study was conducted, reviewing 1,875 studies and selecting 51 primary studies published between 2020 and 2025. Summary of Results: The findings indicate a diversity of evaluation practices, combining traditional criteria and NLP metrics, with advantages in automation and standardization, but limitations regarding the absence of benchmarks and validation in industrial contexts, especially in the development of information systems. Contributions and Impact to IS area: The study contributes to academia by consolidating evaluation approaches and identifying gaps, and to industry by supporting the understanding of risks and opportunities in the use of LLMs in the requirements engineering of information systems.