Context Dependency in LLM-Based Requirements Clarification: Emerging Results from a Controlled SDD Study
Resumo
Specification-Driven Development (SDD) workflows increasingly use Large Language Model (LLM)-based agents to analyze natural-language requirements and generate clarification questions. However, User Stories are intentionally concise, and relevant information may be distributed across glossaries, business rules, prior decisions, and system documentation. This paper reports emerging results from a controlled exploratory study on how different context types affect LLM-based requirements clarification. We analyze four User Stories from different domains under six conditions: no additional context, lexical (terms and definitions), operational (rules and exceptions), decisional (prior decisions), systemic (actors and dependencies), and all contexts combined. The 24 executions produced 118 primary questions, which were segmented into 42 normalized informational gaps and classified by context category, coverage, relevance, and behavior across conditions. Operational and combined-context conditions were associated with fewer observed necessary-and-unanswered gaps, while richer context may also surface clarification needs not observed without additional context. The study contributes an evaluation structure for analyzing context dependency in SDD clarification workflows through normalized informational gaps, coverage, relevance, and behavior across conditions.
Referências
Haowei Cheng, Jati H. Husen, Yijun Lu, Teeradaj Racharak, Nobukazu Yoshioka, Naoyasu Ubayashi, and HironoriWashizaki. 2026. Generative AI for Requirements Engineering: A Systematic Literature Review. Software: Practice and Experience 56, 2 (2026), 141–170. DOI: 10.1002/spe.70029
Fabiano Dalpiaz. 2018. Requirements Data Sets (User Stories). Mendeley Data. DOI: 10.17632/7zbk8zsd8y.1
Fabiano Dalpiaz, Ivor van der Schalk, Sjaak Brinkkemper, Fatma Başak Aydemir, and Garm Lucassen. 2019. Detecting Terminological Ambiguity in User Stories: Tool and Experimentation. Information and Software Technology 110 (2019), 3–16. DOI: 10.1016/j.infsof.2018.12.007
GitHub. 2026. Spec Kit: A Specification-Driven Development Toolkit. [link]. Accessed: 2026-07-08.
Kaicheng Huang, Fanyu Wang, Yutan Huang, and Chetan Arora. 2025. Prompt Engineering for Requirements Engineering: A Literature Review and Roadmap. In 2025 IEEE 33rd International Requirements Engineering Conference Workshops (REW). IEEE, 548–557. DOI: 10.1109/REW66121.2025.00081
Garm Lucassen, Fabiano Dalpiaz, Jan Martijn E. M. van der Werf, and Sjaak Brinkkemper. 2016. Improving Agile Requirements: The Quality User Story Framework and Tool. Requirements Engineering 21, 3 (2016), 383–403. DOI: 10.1007/s00766-016-0250-x
Lloyd Montgomery, Davide Fucci, Abir Bouraffa, Lisa Scholz, and Walid Maalej. 2022. Empirical Research on Requirements Quality: A Systematic Mapping Study. Requirements Engineering 27, 2 (2022), 183–209. DOI: 10.1007/s00766-021-00367-z
Mohammad Amin Zadenoori, Jacek Dąbrowski, Waad Alhoshan, Liping Zhao, and Alessio Ferrari. 2025. Large Language Models (LLMs) for Requirements Engineering (RE): A Systematic Literature Review. arXiv:2509.11446 [cs.SE] DOI: 10.48550/arXiv.2509.11446
