Adoption of LLMs in Requirements Engineering: What Practitioners Are Worried About?

• Reine Santos UFAM
• Igor Steinmacher Northern Arizona University
• Tayana Conte UFAM
• Ana Carolina Oran UFAM
• Bruno Gadelha UFAM

DOI: https://doi.org/10.5753/sbqs.2025.15089

Abstract

Requirements engineers face persistent challenges related to stakeholder communication, information ambiguity, and the precise definition of system needs. In this context, user stories have gained traction as a structured, user-centered approach to representing requirements. At the same time, there is growing interest in using Large Language Models (LLMs) to automate such tasks. However, before proposing automated solutions, it is essential to understand how professionals currently elicit requirements, what techniques and tools they use, and how open they are to adopting LLMs for user story generation. To investigate this, an online survey was conducted with 21 software industry professionals experienced in requirements engineering. The questionnaire, divided into three sections, explored current practices in requirements elicitation and validation, as well as the use of LLMs in these activities. The results showed that 57.1% of participants currently use user stories in their work. Among them, 66.7% reported using LLMs, primarily Chat-GPT, for tasks such as brainstorming and content review. Of these LLM users, 75% also apply them to user story generation, citing benefits such as increased speed, standardization, and improved story quality. However, limitations were also identified, including lack of context, potential over-reliance, and legal concerns. These findings indicate that there is growing interest in adopting LLMs in requirements engineering, including user story generation, as long as their use aligns with established practices and is supported by effective prompting, contextual input, and data privacy safeguards.

Keywords: Requirements Engineering, User Stories, Large Language Models, Automated Requirements Generation, Survey

References

A. Alhazmi and S. Huang. 2020. Survey on differences of requirements engineering for traditional and agile development processes. In 2020 SoutheastCon. IEEE, 1–9.

L. Belzner, T. Gabor, and M. Wirsing. 2023. Large language model assisted software engineering: prospects, challenges, and a case study. In International Conference on Bridging the Gap between AI and Reality. Springer Nature Switzerland, Cham, 355–374. DOI: 10.1007/978-3-031-46002-9_23

Fabiano Dalpiaz and Sjaak Brinkkemper. 2018. Agile Requirements Engineering with User Stories. In 2018 IEEE 26th International Requirements Engineering Conference (RE). 506–507. DOI: 10.1109/RE.2018.00075

J. Devlin, M. W. Chang, K. Lee, and K. Toutanova. 2018. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. arXiv preprint arXiv:1810.04805 (2018). DOI: 10.48550/arXiv.1810.04805

Hugo Ferreira Martins, Antônio Carvalho de Oliveira Junior, Edna Dias Canedo, Ricardo Ajax Dias Kosloski, Roberto Ávila Paldês, and Edgard Costa Oliveira. 2019. Design thinking: Desafios para elicitação de requisitos de software. Informação 10, 12 (2019), 371. DOI: 10.3390/info10120371

Aleksander Jarzębowicz and Natalia Sitko. 2019. Communication and documentation practices in agile requirements engineering: a survey in polish software industry. In EuroSymposium on Systems Analysis and Design. Springer, 147–158.

Barbara A. Kitchenham and Shari Lawrence Pfleeger. 2008. Pesquisas de Opinião Pessoal. In Guide to Advanced Empirical Software Engineering, Forrest Shull, Janice Singer, and Dag I. K. Sjøberg (Eds.). Springer, London. DOI: 10.1007/978-1-84800-044-5_3

Johan Linåker, Sardar Muhammad Sulaman, Rafael Maiani de Mello, and Martin Höst. 2015. Guidelines for conducting surveys in software engineering. (2015).

G. Lucassen, F. Dalpiaz, J. M. E. M. van der Werf, and S. Brinkkemper. 2016. The Use and Effectiveness of User Stories in Practice. In Requirements Engineering: Foundation for Software Quality, M. Daneva and O. Pastor (Eds.). Lecture Notes in Computer Science, Vol. 9619. Springer, Cham, 187–202. DOI: 10.1007/978-3-319-30282-9_14

G. Lucassen, F. Dalpiaz, J. M. E. M. van der Werf, and et al. 2016. Improving Agile Requirements: The Quality User Story Framework and Tool. Requirements Engineering 21, 4 (2016), 383–403. DOI: 10.1007/s00766-016-0250-x

Jay U Oswal, Harshil T Kanakia, and Devvrat Suktel. 2024. Transforming Software Requirements into User Stories with GPT-3.5-: An AI-Powered Approach. In 2024 2nd International Conference on Intelligent Data Communication Technologies and Internet of Things (IDCIoT). IEEE, 913–920.

Shari Lawrence Pfleeger and Barbara A. Kitchenham. 2001. Principles of Survey Research: Part 1: Turning Lemons into Lemonade. ACM SIGSOFT Software Engineering Notes 26, 6 (2001), 16–18.

Teade Punter, Marcus Ciolkowski, Bernd Freimut, and Isabel John. 2003. Conducting on-line surveys in software engineering. In 2003 International Symposium on Empirical Software Engineering, 2003. ISESE 2003. Proceedings. IEEE, 80–88.

A. Radford, J. Wu, R. Child, D. Luan, D. Amodei, and I. Sutskever. 2019. Language Models are Unsupervised Multitask Learners. Technical Report. OpenAI. [link]

Tajmilur Rahman, Yuecai Zhu, Lamyea Maha, Chanchal Roy, Banani Roy, and Kevin Schneider. 2024. Take Loads Off Your Developers: Automated User Story Generation using Large Language Model. In 2024 IEEE International Conference on Software Maintenance and Evolution (ICSME). 791–801. DOI: 10.1109/ICSME58944.2024.00082

Paul Ralph, Sebastian Baltes, Domenico Bianculli, Yvonne Dittrich, Michael Felderer, Robert Feldt, Antonio Filieri, Carlo Furia, Daniel Graziotin, Pinjia He, Rashina Hoda, Natalia Juristo, Barbara Kitchenham, Romain Robbes, Daniel Méndez Fernández, Jefferson Molleri, Diomidis Spinellis, Miroslaw Staron, Klaas-Jan Stol, and Sira Vegas. 2020. ACM SIGSOFT Empirical Standards. DOI: 10.48550/arXiv.2010.03525

I. Rasheed. 2021. Requirement Engineering Challenges in Agile Software Development. Mathematical Problems in Engineering 2021 (2021), 1–18. DOI: 10.1155/2021/6696695

K. Ronanki, B. Cabrero-Daniel, and C. Berger. 2024. ChatGPT as a Tool for User Story Quality Evaluation: Trustworthy Out of the Box?. In Agile Processes in Software Engineering and Extreme Programming – Workshops (Lecture Notes in Business Information Processing, Vol. 489), P. Kruchten and P. Gregory (Eds.). Springer, Cham. DOI: 10.1007/978-3-031-48550-3_17

Daniel Russo. 2024. Navigating the Complexity of Generative AI Adoption in Software Engineering. ACM Trans. Softw. Eng. Methodol. 33, 5, Article 135 (June 2024), 50 pages. DOI: 10.1145/3652154

Ian Sommerville. 2011. Software Engineering (9th ed.). Pearson, Boston. [link]

Andreas Vogelsang and Jonas Fischbach. 2025. Usando Modelos de Linguagem de Grande Porte para Tarefas de Processamento de Linguagem Natural em Engenharia de Requisitos: Uma Diretriz Sistemática. In Manual de Processamento de Linguagem Natural para Engenharia de Requisitos, Alessia Ferrari and Gaurav Ginde (Eds.). Springer, Cham. DOI: 10.1007/978-3-031-73143-3_16

Stefan Wagner, Daniel Méndez Fernández, Marcos Kalinowski, and Michael Felderer. 2018. Agile requirements engineering in practice: Status quo and critical problems. CLEI Electronic Journal 21, 1 (2018), 6–1.

B. Wake and B. Wake. 2003. Invest in good stories, and Smart Tasks. [link] Accessed: 28 April 2023.

Claes Wohlin, Per Runeson, Martin Höst, Magnus C Ohlsson, Björn Regnell, Anders Wesslén, et al. 2012. Experimentation in software engineering. Vol. 236. Springer.

Z. Zhang, M. Rayhan, T. Herda, M. Goisauf, and P. Abrahamsson. 2024. LLM-based agents for automating the enhancement of user story quality: An early report. In International Conference on Agile Software Development (Cham). Springer Nature Switzerland, 117–126.

Liping Zhao, Waad Alhoshan, Alessio Ferrari, Keletso J. Letsholo, Muideen A. Ajagbe, Erol-Valeriu Chioasca, and Riza T. Batista-Navarro. 2021. Natural Language Processing for Requirements Engineering: A Systematic Mapping Study. ACM Comput. Surv. 54, 3, Article 55 (April 2021), 41 pages. DOI: 10.1145/3444689