Template-Driven Specification of Requirements for LLM-Based Chatbots

• Caio V. Melo da Silva USP
• Ana R. Cárdenas Maita USP
• Gregorio Assagra de Almeida Filho USP
• Carlos H. Camillo da Silva USP
• Gabriel Dimant USP
• Renato A. Almeida Instituto de Ciência e Tecnologia Itaú
• Guilherme M. Lopes Costa Instituto de Ciência e Tecnologia Itaú
• Anna M. Cintra Araujo Instituto de Ciência e Tecnologia Itaú
• Barbara Correia Dos Santos Santana Instituto de Ciência e Tecnologia Itaú
• Enio Alterman Blay USP
• Sarajane Marques Peres USP

DOI: https://doi.org/10.5753/wbots.2025.14151

Resumo

The integration of large language models (LLMs) into chatbots introduces specific requirements related to reasoning, regulatory constraints, and user-sensitive interactions. This paper presents a template-driven approach for specifying system requirements in LLM-enhanced chatbots. The method uses a structured requirements card that defines the functional and non-functional roles, input-output behaviors, contextual reasoning dependencies, and compliance or operational constraints. By formalizing these requirements, the approach supports alignment between LLM capabilities and system design, facilitates verification of behavior across dialogue flows, and improves traceability within the software architecture. This specification model is particularly suited for domains where consistency, interpretability, and compliance with business rules are critical. As a proof of concept, a set of requirements for a financial chatbot was elicited and organized into cards derived from the template.

Referências

Amershi, S., Begel, A., Bird, C., DeLine, R., Gall, H., Kamar, E., Nagappan, N., Nushi, B., and Zimmermann, T. (2019). Software engeneering for machine learning: A case study. In IEEE/ACM 41st Int. Conf. on Software Eng.: Software Eng. in Pract.

Bommasani, R., Hudson, D. A., Adeli, E., Altman, R., Arora, S., von Arx, S., Bernstein, M. S., Bohg, J., Bosselut, A., Brunskill, E., et al. (2021). On the opportunities and risks of foundation models. Preprint arXiv:2108.07258.

Bourque, P. and Fairley, R. (2004). Swebok. Nd: IEEE Comput. soc.

Cabrera, C., Bastidas, V., Schooling, J., and Lawrence, N. D. (2024). The Systems Engeneering approach in times of large language models. Preprint arXiv:2411.09050.

Dam, S. K., Hong, C. S., Qiao, Y., and Zhang, C. (2024). A complete survey on llm-based ai chatbots. Preprint arXiv:2406.16937.

Gallegos, I. O., Rossi, R. A., Barrow, J., Tanjim, M. M., Kim, S., Dernoncourt, F., Yu, T., Zhang, R., and Ahmed, N. K. (2024). Bias and fairness in large language models: A survey. Comput. Ling.

Gao, Y., Xiong, Y., Gao, X., Jia, K., Pan, J., Bi, Y., Dai, Y., Sun, J., Wang, M., and Wang, H. (2024). Retrieval-augmented generation for large language models: A survey. Preprint arXiv:2312.10997.

Gerstberger, W. S., Silva, W. A. F., and Guedes, G. T. A. (2024). Bart: Uma técnica de elicitação de requisitos para sistemas multiagentes. In Workshop sobre Bots na Eng. de Software.

Gonçalves, L. P. and Sousa, G. R. (2024). Documentation Artifacts For Conversation-Related Requirements Specification in Chatbots. In Anais do WER24-Workshop em Eng. de Requisitos, Buenos Aires, Argentina.

Gupta, S., Ranjan, R., and Singh, S. N. (2025). Comprehensive framework for evaluating conversational ai chatbots. Preprint arXiv:2502.06105.

Jurafsky, D. and Martin, J. H. (2025). Speech and Language Processing: An Introduction to Natural Language Processing, Computational Linguistics, and Speech Recognition with Language Models. Online manuscript released Jan. 12, 2025.

Liu, P., Yuan, W., Fu, J., Jiang, Z., Hayashi, H., and Neubig, G. (2023). Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing. ACM Comput. Surv.

Mafra, M., Nunes, K., Castro, A., Lopes, A., Oran, A. C., Braz Junior, G., Almeida, J., Paiva, A., Silva, A., Rocha, S., et al. (2022). Defining requirements for the development of useful and usable chatbots: An analyisis of quality attributes from academy and industry. In Int. Conf. on Hum.-Comput. Interact.

Martínez-Fernández, S., Bogner, J., Franch, X., Oriol, M., Siebert, J., Trendowicz, A., Vollmer, A. M., and Wagner, S. (2022). Software engeneering for ai-based systems: a survey. ACM Trans. on Software Eng. and Methodol.

Minaee, S., Mikolov, T., Nikzad, N., Chenaghlu, M., Socher, R., Amatriain, X., and Gao, J. (2024). Large language models: A survey. Preprint arXiv:2402.06196.

Mitchell, M., Wu, S., Zaldivar, A., Barnes, P., Vasserman, L., Hutchinson, B., Spitzer, E., Raji, I. D., and Gebru, T. (2019). Model cards for model reporting. In Proc. of the Conf. on Fairness, Account., and Transparency.

Sarker, I. H. (2024). Llm potentiality and awareness: a position paper from the perspective of trustworthy and responsible ai modeling. Discover Artif. Intell.

Solomon, A., Levy, M., Agur-Cohen, D., Younis, M., and Moshe, E. (2024). Requiremets engeneering for llm: The case of digital inquiries application. In 32nd Int. Requir. Eng. Conf. Workshops (REW).

Tatsat, H. and Shater, A. (2025). Beyond the black box: Interpretability of llms in finance. Preprint arXiv:2505.24650.

Wiegers, K. and Beatty, J. (2013). Software requirements. Pearson Education.

Zhao, W. X., Zhou, K., Li, J., Tang, T., Wang, X., Hou, Y., Min, Y., Zhang, B., Zhang, J., Dong, Z., et al. (2023). A survey of large language models. Preprint arXiv:2303.18223.