Unveiling Quality in Chatbot Conversations: Quantitative Analysis of Chatbot Requirements

• Geovana Ramos Sousa Silva UnB
• Edna Dias Canedo UnB

Resumo

As conversational assistants and natural language interfaces proliferate, the demand for a precise understanding of quality software requirements for chatbots becomes increasingly critical. In this work, we adopted a quantitative methodology, scrutinizing a dataset composed of conversational requirements from a diverse range of agile projects for chatbot development, and identified meaningful patterns in the language and structure utilized. Our investigation led to significant findings, revealing the importance of structured documentation, conversation flow, and user interaction in the development of chatbots, with the most desired quality attributes being capability, naturalness, straightforwardness, and clarity. In addition, a significant emphasis was placed on feature development and meeting acceptance criteria. The research also illuminated the iterative nature of chatbot development, with a recurrent presence of verbs related to improvement or refactoring. While less pronounced, the roles of documentation and testing in ensuring chatbot quality and effectiveness were also noted. This work provides valuable insights into chatbot requirements management and the significance of quality attributes in chatbot development.

Palavras-chave: software requirements, github issues, quality attributes, chatbot, open source