OggyBug: A Test Automation Tool in Chatbots
Resumo
Context: Motivated by the reduction in operating costs, the use of chatbots to automate customer service has been growing. Chatbots have evolved a lot in terms of technologies used as well as in the different application areas. Problem: As it is a recent technology, there is no tool offers to support chatbot test automation with the possibility of testing context information that happens in the dialogue between the chatbot and the human; the existing tools also lack facilities to integrate different data sources that are used during the tests. Objective: Propose and evaluate a new framework for chatbot testing that considers context information and allows the integration test between different data sources. Method: from the analysis of the lack of existing works reported in the literature, a framework for self-testing of chatbots called OggyBug was proposed, which was used by two chatbots development teams that provided feedback on their use. Results: Construction of the framework called OggyBug that allows implementing, manage and report the results of the execution of automatic tests for chatbots, either through an API or through a web interface, with ease of integrating different sources of information within the automation scripts. After collecting the feedback from the teams that used the framework, we can observe the ease in defining scenarios and repeating the execution of the tests. Conclusion: Testing in context information proved to be important to verify or define the information of the conversation session. The configuration of integration tests proved to be complex, due to the need to configure web services in the chatbot's actions.
