Technologies for Hedonic Aspects Evaluation in Text-based Chatbots: A Systematic Mapping Study - De Souza, Ana Carolina Rossi; Mariano, Pamella Augusta De Lima; Guerino, Guilherme Corredato; Chaves, Ana Paula; Valentim, Natasha Malveira Costa (Anais do Simpósio Brasileiro sobre Fatores Humanos em Sistemas Computacionais (IHC); 2023: Anais do XXII Simpósio Brasileiro sobre Fatores Humanos em Sistemas Computacionais, 2023)
Many studies present and evaluate daily-use technologies ranging from information to conversational systems. One of the technologies that have attracted the attention of researchers is chatbots through text or voice messages. In particular, User eXperience (UX) has been pointed out as one of chatbots leading aspects of evaluation. UX evaluation involves pragmatic and hedonic aspects. The first deals with the usability and efficiency of the system, while the second considers aspects related to the originality, innovation, beauty of the system, and the user’s psychological well-being. Although there are previous studies on usability evaluation and human-computer interaction in conversational systems, the absence of specific works that consider the hedonic aspects of UX in chatbots is evident. Therefore, this paper presents a Systematic Mapping Study investigating UX evaluation technologies (questionnaires, methods, techniques, models, among others) from the hedonic aspect of chatbots. We focused our investigation on studies with chatbots that are activated through text, although they may be able to display click interactions, videos, and images in addition to written text. We discovered 29 different technologies used to evaluate hedonic aspects of UX in chatbots, and the most frequent aspect found is trust. Our study provides relevant data on the researched topic, addressing the specific characteristics of human-chatbot interaction, such as identity and social interaction. Moreover, we highlight gaps in the hedonic aspect evaluation in chatbots, such as a few works investigating the assessment of user emotional state.