Mind the Gap Between UX Data and Visualization Proposals: An Approach Emerged From the Grey Literature to Support the Analysis of User Dissatisfaction - Macedo, Maylon Pires; Zaina, Luciana (Anais do Simpósio Brasileiro sobre Fatores Humanos em Sistemas Computacionais (IHC); 2024: Anais do XXIII Simpósio Brasileiro sobre Fatores Humanos em Sistemas Computacionais, 2024)
Software teams usually perform data analysis about user experience data (i.e., UX data) to extract positive and negative issues of users' interaction with the software. The scientific literature has presented forms of visualizing UX data. However, these visualization forms often are not evaluated by software professionals (i.e., designers and developers) in practice to check whether such visualizations can provide enough information to understand users' satisfaction with the product. In addition, software professionals have adapted the visualizations proposed in the literature considering their own learning that emerged from their practice. These professionals' lessons learned have resulted in a significant amount of practical knowledge on UX data visualizations published in UX blogs and websites (known as grey literature). Considering the scattered knowledge available in UX blogs and websites, we searched for publications in the grey literature that contained definitions of UX data and also of visualizations used to explore it. From the 144 articles explored, we found three legs that can support the investigation of causes of users' dissatisfaction reports: the visualization approach, the purpose of using the visualization, and UX data per se. We evaluated with 31 software professionals as a proof of concept on the relevance of the three-leg approach (i.e., visualization, purposes, and UX data) to help software teams identify insights about events that triggered the users' dissatisfaction reports. We selected four users' dissatisfaction reports that emerged from the grey literature analysis and constructed five visualizations to explore UX data related to a mobile app for airline tickets. We chose the airline tickets domain due to the fact that it is a common sense domain that avoids the efforts of interpreting the application domain. From an online questionnaire, we collected the participants' interpretations of the UX data available from the five visualizations. Our primary concern was evaluating how well the visualizations helped participants identify aspects of the mobile application that resulted in reports of user dissatisfaction. Data on participants' interpretation of visualizations were analyzed using descriptive statistics, and open-ended questions on definitions of UX data were analyzed using open coding. The participants evaluated positively the use of the three-leg approach to analyze UX data (i.e., an average of 78% of positive feedback). The findings revealed that most participants recognized that data on product use (e.g., logs and screen recording) was the only suitable relevant UX data source. In addition to evaluating the visualizations positively, a good number of participants (i.e., 84%) pointed out that the proposed visualizations could be appropriated to support their daily work.