Exploring User Experience Factors for Mobile Devices in Textual Data from Questionnaires
Abstract
The evaluation of User Experience (UX) prior to product launch provides insights into end-user acceptance indicators and helps identify potential issues. Therefore, it is important to conduct studies that evaluate user group perceptions, supported by various methodological approaches. Among these, questionnaires may include questions addressing both quantitative and qualitative aspects of user perception. Textual data related to qualitative aspects can be systematically analyzed to identify UX characteristics that contribute to a broader understanding of user perception. This paper presents an experience report on the classification of UX factors based on textual data collected through questionnaires. The purpose of this practice is driven by the belief that a structured analysis of UX factors can deepen the understanding of such data. We analyzed questionnaire results from two mobile device projects involving a total of 92 users, who were the respondents of the questionnaires. For this classification, 26 practitioners analyzed the textual data reported in the questionnaires. The results indicate that UX factors can provide deeper insights into usability and user satisfaction across different components of the devices. Additionally, the practitioners considered this type of classification useful for understanding the overall quality of the projects through questionnaires.
References
Irene Celino and Gloria Re Calegari. 2020. Submitting surveys via a conversational interface: An evaluation of user acceptance and approach effectiveness. International Journal of Human-Computer Studies 139 (2020), 102410.
Adriana Lopes Damian, Marcelo Pereira, Bruno Luz, and João Leite. 2022. Avaliação da Experiência de Usuários por meio da Abordagem Dogfooding. In Simpósio Brasileiro sobre Fatores Humanos em Sistemas Computacionais (IHC). SBC, 17–23.
Adriana Lopes Damian, Ligia Teixeira, Cinthia Carrenho, Bruna Ferreira, Ariel Bentes, Gustavo Tordin, Graziela Martin, Lucas Castro, Bruna Brotto, et al. 2023. Exploring UX Factors through the Dogfooding Approach: An Experience Report. Proceedings of the XXII Brazilian Symposium on Software Quality (2023).
ISO DIS. 2010. 9241-210 2010. Ergonomics of human system interaction-part 210: Human-centred design for interactive systems. Standard. International Organization for Standardization, Tech. rep. International (2010).
Warren Harrison. 2006. Eating your own dog food. IEEE Software 23, 3 (2006), 5–7.
Marc Hassenzahl. 2018. The thing and I: understanding the relationship between user and product. Funology 2: from usability to enjoyment (2018), 301–313.
Leonardo Marques, Patrícia Matsubara, Walter Nakamura, Igor Wiese, Luciana Zaina, and Tayana Conte. 2019. UX-Tips: A UX evaluation technique to support the identification of software application problems. In Proceedings of the XXXIII Brazilian Symposium on Software Engineering. 224–233.
Cecilie Grace Møller, Ke En Ang, María de Lourdes Bongiovanni, Md Saifuddin Khalid, and Jiayan Wu. 2025. Metrics of Success: Evaluating User Satisfaction in AI Chatbots. In Proceedings of the 2024 8th International Conference on Advances in Artificial Intelligence (ICAAI ’24). Association for Computing Machinery, New York, NY, USA, 168–173. DOI: 10.1145/3704137.3704182
Walter Takashi Nakamura, Iftekhar Ahmed, David Redmiles, Edson Oliveira, David Fernandes, Elaine HT de Oliveira, and Tayana Conte. 2021. Are UX evaluation methods providing the same big picture? Sensors 21, 10 (2021), 3480.
Walter T Nakamura, Leonardo C Marques, David Redmiles, Elaine HT de Oliveira, and Tayana Conte. 2023. Investigating the Influence of Different Factors on the UX Evaluation of a Mobile Application. International Journal of Human–Computer Interaction 39, 20 (2023), 3948–3968.
S Rajeshkumar, Ridha Omar, and Murni Mahmud. 2013. Taxonomies of user experience (UX) evaluation methods. In 2013 international conference on research and innovation in information systems (icriis). IEEE, 533–538.
Luis Rivero and Tayana Conte. 2017. A systematic mapping study on research contributions on UX evaluation technologies. In Proceedings of the XVI Brazilian symposium on human factors in computing systems. 1–10.
Luka Rukonić, Vincent Kervyn de Meerendré, and Suzanne Kieffer. 2019. Measuring UX capability and maturity in organizations. In International Conference on Human-Computer Interaction. Springer, 346–365.
Edilson Silva, Edgar Tanaka, and Gustavo Tordin. 2019. Dogfooding: " eating our own dog food" in a large global mobile industry player. In Proceedings of the 14th International Conference on Global Software Engineering. 52–57.
KE Soderquist, L Tirabeni, and P Pisano. 2016. Employee engagement practices in support of open innovation. In 3rd Annual World Open Innovation Conference. 15–16.
Paweł Weichbroth and Anna Baj-Rogowska. 2019. Do online reviews reveal mobile application usability and user experience? The case of WhatsApp. In 2019 Federated Conference on Computer Science and Information Systems (FedCSIS). IEEE, 747–754.
