Machine Learning-Assisted Tools for User eXperience Evaluation: A Systematic Mapping Study
Resumo
Context: Information Systems (IS) have grown exponentially, significantly influencing professional and personal environments. Both scenarios require a distinguished User Experience (UX), which generates positive feelings such as loyalty, learning, and satisfaction from end users. Consequently, tools, software, and applications that integrate Machine Learning (ML) techniques with UX are necessary for enhancing the quality of IS and increasing the productivity of UX specialists. Problem: There is a continued need for more experimental evidence regarding the development, employability/ applicability, evaluation, and evolution of current technologies that automate manual tasks performed by experts. Specifically, such technologies aim to reduce workload, eliminate evaluation biases, and identify patterns that might go unnoticed during assessments. Method: This work aims to summarize and characterize, through a Systematic Mapping Study (SMS), the tools that employ ML techniques to assist in the UX evaluation process. To help us, we defined seven sub-questions that will be addressed based on the data collected from the selected studies. Contributions and Impact: Based on the selected studies, we analyzed and characterized the assessment tools to provide a comprehensive understanding for both the academic and professional communities. This work presents the current state of tools that integrate ML techniques for UX evaluation, offering valuable insights into their effectiveness and application within the IS domain.
Palavras-chave:
User Experience, Machine Learning, Systematic Mapping Study
Referências
Abdallah M. H. Abbas, Khairil Imran Ghauth, and Choo-Yee Ting. 2022. User Experience Design Using Machine Learning: A Systematic Review. IEEE Access 10 (2022), 51501–51514. DOI: 10.1109/ACCESS.2022.3173289
Ighoyota Ben Ajenaghughrure, Sonia C Sousa, Ilkka Johannes Kosunen, and David Lamas. 2019. Predictive model to assess user trust: a psycho-physiological approach. In Proceedings of the 10th Indian conference on human-computer interaction. 1–10.
Asma Musabah Alkalbani, Ahmed Mohamed Ghamry, Farookh Khadeer Hussain, and Omar Khadeer Hussain. 2016. Sentiment analysis and classification for software as a service reviews. In 2016 IEEE 30th international conference on advanced information networking and applications (AINA). IEEE, 53–58.
Renata Araújo and R Suzana. 2017. Grand research challenges in information systems in brazil 2016–2026. Brazilian Computer Society. Clodis Boscarioli Renata Araujo and Rita Suzana (2017), 2016–2026.
Zuriana Abu Bakar and Ong Sze Seong. 2020. The Development of Web-Based Emotion Detection System Using Keyboard Actions (EDS-KA). International Journal on Advanced Science, Engineering and Information Technology (2020). [link]
Elsa Bakiu and Emitza Guzman. 2017. Which feature is unusable? Detecting usability and user experience issues from user reviews. In 2017 IEEE 25th international requirements engineering conference workshops (REW). IEEE, 182–187.
A. Batch, Y. Ji, M. Fan, J. Zhao, and N. Elmqvist. 2024. uxSense: Supporting User Experience Analysis with Visualization and Computer Vision. IEEE Transactions on Visualization and Computer Graphics 30, 07 (jul 2024), 3841–3856. DOI: 10.1109/TVCG.2023.3241581
Regina Bernhaupt, Philippe Palanque, Dimitri Drouet, and Celia Martinie. 2019. Enriching task models with usability and user experience evaluation data. In Human-Centered Software Engineering: 7th IFIP WG 13.2 International Working Conference, HCSE 2018, Sophia Antipolis, France, September 3–5, 2018, Revised Selected Papers 7. Springer, 146–163.
Kim Carmona, Erin Finley, and Meng Li. 2018. The Relationship Between User Experience and Machine Learning. Available at SSRN 3173932 (2018).
Helena de Medeiros Caseli and Maria das Graças Volpe Nunes. 2023. Processamento de Linguagem Natural: Conceitos, Técnicas e Aplicações em Português.
Michael Chromik, Florian Lachner, and Andreas Butz. 2020. ML for UX? - An Inventory and Predictions on the Use of Machine Learning Techniques for UX Research. 1–11. DOI: 10.1145/3419249.3420163
Michael Chromik, Florian Lachner, and Andreas Butz. 2020. Ml for ux?-an inventory and predictions on the use of machine learning techniques for UX research. In Proceedings of the 11th Nordic Conference on Human-Computer Interaction: Shaping Experiences, Shaping Society. 1–11.
Ricardo Luiz Hentges Costa, Tales Schifelbein Soares, Gabriel Machado Lunardi, Pedro Henrique Dias Valle, and Williamson Silva. 2024. Professionals’ Perceptions of the Interaction between User Experience and Machine Learning. In Proceedings of the 20th Brazilian Symposium on Information Systems. 1–9.
Roberto Yuri da Silva Franco, Rodrigo Santos do Amor Divino Lima, Rafael do Monte Paixão, Carlos Gustavo Resque dos Santos, and Bianchi Serique Meiguins. 2019. Uxmood—a sentiment analysis and information visualization tool to support the evaluation of usability and user experience. Information 10, 12 (2019), 366.
Roberto Yuri da Silva Franco, Rodrigo Santos do Amor Divino Lima, Rafael do Monte Paixão, Carlos Gustavo Resque dos Santos, and Bianchi Serique Meiguins. 2019. UXmood—A Sentiment Analysis and Information Visualization Tool to Support the Evaluation of Usability and User Experience. Information 10, 12 (2019). DOI: 10.3390/info10120366
Giuseppe Desolda, Andrea Esposito, Rosa Lanzilotti, and Maria F Costabile. 2021. Detecting emotions through machine learning for automatic UX evaluation. In Human-Computer Interaction–INTERACT 2021: 18th IFIP TC 13 International Conference, Bari, Italy, August 30–September 3, 2021, Proceedings, Part III 18. Springer, 270–279.
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. DOI: 10.48550/ARXIV.1810.04805
ISO DIS. 2009. 9241-210: 2010. Ergonomics of human system interaction-Part 210: Human-centred design for interactive systems. International Standardization Organization (ISO). Switzerland (2009).
Darius Drungilas, Ignas Ramašauskas, and Mindaugas Kurmis. 2024. Emotion Recognition in Usability Testing: A Framework for Improving Web Application UI Design. Applied Sciences 14, 11 (2024). DOI: 10.3390/app14114773
Tore Dyba, Torgeir Dingsoyr, and Geir K Hanssen. 2007. Applying systematic reviews to diverse study types: An experience report. In First international symposium on empirical software engineering and measurement (ESEM 2007). IEEE, 225–234.
Katti Faceli, Ana Carolina Lorena, João Gama, Tiago Agostinho de Almeida, and Andre Carlos Ponce de Leon Ferreira de Carvalho. 2021. Inteligência artificial: uma abordagem de aprendizado de máquina. LTC.
Mingming Fan, Yue Li, and Khai N Truong. 2020. Automatic detection of usability problem encounters in think-aloud sessions. ACM Transactions on Interactive Intelligent Systems (TiiS) 10, 2 (2020), 1–24.
Mingming Fan, Ke Wu, Jian Zhao, Yue Li, Winter Wei, and Khai N. Truong. 2020. VisTA: Integrating Machine Intelligence with Visualization to Support the Investigation of Think-Aloud Sessions. IEEE Transactions on Visualization and Computer Graphics 26, 1 (2020), 343–352. DOI: 10.1109/TVCG.2019.2934797
Joseph L Fleiss, Bruce Levin, and Myunghee Cho Paik. 2013. Statistical methods for rates and proportions. John Wiley & Sons.
Roberto Yuri Da Silva Franco, Alexandre Abreu De Freitas, Rodrigo Santos Do Amor Divino Lima, Marcelle Pereira Mota, Carlos Gustavo Resque Dos Santos, and Bianchi Serique Meiguins. 2019. UXmood-A tool to investigate the user experience (UX) based on multimodal Sentiment analysis and information visualization (InfoVis). In 2019 23rd International Conference Information Visualisation (IV). IEEE, 175–180.
Marc Hassenzahl. 2018. The thing and I: understanding the relationship between user and product. Funology 2: from usability to enjoyment (2018), 301–313.
Steffen Hedegaard and Jakob Grue Simonsen. 2013. Extracting usability and user experience information from online user reviews. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Paris, France) (CHI ’13). Association for Computing Machinery, New York, NY, USA, 2089–2098. DOI: 10.1145/2470654.2481286
Steffen Hedegaard and Jakob Grue Simonsen. 2013. Extracting usability and user experience information from online user reviews. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. 2089–2098.
Kasper Hornbæk and Morten Hertzum. 2017. Technology Acceptance and User Experience: A Review of the Experiential Component in HCI. ACM Trans. Comput.-Hum. Interact. 24, 5, Article 33 (oct 2017), 30 pages. DOI: 10.1145/3127358
Jamil Hussain, Wajahat Ali Khan, Taeho Hur, Hafiz Syed Muhammad Bilal, Jaehun Bang, Anees Ul Hassan, Muhammad Afzal, and Sungyoung Lee. 2018. A multimodal deep log-based user experience (UX) platform for UX evaluation. Sensors 18, 5 (2018), 1622.
Aníbal Iung, João Carbonell, Luciano Marchezan, Elder Rodrigues, Maicon Bernardino, Fabio Paulo Basso, and Bruno Medeiros. 2020. Systematic mapping study on domain-specific language development tools. Empirical Software Engineering 25 (2020), 4205–4249.
Jahanzeb Jabbar, Iqra Urooj, Wu JunSheng, and Naqash Azeem. 2019. Real-time sentiment analysis on E-commerce application. In 2019 IEEE 16th international conference on networking, sensing and control (ICNSC). IEEE, 391–396.
Barbara Kitchenham and Stuart Charters. 2007. Guidelines for performing systematic literature reviews in software engineering. (2007).
Marco Kuhrmann, Daniel Méndez Fernández, and Maya Daneva. 2017. On the pragmatic design of literature studies in software engineering: an experiencebased guideline. Empirical software engineering 22, 6 (2017), 2852–2891.
Karl Kurbel and B. Ulrich. 2008. The making of information systems: Software engineering and management in a globalized world. Springer. 1–591 pages. DOI: 10.1007/978-3-540-79261-1
Feng Liang, Fang Hou, Siamak Farshidi, Slinger Jansen, et al. 2023. Sentiment analysis for software quality assessment. In CEURWorkshop Proceedings, Vol. 3567. CEUR WS, 17–24.
Alexandros Liapis, Evanthia Faliagka, Christos P. Antonopoulos, Georgios Keramidas, and Nikolaos Voros. 2021. Advancing Stress Detection Methodology with Deep Learning Techniques Targeting UX Evaluation in AAL Scenarios: Applying Embeddings for Categorical Variables. Electronics 10, 13 (2021). DOI: 10.3390/electronics10131550
Alexandros Liapis, Christos Katsanos, Nikos Karousos, Michalis Xenos, and Theofanis Orphanoudakis. 2019. UDSP+ stress detection based on user-reported emotional ratings and wearable skin conductance sensor. In Adjunct Proceedings of the 2019 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2019 ACM International Symposium on Wearable Computers. 125–128.
Cesar Eduardo Abud Limas. 2009. Sistemas integrados de gestão-ERP: benefícios esperados e problemas encontrados na implantação em pequenas empresas brasileiras. (2009).
Danilo Lira, Gabriel Mac’Hamilton, Gabriel Stadtler, Myllena Almeida, Simone C dos Santos, and Jéssyka Vilela. 2023. The Use of Users’ Personal Data to Improve Online Retail Services: An Analysis from a Systematic Literature Review. In Proceedings of the XIX Brazilian Symposium on Information Systems. 78–85.
Yuwen Lu, Yuewen Yang, Qinyi Zhao, Chengzhi Zhang, and Toby Jia-Jun Li. 2024. AI Assistance for UX: A Literature Review Through Human-Centered AI. arXiv preprint arXiv:2402.06089 (2024).
Walid Maalej, Zijad Kurtanović, Hadeer Nabil, and Christoph Stanik. 2016. On the automatic classification of app reviews. Requirements Engineering 21 (2016), 311–331.
Maylon Macedo. 2024. UX Data Visualization: Supporting Software Professionals in Exploring. In Engineering Interactive Computer Systems: EICS 2023 International Workshops and Doctoral Consortium: Swansea, UK, June 26-27, 2023, Selected Papers, Vol. 14517. Springer Nature, 207.
C. Manning and H. Schutze. 1999. Foundations of Statistical Natural Language Processing. MIT Press. [link]
Christopher D Manning. 2009. An introduction to information retrieval.
Walaa Medhat, Ahmed Hassan, and Hoda Korashy. 2014. Sentiment analysis algorithms and applications: A survey. Ain Shams engineering journal 5, 4 (2014), 1093–1113.
Emilia Mendes, Claes Wohlin, Katia Felizardo, and Marcos Kalinowski. 2020. When to update systematic literature reviews in software engineering. Journal of Systems and Software 167 (2020), 110607.
M Moellendorff, Marc Hassenzahl, and Axel Platz. 2006. Dynamics of user experience: How the perceived quality of mobile phones changes over time. In User experience—towards a unified view. 74–78.
Saif M Mohammad. 2016. Sentiment analysis: Detecting valence, emotions, and other affectual states from text. In Emotion measurement. Elsevier, 201–237.
Ruth Mugge, Hendrik NJ Schifferstein, and Jan PL Schoormans. 2005. Product attachment and product lifetime: The role of personality congruity and fashion. ACR European Advances (2005).
Kazi Md Munim, Iyolita Islam, Mahmuda Khatun, Md Mahboob Karim, and Muhammad Nazrul Islam. 2017. Towards developing a tool for UX evaluation using facial expression. In 2017 3rd international conference on Electrical Information and Communication Technology (EICT). IEEE, 1–6.
Mie Nørgaard and Kasper Hornbæk. 2006. What do usability evaluators do in practice? an explorative study of think-aloud testing. In Proceedings of the 6th Conference on Designing Interactive Systems (University Park, PA, USA) (DIS ’06). Association for Computing Machinery, New York, NY, USA, 209–218. DOI: 10.1145/1142405.1142439
Martin Obaidi, Lukas Nagel, Alexander Specht, and Jil Klünder. 2022. Sentiment analysis tools in software engineering: A systematic mapping study. Information and Software Technology 151 (2022), 107018. DOI: 10.1016/j.infsof.2022.107018
José Palazzo Moreira de Oliveira. 2003. Sistemas de informação e sociedade. Ciência e Cultura 55, 2 (2003), 39–41.
Kai Petersen, Sairam Vakkalanka, and Ludwik Kuzniarz. 2015. Guidelines for conducting systematic mapping studies in software engineering: An update. Information and Software Technology 64 (2015), 1–18.
Orestis Piskioulis, Katerina Tzafilkou, and Anastasios Economides. 2021. Emotion detection through smartphone’s accelerometer and gyroscope sensors. In Proceedings of the 29th ACM Conference on User Modeling, Adaptation and Personalization. 130–137.
Julio Reis, Pollyanna Gonçalves, Matheus Araújo, Adriano Pereira, and Fabricio Benevenuto. 2015. Uma Abordagem Multilíngue para Análise de Sentimentos. In Anais do IV Brazilian Workshop on Social Network Analysis and Mining (Recife). SBC, Porto Alegre, RS, Brasil, . DOI: 10.5753/brasnam.2015.6767
Luis Rivero and Tayana Conte. 2017. A Systematic Mapping Study on Research Contributions on UX Evaluation Technologies. In Proceedings of the XVI Brazilian Symposium on Human Factors in Computing Systems (Joinville, Brazil) (IHC 2017). Association for Computing Machinery, New York, NY, USA, Article 5, 10 pages. DOI: 10.1145/3160504.3160512
Cecilio Merlotti Rodas, Suellen Elise Timm Barros, Rodrigo Ananias da Silva Souza, and Silvana Aparecida Borsetti Gregorio Vidotti. 2022. Análise de sentimentos sobre as vacinas contra Covid-19: um estudo com algoritmo de machine learning em postagens no twitterr. (2022).
Stuart J Russell and Peter Norvig. 2010. Artificial intelligence: a modern approach. Pearson Education Limited.
J Alfredo Sánchez, Ximena Cortés, Oleg Starostenko, Ofelia Cervantes, and Wanggen Wan. 2015. An extensible platform for seamless integration and management of applications for emotion sensing and interpretation. Journal of Ambient Intelligence and Smart Environments 7, 1 (2015), 5–19.
Rosario Sanchis-Font, Maria Jose Castro-Bleda, José-Ángel González, Ferran Pla, and Lluís-F Hurtado. 2021. Cross-Domain polarity models to evaluate user eXperience in E-learning. Neural processing letters 53, 5 (2021), 3199–3215.
Marina Sokolova and Guy Lapalme. 2009. A systematic analysis of performance measures for classification tasks. Information Processing Management 45 (07 2009), 427–437. DOI: 10.1016/j.ipm.2009.03.002
Meishu Song, Adria Mallol-Ragolta, Emilia Parada-Cabaleiro, Zijiang Yang, Shuo Liu, Zhao Ren, Ziping Zhao, and Björn W Schuller. 2021. Frustration recognition from speech during game interaction using wide residual networks. Virtual Reality & Intelligent Hardware 3, 1 (2021), 76–86.
Ehsan Jahangirzadeh Soure, Emily Kuang, Mingming Fan, and Jian Zhao. 2021. CoUX: collaborative visual analysis of think-aloud usability test videos for digital interfaces. IEEE Transactions on Visualization and Computer Graphics 28, 1 (2021), 643–653.
Arnold POS Vermeeren, Effie Lai-Chong Law, Virpi Roto, Marianna Obrist, Jettie Hoonhout, and Kaisa Väänänen-Vainio-Mattila. 2010. User experience evaluation methods: current state and development needs. In Proceedings of the 6th Nordic conference on human-computer interaction: Extending boundaries. 521–530.
Maria Virvou. 2023. Artificial Intelligence and User Experience in reciprocity: Contributions and state of the art. Intelligent Decision Technologies 17, 1 (2023), 73–125.
Tanja Walsh, Jari Varsaluoma, Sari Kujala, Piia Nurkka, Helen Petrie, and Chris Power. 2014. Axe UX: Exploring long-term user experience with iScale and AttrakDiff. In Proceedings of the 18th international academic mindtrek conference: Media business, management, content & services. 32–39.
Jiaqi Zhang, Eiji Kamioka, and Phan Xuan Tan. 2019. Emotions detection of user experience (Ux) for mobile augmented reality (mar) applications. International Journal of Advanced Trends in Computer Science and Engineering 8, 1.4 S1 (2019), 63–67.
Ighoyota Ben Ajenaghughrure, Sonia C Sousa, Ilkka Johannes Kosunen, and David Lamas. 2019. Predictive model to assess user trust: a psycho-physiological approach. In Proceedings of the 10th Indian conference on human-computer interaction. 1–10.
Asma Musabah Alkalbani, Ahmed Mohamed Ghamry, Farookh Khadeer Hussain, and Omar Khadeer Hussain. 2016. Sentiment analysis and classification for software as a service reviews. In 2016 IEEE 30th international conference on advanced information networking and applications (AINA). IEEE, 53–58.
Renata Araújo and R Suzana. 2017. Grand research challenges in information systems in brazil 2016–2026. Brazilian Computer Society. Clodis Boscarioli Renata Araujo and Rita Suzana (2017), 2016–2026.
Zuriana Abu Bakar and Ong Sze Seong. 2020. The Development of Web-Based Emotion Detection System Using Keyboard Actions (EDS-KA). International Journal on Advanced Science, Engineering and Information Technology (2020). [link]
Elsa Bakiu and Emitza Guzman. 2017. Which feature is unusable? Detecting usability and user experience issues from user reviews. In 2017 IEEE 25th international requirements engineering conference workshops (REW). IEEE, 182–187.
A. Batch, Y. Ji, M. Fan, J. Zhao, and N. Elmqvist. 2024. uxSense: Supporting User Experience Analysis with Visualization and Computer Vision. IEEE Transactions on Visualization and Computer Graphics 30, 07 (jul 2024), 3841–3856. DOI: 10.1109/TVCG.2023.3241581
Regina Bernhaupt, Philippe Palanque, Dimitri Drouet, and Celia Martinie. 2019. Enriching task models with usability and user experience evaluation data. In Human-Centered Software Engineering: 7th IFIP WG 13.2 International Working Conference, HCSE 2018, Sophia Antipolis, France, September 3–5, 2018, Revised Selected Papers 7. Springer, 146–163.
Kim Carmona, Erin Finley, and Meng Li. 2018. The Relationship Between User Experience and Machine Learning. Available at SSRN 3173932 (2018).
Helena de Medeiros Caseli and Maria das Graças Volpe Nunes. 2023. Processamento de Linguagem Natural: Conceitos, Técnicas e Aplicações em Português.
Michael Chromik, Florian Lachner, and Andreas Butz. 2020. ML for UX? - An Inventory and Predictions on the Use of Machine Learning Techniques for UX Research. 1–11. DOI: 10.1145/3419249.3420163
Michael Chromik, Florian Lachner, and Andreas Butz. 2020. Ml for ux?-an inventory and predictions on the use of machine learning techniques for UX research. In Proceedings of the 11th Nordic Conference on Human-Computer Interaction: Shaping Experiences, Shaping Society. 1–11.
Ricardo Luiz Hentges Costa, Tales Schifelbein Soares, Gabriel Machado Lunardi, Pedro Henrique Dias Valle, and Williamson Silva. 2024. Professionals’ Perceptions of the Interaction between User Experience and Machine Learning. In Proceedings of the 20th Brazilian Symposium on Information Systems. 1–9.
Roberto Yuri da Silva Franco, Rodrigo Santos do Amor Divino Lima, Rafael do Monte Paixão, Carlos Gustavo Resque dos Santos, and Bianchi Serique Meiguins. 2019. Uxmood—a sentiment analysis and information visualization tool to support the evaluation of usability and user experience. Information 10, 12 (2019), 366.
Roberto Yuri da Silva Franco, Rodrigo Santos do Amor Divino Lima, Rafael do Monte Paixão, Carlos Gustavo Resque dos Santos, and Bianchi Serique Meiguins. 2019. UXmood—A Sentiment Analysis and Information Visualization Tool to Support the Evaluation of Usability and User Experience. Information 10, 12 (2019). DOI: 10.3390/info10120366
Giuseppe Desolda, Andrea Esposito, Rosa Lanzilotti, and Maria F Costabile. 2021. Detecting emotions through machine learning for automatic UX evaluation. In Human-Computer Interaction–INTERACT 2021: 18th IFIP TC 13 International Conference, Bari, Italy, August 30–September 3, 2021, Proceedings, Part III 18. Springer, 270–279.
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. DOI: 10.48550/ARXIV.1810.04805
ISO DIS. 2009. 9241-210: 2010. Ergonomics of human system interaction-Part 210: Human-centred design for interactive systems. International Standardization Organization (ISO). Switzerland (2009).
Darius Drungilas, Ignas Ramašauskas, and Mindaugas Kurmis. 2024. Emotion Recognition in Usability Testing: A Framework for Improving Web Application UI Design. Applied Sciences 14, 11 (2024). DOI: 10.3390/app14114773
Tore Dyba, Torgeir Dingsoyr, and Geir K Hanssen. 2007. Applying systematic reviews to diverse study types: An experience report. In First international symposium on empirical software engineering and measurement (ESEM 2007). IEEE, 225–234.
Katti Faceli, Ana Carolina Lorena, João Gama, Tiago Agostinho de Almeida, and Andre Carlos Ponce de Leon Ferreira de Carvalho. 2021. Inteligência artificial: uma abordagem de aprendizado de máquina. LTC.
Mingming Fan, Yue Li, and Khai N Truong. 2020. Automatic detection of usability problem encounters in think-aloud sessions. ACM Transactions on Interactive Intelligent Systems (TiiS) 10, 2 (2020), 1–24.
Mingming Fan, Ke Wu, Jian Zhao, Yue Li, Winter Wei, and Khai N. Truong. 2020. VisTA: Integrating Machine Intelligence with Visualization to Support the Investigation of Think-Aloud Sessions. IEEE Transactions on Visualization and Computer Graphics 26, 1 (2020), 343–352. DOI: 10.1109/TVCG.2019.2934797
Joseph L Fleiss, Bruce Levin, and Myunghee Cho Paik. 2013. Statistical methods for rates and proportions. John Wiley & Sons.
Roberto Yuri Da Silva Franco, Alexandre Abreu De Freitas, Rodrigo Santos Do Amor Divino Lima, Marcelle Pereira Mota, Carlos Gustavo Resque Dos Santos, and Bianchi Serique Meiguins. 2019. UXmood-A tool to investigate the user experience (UX) based on multimodal Sentiment analysis and information visualization (InfoVis). In 2019 23rd International Conference Information Visualisation (IV). IEEE, 175–180.
Marc Hassenzahl. 2018. The thing and I: understanding the relationship between user and product. Funology 2: from usability to enjoyment (2018), 301–313.
Steffen Hedegaard and Jakob Grue Simonsen. 2013. Extracting usability and user experience information from online user reviews. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Paris, France) (CHI ’13). Association for Computing Machinery, New York, NY, USA, 2089–2098. DOI: 10.1145/2470654.2481286
Steffen Hedegaard and Jakob Grue Simonsen. 2013. Extracting usability and user experience information from online user reviews. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. 2089–2098.
Kasper Hornbæk and Morten Hertzum. 2017. Technology Acceptance and User Experience: A Review of the Experiential Component in HCI. ACM Trans. Comput.-Hum. Interact. 24, 5, Article 33 (oct 2017), 30 pages. DOI: 10.1145/3127358
Jamil Hussain, Wajahat Ali Khan, Taeho Hur, Hafiz Syed Muhammad Bilal, Jaehun Bang, Anees Ul Hassan, Muhammad Afzal, and Sungyoung Lee. 2018. A multimodal deep log-based user experience (UX) platform for UX evaluation. Sensors 18, 5 (2018), 1622.
Aníbal Iung, João Carbonell, Luciano Marchezan, Elder Rodrigues, Maicon Bernardino, Fabio Paulo Basso, and Bruno Medeiros. 2020. Systematic mapping study on domain-specific language development tools. Empirical Software Engineering 25 (2020), 4205–4249.
Jahanzeb Jabbar, Iqra Urooj, Wu JunSheng, and Naqash Azeem. 2019. Real-time sentiment analysis on E-commerce application. In 2019 IEEE 16th international conference on networking, sensing and control (ICNSC). IEEE, 391–396.
Barbara Kitchenham and Stuart Charters. 2007. Guidelines for performing systematic literature reviews in software engineering. (2007).
Marco Kuhrmann, Daniel Méndez Fernández, and Maya Daneva. 2017. On the pragmatic design of literature studies in software engineering: an experiencebased guideline. Empirical software engineering 22, 6 (2017), 2852–2891.
Karl Kurbel and B. Ulrich. 2008. The making of information systems: Software engineering and management in a globalized world. Springer. 1–591 pages. DOI: 10.1007/978-3-540-79261-1
Feng Liang, Fang Hou, Siamak Farshidi, Slinger Jansen, et al. 2023. Sentiment analysis for software quality assessment. In CEURWorkshop Proceedings, Vol. 3567. CEUR WS, 17–24.
Alexandros Liapis, Evanthia Faliagka, Christos P. Antonopoulos, Georgios Keramidas, and Nikolaos Voros. 2021. Advancing Stress Detection Methodology with Deep Learning Techniques Targeting UX Evaluation in AAL Scenarios: Applying Embeddings for Categorical Variables. Electronics 10, 13 (2021). DOI: 10.3390/electronics10131550
Alexandros Liapis, Christos Katsanos, Nikos Karousos, Michalis Xenos, and Theofanis Orphanoudakis. 2019. UDSP+ stress detection based on user-reported emotional ratings and wearable skin conductance sensor. In Adjunct Proceedings of the 2019 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2019 ACM International Symposium on Wearable Computers. 125–128.
Cesar Eduardo Abud Limas. 2009. Sistemas integrados de gestão-ERP: benefícios esperados e problemas encontrados na implantação em pequenas empresas brasileiras. (2009).
Danilo Lira, Gabriel Mac’Hamilton, Gabriel Stadtler, Myllena Almeida, Simone C dos Santos, and Jéssyka Vilela. 2023. The Use of Users’ Personal Data to Improve Online Retail Services: An Analysis from a Systematic Literature Review. In Proceedings of the XIX Brazilian Symposium on Information Systems. 78–85.
Yuwen Lu, Yuewen Yang, Qinyi Zhao, Chengzhi Zhang, and Toby Jia-Jun Li. 2024. AI Assistance for UX: A Literature Review Through Human-Centered AI. arXiv preprint arXiv:2402.06089 (2024).
Walid Maalej, Zijad Kurtanović, Hadeer Nabil, and Christoph Stanik. 2016. On the automatic classification of app reviews. Requirements Engineering 21 (2016), 311–331.
Maylon Macedo. 2024. UX Data Visualization: Supporting Software Professionals in Exploring. In Engineering Interactive Computer Systems: EICS 2023 International Workshops and Doctoral Consortium: Swansea, UK, June 26-27, 2023, Selected Papers, Vol. 14517. Springer Nature, 207.
C. Manning and H. Schutze. 1999. Foundations of Statistical Natural Language Processing. MIT Press. [link]
Christopher D Manning. 2009. An introduction to information retrieval.
Walaa Medhat, Ahmed Hassan, and Hoda Korashy. 2014. Sentiment analysis algorithms and applications: A survey. Ain Shams engineering journal 5, 4 (2014), 1093–1113.
Emilia Mendes, Claes Wohlin, Katia Felizardo, and Marcos Kalinowski. 2020. When to update systematic literature reviews in software engineering. Journal of Systems and Software 167 (2020), 110607.
M Moellendorff, Marc Hassenzahl, and Axel Platz. 2006. Dynamics of user experience: How the perceived quality of mobile phones changes over time. In User experience—towards a unified view. 74–78.
Saif M Mohammad. 2016. Sentiment analysis: Detecting valence, emotions, and other affectual states from text. In Emotion measurement. Elsevier, 201–237.
Ruth Mugge, Hendrik NJ Schifferstein, and Jan PL Schoormans. 2005. Product attachment and product lifetime: The role of personality congruity and fashion. ACR European Advances (2005).
Kazi Md Munim, Iyolita Islam, Mahmuda Khatun, Md Mahboob Karim, and Muhammad Nazrul Islam. 2017. Towards developing a tool for UX evaluation using facial expression. In 2017 3rd international conference on Electrical Information and Communication Technology (EICT). IEEE, 1–6.
Mie Nørgaard and Kasper Hornbæk. 2006. What do usability evaluators do in practice? an explorative study of think-aloud testing. In Proceedings of the 6th Conference on Designing Interactive Systems (University Park, PA, USA) (DIS ’06). Association for Computing Machinery, New York, NY, USA, 209–218. DOI: 10.1145/1142405.1142439
Martin Obaidi, Lukas Nagel, Alexander Specht, and Jil Klünder. 2022. Sentiment analysis tools in software engineering: A systematic mapping study. Information and Software Technology 151 (2022), 107018. DOI: 10.1016/j.infsof.2022.107018
José Palazzo Moreira de Oliveira. 2003. Sistemas de informação e sociedade. Ciência e Cultura 55, 2 (2003), 39–41.
Kai Petersen, Sairam Vakkalanka, and Ludwik Kuzniarz. 2015. Guidelines for conducting systematic mapping studies in software engineering: An update. Information and Software Technology 64 (2015), 1–18.
Orestis Piskioulis, Katerina Tzafilkou, and Anastasios Economides. 2021. Emotion detection through smartphone’s accelerometer and gyroscope sensors. In Proceedings of the 29th ACM Conference on User Modeling, Adaptation and Personalization. 130–137.
Julio Reis, Pollyanna Gonçalves, Matheus Araújo, Adriano Pereira, and Fabricio Benevenuto. 2015. Uma Abordagem Multilíngue para Análise de Sentimentos. In Anais do IV Brazilian Workshop on Social Network Analysis and Mining (Recife). SBC, Porto Alegre, RS, Brasil, . DOI: 10.5753/brasnam.2015.6767
Luis Rivero and Tayana Conte. 2017. A Systematic Mapping Study on Research Contributions on UX Evaluation Technologies. In Proceedings of the XVI Brazilian Symposium on Human Factors in Computing Systems (Joinville, Brazil) (IHC 2017). Association for Computing Machinery, New York, NY, USA, Article 5, 10 pages. DOI: 10.1145/3160504.3160512
Cecilio Merlotti Rodas, Suellen Elise Timm Barros, Rodrigo Ananias da Silva Souza, and Silvana Aparecida Borsetti Gregorio Vidotti. 2022. Análise de sentimentos sobre as vacinas contra Covid-19: um estudo com algoritmo de machine learning em postagens no twitterr. (2022).
Stuart J Russell and Peter Norvig. 2010. Artificial intelligence: a modern approach. Pearson Education Limited.
J Alfredo Sánchez, Ximena Cortés, Oleg Starostenko, Ofelia Cervantes, and Wanggen Wan. 2015. An extensible platform for seamless integration and management of applications for emotion sensing and interpretation. Journal of Ambient Intelligence and Smart Environments 7, 1 (2015), 5–19.
Rosario Sanchis-Font, Maria Jose Castro-Bleda, José-Ángel González, Ferran Pla, and Lluís-F Hurtado. 2021. Cross-Domain polarity models to evaluate user eXperience in E-learning. Neural processing letters 53, 5 (2021), 3199–3215.
Marina Sokolova and Guy Lapalme. 2009. A systematic analysis of performance measures for classification tasks. Information Processing Management 45 (07 2009), 427–437. DOI: 10.1016/j.ipm.2009.03.002
Meishu Song, Adria Mallol-Ragolta, Emilia Parada-Cabaleiro, Zijiang Yang, Shuo Liu, Zhao Ren, Ziping Zhao, and Björn W Schuller. 2021. Frustration recognition from speech during game interaction using wide residual networks. Virtual Reality & Intelligent Hardware 3, 1 (2021), 76–86.
Ehsan Jahangirzadeh Soure, Emily Kuang, Mingming Fan, and Jian Zhao. 2021. CoUX: collaborative visual analysis of think-aloud usability test videos for digital interfaces. IEEE Transactions on Visualization and Computer Graphics 28, 1 (2021), 643–653.
Arnold POS Vermeeren, Effie Lai-Chong Law, Virpi Roto, Marianna Obrist, Jettie Hoonhout, and Kaisa Väänänen-Vainio-Mattila. 2010. User experience evaluation methods: current state and development needs. In Proceedings of the 6th Nordic conference on human-computer interaction: Extending boundaries. 521–530.
Maria Virvou. 2023. Artificial Intelligence and User Experience in reciprocity: Contributions and state of the art. Intelligent Decision Technologies 17, 1 (2023), 73–125.
Tanja Walsh, Jari Varsaluoma, Sari Kujala, Piia Nurkka, Helen Petrie, and Chris Power. 2014. Axe UX: Exploring long-term user experience with iScale and AttrakDiff. In Proceedings of the 18th international academic mindtrek conference: Media business, management, content & services. 32–39.
Jiaqi Zhang, Eiji Kamioka, and Phan Xuan Tan. 2019. Emotions detection of user experience (Ux) for mobile augmented reality (mar) applications. International Journal of Advanced Trends in Computer Science and Engineering 8, 1.4 S1 (2019), 63–67.
Publicado
19/05/2025
Como Citar
SOARES, Tales Schifelbein; COSTA, Ricardo Luiz Hentges; SOARES, Estefano; CALDERON, Ivanilse; LUNARDI, Gabriel Machado; VALLE, Pedro Henrique Dias; GUEDES, Gilleanes T. A.; SILVA, Williamson.
Machine Learning-Assisted Tools for User eXperience Evaluation: A Systematic Mapping Study. In: SIMPÓSIO BRASILEIRO DE SISTEMAS DE INFORMAÇÃO (SBSI), 21. , 2025, Recife/PE.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2025
.
p. 379-388.
DOI: https://doi.org/10.5753/sbsi.2025.246512.
