An Evaluation Framework for User Experience Using Eye Tracking, Mouse Tracking, Keyboard Input, and Artificial Intelligence: A Case Study

• Kennedy E. S. Souza Federal University of Pará http://orcid.org/0000-0003-0056-2436
• Igor L. Aviz Federal University of Pará http://orcid.org/0000-0002-1260-0009
• Harold D. De Mello Jr. State University of Rio de Janeiro https://orcid.org/0000-0001-6841-1154
• Karla Figueiredo State University of Rio de Janeiro http://orcid.org/0000-0001-8420-3937
• Marley M. B. R. Vellasco Pontifical Catholic University of Rio de Janeiro http://orcid.org/0000-0002-9790-1328
• Fernando Augusto R. Costa Federal University of Pará https://orcid.org/0000-0002-0226-7505
• Marcos C. R. Seruffo Federal University of Pará https://orcid.org/0000-0002-8106-0560

DOI: https://doi.org/10.5753/ihc_estendido.2022.224361

Abstract

User eXperience (UX) has been used to achieve improvements in digital information systems based on how people perceive them. In particular, this paper establishes a framework that employs methods for eye and mouse tracking, keyboard input, self-assessment questionnaire and artificial intelligence algo rithms to evaluate user experience and categorize users in terms of performance profiles. The results obtained with this framework are artifacts that can be used to support customizations of the User Interface (UI) on the websites. Moreover, the established framework is generic and flexible and can be applied to any information system, such as the case study shown in the website of the Federal Revenue of Brazil (RFB). The main objectives of this paper are as follows: (i) to set out a powerful UX framework based on three tracking techniques – the AIT2-UX; (ii) to provide the T2-UXT to collect, collate, process and visualize data obtained from users’ interactions (iii) to use and compare machine learning algo rithms with the classification of user performance profiles; (iv) to use the artifacts generated by the framework to manually customize the UI with the website.

Keywords: User Experience, Assessment Methods, Artificial Intelligence, Eye Tracking, Mouse and Keyboard Tracking

References

Laura Maria Badea. 2014. Predicting consumer behavior with artificial neural networks. Procedia Economics and Finance 15 (2014), 238–246.

Adriano Bernardo Renzi. 2017. UX heuristics for cross-channel interactive scenarios. In International Conference of Design, User Experience, and Usability. Springer, 481–491.

K. E. S. Souza, M. C. R. Seruffo, H. D. De Mello, D. D. S. Souza, and M. M. B. R. Vellasco. 2019. User Experience Evaluation Using Mouse Tracking and Artificial Intelligence. IEEE Access 7 (2019), 96506–96515. Issue 7. https://doi.org/10.1109/ACCESS.2019.2927860