A mixed factorial experiment with colors and adaptive web user interfaces to change emotions


As each person feels emotions differently, web interactive systems may deliver personalized experiences trying to support users to achieve a desired emotional state. A common way of doing so is by providing user interface (UI) adaptation. However, the impact of such adaptations on the users’ emotions is not thoroughly studied. In this work, we describe a mixed factorial experiment with 44 participants in which each one should perform two tasks on the web: reading and transcription. Emotional data was provided by users through a form and also facial expressions via webcam. We represented the users’ emotional paths while performing the tasks and analyzed the most common emotional paths performed by participants. Although most participants reached their desired emotional state in both the experimental and placebo groups, we did not have statistical evidence to prove that users were more likely to achieve the desired emotion while using the adapted UI.
Palavras-chave: User interfaces, Adaptation, Emotion, FACS, Web, Experiment


Ahamed Altaboli. 2013. Investigating the effects of font styles on perceived visual aesthetics of website interface design. In International Conference on Human-Computer Interaction. Springer, 549–554.

Victor Alvarez-Cortes, Benjamin E Zayas-Perez, Victor Huga Zarate-Silva, and Jorge A Ramirez Uresti. 2007. Current trends in adaptive user interfaces: Challenges and applications. In Electronics, Robotics and Automotive Mechanics Conference (CERMA 2007). IEEE, 312–317.

Todd Bentley, Lorraine Johnston, and Karola von Baggo. 2005. Evaluation using cued-recall debrief to elicit information about a user’s affective experiences. In Proceedings of the 17th Australia conference on Computer-Human Interaction: Citizens Online: Considerations for Today and the Future. 1–10.

Renata Germano Bianchi. 2016. Studies on emotional aspects in the design of user interfaces. Master’s thesis. Universidade Federal de São Carlos.

Renata Germano Bianchi and Vânia Paula de Almeida Neris. 2015. Which color should i pick? a study on colors and emotions in human-computer interaction. In Proceedings of the 14th Brazilian Symposium on Human Factors in Computing Systems. 1–10.

Renata G Bianchi, Vânia PA Neris, and Anderson L Ara. 2019. Tags vs. observers–a study on emotions tagged and emotions felt with Flickr pictures. Multimedia Tools and Applications 78, 15 (2019), 21805–21826.

Margaret M Bradley and Peter J Lang. 1994. Measuring emotion: the self-assessment manikin and the semantic differential. Journal of behavior therapy and experimental psychiatry 25, 1 (1994), 49–59.

Amani Braham, Félix Buendía, Maha Khemaja, and Faiez Gargouri. 2021. User interface design patterns and ontology models for adaptive mobile applications. Personal and Ubiquitous Computing (2021), 1–17.

John M Carroll. 2006. Human–computer interaction. Encyclopedia of Cognitive Science (2006).

Shiwei Cheng and Ying Liu. 2012. Eye-tracking based adaptive user interface: implicit human-computer interaction for preference indication. Journal on Multimodal User Interfaces 5, 1 (2012), 77–84.

Elena N Chujkova, Andrey R Aidinyan, and Olga L Tsvetkova. 2020. Adaptation Algorithm for Application Menus. Programming and Computer Software 46, 6 (2020), 397–405.

Irina Cristescu 2008. Emotions in human-computer interaction: the role of nonverbal behavior in interactive systems. Revista Informatica Economică nr 2, 46 (2008), 110–116.

Dianne Cyr. 2013. The Encyclopedia of Human-Computer Interaction. Human Computer Interaction-emotion and website design (2013).

Kasra Dalvand and Mohammad Kazemifard. 2012. An adaptive user-interface based on user’s emotion. In 2012 2nd International eConference on Computer and Knowledge Engineering (ICCKE). IEEE, 161–166.

Tatiana Silva de Alencar and Vânia Paula de Almeida Neris. 2014. Towards design guidelines for software applications that collect user data for ubicomp. In Proceedings of the 13th Brazilian Symposium on Human Factors in Computing Systems. 246–254.

Mailson de Queiroz Proença and Vânia Paula de Almeida Neris. 2017. UIFlex: a meta-design solution for the web. In Proceedings of the XVI Brazilian Symposium on Human Factors in Computing Systems. 1–10.

Isaque E De Souza. 2019. Classificação de sinais fisiológicos para inferência do estado emocional de usuários. Master’s thesis. Universidade Federal de São Carlos (UFSCar).

Pieter Desmet. 2003. Measuring emotion: Development and application of an instrument to measure emotional responses to products. In Funology. Springer, 111–123.

Michela Donati, Giulio Mori, and Fabio Paternò. 2020. Understanding the transitions between web interfaces designed to stimulate specific emotions. Universal Access in the Information Society 19, 2 (2020), 391–407.

Ralf Dörner, Stefan Göbel, Wolfgang Effelsberg, and Josef Wiemeyer. 2016. Serious games. Springer.

Sophie Dupuy-Chessa, Yann Laurillau, and Eric Céret. 2016. Considering aesthetics and usability temporalities in a model based development process. In Actes de la 28ième conférence francophone sur l’Interaction Homme-Machine. 25–35.

Paul Ekman and Erika L Rosenberg. 1997. What the face reveals: Basic and applied studies of spontaneous expression using the Facial Action Coding System (FACS). Oxford University Press, USA.

Julián Andrés Galindo. 2018. Perso2U: Exploration of User Emotional States to Drive Interface Adaptation. In International Conference on Technology Trends. Springer, 288–301.

Julián Andrés Galindo, Sophie Dupuy-Chessa, and Éric Céret. 2017. Toward a UI adaptation approach driven by user emotions. In ACHI’2017 - The Tenth International Conference on Advances in Computer-Human Interactions. 12–17.

Giuseppe Ghiani, Marco Manca, and Fabio Paternò. 2015. Authoring context-dependent cross-device user interfaces based on trigger/action rules. In Proceedings of the 14th International Conference on Mobile and Ubiquitous Multimedia. 313–322.

Patícia Deud Guimarães, Fernando Miguelão da Silva, Valter Vieira de Camargo, and Vânia Paula de Almeida Neris. 2022. An easy-to-maintain framework to adapt web user interfaces consideringusers’ emotions. (2022). Not yet submitted..

Thomas Guntz. 2020. Estimating Expertise from Eye Gaze and Emotions. Ph. D. Dissertation. Université Grenoble Alpes [2020-....].

Joakim Gustafson, Linda Bell, Jonas Beskow, Johan Boye, Rolf Carlson, Jens Edlund, Björn Granström, David House, and Mats Wirén. 2000. AdApt—a multimodal conversational dialogue system in an apartment domain. In The Sixth International Conference on Spoken Language Processing (ICSLP), Beijing, China. 134–137.

Marc Hassenzahl, Michael Burmester, and Franz Koller. 2003. AttrakDiff: Ein Fragebogen zur Messung wahrgenommener hedonischer und pragmatischer Qualität. In Mensch & computer 2003. Springer, 187–196.

Christian Herdin and Christian Märtin. 2020. Modeling and Runtime Generation of Situation-Aware Adaptations. In International Conference on Human-Computer Interaction. Springer, 71–81.

Terri L Holtze. 2006. The web designer’s guide to color research. Internet reference services quarterly 11, 1 (2006), 87–101.

Nan Jiang, Xinyu Feng, Hui Liu, and Juan Liu. 2008. Emotional design of web page. In 2008 9th International Conference on Computer-Aided Industrial Design and Conceptual Design. IEEE, 91–95.

Reed Larson and Mihaly Csikszentmihalyi. 2014. The experience sampling method. In Flow and the foundations of positive psychology. Springer, 21–34.

Youn-kyung Lim, Justin Donaldson, Heekyoung Jung, Breanne Kunz, David Royer, Shruti Ramalingam, Sindhia Thirumaran, and Erik Stolterman. 2008. Emotional experience and interaction design. In Affect and emotion in human-computer interaction. Springer, 116–129.

Christine Lætitia Lisetti and Fatma Nasoz. 2004. Using noninvasive wearable computers to recognize human emotions from physiological signals. EURASIP Journal on Advances in Signal Processing 2004, 11 (2004), 929414.

Sascha Mahlke and Michael Minge. 2008. Consideration of multiple components of emotions in human-technology interaction. In Affect and emotion in human-computer interaction. Springer, 51–62.

Sascha Mahlke, Michael Minge, and Manfred Thüring. 2006. Measuring multiple components of emotions in interactive contexts. In CHI’06 extended abstracts on Human factors in computing systems. 1061–1066.

Camila Loiola Brito Maia and Elizabeth Sucupira Furtado. 2019. An approach to analyze user’s emotion in HCI experiments using psychophysiological measures. IEEE Access 7 (2019), 36471–36480.

Silvia U Maier and Marcus Grueschow. 2021. Pupil dilation predicts individual self-regulation success across domains. Scientific reports 11, 1 (2021), 1–18.

Leandro Y Mano, Bruno S Faiçal, Vinícius P Gonçalves, Gustavo Pessin, Pedro H Gomes, André CPLF de Carvalho, and Jó Ueyama. 2020. An intelligent and generic approach for detecting human emotions: a case study with facial expressions. Soft Computing 24, 11 (2020), 8467–8479.

Sascha Meudt, Miriam Schmidt-Wack, Frank Honold, Felix Schüssel, Michael Weber, Friedhelm Schwenker, and Günther Palm. 2016. Going further in affective computing: how emotion recognition can improve adaptive user interaction. In Toward Robotic Socially Believable Behaving Systems-Volume I. Springer, 73–103.

Giulio Mori, Fabio Paternò, and Ferdinando Furci. 2015. Design criteria for stimulating emotions in web applications. In IFIP Conference on Human-Computer Interaction. Springer, 165–182.

Donald A Norman. 2004. Emotional design: Why we love (or hate) everyday things. Basic Civitas Books.

Reinhard Oppermann and R Rasher. 1997. Adaptability and adaptivity in learning systems. Knowledge transfer 2 (1997), 173–179.

Mailson de Queiroz Proença, Vivian Genaro Motti, Kamila Rios da Hora Rodrigues, and Vânia Paula de Almeida Neris. 2021. Coping with Diversity-A System for End-users to Customize Web User Interfaces. Proceedings of the ACM on Human-Computer Interaction 5, EICS (2021), 1–27.

Karen S Quigley, Kristen A Lindquist, and Lisa Feldman Barrett. 2014. Inducing and measuring emotion and affect: Tips, tricks, and secrets. (2014).

Katharina Reinecke and Abraham Bernstein. 2011. Improving performance, perceived usability, and aesthetics with culturally adaptive user interfaces. ACM Transactions on Computer-Human Interaction (TOCHI) 18, 2 (2011), 1–29.

James A Russell. 1980. A circumplex model of affect.Journal of personality and social psychology 39, 6 (1980), 1161.

Andrew Schall. 2014. New Methods for Measuring Emotional Engagement. In International Conference of Design, User Experience, and Usability. Springer, 347–357.

Klaus R Scherer. 2001. Appraisal processes in emotion: theory, method, research. Appraisal considered as a process of multilevel sequential checking 4, 4 (2001), 92–120.

Klaus R Scherer. 2004. Feelings integrate the central representation of appraisal-driven response organization in emotion. In Feelings and emotions: The Amsterdam symposium. 136–157.

Klaus R Scherer. 2005. What are emotions? And how can they be measured?Social science information 44, 4 (2005), 695–729.

Klaus R Scherer 1984. On the nature and function of emotion: A component process approach. Approaches to emotion 2293, 317 (1984), 31.

Letícia G Zacano da Silva, Patrícia Deud Guimarães, Luciana Oliveira de Souza Gomes, and Vânia Paula de Almeida Neris. 2020. A comparative study of users’ subjective feeling collection instruments. In Proceedings of the 19th Brazilian Symposium on Human Factors in Computing Systems. 1–10.

Olga Sourina and Yisi Liu. 2014. EEG-enabled affective human-computer interfaces. In International Conference on Universal Access in Human-Computer Interaction. Springer, 536–547.

Bastiaan Reinier Steunebrink. 2010. The logical structure of emotions. Ph. D. Dissertation. Utrecht University.

Wolfgang Stuerzlinger, Olivier Chapuis, Dusty Phillips, and Nicolas Roussel. 2006. User interface façades: towards fully adaptable user interfaces. In Proceedings of the 19th annual ACM symposium on User interface software and technology. 309–318.

Vasileios Terzis, Christos N Moridis, and Anastasios A Economides. 2010. Measuring instant emotions during a self-assessment test: the use of FaceReader. In Proceedings of the 7th international conference on methods and techniques in behavioral research. 1–4.

Weihong Wang, Zhidong Li, Yang Wang, and Fang Chen. 2013. Indexing cognitive workload based on pupillary response under luminance and emotional changes. In Proceedings of the 2013 international conference on Intelligent user interfaces. 247–256.

Rogério Aparecido Campanari Xavier and Vânia Paula de Almeida Neris. 2012. A Hybrid Evaluation Approach for the Emotional State of Information Systems Users. In ICEIS (3). 45–53.

Rogério Aparecido Campanari Xavier and Vânia Paula de Almeida Neris. 2014. Measuring the Emotional Experience of Users Through a Hybrid Semantic Approach. In IHC - Brazilian Symposium on Human Factors in Computing Systems.226–235.
GUIMARÃES, Patrícia Deud; SILVA, Fernando Miguelão; ARA, Anderson; NERIS, Vânia Almeida. A mixed factorial experiment with colors and adaptive web user interfaces to change emotions. In: SIMPÓSIO BRASILEIRO SOBRE FATORES HUMANOS EM SISTEMAS COMPUTACIONAIS (IHC), 22. , 2023, Maceió/AL. Anais [...]. Porto Alegre: Sociedade Brasileira de Computação, 2023 .