Human-ChatBot Interaction: measuring the psychophysiological reactions of chatbot users
Resumo
Affective computing and its applications have gained popularity in recent years. However, recognizing emotions is a challenge. In Computing, the artifacts used to recognize emotions are instruments of self-report or data collection through biological sensors. Interaction through chatbots, in turn, has grown and brings with it challenges related to how effective the system is and how satisfied the user is when using it. We carried out an experiment with the aim of learning more about the characteristics of human-non-human interaction. As a result, the expressive responses of users and the chatbots with whom they interacted are the focus of this human-chatbot interaction paper. We gathered face registration, body movements, self-report, and peripheral signals such as the electrocardiogram (ECG), valence, and arousal estimations of the various emotional changes during the engagement period with the Bob chatbot, a conversational recommendation system, that operates through the WhatsApp app, and recommends restaurants to the user based on information such as location. The results were encouraging; users felt comfortable and were receptive to a new conversational tool and the ECG sensors attached to the thorax, which motivates us to make improvements for future tests.
Palavras-chave:
Emotional responses, chat-bot, sensors, physiological data, self-report data
Referências
Foteini Agrafioti, Dimitris Hatzinakos, and Adam K. Anderson. 2012. ECG Pattern Analysis for Emotion Detection. IEEE Transactions on Affective Computing 3, 1 (2012), 102–115. https://doi.org/10.1109/T-AFFC.2011.28
Amani Albraikan, Basim Hafidh, and Abdulmotaleb El Saddik. 2018. iAware: A real-time emotional biofeedback system based on physiological signals. IEEE Access 6 (2018), 78780–78789. https://doi.org/10.1109/ACCESS.2018.2885279
R S Bexton, H O Vallin, and A J Camm. 1986. Diurnal variation of the QT interval–influence of the autonomic nervous system. Heart 55, 3 (1986), 253–258. arXiv: https://heart.bmj.com/content/55/3/253.full.pdf https://doi.org/10.1136/hrt.55.3.253
Margaret M Bradley and Peter J Lang. 1994. Measuring emotion: the self-assessment manikin and the semantic differential. Journal of behavior therapy and experimental psychiatry 25, 1 (1994), 49–59. https://doi.org/10.1016/0005-7916(94)90063-9
Paul Ekman, Wallace V. Freisen, and Sonia Ancoli. 1980. Facial signs of emotional experience. Journal of Personality and Social Psychology 39 (1980), 1125–1134. https://doi.org/10.1037/h0077722
Elaine CS Hayashi, Julián E Gutiérrez Posada, Vanessa RML Maike, and M Cecília C Baranauskas. 2016. Exploring new formats of the Self-Assessment Manikin in the design with children. In Proceedings of the 15th Brazilian Symposium on Human Factors in Computing Systems. ACM, São Paulo, Brazil, 1–10. https://doi.org/10.1145/3033701.3033728
Paris Hsu, Jingshu Zhao, Kehan Liao, Tianyi Liu, and Chen Wang. 2017. AllergyBot: A Chatbot technology intervention for young adults with food allergies dining out. In Proceedings of the 2017 CHI conference extended abstracts on human factors in computing systems. Association for Computing Machinery, New York, NY, USA, 74–79. https://doi.org/10.1145/3027063.3049270
Lorenz Cuno Klopfenstein, Saverio Delpriori, Silvia Malatini, and Alessandro Bogliolo. 2017. The rise of bots: A survey of conversational interfaces, patterns, and paradigms. In Proceedings of the 2017 conference on designing interactive systems. New York, NY, 555–565. https://doi.org/10.1145/3064663.3064672
Rollin McCraty. 2015. Science of the heart: Exploring the role of the heart in human performance. HeartMath Research Center, Institute of HeartMath, CA.
Sintija Petrovica, Alla Anohina-Naumeca, and Hazım Kemal Ekenel. 2017. Emotion recognition in affective tutoring systems: Collection of ground-truth data. Procedia Computer Science 104 (2017), 437–444. https://doi.org/10.1016/j.procs.2017.01.157
Kanlaya Rattanyu and Makoto Mizukawa. 2011. Emotion Recognition Based on ECG Signals for Service Robots in the Intelligent Space During Daily Life. Journal of Advanced Computational Intelligence and Intelligent Informatics 15, 5 (2011), 582–591. https://doi.org/10.20965/jaciii.2011.p0582
Byron Reeves and Clifford Nass. 1996. The Media Equation: How People Treat Computers, Television, and New Media like Real People and Places. Cambridge University Press, USA. https://doi.org/10.1300/J105v24n03_14
Luca Romeo, Andrea Cavallo, Lucia Pepa, Nadia Berthouze, and Massimiliano Pontil. 2019. Multiple instance learning for emotion recognition using physiological signals. IEEE Transactions on Affective Computing 13, 1 (2019), 389 – 407. https://doi.org/10.1109/TAFFC.2019.2954118
Jim Torresen. 2018. A review of future and ethical perspectives of robotics and AI. Frontiers in Robotics and AI 4 (2018), 75. https://doi.org/10.3389/frobt.2017.00075
Rafael Wampfler, Severin Klingler, Barbara Solenthaler, Victor Schinazi, and Markus Gross. 2019. Affective state prediction in a mobile setting using wearable biometric sensors and stylus. In Proceedings of the 12th International Conference on Educational Data Mining, EDM 2019, Montréal, Canada, July 2-5, 2019. International Educational Data Mining Society (IEDMS) 2019. Université du Québec; Polytechnique Montréal, International Educational Data Mining Society (IEDMS), Montréal, Canada, 198–207. https://doi.org/10.3929/ethz-b-000393912
Tianhua Xie, Mingliang Cao, and Zhigeng Pan. 2020. Applying self-assessment manikin (sam) to evaluate the affective arousal effects of vr games. In Proceedings of the 2020 3rd International Conference on Image and Graphics Processing. ACM, Singapore, 134–138. https://doi.org/10.1145/3383812.3383844
Bobo Zhao, Zhu Wang, Zhiwen Yu, and Bin Guo. 2018. EmotionSense: Emotion recognition based on wearable wristband. In 2018 IEEE SmartWorld, Ubiquitous Intelligence & Computing, Advanced & Trusted Computing, Scalable Computing & Communications, Cloud & Big Data Computing, Internet of People and Smart City Innovation (SmartWorld/SCALCOM/UIC/ATC/CBDCom/IOP/SCI). IEEE, IEEE, Guangzhou, China, 346–355. https://doi.org/10.1109/SmartWorld.2018.00091
Amani Albraikan, Basim Hafidh, and Abdulmotaleb El Saddik. 2018. iAware: A real-time emotional biofeedback system based on physiological signals. IEEE Access 6 (2018), 78780–78789. https://doi.org/10.1109/ACCESS.2018.2885279
R S Bexton, H O Vallin, and A J Camm. 1986. Diurnal variation of the QT interval–influence of the autonomic nervous system. Heart 55, 3 (1986), 253–258. arXiv: https://heart.bmj.com/content/55/3/253.full.pdf https://doi.org/10.1136/hrt.55.3.253
Margaret M Bradley and Peter J Lang. 1994. Measuring emotion: the self-assessment manikin and the semantic differential. Journal of behavior therapy and experimental psychiatry 25, 1 (1994), 49–59. https://doi.org/10.1016/0005-7916(94)90063-9
Paul Ekman, Wallace V. Freisen, and Sonia Ancoli. 1980. Facial signs of emotional experience. Journal of Personality and Social Psychology 39 (1980), 1125–1134. https://doi.org/10.1037/h0077722
Elaine CS Hayashi, Julián E Gutiérrez Posada, Vanessa RML Maike, and M Cecília C Baranauskas. 2016. Exploring new formats of the Self-Assessment Manikin in the design with children. In Proceedings of the 15th Brazilian Symposium on Human Factors in Computing Systems. ACM, São Paulo, Brazil, 1–10. https://doi.org/10.1145/3033701.3033728
Paris Hsu, Jingshu Zhao, Kehan Liao, Tianyi Liu, and Chen Wang. 2017. AllergyBot: A Chatbot technology intervention for young adults with food allergies dining out. In Proceedings of the 2017 CHI conference extended abstracts on human factors in computing systems. Association for Computing Machinery, New York, NY, USA, 74–79. https://doi.org/10.1145/3027063.3049270
Lorenz Cuno Klopfenstein, Saverio Delpriori, Silvia Malatini, and Alessandro Bogliolo. 2017. The rise of bots: A survey of conversational interfaces, patterns, and paradigms. In Proceedings of the 2017 conference on designing interactive systems. New York, NY, 555–565. https://doi.org/10.1145/3064663.3064672
Rollin McCraty. 2015. Science of the heart: Exploring the role of the heart in human performance. HeartMath Research Center, Institute of HeartMath, CA.
Sintija Petrovica, Alla Anohina-Naumeca, and Hazım Kemal Ekenel. 2017. Emotion recognition in affective tutoring systems: Collection of ground-truth data. Procedia Computer Science 104 (2017), 437–444. https://doi.org/10.1016/j.procs.2017.01.157
Kanlaya Rattanyu and Makoto Mizukawa. 2011. Emotion Recognition Based on ECG Signals for Service Robots in the Intelligent Space During Daily Life. Journal of Advanced Computational Intelligence and Intelligent Informatics 15, 5 (2011), 582–591. https://doi.org/10.20965/jaciii.2011.p0582
Byron Reeves and Clifford Nass. 1996. The Media Equation: How People Treat Computers, Television, and New Media like Real People and Places. Cambridge University Press, USA. https://doi.org/10.1300/J105v24n03_14
Luca Romeo, Andrea Cavallo, Lucia Pepa, Nadia Berthouze, and Massimiliano Pontil. 2019. Multiple instance learning for emotion recognition using physiological signals. IEEE Transactions on Affective Computing 13, 1 (2019), 389 – 407. https://doi.org/10.1109/TAFFC.2019.2954118
Jim Torresen. 2018. A review of future and ethical perspectives of robotics and AI. Frontiers in Robotics and AI 4 (2018), 75. https://doi.org/10.3389/frobt.2017.00075
Rafael Wampfler, Severin Klingler, Barbara Solenthaler, Victor Schinazi, and Markus Gross. 2019. Affective state prediction in a mobile setting using wearable biometric sensors and stylus. In Proceedings of the 12th International Conference on Educational Data Mining, EDM 2019, Montréal, Canada, July 2-5, 2019. International Educational Data Mining Society (IEDMS) 2019. Université du Québec; Polytechnique Montréal, International Educational Data Mining Society (IEDMS), Montréal, Canada, 198–207. https://doi.org/10.3929/ethz-b-000393912
Tianhua Xie, Mingliang Cao, and Zhigeng Pan. 2020. Applying self-assessment manikin (sam) to evaluate the affective arousal effects of vr games. In Proceedings of the 2020 3rd International Conference on Image and Graphics Processing. ACM, Singapore, 134–138. https://doi.org/10.1145/3383812.3383844
Bobo Zhao, Zhu Wang, Zhiwen Yu, and Bin Guo. 2018. EmotionSense: Emotion recognition based on wearable wristband. In 2018 IEEE SmartWorld, Ubiquitous Intelligence & Computing, Advanced & Trusted Computing, Scalable Computing & Communications, Cloud & Big Data Computing, Internet of People and Smart City Innovation (SmartWorld/SCALCOM/UIC/ATC/CBDCom/IOP/SCI). IEEE, IEEE, Guangzhou, China, 346–355. https://doi.org/10.1109/SmartWorld.2018.00091
Publicado
07/11/2022
Como Citar
PICOLI, Helen Santos; OLIVEIRA, Cynthya Letícia Teles De; OLIVEIRA, Lucas Padilha M. De; MANZATO, Marcelo G.; RODRIGUES, Kamila Rios Da Hora.
Human-ChatBot Interaction: measuring the psychophysiological reactions of chatbot users. In: BRAZILIAN SYMPOSIUM ON MULTIMEDIA AND THE WEB (WEBMEDIA), 28. , 2022, Curitiba.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2022
.
p. 405-408.
