Perceived emotion recognition from nonverbal communication cues in images and videos
Resumo
Identificar emoções permite que sistemas inteligentes monitorem o comportamento de usuários, levando a uma compreensão mais profunda da pessoa. A percepção emocional é um processo que ocorre naturalmente em humanos por meio da comunicação de sinais não verbais, nos quais características emocionais são comunicadas implicitamente por meio de múltiplos canais. Nesta tese, propomos três técnicas que são respaldadas por evidências da literatura de psicologia de comportamento para o reconhecimento de emoções: (1) uma abordagem de reconhecimento de emoções baseada exclusivamente no contexto situacional, (2) um modelo de linguagem corporal que utiliza características de marcha para prever emoções a partir de estilos de caminhada em vídeos, e (3) um modelo que recebe múltiplos sinais extraídos de expressões faciais, contexto situacional e linguagem corporal para perceber emoções em imagens. Os resultados obtidos por nossos modelos se igualam ao estado da arte, mas com melhorias significativas relacionadas ao custo computacional.Referências
Buck, R. (1991). Motivation, emotion and cognition: A developmental-interactionist view. International review of studies on emotion, 1:101–142.
Chen, J., Yang, T., Huang, Z., Wang, K., Liu, M., and Lyu, C. (2023). Incorporating structured emotion commonsense knowledge and interpersonal relation into context-aware emotion recognition. Applied Intelligence, 53(4):4201–4217.
Costa, W., Macêdo, D., Zanchettin, C., Talavera, E., Figueiredo, L. S., and Teichrieb, V. (2022). A fast multiple cue fusing approach for human emotion recognition. SSRN preprint 4255748.
Ekman, P. (1993). Facial expression and emotion. American psychologist, 48(4):384.
Halford, G. S. and Hine, T. J. (2016). Fundamental differences between perception and cognition aside from cognitive penetrability. Behavioral and Brain Sciences, 39.
Jacob, H., Kreifelts, B., Nizielski, S., Schütz, A., and Wildgruber, D. (2016). Effects of emotional intelligence on the impression of irony created by the mismatch between verbal and nonverbal cues. PloS one, 11(10):e0163211.
Kosti, R., Alvarez, J. M., Recasens, A., and Lapedriza, A. (2017). Emotion recognition in context. IEEE Conference on Computer Vision and Pattern Recognition, pages 1667–1675.
Le, N., Nguyen, K., Nguyen, A., and Le, B. (2021). Global-local attention for emotion recognition. Neural Computing and Applications, pages 1–15.
Lee, J., Kim, S., Kim, S., Park, J., and Sohn, K. (2019). Context-aware emotion recognition networks. IEEE International Conference on Computer Vision, pages 10143–10152.
Marinho, I., Padilha, R., Vitorino, G., Batista, M., Oliveira, Y., Almeida, J. V., Costa, W., Araújo, C., and Teichrieb, V. (2024). Eyes of fear: Leveraging emotion recognition for virtual reality experience. In Proceedings of the 26th Symposium on Virtual and Augmented Reality, pages 90–96.
Mesquita, B., Boiger, M., and De Leersnyder, J. (2017). Doing emotions: The role of culture in everyday emotions. European Review of Social Psychology, 28(1):95–133.
Mesquita, B. and Frijda, N. H. (1992). Cultural variations in emotions: a review. Psychological bulletin, 112(2):179.
Mittal, T., Guhan, P., Bhattacharya, U., Chandra, R., Bera, A., and Manocha, D. (2020). Emoticon: Context-aware multimodal emotion recognition using frege’s principle.
Nes, A., Sundberg, K., and Watzl, S. (2023). The perception/cognition distinction. Inquiry, 66(2):165–195.
Picard, R. W. (2000). Affective computing. MIT press.
Tracy, J. L. and Matsumoto, D. (2008). The spontaneous expression of pride and shame: Evidence for biologically innate nonverbal displays. Proceedings of the National Academy of Sciences, 105(33):11655–11660.
Valli, A. (2007). Natural interaction. White Paper.
Valli, A. (2008). Alessandro Valli - Notes on Natural Interaction.
Zhang, M., Liang, Y., and Ma, H. (2019). Context-aware affective graph reasoning for emotion recognition. In 2019 IEEE International Conference on Multimedia and Expo (ICME), pages 151–156. IEEE.
Chen, J., Yang, T., Huang, Z., Wang, K., Liu, M., and Lyu, C. (2023). Incorporating structured emotion commonsense knowledge and interpersonal relation into context-aware emotion recognition. Applied Intelligence, 53(4):4201–4217.
Costa, W., Macêdo, D., Zanchettin, C., Talavera, E., Figueiredo, L. S., and Teichrieb, V. (2022). A fast multiple cue fusing approach for human emotion recognition. SSRN preprint 4255748.
Ekman, P. (1993). Facial expression and emotion. American psychologist, 48(4):384.
Halford, G. S. and Hine, T. J. (2016). Fundamental differences between perception and cognition aside from cognitive penetrability. Behavioral and Brain Sciences, 39.
Jacob, H., Kreifelts, B., Nizielski, S., Schütz, A., and Wildgruber, D. (2016). Effects of emotional intelligence on the impression of irony created by the mismatch between verbal and nonverbal cues. PloS one, 11(10):e0163211.
Kosti, R., Alvarez, J. M., Recasens, A., and Lapedriza, A. (2017). Emotion recognition in context. IEEE Conference on Computer Vision and Pattern Recognition, pages 1667–1675.
Le, N., Nguyen, K., Nguyen, A., and Le, B. (2021). Global-local attention for emotion recognition. Neural Computing and Applications, pages 1–15.
Lee, J., Kim, S., Kim, S., Park, J., and Sohn, K. (2019). Context-aware emotion recognition networks. IEEE International Conference on Computer Vision, pages 10143–10152.
Marinho, I., Padilha, R., Vitorino, G., Batista, M., Oliveira, Y., Almeida, J. V., Costa, W., Araújo, C., and Teichrieb, V. (2024). Eyes of fear: Leveraging emotion recognition for virtual reality experience. In Proceedings of the 26th Symposium on Virtual and Augmented Reality, pages 90–96.
Mesquita, B., Boiger, M., and De Leersnyder, J. (2017). Doing emotions: The role of culture in everyday emotions. European Review of Social Psychology, 28(1):95–133.
Mesquita, B. and Frijda, N. H. (1992). Cultural variations in emotions: a review. Psychological bulletin, 112(2):179.
Mittal, T., Guhan, P., Bhattacharya, U., Chandra, R., Bera, A., and Manocha, D. (2020). Emoticon: Context-aware multimodal emotion recognition using frege’s principle.
Nes, A., Sundberg, K., and Watzl, S. (2023). The perception/cognition distinction. Inquiry, 66(2):165–195.
Picard, R. W. (2000). Affective computing. MIT press.
Tracy, J. L. and Matsumoto, D. (2008). The spontaneous expression of pride and shame: Evidence for biologically innate nonverbal displays. Proceedings of the National Academy of Sciences, 105(33):11655–11660.
Valli, A. (2007). Natural interaction. White Paper.
Valli, A. (2008). Alessandro Valli - Notes on Natural Interaction.
Zhang, M., Liang, Y., and Ma, H. (2019). Context-aware affective graph reasoning for emotion recognition. In 2019 IEEE International Conference on Multimedia and Expo (ICME), pages 151–156. IEEE.
Publicado
30/09/2025
Como Citar
COSTA, Willams de Lima; MARTINEZ, Estefania T.; FIGUEIREDO, Lucas S.; TEICHRIEB, Veronica.
Perceived emotion recognition from nonverbal communication cues in images and videos. In: CONCURSO DE TESES E DISSERTAÇÕES - SIMPÓSIO DE REALIDADE VIRTUAL E AUMENTADA (SVR), 27. , 2025, Salvador/BA.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2025
.
p. 146-151.
DOI: https://doi.org/10.5753/svr_estendido.2025.15756.
