TF-MVSA: Multimodal Video Sentiment Analysis Tool using Transfer Learning - Tomita, Victor Akihito Kamada; Marcacini, Ricardo Marcondes (Anais Estendidos do Simpósio Brasileiro de Sistemas Multimídia e Web (WebMedia); 2023: Anais Estendidos do XXIX Simpósio Brasileiro de Sistemas Multimídia e WebDO - 10.5753/webmedia_estendido.2023.235544, 2023)
Existing methods for sentiment analysis in videos rely on extensive training on large labeled datasets, making them expensive and impractical for real-world applications. This challenge becomes even more complex when dealing with labeled data in different modalities. To address these limitations, we proposed a transfer learning method and a computational tool that leverage pre-trained models for each modality and employ modality consensus to automatically annotate video segments. Our tool implements neural networks with attention mechanisms to learn the significance of each modality during the learning process. The experimental results demonstrate that our tool surpasses unimodal methods and remains competitive with multimodal approaches, even when labeled data for analyzing new videos are unavailable. Moreover, the tool is publicly available, thereby serving as a competitive baseline for similar multimodal sentiment analysis methods.