Musical Hyperlapse: A Multimodal Approach to Accelerate First-Person Videos - Matos, Diognei de; Nascimento, Erickson R. (Anais Estendidos da Conference on Graphics, Patterns and Images (SIBGRAPI); 2022: Anais Estendidos da XXXV Conference on Graphics, Patterns and ImagesDO - 10.5753/sibgrapi.est.2022.23258, 2022)
With the advance in technology and social media usage, first-person recording videos has become a common habit. These videos are usually very long and tiring to watch, bringing the need to speed up them. Despite recent progress of fast-forward methods, they do not consider inserting background music in the videos, which could make them more enjoyable. This thesis presents a new method that creates accelerated videos and includes the background music keeping the same emotion induced by visual and acoustic modalities. Our approach is based on the automatic recognition of emotions induced by music and video contents and an optimization algorithm that maximizes the visual quality of the output video and seeks to match the similarity of the music and the video’s emotions. Quantitative results show that our method achieves the best performance in matching emotion similarity while maintaining the visual quality of the output video compared with other literature methods. Visual results can be seen through the link: https://youtu.be/9ykQa9zhcz8 .