Semi-Automatic Authoring of Sensory Effects in 360° Videos

• Raphael Abreu UFF
• Joel A. F. dos Santos CEFET-RJ
• Débora C. Muchaluat-Saade UFF

DOI: https://doi.org/10.5753/webmedia_estendido.2025.16432

Abstract

Creating multisensory experiences (mulsemedia) in 360° videos faces the challenge of synchronizing effects, a process that is laborious when manual and inflexible when fully automatic. This thesis proposes a semi-automatic approach that integrates deep learning (DL) recommendations while maintaining author control. The core proposal is a scalable cloud architecture with multiple recognition modules and an object tracking algorithm for spatial continuity, implemented in the SensorySynch360 web tool. A study with 81 users demonstrated that the approach significantly reduces authoring effort and time, while preserving usability and quality of experience.

Keywords: Autoria 360°, mulsemedia, autoria semi-automática, efeitos sensoriais, análise de conteúdo

References

Raphael Silva de Abreu, Douglas Mattos, Joel dos Santos, Gheorghita Ghinea, and Débora Christina Muchaluat-Saade. 2021. Toward Content-Driven Intelligent Authoring of Mulsemedia Applications. IEEE MultiMedia 28, 1 (2021), 7–16. DOI: 10.1109/MMUL.2020.3011383

Douglas Paulo de Mattos, Débora C Muchaluat-Saade, and Gheorghita Guinea. 2020. An Approach for Authoring Mulsemedia Documents Based on Events. In International Conference on Computing, Networking and Communications, 2020. IEEE, 7.

Gheorghita Ghinea, Christian Timmerer,Weisi Lin, and Stephen R. Gulliver. 2014. Mulsemedia : State of the Art, Perspectives, and Challenges. ACM Transactions on Multimedia Computing, Communications, and Applications 11, 1s (2014), 1–23. DOI: 10.1145/2617994