SensoryX ’22 Workshop on Multisensory Experiences at ACM IMX ’22
Abstract
Our interactions with the world are multisensory in nature - the senses move us through spaces, mix with our memories and are constantly connected by our brains. Focused only on vision for a long time, the field of human computer interaction (HCI) started to meaningfully bring together all our senses in designing interactions for a variety of media. With this workshop, we look at different aspects of multisensory design - from authoring tools to the evaluation of multisensory experiences with the aim of identifying the current challenges and opportunities of mulsemedia.
Keywords:
Multisensory experiences, Mulsemedia, Sensory effects, Immersive media
References
O. Ademoye and G. Ghinea. 2013. Information recall task impact in olfaction-enhanced multimedia. ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM) 9, 3 (2013), 17.
O. A. Ademoye, N. Murray, G-M. Muntean, and G. Ghinea. 2016. Audio masking effect on inter-component skews in olfaction-enhanced multimedia presentations. ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM) 12, 4 (2016), 51.
Anas Ali Alkasasbeh and Gheorghita Ghinea. 2020. Using olfactory media cues in e-learning–perspectives from an empirical investigation. Multimedia Tools and Applications (2020), 1–23.
J. H. Chu, D. Harley, J. Kwan, M. McBride, and A. Mazalek. 2016. Sensing History: Contextualizing Artifacts with Sensory Interactions and Narrative Design. In Proceedings of the 2016 ACM Conference on Designing Interactive Systems (Brisbane, QLD, Australia) (DIS ’16). ACM, New York, NY, USA, 1294–1302. https://doi.org/10.1145/2901790.2901829
C. Claisse, D. Petrelli, N. Dulake, M. Marshall, and L. Ciolfi. 2018. Multisensory interactive storytelling to augment the visit of a historical house museum. In Proceedings of the 2018 Digital Heritage International Congress. IEEE. http://shura.shu.ac.uk/22646/
I-S Comsa, E. B. Saleme, A. Covaci, G. Mesfin, R. Trestian, C. A. S. Santos, and G. Ghinea. 2019. Do I Smell Coffee? The Tale of a 360◦ Mulsemedia Experience. IEEE MultiMedia (2019), 10 pages.
Ioan-Sorin Comsa, Ramona Trestian, and Gheorghita Ghinea. 2018. 360 mulsemedia experience over next generation wireless networks-a reinforcement learning approach. In 2018 Tenth International Conference on Quality of Multimedia Experience (QoMEX). IEEE, 1–6.
A. Covaci, G. Ghinea, C-H. Lin, S-H. Huang, and J-L. Shih. 2018. Multisensory games-based learning-lessons learnt from olfactory enhancement of a digital board game. Multimedia Tools and Applications (2018), 1–19.
Alexandra Covaci, Estevao Bissoli Saleme, Gebremariam Assres, Ioan-Sorin Comsa, Ramona Trestian, Celso Alberto Saibel Santos, and Gheorghita Ghinea. 2022. Multisensory 360 videos under varying resolution levels enhance presence. IEEE Transactions on Visualization and Computer Graphics (2022).
Alexandra Covaci, Estevao Bissoli Saleme, Gebremariam Mesfin, Nadia Hussain, Elahe Kani-Zabihi, and Gheorghita Ghinea. 2019. How do we experience crossmodal correspondent mulsemedia content? IEEE Transactions on Multimedia 22, 5 (2019), 1249–1258.
A. Covaci, R. Trestian, E. B. Saleme, I-S Comsa, G. Mesfin, C. A. S. Santos, and G. Ghinea. 2019. 360◦ Mulsemedia: A Way to Improve Subjective QoE in 360◦ Videos. In Proceedings of the 27th ACM International Conference on Multimedia (Nice, France) (MM ’19). ACM, New York, NY, USA, 2378–2386. https://doi.org/10.1145/3343031.3350954
A. Covaci, L. Zou, I. Tal, G-M. Muntean, and G. Ghinea. 2018. Is Multimedia Multisensorial? - A Review of Mulsemedia Systems. Comput. Surveys 51, 5, Article 91 (Sept. 2018), 35 pages. https://doi.org/10.1145/3233774
Anne-Sylvie Crisinel and Charles Spence. 2009. Implicit association between basic tastes and pitch. Neuroscience letters 464, 1 (2009), 39–42.
Raphael Silva de Abreu, Douglas Mattos, Joel dos Santos, Gheorghita Ghinea, and Débora Christina Muchaluat-Saade. 2020. Toward content-driven intelligent authoring of mulsemedia applications. IEEE MultiMedia 28, 1 (2020), 7–16.
Sophie Forster and Charles Spence. 2018. “What smell?” Temporarily loading visual attention induces a prolonged loss of olfactory awareness. Psychological Science 29, 10 (2018), 1642–1652.
David Gal, S Christian Wheeler, and Baba Shiv. 2007. Cross-modal influences on gustatory perception. (2007).
G. Ghinea and O. Ademoye. 2012. The sweet smell of success: Enhancing multimedia applications with olfaction. ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM) 8, 1 (2012), 2.
G. Ghinea and O. Ademoye. 2012. User perception of media content association in olfaction-enhanced multimedia. ACM Trans. on Multimedia Computing, Communications, and Applications (TOMM) 8, 4 (2012), 52.
G. Ghinea, F. Andres, and S. R. Gulliver. 2011. Multiple Sensorial Media Advances and Applications: New Developments in MulSeMedia: New Developments in MulSeMedia. Information Science Reference.
G. Ghinea, C. Timmerer, W. Lin, and S. R. Gulliver. 2014. Mulsemedia: State of the Art, Perspectives, and Challenges. ACM Trans. Multimedia Comput. Commun. Appl. 11, 1s, Article 17 (Oct. 2014), 23 pages. https://doi.org/10.1145/2617994
Kexin Guo, Alei Fan, Xinran Lehto, and Jonathon Day. 2021. Immersive digital tourism: the role of multisensory cues in digital museum experiences. Journal of Hospitality & Tourism Research (2021), 10963480211030319.
Henrik Hagtvedt and S Adam Brasel. 2016. Cross-Modal Communication: Sound Frequency Influences Consumer Responses to Color Lightness. Journal of Marketing Research 53, 4 (2016), 551–562.
Jasmin Hopf, Melina Scholl, Barbara Neuhofer, and Roman Egger. 2020. Exploring the impact of multisensory VR on travel recommendation: A presence perspective. In Information and communication technologies in tourism 2020. Springer, 169–180.
Céline Jost, Brigitte Le Pévédic, and Gérard Uzan. 2021. Using Multisensory Technologies to Stimulate People: a Reflexive Paper on Scenagrams. In Proceedings of the 1st Workshop on Multisensory Experiences-SensoryX’21. SBC.
Yamen Koubaa and Amira Eleuch. 2021. Multimodal perceptual processing of cues in food ads: Do you smell what you see?: Visual-induced olfactory imagery and its effects on taste perception and food consumption. Journal of Advertising Research 61, 1 (2021), 78–94.
Weisi Lin and Gheorghita Ghinea. 2021. Progress and Opportunities in Modelling Just-Noticeable Difference (JND) for Multimedia. IEEE Transactions on Multimedia (2021).
Lawrence E Marks. 1974. On associations of light and sound: The mediation of brightness, pitch, and loudness. The American journal of psychology (1974), 173–188.
Lawrence E Marks. 1987. On cross-modal similarity: Auditory–visual interactions in speeded discrimination. Journal of Experimental Psychology: Human Perception and Performance 13, 3 (1987), 384.
Douglas Paulo De Mattos, Débora C Muchaluat-Saade, and Gheorghita Ghinea. 2021. Beyond multimedia authoring: On the need for mulsemedia authoring tools. ACM Computing Surveys (CSUR) 54, 7 (2021), 1–31.
Miguel Melo, Hugo Coelho, Guilherme Gonçalves, Nieves Losada, Filipa Jorge, Mário Sérgio Teixeira, and Maximino Bessa. 2022. Immersive multisensory virtual reality technologies for virtual tourism. Multimedia Systems 28, 3 (2022), 1027–1037.
Gebremariam Mesfin, Nadia Hussain, Alexandra Covaci, and Gheorghita Ghinea. 2019. Using eye tracking and heart-rate activity to examine crossmodal correspondences QoE in mulsemedia. ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM) 15, 2 (2019), 1–22.
G. A. Mesfin, N. Hussain, E. Kani-Zabihi, A. Covaci, E. B. Saleme, and G. Ghinea. 2019. QoE of Cross-modally Mapped Mulsemedia: an Assessment Using Eye Gaze and Heart Rate. Multimedia Tools and Applications (2019), 21 pages.
N. Murray, O. A. Ademoye, G. Ghinea, and G-M. Muntean. 2017. A Tutorial for Olfaction-Based Multisensorial Media Application Design and Evaluation. ACM Comp. Surveys 50, 5, Article 67 (2017), 30 pages. https://doi.org/10.1145/3108243
Niall Murray, Oluwakemi A Ademoye, George Ghinea, Yuansong Qiao, Gabriel-Miro Muntean, and Brian Lee. 2017. Olfactory-enhanced multimedia video clips datasets. In 2017 Ninth International Conference on Quality of Multimedia Experience (QoMEX). IEEE, 1–5.
Niall Murray, Brian Lee, Yuansong Qiao, and Gabriel Miro-Muntean. 2016. The impact of scent type on olfaction-enhanced multimedia quality of experience. IEEE Transactions on Systems, Man, and Cybernetics: Systems 47, 9 (2016), 2503–2515.
Niall Murray, Gabriel-Miro Muntean, Yuansong Qiao, Sean Brennan, and Brian Lee. 2018. Modeling user quality of experience of olfaction-enhanced multimedia. IEEE Transactions on Broadcasting 64, 2 (2018), 539–551.
M. Obrist, E. Gatti, E. Maggioni, C. T. Vi, and C. Velasco. 2017. Multisensory Experiences in HCI. IEEE MultiMedia 24, 2 (Apr 2017), 9–13. https://doi.org/10.1109/MMUL.2017.33
M. Obrist, A. N. Tuch, and K. Hornbaek. 2014. Opportunities for odor: experiences with smell and implications for technology. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 2843–2852.
M. Obrist, C. Velasco, C. Vi, N. Ranasinghe, A. Israr, A. Cheok, C. Spence, and P. Gopalakrishnakone. 2016. Sensing the future of HCI: touch, taste, and smell user interfaces. interactions 23, 5 (2016), 40–49.
N. Ranasinghe, P. Jain, N. Thi Ngoc Tram, K. C. R. Koh, D. Tolley, S. Karwita, L. Lien-Ya, Y. Liangkun, K. Shamaiah, C. Eason Wai Tung, C. C. Yen, and E. Y-L. Do. 2018. Season Traveller: Multisensory Narration for Enhancing the Virtual Reality Experience. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (Montreal QC, Canada) (CHI ’18). ACM, New York, NY, USA, Article 577, 13 pages. https://doi.org/10.1145/3173574.3174151
N. Ranasinghe, T. N. T. Nguyen, Y. Liangkun, L-Y. Lin, D. Tolley, and E. Y-L. Do. 2017. Vocktail: A Virtual Cocktail for Pairing Digital Taste, Smell, and Color Sensations. In Proceedings of the 2017 ACM on Multimedia Conference (Mountain View, California, USA) (MM ’17). ACM, New York, NY, USA, 1139–1147. https://doi.org/10.1145/3123266.3123440
Estêvão B Saleme, Alexandra Covaci, Gebremariam Assres, Ioan-Sorin Comsa, Ramona Trestian, Celso AS Santos, and Gheorghita Ghinea. 2021. The influence of human factors on 360 mulsemedia QoE. International Journal of Human-Computer Studies 146 (2021), 102550.
E. B. Saleme, A. Covaci, G. Mesfin, C. A. S. Santos, and G. Ghinea. 2019. Mulsemedia DIY: A Survey of Devices and a Tutorial for Building Your Own Mulsemedia Environment. Comput. Surveys 52, 3, Article 58 (June 2019), 29 pages.
E. B. Saleme, C. A. S. Santos, R. A. Falbo, G. Ghinea, and F. Andres. 2019. MulseOnto: a Reference Ontology to Support the Design of Mulsemedia Systems. Journal of Universal Computer Science (2019), 26 pages.
E. B. Saleme, C. A. S. Santos, and G. Ghinea. 2019. Coping With the Challenges of Delivering Multiple Sensorial Media. IEEE MultiMedia 26, 2 (April 2019), 66–75. https://doi.org/10.1109/MMUL.2018.2873565
E. B. Saleme, C. A. S. Santos, and G. Ghinea. 2019. A mulsemedia framework for delivering sensory effects to heterogeneous systems. Multimedia Systems 25, 4 (01 Aug 2019), 421–447. https://doi.org/10.1007/s00530-019-00618-8
H-S. Seo, A. Arshamian, K. Schemmer, I. Scheer, T. Sander, G. Ritter, and T. Hummel. 2010. Cross-modal integration between odors and abstract symbols. Neuroscience letters 478, 3 (2010), 175–178.
B. Serrano, R. M. Baños, and C. Botella. 2016. Virtual reality and stimulation of touch and smell for inducing relaxation: A randomized controlled trial. Computers in Human Behavior 55 (2016), 1–8.
Charles Spence. 2010. The color of wine–Part 1. The World of Fine Wine 28 (2010), 122–129.
Charles Spence. 2011. Crossmodal correspondences: A tutorial review. Attention, Perception, & Psychophysics 73, 4 (2011), 971–995.
B. S. Spencer. 2006. Incorporating the sense of smell into patient and haptic surgical simulators. IEEE Transactions on Information Technology in Biomedicine 10, 1 (Jan 2006), 168–173. https://doi.org/10.1109/TITB.2005.856851
R. J. Stone, R. Guest, P. Mahoney, D. Lamb, and C. Gibson. 2017. A 'mixed reality' simulator concept for future Medical Emergency Response Team training. Journal of the Royal Army Medical Corps 163, 4 (2017), 280–287. https://doi.org/10.1136/jramc-2016-000726
Y. Sulema. 2016. Mulsemedia vs. Multimedia: State of the art and future trends. In 2016 International Conference on Systems, Signals and Image Processing (IWSSIP). 1–5. https://doi.org/10.1109/IWSSIP.2016.7502696
Irina Tal, Longhao Zou, Alexandra Covaci, Eva Ibarrola, Marilena Bratu, Gheorghita Ghinea, and Gabriel-Miro Muntean. 2019. Mulsemedia in Telecommunication and Networking Education: A Novel Teaching Approach that Improves the Learning Process. IEEE Communications Magazine 57, 11 (2019), 60–66.
Erik van der Meulen, Marina A Cidota, Stephan G Lukosch, Paulina JM Bank, Aadjan JC van der Helm, and Valentijn T Visch. 2016. A haptic serious augmented reality game for motor assessment of parkinson’s disease patients. In 2016 IEEE International Symposium on Mixed and Augmented Reality (ISMAR-Adjunct). IEEE, 102–104.
C. Velasco and M. Obrist. 2021. Multisensory Experiences: A Primer. Frontiers in Computer Science 3 (2021), 12. https://doi.org/10.3389/fcomp.2021.614524
C. T. Vi, D. Arthur, and M. Obrist. 2018. TasteBud: Bring Taste Back into the Game. In Proceedings of the 3rd International Workshop on Multisensory Approaches to Human-Food Interaction (Boulder, CO, USA) (MHFI’18). ACM, New York, NY, USA, Article 1, 5 pages. https://doi.org/10.1145/3279954.3279955
Ryan J Ward, Sophie M Wuerger, and Alan Marshall. 2021. Smelling sensations: olfactory crossmodal correspondences. Journal of Perceptual Imaging (2021).
B. Yecies. 2016. Transnational collaboration of the multisensory kind: exploiting Korean 4D cinema in China. Media International Australia 159, 1 (2016), 22–31.
L. Zou, I. Tal, A. Covaci, E. Ibarrola, G. Ghinea, and G-M. Muntean. 2017. Can Multisensorial Media Improve Learner Experience?. In Proceedings of the 8th ACM on Multimedia Systems Conference. ACM, 315–320.
O. A. Ademoye, N. Murray, G-M. Muntean, and G. Ghinea. 2016. Audio masking effect on inter-component skews in olfaction-enhanced multimedia presentations. ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM) 12, 4 (2016), 51.
Anas Ali Alkasasbeh and Gheorghita Ghinea. 2020. Using olfactory media cues in e-learning–perspectives from an empirical investigation. Multimedia Tools and Applications (2020), 1–23.
J. H. Chu, D. Harley, J. Kwan, M. McBride, and A. Mazalek. 2016. Sensing History: Contextualizing Artifacts with Sensory Interactions and Narrative Design. In Proceedings of the 2016 ACM Conference on Designing Interactive Systems (Brisbane, QLD, Australia) (DIS ’16). ACM, New York, NY, USA, 1294–1302. https://doi.org/10.1145/2901790.2901829
C. Claisse, D. Petrelli, N. Dulake, M. Marshall, and L. Ciolfi. 2018. Multisensory interactive storytelling to augment the visit of a historical house museum. In Proceedings of the 2018 Digital Heritage International Congress. IEEE. http://shura.shu.ac.uk/22646/
I-S Comsa, E. B. Saleme, A. Covaci, G. Mesfin, R. Trestian, C. A. S. Santos, and G. Ghinea. 2019. Do I Smell Coffee? The Tale of a 360◦ Mulsemedia Experience. IEEE MultiMedia (2019), 10 pages.
Ioan-Sorin Comsa, Ramona Trestian, and Gheorghita Ghinea. 2018. 360 mulsemedia experience over next generation wireless networks-a reinforcement learning approach. In 2018 Tenth International Conference on Quality of Multimedia Experience (QoMEX). IEEE, 1–6.
A. Covaci, G. Ghinea, C-H. Lin, S-H. Huang, and J-L. Shih. 2018. Multisensory games-based learning-lessons learnt from olfactory enhancement of a digital board game. Multimedia Tools and Applications (2018), 1–19.
Alexandra Covaci, Estevao Bissoli Saleme, Gebremariam Assres, Ioan-Sorin Comsa, Ramona Trestian, Celso Alberto Saibel Santos, and Gheorghita Ghinea. 2022. Multisensory 360 videos under varying resolution levels enhance presence. IEEE Transactions on Visualization and Computer Graphics (2022).
Alexandra Covaci, Estevao Bissoli Saleme, Gebremariam Mesfin, Nadia Hussain, Elahe Kani-Zabihi, and Gheorghita Ghinea. 2019. How do we experience crossmodal correspondent mulsemedia content? IEEE Transactions on Multimedia 22, 5 (2019), 1249–1258.
A. Covaci, R. Trestian, E. B. Saleme, I-S Comsa, G. Mesfin, C. A. S. Santos, and G. Ghinea. 2019. 360◦ Mulsemedia: A Way to Improve Subjective QoE in 360◦ Videos. In Proceedings of the 27th ACM International Conference on Multimedia (Nice, France) (MM ’19). ACM, New York, NY, USA, 2378–2386. https://doi.org/10.1145/3343031.3350954
A. Covaci, L. Zou, I. Tal, G-M. Muntean, and G. Ghinea. 2018. Is Multimedia Multisensorial? - A Review of Mulsemedia Systems. Comput. Surveys 51, 5, Article 91 (Sept. 2018), 35 pages. https://doi.org/10.1145/3233774
Anne-Sylvie Crisinel and Charles Spence. 2009. Implicit association between basic tastes and pitch. Neuroscience letters 464, 1 (2009), 39–42.
Raphael Silva de Abreu, Douglas Mattos, Joel dos Santos, Gheorghita Ghinea, and Débora Christina Muchaluat-Saade. 2020. Toward content-driven intelligent authoring of mulsemedia applications. IEEE MultiMedia 28, 1 (2020), 7–16.
Sophie Forster and Charles Spence. 2018. “What smell?” Temporarily loading visual attention induces a prolonged loss of olfactory awareness. Psychological Science 29, 10 (2018), 1642–1652.
David Gal, S Christian Wheeler, and Baba Shiv. 2007. Cross-modal influences on gustatory perception. (2007).
G. Ghinea and O. Ademoye. 2012. The sweet smell of success: Enhancing multimedia applications with olfaction. ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM) 8, 1 (2012), 2.
G. Ghinea and O. Ademoye. 2012. User perception of media content association in olfaction-enhanced multimedia. ACM Trans. on Multimedia Computing, Communications, and Applications (TOMM) 8, 4 (2012), 52.
G. Ghinea, F. Andres, and S. R. Gulliver. 2011. Multiple Sensorial Media Advances and Applications: New Developments in MulSeMedia: New Developments in MulSeMedia. Information Science Reference.
G. Ghinea, C. Timmerer, W. Lin, and S. R. Gulliver. 2014. Mulsemedia: State of the Art, Perspectives, and Challenges. ACM Trans. Multimedia Comput. Commun. Appl. 11, 1s, Article 17 (Oct. 2014), 23 pages. https://doi.org/10.1145/2617994
Kexin Guo, Alei Fan, Xinran Lehto, and Jonathon Day. 2021. Immersive digital tourism: the role of multisensory cues in digital museum experiences. Journal of Hospitality & Tourism Research (2021), 10963480211030319.
Henrik Hagtvedt and S Adam Brasel. 2016. Cross-Modal Communication: Sound Frequency Influences Consumer Responses to Color Lightness. Journal of Marketing Research 53, 4 (2016), 551–562.
Jasmin Hopf, Melina Scholl, Barbara Neuhofer, and Roman Egger. 2020. Exploring the impact of multisensory VR on travel recommendation: A presence perspective. In Information and communication technologies in tourism 2020. Springer, 169–180.
Céline Jost, Brigitte Le Pévédic, and Gérard Uzan. 2021. Using Multisensory Technologies to Stimulate People: a Reflexive Paper on Scenagrams. In Proceedings of the 1st Workshop on Multisensory Experiences-SensoryX’21. SBC.
Yamen Koubaa and Amira Eleuch. 2021. Multimodal perceptual processing of cues in food ads: Do you smell what you see?: Visual-induced olfactory imagery and its effects on taste perception and food consumption. Journal of Advertising Research 61, 1 (2021), 78–94.
Weisi Lin and Gheorghita Ghinea. 2021. Progress and Opportunities in Modelling Just-Noticeable Difference (JND) for Multimedia. IEEE Transactions on Multimedia (2021).
Lawrence E Marks. 1974. On associations of light and sound: The mediation of brightness, pitch, and loudness. The American journal of psychology (1974), 173–188.
Lawrence E Marks. 1987. On cross-modal similarity: Auditory–visual interactions in speeded discrimination. Journal of Experimental Psychology: Human Perception and Performance 13, 3 (1987), 384.
Douglas Paulo De Mattos, Débora C Muchaluat-Saade, and Gheorghita Ghinea. 2021. Beyond multimedia authoring: On the need for mulsemedia authoring tools. ACM Computing Surveys (CSUR) 54, 7 (2021), 1–31.
Miguel Melo, Hugo Coelho, Guilherme Gonçalves, Nieves Losada, Filipa Jorge, Mário Sérgio Teixeira, and Maximino Bessa. 2022. Immersive multisensory virtual reality technologies for virtual tourism. Multimedia Systems 28, 3 (2022), 1027–1037.
Gebremariam Mesfin, Nadia Hussain, Alexandra Covaci, and Gheorghita Ghinea. 2019. Using eye tracking and heart-rate activity to examine crossmodal correspondences QoE in mulsemedia. ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM) 15, 2 (2019), 1–22.
G. A. Mesfin, N. Hussain, E. Kani-Zabihi, A. Covaci, E. B. Saleme, and G. Ghinea. 2019. QoE of Cross-modally Mapped Mulsemedia: an Assessment Using Eye Gaze and Heart Rate. Multimedia Tools and Applications (2019), 21 pages.
N. Murray, O. A. Ademoye, G. Ghinea, and G-M. Muntean. 2017. A Tutorial for Olfaction-Based Multisensorial Media Application Design and Evaluation. ACM Comp. Surveys 50, 5, Article 67 (2017), 30 pages. https://doi.org/10.1145/3108243
Niall Murray, Oluwakemi A Ademoye, George Ghinea, Yuansong Qiao, Gabriel-Miro Muntean, and Brian Lee. 2017. Olfactory-enhanced multimedia video clips datasets. In 2017 Ninth International Conference on Quality of Multimedia Experience (QoMEX). IEEE, 1–5.
Niall Murray, Brian Lee, Yuansong Qiao, and Gabriel Miro-Muntean. 2016. The impact of scent type on olfaction-enhanced multimedia quality of experience. IEEE Transactions on Systems, Man, and Cybernetics: Systems 47, 9 (2016), 2503–2515.
Niall Murray, Gabriel-Miro Muntean, Yuansong Qiao, Sean Brennan, and Brian Lee. 2018. Modeling user quality of experience of olfaction-enhanced multimedia. IEEE Transactions on Broadcasting 64, 2 (2018), 539–551.
M. Obrist, E. Gatti, E. Maggioni, C. T. Vi, and C. Velasco. 2017. Multisensory Experiences in HCI. IEEE MultiMedia 24, 2 (Apr 2017), 9–13. https://doi.org/10.1109/MMUL.2017.33
M. Obrist, A. N. Tuch, and K. Hornbaek. 2014. Opportunities for odor: experiences with smell and implications for technology. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 2843–2852.
M. Obrist, C. Velasco, C. Vi, N. Ranasinghe, A. Israr, A. Cheok, C. Spence, and P. Gopalakrishnakone. 2016. Sensing the future of HCI: touch, taste, and smell user interfaces. interactions 23, 5 (2016), 40–49.
N. Ranasinghe, P. Jain, N. Thi Ngoc Tram, K. C. R. Koh, D. Tolley, S. Karwita, L. Lien-Ya, Y. Liangkun, K. Shamaiah, C. Eason Wai Tung, C. C. Yen, and E. Y-L. Do. 2018. Season Traveller: Multisensory Narration for Enhancing the Virtual Reality Experience. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (Montreal QC, Canada) (CHI ’18). ACM, New York, NY, USA, Article 577, 13 pages. https://doi.org/10.1145/3173574.3174151
N. Ranasinghe, T. N. T. Nguyen, Y. Liangkun, L-Y. Lin, D. Tolley, and E. Y-L. Do. 2017. Vocktail: A Virtual Cocktail for Pairing Digital Taste, Smell, and Color Sensations. In Proceedings of the 2017 ACM on Multimedia Conference (Mountain View, California, USA) (MM ’17). ACM, New York, NY, USA, 1139–1147. https://doi.org/10.1145/3123266.3123440
Estêvão B Saleme, Alexandra Covaci, Gebremariam Assres, Ioan-Sorin Comsa, Ramona Trestian, Celso AS Santos, and Gheorghita Ghinea. 2021. The influence of human factors on 360 mulsemedia QoE. International Journal of Human-Computer Studies 146 (2021), 102550.
E. B. Saleme, A. Covaci, G. Mesfin, C. A. S. Santos, and G. Ghinea. 2019. Mulsemedia DIY: A Survey of Devices and a Tutorial for Building Your Own Mulsemedia Environment. Comput. Surveys 52, 3, Article 58 (June 2019), 29 pages.
E. B. Saleme, C. A. S. Santos, R. A. Falbo, G. Ghinea, and F. Andres. 2019. MulseOnto: a Reference Ontology to Support the Design of Mulsemedia Systems. Journal of Universal Computer Science (2019), 26 pages.
E. B. Saleme, C. A. S. Santos, and G. Ghinea. 2019. Coping With the Challenges of Delivering Multiple Sensorial Media. IEEE MultiMedia 26, 2 (April 2019), 66–75. https://doi.org/10.1109/MMUL.2018.2873565
E. B. Saleme, C. A. S. Santos, and G. Ghinea. 2019. A mulsemedia framework for delivering sensory effects to heterogeneous systems. Multimedia Systems 25, 4 (01 Aug 2019), 421–447. https://doi.org/10.1007/s00530-019-00618-8
H-S. Seo, A. Arshamian, K. Schemmer, I. Scheer, T. Sander, G. Ritter, and T. Hummel. 2010. Cross-modal integration between odors and abstract symbols. Neuroscience letters 478, 3 (2010), 175–178.
B. Serrano, R. M. Baños, and C. Botella. 2016. Virtual reality and stimulation of touch and smell for inducing relaxation: A randomized controlled trial. Computers in Human Behavior 55 (2016), 1–8.
Charles Spence. 2010. The color of wine–Part 1. The World of Fine Wine 28 (2010), 122–129.
Charles Spence. 2011. Crossmodal correspondences: A tutorial review. Attention, Perception, & Psychophysics 73, 4 (2011), 971–995.
B. S. Spencer. 2006. Incorporating the sense of smell into patient and haptic surgical simulators. IEEE Transactions on Information Technology in Biomedicine 10, 1 (Jan 2006), 168–173. https://doi.org/10.1109/TITB.2005.856851
R. J. Stone, R. Guest, P. Mahoney, D. Lamb, and C. Gibson. 2017. A 'mixed reality' simulator concept for future Medical Emergency Response Team training. Journal of the Royal Army Medical Corps 163, 4 (2017), 280–287. https://doi.org/10.1136/jramc-2016-000726
Y. Sulema. 2016. Mulsemedia vs. Multimedia: State of the art and future trends. In 2016 International Conference on Systems, Signals and Image Processing (IWSSIP). 1–5. https://doi.org/10.1109/IWSSIP.2016.7502696
Irina Tal, Longhao Zou, Alexandra Covaci, Eva Ibarrola, Marilena Bratu, Gheorghita Ghinea, and Gabriel-Miro Muntean. 2019. Mulsemedia in Telecommunication and Networking Education: A Novel Teaching Approach that Improves the Learning Process. IEEE Communications Magazine 57, 11 (2019), 60–66.
Erik van der Meulen, Marina A Cidota, Stephan G Lukosch, Paulina JM Bank, Aadjan JC van der Helm, and Valentijn T Visch. 2016. A haptic serious augmented reality game for motor assessment of parkinson’s disease patients. In 2016 IEEE International Symposium on Mixed and Augmented Reality (ISMAR-Adjunct). IEEE, 102–104.
C. Velasco and M. Obrist. 2021. Multisensory Experiences: A Primer. Frontiers in Computer Science 3 (2021), 12. https://doi.org/10.3389/fcomp.2021.614524
C. T. Vi, D. Arthur, and M. Obrist. 2018. TasteBud: Bring Taste Back into the Game. In Proceedings of the 3rd International Workshop on Multisensory Approaches to Human-Food Interaction (Boulder, CO, USA) (MHFI’18). ACM, New York, NY, USA, Article 1, 5 pages. https://doi.org/10.1145/3279954.3279955
Ryan J Ward, Sophie M Wuerger, and Alan Marshall. 2021. Smelling sensations: olfactory crossmodal correspondences. Journal of Perceptual Imaging (2021).
B. Yecies. 2016. Transnational collaboration of the multisensory kind: exploiting Korean 4D cinema in China. Media International Australia 159, 1 (2016), 22–31.
L. Zou, I. Tal, A. Covaci, E. Ibarrola, G. Ghinea, and G-M. Muntean. 2017. Can Multisensorial Media Improve Learner Experience?. In Proceedings of the 8th ACM on Multimedia Systems Conference. ACM, 315–320.
Published
2022-06-22
How to Cite
COVACI, Alexandra; SALEME, Estevão Bissoli; JOST, Celine; DOS SANTOS, Joel; GHINEA, Gheorghita.
SensoryX ’22 Workshop on Multisensory Experiences at ACM IMX ’22. In: WORKSHOP ON MULTISENSORY EXPERIENCES (SENSORYX), 2. , 2022, Aveiro.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2022
.
DOI: https://doi.org/10.5753/sensoryx.2022.20000.
