Quality of Experience of Immersive Media – New Challenges

• Mylene C. Q. Farias Texas State University

Resumo

In the past decade, significant progress in imaging and computing technologies has unleashed the creation of more accurate representations of the physical world, enabling immersive and interactive experiences that more closely resemble reality [1]. As a consequence, immersive applications have attracted a lot of interdisciplinary attention, with the development of multimodal human-computer interactions that allow users to fully immerse themselves within realistic or virtual environments or to interact with real/virtual elements seamlessly blended with the physical world [6]. Immersive experiences can be found at different points along the “virtuality continuum,” ranging from photorealistic settings, through mixed reality approaches, to completely virtual environments. In this scenario, high-speed connections and high-quality imaging systems are essential for the development of next-generation immersive experiences and applications in areas such as healthcare, education/training, arts & entertainment, remote work, marketing, and automotive. But the success of these emerging applications will depend on the quality of experience (QoE) they provide users [5]. As defined by Qualinet [2], QoE refers to the “degree of delight or annoyance of applications or services resulting from the fulfillment of his or her expectations with respect to the utility and/or enjoyment of the application or service in the light of the users personality and current state.” For immersive technologies, immersive media experiences (IMEx) extend the concept of QoE by encompassing elements such as the sense of presence, immersion, and motion sickness, among others [7]. Both QoE and IMEx are shaped by three influencing factors (IF): the system, the context, and the human user. For immersive experiences, the system IF plays an important role, affecting immersion, presence, and interaction itself. This impact has been extensively studied in the literature, with several studies analyzing the effect of the design of the device itself on IMEx [3, 4]. The context IF, which includes the user’s environment, is often studied together with the system IF, since it is difficult to separate these two IFs. Finally, an important IF for immersive media are human factors, which include perceptual characteristics such as visual, audio, and spatial perception that make each user unique. It is well known that impairments may reduce perception and cause imbalances, resulting in uncomfortable symptoms and cybersickness. Subjective and instrumental assessments are commonly used to study human IFs in IMEx. In this context, an open area of research is the design of subjective and instrumental assessment methods to estimate the user immersive media experience. For example, by tracking human behavior and psycho-physiological signals, it is possible to construct models of influential factors that can be employed to identify and potentially forecast phenomena like cybersickness, the user’s perception of immersion and sense of presence, and the overall quality of the experience. In this talk, I discuss several aspects of IMEx, including subjective and instrumental methods, and the several challenges of this area.

Palavras-chave: image processing, immersive media, quality of experience, machine learning, virtual reality, point clouds, light lields

Referências

John G Apostolopoulos, Philip A Chou, Bruce Culbertson, Ton Kalker, Mitchell D Trott, and Susie Wee. 2012. The road to immersive communication. Proc. IEEE 100, 4 (2012), 974–990

Kjell Brunnström, Sergio Ariel Beker, Katrien De Moor, Ann Dooms, Sebastian Egger, Marie-Neige Garcia, Tobias Hossfeld, Satu Jumisko-Pyykkö, Christian Keimel, Mohamed-Chaker Larabi, 2013. Qualinet white paper on definitions of quality of experience. (2013)

Hind Kharoub, Mohammed Lataifeh, and Naveed Ahmed. 2019. 3D user interface design and usability for immersive VR. Applied sciences 9, 22 (2019), 4861

Yong Min Kim, Ilsun Rhiu, Minshik Rhie, Hye Seon Choi, and Myung Hwan Yun. 2019. Current state of user experience evaluation in virtual reality: a systematic review from an ergonomic perspective. In Proceedings of the human factors and ergonomics society annual meeting, Vol. 63. Sage Publications Sage CA: Los Angeles, CA, 1274–1275

Marc-Antoine Moinnereau, Alcyr Alves de Oliveira Jr, and Tiago H Falk. 2022. Immersive media experience: a survey of existing methods and tools for human influential factors assessment. Quality and User Experience 7, 1 (2022), 5

Pablo Pérez, Ester Gonzalez-Sosa, Jesús Gutiérrez, and Narciso García. 2022. Emerging immersive communication systems: overview, taxonomy, and good practices for QoE assessment. Frontiers in Signal Processing 2 (2022), 917684

Andrew Perkis, Christian Timmerer, Sabina Baraković, Jasmina Baraković Husić, Søren Bech, Sebastian Bosse, Jean Botev, Kjell Brunnström, Luis Cruz, Katrien De Moor, 2020. QUALINET white paper on definitions of immersive media experience (IMEx). arXiv preprint arXiv:2007.07032 (2020).