Synesthesia : A Study on Immersive Features of Electronic Games

Immersion is a quality that turns user experiences more appealing, despite the definition of immersion itself still being a source of controversy. Modern electronic games became a well-established immersive media which already has a number of consumer-level virtual reality hardware and software available. Endowing a game with immersion requires not only theoretical background found in literature but more practical guidelines for assisting developers to glimpse possibilities and make design decisions. In this paper, we investigate specific features that bestow immersion to an electronic game. So, we first analyzed game titles the audience, the critics and developers themselves consider immersive in order to enumerate potentially immersive features found in these games. We then developed a potentially immersive game prototype based on these features. Results of a comparative evaluation of our prototype and the selected titles under different settings suggest that six features were able to provide an immersive experience.


INTRODUCTION
The first generations of electronic games did not have sufficient computational resources to provide realistic depictions of virtual worlds.With the advent of three-dimensional graphics, a paradigm was established in which representation gave way to imitation of the real [55].Analogously, sound technologies have experienced a remarkable evolution.The limited chiptune sounds gave rise to the simultaneous reproduction of several high quality audio tracks and binaural perceptual models with sophisticated spatialization effects [42] [40].
In addition, the evolution of processing power has provided the creation of characters that act in a more complex and realistic manner [77] [29] [55].It has been realized over time that audiovisual and narrative features can cause immersion [63] [18] [21].This, in turn, besides contributing to the satisfaction of the player, colorredbecame a selling point [76].Immersion, however, is a phenomenon that, although often mentioned by players, researchers and the media, is still a source of debate regarding its meaning [63].
Modern electronic games stand out as a potentially immersive media, especially when taking into account the recent advances in affordable Virtual Reality (VR) hardware such as the increasingly popular VR headsets [36] [14] [37] [16] [25] [75].However, our investigation does not consider experimentation with titles released for VR headsets.We assume this once again emergent technology still requires specific investigations regarding its effectiveness.Obtaining presence, for example, is inherently less challenging when VR headsets are employed as a supporting technology [59].
On the other hand, immersive games are demanded by the industry.Practical development of such titles therefore requires a set of clear, effective guidelines for assisting developers to endow their experiences with actual immersion [59] [7].Moreover, measuring an immersive experience becomes a challenge because immersion is an emotional phenomenon [26].In this paper, we investigate features that confer the quality of immersion to an electronic game based on the analysis of titles the audience consider immersive.We developed and evaluated a game prototype designed on top of these potentially immersive features we found.

Contribution
This paper contributes to the study of immersion in electronic games by: • Presenting an analysis of games considered immersive by their target audience.
• Polling features that could possibly have caused immersion.
• Developing a prototype of immersive game based on these features.
• Comparing the analysis of the prototype and the chosen games in order to determine whether there was actual immersion.
• Creating a reference table enumerating features that might turn games immersive.

Text Organization
In this paper we describe the whole investigation process [30], which is composed of three sequential steps: research, development, and evaluation.Each step is divided into three smaller steps: • The research step, described in sections 2 to 4, is divided into the study of a theoretical background, related works, and immersive games.
• The development step is divided into game design, implementation, and playtesting.The latter aims to assure the quality of the final product.This step is described across the section 5.
• The evaluation step is divided into the scheduling of an event, testing, and data analysis.These are detailed in sections 6 and 7.

BACKGROUND
The theoretical fundamentals of this work are presented in this section.The concepts of telepresence, mixed reality, flow, and immersion, as adopted in this work, are discussed throughout the subsections.

Presence
A typical electronic game creates and manipulates a virtual world, i.e., an environment representing the space in which the user interaction occurs.Virtual worlds in games usually adopt an audiovisual representation, but others are also possible.The player interacts with the game by taking control of specific elements of such world, typically a game character known in literature as avatar [67] [63].
The feeling of being in a virtual environment mediated by computer technology is reinforced by controlling an avatar.Such felling is known as telepresence, or just presence [67].Jonathan Steuer affirms that telepresence occurs in environments that exhibit vividness and interactivity [67].Vividness refers to the wealth of formal details, i.e., how rich the audiovisual representation aspects are.Interactivity refers to the user's ability to act and modify the virtual environment and the way in which the latter reacts to such acts in real time.
Interactivity is closely related to the concept of agency in the field of Interactive Storytelling [62] [21].Agency is an aesthetic quality defined by how much the player affects the happenings in the virtual world, especially when the experience evolves around a story that develops around actions taken by the user.
Environments that employ elements from the real, physical space as a mean to have direct impact on virtual space are called Mixed Reality (MR) environments [8] [53].The concept of mixed reality differs from Augmented Reality (AR) and Virtual Reality.
The phenomenon of superimposition of virtual space in real space is called augmented reality [4].Virtual objects coexist with real objects and are distinguishable.The key difference between mixed reality and augmented reality is that in the latter, virtual objects are projected to enhance real world environment and are often interactable and visible exclusively through another device, such as a smartphone.An example of mixed reality application is the 2007 game Wii Sports [58], in which players simulate a variety of sports (such as tennis) mimicking movements with the controller.An example of augmented reality application is the mobile game Pokémon Go! [39], in which players may walk around and find creatures lurking in real world locations based on their phone's GPS.The phenomenon of virtual reality occurs when there is a complete use of the virtual space, i.e., when all interactions take place entirely through a synthetic environment.VR often happens through special headsets or other devices that prevent users from seeing the real world [5] [45] [23].On the other hand, both augmented and virtual realities can be considered to be mixed realities based on the concept of how interaction occurs [8].
Tracking and motion capture technologies have provided a breakthrough for applications in mixed reality [10].Kinect™, for example, is a camera with sensors capable of detecting and tracking the human skeleton, thus allowing players to use their own bodies as controllers.In addition, the device's features can be used in applications beyond electronic games, such as biometrics [45] [78], or in order to automatically associate recognized users with their respective personal settings.

Flow
Nakamura and Csikszentmihalyi [57] assert that flow is a state of consciousness in which there are deep focus and concentration on a pleasurable activity.Consequently, an individual in flow state does not differ action from perception, demonstrates diminished self-consciousness and has a distorted notion of time.
These authors also argue that autotelic activity, which causes a complete absorption, is a reasonable definition of good life since the individuals experiencing it may persist developing such activities disregarding hunger, fatigue, and discomfort.Despite the criticism regarding addictive behavior [13], flow can be used to yield interesting results in obtaining better life quality through optimal experiences in the field of entertainment.
Despite flow being a highly individual and subjective experience, scientists have found biological signs of flow.The sensation of flow was reported to be related to the increase of dopamine, which is a neurotransmitter often associated with pleasure, increased attention, and performance [69] [48].
On the other hand, Raph Koster [44,Chapter 5] elaborates the concept of fun and its importance for game development.The author considers "fun" as, among many other definitions, "the act of mastering a problem mentally".He states that flow can be achieved when a game perfectly matches its challenges with its player's capability, therefore providing a fun experience.He also states that a game becomes frustrating when its challenges' demands surpass the player's capability.Conversely, the game becomes boring when the player's capability surpass the challenges' demands.Balancing is thus necessary in order to provide optimal experiences.
Even though flow and fun having a strong connection, one does not imply the other.It is possible to have fun in games from aesthetic appreciation, social interactions, narrative interpretation, and many other ways.Similarly, the flow state can be achieved in relaxing activities such as yoga and meditation [57].

Immersion
In this paper, we adopt the unbiased concept proposed by Ermi and Mäyrä, who define immersion as the phenomenon in which the subject feels part of the experience as a whole, dedicating his or her entire perceptual apparatus to an experience [26].These authors state that immersion in games can be achieved through three fundamental aspects: audiovisual quality; challenges and interaction; and narrative.These aspects correspond to sensory immersion, challengebased immersion, and imaginative immersion, respectively.So, these provide three distinct perspectives for understanding immersion.The process is subjective and depends on factors such as mood, social and psychological context, expertise in games, and motivation.A game needs to involve players in order to further the feeling of empathy.
Concerning how the actual immersion state is achieved, based on the studies of Brown and Cairns [11], immersion can be separated into three stages: engagement, engrossment, and total immersion.At the engagement stage, the player begins his or her playing experience.For this step to occur, the player needs to be willing to focus on a pleasant game.The second stage, engrossment, occurs when the player feels emotionally connected to the game.The last step, total immersion, occurs when there is detachment from the player from her physical surroundings, that is, complete empathy with the virtual world.
This total immersion might lead to the phenomenon known as synesthesia.Neurologically, the term synesthesia refers to the capacity of mixing senses.For example, people with this capacity might be able to "hear the sound of a color" or "see the color of a flavor".This is a very rare condition observed only on people who either were born with it (genuine synesthesia), acquired it as a consequence from brain damage, or people who temporarily forced it by using drugs such as LSD (lysergic acid diethylamide), mescaline, and psilocybin [65].The exact mechanisms of how stimuli in a sensory channel triggers a different channel are still unclear.
Synesthesia can also refer to the fusion of senses.Frequently used in the literature through expressions such as "cold person" or "warm colors", this process is driven by metaphors and relies heavily on cognition and interpretation [19].In electronic games, this can be achieved primarily by providing associations between visuals and sound in an interactive environment [60].
In her study, Anja Volk shows [74] that music plays a pivotal role in games because it amplifies the sense of interaction.She categorizes game music as: linear music, which helps ambiance; reactive music, which is triggered from the player's inputs; and proactive music, which insinuates or encourages the player to perform some kind of action.The reader might readily come up with example situations of such game music categories found in popular modern game titles.Finally, there are technical factors that also help in obtaining immersion, such as screen aspect ratio and resolution, image contrast and brightness, color space, color bit depth, image persistence, and refresh rate [47].

Working Definitions
For the purposes of developing this work, we present succinct definitions for the aforementioned concepts: • Presence is the feeling of "being there".
• Flow is the absorbing feeling of "be doing something".
• Immersion is therefore a feeling that can be defined as a combination of the two prior feelings: "being there doing something".

RELATED WORK
Hu and his collaborators [32] developed an experiment on engagement and immersion in mixed reality.They proposed a game that simulates the classic title Pong [24] using tablets, dynamic environmental lighting, and Kinect for motion capture.Their investigation presented results indicating that divided attention and physical challenge produce a positive effect on the quality of immersion.Mika Suokas [68] conducted tests to evaluate whether the use of eye and head movement tracking technologies can influence the user's immersive experience.The author focused his study in first-person shooter games, usually played using joypads or the classical combination of keyboard and mouse.However, no clear evidence has been obtained in his investigation pointing out that an interaction based on movement tracking provides better immersion when compared to its "ordinary" counterpart using conventional interaction devices.
On the other hand, Gazzard [31] found that motion capture technologies can, at least, make a game feel more natural because they introduce muscle memory, proprioception 1 , and reflexes into the gameplay.
Konishi and his collaborators [43] developed a suit, which has motor devices that vibrate according to a song mapped to specific parts of this wearable device.According to the authors, wearers "felt the music flow through their body".This suit was exhibited to the audience during a gaming event in 2015 and its users reported the sensation of "being plunged into a new world".However, at the 2017 edition of the SWSX (South by Southwest) Conference, Sony's Future Labs presented a demonstration of the suit.Nevertheless it was reported to be confusing [38] as there was no clear perception about what is the suit syncing to, and it did not significantly contribute to the overall experience.
The experimental evaluation conducted by Birk et al. [9] analyzed data from 126 participants playing a custom endless runner game.The authors point out that the use of customizable avatars stimulates player identification with the game, so this feature reinforces users' empathy with the game by considering analyses for similar, embodied, and wishful identification.Birk et al. argue that avatar customization increased not only the experienced immersion, but also autonomy, enjoyment, invested effort, and positive affect.Despite the endeavors made in this evaluation, similar results were already known in the literature for character customization [6].
On the other hand, the investigations conducted by Patton and Schell [59], followed by the careful observations by Baker [7] try to unveil the latent potential of VR headsets for gaming purposes due to the uniqueness of this kind of application.These authors recommend building real world mock-ups of elements found in the virtual world as an important aspect of the development process.In addition, they provide important design goals for building such VR games: provide a comfortable experience; combine presence and object manipulation; inspire players to look around; explore the specific experiences provided by each interaction device; capitalize from design errors during the development process, since it is error-prone; and focus on immersion more than gameplay.
The studies described above suggest the existence of a relationship between physical and virtual environments in immersive games.We can conclude that reinforcing such a relationship could make the players' experience more pleasant and meaningful.However, there are no clear features or guidelines that can be readily explored by game developers in order to endow their titles with immersion.Koos de Beer [20] proposed an analysis on Alternate Reality Games (ARGs), defined as games in which players must collaboratively interpret information across various platforms and solve problems.ARGs hide a narrative, which should be discovered in a transmedia context.Finally, ARGs blend their own virtual worlds with the existing reality, thus creating a unique environment where the game's rules, time, and space become, to a certain extent, unclear.Though the author's thesis did not focus on the immersive qualities of games, it provides extensive data supporting the claim that ARGs are highly immersive.

IMMERSIVE GAMES AND THEIR FEATURES
There are many games considered to be immersive from the players' perspective.In popular online game stores such as Steam, users can tag games they believe to provide this kind of experience.Tags like "Atmospheric", "Story Rich", "Great Soundtrack" and "Cinematic" [17] indicate a multitude of possible titles and provide a baseline for our investigation.As mentioned previously, titles developed exclusively for VR headsets were considered to be out of the scope of this work.This particular choice was taken in order to keep focus on the potentially immersive characteristics found in games themselves instead of those provided by specific VR devices, since this study aims at obtaining clear design guidelines for immersive games.
Rez [35] and Child of Eden [72] were chosen as research subjects among all possible candidates.Several factors contribute to this choice: • The merchandise: both titles claim to be immersive or synesthetic in their release trailers [61] [71].
• The players' opinion: users constantly highlight what they felt while playing these titles [52] [51].
• The critic's opinion: mainstream gaming websites often state how these titles offer rich and unique experiences [12] [15].
• The developers statements: Tetsuya Mizuguchi, creator of both games, draws inspiration from art, music and the concept of synesthesia itself [2].Rez's motivation was to experiment and introduce synesthesia through audiovisual resources [49], while Child of Eden's was to bring feelings of joy and happiness through sounds and positive images [27].
Other researchers have also explored Rez's and Child of Eden's remarkable efforts regarding immersion.Janet Murray [56] states that an immersive, interactive experience is evoked when an environment constantly affords rich and affective feedback to its users.Such potentially autotelic experiences have even generated debates about electronic games being included as artistic avant-garde [22].
Finally, both games appeal to a similar audience, which eases the research process.They present highly reactive music and environments, whose aesthetics fit their main theme accordingly.These titles are both rail-shooter games with similar patterns of level design.Finally, these title also support VR and MR devices in the form of VR headsets and the Kinect, respectively, so further investigations under the light of this kind of user experience are also possible.The game features wireframe-like graphics inspired by the paintings of Wassily Kandinsky [22], which evolve into more complex shapes.A sample game screen can be seen in Figure 1 as published by Xbox Live Market Place [46].Gameplay elements are synchronized with music and sound effects.Among those, both the scenery and the main character vibrate in conjunction with the music's beat.Enemy characters, in their turn, appear or disappear from the screen according to the song.The player cannot fully control the camera, but is allowed to shoot at enemies and obstacles.When destroyed, both of these emit harmonic tunes that are combined with the soundtrack.Consequently, the music gets richer as the player successfully destroys something.

Rez ©
Rez's levels songs are segmented into ten parts.A new part is stacked with the previous ones as the player progresses.The music parallels the player's progress: the more he or she advances, the more "complete" the music becomes.Rez's game interface depicts a terminal which simulates the execution of an algorithm.
Concerned about immersion, the game developers created the Trance Vibrator © , an accessory that made the player's wrist vibrate according to the song [54].The accessory was compatible only with early versions of the game.In subsequent releases, tactile sensations were stimulated using the joypads' motors.The latest release supports PlayStation ® VR, a virtual reality headset on the PlayStation ® 4, and other VR headsets on PC and Android.These "remaster" versions feature an exclusive area specifically designed to offer a unique VR experience [50] Mark MacDonald, who worked in the development of Rez Infinite, states that it benefits greatly from the sensory deprivation caused by the combination of a VR headset and a headphone [64], even though the game itself allows full 3D interaction.

Child of Eden ©
While Rez presents a cybernetic world, Child of Eden shows a more natural and subjective world, as seen in Figure 2 from PlayStation  Blog [73].The player's avatar is never seen onscreen and the interface is as clean as possible.In Child of Eden, the consciousness of the the last human being, called Lumi, is preserved in the Eden network, some kind of future ubiquitous internet.When trying to reunite the memories of Lumi in an avatar, viruses attack Eden and it is up to the player to stop them.
Unlike its predecessor Rez, Child of Eden features songs with lyrics in several segments, without necessarily having synchronization.The visuals are more complex than Rez's: images are colorful, with assets that feel organic and alive.Moreover, the camera is allowed to move in all directions.Enemies move smoothly and produce complimentary sound effects when hit.The colors and sounds were carefully designed to boast an air of optimism to the game.Regarding interaction, there are two kinds of shots that must match the enemies' weaknesses in order to have an actual effect, thus making the gameplay more complex and challenging.
The game can be played using either joypads or the Kinect.The latter allows players use both hands to fire shots, so there is an association between each hand and a specific type of shot fired.

Potentially Immersive Features
Our analysis of these games resulted in seven outstanding key features that were listed as potentially immersive, as corroborated by Ermi and Mäyrä [26]: • Audiovisual synchronization: the fact that both the video and sound react to each other, and to the players' actions makes the world feel more vivid and compelling.This factor might cause sensory immersion.
• Impressive graphics: the unusual art style, 3D models, special effects or animations inspire curiosity and wonder.Therefore, this factor might cause sensory immersion.
• Player's presentation: how the players are depicted in the virtual world, how the user interface aids gameplay, and what is their mission in the game should facilitate the feeling of empathy, i.e., a factor that also might cause sensory immersion.
• Mixed-reality technologies: technologies such as motion capture and skeleton tracking should modify the overall experience positively.This feature might cause sensory immersion as the sense of proprioception is being stimulated.
• Gameplay that inspires constant attention: the game must provide a pleasant activity that absorbs players into it.As complete focus is required, this is a factor that might cause challenge-based immersion.
• Feeling of danger or threat during gameplay: there should be a challenge that must be overcome.The notion of failure and success are great motivators, hence it is a factor that also might cause challenge-based immersion.
• Implicit story and symbology: the idea that players must come to their own conclusions about what is happening in the game inspires curiosity.This aggregates more to the experience rather than having texts to tell the story in an explicit manner.
It is a factor that might cause imaginative immersion.

IMMERSIVE GAME PROTOTYPE: SYNESTHESIA
A game prototype entitled "Synesthesia" was built based on the investigation we carried out concerning immersive game titles.Synesthesia serves as a tool for experimental validation of our hypotheses, hence it aims to meet all the seven potentially immersive features listed in section 4.

Game Concept
Synesthesia is a runner game.That said, the main character moves automatically along one axis (Z) and the player controls the movement along the other two axes (XY plane).This gameplay style was chosen due to the constant attention and commitment it requires from players.Moreover, the game rules are fairly easy to understand, speeding up the learning process when a new player joins our experimentations.
The main objective of Synesthesia is to complete all levels by avoiding obstacles.The avatar is allowed to move left or right, plus squatting or jumping.All obstacles are always presented in black with red edges.The player has four lives.The game ends when all levels are completed or all lives are lost.An early concept of the scenario can be seen in Figure 3.

Limitations
Due to technical, financial, and time constraints, the prototype could not comprise all of the exposed up to this topic.
In particular, the inclusion of customizable avatars was set aside to keep our focus on developing the seven key features aforementioned and to fasten the overall duration of the game.As a result, the players would engage in gameplay as soon as they start the game.We must recall that audiovisual synchronization, impressive visuals, and the use of mixed technologies are key requirements.It is worthy to mention that complex models require an extensive knowledge of 3D modeling.Moreover, organic, natural, and reactive enemy behaviors require complex artificial intelligence systems.Finally, reactive music requires a deep level design and music theory background.High-quality VR headsets such as the PlayStation VR were not accessible for development in our institution, so these were left out of the research.
An alternative to those features was the development of a physical installation.This environment was designed to aid gameplay and stimulate players' curiosity.It is composed of lights that automatically change colors and oscillate according to the music.The lights are positioned in strategic locations that provide visual cues to the players, so they are aware of their position in the real world without diverting their attention from the game.Our prototype supports the Kinect sensor as a controller.It is expected that the immersive impact of the customizable avatars could be replicated with Kinect.A minimal level of empathy can be achieved in the game through a graphic representation of the player, which is reinforced by providing skeleton tracking-driven avatars that mimic the player's movements and posture [6].

Game Design
In Synesthesia, players control a stylized character made of simple polygons, which is on the run to escape from an old arcade booth.Colors for the game palette and LED strips were chosen to represent the 80's aesthetics.Moreover, the game music was composed especially for this prototype in order to reinforce this aesthetic choice.The protagonist avatar becomes more red-colored according to how much health it loses.There are no items in the game capable of restoring health.This design decision was taken for the sake of challenge reinforcement.
All levels were designed to be quick by considering there is an actual physical effort to play the game.The full game is expected to last about five minutes in the best case, i.e., when the avatar does not lose a single life.A total of three levels have been carefully designed.Moreover, there is a "boss fight" at the end of the third level.
No texts besides the game logo were used in Synesthesia in order to accelerate the testing process, to facilitate the learning of the mechanics, to reinforce the aspect of implied narrative, and to avoid distractions.
Table 1 summarizes the design solutions we employed in order to meet all requirements stated by the seven potentially immersive features listed in section 4.

Physical Setup
The physical installation is organized as depicted by Figure 4, where the player stands in the center of the installation looking at a projection screen.A projector (3), the Kinect (2), and a computer (4) are placed at the front.The player is at a maximum distance of 7 meters from the Kinect.A computer attached to an Arduino board controls two 2-meter-side RGB LED strips placed at the sides ( 5), each about 3.5 meters apart from the center.There is another 1-meter LED strip put behind the player for decorative and ambiance purposes.These LED strips extend to a height of about one meter from the ground.Sound can be output from either the computer or the device being used for projection, e.g., an HDMI-compatible projector.Alternatively, a television can replace both the projector and it's projection screen when these are not available.Figure 5 depicts the full setup before connecting the Kinect.

Implementation
The game was developed using the Unity game engine [70] and the C# programming language.Kinect support was implemented on top of the Zigfu plugin [79] for Unity.Ambient lighting was installed from a 5-meter LED strip cut into smaller tapes.All lights blink according to the music.Beat detection is performed using spectral analysis over the music and consists of analyzing, at each instant of the audio clip, the relative volume at predetermined frequencies computed by means of a Fourier Transform.The communication between the game and the lights is handled by an Arduino board [3].The Fruity Loops Studio software [66] was used for music composition, Photoshop [1] for creating the arts and textures, and Blender [28] was the tool of choice for 3D modeling.

EXPERIMENTAL EVALUATION
Development of Synesthesia fundamentally aims to find out which features found in games can actually provide an immersive experience.Due to the subjective nature of immersion, this research experimental approach attempts to find evidence that these features can be applied into any minimally suitable game in order to obtain similar results.

Methodology
The experimental methodology adopted in this work was derived from the study of Charlene Jannett et al. [41], and consists of an attempt to quantify the emotions felt by the player during his or her experience.For that, a survey with 15 statements was elaborated, enumerated from 1 to 5 according to the degree of agreement.Values of 1 and 5 indicates, respectively, a total disagreement response and a total agreement response.The order of each sentence is shuffled randomly in the survey whenever a new user starts to respond it.
Sentences encompass features such as ambiance, feelings of suspense and danger, curiosity, responsiveness of controls, attention, and sense of presence.Backed up by the fundamentals discussed in Section 2, each sentence fits primarily into one of the following questions: "Does the game distract or draw attention into it?","Does the player feel present in the game?", "Is playing the game a pleasant experience?", "Is the game properly balanced?"and "Is the game fun to play?".In order words, each sentence could be separated in groups as follows: attention-related (A), presence-related (B), pleasure-related (C), balance-related (D) and fun-related (E).This organization is shown in Table 2.The visuals pleased me.9 The controls responded adequately.10 I didn't care about what was going on in the game.
(D) Balance-related statements 11 I was anxious about losing.12 I think this game is hard.
(E) Fun-related statements 13 I had fun playing this game.14 Winning didn't seem important to me.15 I would play this game again if I could.
Demographic data include sex, age, how many hours per week a participant spends on playing games, and what kind of gamingrelated label a participant would use to describe himself or herself.
After the quantitative test, there were a room for free commentary and an option to mark, based on the participant's own concept of immersion, how much he or she felt immersed in the game on an integer scale ranging from 0 to 10.
There was a brief interview about the user experience accompanied by the survey.Case-specific questions were asked during the interview: in the Rez experiment, for example, it was asked if the player realized whether the joypad vibration option was set on.For Child of Eden, the questions were related to the experience of playing with the Kinect.Finally, in Synesthesia, participants answered about the relevance of the physical setting.Details on the experiments will be described next.
Unfortunately, this methodology might bring a few inaccuracies: a participant' idiosyncratic understanding of survey statements affects the observation of her or his opinions.The meaning of terms such as "casual gamer" and ' hardcore gamer", as used in the demographics section of the survey, may vary according to each person.The very concepts of "fun", "presence", and "immersion" may also prove rather difficult to evaluate, therefore it was necessary to observe each participants' gameplay.
Finally, pertinent equipment such as eye-tracking devices, heartbeat monitors, and screen recorders had to be left out of this experiment due to time and budget limitations.So, analyses described in this work are mostly based on the interpretation of qualitative data.

Experimental Setup
The games were tested in dark, closed, properly air-conditioned, and quiet environments.Each game title was displayed using an Epson Powerlite 93 projector [34] via HDMI connection and the game audio was emitted through that projector's speakers.Tests were considered time-consuming, since each person took between 10 and 20 minutes to complete the proposed tasks.For this reason, about 10 people were expected in each of the tests.A total of 36 participants filled the survey.Corresponding results for each test are described in the next section.

Experiments
Tests carried out in December 2016 are described in subsections 6.3.1, 6.3.3, and 6.3.4.In their turn, subsections 6.3.2 and 6.3.5 address tests performed in January 2018.

Rez HD (Demo)
A free demo version of Rez HD on the Xbox360 was used for evaluation purposes.This experiment sought to analyze the immersive impact of audiovisual synchronization and challenge.Participants interacted with this game using the console's standard controller with the joypad vibration feedback turned on.By doing this, the controller vibrates with different strengths, frequencies, and balances depending on the current beat of the song.The demo version of Rez HD ends when the player deals damage equivalent to a quarter of the boss's health at the end of the first level.
Ten people took part in the experiment, whose game sessions lasted for about seven minutes.All participants agreed to complete the survey after the interview.Interestingly enough, two people actually asked to play again a few times.

Rez Infinite
In the test using Rez Infinite, players were asked to play the first level using the default control schemes, this time with no restrictions.Joypad vibration was turned off.Unlike the Rez HD test, participants played with headphones with adequate volume so they could not hear external sounds.
The main goal of this experiment was to the contrast the tests between the demo and full versions.Moreover, we could analyze the impact of haptic feedback and sound effects.Though there are revamped textures in the Infinite version, the Xbox 360 version did not look graphically inferior enough to bother players.
Four people participated in this test.Of those four participants, two played more than once.All participants readily agreed to complete the survey.

Child of Eden
Tests of the game Child of Eden were also carried out on Xbox360.However, the user input provided via Kinect was adopted.This allowed us to analyze the immersive impact of motion capture technologies.Moreover, the game was played with the "Feel Eden" mode turned on.This makes players' avatars invincible, therefore it was expected that participants would report a low level, if any, of challenge-based immersion.
Only the first level was played since it takes a considerable amount of time.Eight people participated in the tests, whose game sessions lasted from 8 to 12 minutes.All participants agreed to complete the survey after the interview.Synesthesia was first evaluated with the LED lighting feature turned on.Synesthesia ran on a 15-inch MacBook Pro (mid 2014) with a dedicated graphics card [33].This experiment sought to analyze the importance of the physical environment as well as to assess the effectiveness of the immersive features we implemented in this prototype game.
The tests were conducted between 2 p.m. and 5 p.m. in a room with a slight incidence of sunlight, which made the LED seem less brighter than normal.In preliminary tests, it was noticed that the brightness was excessively strong and bothered players.The interference of sunlight has become a intensity regulator factor for the LED strips.Figure 6 shows the light intensity at 5 p.m., when there was negligible interference from the sunlight coming from the windows.Eleven participants attended this test.

Synesthesia (without LED Lights)
Players were asked to play the full game, as many times as they wanted, without the LED lights arranged in the room.The main purpose of this test was to assess the relevance of the physical installation to the overall experience.
Despite attracting many people around the room interested in the experiment and the prototype, only three users participated in this test.Tests lasted about ten minutes per person.

RESULTS
In this section we discuss about the observations of player behavior during the tests.We also analyze data obtained through the survey and interviews.

Participant Profile and Reported Immersion
An average participant was a male aged between 18 and 22 years old who usually plays games for at least 2 hours a week and considers himself a casual player.This is shown in Figure 7. Interestingly though, no teenagers toke part in the experiments and, despite describing themselves as casual gamers, only about 10% of those who responded the surveys play less than 2 hours a week.Hence, the terms "casual" and "hardcore" are not perceived as time-dependent.Thirteen participants labeled themselves as casual players while stating they play at least 5 hours per week.We argue that it is reasonable to assume that most users were skillful enough to play Synesthesia and the titles we selected for this research.
With an average rate above 7, selected games ranged from "slightly immersive" to "immersive".CChild of Eden is the least immersive title according to the results of the survey, whereas Rez stands as the most immersive.Figure 8 presents the average immersion rate reported by users in each experiment and the average of all experiments combined.Each experiment will be analyzed in the following subsections.

Rez HD (demo) experiment analysis
Many players have had some strangeness with the graphics.The interface, according to them, was confusing at times, and showed unnecessary elements.Players seemed to have had a bit of difficulty understanding that it was necessary to destroy a floating cube in the background to proceed to the next area.They also did not immediately figure it was possible to shoot several enemies simultaneously before firing by holding the aim button.Five of the ten players  2.
reported that they play more than eight hours a week, which is consistent with the fact that despite these initial difficulties, players had quickly mastered the mechanics.Furthermore, despite being considered confusing, Rez's visuals were regarded as beautiful: most players responded to statement 8, "The visuals pleased me", with an average score of 4 out of 5 (see Figure 9).
Several users displayed clear signs of attention to the game: they beat their feet to the rhythm of the music, tilted their spine toward the screen or gaped in wonder.The controller vibration was on, however only two players reported feeling it during the interview.Some participants stated the vibration was not on and, after the interview, played the game again.They felt surprised and confused when noticed the vibration was actually on.Only one player noticed that the protagonist both vibrates to the music and changes its forms according to the player's performance.These reactions match the values for responses on attention-related statements (group A).Statement 4 ("For a few moments I forgot what was happening in mysurroundings"), for example, shows a high value of 4.4.This suggests that players felt very absorbed by the game, moreover the minimalist design seemed to please and facilitate learning.
Three players did not manage to complete the game, but replayed it until the end of the available content.Players in general felt frustrated when the demo version's boundary message appeared during the battle against the boss and locked the controls.Eight players said that the game made them feel curious.Nine stated that Rez HD is fun and would play it again.Due to this, statement 15 ended up having a notably high average score of 4.7.
According to the participants, the sound was the game element that stood out the most.They stated that the music matches the graphics and overall theme.Much of the psychological atmosphere was due to the beat of the music, which indicated moments of tension and relaxation.
It can be concluded that vibration, by providing tactile feedback, has made the experience more immersive.The stress generated by the challenge was seen as a positive factor.Balance-related statements (Group D) average responses range between 2 and 3, meaning Rez HD's challenge level fit the participant's capacity adequately.Players showed greater interest in the battle against the boss.

Rez Infinite Experiment Analysis
Players in general demonstrated awe upon reaching level layer 4, where it becomes clear that the music builds up matching their progress in the game.
The game was described as easy to learn, though the interface was reported as insufficiently clear.There seemed to be an initial disapproval of graphics, that looked dated according to some par-  2.
ticipants.This discomfort was later subdued by the constant action and animations, as well the construction of scenarios during the gameplay.
Volunteers demonstrated signs of attention such as playing with mouths open in moments that required faster reactions, leaning the body towards the screen, and beating their feet according to the background music.Two players felt uncomfortable, which might point that the test was lengthy.One volunteer who spent over twenty minutes playing the game stated that he felt time passing faster than usual.The overall sound composition received unanimous praise.
Results obtained from this experiment were similar to the experiment in Section 7.2.However, Rez Infinite was perceived as being more difficult than Rez HD.It could be assumed that most of the first level's challenge was placed into the boss fight, which is purposely absent in the Rez HD demo version.Scores given to Statement 10 ("I didnt care about what was going on in the game.") were notably lower, which might indicate the participants who played Rez Infinite were more interested in understanding the overall plot and theme than the one ones who played Rez HD (see Figure 10).The participants of this experiment were considerably more interested in reaching the end of the level than the ones who participated playing the demo version.The possible reason behind that could be the in the demo release: it is simply impossible to reach the game's end, as a message blocks the player's progress upon damaging the boss.
There was no conclusive evidence that haptic feedback by itself enhanced the user experience.Players in this test did not miss this feature when asked nor felt that their experience would be improved by using it.However, since players in the previous test were consistently surprised by the fact that they were playing with vibration on, further studies might be required in order to evaluate whether this particular feature really reinforces sensory immersion.There is no clear evidence supporting whether their surprise was due to vibration becoming something that "feels natural" or because it made no difference at all in gameplay.

Child of Eden experiment analysis
The first thing the participants noticed was the beauty of the game.Participants reported several times that the visual aspect and the audio treatment, besides their interactions, were quite impressive.The sci-fi theme was also appreciated.The game was taken as relaxing and de-stressing.It has been said that the experience of playing Child of Eden in general is pleasant.Two of the participants finished the game session in wonder.
On the other hand, some players reported negative points as well.They stated in the interview that they felt uncomfortable with some parts of the levels, because the enemies did not stand out enough, or did not seem to have an "enemy-like behavior".The colors Players also complained that they often shot everything that moved simply because they did not know what they had to shoot.They also complained about the use of their hands as pointers, which was considered tiring.Also, Kinect's motion capture was not accurate enough for moments that required quick reaction.Statement 9 ("The controls responded adequately.") received an average response of 2.5, pointing to a general disapproval of this input method.It was verified that, when the Kinect lost track, players "returned to their body" and lost focus: this suggests a break of immersion.In addition, they found the game's viewport was narrow and did not help to look in the right direction.It was also mentioned that the narrative could not be easily understood.
When asked about the use of the Kinect as input device, most participants said this is an interesting choice since the device offers a different way to play, but it is not essential and it does not necessarily make the user experience better than playing with a traditional controller.
After analyzing the survey, it was observed that Child of Eden easily catches people's attention, but can not maintain it in a consistent way for a long time since it does not arouse suspense or curiosity.The game was not considered difficult, but was classified as having a slow learning curve.
The experience was set to be purposely contemplative when using the "Feel Eden mode".Statement 11 ("I was anxious about losing.")was the lowest among all tests and statement 14 ("Winning didnt seem important to me.") was the highest, confirming this.Removing the element of stress, which is the fact that the player cannot lose, may have influenced players' perception of the game's challenge.On the other hand, players felt frustrated even with endless lives.This fact raises questions about whether the lack of challenge was in fact an important factor for this experiment.
The audiovisual aspect proved to be essential.The story, however, did not.Players did not praise the story or understood what was exactly happening in the game by themselves, which reinforces that immersion in Child of Eden happens because of its audiovisual qualities.
Comparatively, Child of Eden's responses tended to demonstrate players did not enjoy this game as much as the others.During the interviews, participants spent more time highlighting what they did not like more than what they enjoyed in the game.This is also seen in the chart presented in Figure 11.It is important to point out that the results of this experiment should not be used as evidence of the game's quality.Instead, this result shows that many of the parameters chosen -notably user input and gameplay modes -hindered the overall user experience.

Synesthesia with Lights Experiment Analysis
Figure 12: Average responses for the Synesthesia with lights experiment.Labels described in Table 2.
Only of eleven of seventeen players responded the survey.The results are shown in Figure 12.Among these seventeen, two did not finish playing all game levels.These also neither responded to the survey nor participated in the interview.
Initially, players complained about the viewport, which made the perception of three-dimensionality complicated as shown in Figure 13.Artifacts such as tiled flooring and differentiated colors on the tile where the player stands have not proved to be sufficient.Many obstacles have become frustrating as they seemed to record mistaken collisions with the avatar.The avatar had really collided with them but players misinterpreted it due to the camera's perspective.
Only one of the seventeen participants was able to complete the task without ever dying in-game or having any difficulty.There were complaints about the accuracy of capturing the extremities of the body via Kinect, specifically the head and feet, which were often lost by the camera.Contrary to this, Statement 9 had an average score of 4.27, therefore it is safe to assume these tracking imprecisions did not disturb players.There was no difficulty in understanding game rules and objectives.
Players praised the physical setting and the theme of the experiment.The LED were regarded as an accessory and served as a mark of the boundaries of horizontal movement in the real world.Eight players noticed that the lights blinked to the beat of the music.Six of the eleven players who responded the survey stated that the lights improved the experience.Finally, all seventeen participants stated that the experience would be less interesting without LEDs.It was suggested that, in order to make the use of lights more relevant to the game, they should react to game events as well, e.g., as blinking when an enemy appears or the avatar collides.
It was mentioned three times that using Kinect was a positive factor.When asked why, players said that it is rare to see games created specifically to be played with Kinect and that they are used to play dancing games with it.The Kinect, however, was stated to be tiresome to use.
Although the avatar always moves automatically over the Z-axis with the same velocity and levels last progressively longer, players reported that they felt that the levels became faster as time passed.The use of relevant physical space showed increased interactivity with the system.It also became interesting to watch.People who were watching felt instigated to the gameplay.
The cohesion between thematic, audiovisual resources, and physical space configuration was highlighted by the participants.They easily identified the retro-inspired style without needing texts or further explanations.The game plot was not perceived by players, which fortunately did not appear to undermine the experience.
The music and amount of obstacles on the screen combined with the flashing lights altered players' perception of time.The participants were quite surprised with the boss (Figure 14), at which time there was the highest peak of interest in the game.This enemy constantly threw balls that moved towards the player.Unlike other obstacles, the boss stalks players before unleashing its attack, so players could not find a safe spot and keep being there.
Overall, players felt satisfied with the experience.Statement 3 ("The game didnt catch my attention.")had the lowest score among all experiments, meaning the game was indeed unusual and attractive.Statements 13 ("I had fun playing this game.")and 15 ("I would play this game again if I could.")had high average scores of 5 and 4.81, respectively, which further corroborates the hypothesis that the game was also fun and engaging.

Synesthesia without Lights Experiment Analysis
Players found the game unusual and praised the use of motion capture.Two out of three stated they wanted to play the game because they felt curious about it.
Contrary to the experiment with a proper real-world ambiance (subsection 7.5), participants did not report full immersion, as they were aware of their presence in both virtual and real worlds most of the time.Nevertheless they reported a feeling of "time passing  2.
faster".One of them even danced to the music, even though it was not necessary in the gameplay.
Visuals were well received, but most players complained about collision boxes and motion capture.These players exhibited clear expressions of confusion while playing, which harmed the overall experience.None of them wanted to finish the game.
The game was not regarded as easy nor hard.This report might be misleading as imprecision with the Kinect might lead to an unintentional difficulty.All users reported the camera was not clear enough to estimate the distance between the avatar and obstacles.Two out of three players quit the game after losing consecutive times in the same level section.One of these players refused to play even with the possibility of skipping the entire level, while the other played until level 3's midsection.
The overall experience seemed really engaging.Players looked happy while playing, even with imperfections and inaccuracies during gameplay, which attracted a large crowd nearby to watch it.By having different themes on each level, the game successfully caught players' attention, who unanimously affirmed in the survey to feel curious about what would happen next.
Repeating the evidence found in the test without lights, the game was regarded as easy to learn, highly re-playable, and fun.This fun was not necessarily linked to the game: many users danced or experimented with the Kinect in a joyous manner similar to a child learning how to play with a new toy.Even though the three participants did not have the chance to play the games with a proper multimedia installation, the possibility of having LED lights synchronized with the music was admired.Some participants asked if there would be another test, this time with a full setup.
Results from this experiment (see Figure 15) were similar to the ones from the experiment in Subsection 7.5.Statement 3 had a higher average response in this experiment and Statement 4 had a lower one.This means that the lights actually help players to keep their attention into the game.Results on group B differ only in Statement 6 ("I was aware I was in a room playing this game while playing it."),suggesting that a proper physical setup helps to immerse players.The results also show that the synchronized LED lights compounded a significant part of the visual presentation.So, removing them makes the game less impressive in that regard.Group D largely stays the same, which was expected as the lights were not a crucial gameplay element.Group E's only change was that players cared less about winning the game.
This experiment provided sufficient evidence to conclude, based on the participants' actions, behaviors, and statements, that LED lights were indeed accessory and actually help players to keep focused on the game.Despite that, it is still inconclusive whether the lights themselves could be considered an immersive element, since no volunteer participated in more than one of these experiments.Further investigations and a different testing methods would be required to prove this point.Particularly, since even if there were common users between the two tests, their experience would be slightly different due to each game's predefined paths.As described in all tests, presenting players with a level design which constantly surprises and amazes them was found to be a conspicuous factor regarding the achievement of immersion in a game.

CONCLUDING REMARKS
Our experimental results lead to several conclusions about the immersive experience found in games.Synesthesia successfully fulfilled 5 of the 7 potentially immersive features pointed out in this work.However, it is reasonable to assume that we were able to achieve 5 out of 6 of those features since we purposely did not implement a credible player representation in our prototype.Table 3 shows, according to the tests performed, in which experiments the characteristics listed were proven to be actually "immersive" from the users' perspective.The cohesion of audiovisual resources was identified as the most immersive factor.According to the interviews and the survey, such information suggests that, first and foremost, developers must design a credible virtual world to provide the feeling of presence in that world.The method of synchronization between audio and video in the Synesthesia game did not have sufficient prominence to be considered an immersive characteristic as the physical environment was admired but was not considered essential in the way it was implemented.However, it is worthy to observe that the method used in Rez was much appreciated.
Experiments have shown that a player's representation on screen gives better identification of their persona in the virtual world.Rez showed that it becomes easier to feel present in a game with a credible avatar along with a cohesive user interface.However, implementation of this feature was discarded in Synesthesia and left as a future work.
The use of mixed-reality technologies has proved to be a differential when the game in question is developed around it.In Child of Eden, its use is optional and it gives rise to imprecision during user interaction.This issue does not manifest when the game played using a joypad.In the case of Synesthesia, such inaccuracies inherent to Kinect did not make the game experience impracticable or even reportedly less fun.
The gameplay that demands constant attention was found to contrast with another point in this research: that of implicit plot and symbology.Rapid and intense action prevents players from spending too much time admiring the background and looking for information about the game's story.In contrast, the gameplay reinforced one of our working hypotheses: the sense of danger and threat are strong immersive factors.This was specially true in the experiment using Child of Eden in "Feel Eden" mode, where no challenge at all was proposed and the game was regarded as less immersive than the other two evaluated in this research.
Based on the exposed, we propose the following guidelines for building immersive experiences in electronic Games: 1. Mixed Reality Technologies: games that use MR technologies should be designed with these in mind from the beginning.Users reported more enjoyment when MR technologies become an essential part of the experience instead of an accessory or alternative way to play.
2. Audiovisual Synchronization and Player's presentation: worlds become more vivid and users play a more agent role when the virtual world constantly reacts to their actions.Having an avatar or user interface which displays a real-time representation of progress (and regress) motivates users to remain playing the game, as seen in Rez.
3. Graphics: virtual worlds must become credible to propitiate the feeling of presence.This should be achieved by the quality of 3D models, art style and direction, user interface, environmental design, and most importantly, how cohesive the visuals look with other elements such as gameplay and story.
4. Challenge: An adequate difficulty balance along with peaks of tension, such as a boss battle in the end of a level.Such balance in a game that requires constant attention and quick reaction times from players facilitates immersion by challenge.
There should be as few distracting elements as possible, as the action was reported to be the most important immersive factor.
Finally, our experimentation also raised more questions that could be answered in subsequent works: The experiment with Rez opens the possibility to test the immersive effects of virtual reality in VR headset support technologies, such as the PlayStation VR, available in the Rez Infinite version.Moreover, observation of players' behavior may also suggest the existence of physical signs of immersion.For future studies, heart rate monitors and eye tracking devices can be used in order to evaluate if, starting from the hypothesis that Rez is an immersive game, oneself can deduce what physiological changes could happen to an immersed player.
The experiment with Child of Eden opens two specific questions: what would be the result if the tests were done without the Feel Eden mode checked?Given that the game allows the use of traditional control and motion capture, what would be the opinion of the players if they had tested both cases?The experiment with Synesthesia opens the possibility of tests related to media besides games.What would be the effect of the LED strips if the tests had been done at night, with no incidence of sunlight in the room?What if ambient lights had been applied to other media, such as movies?The evolution of the proposed environment for the developed prototype also gives precedence to the creation of immersive interactive multimedia installations.

Figure 1 :
Figure 1: In Rez, players play a hacker whose avatar navigates through a geometric sci-fi world.Source: Xbox Live Market Place Originally released for PlayStation ® 2 and Dreamcast ® in 2001, and remastered for Xbox ® 360 and PlayStation ® 4 in 2008 and 2016 respectively, Rez tells the story of a hacker who aims to eliminate viruses in Eden, an artificial intelligence that controls a super computer network.As of 2017, the Infinite version is also available to personal computers and mobile devices running Android compatible with Google Daydream.The game features wireframe-like graphics inspired by the paintings of Wassily Kandinsky[22], which evolve into more complex shapes.A sample game screen can be seen in Figure1as published by Xbox Live Market Place[46].Gameplay elements are synchronized with music and sound effects.Among those, both the scenery and the main character vibrate in conjunction with the music's beat.Enemy characters, in their turn, appear or disappear from the screen according to the song.The player cannot fully control the camera, but is allowed to shoot at enemies and obstacles.When destroyed, both of these emit harmonic tunes that are combined with the soundtrack.Consequently, the music gets richer as the player successfully destroys something.Rez's levels songs are segmented into ten parts.A new part is stacked with the previous ones as the player progresses.The music parallels the player's progress: the more he or she advances, the more "complete" the music becomes.Rez's game interface depicts a terminal which simulates the execution of an algorithm.Concerned about immersion, the game developers created the Trance Vibrator © , an accessory that made the player's wrist vibrate according to the song[54].The accessory was compatible only with early versions of the game.In subsequent releases, tactile sensations were stimulated using the joypads' motors.The latest release supports PlayStation ® VR, a virtual reality headset on the PlayStation ® 4, and other VR headsets on PC and Android.These "remaster" versions feature an exclusive area specifically designed to offer a unique VR experience[50] Mark MacDonald, who worked in the development of Rez Infinite, states that it benefits greatly from the sensory deprivation caused by the combination of a VR headset and a headphone[64], even though the game itself allows full 3D interaction.

Figure 2 :
Figure 2: Child of Eden art style presents a cybernetic though organic world with vivid colors and natural shapes.

Figure 3 :
Figure 3: Concept art of the scenario of the second level of Synesthesia.Aesthetics are inspired by arcades popular back in the 80's.

Figure 5 :
Figure 5: Actual physical installation following the proposed layout, as described by Figure 4.The Kinect should be connected to the computer and placed anywhere in front of it.

(
A) Attention-related statements 1 The game distracted me onto it.2 I was curious to know what happened next.3 The game didn't catch my attention.4 For a few moments I forgot what was happening in my surroundings.(B) Presence-related statements 5 I felt like I was inside the game for a few moments.6 I was aware I was in a room playing this game while playing it.7 I had the impression that time flowed fast.(C) Pleasure-related statements 8

6. 3 . 4 Figure 6 :
Figure 6: The room where the project was installed and tested.Picture taken approximately at 5 p.m.

Figure 7 :
Figure 7: Age, gender, weekly gameplay hours and self-assigned gaming labels distributions among participants from all experiments.

Figure 8 :
Figure 8: Players were asked to rate how immersed they felt using a score ranging from 0 to 10.This chart shows the average response from every experiment.

Figure 9 :
Figure 9: Average responses for the Rez HD experiment.Labels are described in Table2.

Figure 10 :
Figure 10: Average responses for the Rez Infinite experiment.Labels described in Table2.

Figure 11 :
Figure 11: Average responses for the Child of Eden experiment.Labels described in Table2.

Figure 13 :
Figure13: The camera's perspective in Synesthesia was reported to hinder players' perception of the 3D space.

Figure 14 :
Figure 14: Synesthesia's boss.Players felt surprised by the contrast between the static nature of obstacles and the reactive nature of this enemy.

Figure 15 :
Figure 15: Average responses for the Synesthesia without lights experiment.Labels described in Table2.

Table 1 :
Application of the listed features in Synesthesia

Table 2 :
Table listing and categorizing all statements present in the survey

Table 3 :
Potentially immersive features and corresponding games in which each feature was found to be immersive.