Analysing Balance Loss in VR Interaction with HMDs

— Immersive virtual reality (VR) has been used in different fields such as training, educational programs, entertainment, psychological treatments, and rehabilitation. Despite its broad utilization, some issues limit its application such as the loss of balance. Balance is disturbed because visual stimuli received from the virtual scenario are not in harmony with perceived stimuli by the proprioception and vestibular systems that remain in contact with the real environment. With the increasing popularity and accessibility of high-quality VR systems, concerns have been raised about the propensity of VR to induce balance loss. Balance is essential for safe use of VR experience and its loss can result in severe injury. In this work, we present a methodology and the necessary tools to quantify the influence of VR on the user’s balance and assess risk of falls during VR interaction. By means of an experiment making use of an Oculus Rift and a MS Kinect Sensor, we observe, quantify and compare the effect of VR scenes with different levels of danger on the balance of users, as well as the effect of visual and auditory warnings of balance loss. Results suggest that auditory signs were not effective in warning users about risk of fall, and that the order which the scenes are presented to users affects their behavior. Users who were first presented to a more challenging scene proceeded more carefully and most of the time carried this behavior to the other less challenging scenes.


I. INTRODUCTION
Up to a few years ago, Virtual Reality (VR), as an emerging technology, seemed to be far from mainstream adoption, mostly due to the high cost of VR gear.However, with the launch of more affordable and easy-to-use devices, such as the Oculus Rift [https://www.oculus.com/rift/],HTC Vive [https://www.vive.com/]or Samsung's mobile gear VR [http://www.samsung.com/global/galaxy/gear-vr/],the use of VR is rapidly gaining popularity among consumers.In 2016, 1.43 million units were sold, and sales are estimated to exceed 6 million by the end of 2017 [1].
Balance is a complex skill composed of three subsystems that work together to keep us aware of our surroundings and give us the ability to react to current conditions and prepare for future changes.These subsystems are the proprioceptive system, the vestibular system, and vision.Immersive VR completely changes our visual perception of the surrounding environment and this can, and most of the time does, provide information contradictory to that which we perceive through the other subsystems involved in balance, like the proprioceptive system and the vestibular system, producing discomfort, motion sickness, and loss of balance [2].This problem must be addressed if VR is to become as ubiquitous as mobile technology.This is especially true for the growing use of VR in areas other than entertainment, like rehabilitation [3] [4] [5] and education for training simulators [6] [7].
In this work, we study the problem of fall risk during VR interaction.In order to do this, we present a methodology and the necessary tools to quantify the influence of VR on the user's balance and assess the user's fall risk during VR interaction.By means of an experiment making use of an Oculus Rift and a MS Kinect Sensor, we observe, quantify and compare the effect of different VR scenes on the balance of the users, as well as the effect of visual and auditory warnings of balance loss.This information can serve as a knowledge base for future works aimed at tackling this important issue.
To address the aforementioned problem, we created a tool able to measure the balance of users during interaction with VR systems.In order to do this, we researched about balance and the means that the human body achieves it from a physical and physiological standpoint.The literature suggests as a good measure of user balance, the analysis of the speed and location of the center of mass (CoM) within the user's support area.
To calculate the center of mass and the support area, we revised current literature searching for an algorithm that could calculate the user´s CoM without requiring expensive equipment or a calibration process.We implemented a kinematic method capable of calculating a user's CoM with the body tracking data obtained from the Microsoft Kinect SDK as presented in [8].This algorithm relies on anthropometric data to calculate the location of CoM of different limbs and then calculates CoM as a weighted average.Fall risk was calculated based on the distance of the projection of CoM onto the supporting plane, to the closest edge of user's support area.
We then used these tools to create a warning that could help the users maintain their balance during their interaction with VR.We created two types of warnings: visual and an auditory.Auditory feedback was based on the hypothesis that it would act as a biofeedback reinforcing the sense of balance.The visual feedback was based on the hypothesis that diminishing the visual information would make the balance system rely more on other sensory information that was not being affected by VR.The auditory warning consisted of a three-dimensional (3D) sound that indicated the direction in which the user was falling.The visual warning was a red tinting of the environment whenever the user was at fall risk.
To test the developed tools, we designed and conducted different experiments in which we explored the influence of different VR environments on the user´s balance.We recorded users while walking on a set of wooden plates laid on the floor of test area in a specific order back and forth without the use of a VR headset and with the use of Oculus Rift VR headset in two VR environments.One VR environment similar to the testing area, this environment contained 3D models of the wooden plates on which the users walked on.Another VR environment had the wooden plates were represented by tall wooden pillars over a dangerous rocky sea.We then compared user´s balance states in the different situations with and without fall risk warnings.This was achieved by recording the time series of the trajectory of the user's CoM denoising CoM trajectories using the Ramer-Douglas-Peucker algorithm [9] and comparing the trajectories.To compare the CoM trajectory of the users in the different environments, we implemented a shape comparison algorithm based on the turning function of the CoM's trajectory.We also compared the user's average speed and time of completion of the experiment.Summarizing, this work presents a methodology and the necessary tools to quantify the influence of VR on the user's balance and assess the user's fall risk during VR interaction.This paper is structured as follows.Section 2 presents the basic concepts of this work and summarizes the related work.Section 3 covers the proposed solution and its technical aspects.Section 4 presents the testing methodology of our experiments.Section 5 presents the results along with the corresponding statistical analysis and discusses different hypothesis about the meaning of the results.Finally, Section 6 concludes the paper.

II. BACKGROUND
This section provides an overview of the main concepts related to this work.Initially we cover concepts related to human balance, and then cover topics related to the center of gravity/center of mass.Finally, we discuss related work such as the center of mass calculation and previous studies of visual induced motion sickness and fall risk in virtual reality interaction.

A. Balance and Center of Mass
Balance is the ability to maintain the body's center of mass over its base of support [10] and it depends on the forces and the moments (torque) applied to that body.Balance is achieved by the joint effort of three subsystems working together to keep us aware of our surroundings.They give us the ability to react to current conditions and prepare for future changes preventing us from falling.These subsystems are the proprioceptive system, vestibular system, and vision [2].
The proprioceptive system gathers information from the skin, muscles, and joints, and involves sensory receptors that are sensitive to stretch or pressure in the surrounding tissues to achieve spatial awareness of our own body, this means, we are subconsciously aware of the location of every limb relative to our body.
Sensory information about motion, equilibrium, and spatial orientation are provided by the vestibular apparatus, which in each ear includes the utricle, saccule, and three semicircular canals.The utricle and saccule detect gravity (information in a vertical orientation) and linear movement.The semicircular canals, which detect rotational movement, are located at right angles to each other and are filled with a fluid called endolymph.When the head rotates in the direction sensed by a particular canal, the endolymphatic fluid within it lags behind because of inertia and exerts pressure against the canal's sensory receptor.The receptor then sends impulses to the brain about the movement from the specific canal that is stimulated.When the vestibular organs on both sides of the head are functioning properly, they send symmetrical impulses to the brain.(Impulses originating from the right side are consistent with impulses originating from the left side) [2].
Particularly, vision greatly influences our perception of gravity and is our main information cue to prepare for changes.These complex interlacing feedback mechanisms can be disrupted when our brain receives visual information contradictory to the information received from the other subsystems.This is what happens most of the times in immersive VR, producing discomfort, sickness, and loss of balance.
Fig. 1 Example of poses with CoM inside support area [14] We know from physics that objects and users under the force of gravity will not fall as long as its center of mass (CoM) remains inside its support area.Moreover, within the support area there is a limit of stability, which expresses the area inside the support base region that an individual utilizes to get balance [11].In physics, the center of mass of a distribution of mass in space is the unique point where the weighted relative position of the distributed mass sums to zero or the point where if a force is applied causes it to move in the direction of the force without rotation [12].The normal force acts on the user's support area perpendicular to the floor, the gravity force acts down on the user´s body.These forces will cancel out as long as the user's CoM remains inside the support area.In order to maintain static equilibrium, the user's vertical projection of CoM must remain inside of its support region.The support region is the convex hull of the user's feet [13].An example can be seen in Figure 1 obtained from [14].There are many known good methods to calculate the user's CoM.In [8] a comparison of such methods is presented and the kinematic method is the most commonly used in balance studies.Further description of this algorithm proposed by [15] will be discussed in the following sections.
CoM is the unique point at the center of a distribution of mass in space that has the property that the weighted position vectors relative to this point sum zero.In analogy to statistics, the center of mass is the mean location of a distribution of mass in space [12].This physical imaginary point, the CoM of the human body, represents a relevant gait analysis variable.CoM behaviour can summarize the whole-body movement and the translational vector for the momentum of the body mass.The threedimensional (3D) trajectory of CoM is the most representative variable in order to analyze human locomotion in various conditions.
The knowledge of the exact position of CoM of an object or a body is very important to establish the stability of the system [16].Stability is "the capacity of an object to return to equilibrium or to its original position after being displaced" [17].The stability of an object is affected by "the height of the center of mass, the size of the base of support and the weight of the object" [18].It´s clear that the location of CoM relative to the user´s support area is a key factor in determining body stability and the fall risk.

B. CoM Calculation
Due to the importance of the estimation of the CoM for the study of human biomechanics, many studies have been done in this area.The literature presents three different methods to obtain the CoM.In the kinematic method, CoM is obtained by getting the position of body segments in a determined instant, combine them with inertial parameters, such as segments weight, and estimated CoM of each segment.It requires the use of cinemetry and anthropometric models of human body.Another method estimated the horizontal component of the CoM using a double integration of horizontal acceleration (applied horizontal force divided by body mass).However, it is very difficult to find the initial position and initial velocity of a body after the double integration.Finally, the third method is based on the relationship between the center of pressure and the horizontal displacement of CoM in the frequency domain.This method models the body as an inverted pendulum.Some studies propose the use of force platforms [8].Force platforms can estimate CoM displacements since they track the resultant vector of the ground reaction forces that is equal and opposite to the weighted average of the location of all forces acting over the platform.Then, the force platform captures the combined response of CoM displacement and its position [11].Center of Mass projection can be calculated by means of a low pass filter on the pressure points obtained with a force plate.Other works suggest the use of pressure plates (force plates) and cameras such as the MS Kinect [19].In this work, the authors present a method that calculates a statistically equivalent serial chain (SESC) obtaining a subject specific estimation of CoM.This allowed an average error of 26.6 ± 6.0 mm compared to an average error of 118.4 ± 50.0 mm for the estimation of the subjects CoM with an anthropometric model based CoM estimation described by [16].This came at the cost of using a force platform, having the subject standing on top of it and holding 40 static postures, each lasting 5 s.For our purposes of quantifying the influence of VR on the user´s balance and creating fall risk warning, the average error of 118.4 ± 50.0 mm was sufficient, and we wanted to present a methodology that could evolve into a non-intrusive and a low cost solution.The requirement of a force plate and a preparation phase was incompatible with our objectives.This prevented the use of the method proposed by [19].After analyzing different methods, we concluded that the method that best meets our needs and restrictions is the kinematic method as suggested by [8] and presented in [15].This method will be described later.

C. Balance and visual induced motion sickness studies in VR
The safe, standardized and controlled virtual environment allows analysis and tests that would be very difficult to be replicated in the real world [20].This feature has been motivating the use of immersive technologies in many fields, especially for the assessment of balance or for its training [21], [22], [23].In parallel, the launch of relatively low-cost devices, such as Oculus Rift, HTC Vive, Microsoft Kinect and Nintendo Wii, for example, spurred research and new applications using VR.However, VR systems still present a series of limitations that several times are omitted or underestimated [20].
Several studies have been conducted aiming to evaluate or to minimize the conflicts among visual, proprioceptive and vestibular systems when participants were immersed in VR experiments, such as [24], which emphasized the importance to minimize the latency of VR systems.
In addition, many studies have been conducted regarding fall risk analysis, especially on the elderly and physically impaired.This is due to the high economic and humanitarian cost of accidents involving these groups.Despite having a great volume of studies on fall risk, we managed to find just a few regarding fall risk during the use of Virtual Reality.Chiarovano and colleagues [25] conducted a study on the influence of a VR three-dimensional visual scene and a VR three-dimensional field of dots moving in one particular trajectory on the balance of test subjects.The balance score was calculated using a Wii Balance Board (WBB).The WBB is capable of measuring the center of pressure.Under no other external forces other than the force of gravity, the center of pressure (CoP) is the projection of CoM on the support surface.The study used CoP measurements gathered with the WBB, these were standardized by calculation of a score measured (in %).For each CoP measurement, a value between 0 and 1 was calculated according to the formula:  = 1where  is the distance from the CoP measurement to the center (upright) position, and  is the distance from the subject's limit of stability to the center position.The value of  for a CoP measurement placed on the center was 1; for a CoP measurement placed on the limit of stability  was 0; and for a CoP measurement placed at 3/4 of the distance to the limit of stability  was 0.25.An average of these values over the whole time-series was calculated to give the balance score.This is an intuitive measure that we will use and further discuss in following sections.The virtual environment used in [25] was a version of the "Oculus Tuscany Demo" developed by FenixFire [http://fenixfire.com/] and Oculus VR modified to rotate around pitch, roll, or yaw axes unpredictably by a sum-of-sines pseudorandom waveform.
It is important to reinforce that CoP represents the vertical projection of CoM on the ground when the gravity force acts exclusively, and this approximation is good for the purposes of this study.Nevertheless, during gait other forces beyond gravity are acting on the body.For this reason, Hof et al [26] suggested the use of a simple inverted pendulum model in order to measure dynamic stability.The position of the CoP added to its velocity times a factor (square root of leg's length/gravity acceleration) should be within the support base.Therefore, this model considers the height of the CoM and displacement speed forces, being theoretically more accurate.The authors named the projection of this point as the "extrapolated center of mass position".
In the research conducted by Robert et.al.[27], static and dynamic balance was measured recording CoP displacements using a force plate embedded in the floor with a sampling frequency of 1000 Hz.The maximum and the mean velocity of CoP displacements along the anteroposterior (A/P) and mediallateral (M/L) axes, as well as the total CoP path length was compared during the test subject´s interaction with the virtual environment.The virtual environment was a reproduction of the gait laboratory created through 3D filming, which allowed a full 360 ° photography and its recreation.In that experiment, the virtual and the physical environments were essentially indistinguishable according to the authors; but unfortunately, no images of the scenario were published.Results showed that in an environment with photo realistic condition there was no significant difference in the observed variables between the virtual and real testing environment.
Other related work presented experiments that were performed with the use of 3D Cave projectors and polarized lenses presenting the test subjects with visualvestibulosomatosensory conflicting information [28].Most recent work presented in [29] analyses postural stability in VR using the HTC Vive.Postural stability was measured through body sway using the tracking system provided by the HTC Vive.This was done while the user was immersed in a simple VR scene providing only a skybox and a visual fixation point for the user to look at.Our work's goal is different from the goals of the aforementioned studies.For that reason, a different methodology was conceived to answer questions that are related to such previous studies.This study allowed us to borrow interesting ideas such as the balance score proposed by [25] and using the mean velocity of CoP presented in [27] to measure the impact of the VR scenes on the user's balance.The next section covers the usage and implementation of previously cited algorithms in our experiments.

III. PROPOSED SOLUTION
To achieve our goals, this work was separated into phases.The first phase required recording the necessary data to calculate the user's balance state, detect falls and estimate fall risk.The second phase consists of the development of loss-ofbalance inducing VR experiences.In the final phase, we conducted a series of comparison experiments to see the effect of the different VR environments on the user's equilibrium.We also studied the effect of warnings alerting the user of fall risk during interaction with VR environments.The following subsections will describe each of these phases.

A. Balance state and CoM calculations
During the experiments positions and orientations of the user's limbs and torso was obtained using the MS Kinect SDK.To assess fall risk, we needed to calculate the projection of the user's CoM inside the user's support area.The distance from CoM's projection (CoP) to the center of the support area and its distance to the edge of the support area were used as a measure of fall risk.As proposed by [25] the formula:  = 1was used, where  is the distance from CoP measurement to the center (upright) position, and  is the distance from the subject's limit of stability (edge of the support area) to the center position (Figures 2 and 3).The calculation of the support area is a hard problem.The MS Kinect SDK provides the position of the ankle joint and the tip of the foot.The correct tracking of the tip of the foot was inconsistent from user to user.The position of the heel is not tracked by the SDK.Because of these limitations, we recurred to anthropometric data in order to estimate the user´s foot length.Anthropometric data presented in [30] established proportions between height and foot dimensions such as length and width (Figure 4).Using this estimation, the support area was calculated as the trapezoid defined by the tip and heel of each foot.Current estimations of the user's CoM were obtained using the kinematic method.This algorithm is presented in [15], and makes use of anthropometric data to estimate the position of CoM.This method is based on the definition of CoM.An accurate anthropometric model and full kinematics description of specific positions of several segments are required.In particular, the accuracy of CoM location is related to the validity of the mass inertia parameters (MIP) providing the CoM position and mass fraction of each segment of the model.In the present work, the anthropometric model was composed of 10 segments (2 feet/shin, 2 thighs, torso, head, 2 forearm/arm, 2 upper arms).The anthropometric data from [16] was used to estimate CoM location in each segment.The tracking provided by the MS Kinect SDK was used to locate the position of each segment as shown in the highlighted cubes, which represent the tracked joints (Figure 5).
In this algorithm, the CoM of the body is calculated as a weighted average of the center of mass of known particles.Each body part is represented by a particle located at its center of mass (Figure 6).The locations of these points are estimated based on anthropometric data to determine the center of mass of each body part as shown in Table 1.This table describes the location of CoM of body segments in percentage from proximal to distal, as well as the total body mass percentage of each limb.For example, the thigh's (Upper Leg) CoM location is 43.3% half way from the base at the hip (proximal) to the knee (distal).
Given knowledge of the size, location, and orientation of a certain body part, we can correctly estimate its CoM.As stated, this method is not 100% accurate as its accuracy depends on the accuracy of the estimated CoM for each segment.As shown in the comparison made in [8] this algorithm is sufficiently robust for our objectives and suffices the requirements of using relatively inexpensive and well available equipment, in this case, the MS Kinect Sensor.As future work, we intend to improve our current algorithm using a statically equivalent serial chain (SESC).As described in [19], it is possible to express CoM position of any chained structure as the end effector position of an open serial virtual chain.This virtual structure is known as a statically equivalent serial chain.The advantage of this method is that it incorporates a phase where the user is asked to hold a number of positions of which we know the location of CoM.This gathered data allows us to correctly calculate the locations of CoM for each segment of the user´s model.This method removes the error of estimating the segment´s CoM with anthropometric data.

B. Creating the fall risk warnings
In an effort to investigate if the negative effects of VR could be mitigated, we experimented with different types of feedbacks on the user's balance state.In our first approach, the warning consisted of sound and visual information.The visual information displayed the support area, the center of the support area and the location of the center of gravity within the support area as shown in Figure 7. Auditory information was provided through stereo headsets.The auditory information consisted of a 3D procedurally generated beep.The sound indicated the direction toward where the user should move in order to re-center its CoM.Initial tests showed these warnings were ineffective.The auditory warning was hard for the user to process and locate.Despite generating the sound with a 3D sound source in the VR environment, it was impossible to distinguish if the sound was in front or behind due to the stereo output headphones.The constant auditory cue was, in fact, confusing and uncomfortable.The visual feedback of the location was too complex and interfered with the user´s vision of the scene.After further research, we chose another visual warning.This warning was designed taking into consideration partial results of the tests.We observed that in the majority of cases users lost their balance due to unexpectedly stepping on the edge of wooden plates set as part of the scenario.This was caused by a number of problems: incorrect location of the user's point of view, the users misjudged the location of their feet due to incorrect camera location and lack of a more realistic shadow.The proposed warning consists of a red dye applied on the whole environment that varied according to the distance of the user's CoP to the edge of the support area.This helped the users in identifying when they could safely take the step.The tinting of the environment is achieved varying the intensity of a red fog.The objective of this warning was to examine if this kind of subtle non-intrusive warning could improve user balance.In low vision scenarios and in subjects with vision impairments it has been shown that subjects are more cautious while taking steps [31].Thus, this warning would not only trigger a response at a conscious level but could also trigger a response at a subconscious level.Fog density was varied using the position of CoP relative to the support area using the formula described previously:  = 1-.Fog color was interpolated from gray, balance coefficient value of 1, to red balance coefficient value of 0. The effect can be seen in Figures 8 and 9

C. VR experiences
The VR experiences were developed using Unity3D, a popular game engine that provides the tools for fast prototyping of the VR environments.The VR environment was presented to the users through the Oculus Rift VR headset, a lowpersistence, high-resolution (2160 x 1200) and low tracking delay headset.VR experiences tend to induce balance loss due to two main factors.Information obtained from the visual system relies on prior experience and most of the information the visual system processes and contributes to the balance system is based on immutable cues in the real world.However, in virtual reality, these cues are either inexistent or distorted.This can be caused by many factors in a virtual environment, such as incorrect lighting, misalignment of the floor in the VR environment with the floor in the real world, or incorrect orientation of the camera.The visual system can also produce information contradictory to the proprioceptive system.Delays in the body tracking system or incorrect positioning of the user's point of view may cause these contradictions.Current body tracking systems, like the Microsoft Kinect, provide near real-time representation of the user's body position, yet the combination of the delay and tracking imprecision are sufficient to provoke this undesired effect.Whenever visual information contradicts to the information obtained from the proprioceptive or the vestibular systems, the balance will be impaired [2].
The aforementioned flaws in VR environments were taken into consideration in the development phase and were minimized or solved whenever possible.We calibrated the tracking orientation of the Oculus Rift for every user, minimizing this problem.We also set the MS Kinect Sensor in a position where it was able to calculate the floor plane, and we corrected skeleton tracking so that the orientation of the virtual world was aligned with the real world.Users interact in first person view, and are represented in the VR environment with a simplistic stick figure avatar (Figure 10).Avatar movement replication presented very little lag as the movements of the avatar were refreshed 30 times per second.By design, this avatar could provide conflicting proprioceptive information.Although an accurate representation of the location of the user's position and posture temporal and spatial wise, it is not an accurate visual representation of the user's body.The selection of this nonrealistic avatar was intentional because, in most VR scenarios that make use of user avatars, avatars are pre-defined and selected from a fixed set, often with humanoid appearance but not a realistic human body rendering.
Two VR environments were developed, these aimed at representing different levels of danger.The first environment is a simple representation of the real testing scenario with four wooden plates on the floor (Figure 8).The other environment consisted of a set of wooden pillars in the same position as the wooden plates, high above rocks in the middle of an ocean (Figure 9).The wooden plates in the first scenario and the wooden pillars were placed within the virtual environment in positions matching wooden plates in the test area.Registration of these positions was estimated using the MS Kinect sensor as a reference and manually corrected before each round of tests.The next section explains the design of both scenarios and their purpose in our tests.

IV. TESTING METHODOLOGY
Two groups of twelve subjects with healthy vision and motor skills were asked to walk on a set of wooden plates laid on the floor in a specific order back and forth.The first group of subjects was recorded while doing this activity in three different scenarios, without the use of a VR headset and with the use of Oculus Rift VR headset in two different VR environments.A VR environment was similar to the testing area, this environment contained 3D models of the wooden plates on which the users walked on.The other VR environment had the wooden plates represented by tall wooden pillars over a dangerous rocky sea.The order of presentation of the different scenarios was varied; with three different types of scenarios six different orders are possible.They were randomly assigned to users so that the variation of the presentation order was evenly distributed among the users.This method assigned two users per variation for each group.The second group of users was presented to the same experiment while being provided with fall risk warnings throughout the tests.Recordings of the user's CoM and support area were made throughout the tests.The gathered data was denoised and compared in order to estimate the influence of the different scenarios on the subject's balance.A side by side comparison of the three different scenarios can be seen in Figure 10.

A. Subjects
Twenty-four adult subjects between 20 and 36 years of age were recruited for this study.Inclusion criteria were normal vision and normal gait.The users themselves informed theirs condition of normal vision and gait; we did not conduct any medical examination to verify the state of their vision or gait.
The tests were explained to the subjects and an informed consent document was read and approved.

B. Test
The test consisted of having the users traverse a set of wooden plates laid on the floor of the test area in a specific order while recording their CoM with the MS Kinect Sensor.The plates were laid in a configuration that made the users take two steps forward with their right foot and two steps backward with their left.Figure 11 shows the layout of the wooden plates and the order on which users took the steps.This test was conducted with three different degrees of interaction.The first type of interaction was without the use of a VR headset.This was intended to establish a baseline of each user´s state of equilibrium while traversing the set course without the interference of the VR system.
The second and third tests made use of the Oculus VR headsets for the representation of virtual environments.In these tests, the user could see himself/herself as an avatar, a stick figure, inside the virtual environment.The user´s avatar could then traverse the virtual environment while the user traversed the test area.The avatar´s movements were mapped to the user´s movement making use of the MS Kinect SDK and the MS Kinect Sensor.
The second test´s virtual environment was a minimalistic representation of the current test scene.It consisted of a gray floor and the wooden plates placed in the virtual environment in the same position as the real test area.This test intended to measure the user's balance state with minimal intervention of the virtual environment (Figure 12).The third test's virtual environment differed from the second only in its visual appearance.This third test was designed to maximize balance impairment through vertigo.In this environment, the path was represented by tall virtual trunks placed on top of rocks in an ocean, (Figure 13).This created an illusion of walking at high altitude in a dangerous situation.This test aimed at maximizing the impact of the virtual environment on the user´s balance.These scenarios will be referred to as Real (r), Office (o) and Sea (s).The order of presentation of the tests was randomly and uniformly distributed among users as shown in Table 2.This was done to minimize the influence of user adaptation in the results.To further explore the influence of VR a variation of the 2nd and 3rd tests was presented to half of the users.The variation showed the user a subtle visual warning.The visual warning consisted of a red tinting of the environment.The tinting varied depending on the balance state of the user.The balance state was calculated using the position of the CoP relative to the user's support area as previously described.

C. Data gathering and processing
The type of environment and the trajectory of CoM during tests were recorded as a time series.For each skeleton frame captured with the Kinect Sensor, CoM is calculated and recorded with its corresponding time stamp.Skeleton frames are calculated an average of 30 times per second.In the following, we describe a method for denoising the CoM paths as well as methods applied to extract information from the data.

1) Denoising:
The Ramer-Douglas-Peucker algorithm is an approximation algorithm used to simplify polygons and curves which produce a smaller number of vertices that lie on the given curve.This is done by recursively identifying key points and dismissing non-relevant points [9].
The algorithm expects points of a polygon or curve to be ordered in sequence.The algorithm first approximates the whole curve with a straight line from the first point to the last.Then a key point is defined as the point farthest from the line if the distance is greater than ε.This key point will split the set of ordered points in two sets, delimited by the first point, the key point and the last point.It improves the current approximation by recursively calling the algorithm on the two new created sets.Finally, all points not marked as key points are discarded.
With this method, we have a denoised 3D representation of the trajectory of CoM during the displacement within the virtual environments.Figure 14 shows a comparison of a denoised segment of a CoM path and the original CoM path.From this data, we can obtain useful information such as speed, path length, and duration of execution.We can also compare the shapes of the trajectories in the different scenarios without introducing errors in the results.2) Information extraction and shape comparison: To evaluate the influence of the different VR environments and warnings on the user's balance, for each user, denoised CoM path recorded in the experiment without the use of the VR headset was chosen as the reference path (RP).This type of path will be colored red when shown in images.With the reference path selected, it was necessary to measure the differences between the reference path and the paths recorded during VR interaction.These different paths will be referred to as office path (OP) and sea path (SP); they will be colored in yellow and blue respectively.To differentiate the paths obtained with fall risk warning and without fall risk warning, the letter W was added to the nomenclature of the paths with the warning: RPW, OPW and SPW. Figure 15 shows the three types of recorded paths side by side.As discussed previously, Chiarovano and colleagues [25] suggest that the maximum and the mean velocity of CoP displacements along the anteroposterior (A/P) and mediallateral (M/L) axes, as well as the total CoP path length, can be used as an indicative of the user's balance state.We will analyze the effect of the VR environments on the user's speed without separating the velocity along the body's axes.We wanted to also compare the spatial properties of the path.An appropriate method presented in the literature for shape comparison is the Turning Function Distance [32].The cumulative angle function, or turning function,  () of a polygon or polyline  gives the angle between the counterclockwise tangent and the -axis as a function of the arc length   () keeps track of the turning that takes place, increasing with left hand turns, and decreasing with right hand turns (Figure 16).A dissimilarity measure for two polygons A and B using the turning function: (, ) = ∫| () −  ()|, is used in [33].This calculates the area between the turning functions and the dotted lines shown in Figure 17.To apply this distance measurement, both polygons should have the same perimeter.We apply this distance measurement and compare the dissimilarity between the paths captured without the use of VR(RP) and each of the paths of the different scenarios tested.To compare effectively captured paths, we first discard the initial and final portions of the path with a clipping plane located at 10 cm from the starting line to ensure both start and end at the same position along the main axis of displacement.Then we scale them to a 1-meter length path to comply with the aforementioned requirement.Figure 18 shows recorded paths for one user, and their respective turning functions.

A. Statistical Analysis
A statistical analysis was performed to determine if there was a statistical difference between the data gathered for each of the conditions studied.We determined shape difference between SP and RP (SP-RP) and OP and RP (OP-RP) -Table 3.An independent-samples t-Test suggests that there is no significant difference between the calculated path difference values for each scene with a p-value of 0.18276.We also determined shape difference between SPW and RP (SPW-RP) and OPW and RP (OPW-RP) -Table 4. Results also showed no significant statistical difference with a p-value of 0.17529.
We then analyzed the average speed of the users in the different environments -Tables 5 and 6.A two-Way ANOVA test using average speed as the dependent variable and the scenario as the independent variable suggests that there is significant difference between the two scenarios, the office and sea environments with a p-value of 0.00032 and a 20% decrease in average speed in the sea environment, Figure 19 shows the results of the two-Way ANOVA test.It also suggests that there is a significant difference between experiments with a p-value of 0.01986 and a 16% reduction of speed in the experiment with fall risk warning.We conducted a t-Test of the average speed of SP, versus average speed of SPW as shown in Figure 20.The t-Test suggests, as expected, that there is a significant difference between the user's average speeds of the two scenarios.Furthermore, the mean of the user's average speed with the warning is 12% lower and has 71% less variance.

B. Discussion
In this section, we present our interpretation of the obtained results as well as our consideration of the results of the statistical analysis.
It is important to mention that neither positive nor negative results of the statistical analysis should be interpreted as conclusive due to the small size of the test samples.Further experimentation is needed to reach results that are more conclusive.The statistical analysis of path difference suggested there is no significant difference between OP and SP in relation to RP.Neither do OPW and SPW in relation to RPW.This result can be seen in Figure 21, as there is no evident visual difference between the plotted paths.We attribute this to the fact that the order of presentation of the tests influences greatly on the paths of the users.It is counter intuitive that users would do worse in the office environment than in the sea environment, as the path renderings may indicate.We could not identify the reason for this behavior, but this can be the result of various reasons: we hypothesize that users feel more comfortable in the office environment, completely trusting visual information, which leads them to make more mistakes and taking bad steps on the wooden plates.On the other hand, the sea environment triggers a more cautious approach during gait.This somehow corroborated by the significant differences in the user speed in these scenarios.
We present a more detailed comparison of the paths of users who were exposed to the office environment before the sea environment (Figures 22 to 24) and users who were exposed to the office environment after the sea environment (Figures 25 to 27).As previously explained, OP and OPW are represented in yellow, SP and SPW are represented in blue and the paths captured without the use of the VR headset, the reference paths are represented in red.In Figures 22 to 27, the first two paths are from users without warning, and the last two, from users with warning.
We believe OP and OPW visually differ more from RP than SP from SPW.Furthermore, we could argue that when the first scenario presented to the user was the office environment, the deviation from the reference path was even greater as seen in Figures 23 and 24.This observation is opposed to the users who's first presented environment was the sea, where CoP path in the sea environments did not deviate as much from the RP.
We assume this happened due to the fear induced by the risk of falling present in the sea environment.It appears evident that there are differences between the VR paths, yet it is necessary to work on the shape comparison algorithm.We will be aiming at improving this methodology and experimenting with other methods of balance quantification.
However, statistical analysis on user speed during VR interaction suggests there is significant difference between scenarios and between groups with lower average speed in the warning group and the sea environment.The office environment should make the user feel as if she/he were not using a VR headset, as opposed to the sea environment.The visual warning effect on speed is also in agreement with the literature, as discussed in [31], which showed that reduced visibility affects speed: "The low-vision group showed greater step width (p≤0.001) and slower gait speed (p≤0.004)"; a greater step width increases the support area which reinforces balance stability.Although we were unable to demonstrate statistical significant impact in our shape difference measurements, we could argue that virtual environments that induce a sense of danger tend to augment user awareness and induce a cautious behavior, which in turn could reduce fall risk.Red is a color we inherently perceive as one that is associated with danger and mistakes [34].This leads us to believe that the red tinting warning could be a viable alternative that needs to be further investigated and improved.In this work, we have elaborated and tested a methodology for the fall risk analysis of users during their interaction with VR environments.For this methodology, we implemented three main algorithms, which create a pipeline for analyzing and comparing user balance.We implemented a whole-body CoM and CoP estimation algorithm that is based on anthropomorphic data and makes use of a relatively low-cost and commercially available tracking sensor (MS Kinect Sensor).
Two different VR environments were designed and implemented, one with a high perception of danger and another representing the area where the tests were conducted.A measure of balance state was calculated based on the position of the user's CoP relative to the support area; this was used to implement a visual fall risk warning.An experiment was conducted with two groups of twelve users each.
We implemented the Ramer-Douglas-Peucker approximation algorithm and used it to denoise CoP path recordings.Finally, we implemented a shape comparison algorithm based on the cumulative turning function for comparing the different CoP paths recorded.
We recorded the users while walking on a set of wooden plates laid on the floor of the test area in a specific order back and forth without the use of a VR headset (Oculus Rift VR) while in the VR environments.
The abovementioned methodology is one of the contributions of this work.Another contribution is the issues raised by the conducted experiments.
The current amount of results does not allow us to reach a conclusive stance on the effectiveness of shape comparison algorithms on CoP paths to measure user's fall risk.However, they do suggest that the balance state value calculated by the formula proposed by [25] can be successfully used as a fall risk measure, evidenced by its impact on user speed.It also resulted in lower path shape difference in the sea environment compared to the office environment.Yet we were not able to prove statistical significance of this effect with the current amount of experimentation, but the experiment may shed a light on some issues regarding the user's fall risk during VR interaction.
We also observed that user behavior varied depending on the presentation order of the different scenarios.This was expressed in some cases in the user path; users who were presented first to the sea environment usually proceeded more carefully and most of the times, carried this behavior over to the office environment.In opposition, users who were first presented to the office environment proceeded to walk naturally without paying attention to the possible difference in perception between the virtual environment and the real scenario, often leading to miss steps and temporary loss of balance.We noticed that once the user had a miss step or an imbalance period, in general, they proceeded more cautiously and continued with this behavior through the remaining tests.For this reason, we hypothesize that the presentation of the tests has a major impact on user behavior, and this is an important issue to be considered in future works.

Fig. 2
Fig. 2 CoP and Support Area

Fig. 6
Fig. 6 Body segments CoM locations, whole body CoM and CoP locations.

Fig 7
Fig 7 Feedback of CoM's projection in the support area .

Fig. 8 Fig 9
Fig. 8 Warning tint effect on Office environment

Fig. 10
Fig. 10 Test scenarios, from left to right: testing area, office and sea virtual environments

Fig. 11
Fig. 11 Wooden plates layout and step order

Fig. 14 Original
Fig. 14 Original CoM path in red, denoised CoM in blue

Fig. 16
Fig. 16 Curve and turning function

Fig. 17
Fig. 17 Evaluation of dissimilarity with turning functions

Fig. 19
Fig. 19 Two-Way ANOVA on mean speed of OP and SP vs OPW and SPW

Fig. 20 t
Fig. 20 t-Test of average speed of SP vs SPW

Fig 21 RP
Fig 21 RP (red) vs SP (blue) vs OP (yellow) of all 24 users relative to wooden plates

Fig. 22
Fig. 22 No warning vs warning; order of experiment: r,o,s

Fig. 25
Fig. 25 No warning vs warning; order of experiment: r,s,o

TABLE 1 .
ANTHROPOMETRIC DATA USED TO ESTIMATE COM LOCATIONS OF DIFFERENT BODY SEGMENTS

TABLE 2 .
TEST PRESENTATION ORDER

TABLE 3 .
PATH DIFFERENCE OF FIRST GROUP OF USERS

TABLE 4 .
PATH DIFFERENCE OF SECOND GROUP OF USERS