Proceedings of the Symposium on Virtual and Augmented Reality (SVR)
Sociedade Brasileira de Computação - SBC
Adaptive Virtual Reality Solutions: A Literature Review
Virtual Reality technology emerges as a versatile tool with applications ranging from gaming to therapy. Particularly, Adaptive Virtual Reality (AVR) stands out as a promising field, employing adaptive mechanisms to automatically customize the user experience according to individual preferences and needs. The use cases for this technology are broad, and so is the technical landscape. AVR technology enables, for instance, educational platforms that adapt to the user’s learning style, upper limb rehabilitation with auto-adjustable difficulty, and horror games that detect and incorporate the user’s most feared elements. Implementation-wise, some studies utilize Neural Networks and Support Vector Machines to adapt the system’s logic, while others use Fuzzy Logic Systems, for example. Additionally, some works integrate physiological sensors to estimate user stress, cognitive workload, or cybersickness. This technological and practical diversity poses a challenge for researchers and developers seeking to build AVR-based applications. To assist these professionals, this work reviews 30 studies that propose, develop, and assess AVR-based solutions. It provides a technological overview of these systems and classifies them into four domain groups: healthcare, learning/training, entertainment, and cybersickness mitigation. This work also presents key considerations and guidelines for designing and assessing AVR-based systems.
Analysis of Cybersickness through Biosignals: an approach with Symbolic Machine Learning
Cybersickness represents one of the main obstacles to the use of Virtual Reality, often triggered by the use of Head-mounted Display devices. The symptoms associated with cybersickness can vary among individuals and include nausea, dizziness, eye strain, and headache, which may persist for minutes or even hours after exposure to Virtual Reality. According to the literature, cybersickness has a considerable impact on physiological signals such as delta waves in the Electroencephalogram; Heart Rate and Heart Rate Variability, derived from the Electrocardiogram; Electrodermal Activity; and Electrogastrography, all of which show a significant correlation with this condition. In this study, we investigated the use of biosignals to identify the possible causes associated with cybersickness in Virtual Reality. Our main hypothesis is that the combination of quantitative and subjective assessments, combined with Symbolic Machine Learning techniques, is effective in creating a ranking of the main causative/indicative factors of this condition. The results of this study highlight significant contributions to the understanding of factors influencing cybersickness symptoms. Statistical analyses confirmed the relationship between physiological changes and cybersickness symptoms. By including biosignals in our model, we achieved a significant gain, with an AUC of 0.95. The rankings of the main factors, both for the model without and with the inclusion of biosignals, confirmed previous research described in the literature. To the best of our knowledge, this is the first work to employ Symbolic Machine Learning models to detect the causes of cybersickness and generate a ranking of the most relevant factors.
Analysing Hybrid Neural and Ray Tracing Perception for Foveated Rendering
Foveated Rendering is a fundamental approach for optimizing latency and achieving high fidelity graphics in Virtual Reality. With the advent of real-time ray tracing, different global illumination effects are becoming feasible in interactive media, although they still require solutions for optimally displaying such effects. At the same time, neural rendering techniques, such as Neural Radiance Fields (NeRF) and Gaussian Splatting, are featuring realistic reconstruction of objects and scenes through real or pre-rendered images. Previous works introduced the possibility of hybrid rendering approaches, using full Ray Tracing techniques for the Fovea regions of a Head Mounted Display and Neural Rendering approaches, using baked and previously trained Networks for inferring realistic light effects in a more optimal and faster way for the peripheral region. In this work, we make a deep analysis of human perception related to five different lighting conditions, commonly achieved by Ray Tracing techniques when combining NeRF for approaching the peripheral region. To this end, we conducted a Double-Stimulus Impairment Scale (DSIS) test with 42 users, half of whom were tested on their perception of lighting effects with the reticule present in the image to fixate the user’s gaze, and half of whom were tested without the reticule. Results analysis enhances our understanding of visual perception in virtual reality and the limitations of NeRF on reproducing ray tracing effects for VR environments.
An Underwater Sound Propagation Model based on Ray Tracing for Virtual Reality
The emergence of new technologies for Virtual Reality has provided the user with a greater feeling of immersion. This immersion includes visual and sound perception, making realistic models for the light and sound behavior necessary. The Computer Graphics field is already well consolidated, but studies still need to be conducted on sound behavior in Virtual Reality. In this context, this work describes a new simplified underwater sound propagation model using the Unity game engine. The proposed approach allows the development of virtual environments with realistic underwater sound effects. Based on the ray-tracing algorithm, the implementation presents satisfactory results considering static sound sources. This paper describes the proposed model and a few test scenarios, and discusses the results obtained.
Enhancing Empathic Communication in Healthcare Education Through Virtual Conversations: Leveraging Large Language Models for Real-Time Feedback
Virtual conversations are increasingly utilized in healthcare education to enhance verbal empathic communication skills through tailored feedback on trainees’ responses. These conversations, supported by modalities such as speech, animation, and gestures, are highly customizable and accessible via the internet, bypassing the need for head-mounted displays (HMDs). However, training models to accurately evaluate empathic responses and generate human-like language with actionable suggestions remains a challenge. The advent of large language models (LLMs) provides new solutions to these challenges. This paper examines the impact of GPT-4-generated feedback on the empathic expressions of health professions trainees during virtual conversations. We enrolled 72 students from nursing and dental disciplines in an Interprofessional Collaborative Care course at the University of Florida. Participants were randomly assigned to one of two groups: one received feedback and suggestions from GPT-4 during conversations, while the other did not. We collected data on the perceived accuracy and helpfulness of the feedback from the intervention group. Using the Empathic Communication Coding System (ECCS) and GPT-4 Turbo, we assessed and compared the empathy levels of participants between the two groups. A Mann-Whitney U test was used to determine differences in average empathy levels. Results showed that participants receiving GPT-4 feedback had significantly higher median empathy levels than those without feedback. Feedback’s accuracy and utility were also affirmed by the participants. This study highlights the effectiveness of integrating LLMs like GPT-4 into virtual conversations for enhancing training outcomes in healthcare education.
Enhancing Virtual Human Interactions by Designing a Real-Time Dialog Filter for Mitigating Nonsensical Responses
Virtual Humans (VHs) are crucial in facilitating discussions on sensitive topics and training interpersonal interactions. However, conversational errors, like nonsensical responses, challenge VH simulation effectiveness. This paper explores real-time dialog filters to detect such undesired exchanges. We employ a five-step prompt design iteratively and leverage OpenAI’s GPT large language model to demonstrate feasibility. Our filter distinguishes meaningful from nonsensical responses generated by a rule-based system, achieving high F1 scores (0.84) and accuracy (0.78). Comparison with human-expert classifications validates its efficacy. Filtering nonsensical responses ensures coherent and relevant interactions, significantly enhancing efficacy. This study underscores how leveraging large language models can refine existing VH systems and improve virtual human dialogues.
Evaluating Usability, User Experience, and Playability of a Puzzle Game in Mixed Reality
This study investigates the usability, user experience (UX), and playability of a mixed reality (MR) game called Cubism, which allows participants to interact with holograms using hand gestures. We conducted an experiment with seven participants to evaluate their interaction with the game. The Usability and User Experience Evaluation in Touchable Hologram (UUXE-ToH) questionnaire was the primary data collection instrument to capture participants' perceptions of usability, satisfaction, and overall experience. Our findings indicate that the mixed reality environment of Cubism offers an engaging and intuitive user experience, though several challenges were identified. These insights contribute to the growing body of knowledge on human-computer interaction in mixed-reality contexts and suggest avenues for improving the design of future mixed-reality games. This research is valuable to the VR/AR/MR community by highlighting the critical factors influencing user interaction and satisfaction in MR games.
Exploring Mixed Reality in Digital Board Games: A Comparative Analysis of Cooperative and Competitive Modes
The integration of mixed reality (MR) into digital board games introduces new dimensions of player engagement and satisfaction. This study examines the impact of MR on user experience by comparing cooperative and competitive game modes. Using the games "Demeo" (cooperative) and "Demeo Battles" (competitive) as case studies, we conducted playtesting sessions and applied the GUESS-18 scale to measure various aspects of player satisfaction, including usability, narrative engagement, pleasure, and social connectivity. Our findings revealed significant differences in user experience between the two game modes. Cooperative play in "Demeo" was associated with higher levels of social connectivity, pleasure, and in-game engagement, indicating a preference for collaborative interactions and immersive experiences. By contrast, "Demeo Battles" excelled in personal gratification and visual aesthetics, highlighting the competitive mode's focus on individual performance and strategic depth. Additionally, we evaluated the impact of different VR headsets (Quest 2, Quest 3, and Quest Pro) on the mixed reality experience. Quest Pro provided the highest overall satisfaction due to its more ergonomic and comfortable design, which enhanced comfort and facilitated more immersive mixed reality experiences. These insights offer valuable guidance for future game design, emphasizing the importance of tailoring MR experiences to player preferences, in both cooperative and competitive contexts.
Expanding the VR for the Social Good Initiative to Brazilian Higher Education
This paper explores the expansion of the “VR for the Social Good” (VR4SG) initiative from the University of Florida to the Brazilian higher education context. In Brazil, the general public predominantly views Virtual Reality (VR) as just an entertainment tool, yet we intend to highlight its potential for addressing local Social Good issues. Therefore, this paper proposes a structured adaptation of the VR4SG course tailored to contextual needs, resources, implementation requirements, and desired social changes within Brazilian communities. This proposal is informed by our analysis of various versions of the course over the years, utilizing the Service Blueprint tool to outline a clear and effective strategy for implementation. Overall, the proposed expansion of the VR4SG course in the Brazilian Higher Education scenario aims to empower students from different disciplines into proactive, real problem solvers using VR technology. With this approach, we expect to inspire and qualify Brazilian students to go beyond consuming VR media to create immersive narratives capable of shifting perspectives and fostering empathy. By promoting teamwork and effective communication, the course can not only enhance personal skills but also contribute to social change, aligning with the United Nations’ Sustainable Development Goals.
Eyes of Fear: Leveraging Emotion Recognition for Virtual Reality Experience
Given the growing interest in immersive gaming experiences, particularly within the horror genre, there is a rising demand for sophisticated and engaging game mechanics. However, creating a truly immersive horror experience requires a balance between fear-inducing elements and player engagement. In this work, we present Eyes of Fear, a horror game that explores human emotions and eye movement as interactive mechanisms in virtual reality. We used a state-of-the-art emotion recognition algorithm and Tobii Eye Tracker for our game prototype and evaluated their effectiveness through player feedback. Our findings show that emotion-sensitive enemy behavior elevates unpredictability and intense emotions while amusing players.
Facial Conformation Modeling by Anthropometric Measurements
This paper presents a method for modeling facial conformation using anthropometric measurements. The technique is designed to ensure seamless surface continuity and scalability for incorporating new measurements. The method allows for the manipulation of both global and local facial characteristics, provided the anthropometric points are present. It organizes the anthropometric measurements into a three-level hierarchy with global, regional, and local facial features. Then, deformations are applied to a base facial mesh representation to adjust it to the landmarks and achieve the desired facial measures. In this work, the results focus on modifications of the mouth region of a base model, showcasing the method’s ability to alter features such as mouth width and lip thickness. The paper discusses the current limitations of the proposed technique and points to some improvements in future works.
Generalization of Real-Time Motion Control with DRL Using Conditional Rewards and Symmetry Constraints
Deep Reinforcement Learning has been increasingly explored as a method for generating physics-based motions in articulated characters. However, effective control tools are still necessary to better guide the learning process and provide animators with greater control and reliability over the resulting animations. This paper proposes new control tools, including the generalization of real-time control, conditional rewards, symmetry constraints, and a user interface. Real-time control allows dynamic adjustment of chosen parameters, conditional rewards simplify the competition between rewards, symmetry constraints reduce uncoordinated movements, and the user interface facilitates training and animation parameter specification. The proposed control tools show promise in improving the quality and control of physics-based character animation.
Inspection and Training using Virtual Reality Applied a new Wheel Loader Model
This study introduces a new wheel loader model designed to enhance the inspection and training process for the handlers at a mining industry. This model incorporates advanced features, providing a comprehensive and realistic training experience. A key component is a virtual reality simulator, developed using a cross-platform game engine. This simulation serves as an interactive tool for performing maintenance inspection procedures, addressing a critical need for effective pre-operation safety instruction. The software was tested by 16 qualified professionals from the industry, across different conditions. The results demonstrated that the simulator significantly improves the initial training experience. Therefore, it allows employees to gain essential skills and knowledge without exposure to real-world risks. In addition, the new wheel loader model incorporates more real-life immersion as the interior of the cabin. It is constituted of the buttons and proper texturing, enabling operators to identify and address potential issues. Thus, further minimizing operation disruptions. Overall, the integration of this advanced wheel loader model with a sophisticated virtual reality apprenticeship not only enhances safety but also reduces machine downtime. Moreover, the integration allows reduced training costs and it provides a highly effective solution. Consequently, it ensures that operators are well-prepared for a wide range of situations that they may come across in the real world.
Interactive 2D vs. 3D Loading Screens in VR: Impact on User Experience and Perceived Wait Time
Loading screens in games and VR tend to be frustrating, so optimizing them to maintain user engagement is crucial. Previous research suggests that interactions can significantly reduce perceived wait time. However, the complexity of 3D interactions may impose additional computational overhead. To address this, we investigate the possibility of using interactive loading screens with 2D elements, aiming to balance user engagement and performance. Through a user study, with 21 participants, and performance benchmarks, we evaluate the effectiveness of this technique in enhancing UX and reducing perceived wait times. Our findings indicate that interactive 2D screens, while less computationally intensive, can offer comparable user engagement benefits to fully interactive 3D screens, providing a viable alternative for optimizing VR loading experiences.
Introduction to Gauss’s Law in Electromagnetism through a Virtual Reality Application
Virtual reality (VR) has emerged as a transformative technology in recent years, while Gauss’s law of electromagnetism remains a fundamental principle within Maxwell’s equations. Gauss’s law relates the net electric flux through a closed surface to the charge enclosed within that surface and is particularly useful in scenarios with high symmetry in charge or field configurations. This paper explores the application of Gauss’s law through VR technology to enhance educational outcomes. By providing immersive and interactive experiences, VR enables students to visualize electric fields and flux in three dimensions, interact with closed surfaces, and experiment with dynamic charge configurations. This approach not only strengthens theoretical understanding but also promotes collaborative learning and engagement. Integrating Gauss’s law with VR technology helps bridge the gap between abstract electromagnetic concepts and practical comprehension, making advanced studies in electromagnetism more accessible and engaging. This work focuses on the development of the computational framework, with initial qualitative results from one-semester undergraduate students presented. Quantitative analysis of the impact will be addressed in future research.
Machine Learning Applied to Locomotion in Virtual Reality
The objective of this research project is to recognize a user's walking pattern on a treadmill-like platform for navigation in Virtual Reality (VR) environments. To achieve this, a walking recognition software solution was developed using Convolutional Neural Networks (CNNs), a type of Machine Learning (ML) architecture. The ML model was trained on a dataset collected from users' movements in a virtual simulation, tracking the positions and rotations of their feet as they walked in various directions and orientations. Tracking was accomplished using 6 degrees of freedom (6DoF) trackers placed on the users' feet. The neural network achieved a 94% accuracy rate during testing. Due to the network's lightweight configuration, the response time is, on average, 0.1 seconds, demonstrating its potential as an efficient natural controller for real-time user navigation in multiple directions.
Methods for Evaluating Immersive 3D Virtual Environments: a Systematic Literature Review
Although evaluation methods for Virtual Reality systems have been proposed in many studies, most of them are developed for particular applications with no possibility of generalization. Additionally, few studies proposed compilation and categorization of these studies, which turn difficulty obtaining an overview of the state-of-art. This systematic literature review aims to analyze and categorize evaluation methods for interaction in immersive three-dimensional virtual environments. By reviewing studies published from 2019 to April 2024, diverse approaches and methodologies used to assess Virtual Reality interaction techniques were identified. The studies were categorized into five main areas: locomotion, input, visualization and selection, evaluation of haptic interfaces, and transitions and scenarios. Our findings highlight that most studies employed established evaluation methods, with only one study proposing new method. Objective method, particularly time-related measures, were frequently used, alongside subjective method obtained through questionnaires and interviews. The review emphasizes the importance of combining subjective and objective method to provide a comprehensive analysis of user experience and interaction effectiveness. A significant need for standardized evaluation frameworks to enable consistent comparisons across studies was identified.
ReBase: a framework for acquiring and managing body tracking data
There is a great deal of interest in developing virtual reality and augmented reality applications that require data storage, with which developers can interact and create intelligent systems for various purposes, such as applications for neuromotor rehabilitation. These applications provide the benefits of traditional rehabilitation without becoming overwhelming, as they allow the user to be immersed in various scenarios and situations. In addition, the way these applications interact makes it possible to retrieve and analyze the data generated by users’ movements during treatment. However, it is tough to find datasets of body movements, and those that do exist do not include movements specific to the rehabilitation context. This paper, therefore, presents ReBase, a framework that provides a standardized storage system for body movement data. ReBase was designed to allow the collection of data generated by any application, regardless of the form of body tracking used. This paper presents an open API that can be used to communicate with the ReBase database, a package for the Unity game engine, and a library for the Python programming language, which facilitates the integration of new applications into the system. In addition, two applications are shown as case studies within and outside the health area, developed using the tool.
Shared Reconstructed Environments in Extended Reality
Modern extended reality headsets can identify the geometry of the physical surrounding space, which can be shared and synchronized with remote users, allowing them to view the space through their own extended reality headsets from different points of view. Conversely, remote users can augment this shared space with additional virtual geometry. For instance, imagine wearing an augmented reality (AR) headset that identifies and shares your surrounding space with a remotely located interior designer. This interior designer can then virtually teleport into your environment and suggest furniture placements, which you can see in real-time through your AR glasses. Moreover, you can interact with each other via avatars representing each user. This research project demonstrates how two people in different locations can experience one’s real physical environment in real-time and interact with each other. The strategies used to share geometry and user poses are discussed, along with the limitations of the hardware used for the prototype. The developed prototype involves one user wearing a Magic Leap 1 headset to scan their space, enabling a second user to remotely explore this space using an HTC Vive virtual reality (VR) headset. The VR user can also add virtual objects that both users can see.
João Guilherme Cintra de Freitas Almeida
Simulation Sickness in Virtual Reality Games, How to Relieve it - Systematic Literature Study
Immersive technologies have introduced innovative possibilities across various fields, including entertainment, sports, education, healthcare, training, and rehabilitation. Despite the increasing popularity and feasibility of these technologies, owing to their ability to create a heightened sense of presence, a significant challenge remains in the widespread adoption of virtual reality: simulation sickness, also known as cybersickness. This phenomenon manifests as symptoms such as nausea, dizziness, and fatigue, arising from factors like sensory discrepancies between real and virtual movements, latency, graphic quality, and individual susceptibility. This article aims to provide a comprehensive review of the current literature on cybersickness, exploring its causes and identifying methods to mitigate its adverse effects through advancements in hardware, software, and usability settings, ultimately striving for a more comfortable and positive virtual experience.
The challenge of stress induction in serious games, considering gameplay
Stress induction is crucial for training personnel in high-pressure situations, such as military, first responders, and healthcare professionals. The ability to perform effectively under stress is a critical skill for these professionals, as their decisions can have significant consequences. One of the most effective methods for inducing stress in training scenarios is through the use of virtual environments. Virtual Reality (VR) and Augmented Reality (AR) technologies have become increasingly sophisticated, allowing for the creation of highly realistic and immersive training environments. These technologies can simulate a wide range of scenarios, from battlefield conditions for military personnel to emergency response situations for first responders and complex surgical procedures for healthcare professionals. This type of training aims to prepare individuals to respond effectively in real-life scenarios, enhancing resilience and decision-making abilities under stress. Measuring stress levels during these trainings is essential to assess program effectiveness and adjust methodologies as needed. By monitoring physiological and psychological stress indicators, such as heart rate variability, cortisol levels, and self-reported stress scales, trainers can determine if the scenarios are inducing the desired level of stress. This measurement ensures that participants are genuinely engaged in the training, allowing for real-time adjustments to the intensity and nature of the scenarios to maximize the training benefits. Additionally, it helps in verifying that the participant is fully immersed and responding realistically to the simulated high-pressure environment, ensuring the training’s relevance and effectiveness. Therefore, through a Systematic Literature Review, this study investigated the best biomarkers and their combinations and protocols used, aiming to determine the most effective way to measure stress induction in training to improve their efficiency.
Using learning environments for military study
One of the biggest concerns in planning a military operation is how to explore the terrain, avoid collateral damage, and accomplish the mission. During that process, military planners study the topographic and tactical aspects, gathering detailed information to better advise superiors in their decisions. Learning how to prepare for that terrain study takes time and practice, and inside the classroom is difficult. Because of that, sometimes students are taken into regions in a Field Exercise (FE) to understand how to conduct this kind of activity. Even participating in those FEs, it is quite difficult for the instructor to explain to students the subject given the abstraction needed to visualize, for example, the coordination measures for an operation in the area, the company moving to an objective, and so on. Another problem happens when the instructor points to a place and makes observations about that area, while the student accidentally looks at the wrong place and starts learning the content in the wrong way. Trying to solve similar problems, Wenqiong Du et al. developed a Mixed Reality-based platform addressing nontechnical skills personnel for Battlefield First Aid (BFA) training with satisfactory effectiveness. In this work, the interaction between the real and the virtual world was fulfilled by using the HTC VIVE VR Headset. In this context, we propose “Simulador Virtual para Estudo Topotático do Terreno (SVETT),” a simulator that uses resources usually found in classrooms involving 3 interfaces (VR, projection, and constructive) that can help instructors ease teaching the abstract subjects of a terrain study and standardize the knowledge learned from students. The Brazilian Marines Simulation Center developed SVETT and evaluated it with 54 students. The results suggest that using SVETT first, students found participating in the field exercise easier because of their previous contact with the region.
Virtual Reality Bank: A Novel Banking Experience in Immersive World
As Virtual Reality (VR) technology continues to advance and wide-spread through various industries, the potential of immersive technologies in the banking sector has also begun to be acknowledged more commonly. VR banking, a 3D bank environment developed in an immersive world to provide customers with bank services, has the potential to substantially enhance the customer experience. However, a clear insight into an end-to-end pipeline for developing a banking system in VR is yet to be presented. This paper presents a novel VR banking system that facilitates users with personalized full-body avatars for representation within a VR bank, a speech-to-text integration for providing user information for bank services, and real-time dual avatar interaction and communication with a bank employee. Through a subjective survey, we evaluate the performance of the proposed facilitates and the overall VR bank application in comparison to mobile and physical banking. Our study reveals that integrated features have successfully achieved their expected level of performance and contributed to creating a convenient, inventive, personalized, and accessible VR bank application.
What if HMDs could generate audio descriptions of real scenes for visually impaired people?
This paper explores the feasibility and effectiveness of using the Meta Quest 3 head-mounted display to generate real-time audio descriptions of real scenes for visually impaired individuals. The objective is to enhance the spatial awareness and interaction capabilities of visually impaired users by providing continuous, detailed audio descriptions of their surroundings. The methodology involves leveraging the Meta Quest 3 to capture real-time visual data, which is then processed and converted into audio descriptions using AI-driven software. Experiments were conducted in various real-world environments to test the accuracy and usability of the audio description system. Key findings indicate that the Meta Quest 3 can effectively deliver real-time audio descriptions with high accuracy, significantly improving the navigation and situational awareness of visually impaired users. This study demonstrates the potential of HMD technology to create more inclusive and accessible experiences, highlighting the importance of continued innovation in assistive technologies for the visually impaired​.
What if Video See-Through in HMDs Changes How Accurately We Perform Tasks?
This paper explores the impact of video see-through head-mounted display devices on user perception and accuracy during activity execution. As augmented reality technologies evolve from cumbersome systems to advanced, self-contained units like HMDs, they offer enhanced freedom and immersive interactions with digitally augmented environments. Despite significant strides in this field, there is a comparative scarcity of research evaluating the practical effectiveness of these technologies in real-world applications. This study focuses on VST systems, which have gained substantial commercial traction over optical see-through systems due to their cost-effectiveness and ease of integration. This research assesses the precision and accuracy of these devices through a two-stage experiment involving dart throwing and bottle handling tasks to determine their applicability in both precise and everyday activities. Using a Meta Quest 3 as the target HMD, the experiment evaluates performance with and without the device, aiming to identify any significant differences in task execution. This paper aims to fill the gap in literature by providing empirical evidence on the feasibility of VST HMDs in enhancing user interaction within a mixed reality context, thereby contributing to the broader application and democratization of VST technology.
What if... we Knew the Effects and Benefits of Virtual Reality in Childhood?
This article provides a systematic review of Head-Mounted Display Virtual Reality devices intended for children. The primary objective of this study is to investigate inquiries concerning the safe utilization of HMD devices for children, potential associated risks, and the recommended age group for usage. By examining the impacts, advantages, and obstacles associated with the exploration of this emergent technology, this research strives to amalgamate findings on the repercussions of HMDs on children’s physical, cognitive, learning, and safety well-being. It offers pertinent insights for the safe administration of this technology to children across various age groups. The review encompassed 59 research articles conducted in various countries, which highlighted both the positive effects, such as the reduction of anxiety and improvement of social skills, and the negative effects, such as physical discomfort and fatigue, associated with prolonged HMD use. Finally, the analysis included a comparison of different device models and exposure times, offering a comprehensive overview of best practices and recommendations for future studies.
XiloVR: Preserving Cultural Heritage through a Virtual Reality-Based Woodcutting Simulation
Woodcut (in Portuguese, Xilogravura) holds significant historical and cultural importance in the context of Northeast, playing a crucial role in preserving the region’s artistic heritage. Meanwhile, Virtual Reality (VR) emerges as a powerful technology capable of creating immersive environments that simulate real-world activities, offering new possibilities for education, training, and information dissemination. This paper presents the development of an interactive Virtual Reality (VR) environment that explores the educational potential of this technology to promote woodcut as a cultural heritage of Pernambuco. The proposal aims to provide a practical and engaging experience in art creation, innovating teaching methodologies and ensuring the preservation and transmission of knowledge about woodcut to future generations. The project’s implementation is based on the work of master artisan Edilson Oliveira, a young craftsman from Pernambuco, and simulates all stages of artistic creation, integrating contributions from experts to create a highly realistic experience. This approach seeks to preserve and disseminate knowledge about the technique for future generations.
3D Visualization of Molecules from Structural Formulas: An Interactive Approach
This work presents the development of a system whose main objective is to convert two-dimensional (2D) representations of chemical molecules into interactive three-dimensional (3D) models. The aim is to enhance the understanding of molecular structures through a more intuitive and dynamic visualization. The system allows users to either capture a photo of a hand-drawn 2D molecule or draw a molecule directly on the screen of a mobile device. The input image is processed to generate a 3D model that can be manipulated and examined from different angles, facilitating the analysis of molecular geometries. This approach not only bridges the gap between traditional 2D chemical diagrams and modern 3D visualization but also provides a valuable educational tool for chemistry students and professionals. Our ongoing work aims to further improve the user interface and enhance the accuracy of 3D rendering to support a broader range of molecular structures.
Attention Guidance through Video Script: A Case Study of Object Focusing on 360º VR Video Tours
Within the expansive domain of virtual reality (VR), 360º VR videos immerse viewers in a spherical environment, allowing them to explore and interact with the virtual world from all angles. While this video representation offers unparalleled levels of immersion, it often lacks effective methods to guide viewers’ attention toward specific elements within the virtual environment. This paper combines the models Grounding Dino and Segment Anything (SAM) to guide attention by object focusing based on video scripts. As a case study, this work conducts the experiments on a 360º video tour on the University of Reading. The experiment results show that video scripts can improve the user experience in 360º VR Videos Tour by helping in the task of directing the user’s attention.
Augmented Reality Support for Oncological Neurosurgery
This project aims to enhance oncological neurosurgery by displaying brain tumors using Augmented Reality (AR) as an overlay image during surgical procedures. Its purpose is to assist neurosurgeons in tasks such as brain biopsies by providing a clear and precise visualization of the affected area, thus enabling faster and less invasive surgeries, ultimately offering a more comfortable experience for patients. Currently, medical procedures achieve this through neuronavigation systems, which display two-dimensional images of the patient’s brain on a screen, typically positioned beside the patient or in separate imaging support rooms. By utilizing AR, surgeons can directly visualize the tumor on the patient’s body through a virtual 3D model, eliminating the need to frequently refer to external images. This project was developed using Microsoft HoloLens, a specialized AR headset, and Unity, a game engine widely used for 3D environment development. The system uses DICOM files as input, which are converted into 3D objects displayed through the AR headset.
Construction of Interfaces in Virtual Reality: a systematic review focusing on Ergonomics and User Experience
Virtual reality (VR) technology has generated significant global interest, leading to ongoing advancements. However, the development of VR interfaces, particularly concerning ergonomics and user experience (UX), remains underexplored. This paper addresses this gap by conducting a systematic literature review to establish guidelines for optimizing VR interfaces. By analyzing existing studies, we aim to develop guidelines that enhance UX and ergonomic considerations in VR, thereby advancing the effectiveness and accessibility of this technology.
Developing an University Immersive Campus: Guidelines for Beginner Developers
Immersive campuses are becoming an increasingly prevalent trend in how universities interact with their academic community and technologies like the metaverse emerge as promising resources to enhance the educational experience. The metaverse can provide more engaging and interactive learning experiences, allowing students to explore information in a practical and immersive way. This study provides guidelines for beginner developers creating immersive campuses for exploring a university. Using the design science research approach, we developed a three-stage simulation with progressive challenges and interactions in the Unity game engine and Meta Quest 2 device. Then an exploratory group evaluated the potential user acceptance and usage with the UTAUT2 model and we finally carried out a focus group. The results indicated that the immersive Campus was perceived as easy and enjoyable to use, though recommendations were not a major influence for most users. This discrepancy may have led many participants to disagree that influential individuals believed they should use the tools. Despite this, participants expressed an intention to utilize RV authoring tools in the future, albeit without a commitment to regular usage. We compiled seven guidelines, including prioritizing seated immersive experiences, enhancing visual accessibility, optimizing code post-update, organizing interface components, using progressive loading for responsiveness, and optimizing texture loading functions to prevent performance issues, among others.
Enhancing Robustness in Audio Deepfake Detection for VR Applications using data augmentation and Mixup
The rapid advancement of virtual reality (VR) technology has heightened the need for robust and reliable deepfake audio detection to ensure the authenticity and integrity of virtual interactions. Al-though current state-of-the-art models exhibit promising results, they are often overconfident, which can lead to poor generalization and reduced effectiveness against novel or slightly altered deepfake attacks. In this work, we investigate the application of data augmentation techniques and Mixup techniques to increase the diversity of training data and improve the generalization of deepfake audio detection models. Mixup creates new training examples by combining pairs of existing examples, promoting smoother and more robust decision boundaries, while data augmentation creates new training examples altering a sample with a given probability. Our results demonstrate that applying such techniques to the Wav2vec 2.0 model significantly improves its generalization ability, leading to more reliable deepfake detection in VR environments
Isadora Stéfany Rezende Remigio Mesquita
Evaluation of Navigation in Immersive Large-Scale VR Environments Using Minimap-assisted Teleportation
In virtual reality (VR), large-scale applications present numerous challenges in navigation that must be addressed. Providing users with the freedom to move anywhere at any time is crucial for delivering an optimal immersive experience, particularly in large-scale environments. This paper evaluates the use of 2D minimaps for orientation and locomotion, with the goal of enhancing user navigation efficiency and precision. We discuss various methods for representing terrain information within a 2D panel, incorporating both real-world data and in-game elements to recreate 3D terrains accurately. An empirical user study involving 24 participants evaluated three teleportation techniques: the World in Miniature (WIM), the WIM in a 2D panel, and the 2D minimap. Metrics such as completion time, teleportation steps, and user workload indicated that 2D minimaps significantly improve navigation efficiency. Considering the time of completion, the 2D representation had a difference of 23.12% compared with WIM implementations. Additionally, considering the number of teleportation steps taken, the usage of a minimap proved more precise with a considerable difference of 138.65% compared to the WIM technique, leading to reduced task completion times and teleportation steps.
Fine-Tuning a Video Masked Autoencoder to Develop an Augmented Reality Application for Brazilian Sign Language Interpretation
Approximately 5% of the Brazilian population experiences some level of hearing loss. Over the years, various methods have been proposed to facilitate communication between individuals with and without hearing disabilities. Typically, these solutions focus on developing techniques or mechanisms that help people with disabilities understand communication from those without disabilities. Most methods rely on neural networks or similar architectures, often neglecting the large vision models developed in recent years. In this paper, we propose a high-level design for a head-mounted device (HMD) that can translate Brazilian Sign Language into spoken or written words. As an initial result, we used the MINDS-Libras video dataset for Brazilian Sign Language and fine-tuned a Vision Transformer architecture known as VideoMAE, achieving promising results. The accuracy reached up to 84% with only 50 videos in the training subset for each word.
Gamification and Tourism: the experience of built cultural heritage mediated by digital technology
The way the world consumes culture has changed. Museums, cultural centers, and historical monuments have been transforming to adapt to these contemporary trends. Digital technology has been one of the allies in expanding the user experience in the cultural sphere, whether in virtual mode or places of visitation. The means of interacting with the user have taken on playful and participatory approaches to increase attractiveness and facilitate engagement, with gamification strategies standing out. This research aimed to understand how a playful experience, through a digital interface, can contribute to the leisure tourism experience while enhancing the historical and cultural heritage. To this end, a review of the literature was conducted to understand the characteristics and use of gamification in tourism and the development of a practical experiment, a gamified application for tourist visits to Matriz Nossa Senhora do Pilar church, a 18th-century Baroque church in the city of São João del-Rei, Minas Gerais.
Gesture Recognition in Smartwatches Using LSTM for Interaction in Low-Cost Virtual Environments
This work presents a methodology to enhance interaction in VR environments using accessible devices such as Google Cardboard and smartwatches. The main contribution is the implementation of a recurrent neural network model of the LSTM type, designed to recognize gestures captured by smartwatches. This enables more natural and fluid interaction with the virtual environment, significantly elevating the level of immersion and responsiveness perceived by users. Preliminary results demonstrate that the LSTM model achieves robust performance in accurately identifying gestures, which is essential for providing an immersive and engaging experience. We hope that this approach expands the possibilities for interaction in virtual environments and represents a significant advancement in the field of wearable computing applied to Virtual Reality.
Improving VR Accessibility Through Automatic 360 Scene Description Using Multimodal Large Language Models
Advancements in Virtual Reality (VR) technology hold immense promise for enriching immersive experiences. Despite the advancements in VR technology, there remains a significant gap in addressing accessibility concerns, particularly in automatically providing descriptive information for VR scenes. This paper combines the potential of leveraging Multimodal Large Language Models (MLLMs) to automatically generate text descriptions for 360 VR scenes according to Speech-to-Text (STT) prompts. As a case study, we conduct experiments on educational settings in VR museums, improving dynamic experiences across various contexts. Despite minor challenges in adapting MLLMs to VR Scenes, the experiments demonstrate that they can generate descriptions with high quality. Our findings provide insights for enhancing VR experiences and ensuring accessibility to individuals with disabilities or diverse needs.
Interactive Educational Game for Chemistry in Virtual Reality with User Gesture Interactions via Smartwatches
This article explores a ludic approach to disseminating knowledge through a prototype of an interactive mobile virtual reality (mVR) game centered on chemistry education. Targeted at high-school students and chemistry undergraduates, with varying levels of prior knowledge, the game utilizes immersive experiences to encourage and facilitate learning. It discusses various interaction methods suitable for exploration-based mVR games, highlighting the integration of external devices like smartwatches. These devices enable dynamic ways of interaction via the continuous gesture recognition algorithm, enhancing user engagement and interaction within the virtual environment.
Simulating a Virtual Reality-based Electrical Substation: A Pedagogical Proposal
Virtual reality is a promising educational tool for electrical engineering because it can replicate complex environments. Immersive experiences can enhance knowledge retention and allow students to safely practice operation procedures and protocols, and develop their skills without the risks of handling high-voltage equipment. This work provides guidelines for developing a virtual reality-based simulation of an electrical substation for education. Using the design science research approach, we developed a three-stage educational simulation with progressive challenges and interactions in the Unity game engine and Meta Quest 2 device, and then an exploratory group evaluated the potential user acceptance and usage with the UTAUT2 model. Results highlighted that the simulation was intuitive and impacted positively on the learning process for the users while also being fun and engaging. Based on these encouraging outcomes, future classroom testing sessions will be conducted to further assess the impact of the simulation on the learning process.
Situated Visualization in Public Spaces: an approach for civic media strategies
Situated visualizations integrate technologies to create data visualization experiences, allowing people to obtain information beyond the computer screen by connecting information to the real world. They can function as decision support systems by facilitating understanding of a phenomenon. In addition, situated visualizations in public spaces promote the democratization of data and information performing as civic media, which are inclusive and collaborative in mobilizing citizens about urban problems. This article presents a practical experiment: an interactive installation that presents data on pedestrian accidents in the Região Metropolitana de Vitória. The installation presents an information panel, a data selector panel, an interactive totem, and a carpet/map.The results show that users had a good experience with the installation, obtaining information interactively and entertainingly. Furthermore, this installation can display data on other urban problems, thus democratizing information. The experiment shows that situated visualization can promote civic action, making citizens aware of urban problems, stimulating reflection, and, potentially, the search for improvements in urban communities.
Technique for Identification of Electrical Substation Equipment Through Auto-Framing Interest Points and OCR Recognition of Text Tags in Environment for Augmented Reality Systems
Electrical power substation systems are considered critical environments with a high impact factor on society. Although Augmented Reality (AR) solutions are becoming increasingly prevalent in the future of Industry 4.0, there is a concern about the practicality of using these systems. AR has significant potential to support assisted maintenance of substation components by projecting field asset information onto an AR headset. To enhance this process, a technique was proposed to identify equipment by automatically reading text from its tags using OCR (generally manual) and using auto-framing through object detection (with Neural Networks). The developed solution can be tested and evaluated in a laboratory setting. The conditions evaluated when pointing a camera at the image of the equipment's operation box with his text identification tag showed that the method employed by the technique can achieve relatively better results than manual framing, making the equipment identification process efficient and potentially promising for implementation in AR devices.
VREnergy: A simulator for critical operating procedures in electrical substations with evaluation process
This article presents a VR simulator developed with Unity 3D for critical electrical maintenance procedures in a factory setting. The simulator offers two modes: tutorial, for guided step-by-step instruction, and free mode, for independent skill testing. Upon completion, operators receive detailed performance reports with scores based on their procedural errors. The simulator has proven effective in reducing factory accidents and increasing operator engagement in training sessions.
What if Surgeons Could Practice Anywhere?
Surgical students must practice procedures in a controlled and safe to make mistakes environment before they perform these procedures on patients. Despite that, surgical practices in these conditions are traditionally limited by the tutor's time availability to provide feedback and by the usage of consumable materials to simulate surgery in a patient. In this context, we developed the "Hand Guide", a virtual reality application that allows expert surgeons to record their hand movements during a simulated surgical procedure and offer it to a surgical student to try to replicate it. The application provides real-time feedback and also a final score at the end of the recorded procedure. Validation with a specialist revealed that the application may be used to teach theoretical content to students with no surgical background and to demonstrate new techniques to experienced surgeons.
What if Virtual Reality could be used as a Tool in Positive Psychotherapy in Brazil?
Virtual Reality allows the user to experience a sense of presence and immersion in a three-dimensional virtual environment. Such sense is provided by tracking head movement, that naturally changes the virtual images and motion. VR can be used as a tool in clinical psychology, being its incorporation by psychologists advantageous. Thus, the effectiveness of psychotherapy integrated with VR for the treatment of psychological disorders has been shown in some studies. However, to ensure the appropriate use of VR, psychologists must have a clear understanding of the opportunities and challenges this tool may provide in professional practice. This article describes the first steps in incorporating VR into psychotherapy procedures in Brazil. In particular, the paper focuses on the validation of the developed virtual environments by psychologists, the introduction of narratives, and the prospects to implement VR in psychotherapy to improve the well-being of patients.
What if… we could virtually restore and visit a pre-war church to the highest fidelity possible with no 3D scanning available?
The digital study of the historical and sacred parts of the church of Thillois aimed to enhance understanding and preservation of this significant heritage site, severely impacted by the First World War. By employing digital tools and technologies, this study facilitated the creation of a comprehensive 3D model of the church, incorporating detailed analyses of historical archives and the restored building. This effort not only reconstructed the physical attributes of the church but also addressed the sociocultural impact of the war on the local community. Key stages of the study included detailed computational modeling, photogrammetry, and the integration of VR to provide an immersive experience for users. The study highlighted the challenges and limitations of digital restoration, such as the passive nature of video animations and the physical distance between the observer and the digital model. Nevertheless, the VR experience allowed users to interact with the model, enhancing accessibility and engagement with the heritage site. The research underscored the importance of consistent and regular documentation to support future studies and enhance collective memory. Digital methods and tools complemented traditional documentation, offering new ways to access, reformulate, and understand historical information. The studys findings have significant implications for similar heritage sites in the Reims region and beyond, demonstrating the social and cultural potentials of digital tech- nologies, fanning the flames of collective memory as a source of social and civic cohesion, and virtually and more intensely restoring architectural spaces lost or altered by historical events.