Virtual Reality Group at Tecgraf / PUC-Rio

This paper describes the Virtual Reality Group of Tecgraf/PUC-Rio and some of its research and development activities in the areas of virtual and augmented reality, 3D user interaction and collaborative systems. The main projects of the group are related to the oil & gas industry, and include the development of real-time photorealist interaction and visualization tools for virtual environments, which support programs for the representation and inspection of engineering models as well as for the representation of natural models such as terrains and oil reservoirs. Keywordsvirtual reality,3D user interaction,augmented reality

Tecgraf's great commitment, and perhaps its major challenge, is to conduct its projects, fulfilling the schedule and the demand for a proper documentation, while priming for the scientific output and the academic formation of masters and doctors.Apart from producing human resources in-house, the group enhances its partners' technical qualification by presenting innovative solutions to their problems.As a consequence of its research activities, Tecgraf publishes a large number of papers and graduates several doctors, masters, and computer science and civil engineers.Through its projects, the group has developed over 60 products in the fields of Computational Mechanics, Naval Projects, Reservoirs and Geology, Geographic Information Systems and Environment, Program Development Tools, Geometric Modelling, Scientific Visualization, Virtual Reality and Computer-Assisted Projects and Supervision.

II. VIRTUAL REALITY GROUP
Virtual Reality has been rising as a strong tendency in Computer Graphics, being increasingly used operationally in the industry.Following this tendency, Tecgraf and Petrobras improved their cooperation in this area in 2001, when the VR group of Tecgraf started its activities.The VR group has worked jointly with Petrobras in the implementation, operation and support of VR rooms at Petrobras.The mission of the group is to find innovative solutions and to develop applications and tools for real-time photorealist visualization and interaction, keeping the focus on making them operational for the industry partner.
Currently this group is composed of about 30 people, among researchers, students and consultant professors.Its main projects are in cooperation with Petrobras, in the areas of geosciences and offshore engineering.There are also projects sponsored by CNPq (Brazilian National Research Council), FINEP (The Brazilian Innovation Agency), FAPERJ (the state agency of Rio de Janeiro for funding research) and RNP (National Education and Research Network).
Among the activities performed by the group, we can highlight the development of real-time photorealist interaction and visualization tools for virtual environments, which support programs for the representation and inspection of engineering models as well as for the representation of natural models such as terrains and oil reservoirs.The group also develops tools to implement applications in distributed environments, allowing users to manipulate the models remotely and to synchronize the visualization of distributed applications.
Another research activity of the group is the development and experimentation with new interaction environments, devices and techniques, such as multi-projected immersive environments, tracking devices, gesture and multi-touch recognition, among others.
The main current projects of the group are described in the following section.

III. MAIN PROJECTS
The VR group of Tecgraf / PUC-Rio is a multidisciplinary research team, and its ongoing projects intrinsically involves not only virtual reality, but some "adjacent" areas, such as computer graphics, human-computer interaction, design, collaborative work, among others.For the sake of simplicity, we are going to divide the projects in three main areas in this text: Virtual and Augmented Reality, 3D User Interaction (3DUI), and Collaborative Systems.

A. Virtual and Augmented Reality
The use of Computer Graphics and Virtual Reality has revolutionized several activities, since the possibility to interactively visualize and manipulate virtual models improves the comprehension and analysis of a large amount of information of spatial nature, exploring the human capability of visual communication and reasoning.
Particularly related to the oil & gas industry, earth sciences and engineering are challenged to manage and interpret increasing amounts of data coming from the captured information of the environment or generated by computer simulations.
Since VR's early development, it has been seen as a potential tool for engineering activities.Initially, VR application in conjunction with 3D geometric CAD (Computer Aided Design) models was restricted to design review, virtual prototyping, and marketing purposes.These models are now showing their potential in VR applications for such diverse purposes as ergonomic studies, safety training, and physicalsimulation visualization.However, a significant gap remains between CAD models and VR models owing to CAD and VR tools' differing purposes.CAD tools create detailed models, aimed at the execution process.VR tools support activities with a high visualization demand, to provide the best possible immersion in the physical setting by means of the virtual model.To bridge the gap, researchers have proposed several VR-CAD integration solutions.One such solution is the Environ application, one of our main projects (Figure 1) [1].Environ's main goal is to offer 3D visualization resources for CAD models with enough realism to serve as an integration tool for several engineering activities in the oil and gas industry.SiVIEP is another system being developed by the group, that involves not only the engineering scope, but also geosciences.SiVIEP supports a comprehensive visualization of several types of models comprising an oil exploration and production enterprise.For example, it is possible to load from oil platforms to wells and reservoirs in a single scene (Figure 2).One challenging characteristic of the virtual environments resulting from this integration is that they are multiscale, which are environments that can provide information in different levels of detail, from a single screw to an oil field spanning dozens of miles.Navigation in 3D environments still presents issues and is plenty of challenges.Multiscale environments, which are becoming more common, bring further problems to be solved in terms of navigation.We have been working on solutions to some of the problems identified regarding navigation in these environments [2].
Environ, SiVIEP, and other challenges brought by the industry constitute the basis for our research and development efforts in many related areas, such as: new rendering algorithms for massive models [3], integration with information systems [4], GPU programming [5] and frameworks for applications development [6].
Regarding Augmented Reality (AR), we are living exciting days, since it is increasingly being used in videogames and marketing campaigns.In parallel with these noticeable advances in entertainment and marketing, there are also less visible revolutions taking place in many other scenarios due to the convergence of Computer Vision, AR, Scientific Visualization, and VR.Roughly speaking, the progresses in Computer Vision are enhancing the performance of AR applications which, together with Scientific Visualization and VR, provide the possibility to improve the understanding and analysis of a large amount of information of spatial nature [7].
While these progresses demonstrate the evolution of this technology, they also indicate challenges posed for further research.Our research in this area includes the definition of a new camera calibration pattern [8], and applications in construction management [9] and cultural heritage [10].

B. 3DUI
Many of the tasks performed in a conventional desktop computer are related to text editing, file organization, use of tables, and mathematical calculations, among others.However, other areas of application emerged, e.g., 2D image editing, animation, CAD, interactive 3D visualization, entertainment and games.The inherent characteristics of these classes of applications have created a demand which was not fully met by the desktop metaphor as it was originally proposed to represent an office desktop through WIMP interfaces.Such a conventional setup does not meet all the interaction needs related to the additional dimension in this new environment.
The main evidence supporting this fact is the existence of a research line focused exclusively on 3DUIs (3D User Interfaces) which is as old as the very development of WIMP interfaces.The early efforts were guided by the evolution of scientific visualization and flight simulator applications driving the development of a new set of interactive hardware, such as head mounted displays (HMD), data gloves and CAVES.
The gradual convergence of interfaces from different natures (such as WIMP and 3DUIs) resulted in the emergence of other interfaces that share characteristics of both.The wide variety of hardware and software, and the different interaction techniques proposed are potential "ingredients" for a research field called Hybrid User Interfaces (HUI), where efforts are being joined in order to blend harmoniously both virtual (applications, interaction techniques, graphics, etc.) and physical (the input and output) elements from one or more interfaces.
In the field of HUI, the group developed a workspace called HybridDesk, featuring a mix of different devices to allow the use of three interactive environments for performing tasks with different hardware requirements (Figure 3).This workplace attempted to use semi-immersive visualization using low cost equipment instead of expensive commercial solutions.It was designed as a low-cost, easy-to-use system to support 3D annotation task in an oil platform visualization application.[11].Recently, interfaces based on single or multi-touch gestures have become quite common in everyday use, emerging as an attractive area for research.Their success has been due to their intuitiveness, the absence of intermediary tools and fast learning curve, composing an adequate environment for the general public.We have developed a 50 inch multi-touch table (Figure 4) based on Rear Diffuse Illumination setup and an infrared camera along with computer vision algorithms.
Based on the successful experience with multi-touch interfaces, it is supposed that this kind of interaction can also be applied to immersive virtual reality environments.We explored the possibilities of using a touch-like interface in an immersive virtual reality environment by means of the development of v-Glove [12].It is a wireless glove, which has two main functionalities: tracking the position of the user's index finger and vibrate the fingertip when it reaches an area mapped in the interaction space to simulate a touch feeling.Interaction with 3D virtual or hybrid environments, although rapidly growing, still lacks critical mass and is distant from standardization.There is still not enough knowledge about the best 3D interaction techniques.In this regard, the definition of adequate evaluation methods poses various challenges, particularly due to the diversity of interaction possibilities with VEs, including new interaction devices, like touch screens, wands, 3D glasses, head-mounted displays, etc.There is a research effort in the group to study evaluation methods for 3DUI [13,14].

C. Collaborative Systems
VR and visualization technologies enhance the content knowledge within any engineering design activity.When used in conjunction with collaboration, they provide valuable insights for better decision support with risk mitigation.The research challenge is to build systems adequate for highly heterogeneous scenarios, composed of not only geographically distributed teams, but also teams of specialists in different fields.This envisioned solution must provide scientific tools and technologies coupled with collaborative environments to support not only the modeling and simulation of complex scientific problems, but also the decision process, essential in the management of complex projects.
The research project we develop in this area is called Collaborative Engineering Environment (CEE) and integrates VR techniques into a collaborative environment where the execution of different sequences of engineering simulations is modeled as scientific workflows [15].The focus of this research is on the oil & gas industry, specially the Offshore Engineering domain, where the project of a new production unit is a lengthy and expensive process and usually is conducted by different specialists geographically distributed.Among the integrated engineering simulations it is possible to highlight the ones from structural calculus, hydrodynamics, naval engineering with mooring systems, meteo-oceanography, etc.Thus, CEE main objective is to improve users' interpretation capacity and skills while providing visualization tools for a better comprehension of the results of executed simulations.
CEE is composed of the following tools for distributed group work: i) a VR visualization tool for collaborative visualization of engineering simulations in an immersive or desktop environment; ii) a scientific workflow management system to control the execution of simulations; and iii) a videoconference system.Figure 5 shows an example of collaborative session using CEE for risers' simulation analysis.

IV. FINAL COMMENTS
We are living an interesting moment in the areas of VR, AR, and 3DUI.Technologies that have been restricted to research laboratories for decades are now mainstream and reaching ordinary people.3D cinema and TV seem to be finally mature, massively multiuser online games popularized VR on the web, Wii, Kinect, and similar technologies brought real 3D interaction experiences to the general public, and multi-touch interaction is becoming a standard in mobile devices.
While all these technologies provide great opportunities for innovative applications, they also indicate the challenges posed for further research.Interaction with VR and AR applications, although rapidly growing, still lacks critical mass and is distant from standardization.There is still not enough knowledge about the best 3D interaction techniques and how these techniques can improve many activities.This exciting context and the crescent demand from our industrial partners permeate the current research of the group, which intends to contribute to the development in these fields.
We are continuously looking for post-graduate students aiming to develop their research in this industry-driven scenario, in which the research challenges are brought to us by real-world problems.

Figure 1 .
Figure 1.Environ: (a) User stand in front of a cave composed by three view screens.(b)Virtual representation of the cave in (a) and user's position using multi-projection calculations.

Figure 2 .
Figure2.SiVIEP: visualization of an oil enterprise.In the first image, a complete oil field can be seen.In the second image, the camera reaches the scale of a platform and the interior of the platform can be navigated.

Figure 3 .
Figure 3. HybridDesk scheme with its three interactive environments and its use in the navigation mode.

Figure 5 .
Figure 5. Collaborative session using CEE for riser analysis.