Applied Multimedia and Virtual Environments Research at the ACiMA ( aka ABoVE ) Laboratory

This manuscript briefly introduces several prototypes developed at the ACiMA Laboratory in the last few years. It aims at introducing the lab to anyone interested in joining us or other research labs that may have interest in establishing collaboration in the areas listed below.

Later that year the control software was developed.After that, a number of prototypes has been developed using it.We have acquired a comprehensive set of equipment, which will be listed throughout this manuscript.We have just started the process of acquiring a CAE LaparoscopyVR simulator [11].Such system will allow us to enhance considerably the realism and usefulness of surgical simulations on that specialty.Although this manuscript seems to indicate that we are heavily biased towards medical applications, the laboratory also carries out research in other areas such as oil and gas industry [8], bioinformatics [2] and defense [10], amongst others.

II. PROTOTYPES
We describe here results achieved in the last few years.

A. MiniVR -Portable VR System
This prototype aims at allowing users which would not otherwise have access to 3D content, to take advantage of interactive 3D applications [1].Considering the way it was designed, most applications developed to work in the CAVE system at LNCC will also work in the MiniVR system.The MiniVR consists of an aluminum box with approximately 30 cm on each side.It contains everything that is needed to project 3D content on a screen.The idea is that just a power cable connection is required to have all working.
Internally the MiniVR has a processing unit, two compact projectors, polarizing filters as well as all internal connections and cooling solution.Figure 1 shows the MiniVR system, including a quick glance on its internal components.
MiniVR allows VR applications to be taken where the users are, rather than requiring the users to move to a installation which would provide 3D interaction.The unit weights 8Kg (17.5 pounds).The image resolution is of 1024x768 pixels.The MiniVR works best in an environment which is not too bright, in which case the image can be enlarged considerably.As the environment gets brighter the scene needs to be smaller (projectors closer to the screen) so that the content is bright enough to be properly seen.
We use passive stereoscopy, more precisely circular polarizing filters and matching glasses.The two projectors project images addressed to either the right or left eye.The polarization allows each eye to see only its own image, which provides the brain with two separated views of the 3D scene, which is automatically used to position the objects in the 3D space.Figure 2 shows a 3D scene being generated by MiniVR.The scene needs to be projected in a surface which keeps the polarization of the light.We use a piece of Stewart Silver Screen (Figure 2 left), but any surface that maintains the polarization is useable.
The MiniVR system uses X3D based on InstantReality, which is the same setup used in the CAVE at LNCC.For interaction, we often use a wireless keyboard and mouse, a wireless joystick Logitech Rumblepad.When a more precise control is called for we use a (wired) 3D mouse (3D Connexxion Space Navigator).The setup has been demonstrated to the public in two opportunities: a) FAPERJ's Science, Technology and Innovation Fair (Rio de Janeiro, RJ) -FAPERJ is the State of Rio de Janeiro Research Support Agency and b) SBPC ExpoT&C Conference and Exposition (Goiânia, GO) -SBPC is the Brazilian Society for the Advancement of Science.We are designing an yet smaller unit, with built-in power, which is to increase the portability even further.

B. Automatic Evaluation of the Quality of Cardiopulmonary Resuscitation Procedure
The training of Cardiopulmonary Resuscitation (CPR) is often performed in physical manikins which mimic the behavior of the human body.One example of such models is the Laerdal Little Anne (Figure 3).The manikin allows one to practice the CPR procedure, getting used to the pressure to be applied.One may manually calculate the frequency used in the procedure, but the system itself does not do it.We have added to this training setup some hardware and software that automatically measures three parameters: a) The pressure the user makes, with a historic in a timeline graph; b) The frequency in the procedure, with a historic in a timeline graph and c) The angle at which the pressure is being made.One shall remember that the proper procedure uses pressure which is normal to the surface, i.e. it is performed with the pressure down, without inclination.There is also a historic timeline graph which documents this data.
The system has been implemented using a Nintendo Wii Fit Plus scale, which is placed under the manikin.The Wii Fit scale is connected to a computer through Bluetooth.As the pressure is performed in the manikin, the weight is distributed in the corners of the scale.If the pressure is properly applied the weight is uniformly distributed.If it is performed at an angle the corners of the scale will indicate it.The system shows the information in an easy to check interface (Figure 4).The interface is composed of three sections.The section to the left has a dial which shows the pressure applied at that instant.The green level displays the acceptable values.The section to the right has a dial which shows the frequency at which the massage is being applied, with the values in the green range showing the ideal frequency.Finally, the section in the center has two concentric circles.If the massage is made properly, the mark stays within the smaller circle, whereas if the massage is being made at a wrong angle, the mark will move to the external circle and eventually outside of it (proportional to the angle applied).
At the lower part of all sections we have a graph that shows how the user has done throughout the procedure.By looking at the graphs shown, one can quickly see how efficient the procedure was performed, if it kept performance consistency or if it faded as time went by (due to tiredness). Figure 5 shows the system being used in the FAPERJ Science, Technology and Innovation fair.This prototype has been shown to many individuals who actually perform CPR in their daily tasks.The results are very positive so far.We expect to add to the system a way to record the data for later evaluation of one's performance.

C. Virtual Grasping in the CAVE
This prototype aimed at allowing a user to grasp a virtual object with his/her own hand to move it around in the CAVE.The virtual world consists of a 3D surgery room (see Figure 6 left) in which we have a set of items on a table.Using a CyberGlove Wireless II (Figure 6 right) the user can reach out and grasp an object.The system detects the collision of the user's hand with the virtual object and allows the user to push the virtual object, as well as to grasp it and move it around.The system also makes use of a magnetic position tracker (Ascension Nest of Birds).One sensor is attached to the user hand so that the system knows where the hand is located as well as the movements performed (through the data glove).The CyberGlove allows the system to know the configuration of the user's hand, which includes the grasping and release movements.All motions performed by the user are mimicked by a virtual hand model which is within the virtual world.This system enables fine interaction between a real user and the virtual world.Figure 7 shows a hand that represents the user's hand approaching a virtual model of some objects (left) as well as the user grasping a bottle (right) and moving it.The bottle moves along with the user hand as it would in real life.

D. Low-Cost CAVE
We have developed a low-cost CAVE at the LNCC.We use two setups with it: a) a cluster of 4 computers (each one controlling one of the three walls and floors) and b) a single computer with a nVidia QuadroPlex 2100 D4 which provides 8 video outputs.We use passive circular polarizing stereoscopic projection, with two off-the-shelf NEC LT245 DLP projectors for each wall.The CAVE structure is built out of PVC pipes.The hardware is controlled by InstantReality plus some customs code and the performance has been quite acceptable.Figure 8 shows the setup, Figure 9 shows the CAVE in use.

E. AVIDHa -3D Haptic Anatomy Atlas
The application, called AVIDHa is a 3D human body atlas for the purpose of anatomy study [6].AVIDHa allows students to interactively explore several human body systems through the senses of touch and stereoscopic vision.The human body systems are available as high definition 3D models with photorealistic textures, as shown in Figure 10.
The application allows the anatomy student to fly through and move inside of the human body.The flight and exploration modes are done with either 3D mouse or a joystick.The student may also choose to investigate each system separately, change organ's opacity to visualize internal parts, capture screenshots for later examination, manipulate clipping planes to explore the inner parts of a given system, or even use a haptic device, such as the Sensable's Phantom Omni, to feel the organ's density and contours.AVIDHa may also run as a distributed collaborative application, allowing users geographically apart to one another to interact through the network.In this case a mediator may drive the simulation and share his/her knowledge with other participants.The system also works in the CAVE.AVIDHa was developed using a rendering engine developed at the ACiMA Lab.Such engine, called EnCIMA (An Engine for Collaborative and Immersive Multimedia Applications), will not be discussed here, but can be found in [3,4,5].

A. Head Mounted Display Based Medical Application
This prototype aims at allowing a user to view a 3D scene through a Head Mounted Display (HMD) and interact with such world.Figure 10 shows a user wearing the Sensics xSight 3120 panoramic HMD.We can notice that the user's vision is blocked by the HMD.The user only sees what is displayed in the HMD screens.In this application the user sees a surgery room, with a patient lying in the surgery table.Figure 11 shows the interface the user sees.There is a number of displays with information controlled by the application.A menu allows a user to set transparency in the patient model, so that internal structures can be seen (Figure 12).The user can then move a needle around and apply it to the patient.A message in one of the virtual world screen shows the target reached by the needle.This is an early stage of a prototype which shows some functionality that we expect to use in a virtual surgery application.

B. NanoVR
As mentioned quickly in the MiniVR section above, we are designing an yet more portable solution.We are experimenting with smaller projectors which, if found fit for stereoscopic projection, would allow us to build a truly portable setup, controlled by two cell phones connected to cell-phone-sized projectors.Since all components work on batteries, that shall allow one to deploy 3d content anywhere, without the need for a power connection.We are experimenting with various polarizing filters, as the small projectors have low brightness, which is further reduced by the filters.

C. CyberForce/CyberGrasp/CyberGlove System
We have recently received a CyberForce/CyberGrasp/ CyberGlove system.Such system is to be merged with the HMD setup shown above to allow tactile-rich simulations.It allows full control and force feedback to the user hand.

D. HMD Head Tracking
The HMD shown above requires the user's head to be tracked, so that movements are compensated by adjusting the images displayed.We expect to use Nintendo Wii's Wiimote and/or Microsoft Kinect to track the HMDs motion.
INTRODUCTION The Applied Multimedia and Virtual Environments Laboratory (ABoVE, aka ACiMA) started activities in May 2003.It concentrates the research efforts on Collaborative Virtual Environments, Virtual Reality and Multimedia Systems at the National Laboratory for Scientific Computing (LNCC), a research institute that belongs to the Brazilian Ministry of Science, Technology and Innovation.Throughout the years many prototypes were developed [1-10] as a result of the research carried out at the Lab.Here we describe briefly a few of such results.The ACiMA Laboratory keeps strong collaboration with both DISCOVER Laboratory (Distributed & Collaborative Virtual Environment Research Laboratory) and MCRLab (Multimedia Communication Research Laboratory) at the University of Ottawa (Canada).In 2007 we started to design and assemble a low cost CAVE, which was finished in 2008.

Figure 12 .
Figure 12.User interface seen within the HMD.

Figure 13 .
Figure 13.Setting transparency in the model.Internal structures in display.