LApIS : developing research applied to the health care area

The Laboratory of Computer Applications for Health Care is a Brazilian lab investigating virtual reality techniques, content-based image retrieval, and image processing areas, mainly developing applications to the health care area, although many of the techniques and tools extrapolate this application scope. A brief history of the laboratory, its mission as well as current projects and collaborations are presented. Current developed projects are mainly related to biopsy simulation by using virtual reality, content-based image retrieval (CBIR), testing of software with graphical outputs, efficient data structures to represent modeled objects, information visualization, methodologies to assess virtual environments and virtual atlas with direct interaction.


I. INTRODUCTION
The LApIS (acronym in Portuguese for Health Informatics Applications Laboratory -Laboratório de Aplicações de Informática em Saúde) is a laboratory located at the School of Arts, Sciences and Humanities of the University of São Paulo.Its main goal is the conception and development of low-cost techniques and tools for the -but not limited to -health care area.It was established with funding from the CNPq (research funding agency in Brazil) with an initial project to build a prototype of a simulator for breast biopsy.Among its early research, tools for dynamic building of three-dimensional (3D) structural of fetuses by using measurements extracted from two-dimensional ultrasound, visualization of information from medical image databases, and a virtual atlas of breast cancer were developed.
The LApIS mission is developing computational techniques and low-cost tools directed to contribute with research in the virtual reality, image processing and content-based image retrieval (CBIR) areas.In general, the developed methods and techniques have the health area as their application field; however, results obtained can be applied to several knowledge areas.

II. RESEARCH LINES
The main research lines of the LApIS are virtual medical training, CBIR, information visualization and evaluation of virtual tools.
In the medical training line, initially a prototype of a tool for simulating the breast biopsy exam was developed, with characteristics of non-immersive virtual reality considering a conventional platform [1].The need for improving this prototype led to projects on accurate collision detection, stereoscopy and deformation, which are currently joined in a framework that creates tools for training biopsy exams in general.Also in this area, virtual atlases for anatomy studies were created, and currently efforts are focused on developing natural interaction for these virtual atlases.
In the CBIR context, there are projects developed to provide retrieval of images from different nature, such as medical images and human faces.Nowadays, the research of this subject is directed towards the retrieval of threedimensional models based on content (model retrieval) and the development of techniques to provide adequate visualization of the results obtained from CBIR systems.The information visualization line aims at supporting the understanding of large volumes of abstract data.Initial works in this area were developed using virtual reality to compose scenes and to allow interaction with the medical and business areas.Research was also conducted considering different types of metadata from several Database Management Systems.The current direction of this line lies in defining visualization metaphors in an automated manner and evaluating the effectiveness of scenes from three-dimensional interactive visualization environments.
Finally, the line of virtual tools evaluation considers both technical aspects of the tools and the user.Along this line, software testing techniques for evaluating tools are developed, works aiming at measuring knowledge and skill acquisition in 3D virtual environments.Also serious games have been investigating as a way to evaluate the user.

III. MAIN ONGOING PROJECTS
In the context of the research lines mentioned in the previous section, the main ongoing projects of the LApIS are presented as follows.

A. ViMeT Framework
The ViMeT (Virtual Medical Training) Framework is an open source Virtual Reality object-oriented framework to generate tools for simulating biopsy exams [2].Biopsy simulation procedures present common tasks, independent of the object under study and the medical instrument used.Therefore, the construction of a framework can provide time saving functionalities in applications development.
In the current version, ViMeT provides functionalities and characteristics to create applications for simulating biopsy exams, such as stereoscopy; modeling of 3D objects to represent the medical instrument, the user's hand and the human organ; collision detection between objects; deformation of the object that represents the human organ at the moment of collision and support to non-conventional devices, such as dataglove and haptic equipment.Two examples of applications generated by using ViMeT are presented in Figure 1.ViMeT Framework is available at http://www.each.usp.br/lapis/.Some current works related to this project are: realism in relation to visualization, methods for providing precise collision detection in real time, realistic deformation of complex objects, use of serious games to evaluate students (Figure 2) and definition of methodologies for evaluating knowledge and abilities acquisition by users when trained in three-dimensional virtual environments.

B. Computational Geometry
In order to provide realism in simulations generated by ViMeT Framework, one must find a tradeoff between the detail level of the graphic objects represented by the environment (essentially human organs and medical instruments) and the algorithms performance.In the ViMeT, the objects are represented by a triangulated geometrical mesh.The geometry is represented by the three-dimensional coordinates of the vertices and the topology by the mesh connectivity.In this version, all objects are two-dimensional meshes, although they are immersed in a three-dimensional space.A two-dimensional triangulation can be defined as a set T of triangulated faces such that: 1. Any two faces of T are either disjoint, or share a face of smaller dimension (edge or vertex); 2. The set of faces of T is connected; 3. The union of all faces of T has no singularities.
For each triangle of T, an orientation can be defined which induces the same orientation in its sub-faces.An orientation is consistent if faces that share the same edge have opposite orientations.
It is worth to mentioning that the greater the number of triangles used to represent an object, the higher the resolution and therefore, the realism.At the same time, due to the high computational cost of handling a great number of elements, the simulation can become unfeasible.
It is necessary to find and implement efficient representations to the objects used in the ViMeT Framework.Two topological data structures are being implemented in order to improve the efficiency of the environment.One of these data structures is the CHE -Compact Half-Edge [4] which is an evolution of the classical data structure Half-Edge and can have several levels of representations.The second one is the Mate-Face data structure [5] which is also an evolution of the corner-table data structure.The Mate-Face is more intuitive and can be extensible to three-dimensional objects.The two data structures are being tested and both should be incorporated in the environment.Figure 3 shows two examples of meshes represented with CHE and Mate-Face, respectively.
As the simulations in the ViMeT Framework are becoming more sophisticated, some new possibilities of investigation appear: 1. the addition of several two-dimensional meshes each representing a different layer of a human organ, skin, muscle, bones and other structures of the human body; 2. the use of volumetric meshes instead of using twodimensional meshes to represent the different layers of the objects.This approach is more expensive, but tends to provide a better realism in the simulation; 3. another possible approach is a totally meshless approach.
Here, the objects would be represented by "particles" that store its specification.This option is not being tried in ViMeT, so far.

C. Content-based Image Retrieval
The studies on CBIR address mainly the problem of querying, in a database, images which are similar to a given reference image.CBIR is defined as any technology that helps to organize digital image files based on their visual content [6].In general, CBIR systems consist of computer programs that aim at finding in a database those images most similar to a model image, in accordance with one or more provided criteria.
The criteria for similarity are obtained by extracting features of the image, such as color, texture and shape.The automated CBIR systems involve several areas of computer science, mainly image processing and databases.In a simple way, a CBIR system is essentially composed of four parts: extractors, similarity functions, indexing structures and sorting algorithms.
After developing projects to retrieve images in an mammographic database and human face database considering relevance feedback (Figure 4) and also some work to evaluate the behavior of similarity functions, the current projects are directed to defining extractor and tools to allow model retrieval (CBIR in the three-dimensional context), visualization of CBIR systems results and using CBIR concepts to evaluate computeraided diagnosis systems.

D. VIDA Project
The VIDA (Virtual and Interactive Distance-learning on Anatomy) Project aims at developing learning in virtual environments to teach anatomy by using virtual atlases.The atlases can be accessed remotely, via Internet, with conventional equipment (such as mouse and keyboard) as well as by the use of natural interfaces, i.e., use of the human body for interaction (Figure 5).
Currently, researches related to this project are directed to studying ways to get more natural interactions, by using gestures, voice, game controllers (such as Kinect) and others, besides building tools to allow teachers to mount their own virtual atlas, without the need of a programmer.

E. Support for the test of medical applications
Developing medical applications is not an easy task.It demands quality and specific development techniques.In this direction, two projects are being developed aiming to support the test of such applications.
The first is the development of a framework to create testing oracles for graphical programs, such as medical image processing.A testing oracle is a method used to verify whether the system under test behaves correctly for a particular execution.The role of an oracle can be played by another program -in the case of an automated oracle -or a human being, based on a specification, who decides whether the behavior obtained is that expected.
In order to try the concepts of CBIR in the construction of graphical oracles, a framework named O-FIm (Oracle For Images) was implemented, following the open source paradigm (http://ccsl.icmc.usp.br/pt-br/content/O-FIm-oracleimages).
As a result of using this framework, the tester gets a Java program that is able to compare two images (usually stored in files) responding whether they are similar or not, according to the chosen characteristics.Also to facilitate the creation of an oracle, the O-FIm framework has a wizard tool that generates the textual description of the oracle.This is a graphical interface that allows the tester's intervention to create a description as the one presented in Figure 6.In the figure, the tester creates an oracle which can decide whether a segmentation algorithm, applied to mammographic images behaves as expected.
The second project in this subject uses scene graphs to define testing criteria to test virtual reality applications.Scene Graphs (SGs) are structures which represent a virtual environment and in a certain way specify its behavior.As such, SGs can be used to derive requirements that must be satisfied during the test activity.
For instance, exercising each node or each transformation present in the SG is a minimal requirement one should expect to meet during testing.Thus, a series of testing criteria, along with a support tool to monitor its coverage have been developed in this project.The ultimate goal is to apply them to other products produced in the group.

Figure 1 .
Figure 1.Examples of applications generated by using ViMeT Framework.

Figure 2 .
Figure 2. Example of a serious game for student evaluation [3].

Figure 3 .
Figure 3. a) Mesh with CHE representation b) Mesh in Mate-Face Representation