Advanced Compiler and Runtime Support for Data Intensive Applications - Ferreira, Renato; Agrawal, Gagan; Saltz, Joel (Anais do International Symposium on Computer Architecture and High Performance Computing (SBAC-PAD), 2000)
Processing and analyzing large volumes of data play an increasingly important role in many domains of scientific research. However, high-level language and compiler support for developing such applications have been so far lacking. We are developing compiler and runtime techniques necessary to generate efficient implementations of such applications from a high-level programming language. Through advanced compiler analysis, our system is able to extract necessary information from the source. This information is used to generate an executable that makes extensive use of a runtime system called Active Data Repository (ADR). In this paper, we present advanced techniques thar can enhance the performance of compiler generated data intensive codes. We show how characterization of access patterns into dense and sparse and choosing appropriate execution strategy for each leads to better performance. We also show how demand-driven output space allocation on each processor, managed through a sparse data-structure, can reduce the number of tiles that need to be allocated on each processor and enhances performance.