Improving In-memory Column-Store Database Predicate Evaluation Performance on Multi-core Systems

• Hong Min IBM Thomas J. Watson Research Center
• Hubertus Franke IBM Thomas J. Watson Research Center

Abstract

The ability to analyze a large volume of data for the purpose of business intelligence has led to various innovations in database technology. One example is the increased interest of using column-oriented data layout to address query performance in analytical and warehousing workloads. As system architectures move towards multi-core designs, it is important to address optimizing performance for these workloads on these platforms. In this paper we present SPHINX, an architecture that utilizes multi-core systems for search-based predicate evaluation operations in analytical query workloads against in-memory column store. We discuss the natural parallelism of predicate evaluations and various bottlenecks that impact search performance. We present several performance improvement techniques and apply a scan sharing technique based on cache reuse efficiency to further improve the performance. We demonstrate the performance benefits of our scan sharing scheduler over other scheduling approaches in a workload of mixed search queries.

Keywords: Prefetching, Layout, Databases, Filtering, Throughput, Bandwidth, multi-core, in-memory database, column store, scan sharing