Accelerating GNN Inference via Automated Parallel Execution on Edge Heterogeneous Platforms - Lin, Yi-Chien; Fan, Haoyang; Gobriel, Sameh; Jain, Nilesh; Prasanna, Viktor K. (Anais do International Symposium on Computer Architecture and High Performance Computing (SBAC-PAD), 2025)
Recently, Graph Neural Networks (GNN) have been integrated into various local applications, such as local community detection and local code assistant, making edge inference increasingly important. To support diverse workloads, state-of-the-art edge devices have evolved into heterogeneous platforms, integrating components like CPU, GPU, and NPU. To this end, we propose GNX, a novel GNN system that accelerates GNN inference on edge heterogeneous platforms by leveraging all the heterogeneous processing units. Given a GNN model and a heterogeneous platform, GNX automatically generates parallel execution plans, consisting of both data and pipeline parallelism. To reduce the complexity of the design space, GNX converts GNN models into coarse-grained blocks and performs the search at the block level. By leveraging the APIs provided by state-of-the-art heterogeneous frameworks, GNX can flexibly schedule various parallel execution plans and seamlessly adjust the workload across the heterogeneous processing units for load-balanced execution. Our study shows that GNX effectively accelerates three widely-used GNN models on two state-of-the-art edge heterogeneous platforms. Compared with the baseline approach that uses only a single processing unit, GNX achieves up to a 2.57× speedup. Compared with adopting data parallelism and a state-of-the-art scheduler, GNX achieves up to 1.90× and 1.79× speedup, respectively. We also discuss the applicability of and extensions to GNX to support other GNN models.