Accelerating GNN Inference via Automated Parallel Execution on Edge Heterogeneous Platforms

• Yi-Chien Lin USC
• Haoyang Fan USC
• Sameh Gobriel Intel Labs
• Nilesh Jain Intel Labs
• Viktor K. Prasanna USC

Resumo

Recently, Graph Neural Networks (GNN) have been integrated into various local applications, such as local community detection and local code assistant, making edge inference increasingly important. To support diverse workloads, state-of-the-art edge devices have evolved into heterogeneous platforms, integrating components like CPU, GPU, and NPU. To this end, we propose GNX, a novel GNN system that accelerates GNN inference on edge heterogeneous platforms by leveraging all the heterogeneous processing units. Given a GNN model and a heterogeneous platform, GNX automatically generates parallel execution plans, consisting of both data and pipeline parallelism. To reduce the complexity of the design space, GNX converts GNN models into coarse-grained blocks and performs the search at the block level. By leveraging the APIs provided by state-of-the-art heterogeneous frameworks, GNX can flexibly schedule various parallel execution plans and seamlessly adjust the workload across the heterogeneous processing units for load-balanced execution. Our study shows that GNX effectively accelerates three widely-used GNN models on two state-of-the-art edge heterogeneous platforms. Compared with the baseline approach that uses only a single processing unit, GNX achieves up to a 2.57× speedup. Compared with adopting data parallelism and a state-of-the-art scheduler, GNX achieves up to 1.90× and 1.79× speedup, respectively. We also discuss the applicability of and extensions to GNX to support other GNN models.

Palavras-chave: Schedules, Image edge detection, High performance computing, Pipelines, Graphics processing units, Machine learning, Parallel processing, Graph neural networks, Heterogeneous networks, Load modeling, Heterogeneous computing, Edge inference, GNN