Can OpenMP Scale Beyond the Node? A Performance Evaluation of Remote Offloading via the MPI Proxy Plugin - Cléto, Jhonatan; Valarini, Guilherme; Yviquel, Hervé (Anais do Simpósio em Sistemas Computacionais de Alto Desempenho (SSCAD); 2025: Anais do XXVI Simpósio em Sistemas Computacionais de Alto DesempenhoDO - 10.5753/sscad.2025.16707, 2025)
We evaluate the MPI Proxy Plugin (MPP), an extension to LLVM/OpenMP that enables remote offloading of target regions via MPI, allowing distributed GPU execution without modifying application code. Using benchmarks on NVIDIA H100 and AMD MI300A nodes, we show that MPP delivers competitive performance compared to traditional MPI+OpenMP, particularly for coarse-grained, compute-intensive workloads. While MPP simplifies development and enables communication–computation overlap, it introduces higher runtime overheads and task granularity requirements. Our results position MPP as a promising step toward unifying shared and distributed heterogeneous programming under the OpenMP model.