Comparing MPI, MPP, and StarPU Runtimes for Distributed GPU Execution: A Case Study Using N-Body Simulations

• Jhonatan Cléto UNICAMP
• Nicolas Vanz UFSC
• Hervé Yviquel UNICAMP
• Márcio Castro UFSC

DOI: https://doi.org/10.5753/sscad.2026.34916

Resumo

Graphics Processing Unit (GPU) accelerators are a central component of modern High Performance Computing (HPC) systems, but scaling applications across multiple GPUs and nodes requires efficient coordination of computation, communication, data movement, and task dependencies. Different programming models and runtime systems expose these concerns at different abstraction levels, trading programmer control for runtime automation. In this work, we compare three approaches to distributed GPU execution: the Message Passing Interface (MPI), the MPI Proxy Plugin (MPP), and StarPU. Our evaluation uses a direct-method N-Body simulation and Task Bench on up to 32 AMD Instinct MI210 GPUs across eight nodes. The results show that distributed GPU performance depends jointly on runtime organization, kernel implementation, task granularity, and dependency structure, highlighting trade-offs between explicit control, programming portability, and runtime-managed execution.

Referências

Agullo, E., Aumage, O., Bramas, B., Coulaud, O., and Pitoiset, S. (2017). Bridging the Gap Between OpenMP and Task-Based Runtime Systems for the Fast Multipole Method. IEEE Transactions on Parallel and Distributed Systems, 28(10):2794–2807.

AMD (2022). AMD Instinct MI210 Accelerators Specifications. Product specifications page.

AMD (2026). Introduction to the HIP programming model. ROCm Documentation. Accessed: 2026-08-14.

Augonnet, C., Thibault, S., Namyst, R., and Wacrenier, P.-A. (2010). StarPU: a unified platform for task scheduling on heterogeneous multicore architectures. Concurrency and Computation: Practice and Experience, 23(2):187–198.

Barnes, J. and Hut, P. (1986). A hierarchical o(n log n) force-calculation algorithm. Nature, 324(6096):446–449.

Ceccato, R., Cléto, J., Leite, G., Rigo, S., Diaz, J. M. M., and Yviquel, H. (2024). Spinner: Enhancing HPC Experimentation with a Streamlined Parameter Sweep Tool. In 2024 International Symposium on Computer Architecture and High Performance Computing Workshops (SBAC-PADW), pages 1–11.

Cléto, J., Valarini, G., Pereira, M. M., Araujo, G., and Yviquel, H. (2025a). Scalable OpenMP Remote Offloading via Asynchronous MPI and Coroutine-Driven Communication. In Nagel, W. E., Goehringer, D., and Diniz, P. C., editors, Euro-Par 2025: Parallel Processing, pages 254–267, Cham. Springer Nature Switzerland.

Cléto, J., Valarini, G., and Yviquel, H. (2025b). Can OpenMP Scale Beyond the Node? A Performance Evaluation of Remote Offloading via the MPI Proxy Plugin. In Anais do XXVI Simpósio em Sistemas Computacionais de Alto Desempenho, pages 278–289, Porto Alegre, RS, Brasil. SBC.

Ferat, M., Pereira, R., Roussel, A., Carribault, P., Steffenel, L.-A., and Gautier, T. (2022). Enhancing MPI+OpenMP Task Based Applications for Heterogeneous Architectures with GPU Support. In OpenMP in a Modern World: From Multi-Device Support to Meta Programming, Lecture Notes in Computer Science, pages 3–16. Springer.

Heggie, D. (2005). The Classical Gravitational N-Body Problem. Encyclopedia of Mathematical Physics.

Miletto, M. C. and Schnorr, L. M. (2019). OpenMP and StarPU Abreast: the Impact of Runtime in Task-Based Block QR Factorization Performance. In Anais do XX Simpósio em Sistemas Computacionais de Alto Desempenho (SSCAD 2019), pages 25–36.

Netto, M. A. S., Calheiros, R. N., Rodrigues, E. R., Cunha, R. L. F., and Buyya, R. (2018). HPC Cloud for Scientific and Business Applications: Taxonomy, Vision, and Research Challenges. ACM Computing Surveys, 51(1).

Nyland, L., Harris, M., and Prins, J. (2007). Fast N-Body Simulation with CUDA. In Nguyen, H., editor, GPU Gems 3, chapter 31. Addison-Wesley.

Sharma, R. and Kanungo, P. (2011). Performance Evaluation of MPI and Hybrid MPI+OpenMP Programming Paradigms on Multi-Core Processors Cluster. In 2011 International Conference on Recent Trends in Information Systems, pages 137–140.

Slaughter, E., Wu, W., Fu, Y., Brandenburg, L., Garcia, N., Kautz, W., Marx, E., Morris, K. S., Cao, Q., Bosilca, G., Mirchandaney, S., Leek, W., Treichlerk, S., McCormick, P., and Aiken, A. (2020). Task Bench: A Parameterized Benchmark for Evaluating Par allel Runtime Performance. In SC20: International Conference for High Performance Computing, Networking, Storage and Analysis, pages 1–15.

Vanz, N., Munhoz, V., and Castro, M. (2025a). Performance Evaluation of N-Body Simulations on AWS with StarPU, OpenMP and MPI Runtime Systems. In Anais do XXVI Simpósio em Sistemas Computacionais de Alto Desempenho, pages 61–72, Porto Alegre, RS, Brasil. SBC.

Vanz, N., Munhoz, V., Castro, M., Lima Pilla, L., and Aumage, O. (2025b). Task-Based HPC in the Cloud: Price-Performance Analysis of N-Body Simulations with StarPU. In IC2E 2025 - 13th IEEE International Conference on Cloud Engineering, Rennes, France.

Yviquel, H., Pereira, M., Francesquini, E., Valarini, G., Leite, G., Rosso, P., Ceccato, R., Cusihualpa, C., Dias, V., Rigo, S., Souza, A., and Araujo, G. (2023). The OpenMP Cluster Programming Model. In Workshop Proceedings of the 51st International Conference on Parallel Processing, New York, NY, USA.

Zwart, S. F. P., Belleman, R. G., and Geldof, P. M. (2007). High-performance direct gravitational N-body simulations on graphics processing units. New Astronomy, 12(8):641–650.