Experimental Validation of SimGrid Predictions for RTM Applications in Multi-Node Multi-GPU Environments
Resumo
Simulation models calibrated on one hardware generation are reused without validation. A SimGrid/SMPI model of Fletcher, an anisotropic reverse time migration (RTM) application, calibrated on one GPU per node at 1 Gbit/s, predicted higher bandwidths raise the computing-time fraction, untested with multi-GPU nodes. We run Fletcher’s CUDA backend on the Grid’5000 chuc cluster, four A100 GPUs per node, one to four nodes, at 1, 10, and 25 Gbit/s. The trend holds qualitatively, but diverges quantitatively with scale: overlap spreads from 1.3 to 38.2 points, with limited gains above 10 Gbit/s. A same-hardware simulation matches within 13% at 10–25 Gbit/s, but underestimates throughput by 40–50% at 1 Gbit/s beyond one node, from message sizes outside calibration. Direction generalizes across generations; magnitude does not.
Referências
Balouek, D., Carpen Amarie, A., Charrier, G., Desprez, F., Jeannot, E., Jeanvoine, E., Lèbre, A., Margery, D., Niclausse, N., Nussbaum, L., Richard, O., Pérez, C., Quesnel, F., Rohr, C., and Sarzyniec, L. (2013). Adding virtualization capabilities to the Grid’5000 testbed. In Ivanov, I. I., van Sinderen, M., Leymann, F., and Shan, T., editors, Cloud Computing and Services Science, volume 367 of Communications in Computer and Information Science, pages 3–20. Springer.
Bédaride, P., Degomme, A., Genaud, S., Legrand, A., Markomanolis, G. S., Quinson, M., Stillwell, M., Suter, F., and Videau, B. (2013). Toward better simulation of MPI applications on Ethernet/TCP networks. In High Performance Computing Systems. Performance Modeling, Benchmarking and Simulation (PMBS 2013), volume 8324 of Lecture Notes in Computer Science, pages 1–20, Cham. Springer.
Boughzala, D., Lefèvre, L., and Orgerie, A.-C. (2020). Predicting the energy consumption of CUDA kernels using SimGrid. In 2020 IEEE 32nd International Symposium on Computer Architecture and High Performance Computing (SBAC-PAD), pages 191–198, Porto, Portugal. IEEE.
Casanova, H., Giersch, A., Legrand, A., Quinson, M., and Suter, F. (2014). Versatile, scalable, and accurate simulation of distributed applications and platforms. Journal of Parallel and Distributed Computing, 74(10):2899–2917.
Cherrueau, R.-A., Delavergne, M., Van Kempen, A., Lebre, A., Pertin, D., Rojas Balderrama, J., Simonet, A., and Simonin, M. (2022). EnOSlib: A library for experiment-driven research in distributed computing. IEEE Transactions on Parallel and Distributed Systems, 33(6):1464–1477.
Cornebize, T. (2021). High Performance Computing: towards better Performance Predictions and Experiments. PhD thesis, Université Grenoble Alpes.
Degomme, A., Legrand, A., Markomanolis, G. S., Quinson, M., Stillwell, M., and Suter, F. (2017). Simulating MPI applications: The SMPI approach. IEEE Transactions on Parallel and Distributed Systems, 28(8):2387–2400.
Denis, A. and Trahay, F. (2016). MPI overlap: Benchmark and analysis. In 2016 45th International Conference on Parallel Processing (ICPP), pages 258–267, Philadelphia, PA, USA. IEEE.
Energy Sciences Network (ESnet) (2026). iPerf3: A TCP, UDP, and SCTP network bandwidth measurement tool. [link]. Version 3.21.
Fletcher, R. P., Du, X., and Fowler, P. J. (2009). Reverse time migration in tilted transversely isotropic (TTI) media. Geophysics, 74(6):WCA179–WCA187.
Hoefler, T. and Belli, R. (2015). Scientific benchmarking of parallel computing systems: twelve ways to tell the masses when reporting performance results. In Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis (SC ’15), New York, NY, USA. Association for Computing Machinery.
Keller Tesser, R., Mello Schnorr, L., Legrand, A., Dupros, F., and Navaux, P. O. A. (2017). Using simulation to evaluate and tune the performance of dynamic load balancing of an over-decomposed geophysics application. In Rivera, F. F., Pena, T. F., and Cabaleiro, J. C., editors, Euro-Par 2017: Parallel Processing, volume 10417 of Lecture Notes in Computer Science, pages 192–205, Cham. Springer.
Künas, C. A., Freytag, G., Araújo, T., Machado, R., Morales, B., Sardinha, A., and Navaux, P. O. A. (2025). Compressão de dados para redução de E/S em simulações sísmicas de alto desempenho. In Anais do XXVI Simpósio em Sistemas Computacionais de Alto Desempenho (SSCAD), pages 326–337, Bonito, Brazil. Sociedade Brasileira de Computação. (in Portuguese).
Künas, C. A., Freytag, G., and Navaux, P. O. A. (2024). Enhancing reverse time migration simulations in HPC systems through I/O and computation overlapping. In Guerrero, G., San Martín, J., Meneses, E., Barrios Hernández, C. J., Osthoff, C., and Monsalve Diaz, J. M., editors, High Performance Computing: 11th Latin American High Performance Computing Conference, CARLA 2024, Santiago de Chile, Chile, Revised Selected Papers, volume 2270 of Communications in Computer and Information Science, pages 68–82. Springer, Cham.
Medvedev, A. V. (2022). IMB-ASYNC: a revised method and benchmark to estimate MPI-3 asynchronous progress efficiency. Cluster Computing, 25(4):2683–2697.
Micikevicius, P. (2009). 3D finite difference computation on GPUs using CUDA. In Proceedings of the 2nd Workshop on General Purpose Processing on Graphics Processing Units (GPGPU-2), pages 79–84, NY, USA. Association for Computing Machinery.
Pearson, C., Hidayetoğlu, M., Almasri, M., Anjum, O., Chung, I.-H., Xiong, J., and Hwu, W.-M. W. (2020). Node-aware stencil communication for heterogeneous supercomputers. In 2020 IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW), pages 796–805. IEEE.
Potluri, S., Hamidouche, K., Venkatesh, A., Bureddy, D., and Panda, D. K. (2013). Efficient inter-node MPI communication using GPUDirect RDMA for InfiniBand clusters with NVIDIA GPUs. In 2013 42nd International Conference on Parallel Processing (ICPP), pages 80–89. IEEE.
Rigon, P. H. C., Schussler, B. S., Künas, C. A., Lorenzon, A. F., Carissimi, A., and Navaux, P. O. A. (2024). Otimizando a implementação multi-GPU do método fletcher através da paralelização eficiente na computação e comunicação de dados. In Anais da XXIV Escola Regional de Alto Desempenho da Região Sul (ERAD-RS), pages 5–8, Florianópolis, Brazil. Sociedade Brasileira de Computação. (in Portuguese).
Salim, J. H. (2015). Linux traffic control classifier-action subsystem architecture. In Proceedings of netdev 0.1, Ottawa, ON, Canada.
Spadotto, V. D. (2026). Distributed implementation of anisotropic RTM using the fletcher method with MPI and ghost cells. B.s. thesis, Universidade Federal do Rio Grande do Sul, Porto Alegre, Brazil.
Spadotto, V. D. and Mello Schnorr, L. (2026a). MPI-based 3D anisotropic RTM with fletcher’s method and ghost cell halo exchange. In 2026 IEEE 38th International Symposium on Computer Architecture and High Performance Computing (SBAC-PAD), Madrid, Spain. To appear.
Spadotto, V. D. and Mello Schnorr, L. (2026b). SimGrid to evaluate ghost cells effectiveness in high-performance computing applications. In Latin America High Performance Computing Conference (CARLA), Córdoba, Argentina. To appear.
Stanisic, L., Thibault, S., Legrand, A., Videau, B., and Méhaut, J.-F. (2015). Faithful performance prediction of a dynamic task-based runtime system for heterogeneous multi-core architectures. Concurrency and Computation: Practice and Experience, 27(16):4075–4090.
Unat, D., Turimbetov, I., Issa, M. K. T., Sağbili, D., Vella, F., De Sensi, D., and Ismayilov, I. (2026). The landscape of GPU-centric communication. ACM Computing Surveys, 58(12):1–36.
Velho, P., Schnorr, L. M., Casanova, H., and Legrand, A. (2013). On the validity of flow-level TCP network models for grid and cloud simulations. ACM Transactions on Modeling and Computer Simulation, 23(4):23:1–23:26.
Zhou, H.-W., Hu, H., Zou, Z., Wo, Y., and Youn, O. (2018). Reverse time migration: A prospect of seismic imaging methodology. Earth-Science Reviews, 179:207–227.
Bédaride, P., Degomme, A., Genaud, S., Legrand, A., Markomanolis, G. S., Quinson, M., Stillwell, M., Suter, F., and Videau, B. (2013). Toward better simulation of MPI applications on Ethernet/TCP networks. In High Performance Computing Systems. Performance Modeling, Benchmarking and Simulation (PMBS 2013), volume 8324 of Lecture Notes in Computer Science, pages 1–20, Cham. Springer.
Boughzala, D., Lefèvre, L., and Orgerie, A.-C. (2020). Predicting the energy consumption of CUDA kernels using SimGrid. In 2020 IEEE 32nd International Symposium on Computer Architecture and High Performance Computing (SBAC-PAD), pages 191–198, Porto, Portugal. IEEE.
Casanova, H., Giersch, A., Legrand, A., Quinson, M., and Suter, F. (2014). Versatile, scalable, and accurate simulation of distributed applications and platforms. Journal of Parallel and Distributed Computing, 74(10):2899–2917.
Cherrueau, R.-A., Delavergne, M., Van Kempen, A., Lebre, A., Pertin, D., Rojas Balderrama, J., Simonet, A., and Simonin, M. (2022). EnOSlib: A library for experiment-driven research in distributed computing. IEEE Transactions on Parallel and Distributed Systems, 33(6):1464–1477.
Cornebize, T. (2021). High Performance Computing: towards better Performance Predictions and Experiments. PhD thesis, Université Grenoble Alpes.
Degomme, A., Legrand, A., Markomanolis, G. S., Quinson, M., Stillwell, M., and Suter, F. (2017). Simulating MPI applications: The SMPI approach. IEEE Transactions on Parallel and Distributed Systems, 28(8):2387–2400.
Denis, A. and Trahay, F. (2016). MPI overlap: Benchmark and analysis. In 2016 45th International Conference on Parallel Processing (ICPP), pages 258–267, Philadelphia, PA, USA. IEEE.
Energy Sciences Network (ESnet) (2026). iPerf3: A TCP, UDP, and SCTP network bandwidth measurement tool. [link]. Version 3.21.
Fletcher, R. P., Du, X., and Fowler, P. J. (2009). Reverse time migration in tilted transversely isotropic (TTI) media. Geophysics, 74(6):WCA179–WCA187.
Hoefler, T. and Belli, R. (2015). Scientific benchmarking of parallel computing systems: twelve ways to tell the masses when reporting performance results. In Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis (SC ’15), New York, NY, USA. Association for Computing Machinery.
Keller Tesser, R., Mello Schnorr, L., Legrand, A., Dupros, F., and Navaux, P. O. A. (2017). Using simulation to evaluate and tune the performance of dynamic load balancing of an over-decomposed geophysics application. In Rivera, F. F., Pena, T. F., and Cabaleiro, J. C., editors, Euro-Par 2017: Parallel Processing, volume 10417 of Lecture Notes in Computer Science, pages 192–205, Cham. Springer.
Künas, C. A., Freytag, G., Araújo, T., Machado, R., Morales, B., Sardinha, A., and Navaux, P. O. A. (2025). Compressão de dados para redução de E/S em simulações sísmicas de alto desempenho. In Anais do XXVI Simpósio em Sistemas Computacionais de Alto Desempenho (SSCAD), pages 326–337, Bonito, Brazil. Sociedade Brasileira de Computação. (in Portuguese).
Künas, C. A., Freytag, G., and Navaux, P. O. A. (2024). Enhancing reverse time migration simulations in HPC systems through I/O and computation overlapping. In Guerrero, G., San Martín, J., Meneses, E., Barrios Hernández, C. J., Osthoff, C., and Monsalve Diaz, J. M., editors, High Performance Computing: 11th Latin American High Performance Computing Conference, CARLA 2024, Santiago de Chile, Chile, Revised Selected Papers, volume 2270 of Communications in Computer and Information Science, pages 68–82. Springer, Cham.
Medvedev, A. V. (2022). IMB-ASYNC: a revised method and benchmark to estimate MPI-3 asynchronous progress efficiency. Cluster Computing, 25(4):2683–2697.
Micikevicius, P. (2009). 3D finite difference computation on GPUs using CUDA. In Proceedings of the 2nd Workshop on General Purpose Processing on Graphics Processing Units (GPGPU-2), pages 79–84, NY, USA. Association for Computing Machinery.
Pearson, C., Hidayetoğlu, M., Almasri, M., Anjum, O., Chung, I.-H., Xiong, J., and Hwu, W.-M. W. (2020). Node-aware stencil communication for heterogeneous supercomputers. In 2020 IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW), pages 796–805. IEEE.
Potluri, S., Hamidouche, K., Venkatesh, A., Bureddy, D., and Panda, D. K. (2013). Efficient inter-node MPI communication using GPUDirect RDMA for InfiniBand clusters with NVIDIA GPUs. In 2013 42nd International Conference on Parallel Processing (ICPP), pages 80–89. IEEE.
Rigon, P. H. C., Schussler, B. S., Künas, C. A., Lorenzon, A. F., Carissimi, A., and Navaux, P. O. A. (2024). Otimizando a implementação multi-GPU do método fletcher através da paralelização eficiente na computação e comunicação de dados. In Anais da XXIV Escola Regional de Alto Desempenho da Região Sul (ERAD-RS), pages 5–8, Florianópolis, Brazil. Sociedade Brasileira de Computação. (in Portuguese).
Salim, J. H. (2015). Linux traffic control classifier-action subsystem architecture. In Proceedings of netdev 0.1, Ottawa, ON, Canada.
Spadotto, V. D. (2026). Distributed implementation of anisotropic RTM using the fletcher method with MPI and ghost cells. B.s. thesis, Universidade Federal do Rio Grande do Sul, Porto Alegre, Brazil.
Spadotto, V. D. and Mello Schnorr, L. (2026a). MPI-based 3D anisotropic RTM with fletcher’s method and ghost cell halo exchange. In 2026 IEEE 38th International Symposium on Computer Architecture and High Performance Computing (SBAC-PAD), Madrid, Spain. To appear.
Spadotto, V. D. and Mello Schnorr, L. (2026b). SimGrid to evaluate ghost cells effectiveness in high-performance computing applications. In Latin America High Performance Computing Conference (CARLA), Córdoba, Argentina. To appear.
Stanisic, L., Thibault, S., Legrand, A., Videau, B., and Méhaut, J.-F. (2015). Faithful performance prediction of a dynamic task-based runtime system for heterogeneous multi-core architectures. Concurrency and Computation: Practice and Experience, 27(16):4075–4090.
Unat, D., Turimbetov, I., Issa, M. K. T., Sağbili, D., Vella, F., De Sensi, D., and Ismayilov, I. (2026). The landscape of GPU-centric communication. ACM Computing Surveys, 58(12):1–36.
Velho, P., Schnorr, L. M., Casanova, H., and Legrand, A. (2013). On the validity of flow-level TCP network models for grid and cloud simulations. ACM Transactions on Modeling and Computer Simulation, 23(4):23:1–23:26.
Zhou, H.-W., Hu, H., Zou, Z., Wo, Y., and Youn, O. (2018). Reverse time migration: A prospect of seismic imaging methodology. Earth-Science Reviews, 179:207–227.
Publicado
03/11/2026
Como Citar
SILVA, Arthur Andrade da; SPADOTTO, Vinícius Daniel; KÜNAS, Cristiano Alex; SCHNORR, Lucas Mello; NAVAUX, Phillipe Olivier Alexandre.
Experimental Validation of SimGrid Predictions for RTM Applications in Multi-Node Multi-GPU Environments. In: SIMPÓSIO EM SISTEMAS COMPUTACIONAIS DE ALTO DESEMPENHO (SSCAD), 27. , 2026, Natal/RN.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2026
.
p. 637-650.
DOI: https://doi.org/10.5753/sscad.2026.34996.
