Data mapping strategies for multi-GPU implementation of a seismic application
Resumo
Seismic imaging applications are computationally costly, and the industry’s demand is continuously increasing due to the availability of better data, larger data, and the need for better resolution images. It means that the computational capacity needed tends to increase both in terms of FLOPS calculation and memory. Nowadays, many HPC clusters have nodes with multiple GPUs (e.g., 2, 4, and 8). In this paper, we investigate mechanisms and strategies for the data exchange (of the halo zones) of a finite differences grid of a wave simulator implemented in OpenMP. We compare the performance and programming effort of four data mapping mechanisms supported by OpenMP and CUDA. Our best strategy has achieved speedups of 3.87 on four V100 GPUs with NVLink.
Referências
Bohlen, T. (2002). Parallel 3-d viscoelastic finite difference seismic modelling. Computers & Geosciences, 28(8):887–899.
Cai, X., Liu, Y., and Ren, Z. (2018). Acoustic reverse-time migration using gpu card and posix thread based on the adaptive optimal finite-difference scheme and the hybrid absorbing boundary condition. Computers & Geosciences, 115:42–55.
Datta, D., Appelhans, D., Evangelinos, C., and Jordan, K. (2018). An asynchronous two-level checkpointing method to solve adjoint problems on hierarchical memory spaces. Computing in Science and Engineering, 20:39–55.
de Souza, J., Moreira, J., Oliveira, A., Roberts, K., Gomi, E., Silva, E., and Senger, H. (2022a). A GPU accelerated modeling and simulation of acoustic waves for seismic applications. In Third EAGE Workshop on HPC in Americas, pages 1–5. European Association of Geoscientists & Engineers.
de Souza, J. F., Machado, L. S. F., Gomi, E. S., Tadonki, C., Mcintosh-Smith, S., and Senger, H. (2022b). Performance of openmp loop transformations for the acoustic wave stencil on gpus. In The International Conference for High Performance Computing, Networking, Storage, and Analysis (Supercomputing 22 Poster track).
de Souza, J. F., Moreira, J. B. D., Roberts, K. J., di Ramos Alves Gaioso, R., Gomi, E. S., Silva, E. C. N., and Senger, H. (2022c). simwave A finite difference simulator for acoustic waves propagation. CoRR, abs/2201.05278.
Deng, J., Rogez, Y., Zhu, P., Herique, A., Jiang, J., and Kofman, W. (2021). 3d time-domain electromagnetic full waveform inversion in debye dispersive medium accelerated by multi-gpu paralleling. Computer Physics Communications, 265.
Gokhberg, A. and Fichtner, A. (2016). Full-waveform inversion on heterogeneous hpc systems. Computers & Geosciences, 89:260–268.
Hölzle, U. (2017). Schlumberger chooses GCP to deliver new oil and gas technology platform. [link]. Last accessed on 2nd May, 2023.
Kim, Y., Shin, C., and Calandra, H. (2012). 3D Hybrid Waveform Inversion With GPU Devices. SEG Int. Exposition and Annual Meeting. SEG-2012.
Mattson, T., He, Y., and Koniges, A. (2019). The OpenMP Common Core: Making OpenMP Simple Again. Scientific and Engineering Computation. MIT Press.
Meng, J. and Skadron, K. (2009). Performance modeling and automatic ghost zone optimization for iterative stencil loops on gpus. In The International Conference on Supercomputing, pages 256–265.
Micikevicius, P. (2009). 3d finite difference computation on gpus using cuda. In Workshop on General Purpose Processing on Graphics Processing Units, page 79–84, New York, NY, USA. ACM.
Virieux, J. and Operto, S. (2009). An overview of full-waveform inversion in exploration geophysics. GEOPHYSICS, 74(6):WCC1–WCC26.
Xu, R., Tian, X., Chandrasekaran, S., and Chapman, B. (2015). Multi-gpu support on single node using directive-based programming model. Scientific Programming, 2015:621730.
Cai, X., Liu, Y., and Ren, Z. (2018). Acoustic reverse-time migration using gpu card and posix thread based on the adaptive optimal finite-difference scheme and the hybrid absorbing boundary condition. Computers & Geosciences, 115:42–55.
Datta, D., Appelhans, D., Evangelinos, C., and Jordan, K. (2018). An asynchronous two-level checkpointing method to solve adjoint problems on hierarchical memory spaces. Computing in Science and Engineering, 20:39–55.
de Souza, J., Moreira, J., Oliveira, A., Roberts, K., Gomi, E., Silva, E., and Senger, H. (2022a). A GPU accelerated modeling and simulation of acoustic waves for seismic applications. In Third EAGE Workshop on HPC in Americas, pages 1–5. European Association of Geoscientists & Engineers.
de Souza, J. F., Machado, L. S. F., Gomi, E. S., Tadonki, C., Mcintosh-Smith, S., and Senger, H. (2022b). Performance of openmp loop transformations for the acoustic wave stencil on gpus. In The International Conference for High Performance Computing, Networking, Storage, and Analysis (Supercomputing 22 Poster track).
de Souza, J. F., Moreira, J. B. D., Roberts, K. J., di Ramos Alves Gaioso, R., Gomi, E. S., Silva, E. C. N., and Senger, H. (2022c). simwave A finite difference simulator for acoustic waves propagation. CoRR, abs/2201.05278.
Deng, J., Rogez, Y., Zhu, P., Herique, A., Jiang, J., and Kofman, W. (2021). 3d time-domain electromagnetic full waveform inversion in debye dispersive medium accelerated by multi-gpu paralleling. Computer Physics Communications, 265.
Gokhberg, A. and Fichtner, A. (2016). Full-waveform inversion on heterogeneous hpc systems. Computers & Geosciences, 89:260–268.
Hölzle, U. (2017). Schlumberger chooses GCP to deliver new oil and gas technology platform. [link]. Last accessed on 2nd May, 2023.
Kim, Y., Shin, C., and Calandra, H. (2012). 3D Hybrid Waveform Inversion With GPU Devices. SEG Int. Exposition and Annual Meeting. SEG-2012.
Mattson, T., He, Y., and Koniges, A. (2019). The OpenMP Common Core: Making OpenMP Simple Again. Scientific and Engineering Computation. MIT Press.
Meng, J. and Skadron, K. (2009). Performance modeling and automatic ghost zone optimization for iterative stencil loops on gpus. In The International Conference on Supercomputing, pages 256–265.
Micikevicius, P. (2009). 3d finite difference computation on gpus using cuda. In Workshop on General Purpose Processing on Graphics Processing Units, page 79–84, New York, NY, USA. ACM.
Virieux, J. and Operto, S. (2009). An overview of full-waveform inversion in exploration geophysics. GEOPHYSICS, 74(6):WCC1–WCC26.
Xu, R., Tian, X., Chandrasekaran, S., and Chapman, B. (2015). Multi-gpu support on single node using directive-based programming model. Scientific Programming, 2015:621730.
Publicado
17/10/2023
Como Citar
FREIRE, Yuri Nicolau; GOMI, Edson; SENGER, Hermes.
Data mapping strategies for multi-GPU implementation of a seismic application. In: SIMPÓSIO EM SISTEMAS COMPUTACIONAIS DE ALTO DESEMPENHO (SSCAD), 24. , 2023, Porto Alegre/RS.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2023
.
p. 109-120.
DOI: https://doi.org/10.5753/wscad.2023.235856.
