HPC Data Storage at a Glance: The Santos Dumont Experience

  • André Ramos Carneiro UFRGS / LNCC
  • Jean Luca Bez UFRGS
  • Carla Osthoff LNCC
  • Lucas Mello Schnorr UFRGS
  • Philippe O. A. Navaux UFRGS

Resumo


High-Performance Computing (HPC) platforms are used to solve the most diverse scientific problems in research areas, such as biology, chemistry, physics, and health sciences. Researchers use a multitude of scientific software, which have different requirements. These requirements include input and output operations, which directly impact performance due to the existing difference in processing and data access speeds. Thus, supercomputers must efficiently handle a mixed workload scenario when storing data from the applications. Knowledge of the application set and its performance running in a supercomputer is needed to understand the storage system's usage, pinpoint possible bottlenecks, and guide optimization techniques. This research proposes a methodology and visualization tool to evaluate a supercomputer's data storage infrastructure's performance, taking into account the diverse workload and demands of the system over a long period of operation. As a study case, we focus on the Santos Dumont supercomputer, where we were able to identify inefficient usage and problematic factors of performance.
Palavras-chave: File systems, System performance, Scalability, Memory, Metadata, Supercomputers, Usability, Lustre, Parallel File System, High-Performance Storage, I/O Workload, I/O Characterization, Visualization
Publicado
26/10/2021
CARNEIRO, André Ramos; BEZ, Jean Luca; OSTHOFF, Carla; SCHNORR, Lucas Mello; NAVAUX, Philippe O. A.. HPC Data Storage at a Glance: The Santos Dumont Experience. In: INTERNATIONAL SYMPOSIUM ON COMPUTER ARCHITECTURE AND HIGH PERFORMANCE COMPUTING (SBAC-PAD), 33. , 2021, Belo Horizonte. Anais [...]. Porto Alegre: Sociedade Brasileira de Computação, 2021 . p. 157-166.