HPC Data Storage at a Glance: The Santos Dumont Experience

• André Ramos Carneiro UFRGS / LNCC
• Jean Luca Bez UFRGS
• Carla Osthoff LNCC
• Lucas Mello Schnorr UFRGS
• Philippe O. A. Navaux UFRGS

Resumo

High-Performance Computing (HPC) platforms are used to solve the most diverse scientific problems in research areas, such as biology, chemistry, physics, and health sciences. Researchers use a multitude of scientific software, which have different requirements. These requirements include input and output operations, which directly impact performance due to the existing difference in processing and data access speeds. Thus, supercomputers must efficiently handle a mixed workload scenario when storing data from the applications. Knowledge of the application set and its performance running in a supercomputer is needed to understand the storage system's usage, pinpoint possible bottlenecks, and guide optimization techniques. This research proposes a methodology and visualization tool to evaluate a supercomputer's data storage infrastructure's performance, taking into account the diverse workload and demands of the system over a long period of operation. As a study case, we focus on the Santos Dumont supercomputer, where we were able to identify inefficient usage and problematic factors of performance.

Palavras-chave: File systems, System performance, Scalability, Memory, Metadata, Supercomputers, Usability, Lustre, Parallel File System, High-Performance Storage, I/O Workload, I/O Characterization, Visualization