Spinner: Enhancing HPC Experimentation with a Streamlined Parameter Sweep Tool

• Rodrigo Ceccato UNICAMP
• Jhonatan Cléto UNICAMP
• Gustavo Leite UNICAMP
• Sandro Rigo UNICAMP
• Jose M. Monsalve Diaz UNICAMP
• Hervé Yviquel UNICAMP

Resumo

We introduce Spinner, an open-source, parameterized sweep benchmark tool for high-performance computing (HPC). Spinner employs a YAML file format to facilitate the execution of applications, data collection, and metadata versioning and sharing. This setup simplifies version control and sharing of experimental configurations. By including dataframes that describe experiments and raw data, it allows readers and reviewers to gain a clearer understanding than summary statistics alone. Spinner's design is non-intrusive and efficient, executing user-supplied commands and dynamically replacing parameters in the command template. It also features the capability to rerun failed experiments, which is particularly useful for testing unstable applications or identifying unstable parameter combinations. We have demonstrated Spinner's functionality with minimal configuration by using it in tests with LAMMPS, GROMACS, Task Bench, XSBench, and RSBench on two different clusters. These tests measured wall time and captured application outputs, underscoring Spinner's practicality and adaptability.

Palavras-chave: High performance computing, Memory management, Benchmark testing, Metadata, Time measurement, Reproducibility of results, Planning, Standards, Monitoring, Software development management