Mainstream vs. Emerging HPC: Metrics, Trade-Offs and Lessons Learned - Radulovic, Milan; Asifuzzaman, Kazi; Zivanovic, Darko; Rajovic, Nikola; Verdiére, Guillaume Colin de; Pleiter, Dirk; Marazakisl, Manolis; Kallimanis, Nikolaos; Carpenter, Paul; Radojković, Petar; Ayguadé, Eduard (Anais do International Symposium on Computer Architecture and High Performance Computing (SBAC-PAD); 2018: Anais do XXX International Symposium on Computer Architecture and High Performance Computing, 2018)
Various servers with different characteristics and architectures are hitting the market, and their evaluation and comparison in terms of HPC features is complex and multidimensional. In this paper, we share our experience of evaluating a diverse set of HPC systems, consisting of three mainstream and five emerging architectures. We evaluate the performance and power efficiency using prominent HPC benchmarks, High-Performance Linpack (HPL) and High Performance Conjugate Gradients (HPCG), and expand our analysis using publicly available specialized kernel benchmarks, targeting specific system components. In addition to a large body of quantitative results, we emphasize six usually overlooked aspects of the HPC platforms evaluation, and share our conclusions and lessons learned. Overall, we believe that this paper will improve the evaluation and comparison of HPC platforms, making a first step towards a more reliable and uniform methodology.