An MLIR Version of the PolyBench Benchmark Suite for High-Performance Computing
Resumo
Polyhedral optimizations play a central role in improving the performance of loop-intensive programs, yet existing tools such as Pluto, LLVM Polly, and the MLIR Affine dialect, rely on heterogeneous infrastructures, input formats, and compilation pipelines. This fragmentation hinders reproducible experiments and fair comparisons across polyhedral frameworks. To address these challenges, we create a complete PolyBench suite rewritten in the MLIR Affine dialect, PolyBench-MLIR, enabling native experimentation with MLIR-based transformations. We further introduce Kiriko, a unified and automated benchmarking infrastructure designed to systematically evaluate polyhedral optimizers under consistent experimental conditions. Kiriko integrates multiple toolchains into a single workflow, providing end-to-end support for optimization, lowering, compilation, and execution. Using Kiriko and PolyBench-MLIR, we conduct a comprehensive performance study of Clang, Polly, Pluto, and MLIR Affine across the PolyBench benchmarks. The results demonstrate the variability of existing polyhedral approaches across computational domains and highlight the importance of standardized infrastructures for advancing research in loop optimization. Overall, Kiriko and the PolyBench-MLIR suite establish a reproducible foundation for developing, comparing, and analyzing modern polyhedral optimization techniques.
Referências
Agostini, N. B., Curzel, S., Kaeli, D., and Tumeo, A. (2022). Soda-opt an mlir based flow for co-design and high-level synthesis. In CF, page 201–202, New York, NY, USA. Association for Computing Machinery.
Bailey, D., Barszcz, E., Barton, J., Browning, D., Carter, R., Dagum, L., Fatoohi, R., Frederickson, P., Lasinski, T., Schreiber, R., Simon, H., Venkatakrishnan, V., and Weeratunga, S. (1991). The nas parallel benchmarks. Int. J. High Perform. Comput. Appl., 5(3):63–73.
Bondhugula, U., Hartono, A., Ramanujam, J., and Sadayappan, P. (2008). A practical automatic polyhedral parallelizer and locality optimizer. In PLDI, PLDI ’08, page 101–113, New York, NY, USA. Association for Computing Machinery.
Bucek, J., Lange, K.-D., and v. Kistowski, J. (2018). Spec cpu2017: Next-generation compute benchmark. In Companion of the 2018 ACM/SPEC International Conference on Performance Engineering, ICPE ’18, page 41–42, New York, NY, USA. Association for Computing Machinery.
Che, S., Boyer, M., Meng, J., Tarjan, D., Sheaffer, J. W., Lee, S.-H., and Skadron, K. (2009). Rodinia: A benchmark suite for heterogeneous computing. In IISWC, IISWC ’09, page 44–54, USA. IEEE Computer Society.
Grosser, T., Groesslinger, A., and Lengaur, C. (2012). Polly - performing polyhedral optimizations on a low-level intermediate representation. Parallel Processing Letters, 22(04):1250010.
Herten, A., Pearce, O., and Guimarães, F. S. (2026). An hpc benchmark survey and taxonomy for characterization. The International Journal of High Performance Computing Applications, 40(1):42–51.
Lattner, C. and Adve, V. (2004). Llvm: A compilation framework for lifelong program analysis & transformation. In International symposium on code generation and optimization, 2004. CGO 2004., pages 75–86. IEEE.
Lattner, C., Amini, M., Bondhugula, U., Cohen, A., Davis, A., Pienaar, J., Riddle, R., Shpeisman, T., Vasilache, N., and Zinenko, O. (2021). MLIR: Scaling compiler infrastructure for domain specific computation. In 2021 IEEE/ACM International Symposium on Code Generation and Optimization (CGO), pages 2–14.
Lee, J., Kim, J., Seo, S., Kim, S., Park, J., Kim, H., Dao, T. T., Cho, Y., Seo, S. J., Lee, S. H., Cho, S. M., Song, H. J., Suh, S.-B., and Choi, J.-D. (2010). An opencl framework for heterogeneous multicores with local memory. In PACT, PACT ’10, page 193–204, New York, NY, USA. Association for Computing Machinery.
LLVM Project. (2026). Mlir affine dialect. [link].
Moses, W. S., Chelini, L., Zhao, R., and Zinenko, O. (2024). Polygeist: Raising c to polyhedral mlir. In Proceedings of the 30th International Conference on Parallel Architectures and Compilation Techniques, PACT ’21, page 45–59. IEEE Press.
Pouchet, L.-N. et al. (2012). Polybench: The polyhedral benchmark suite.
Queiroz, F., Damasceno, E., and Amaris, M. (2023). Predição de consumo de energia de aplicações openmp multithreads. In Escola Regional de Alto Desempenho Norte 2 (ERAD-NO2) e Escola Regional de Aprendizado de Máquina e Inteligência Artificial Norte 2 (ERAMIA-NO2), pages 21–24. SBC.
Sreenivasan, V., Javali, R., Hall, M., Balaprakash, P., Scogland, T. R. W., and de Supinski, B. R. (2019). A framework for enabling openmp autotuning. In OpenMP: Conquering the Full Hardware Spectrum: 15th International Workshop on OpenMP, IWOMP 2019, Auckland, New Zealand, September 11–13, 2019, Proceedings, page 50–60, Berlin, Heidelberg. Springer-Verlag.
Stratton, J. A., Rodrigues, C., Sung, I.-J., Obeid, N., Chang, L.-W., Anssari, N., Liu, G. D., and Hwu, W.-m. W. (2012). Parboil: A revised benchmark suite for scientific and commercial throughput computing. Center for Reliable and High-Performance Computing, 127(7.2):27.
Tørring, J. O., van Werkhoven, B., Petrovč, F., Willemsen, F.-J., Filipovič, J., and Elster, A. C. (2023). Towards a benchmarking suite for kernel tuners. In 2023 IEEE international parallel and distributed processing symposium workshops (IPDPSW), pages 724–733. IEEE.
Weinhardt, M. (2022). An analysis of mapping polybench kernels to hpc cgras. In 2022 IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW), pages 647–654.
Wu, X., Kruse, M., Balaprakash, P., Finkel, H., Hovland, P., Taylor, V., and Hall, M. (2022). Autotuning polybench benchmarks with llvm clang/polly loop optimization pragmas using bayesian optimization. Concurrency and Computation: Practice and Experience, 34(20):e6683.
Yuki, T. (2014). Understanding polybench/c 3.2 kernels. In International workshop on polyhedral compilation techniques (IMPACT), pages 1–5.
