Analyzing the I/O Performance of Post-Hoc Visualization of Huge Simulation Datasets on the K Computer

• Eduardo C. Inacio UFSC / RIKEN AICS Advanced Visualization Research Team
• Jorji Nonaka RIKEN AICS Advanced Visualization Research Team
• Kenji Ono RIKEN AICS Advanced Visualization Research Team / Kyushu University
• Mario A. R. Dantas UFSC / RIKEN AICS Advanced Visualization Research Team

DOI: https://doi.org/10.5753/wscad.2017.246

Resumo

As computational science simulations produce ever increasing volumes of data, executing part or even the entire visualization pipeline in the supercomputer side becomes more a requirement than an option. Given the uniqueness of the high performance K computer architecture, the HIVE visualization framework was developed, focusing on meeting visualization and data analysis demands of scientists and engineers. In this paper, we present an analysis on the input/output (I/O) performance of post-hoc visualization. The contribution of this research work is characterized by an analysis of a set of empirical study cases considering huge simulation datasets using HIVE on the K computer. Results from the experimental effort, using a dataset produced by a real-world global climate simulation, provide a differentiated knowledge on the impact of dataset partitioning parameters in the I/O performance of large-scale visualization systems, and highlight challenges and opportunities for performance optimizations.

Referências

Ajima, Y., Takagi, Y., Inoue, T., Hiramoto, S., and Shimizu, T. (2011). The Tofu Interconnect. In HOTI '11 Proceedings of the IEEE 19th Annual Symposium on High Performance Interconnects, pages 87–94. IEEE.

Bennett, J. C., Abbasi, H., Bremer, P.-T., Grout, R., Gyulassy, A., Jin, T., Klasky, S., Kolla, H., Parashar, M., Pascucci, V., Pebay, P., Thompson, D., Yu, H., Zhang, F., and Chen, J. (2012). Combining in-situ and in-transit processing to enable extreme-scale In SC '12 Proceedings of the International Conference on High scientic analysis. Performance Computing, Networking, Storage and Analysis. IEEE.

Childs, H., Brugger, E., Whitlock, B., Meredith, J., Ahern, S., Pugmire, D., Biagas, K., Miller, M., Harrison, C., Weber, G. H., Krishnan, H., Fogal, T., Sanderson, A., Garth, C., Bethel, E. W., Camp, D., Rübel, O., Durant, M., Favre, J. M., and Navrátil, P. (2012). VisIt: An End-User Tool For Visualizing and Analyzing Very Large Data. In Bethel, E. W., Childs, H., and Hansen, C., editors, High Performance Visualization: Enabling Extreme-Scale Scientic Insight, pages 357–372. Chapman & Hall/CRC, 1 edition.

Dongarra, J., Meuer, H. W., and Strohmaier, E. (2017). TOP500 Supercomputer Sites.

Dorier, M., Sisneros, R., Gomez, L. B., Peterka, T., Orf, L., Rahmani, L., Antoniu, G., and Bougé, L. (2016). Adaptive Performance-Constrained In Situ Visualization of In CLUSTER '16 Proceedings of the IEEE International Atmospheric Simulations. Conference on Cluster Computing, pages 269–278. IEEE.

Fabian, N., Moreland, K., Thompson, D., Bauer, A. C., Marion, P., Gevecik, B., Rasquin, M., and Jansen, K. E. (2011). The ParaView Coprocessing Library: A scalable, general purpose in situ visualization library. In LDAV '11 Proceedings of the IEEE Symposium on Large Data Analysis and Visualization, pages 89–96. IEEE.

Inacio, E. C., Barbetta, P. A., and Dantas, M. A. R. (2017). A Statistical Analysis of the Performance Variability of Read/Write Operations on Parallel File Systems. Procedia Computer Science Special Issue: International Conference on Computational Science, ICCS 2017, 108:2393–2397.

Inacio, E. C., Pilla, L. L. L., and Dantas, M. A. R. (2015). Understanding the Effect of Multiple Factors on a Parallel File System's Performance. In WETICE '15 Proceedings of the 24th IEEE International Conference on Enabling Technologies: Infrastructure for Collaborative Enterprises, pages 90–92. IEEE.

Miyamoto, Y., Kajikawa, Y., Yoshida, R., Yamaura, T., Yashiro, H., and Tomita, H. (2013). Deep moist atmospheric convection in a subkilometer global simulation. Geophysical Research Letters, 40(18):4922–4926.

Miyazaki, H., Kusano, Y., Shinjou, N., Shoji, F., Yokokawa, M., and Watanabe, T. (2012). Overview of the K computer System. Fujitsu Scientic and Technical Journal, 48(3):255–265.

Nonaka, J., Ono, K., Bi, C., Sakurai, D., Fujita, M., Oku, K., and Kawanabe, T. (2016). HIVE: A Visualization and Analysis Framework for Large-Scale Simulations on the K Computer. In PacicVis '16 Proceedings of the IEEE Pacic Visualization Poster Session. IEEE.

Nonaka, J., Ono, K., and Fujita, M. (2014). Multi-step image compositing for massively parallel rendering. In HPCS '14 Proceedings of the International Conference on High Performance Computing & Simulation, pages 627–634. IEEE.

Ono, K., Kawashima, Y., and Kawanabe, T. (2014). Data Centric Framework for Largescale High-performance Parallel Computation. Procedia Computer Science Special Issue: International Conference on Computational Science, ICCS 2014, 29:2336– 2350.

Roten, D., Cui, Y., Olsen, K. B., Day, S. M., Withers, K., Savran, W. H., Wang, P., and Mu, D. (2016). High-Frequency Nonlinear Earthquake Simulations on Petascale Heterogeneous Supercomputers. In SC '16 Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis. IEEE.

Satoh, M., Tomita, H., Yashiro, H., Kajikawa, Y., Miyamoto, Y., Yamaura, T., Miyakawa, T., Nakano, M., Kodama, C., Noda, A. T., Nasuno, T., Yamada, Y., and Fukutomi, Y. (2017). Outcomes and challenges of global high-resolution non-hydrostatic atmospheric simulations using the K computer. Progress in Earth and Planetary Science, 4(1):24.

Tsujita, Y., Yoshizaki, T., Yamamoto, K., Sueyasu, F., Miyazaki, R., and Uno, A. (2017). Alleviating I/O Interference Through Workload-Aware Striping and Load-Balancing on Parallel File Systems. In Kunkel, J. M., Yokota, R., Balaji, P., and Keyes, D., editors, High Performance Computing, volume 10266 of Lecture Notes in Computer Science, pages 315–333. Springer.