Escalando a E/S do SDDP em Ambientes HPC: da Escrita Serializada às Escritas Paralelas com MPI-IO sobre Lustre
Resumo
A escalabilidade eficiente de E/S é um desafio crítico em aplicações paralelas de alto desempenho, sobretudo em modelos de otimização do setor energético como o SDDP, que demandam acesso frequente e distribuído a grandes conjuntos de dados. Este artigo investiga os gargalos de E/S da fase de simulação final do SDDP e propõe uma arquitetura baseada em MPI-IO sobre Lustre, substituindo a estratégia atual com escritor único. Avaliada nos ambientes AWS (FSx for Lustre) e Santos Dumont (LNCC), de 2 a 32 nós, reduz o tempo total em até 36,7% (AWS) e 40,4% (SDumont) em 32 nós, com ganhos de banda agregada percebida de 113× e 13,7×, respectivamente.
Referências
Amazon Web Services (2024a). Amazon EC2 R7i instances — Memory Optimized. [link].
Amazon Web Services (2024b). Elastic network adapter (ENA) — enhanced networking on Linux. Amazon EC2 User Guide.
Amazon Web Services (2024c). Placement groups for Amazon EC2 instances. Amazon EC2 User Guide.
Amdahl, G. M. (1967). Validity of the single processor approach to achieving large scale computing capabilities. In Proceedings of the April 18–20, 1967, Spring Joint Computer Conference, AFIPS ’67 (Spring), pages 483–485, New York, NY, USA. Association for Computing Machinery.
Braam, P. J. (2002). The Lustre storage architecture. Cluster File Systems Inc.
Carns, P., Latham, R., Ross, R., Iskra, K., Lang, S., and Riley, K. (2009). 24/7 characterization of petascale I/O workloads. In Proceedings of the 2009 IEEE International Conference on Cluster Computing and Workshops, pages 1–10. IEEE.
Cirne, W., Brasileiro, F., Sauvé, J., Andrade, N., Paranhos, D., Santos-Neto, E., and Medeiros, R. (2003). Grid computing for bag of tasks applications. In Proceedings of the 3rd IFIP Conference on E-Commerce, E-Business and E-Government (I3E).
Conejo, A. J., Carrión, M., and Morales, J. M. (2010). Decision Making Under Uncertainty in Electricity Markets, volume 153 of International Series in Operations Research & Management Science. Springer, New York, NY.
Dickens, P. M. and Logan, J. (2008). Towards an understanding of the performance of MPI-IO in Lustre file systems. In Proceedings of the IEEE International Conference on Cluster Computing, CLUSTER ’08, pages 254–263. IEEE.
Dickens, P. M. and Logan, J. (2010). A high performance implementation of MPI-IO for a Lustre file system environment. Concurrency and Computation: Practice and Experience, 22(11):1433–1449.
Garcia-Carballeira, F., Camarmas-Alonso, D., Calderon-Mateos, A., and Carretero, J. (2023). A new Ad-Hoc parallel file system for HPC environments based on the Expand parallel file system. In Proceedings of the 22nd International Symposium on Parallel and Distributed Computing, ISPDC ’23, pages 69–76. IEEE.
Graham, R. L., Shipman, G. M., Barrett, B. W., Castain, R. H., Bosilca, G., and Lumsdaine, A. (2005). Open MPI: A high-performance, heterogeneous MPI. In Proceedings of the 5th International Workshop on Algorithms, Models and Tools for Parallel Computing on Heterogeneous Networks.
Gropp, W., Kimpe, D., Ross, R., Thakur, R., and Träff, J. L. (2008). Self-consistent MPIIO performance requirements and expectations. In Proceedings of the 15th European PVM/MPI Users’ Group Meeting, EuroPVM/MPI ’08, pages 167–176. Springer.
Intel Corporation (2023). Intel Xeon scalable processors — 4th generation (Sapphire Rapids). [link].
Kang, Q., Lee, S., Hou, K.-y., Ross, R., Agrawal, A., Choudhary, A., and Liao, W.k. (2020). Improving MPI collective I/O for high volume non-contiguous requests with intra-node aggregation. IEEE Transactions on Parallel and Distributed Systems, 31(11):2682–2695.
Liao, W.-k., Ching, A., Coloma, K., Choudhary, A., and Kandemir, M. (2007). Improving MPI independent write performance using a two-stage write-behind buffering method. In Proceedings of the 21st IEEE International Parallel and Distributed Processing Symposium, IPDPS ’07. IEEE.
Liao, W.-k. and Choudhary, A. (2008). Dynamically adapting file domain partitioning methods for collective I/O based on underlying parallel file system locking protocols. In Proceedings of the ACM/IEEE Conference on Supercomputing, SC ’08. IEEE.
Luu, H., Winslett, M., Gropp, W., Ross, R., Carns, P., Harms, K., Prabhat, Byna, S., and Yao, Y. (2015). A multiplatform study of I/O behavior on petascale supercomputers. In Proceedings of the 24th International Symposium on High-Performance Parallel and Distributed Computing, HPDC ’15, pages 33–44, New York, NY, USA. Association for Computing Machinery.
Message Passing Interface Forum (1997). MPI-2: Extensions to the message-passing interface. Specification.
Message Passing Interface Forum (2023). MPI: A message-passing interface standard, version 4.1. [link].
Message Passing Interface Forum (2025). MPI: A message-passing interface standard, version 5.0. [link].
OpenSFS and EOFS (2025). Lustre Software Release 2.x: Operations Manual. [link].
Pereira, M. V. F. and Pinto, L. M. V. G. (1991). Multi-stage stochastic optimization applied to energy planning. Mathematical Programming, 52(1–3):359–375.
Petrini, F., Kerbyson, D. J., and Pakin, S. (2003). The case of the missing supercomputer performance: Achieving optimal performance on the 8,192 processors of ASCI Q. In Proceedings of the 2003 ACM/IEEE Conference on Supercomputing, page 55. ACM.
Thakur, R., Gropp, W., and Lusk, E. (1999). On implementing MPI-IO portably and with high performance. Proceedings of the 6th Workshop on I/O in Parallel and Distributed Systems, pages 23–32.
Xie, B., Oral, S., Zimmer, C., Choi, J. Y., Dillow, D., Klasky, S., Lofstead, J. F., Podhorszki, N., and Chase, J. S. (2020). Characterizing output bottlenecks of a production supercomputer: Analysis and implications. ACM Transactions on Storage, 15(4):1–39.
Zakaria, A., Ismail, F. B., Lipu, M. S. H., and Hannan, M. A. (2020). Uncertainty models for stochastic optimization in renewable energy applications. Renewable Energy, 145:1543–1571.
Zhang, Z., Espinosa, A., Iskra, K., Raicu, I., Foster, I., and Wilde, M. (2009). Design and evaluation of a collective IO model for loosely coupled petascale programming. arXiv preprint.
Amazon Web Services (2024b). Elastic network adapter (ENA) — enhanced networking on Linux. Amazon EC2 User Guide.
Amazon Web Services (2024c). Placement groups for Amazon EC2 instances. Amazon EC2 User Guide.
Amdahl, G. M. (1967). Validity of the single processor approach to achieving large scale computing capabilities. In Proceedings of the April 18–20, 1967, Spring Joint Computer Conference, AFIPS ’67 (Spring), pages 483–485, New York, NY, USA. Association for Computing Machinery.
Braam, P. J. (2002). The Lustre storage architecture. Cluster File Systems Inc.
Carns, P., Latham, R., Ross, R., Iskra, K., Lang, S., and Riley, K. (2009). 24/7 characterization of petascale I/O workloads. In Proceedings of the 2009 IEEE International Conference on Cluster Computing and Workshops, pages 1–10. IEEE.
Cirne, W., Brasileiro, F., Sauvé, J., Andrade, N., Paranhos, D., Santos-Neto, E., and Medeiros, R. (2003). Grid computing for bag of tasks applications. In Proceedings of the 3rd IFIP Conference on E-Commerce, E-Business and E-Government (I3E).
Conejo, A. J., Carrión, M., and Morales, J. M. (2010). Decision Making Under Uncertainty in Electricity Markets, volume 153 of International Series in Operations Research & Management Science. Springer, New York, NY.
Dickens, P. M. and Logan, J. (2008). Towards an understanding of the performance of MPI-IO in Lustre file systems. In Proceedings of the IEEE International Conference on Cluster Computing, CLUSTER ’08, pages 254–263. IEEE.
Dickens, P. M. and Logan, J. (2010). A high performance implementation of MPI-IO for a Lustre file system environment. Concurrency and Computation: Practice and Experience, 22(11):1433–1449.
Garcia-Carballeira, F., Camarmas-Alonso, D., Calderon-Mateos, A., and Carretero, J. (2023). A new Ad-Hoc parallel file system for HPC environments based on the Expand parallel file system. In Proceedings of the 22nd International Symposium on Parallel and Distributed Computing, ISPDC ’23, pages 69–76. IEEE.
Graham, R. L., Shipman, G. M., Barrett, B. W., Castain, R. H., Bosilca, G., and Lumsdaine, A. (2005). Open MPI: A high-performance, heterogeneous MPI. In Proceedings of the 5th International Workshop on Algorithms, Models and Tools for Parallel Computing on Heterogeneous Networks.
Gropp, W., Kimpe, D., Ross, R., Thakur, R., and Träff, J. L. (2008). Self-consistent MPIIO performance requirements and expectations. In Proceedings of the 15th European PVM/MPI Users’ Group Meeting, EuroPVM/MPI ’08, pages 167–176. Springer.
Intel Corporation (2023). Intel Xeon scalable processors — 4th generation (Sapphire Rapids). [link].
Kang, Q., Lee, S., Hou, K.-y., Ross, R., Agrawal, A., Choudhary, A., and Liao, W.k. (2020). Improving MPI collective I/O for high volume non-contiguous requests with intra-node aggregation. IEEE Transactions on Parallel and Distributed Systems, 31(11):2682–2695.
Liao, W.-k., Ching, A., Coloma, K., Choudhary, A., and Kandemir, M. (2007). Improving MPI independent write performance using a two-stage write-behind buffering method. In Proceedings of the 21st IEEE International Parallel and Distributed Processing Symposium, IPDPS ’07. IEEE.
Liao, W.-k. and Choudhary, A. (2008). Dynamically adapting file domain partitioning methods for collective I/O based on underlying parallel file system locking protocols. In Proceedings of the ACM/IEEE Conference on Supercomputing, SC ’08. IEEE.
Luu, H., Winslett, M., Gropp, W., Ross, R., Carns, P., Harms, K., Prabhat, Byna, S., and Yao, Y. (2015). A multiplatform study of I/O behavior on petascale supercomputers. In Proceedings of the 24th International Symposium on High-Performance Parallel and Distributed Computing, HPDC ’15, pages 33–44, New York, NY, USA. Association for Computing Machinery.
Message Passing Interface Forum (1997). MPI-2: Extensions to the message-passing interface. Specification.
Message Passing Interface Forum (2023). MPI: A message-passing interface standard, version 4.1. [link].
Message Passing Interface Forum (2025). MPI: A message-passing interface standard, version 5.0. [link].
OpenSFS and EOFS (2025). Lustre Software Release 2.x: Operations Manual. [link].
Pereira, M. V. F. and Pinto, L. M. V. G. (1991). Multi-stage stochastic optimization applied to energy planning. Mathematical Programming, 52(1–3):359–375.
Petrini, F., Kerbyson, D. J., and Pakin, S. (2003). The case of the missing supercomputer performance: Achieving optimal performance on the 8,192 processors of ASCI Q. In Proceedings of the 2003 ACM/IEEE Conference on Supercomputing, page 55. ACM.
Thakur, R., Gropp, W., and Lusk, E. (1999). On implementing MPI-IO portably and with high performance. Proceedings of the 6th Workshop on I/O in Parallel and Distributed Systems, pages 23–32.
Xie, B., Oral, S., Zimmer, C., Choi, J. Y., Dillow, D., Klasky, S., Lofstead, J. F., Podhorszki, N., and Chase, J. S. (2020). Characterizing output bottlenecks of a production supercomputer: Analysis and implications. ACM Transactions on Storage, 15(4):1–39.
Zakaria, A., Ismail, F. B., Lipu, M. S. H., and Hannan, M. A. (2020). Uncertainty models for stochastic optimization in renewable energy applications. Renewable Energy, 145:1543–1571.
Zhang, Z., Espinosa, A., Iskra, K., Raicu, I., Foster, I., and Wilde, M. (2009). Design and evaluation of a collective IO model for loosely coupled petascale programming. arXiv preprint.
Publicado
03/11/2026
Como Citar
CRUZ, Marcelo Barros da; AMORIM, Claudio Luis de; DUTRA, Diego Leonel Cadette.
Escalando a E/S do SDDP em Ambientes HPC: da Escrita Serializada às Escritas Paralelas com MPI-IO sobre Lustre. In: SIMPÓSIO EM SISTEMAS COMPUTACIONAIS DE ALTO DESEMPENHO (SSCAD), 27. , 2026, Natal/RN.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2026
.
p. 15-28.
DOI: https://doi.org/10.5753/sscad.2026.34270.
