Análise de um Log Real de Jobs visando a Previsão de Tempos de Espera para Execução em um Cluster de Alto Desempenho
Resumo
A exploração de petróleo em águas profundas depende de simulações numéricas complexas, executadas em supercomputadores, para otimizar a produção e projetar cenários futuros. Na Petrobras, líder mundial em exploração de petróleo em águas profundas, essa infraestrutura inclui diversos sistemas classificados no TOP500. Contudo, apesar da robusta capacidade computacional, a natureza das operações gera momentos de pico de demanda. Para criar melhores estratégias para lidar com esses picos, este trabalho analisa o comportamento do tempo de espera na fila, concluindo que as métricas de carga de trabalho por usuário — como a soma do tempo de execução e de CPUs de jobs já em espera — são os fatores com a maior correlação.Referências
Brown, N., Gibb, G., Belikov, E., and Nash, R. (2024). Predicting accurate batch queue wait times on production supercomputers by combining machine learning techniques. Concurrency and Computation: Practice and Experience, 36(15):e8112.
Kendall, M. G. (1938). A new measure of rank correlation. Biometrika, 30(1/2):81–93.
Lovell, A., Wisniewski, P., Rodenbeck, S., and Ashish (2024). A hierarchical deep learning approach for predicting job queue times in hpc systems. In SC24-W: Workshops of the International Conference for High Performance Computing, Networking, Storage and Analysis, pages 621–628.
Menear, K., Konate, K., Potter, K., and Duplyakin, D. (2024). Tandem predictions for hpc jobs. In Practice and Experience in Advanced Research Computing 2024: Human Powered Computing, PEARC ’24, New York, NY, USA. Association for Computing Machinery.
Paokin, A. V. and Nikitenko, D. A. (2023). Approbation of methods for supercomputer job queue wait time estimation. Lobachevskii Journal of Mathematics, 44(8):3140–3147.
Pearson, K. (1895). Note on regression and inheritance in the case of two parents. Proceedings of the Royal Society of London, 58:240–242.
Ramachandran, S., Jayalal, M., Vasudevan, M., and Jehadeesan, R. (2024). Combining machine learning & metaheuristic algorithms for predicting waiting time of high performance computing jobs. In 2024 5th International Conference on Innovative Trends in Information Technology (ICITIIT), pages 1–6.
Spearman, C. (1904). The proof and measurement of association between two things. The American Journal of Psychology, 15(1):72–101.
TOP500.org (2025). Top500 list. [link]. Acessado em: 10 de setembro de 2025.
Yoo, A. B., Jette, M. A., and Grondona, M. (2003). Slurm: Simple linux utility for resource management. In Feitelson, D., Rudolph, L., and Schwiegelshohn, U., editors, Job Scheduling Strategies for Parallel Processing, pages 44–60, Berlin, Heidelberg. Springer Berlin Heidelberg.
Kendall, M. G. (1938). A new measure of rank correlation. Biometrika, 30(1/2):81–93.
Lovell, A., Wisniewski, P., Rodenbeck, S., and Ashish (2024). A hierarchical deep learning approach for predicting job queue times in hpc systems. In SC24-W: Workshops of the International Conference for High Performance Computing, Networking, Storage and Analysis, pages 621–628.
Menear, K., Konate, K., Potter, K., and Duplyakin, D. (2024). Tandem predictions for hpc jobs. In Practice and Experience in Advanced Research Computing 2024: Human Powered Computing, PEARC ’24, New York, NY, USA. Association for Computing Machinery.
Paokin, A. V. and Nikitenko, D. A. (2023). Approbation of methods for supercomputer job queue wait time estimation. Lobachevskii Journal of Mathematics, 44(8):3140–3147.
Pearson, K. (1895). Note on regression and inheritance in the case of two parents. Proceedings of the Royal Society of London, 58:240–242.
Ramachandran, S., Jayalal, M., Vasudevan, M., and Jehadeesan, R. (2024). Combining machine learning & metaheuristic algorithms for predicting waiting time of high performance computing jobs. In 2024 5th International Conference on Innovative Trends in Information Technology (ICITIIT), pages 1–6.
Spearman, C. (1904). The proof and measurement of association between two things. The American Journal of Psychology, 15(1):72–101.
TOP500.org (2025). Top500 list. [link]. Acessado em: 10 de setembro de 2025.
Yoo, A. B., Jette, M. A., and Grondona, M. (2003). Slurm: Simple linux utility for resource management. In Feitelson, D., Rudolph, L., and Schwiegelshohn, U., editors, Job Scheduling Strategies for Parallel Processing, pages 44–60, Berlin, Heidelberg. Springer Berlin Heidelberg.
Publicado
05/11/2025
Como Citar
GALLO, Bernardo; MAROTTI, Matheus; DRUMMOND, Lúcia Maria de Assumpção; VITERBO, José; PORTELLA, Felipe A.; ESTRELA, Paulo J. B.; MALINI, Renzo Q..
Análise de um Log Real de Jobs visando a Previsão de Tempos de Espera para Execução em um Cluster de Alto Desempenho. In: ESCOLA REGIONAL DE ALTO DESEMPENHO DA REGIÃO SUDESTE (ERAD-SE), 10. , 2025, Niterói/RJ.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2025
.
p. 11-15.
DOI: https://doi.org/10.5753/eradse.2025.16956.
