Prediction of Reservoir Simulation Jobs Times Using a Real-World SLURM Log

• Alan L. Nunes UFF
• Felipe A. Portella PETROBRAS
• Paulo J. B. Estrela PETROBRAS
• Renzo Q. Malini PETROBRAS
• Bruno Lopes UFF
• Arthur Bittencourt UFF
• Gabriel B. Leite UFF
• Gabriela Coutinho UFF
• Lúcia Maria de Assumpção Drummond UFF

DOI: https://doi.org/10.5753/wscad.2023.235649

Resumo

Modeling petroleum field behavior provides crucial knowledge for risk quantification regarding extraction prospects. Since their processing requires significant computational power and storage capabilities, oil companies run reservoir simulation jobs on high-performance computing clusters. Efficiently using machine learning algorithms in job schedulers to predict the incoming job execution time can increase the effectiveness of cluster resources, such as improving its resource usage rate and reducing the job queue time. This paper introduces a novel and robust predictor, based on SLURM logs from Petrobras, that classifies with more than 74% accuracy the duration time interval of reservoir simulation jobs. The results reveal that our model exceeded the performance of the EASY++ algorithm-based estimator.

Referências

Cheng, F., Huang, Y., Tanpure, B., Sawalani, P., Cheng, L., and Liu, C. (2022). Cost-aware job scheduling for cloud instances using deep reinforcement learning. Cluster Computing, pages 1–13.

Coats, K. H. (1982). Reservoir Simulation: State of the Art. Journal of Petroleum Technology, 34(8):1633–1642.

Gaussier, E., Glesser, D., Reis, V., and Trystram, D. (2015). Improving backfilling by using machine learning to predict running times. In Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis, pages 1–10.

Gaussier, E., Lelong, J., Reis, V., and Trystram, D. (2018). Online Tuning of EASY-Backfilling using Queue Reordering Policies. IEEE Transactions on Parallel and Distributed Systems, 29(10):2304–2316.

Gupta, Y. (2015). Kibana Essentials. Packt Publishing Ltd.

Hall, M., Frank, E., Holmes, G., Pfahringer, B., Reutemann, P., and Witten, I. H. (2009). The WEKA Data Mining Software: An Update. SIGKDD Explor. Newsl., 11(1):10–18.

Kuchnik, M., Park, J. W., Cranor, C., Moore, E., DeBardeleben, N., and Amvrosiadis, G. (2019). This is why ML-driven cluster scheduling remains widely impractical. Technical report, Carnegie Mellon University.

Kuhn, M. and Johnson, K. (2013). Applied Predictive Modeling, volume 26. Springer.

Larose, D. T. and Larose, C. D. (2015). Data Mining and Predictive Analytics. John Wiley & Sons.

Lifka, D. A. (1998). An extensible job scheduling system for massively parallel processor architectures. Illinois Institute of Technology.

Portella, F., Buchaca, D., Rodrigues, J. R., and Berral, J. L. (2022). TunaOil: A tuning algorithm strategy for reservoir simulation workloads. Journal of Computational Science, 63:101811.

Quinlan, J. R. (1993). C4.5: Programs for Machine Learning. Morgan Kaufmann Publishers.

Tanash, M., Dunn, B., Andresen, D., Hsu, W., Yang, H., and Okanlawon, A. (2019). Improving HPC System Performance by Predicting Job Resources via Supervised Machine Learning. In Proceedings of the Practice and Experience in Advanced Research Computing on Rise of the Machines (learning), pages 1–8. Association for Computing Machinery.

Tsafrir, D., Etsion, Y., and Feitelson, D. G. (2007). Backfilling Using System-Generated Predictions Rather than User Runtime Estimates. IEEE Transactions on Parallel and Distributed Systems, 18(6):789–803.

Witt, C., Bux, M., Gusew, W., and Leser, U. (2019). Predictive performance modeling for distributed batch processing using black box monitoring and machine learning. Information Systems, 82:33–52.

Yoo, A. B., Jette, M. A., and Grondona, M. (2003). SLURM: Simple Linux Utility for Resource Management. In Workshop on job scheduling strategies for parallel processing, pages 44–60. Springer.