GraspCC-LB: Dimensionamento de Recursos para Execução de Workflows em Ambientes de Computação de Alto Desempenho
Resumo
Com a crescente complexidade das simulações computacionais e o aumento do volume de dados processados, a execução de workflows científicos em ambientes HPC torna-se cada vez mais necessária. No entanto, dimensionar a quantidade necessária de recursos para essa execução pode ser uma tarefa desafiadora, uma vez que implica considerar a estrutura do workflow e as características do ambiente. Este artigo apresenta a heurística GraspCC-LB, baseada no procedimento de busca adaptativa randomizada gulosa (GRASP), para o dimensionamento de recursos em ambientes HPC. A GraspCC-LB considera a estrutura do workflow em layers para realizar o dimensionamento, o que a difere das abordagens existentes. A GraspCC-LB foi avaliada utilizando traces reais de workflows das áreas de bioinformática e astronomia, demonstrando resultados promissores.
Referências
Abdi, S., Pourkarimi, L., Ahmadi, M., and Zargari, F. (2018). Cost minimization for bag-of-tasks workflows in a federation of clouds. J. Supercomput., 74(6):2801–2822.
Coutinho, R. et al. (2013). Optimization of a cloud resource management problem from a consumer perspective. In Euro-Par 2013, volume 8374 of LNCS, pages 218–227. Springer.
Coutinho, R. et al. (2015). Optimizing virtual machine allocation for parallel scientific workflows in federated clouds. FGCS, 46:51–68.
Coutinho, R. et al. (2016). A dynamic cloud dimensioning approach for parallel scientific workflows: a case study in the comparative genomics domain. J. Grid Comput., 14(3):443–461.
de Oliveira, D. C. M., Liu, J., and Pacitti, E. (2019). Data-Intensive Workflow Management: For Clouds and Data-Intensive and Scalable Computing Environments. Synthesis Lectures on Data Management. Morgan & Claypool Publishers.
Deelman, E. et al. (2018). The future of scientific workflows. Int. J. High Perform. Comput. Appl., 32(1):159–175.
Deldari, A., Naghibzadeh, M., and Abrishami, S. (2017). Cca: a deadline-constrained workflow scheduling algorithm for multicore resources on the cloud. J. of Supercomp., 73(2):756–781.
Ferreira da Silva, R. et al. (2019). Using simple pid-inspired controllers for online resilient resource management of distributed scientific workflows. FGCS, 95:615–628.
Gil, Y. et al. (2007). On the black art of designing computational workflows. In Proc.s of the WORKS, page 53–62, New York, NY, USA.
Juve, G., Chervenak, A., Deelman, E., Bharathi, S., Mehta, G., and Vahi, K. (2013). Characterizing and profiling scientific workflows. FGCS, 29(3):682–692.
Lin, B., Guo, W., Xiong, N., Chen, G., Vasilakos, A., and Zhang, H. (2016). A pretreatment workflow scheduling approach for big data applications in multicloud environments. IEEE Transactions on Network and Service Management, 13(3):581–594.
Liu, J., Pacitti, E., Valduriez, P., de Oliveira, D., and Mattoso, M. (2016). Multi-objective scheduling of scientific workflows in multisite clouds. FGCS, 63:76–95.
Malawski, M. et al. (2015). Scheduling multilevel deadline-constrained scientific workflows on clouds based on cost optimization. Scientific Programming, 2015:5.
Mohammadi, S., Pedram, H., and PourKarimi, L. (2018). Integer linear programming-based cost optimization for scheduling scientific workflows in multi-cloud environments. The Journal of Supercomputing, 74:4717–4745.
Moschakis, I. and Karatza, H. (2015). Multi-criteria scheduling of bag-of-tasks applications on heterogeneous interlinked clouds with simulated annealing. Journal of Systems and Software, 101:1–14.
Ogasawara, E. S. et al. (2011). An algebraic approach for data-centric scientific workflows. VLDB, 4(12):1328–1339.
Rosa, M. J. et al. (2021). Computational resource and cost prediction service for scientific workflows in federated clouds. FGCS, 125:844–858.
Rynge, M. et al. (2014). Producing an infrared multiwavelength galactic plane atlas using montage, pegasus, and amazon web services. Astronomical Data Analysis Software and Systems XXIII, 485:211.
Song, A., Chen, W.-N., Luo, X., Zhan, Z.-H., and Zhang, J. (2020). Scheduling workflows with composite tasks: A nested particle swarm optimization approach. IEEE Transactions on Services Computing, 15(2):1074–1088.
Zhou, J., Wang, T., Cong, P., Lu, P., Wei, T., and Chen, M. (2019). Cost and makespan-aware workflow scheduling in hybrid clouds. Journal of Systems Architecture, 100:101631.
Coutinho, R. et al. (2013). Optimization of a cloud resource management problem from a consumer perspective. In Euro-Par 2013, volume 8374 of LNCS, pages 218–227. Springer.
Coutinho, R. et al. (2015). Optimizing virtual machine allocation for parallel scientific workflows in federated clouds. FGCS, 46:51–68.
Coutinho, R. et al. (2016). A dynamic cloud dimensioning approach for parallel scientific workflows: a case study in the comparative genomics domain. J. Grid Comput., 14(3):443–461.
de Oliveira, D. C. M., Liu, J., and Pacitti, E. (2019). Data-Intensive Workflow Management: For Clouds and Data-Intensive and Scalable Computing Environments. Synthesis Lectures on Data Management. Morgan & Claypool Publishers.
Deelman, E. et al. (2018). The future of scientific workflows. Int. J. High Perform. Comput. Appl., 32(1):159–175.
Deldari, A., Naghibzadeh, M., and Abrishami, S. (2017). Cca: a deadline-constrained workflow scheduling algorithm for multicore resources on the cloud. J. of Supercomp., 73(2):756–781.
Ferreira da Silva, R. et al. (2019). Using simple pid-inspired controllers for online resilient resource management of distributed scientific workflows. FGCS, 95:615–628.
Gil, Y. et al. (2007). On the black art of designing computational workflows. In Proc.s of the WORKS, page 53–62, New York, NY, USA.
Juve, G., Chervenak, A., Deelman, E., Bharathi, S., Mehta, G., and Vahi, K. (2013). Characterizing and profiling scientific workflows. FGCS, 29(3):682–692.
Lin, B., Guo, W., Xiong, N., Chen, G., Vasilakos, A., and Zhang, H. (2016). A pretreatment workflow scheduling approach for big data applications in multicloud environments. IEEE Transactions on Network and Service Management, 13(3):581–594.
Liu, J., Pacitti, E., Valduriez, P., de Oliveira, D., and Mattoso, M. (2016). Multi-objective scheduling of scientific workflows in multisite clouds. FGCS, 63:76–95.
Malawski, M. et al. (2015). Scheduling multilevel deadline-constrained scientific workflows on clouds based on cost optimization. Scientific Programming, 2015:5.
Mohammadi, S., Pedram, H., and PourKarimi, L. (2018). Integer linear programming-based cost optimization for scheduling scientific workflows in multi-cloud environments. The Journal of Supercomputing, 74:4717–4745.
Moschakis, I. and Karatza, H. (2015). Multi-criteria scheduling of bag-of-tasks applications on heterogeneous interlinked clouds with simulated annealing. Journal of Systems and Software, 101:1–14.
Ogasawara, E. S. et al. (2011). An algebraic approach for data-centric scientific workflows. VLDB, 4(12):1328–1339.
Rosa, M. J. et al. (2021). Computational resource and cost prediction service for scientific workflows in federated clouds. FGCS, 125:844–858.
Rynge, M. et al. (2014). Producing an infrared multiwavelength galactic plane atlas using montage, pegasus, and amazon web services. Astronomical Data Analysis Software and Systems XXIII, 485:211.
Song, A., Chen, W.-N., Luo, X., Zhan, Z.-H., and Zhang, J. (2020). Scheduling workflows with composite tasks: A nested particle swarm optimization approach. IEEE Transactions on Services Computing, 15(2):1074–1088.
Zhou, J., Wang, T., Cong, P., Lu, P., Wei, T., and Chen, M. (2019). Cost and makespan-aware workflow scheduling in hybrid clouds. Journal of Systems Architecture, 100:101631.
Publicado
17/10/2023
Como Citar
ALVARENGA, Luis Carlos Ramos; FROTA, Yuri; OLIVEIRA, Daniel de; COUTINHO, Rafaelli.
GraspCC-LB: Dimensionamento de Recursos para Execução de Workflows em Ambientes de Computação de Alto Desempenho. In: SIMPÓSIO EM SISTEMAS COMPUTACIONAIS DE ALTO DESEMPENHO (SSCAD), 24. , 2023, Porto Alegre/RS.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2023
.
p. 145-156.
DOI: https://doi.org/10.5753/wscad.2023.235644.
