Uma Arquitetura P2P de Distribuição de Atividades para Execução Paralela de Workflows Científicos
Abstract
Scientific workflows are composed of activities that model scientific experiments. Many Scientific Workflow Management Systems use High Performance Computing environments to parallelize the execution of these activities in large-scale workflows. Data distribution, control, and optimizing the parallel execution of these activities can be a complex task due to scalability of involved resources. This paper presents DEW, a data and activity distribution mechanism for a parallel workflow execution engine. DEW is based on a hierarchical P2P network that enables distributed control in workflow execution using distributed disk and in the presence of high occurrence of churn events.References
Apache Software Foundation, (2009), "Hadoop", Apache Hadoop Website Chandra, R., (2001), Parallel programming in OpenMP. Morgan Kaufmann.
Costa, F., Silva, V., Oliveira, D., Ocaña, K., Dias, J., Ogasawara, E., Mattoso, M., (2013), "Capturing and Querying Workflow Runtime Provenance with PROV: a Practical Approach". In: Proc. of the International Workshop on Managing and Querying Provenance Data at Scale (BigProv’13), Genova, Italy.
Dias, J., Ogasawara, E., de Oliveira, D., Pacitti, E., Mattoso, M., (2010a), "Improving Many-Task computing in scientific workflows using P2P techniques". In: 1801 Proceedings of the 3rd IEEE Workshop on Many-Task Computing on Grids and Supercomputers, p. 1–10, New Orleans, Louisiana, USA.
Dias, J., Rodrigues, C., Ogasawara, E., Oliveira, D., Braganholo, V., Pacitti, E., Mattoso, M., (2010b), "SciMulator: Um Ambiente de Simulação de Workflows Científicos em Redes P2P". In: Workshop P2P 2010, p. 45–56, Gramado, Rio Grande do Sul Brazil.
Freire, J., Koop, D., Santos, E., Silva, C. T., (2008), "Provenance for Computational Tasks: A Survey", Computing in Science and Engineering, v.10, n. 3, p. 11–21.
Gabriel, E., Fagg, G. E., Bosilca, G., Angskun, T., Dongarra, J. J., Squyres, J. M., Sahay, V., Kambadur, P., Barrett, B., et al., (2004), "Open MPI: Goals, Concept, and Design of a Next Generation MPI Implementation", Recent Advances in Parallel Virtual Machine and Message Passing Interface, , p. 353–377.
Mattoso, A., Silva, F., Ruberg, N., Cruz, M., (2008), "Gerência de Workflows Científicos: Uma Análise Crítica no Contexto da Bioinformática", COPPE/UFRJ, n. Relatório técnico Ogasawara, E., Dias, J., Oliveira, D., Porto, F., Valduriez, P., Mattoso, M., (2011), "An Algebraic Approach for Data-Centric Scientific Workflows", Proc. of VLDB Endowment, v. 4, n. 12, p. 1328–1339.
Ogasawara, E., Dias, J., Oliveira, D., Rodrigues, C., Pivotto, C., Antas, R., Braganholo, V., Valduriez, P., Mattoso, M., (2010), "A P2P approach to many tasks computing for scientific workflows". In: VECPAR’10, p. 327–339, Berlin, Heidelberg.
Ogasawara, E., Dias, J., Silva, V., Chirigati, F., Oliveira, D., Porto, F., Valduriez, P., Mattoso, M., (2013), "Chiron: A Parallel Engine for Algebraic Scientific Workflows", Concurrency and Computation Papuzzo, G., Spezzano, G., (2011), "Autonomic management of workflows on hybrid grid-cloud infrastructure", CNSM ’11 Proceedings of the 7th International Conference on Network and Services Management.
Rahman, M., Ranjan, R., Buyya, R., (2010), "Cooperative and decentralized workflow scheduling in global grids", Future Generation Computer Systems, v. 26, n. 5 (May.), p. 753–768.
Raicu, I., Foster, I. T., Yong Zhao, (2008), "Many-task computing for grids and supercomputers". In: Proceedings of the Workshop on Many-Task Computing on Grids and Supercomputers, p. 1–11, Austin, Texas, USA.
Taylor, I., Shields, M., Wang, I., Harrison, A., (2007), "The Triana Workflow for e-Science, Environment: Architecture and Applications", Workflows Springer, p. 320–339. Wu, D., Tian, Y., Ng, K.-W., Datta, A., (2008), "Stochastic analysis of the interplay between object maintenance and churn", Computer Communications, v. 31, n. 2 (Feb.), p. 220–239.
Costa, F., Silva, V., Oliveira, D., Ocaña, K., Dias, J., Ogasawara, E., Mattoso, M., (2013), "Capturing and Querying Workflow Runtime Provenance with PROV: a Practical Approach". In: Proc. of the International Workshop on Managing and Querying Provenance Data at Scale (BigProv’13), Genova, Italy.
Dias, J., Ogasawara, E., de Oliveira, D., Pacitti, E., Mattoso, M., (2010a), "Improving Many-Task computing in scientific workflows using P2P techniques". In: 1801 Proceedings of the 3rd IEEE Workshop on Many-Task Computing on Grids and Supercomputers, p. 1–10, New Orleans, Louisiana, USA.
Dias, J., Rodrigues, C., Ogasawara, E., Oliveira, D., Braganholo, V., Pacitti, E., Mattoso, M., (2010b), "SciMulator: Um Ambiente de Simulação de Workflows Científicos em Redes P2P". In: Workshop P2P 2010, p. 45–56, Gramado, Rio Grande do Sul Brazil.
Freire, J., Koop, D., Santos, E., Silva, C. T., (2008), "Provenance for Computational Tasks: A Survey", Computing in Science and Engineering, v.10, n. 3, p. 11–21.
Gabriel, E., Fagg, G. E., Bosilca, G., Angskun, T., Dongarra, J. J., Squyres, J. M., Sahay, V., Kambadur, P., Barrett, B., et al., (2004), "Open MPI: Goals, Concept, and Design of a Next Generation MPI Implementation", Recent Advances in Parallel Virtual Machine and Message Passing Interface, , p. 353–377.
Mattoso, A., Silva, F., Ruberg, N., Cruz, M., (2008), "Gerência de Workflows Científicos: Uma Análise Crítica no Contexto da Bioinformática", COPPE/UFRJ, n. Relatório técnico Ogasawara, E., Dias, J., Oliveira, D., Porto, F., Valduriez, P., Mattoso, M., (2011), "An Algebraic Approach for Data-Centric Scientific Workflows", Proc. of VLDB Endowment, v. 4, n. 12, p. 1328–1339.
Ogasawara, E., Dias, J., Oliveira, D., Rodrigues, C., Pivotto, C., Antas, R., Braganholo, V., Valduriez, P., Mattoso, M., (2010), "A P2P approach to many tasks computing for scientific workflows". In: VECPAR’10, p. 327–339, Berlin, Heidelberg.
Ogasawara, E., Dias, J., Silva, V., Chirigati, F., Oliveira, D., Porto, F., Valduriez, P., Mattoso, M., (2013), "Chiron: A Parallel Engine for Algebraic Scientific Workflows", Concurrency and Computation Papuzzo, G., Spezzano, G., (2011), "Autonomic management of workflows on hybrid grid-cloud infrastructure", CNSM ’11 Proceedings of the 7th International Conference on Network and Services Management.
Rahman, M., Ranjan, R., Buyya, R., (2010), "Cooperative and decentralized workflow scheduling in global grids", Future Generation Computer Systems, v. 26, n. 5 (May.), p. 753–768.
Raicu, I., Foster, I. T., Yong Zhao, (2008), "Many-task computing for grids and supercomputers". In: Proceedings of the Workshop on Many-Task Computing on Grids and Supercomputers, p. 1–11, Austin, Texas, USA.
Taylor, I., Shields, M., Wang, I., Harrison, A., (2007), "The Triana Workflow for e-Science, Environment: Architecture and Applications", Workflows Springer, p. 320–339. Wu, D., Tian, Y., Ng, K.-W., Datta, A., (2008), "Stochastic analysis of the interplay between object maintenance and churn", Computer Communications, v. 31, n. 2 (Feb.), p. 220–239.
Published
2013-07-23
How to Cite
SILVA, Vítor; DIAS, Jonas; OLIVEIRA, Daniel de; OGASAWARA, Eduardo; MATTOSO, Marta.
Uma Arquitetura P2P de Distribuição de Atividades para Execução Paralela de Workflows Científicos. In: BRAZILIAN E-SCIENCE WORKSHOP (BRESCI), 7. , 2013, Maceió.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2013
.
p. 1795-1802.
ISSN 2763-8774.
