Apoio à Utilização de Análise de Dados em Aplicações CSE por meio de Contêineres
Abstract
Large-scale scientific applications are characterized by invoking several software libraries and producing large amounts of data through scripts. Containerbased virtualization is a way that is not widely adopted by the HPC community to support the portability of applications, especially approaches for data analysis. To assist the adoption of containers, this article presents ProvDeploy with a model aimed at incorporating services into applications developed for PAD environments.
Keywords:
Aplicações em Agricultura, Biologia, Engenharia, Física, Matemática, Medicina, Mercado Financeiro, Nanociências, Óleo e Gás, Química e outras áreas.
References
Davidson, S. B. and Freire, J. (2008). Provenance and scientific workflows: challenges and opportunities. In Proceedings of the 2008 ACM SIGMOD international conference on Management of data, pages 1345–1350.
Gerhardt, L., Bhimji, W., Fasel, M., Porter, J., Mustafa, M., Jacobsen, D., Tsulaia, V., and Canon, S. (2017). Shifter: Containers for hpc. In J. Phys. Conf. Ser., volume 898, page 082021.
Kurtzer, G. M., Sochat, V., and Bauer, M. W. (2017). Singularity: Scientific containers for mobility of compute. PloS one, 12(5).
McMillan, S. (2018). Making containers easier with hpc container maker. In In HPCSYSPROS18: HPC System Professionals Workshop, Dallas, TX.
Neves, L. (2020). Provdeploy: Apoio a coleta de dados de proveniência em scripts de execução de codigos científicos
Preeth, E., Mulerickal, F. J. P., Paul, B., and Sastri, Y. (2015). Evaluation of docker containers based on hardware utilization. In 2015 International Conference on Control Communication & Computing India (ICCC), pages 697–700. IEEE.
Priedhorsky, R. and Randles, T. (2017). Charliecloud: Unprivileged containers for userdefined software stacks in hpc. In Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis, pages 1–10.
Vahi, K., Rynge, M., Papadimitriou, G., Brown, D. A., Mayani, R., da Silva, R. F., Deelman, E., Mandal, A., Lyons, E., and Zink, M. (2019). Custom execution environments with containers in pegasus-enabled scientific workflows. arXiv preprint arXiv:1905.08204.
Wu, K., Ahern, S., Bethel, E. W., Chen, J., Childs, H., Cormier-Michel, E., Geddes, C., Gu, J., Hagen, H., Hamann, B., et al. (2009). Fastbit: interactively searching massive data. In Journal of Physics: Conference Series, volume 180, page 012053. IOP Publishing.
Gerhardt, L., Bhimji, W., Fasel, M., Porter, J., Mustafa, M., Jacobsen, D., Tsulaia, V., and Canon, S. (2017). Shifter: Containers for hpc. In J. Phys. Conf. Ser., volume 898, page 082021.
Kurtzer, G. M., Sochat, V., and Bauer, M. W. (2017). Singularity: Scientific containers for mobility of compute. PloS one, 12(5).
McMillan, S. (2018). Making containers easier with hpc container maker. In In HPCSYSPROS18: HPC System Professionals Workshop, Dallas, TX.
Neves, L. (2020). Provdeploy: Apoio a coleta de dados de proveniência em scripts de execução de codigos científicos
Preeth, E., Mulerickal, F. J. P., Paul, B., and Sastri, Y. (2015). Evaluation of docker containers based on hardware utilization. In 2015 International Conference on Control Communication & Computing India (ICCC), pages 697–700. IEEE.
Priedhorsky, R. and Randles, T. (2017). Charliecloud: Unprivileged containers for userdefined software stacks in hpc. In Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis, pages 1–10.
Vahi, K., Rynge, M., Papadimitriou, G., Brown, D. A., Mayani, R., da Silva, R. F., Deelman, E., Mandal, A., Lyons, E., and Zink, M. (2019). Custom execution environments with containers in pegasus-enabled scientific workflows. arXiv preprint arXiv:1905.08204.
Wu, K., Ahern, S., Bethel, E. W., Chen, J., Childs, H., Cormier-Michel, E., Geddes, C., Gu, J., Hagen, H., Hamann, B., et al. (2009). Fastbit: interactively searching massive data. In Journal of Physics: Conference Series, volume 180, page 012053. IOP Publishing.
Published
2020-11-30
How to Cite
NEVES, Liliane; PINA, Débora ; DE OLIVEIRA, Daniel ; MATTOSO, Marta .
Apoio à Utilização de Análise de Dados em Aplicações CSE por meio de Contêineres. In: REGIONAL SCHOOL OF HIGH PERFORMANCE COMPUTING FROM RIO DE JANEIRO (ERAD-RJ), 6. , 2020, Evento Online.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2020
.
p. 26-30.
DOI: https://doi.org/10.5753/eradrj.2020.14512.
