The Case for Resource Sharing in Scientific Workflow Executions
Resumo
Scientific workflows have become mainstream for conducting largescale scientific research. The execution of these applications can be very costly in terms of computational resources. Therefore, optimizing their resource utilization and efficiency is highly desirable, even in computational environments where the processing resources are plentiful, such as clouds. In this work, we study the case of exploring shared multiprocessors within a single virtual machine. Using a public cloud provider and real-world applications, we show that the use of dedicated processors can lead to sub-optimal performance of scientific workflows. This is a first step towards the creation of a self-aware resource management system inline with the state-of-the-art multitenant platforms.
Referências
(2015). Google Cloud. https://cloud.google.com/. [Accessed 2015-07-31].
(2015). USC Epigenome Center. http://epigenome.usc.edu/. [Accessed 2015-07-31].
Azarnoosh, S., Rynge, M., Juve, G., et al. (2013). Introducing PRECIP: An API for managing repeatable experiments in the cloud. In The 2013 IEEE International Conference on Cloud Computing Technology and Science Volume 02, CLOUDCOM '13, pages 19–26.
Callaghan, S., Maechling, P., Small, P., et al. (2011). Metrics for heterogeneous scientic workows: A case study of an earthquake science application. Int. J. High Perform. Comput. Appl., 25(3):274–285.
Chen, W. and Deelman, E. (2011). Workow overhead analysis and optimizations. In Proceedings of the 6th workshop on Workows in support of large-scale science, pages 11–20. ACM.
Deelman, E., Juve, G., and Berriman, G. B. (2012). Using clouds for science, is it just kicking the can down the road? In The 2nd International Conference on Cloud Computing and Services Science, CLOSER '12, pages 127–134.
Deelman, E., Singh, G., Livny, M., Berriman, B., and Good, J. (2008). The cost of doing science on the cloud: The Montage example. In International Conference for High Performance Computing, Networking, Storage and Analysis, 2008, SC '08, pages 1– 12.
Deelman, E., Vahi, K., Juve, G., et al. (2015). Pegasus, a workow management system for science automation. Future Generation Computer Systems, 46(0):17–35.
Fard, H. M., Prodan, R., Barrionuevo, J. J. D., and Fahringer, T. (2012). A multiobjective approach for workow scheduling in heterogeneous environments. In The 12th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing, CCGRID '12, pages 300–309.
Ferreira da Silva, R., Juve, G., Deelman, E., et al. (2013). Toward ne-grained online task characteristics estimation in scientic workows. In 8th Workshop on Workows in Support of Large-Scale Science, WORKS '13, pages 58–67.
Frey, J. (2002). Condor dagman: Handling inter-job dependencies. University of Wisconsin, Dept. of Computer Science, Tech. Rep.
Giardine, B., Riemer, C., Hardison, R. C., et al. (2005). Galaxy: a platform for interactive large-scale genome analysis. Genome Res., 15(10):1451–1455.
Hoffa, C., Mehta, G., Freeman, T., et al. (2008). On the use of cloud computing for scientic workows. In The IEEE Fourth International Conference on eScience, eScience '08, pages 640–645.
Jacob, J. C., Katz, D. S., Berriman, G. B., et al. (2009). Montage: a grid portal and software toolkit for science-grade astronomical image mosaicking. Int. J. Comput. Sci. Eng., 4(2):73–87.
Juve, G., Chervenak, A., Deelman, E., et al. (2013). Characterizing and proling scientic workows. Future Generation Computer Systems, 29(3):682–692.
Juve, G., Deelman, E., Vahi, K., et al. (2009). Scientic workow applications on Amazon EC2. In The 5th IEEE International Conference on e-Science, e-Science '09, pages 59–66.
Juve, G., Tovar, B., Ferreira da Silva, R., et al. (2015). Practical resource monitoring for robust high throughput computing. In Workshop on Monitoring and Analysis for High Performance Computing Systems Plus Applications, HPCMASPA'15, page to appear.
Ludäscher, B., Altintas, I., Berkley, C., et al. (2006). Scientic workow management and the Kepler system. Concurr. Comput. : Pract. Exper., 18(10):1039–1065.
Malawski, M., Juve, G., Deelman, E., and Nabrzyski, J. (2012). Costand deadlineIn The constrained provisioning for scientic workow ensembles in IaaS clouds. International Conference on High Performance Computing, Networking, Storage and Analysis, SC '12, pages 22:1–22:11.
Taylor, I., Deelman, E., Gannon, D., and Shields, M. (2007). Workows for e-Science. Springer.
Thain, D., Tannenbaum, T., and Livny, M. (2005). Distributed computing in practice: The Condor experience. Concurr. Comput. : Pract. Exper., 17(2-4):323–356.
Vockler, J. S., Mehta, G., Zhao, Y., Deelman, E., and Wilde, M. (2006). Kickstarting In International Workshop on Grid Computing Environments, remote applications. GCE '06.
Wolstencroft, K., Haines, R., Fellows, D., et al. (2013). The Taverna workow suite: designing and executing workows of web services on the desktop, web or in the cloud. Nucleic Acids Research, 41(W1):W557–W561.
(2015). USC Epigenome Center. http://epigenome.usc.edu/. [Accessed 2015-07-31].
Azarnoosh, S., Rynge, M., Juve, G., et al. (2013). Introducing PRECIP: An API for managing repeatable experiments in the cloud. In The 2013 IEEE International Conference on Cloud Computing Technology and Science Volume 02, CLOUDCOM '13, pages 19–26.
Callaghan, S., Maechling, P., Small, P., et al. (2011). Metrics for heterogeneous scientic workows: A case study of an earthquake science application. Int. J. High Perform. Comput. Appl., 25(3):274–285.
Chen, W. and Deelman, E. (2011). Workow overhead analysis and optimizations. In Proceedings of the 6th workshop on Workows in support of large-scale science, pages 11–20. ACM.
Deelman, E., Juve, G., and Berriman, G. B. (2012). Using clouds for science, is it just kicking the can down the road? In The 2nd International Conference on Cloud Computing and Services Science, CLOSER '12, pages 127–134.
Deelman, E., Singh, G., Livny, M., Berriman, B., and Good, J. (2008). The cost of doing science on the cloud: The Montage example. In International Conference for High Performance Computing, Networking, Storage and Analysis, 2008, SC '08, pages 1– 12.
Deelman, E., Vahi, K., Juve, G., et al. (2015). Pegasus, a workow management system for science automation. Future Generation Computer Systems, 46(0):17–35.
Fard, H. M., Prodan, R., Barrionuevo, J. J. D., and Fahringer, T. (2012). A multiobjective approach for workow scheduling in heterogeneous environments. In The 12th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing, CCGRID '12, pages 300–309.
Ferreira da Silva, R., Juve, G., Deelman, E., et al. (2013). Toward ne-grained online task characteristics estimation in scientic workows. In 8th Workshop on Workows in Support of Large-Scale Science, WORKS '13, pages 58–67.
Frey, J. (2002). Condor dagman: Handling inter-job dependencies. University of Wisconsin, Dept. of Computer Science, Tech. Rep.
Giardine, B., Riemer, C., Hardison, R. C., et al. (2005). Galaxy: a platform for interactive large-scale genome analysis. Genome Res., 15(10):1451–1455.
Hoffa, C., Mehta, G., Freeman, T., et al. (2008). On the use of cloud computing for scientic workows. In The IEEE Fourth International Conference on eScience, eScience '08, pages 640–645.
Jacob, J. C., Katz, D. S., Berriman, G. B., et al. (2009). Montage: a grid portal and software toolkit for science-grade astronomical image mosaicking. Int. J. Comput. Sci. Eng., 4(2):73–87.
Juve, G., Chervenak, A., Deelman, E., et al. (2013). Characterizing and proling scientic workows. Future Generation Computer Systems, 29(3):682–692.
Juve, G., Deelman, E., Vahi, K., et al. (2009). Scientic workow applications on Amazon EC2. In The 5th IEEE International Conference on e-Science, e-Science '09, pages 59–66.
Juve, G., Tovar, B., Ferreira da Silva, R., et al. (2015). Practical resource monitoring for robust high throughput computing. In Workshop on Monitoring and Analysis for High Performance Computing Systems Plus Applications, HPCMASPA'15, page to appear.
Ludäscher, B., Altintas, I., Berkley, C., et al. (2006). Scientic workow management and the Kepler system. Concurr. Comput. : Pract. Exper., 18(10):1039–1065.
Malawski, M., Juve, G., Deelman, E., and Nabrzyski, J. (2012). Costand deadlineIn The constrained provisioning for scientic workow ensembles in IaaS clouds. International Conference on High Performance Computing, Networking, Storage and Analysis, SC '12, pages 22:1–22:11.
Taylor, I., Deelman, E., Gannon, D., and Shields, M. (2007). Workows for e-Science. Springer.
Thain, D., Tannenbaum, T., and Livny, M. (2005). Distributed computing in practice: The Condor experience. Concurr. Comput. : Pract. Exper., 17(2-4):323–356.
Vockler, J. S., Mehta, G., Zhao, Y., Deelman, E., and Wilde, M. (2006). Kickstarting In International Workshop on Grid Computing Environments, remote applications. GCE '06.
Wolstencroft, K., Haines, R., Fellows, D., et al. (2013). The Taverna workow suite: designing and executing workows of web services on the desktop, web or in the cloud. Nucleic Acids Research, 41(W1):W557–W561.
Publicado
18/10/2015
Como Citar
ODA, Ricardo; CORDEIRO, Daniel; DA SILVA, Rafael; DEELMAN, Ewa; BRAGHETTO, Kelly.
The Case for Resource Sharing in Scientific Workflow Executions. In: SIMPÓSIO EM SISTEMAS COMPUTACIONAIS DE ALTO DESEMPENHO (SSCAD), 16. , 2015, Florianópolis.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2015
.
p. 168-179.
DOI: https://doi.org/10.5753/wscad.2015.14281.
