Nebulous: A Framework for Scientific Applications Execution on Cloud Environments
Resumo
This paper presents a framework, called Nebulous, designed to simplify the execution of MPI and OpenMP parallel applications in computational clouds. The framework aims to automate the process of deployment and execution, avoiding the direct user interaction with the cloud. The framework is built on a cloud middleware layer (e.g. OpenNebula, Eucalyptus or Nimbus) and consists of three components: the Resource Description Block, an Application Programming Interface and the Executor. According to the results, Nebulous allows users to deploy applications over varying numbers of nodes in a simple way, uniformly, scalably and with minimum effort.
Referências
C. Bunch, N. Chohan, C. Krintz, and K. Shams. Neptune: a domain specific language for deploying hpc software on cloud platforms. In Proceedings of the 2nd international workshop on Scientific cloud computing, ScienceCloud ’11, pages 59–68, New York, NY, USA, 2011. ACM.
M. Cera, Y. Georgiou, O. Richard, N. Maillard, and P. Navaux. Supporting malleability in parallel architectures with dynamic cpu sets mapping and dynamic mpi. In K. Kant, S. Pemmaraju, K. Sivalingam, and J. Wu, editors, Distributed Computing and Networking, volume 5935 of Lecture Notes in Computer Science, pages 242–257. Springer Berlin / Heidelberg, 2010.
B. Chapman, G. Jost, and R. van der Pas. Using OpenMP: Portable Shared Memory Parallel Programming (Scientific and Engineering Computation). The MIT Press, 2007.
D. de Oliveira and E. Ogasawara. Is cloud computing the solution for brazilian researchers? International Journal of Computer Applications, 6(8):19–23, September 2010. Published By Foundation of Computer Science.
E. Deelman, M. Livny, G. Mehta, and A. Pavlo. Pegasus and dagman from concept to execution: Mapping scientific workflows onto todays cyberinfrastructure. IOS, 2008.
T. Desell, K. E. Maghraoui, and C. A. Varela. Malleable applications for scalable high performance computing. Cluster Computing, 10:323–337, September 2007.
T. Deshane, Z. Shepherd, J. Matthews, M. Ben-Yehuda, A. Shah, and B. Rao. Quantitative comparison of Xen and KVM. In Xen summit, Berkeley, CA, USA, June 2008. USENIX association.
K. El Maghraoui, T. J. Desell, B. K. Szymanski, and C. A. Varela. Malleable iterative mpi applications. Concurrency and Computation: Practice and Experience, 21(3):393–413, 2009.
C. Evangelinos and C. N. Hill. Cloud computing for parallel scientific hpc applications: Feasibility of running coupled atmosphere-ocean climate models on amazonas ec2 . October, 2(2.40):2–34, 2008.
E. Gabriel, G. E. Fagg, G. Bosilca, T. Angskun, J. J. Dongarra, J. M. Squyres, V. Sahay, P. Kambadur, B. Barrett, A. Lumsdaine, R. H. Castain, D. J. Daniel, R. L. Graham, and T. S. Woodall. Open MPI: Goals, concept, and design of a next generation MPI implementation. In Proceedings, 11th European PVM/MPI Users’ Group Meeting, pages 97–104, Budapest, Hungary, September 2004.
W. Gropp, E. Lusk, and R. Thakur. Using MPI-2: Advanced Features of the Message-Passing Interface. MIT Press, Cambridge, MA, USA, 1999.
C. Hoffa, G. Mehta, T. Freeman, E. Deelman, K. Keahey, B. Berriman, and J. Good. On the use of cloud computing for scientific workflows. In Proceedings of the 2008 Fourth IEEE International Conference on eScience, pages 640–645, Washington, DC, USA, 2008. IEEE.
K. R. Jackson, L. Ramakrishnan, K. Muriki, S. Canon, S. Cholia, J. Shalf, H. J. Wasserman, and N. J. Wright. Performance analysis of high performance computing applications on the amazon web services cloud. In CloudCom’10, pages 159–168, 2010.
G. Juve and E. Deelman. Scientific workflows and clouds. ACM Crossroads, pages 14–18, 2010.
J. H. Lienhard and J. H. Lienhard. A Heat Transfer Textbook - 3rd ed. Phlogiston Press: Cambridge, Massachusetts, 2008.
B. Ludäscher, I. Altintas, C. Berkley, D. Higgins, E. Jaeger, M. Jones, E. A. Lee, J. Tao, and Y. Zhao. Scientific workflow management and the kepler system: Research articles. Concurr. Comput. : Pract. Exper., 18:1039–1065, August 2006.
J. Mirkovic, T. Faber., P. Hsieh, G. Malayandisamu, and R. Malavia. Dadl: Distributed application description language. Technical Report ISI-TR-664, USC/ISI, 2010.
N. Project. http://www.nimbusproject.org/.
P. Sempolinski and D. Thain. A comparison and critique of eucalyptus, opennebula and nimbus. 2010 IEEE Second International Conference on Cloud Computing Technology and Science, November:417–426, 2010.
Y. Simmhan, C. Van Ingen, G. Subramanian, and J. Li. Bridging the gap between desktop and the cloud for escience applications. 2010 IEEE 3rd International Conference on Cloud Computing, pages 474–481, 2010.
B. Sotomayor, R. S. Montero, I. M. Llorente, and I. Foster. Virtual infrastructure management in private and hybrid clouds. IEEE Internet Computing, 13:14–22, 2009.
H.-L. Truong and S. Dustdar. Cloud computing for small research groups in computational science and engineering: current status and outlook. Computing, 91:75–91, January 2011.
J.-S. Vöckler, G. Juve, E. Deelman, M. Rynge, and B. Berriman. Experiences using cloud computing for a scientific workflow application. In Proceedings of the 2nd international workshop on Scientific cloud computing, ScienceCloud ’11, pages 15–24, New York, NY, USA, 2011. ACM.
L. Wang, J. Tao, M. Kunze, A. C. Castellanos, D. Kramer, and W. Karl. Scientific cloud computing: Early definition and experience. In Proceedings of the 2008 10th IEEE International Conference on High Performance Computing and Communications, HPCC ’08, pages 825–830, Washington, DC, USA, 2008. IEEE Computer Society.
M. Cera, Y. Georgiou, O. Richard, N. Maillard, and P. Navaux. Supporting malleability in parallel architectures with dynamic cpu sets mapping and dynamic mpi. In K. Kant, S. Pemmaraju, K. Sivalingam, and J. Wu, editors, Distributed Computing and Networking, volume 5935 of Lecture Notes in Computer Science, pages 242–257. Springer Berlin / Heidelberg, 2010.
B. Chapman, G. Jost, and R. van der Pas. Using OpenMP: Portable Shared Memory Parallel Programming (Scientific and Engineering Computation). The MIT Press, 2007.
D. de Oliveira and E. Ogasawara. Is cloud computing the solution for brazilian researchers? International Journal of Computer Applications, 6(8):19–23, September 2010. Published By Foundation of Computer Science.
E. Deelman, M. Livny, G. Mehta, and A. Pavlo. Pegasus and dagman from concept to execution: Mapping scientific workflows onto todays cyberinfrastructure. IOS, 2008.
T. Desell, K. E. Maghraoui, and C. A. Varela. Malleable applications for scalable high performance computing. Cluster Computing, 10:323–337, September 2007.
T. Deshane, Z. Shepherd, J. Matthews, M. Ben-Yehuda, A. Shah, and B. Rao. Quantitative comparison of Xen and KVM. In Xen summit, Berkeley, CA, USA, June 2008. USENIX association.
K. El Maghraoui, T. J. Desell, B. K. Szymanski, and C. A. Varela. Malleable iterative mpi applications. Concurrency and Computation: Practice and Experience, 21(3):393–413, 2009.
C. Evangelinos and C. N. Hill. Cloud computing for parallel scientific hpc applications: Feasibility of running coupled atmosphere-ocean climate models on amazonas ec2 . October, 2(2.40):2–34, 2008.
E. Gabriel, G. E. Fagg, G. Bosilca, T. Angskun, J. J. Dongarra, J. M. Squyres, V. Sahay, P. Kambadur, B. Barrett, A. Lumsdaine, R. H. Castain, D. J. Daniel, R. L. Graham, and T. S. Woodall. Open MPI: Goals, concept, and design of a next generation MPI implementation. In Proceedings, 11th European PVM/MPI Users’ Group Meeting, pages 97–104, Budapest, Hungary, September 2004.
W. Gropp, E. Lusk, and R. Thakur. Using MPI-2: Advanced Features of the Message-Passing Interface. MIT Press, Cambridge, MA, USA, 1999.
C. Hoffa, G. Mehta, T. Freeman, E. Deelman, K. Keahey, B. Berriman, and J. Good. On the use of cloud computing for scientific workflows. In Proceedings of the 2008 Fourth IEEE International Conference on eScience, pages 640–645, Washington, DC, USA, 2008. IEEE.
K. R. Jackson, L. Ramakrishnan, K. Muriki, S. Canon, S. Cholia, J. Shalf, H. J. Wasserman, and N. J. Wright. Performance analysis of high performance computing applications on the amazon web services cloud. In CloudCom’10, pages 159–168, 2010.
G. Juve and E. Deelman. Scientific workflows and clouds. ACM Crossroads, pages 14–18, 2010.
J. H. Lienhard and J. H. Lienhard. A Heat Transfer Textbook - 3rd ed. Phlogiston Press: Cambridge, Massachusetts, 2008.
B. Ludäscher, I. Altintas, C. Berkley, D. Higgins, E. Jaeger, M. Jones, E. A. Lee, J. Tao, and Y. Zhao. Scientific workflow management and the kepler system: Research articles. Concurr. Comput. : Pract. Exper., 18:1039–1065, August 2006.
J. Mirkovic, T. Faber., P. Hsieh, G. Malayandisamu, and R. Malavia. Dadl: Distributed application description language. Technical Report ISI-TR-664, USC/ISI, 2010.
N. Project. http://www.nimbusproject.org/.
P. Sempolinski and D. Thain. A comparison and critique of eucalyptus, opennebula and nimbus. 2010 IEEE Second International Conference on Cloud Computing Technology and Science, November:417–426, 2010.
Y. Simmhan, C. Van Ingen, G. Subramanian, and J. Li. Bridging the gap between desktop and the cloud for escience applications. 2010 IEEE 3rd International Conference on Cloud Computing, pages 474–481, 2010.
B. Sotomayor, R. S. Montero, I. M. Llorente, and I. Foster. Virtual infrastructure management in private and hybrid clouds. IEEE Internet Computing, 13:14–22, 2009.
H.-L. Truong and S. Dustdar. Cloud computing for small research groups in computational science and engineering: current status and outlook. Computing, 91:75–91, January 2011.
J.-S. Vöckler, G. Juve, E. Deelman, M. Rynge, and B. Berriman. Experiences using cloud computing for a scientific workflow application. In Proceedings of the 2nd international workshop on Scientific cloud computing, ScienceCloud ’11, pages 15–24, New York, NY, USA, 2011. ACM.
L. Wang, J. Tao, M. Kunze, A. C. Castellanos, D. Kramer, and W. Karl. Scientific cloud computing: Early definition and experience. In Proceedings of the 2008 10th IEEE International Conference on High Performance Computing and Communications, HPCC ’08, pages 825–830, Washington, DC, USA, 2008. IEEE Computer Society.
Publicado
26/10/2011
Como Citar
GALANTE, Guilherme; BONA, Luis Carlos E. de.
Nebulous: A Framework for Scientific Applications Execution on Cloud Environments. In: SIMPÓSIO EM SISTEMAS COMPUTACIONAIS DE ALTO DESEMPENHO (SSCAD), 12. , 2011, Vitória.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2011
.
p. 41-48.
DOI: https://doi.org/10.5753/wscad.2011.17266.
