MapReduce na AWS: Uma Análise de Custos Computacionais Utilizando os Serviços FaaS e IaaS
Resumo
Serviços da nuvem oferecem facilidades computacionais bem estabelecidas. Na busca de eficiência para executar aplicações do tipo MapReduce, que lidam com grandes volumes de dados, baixos custos monetários também são almejados. Para delinear os benefícios de diferentes serviços cloud, realizamos uma análise exploratória dos tempos e custos para a execução de uma aplicação MapReduce na nuvem pública da Amazon, a AWS. A partir de implementações com os frameworks Spark e MARLA sob os serviços EC2 e Lambda, respectivamente, apresentamos os impactos associados às quantidades e tipos de recursos escolhidos. Os resultados sugeriram o ambiente MARLA Lambda como o mais rápido e o Spark EC2 como o mais econômico.
Referências
Awaysheh, F. M., Alazab, M., Garg, S., Niyato, D., and Verikoukis, C. (2021). Big data resource management & networks: Taxonomy, survey, and future directions. IEEE Communications Surveys & Tutorials, 23(4):2098-2130.
Dean, J. and Ghemawat, S. (2004). MapReduce: Simplified Data Processing on Large Clusters. In Proceedings of the 6th Symposium on Operating Systems Design and Implementation, OSDI' 04, pages 137-149. USENIX Association.
Giménez-Alventosa, V., Moltó, G., and Caballer, M. (2019). A framework and a performance assessment for serverless MapReduce on AWS Lambda. Future Generation Computer Systems, 97:259-274.
Kapil, D., Mishra, S., and Gupta, V. (2022). A performance perspective of live migration of virtual machine in cloud data center with future directions. International Journal of Wireless and Microwave Technologies, 12:48-56.
Kim, Y. and Lin, J. (2018). Serverless Data Analytics with Flint. In IEEE 11th International Conference on Cloud Computing (CLOUD), pages 451-455, Los Alamitos, CA, USA. IEEE Computer Society.
Malawski, M., Gajek, A., Zima, A., Balis, B., and Figiela, K. (2020). Serverless execution of scientific workflows: Experiments with hyperflow, aws lambda and google cloud functions. Future Generation Computer Systems, 110:502-514.
Muniswamaiah, M., Agerwala, T., and Tappert, C. (2019). Big data in cloud computing review and opportunities. International Journal of Computer Science & Information Technology (IJCSIT), 11.
Nunes, A. L., Melo, A., Boeres, C., de Oliveira, D., and Drummond, L. M. A. (2021). Towards Analyzing Computational Costs of Spark for SARS-CoV-2 Sequences Comparisons on a Commercial Cloud. XXII Symposium in High Performance Computing Systems, pages 192-203.
Shvachko, K., Kuang, H., Radia, S., and Chansler, R. (2010). The Hadoop Distributed File System. In Proceedings of the 26th IEEE Symposium on Mass Storage Systems and Technologies (MSST), pages 1-10. IEEE.
Zaharia, M., Chowdhury, M., Das, T., Dave, A., Ma, J., McCauley, M., Franklin, M. J., Shenker, S., and Stoica, I. (2012). Resilient Distributed Datasets: A Fault-Tolerant Abstraction for In-Memory Cluster Computing. In Proceedings of the 9th USENIX Conference on Networked Systems Design and Implementation, NSDI '12, pages 15-28, USA. USENIX Association.
Dean, J. and Ghemawat, S. (2004). MapReduce: Simplified Data Processing on Large Clusters. In Proceedings of the 6th Symposium on Operating Systems Design and Implementation, OSDI' 04, pages 137-149. USENIX Association.
Giménez-Alventosa, V., Moltó, G., and Caballer, M. (2019). A framework and a performance assessment for serverless MapReduce on AWS Lambda. Future Generation Computer Systems, 97:259-274.
Kapil, D., Mishra, S., and Gupta, V. (2022). A performance perspective of live migration of virtual machine in cloud data center with future directions. International Journal of Wireless and Microwave Technologies, 12:48-56.
Kim, Y. and Lin, J. (2018). Serverless Data Analytics with Flint. In IEEE 11th International Conference on Cloud Computing (CLOUD), pages 451-455, Los Alamitos, CA, USA. IEEE Computer Society.
Malawski, M., Gajek, A., Zima, A., Balis, B., and Figiela, K. (2020). Serverless execution of scientific workflows: Experiments with hyperflow, aws lambda and google cloud functions. Future Generation Computer Systems, 110:502-514.
Muniswamaiah, M., Agerwala, T., and Tappert, C. (2019). Big data in cloud computing review and opportunities. International Journal of Computer Science & Information Technology (IJCSIT), 11.
Nunes, A. L., Melo, A., Boeres, C., de Oliveira, D., and Drummond, L. M. A. (2021). Towards Analyzing Computational Costs of Spark for SARS-CoV-2 Sequences Comparisons on a Commercial Cloud. XXII Symposium in High Performance Computing Systems, pages 192-203.
Shvachko, K., Kuang, H., Radia, S., and Chansler, R. (2010). The Hadoop Distributed File System. In Proceedings of the 26th IEEE Symposium on Mass Storage Systems and Technologies (MSST), pages 1-10. IEEE.
Zaharia, M., Chowdhury, M., Das, T., Dave, A., Ma, J., McCauley, M., Franklin, M. J., Shenker, S., and Stoica, I. (2012). Resilient Distributed Datasets: A Fault-Tolerant Abstraction for In-Memory Cluster Computing. In Proceedings of the 9th USENIX Conference on Networked Systems Design and Implementation, NSDI '12, pages 15-28, USA. USENIX Association.
Publicado
19/10/2022
Como Citar
CAMPBELL, Ronald; NUNES, Alan L.; BOERES, Cristina; DRUMMOND, Lúcia Maria de Assumpção.
MapReduce na AWS: Uma Análise de Custos Computacionais Utilizando os Serviços FaaS e IaaS. In: SIMPÓSIO EM SISTEMAS COMPUTACIONAIS DE ALTO DESEMPENHO (SSCAD), 23. , 2022, Florianópolis/SC.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2022
.
p. 145-156.
DOI: https://doi.org/10.5753/wscad.2022.226308.
