How FaaS with DBaaS performs in different regions: an evaluation by the Orama Framework
Resumo
Studies indicate that cloud services based on the serverless paradigm, such as Function-as-a-Service (FaaS) should become the main mechanisms of the next generation of cloud computing. Given this perspective, public cloud providers have made efforts to expand the coverage of their services in order to meet this need. However, the effort needed to maintain equivalence between different regions highlights the importance of studying the behavior of FaaS environments in different regions of providers. This work presents a study aided by the Orama framework in order to evaluate the performance of the main FaaS integrated with Database-as-a-Service (DBaaS) services in five regions spread across the globe. The results indicate that the Alibaba provider was able to guarantee good equivalence between its regions, in addition to a lower average execution time. AWS and GCP had similar results, although the error rate on AWS was the highest on average. Azure, on the other hand, had the worst performance, with the highest average execution time, in addition to significant failure rates.
Referências
Back, T. and Andrikopoulos, V. (2018). Using a microbenchmark to compare function as a service solutions. In ECSOCC, pages 146–160. Springer.
Barcelona-Pons, D. and García-López, P. (2021). Benchmarking parallelism in faas platforms. Future Generation Computer Systems, 124:268–284.
Carvalho., L. and Araujo., A. (2022). Orama: A benchmark framework for function-as-a-service. In Proceedings of the 12th CLOSER, pages 313–322. INSTICC, SciTePress.
Cloud, A. (2021). Alibaba cloud function. [link]. [online; 11-Aug-2021].
García López, P., Sánchez-Artigas, M., París, G., Barcelona Pons, D., Ruiz Ollobarren, A., and Arroyo Pinto, D. (2018). Comparison of faas orchestration systems. In 2018 IEEE/ACM UCC Companion, pages 148–153.
Google (2021). Cloud functions. [link]. [Online; 10-Aug-2021].
Grambow, M., Pfandzelter, T., Burchard, L., Schubert, C., Zhao, M., and Bermbach, D. (2021). Befaas: An application-centric benchmarking framework for faas platforms. Jain, R. (1991). The art of computer systems: Techniques for experimental design, measurement, simulation, and modeling.
Kuhlenkamp, J., Werner, S., Borges, M. C., El Tal, K., and Tai, S. (2019). An evaluation of faas platforms as a foundation for serverless big data processing. In Proceedings of the 12th IEEE/ACM, UCC’19, page 1–9, NY, USA. ACM.
Malawski, M., Gajek, A., Zima, A., Balis, B., and Figiela, K. (2020). Serverless execution of scientific workflows: Experiments with hyperflow, AWS lambda and Google Cloud Functions. Future Generation Computer Systems, 110:502–514.
MELL, P. and Grance, T. (2011). The NIST definition of cloud computing. National Institute of Standards and Tecnology.
Microsoft (2021). Azure functions. [link]. [online; 11-Aug-2021].
Motta, M. A. C., Carvalho, L. R., Rosa, M. J. F., and Araujo, A. P. F. (2022). Comparison of faas platform performance in private clouds. In Proceedings of the 12th CLOSER,, pages 109–120. INSTICC, SciTePress.
Nupponen, J. and Taibi, D. (2020). Serverless: What it is, what to do and what not to do. In 2020 IEEE ICSA-C, pages 49–50.
Schleier-Smith, J., Sreekanti, V., Khandelwal, A., Carreira, J., Yadwadkar, N. J., Popa, R. A., Gonzalez, J. E., Stoica, I., and Patterson, D. A. (2021). What serverless computing is and should become: The next phase of cloud computing. ACM, 64(5):76–84.
Somu, N., Daw, N., Bellur, U., and Kulkarni, P. (2020). Panopticon: A comprehensive benchmarking tool for serverless applications. In 2020 COMSNETS, pages 144–151.
Wen, J., Liu, Y., Chen, Z., Ma, Y., Wang, H., and Liu, X. (2021). Understanding characteristics of commodity serverless computing platforms.
ZHENG, X. (2018). Database as a service - current issues and its future. CoRR, abs/1804.00465.
