An Analysis of Performance Variability in AWS Virtual Machines

• Miguel de Lima UFF
• Luan Teylo Centre Inria de l’université de Bordeaux
• Lúcia Drummond UFF

DOI: https://doi.org/10.5753/sscad.2024.244526

Resumo

Cloud computing platforms are essential for a wide range of applications, including High-Performance Computing (HPC) and artificial intelligence. However, the performance variability of virtual machines (VMs) in these shared environments presents significant challenges. This paper provides an extensive month-long analysis of the performance variability of C family VMs on Amazon Web Services (AWS) across two regions (us-east-1 and sa-east-1), various instance generations, and market types. Our findings indicate that Graviton processors (c6g.12xlarge and c7g.12xlarge) exhibit minimal performance variability and high cost-effectiveness, with the c7g.12xlarge instance, in particular, offering significantly reduced execution times and lower costs. Intel and AMD instances, while showing performance improvements from generation c6 to c7, exhibited up to 20% variability.

Referências

Bailey, D. H., Barszcz, E., Barton, J. T., Browning, D. S., Carter, R. L., Dagum, L., Fatoohi, R. A., Frederickson, P. O., Lasinski, T. A., Schreiber, R. S., et al. (1991). The nas parallel benchmarks. The International Journal of Supercomputing Applications, 5(3):63–73.

Bakhtiarifard, P., Igel, C., and Selvan, R. (2024). Ec-nas: Energy consumption aware tabular benchmarks for neural architecture search. In ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 5660–5664. IEEE.

Dancheva, T., Alonso, U., and Barton, M. (2024). Cloud benchmarking and performance analysis of an hpc application in amazon ec2. Cluster Computing, 27(2):2273–2290.

Ericson, J., Mohammadian, M., and Santana, F. (2017). Analysis of performance variability in public cloud computing. In 2017 IEEE International Conference on Information Reuse and Integration (IRI), pages 308–314. IEEE.

Ferrari, G. C. F., Castro, M., et al. (2024). Comparing burstable and on-demand aws ec2 instances using nas parallel benchmarks. In Anais da XXIV Escola Regional de Alto Desempenho da Região Sul, pages 61–64. SBC.

Hosseini, S. S., Chuen, A. M., and Chan, W. M. (2024). Computational aerodynamics study of the lift+ cruise vtol concept vehicle components. In Transformative Vertical Flight (TVF) Meeting.

Iosup, A., Yigitbasi, N., and Epema, D. (2011). On the performance variability of production cloud services. In 2011 11th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing, pages 104–113. IEEE.

Jackson, K. R., Ramakrishnan, L., Muriki, K., Canon, S., Cholia, S., Shalf, J., Wasserman, H. J., and Wright, N. J. (2010). Performance analysis of high performance computing applications on the amazon web services cloud. In 2010 IEEE second international conference on cloud computing technology and science, pages 159–168. IEEE.

Munhoz, V., Bonfils, A., Castro, M., and Mendizabal, O. (2023). A performance comparison of hpc workloads on traditional and cloud-based hpc clusters. In 2023 International Symposium on Computer Architecture and High Performance Computing Workshops (SBAC-PADW), pages 108–114. IEEE.

Oliker, L., Canning, A., Carter, J., Shalf, J., and Ethier, S. (2004). Scientific computations on modern parallel vector systems. In SC’04: Proceedings of the 2004 ACM/IEEE Conference on Supercomputing, pages 10–10. IEEE.

Subhlok, J., Venkataramaiah, S., and Singh, A. (2002). Characterizing nas benchmark performance on shared heterogeneous networks. In Proceedings 16th International Parallel and Distributed Processing Symposium, pages 9–pp. IEEE.