Checkpointing Parallel Applications for Azure Spot Virtual Machines: Tools, Runtimes, and Storage Trade-offs
Resumo
Spot virtual machines (VMs) provide access to spare computing capacity at lower prices, but cloud providers may evict them with short notice. We study transparent checkpoint/restart for CPU applications on Azure Spot VMs, using CRIU/Podman for OpenMP and DMTCP for single-node MPI. Local NVMe measurements validate both setups and reveal different checkpoint-time trends as parallelism grows. We also evaluate Managed Disks, Azure Files, and Blob Storage for checkpoint durability and recovery on a replacement VM. A Poisson revocation model relates checkpoint and recovery times to checkpoint intervals and expected VM execution time. Azure’s 30 s eviction notice motivates the proactive persistence of checkpoints.
Referências
Abdel Azeem, B. and Helal, M. (2014). Performance evaluation of checkpoint/restart techniques for MPI applications on Amazon cloud. In Proceedings of the 9th International Conference on Informatics and Systems, pages PDC–49–PDC–56.
Amazon Web Services (2026a). Behavior of spot instance interruptions. [link]. Accessed October 15, 2026.
Amazon Web Services (2026b). Spot instance interruption notices. [link]. Accessed October 15, 2026.
Ansel, J., Arya, K., and Cooperman, G. (2009). DMTCP: Transparent checkpointing for cluster computations and the desktop. In Proceedings of the IEEE International Symposium on Parallel and Distributed Processing, pages 1–12.
CRIU Project (2026). Checkpoint/restore. [link] Restore. Accessed July 13, 2026.
Daly, J. T. (2006). A higher order estimate of the optimum checkpoint interval for restart dumps. Future Generation Computer Systems, 22(3):303–312.
Duell, J., Hargrove, P., and Roman, E. (2002). The design and implementation of Berkeley Lab’s Linux Checkpoint/Restart. Technical Report LBNL-54941, Lawrence Berkeley National Laboratory.
Garg, R., Price, G., and Cooperman, G. (2019). MANA for MPI: MPI-agnostic network-agnostic transparent checkpointing. In Proceedings of the 28th International Symposium on High-Performance Parallel and Distributed Computing, pages 49–60. ACM.
Gong, Y., He, B., and Zhou, A. C. (2015). Monetary cost optimizations for MPI-based HPC applications on Amazon clouds: Checkpoints and replicated execution. In Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis, pages 1–12. ACM.
Jin, H., Frumkin, M., and Yan, J. (1999). The OpenMP implementation of NAS Parallel Benchmarks and its performance. Technical Report NAS-99-011, NASA Ames Research Center.
Microsoft Azure (2026a). About Azure Spot Virtual Machines. [link]. Accessed October 15, 2026.
Microsoft Azure (2026b). Azure Metadata Service: Scheduled events for Linux virtual machines. [link]. Accessed October 15, 2026.
Microsoft Azure (2026c). Build workloads on spot virtual machines. [link]. Accessed October 15, 2026.
Microsoft Azure (2026d). Introduction to Azure Storage. [link]. Accessed October 15, 2026.
Munhoz, V., Castro, M., and Mendizabal, O. (2022). Strategies for fault-tolerant tightly-coupled HPC workloads running on low-budget spot cloud infrastructures. In Proceedings of the 34th International Symposium on Computer Architecture and High Performance Computing, pages 263–272. IEEE.
NASA Advanced Supercomputing Division (2026). NAS Parallel Benchmarks. [link]. Accessed July 13, 2026.
Serrano, L., Lima, M., Portella, F., and Drummond, L. (2025). A multi-cloud approach to cost optimization with aws and azure fleet services. In Anais Estendidos do XXXVII International Symposium on Computer Architecture and High Performance Computing, pages 61–68, Porto Alegre, RS, Brasil. SBC.
Siavvas, M. and Gelenbe, E. (2019). Optimum checkpoints for programs with loops. Simulation Modelling Practice and Theory, 97:101951.
Sodré, D., Serrano, L., Lima, M., Boeres, C., Drummond, L., and Rebello, V. (2025). Spotting the right cloud instances with multiple aws ec2 fleets. In Anais do XXXVII International Symposium on Computer Architecture and High Performance Computing, pages 69–79, Porto Alegre, RS, Brasil. SBC.
Teylo, L., Brum, R. C., Arantes, L., Sens, P., and Drummond, L. M. d. A. (2020). Developing checkpointing and recovery procedures with the storage services of Amazon Web Services. In Proceedings of the 49th International Conference on Parallel Processing Workshops, pages 1–8. ACM.
Amazon Web Services (2026a). Behavior of spot instance interruptions. [link]. Accessed October 15, 2026.
Amazon Web Services (2026b). Spot instance interruption notices. [link]. Accessed October 15, 2026.
Ansel, J., Arya, K., and Cooperman, G. (2009). DMTCP: Transparent checkpointing for cluster computations and the desktop. In Proceedings of the IEEE International Symposium on Parallel and Distributed Processing, pages 1–12.
CRIU Project (2026). Checkpoint/restore. [link] Restore. Accessed July 13, 2026.
Daly, J. T. (2006). A higher order estimate of the optimum checkpoint interval for restart dumps. Future Generation Computer Systems, 22(3):303–312.
Duell, J., Hargrove, P., and Roman, E. (2002). The design and implementation of Berkeley Lab’s Linux Checkpoint/Restart. Technical Report LBNL-54941, Lawrence Berkeley National Laboratory.
Garg, R., Price, G., and Cooperman, G. (2019). MANA for MPI: MPI-agnostic network-agnostic transparent checkpointing. In Proceedings of the 28th International Symposium on High-Performance Parallel and Distributed Computing, pages 49–60. ACM.
Gong, Y., He, B., and Zhou, A. C. (2015). Monetary cost optimizations for MPI-based HPC applications on Amazon clouds: Checkpoints and replicated execution. In Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis, pages 1–12. ACM.
Jin, H., Frumkin, M., and Yan, J. (1999). The OpenMP implementation of NAS Parallel Benchmarks and its performance. Technical Report NAS-99-011, NASA Ames Research Center.
Microsoft Azure (2026a). About Azure Spot Virtual Machines. [link]. Accessed October 15, 2026.
Microsoft Azure (2026b). Azure Metadata Service: Scheduled events for Linux virtual machines. [link]. Accessed October 15, 2026.
Microsoft Azure (2026c). Build workloads on spot virtual machines. [link]. Accessed October 15, 2026.
Microsoft Azure (2026d). Introduction to Azure Storage. [link]. Accessed October 15, 2026.
Munhoz, V., Castro, M., and Mendizabal, O. (2022). Strategies for fault-tolerant tightly-coupled HPC workloads running on low-budget spot cloud infrastructures. In Proceedings of the 34th International Symposium on Computer Architecture and High Performance Computing, pages 263–272. IEEE.
NASA Advanced Supercomputing Division (2026). NAS Parallel Benchmarks. [link]. Accessed July 13, 2026.
Serrano, L., Lima, M., Portella, F., and Drummond, L. (2025). A multi-cloud approach to cost optimization with aws and azure fleet services. In Anais Estendidos do XXXVII International Symposium on Computer Architecture and High Performance Computing, pages 61–68, Porto Alegre, RS, Brasil. SBC.
Siavvas, M. and Gelenbe, E. (2019). Optimum checkpoints for programs with loops. Simulation Modelling Practice and Theory, 97:101951.
Sodré, D., Serrano, L., Lima, M., Boeres, C., Drummond, L., and Rebello, V. (2025). Spotting the right cloud instances with multiple aws ec2 fleets. In Anais do XXXVII International Symposium on Computer Architecture and High Performance Computing, pages 69–79, Porto Alegre, RS, Brasil. SBC.
Teylo, L., Brum, R. C., Arantes, L., Sens, P., and Drummond, L. M. d. A. (2020). Developing checkpointing and recovery procedures with the storage services of Amazon Web Services. In Proceedings of the 49th International Conference on Parallel Processing Workshops, pages 1–8. ACM.
Publicado
03/11/2026
Como Citar
SERRANO, Lucas et al.
Checkpointing Parallel Applications for Azure Spot Virtual Machines: Tools, Runtimes, and Storage Trade-offs. In: SIMPÓSIO EM SISTEMAS COMPUTACIONAIS DE ALTO DESEMPENHO (SSCAD), 27. , 2026, Natal/RN.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2026
.
p. 273-286.
DOI: https://doi.org/10.5753/sscad.2026.34929.
