Proposal of an agnostic model for serverless composition and orchestration of scientific workflows

• André Vieira USP
• Daniel Cordeiro USP

DOI: https://doi.org/10.5753/eradsp.2025.9760

Abstract

Serverless computing offers potential benefits for the execution of scientific workflows, but presents challenges related to the stateless nature of functions and the complexity of orchestration. This work proposes a model for composing and orchestrating scientific workflows in serverless environments, utilizing a platform-agnostic specification language to define the workflow structure. The approach employs an event-driven architecture with persistent state management to coordinate task execution. The goal is to enable portability and scalable execution of complex scientific processes in the cloud, overcoming the inherent limitations of the serverless paradigm.

References

Atkinson, M., Gesing, S., Montagnat, J., and Taylor, I. (2017). Scientific workflows: Past, present and future. Future Generation Computer Systems, 75:216–227.

Baldini, I., Castro, P., Chang, K., Cheng, P., Fink, S., Ishakian, V., Mitchell, N., Muthusamy, V., Rabbah, R., Slominski, A., and Suter, P. (2017). Serverless Computing: Current Trends and Open Problems, pages 1–20. Springer Singapore, Singapore.

Castro, P., Ishakian, V., Muthusamy, V., and Slominski, A. (2019). The rise of serverless computing. Commun. ACM, 62(12):44–54.

Deelman, E., Vahi, K., Juve, G., Rynge, M., Callaghan, S., Maechling, P. J., Mayani, R., Chen, W., Ferreira da Silva, R., Livny, M., and Wenger, K. (2015). Pegasus, a workflow management system for science automation. Future Generation Computer Systems, 46:17–35.

Ristov, S., Pedratscher, S., and Fahringer, T. (2021). Afcl: An abstract function choreography language for serverless workflow specification. Future Generation Computer Systems, 114:368–382.