Abstract
Phylogenomic experiments provide the basis for evolutionary biology inferences. They are data- and CPU-intensive by nature and aim at producing phylogenomic trees based on an input dataset of protein sequences of genomes. These experiments can be modeled as scientific workflows. Although workflows can be efficiently managed by Workflow Management Systems (WfMS), they are not often used by bioinformaticians, which traditionally use scripts to implement their workflows. However, collecting provenance from scripts is a challenging task. In this paper, we specialize the DfAnalyzer tool for the phylogenomics domain. DfAnalyzer enables capturing, monitoring, debugging, and analysing dataflows while being generated by the script. Additionally, it can be invoked from scripts, in the same way bioinformaticians already import libraries in their code. The proposed approach captures strategic domain data, registering provenance and telemetry (performance) data to enable queries at runtime. Another advantage of specializing DfAnalyzer in the context of Phylogenomic experiments is the capability of capturing data from experiments that execute either locally or in HPC environments. We evaluated the proposed specialization of DfAnalyzer using the SciPhylomics workflow and the proposed approach showed relevant telemetry scenarios and rich data analyses.
This work was partially supported by CNPq, CAPES, and FAPERJ.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Abuín, J.M., Pichel, J.C., Pena, T.F., Amigo, J.: SparkBWA: speeding up the alignment of high-throughput DNA sequencing data. PLoS ONE 11(5), e0155461 (2016)
Afgan, E., et al.: The galaxy platform for accessible, reproducible and collaborative biomedical analyses: 2016 update. Nucleic Acids Res. 44(W1), W3–W10 (2016)
Carvalho, L.A.M.C., Wang, R., Gil, Y., Garijo, D.: NIW: converting notebooks into workflows to capture dataflow and provenance. In: Tiddi, I., Rizzo, G., Corcho, Ó. (eds.) Proceedings of Workshops and Tutorials of the 9th International Conference on Knowledge Capture (K-CAP 2017), Austin, Texas, USA, 4 December 2017. CEUR Workshop Proceedings, vol. 2065, pp. 12–16. CEUR-WS.org (2017)
de Oliveira, D.C.M., Liu, J., Pacitti, E.: Data-intensive workflow management: for clouds and data-intensive and scalable computing environments (2019)
Deelman, E., et al.: Pegasus, a workflow management system for science automation. FGCS 46, 17–35 (2015)
Freire, J., Koop, D., Santos, E., Silva, C.T.: Provenance for computational tasks: a survey. CS&E 10(3), 11–21 (2008)
Guedes, T., et al.: Capturing and analyzing provenance from spark-based scientific workflows with SAMbA-RaP. Future Gener. Comput. Syst. 112, 658–669 (2020)
Hondo, F., et al.: Data provenance management for bioinformatics workflows using NoSQL database systems in a cloud computing environment. In: 2017 IEEE International Conference on Bioinformatics and Biomedicine (BIBM), pp. 1929–1934. IEEE (2017)
Marozzo, F., Talia, D., Trunfio, P.: Scalable script-based data analysis workflows on clouds. In: WORKS, pp. 124–133 (2013)
Masulli, F.: Comput. Methods Programs Biomed. 91(2), 182 (2008)
Moreau, L., et al.: PROV-DM: the PROV data model. W3C Recommendation 30, 1–38 (2013)
Oliveira, D., Ocaña, K.A.C.S., Baião, F.A., Mattoso, M.: A provenance-based adaptive scheduling heuristic for parallel scientific workflows in clouds. JGC 10(3), 521–552 (2012)
de Oliveira, D., et al.: Performance evaluation of parallel strategies in public clouds: a study with phylogenomic workflows. Future Gener. Comput. Syst. 29(7), 1816–1825 (2013)
Olma, M., Karpathiotakis, M., Alagiannis, I., Athanassoulis, M., Ailamaki, A.: Slalom: coasting through raw data via adaptive partitioning and indexing. Proc. VLDB Endow. 10(10), 1106–1117 (2017)
Pimentel, J.F., Murta, L., Braganholo, V., Freire, J.: noWorkflow: a tool for collecting, analyzing, and managing provenance from python scripts. Proc. VLDB Endow. 10(12), 1841–1844 (2017)
Pina, D.B., Neves, L., Paes, A., de Oliveira, D., Mattoso, M.: Análise de hiperparâmetros em aplicações de aprendizado profundo por meio de dados de proveniência. In: Anais Principais do XXXIV Simpósio Brasileiro de Banco de Dados, pp. 223–228. SBC (2019)
Silva, V., de Oliveira, D., Valduriez, P., Mattoso, M.: Dfanalyzer: runtime dataflow analysis of scientific applications using provenance. Proc. VLDB Endow. 11(12), 2082–2085 (2018)
The UniProt Consortium: UniProt: the universal protein knowledgebase. Nucleic Acids Res. 45(D1), D158–D169 (2016)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Dias, L.G., Mattoso, M., Lopes, B., de Oliveira, D. (2020). Experiencing DfAnalyzer for Runtime Analysis of Phylogenomic Dataflows. In: Setubal, J.C., Silva, W.M. (eds) Advances in Bioinformatics and Computational Biology. BSB 2020. Lecture Notes in Computer Science(), vol 12558. Springer, Cham. https://doi.org/10.1007/978-3-030-65775-8_10
Download citation
DOI: https://doi.org/10.1007/978-3-030-65775-8_10
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-65774-1
Online ISBN: 978-3-030-65775-8
eBook Packages: Computer ScienceComputer Science (R0)