Lotaru: Locally predicting workflow task runtimes for resource management on heterogeneous infrastructures

J Bader, F Lehmann, L Thamsen, U Leser… - Future Generation …, 2024 - Elsevier
Many resource management techniques for task scheduling, energy and carbon efficiency,
and cost optimization in workflows rely on a-priori task runtime knowledge. Building runtime …

Big data scientific workflows in the cloud: Challenges and future prospects

S Khan, SA Ali, N Hasan, KA Shakil, M Alam - Cloud Computing for …, 2019 - Springer
The concept of workflows was implemented to mitigate the complexities involved in tasks
related to scientific computing and business analytics. With time, they have found …

How workflow engines should talk to resource managers: A proposal for a common workflow scheduling interface

F Lehmann, J Bader, F Tschirpke… - 2023 IEEE/ACM …, 2023 - ieeexplore.ieee.org
Scientific workflow management systems (SWMSs) and resource managers together ensure
that tasks are scheduled on provisioned resources so that all dependencies are obeyed …

Lotaru: Locally estimating runtimes of scientific workflow tasks in heterogeneous clusters

J Bader, F Lehmann, L Thamsen, J Will… - Proceedings of the 34th …, 2022 - dl.acm.org
Many scientific workflow scheduling algorithms need to be informed about task runtimes a-
priori to conduct efficient scheduling. In heterogeneous cluster infrastructures, this problem …

Sweep: Accelerating scientific research through scalable serverless workflows

A John, K Ausmees, K Muenzen, C Kuhn… - Proceedings of the 12th …, 2019 - dl.acm.org
Scientific and commercial applications are increasingly being executed in the cloud, but the
difficulties associated with cluster management render on-demand resources inaccessible …

Feedback-based resource allocation for batch scheduling of scientific workflows

C Witt, D Wagner, U Leser - 2019 International Conference on …, 2019 - ieeexplore.ieee.org
A scientific workflow is a set of interdependent compute tasks orchestrating large scale data
analyses or in-silico experiments. Workflows often comprise thousands of tasks with …

Sizey: Memory-efficient execution of scientific workflow tasks

J Bader, F Skalski, F Lehmann, D Scheinert… - arXiv preprint arXiv …, 2024 - arxiv.org
As the amount of available data continues to grow in fields as diverse as bioinformatics,
physics, and remote sensing, the importance of scientific workflows in the design and …

Computation semantics of the functional scientific workflow language Cuneiform

J Brandt, W Reisig, U Leser - Journal of Functional Programming, 2017 - cambridge.org
Cuneiform is a minimal functional programming language for large-scale scientific data
analysis. Implementing a strict black-box view on external operators and data, it allows the …

Portability of scientific workflows in ngs data analysis: a case study

C Schiefer, M Bux, J Brandt, C Messerschmidt… - arXiv preprint arXiv …, 2020 - arxiv.org
The analysis of next-generation sequencing (NGS) data requires complex computational
workflows consisting of dozens of autonomously developed yet interdependent processing …

Ponder: Online Prediction of Task Memory Requirements for Scientific Workflows

F Lehmann, J Bader, N De Mecquenem… - 2024 IEEE 20th …, 2024 - ieeexplore.ieee.org
Scientific workflows are used to analyze large amounts of data. These workflows comprise
numerous tasks, many of which are executed repeatedly, running the same custom program …