Optimizing load balancing and data-locality with data-aware scheduling
Load balancing techniques (eg work stealing) are important to obtain the best performance
for distributed task scheduling systems that have multiple schedulers making scheduling …
for distributed task scheduling systems that have multiple schedulers making scheduling …
Fusionfs: Toward supporting data-intensive scientific applications on extreme-scale high-performance computing systems
State-of-the-art, yet decades-old, architecture of high-performance computing systems has
its compute and storage resources separated. It thus is limited for modern data-intensive …
its compute and storage resources separated. It thus is limited for modern data-intensive …
Flux: A next-generation resource management framework for large HPC centers
DH Ahn, J Garlick, M Grondona, D Lipari… - 2014 43rd …, 2014 - ieeexplore.ieee.org
Resource and job management software is crucial to High Performance Computing (HPC)
for efficient application execution. However, current systems and approaches can no longer …
for efficient application execution. However, current systems and approaches can no longer …
PapyrusKV: A high-performance parallel key-value store for distributed NVM architectures
This paper introduces PapyrusKV, a parallel embedded key-value store (KVS) for distributed
high-performance computing (HPC) architectures that offer potentially massive pools of …
high-performance computing (HPC) architectures that offer potentially massive pools of …
I/O-aware batch scheduling for petascale computing systems
In the Big Data era, the gap between the storage performance and an application's I/O
requirement is increasing. I/O congestion caused by concurrent storage accesses from …
requirement is increasing. I/O congestion caused by concurrent storage accesses from …
Load‐balanced and locality‐aware scheduling for data‐intensive workloads at extreme scales
Data‐driven programming models such as many‐task computing (MTC) have been
prevalent for running data‐intensive scientific applications. MTC applies over …
prevalent for running data‐intensive scientific applications. MTC applies over …
Next generation job management systems for extreme-scale ensemble computing
With the exponential growth of supercomputers in parallelism, applications are growing
more diverse, including traditional large-scale HPC MPI jobs, and ensemble workloads such …
more diverse, including traditional large-scale HPC MPI jobs, and ensemble workloads such …
Overcoming hadoop scaling limitations through distributed task execution
Data driven programming models like MapReduce have gained the popularity in large-scale
data processing. Although great efforts through the Hadoop implementation and framework …
data processing. Although great efforts through the Hadoop implementation and framework …
Towards scalable distributed workload manager with monitoring-based weakly consistent resource stealing
One way to efficiently utilize the coming exascale machines is to support a mixture of
applications in various domains, such as traditional large-scale HPC, the ensemble runs …
applications in various domains, such as traditional large-scale HPC, the ensemble runs …
Exploring the design tradeoffs for extreme-scale high-performance computing system software
Owing to the extreme parallelism and the high component failure rates of tomorrow's
exascale, high-performance computing (HPC) system software will need to be scalable …
exascale, high-performance computing (HPC) system software will need to be scalable …