A survey and classification of software-defined storage systems
The exponential growth of digital information is imposing increasing scale and efficiency
demands on modern storage infrastructures. As infrastructure complexity increases, so does …
demands on modern storage infrastructures. As infrastructure complexity increases, so does …
Sizing and partitioning strategies for burst-buffers to reduce io contention
G Aupy, O Beaumont… - 2019 IEEE international …, 2019 - ieeexplore.ieee.org
Burst-Buffers are high throughput and small size storage which are being used as an
intermediate storage between the PFS (Parallel File System) and the computational nodes …
intermediate storage between the PFS (Parallel File System) and the computational nodes …
Mapping and scheduling HPC applications for optimizing I/O
In HPC platforms, concurrent applications are sharing the same file system. This can lead to
conflicts, especially as applications are more and more data intensive. I/O contention can …
conflicts, especially as applications are more and more data intensive. I/O contention can …
Analysis and correlation of application I/O performance and system-wide I/O activity
Storage resources in high-performance computing are shared across all user applications.
Consequently, storage performance can vary markedly, depending not only on an …
Consequently, storage performance can vary markedly, depending not only on an …
The Case for Storage Optimization Decoupling in Deep Learning Frameworks
Deep Learning (DL) training requires efficient access to large collections of data, leading DL
frameworks to implement individual I/O optimizations to take full advantage of storage …
frameworks to implement individual I/O optimizations to take full advantage of storage …
Limitless—light-weight monitoring tool for large scale systems
This work presents LIMITLESS, a HPC framework that provides new strategies for
monitoring clusters. LIMITLESS is a scalable light-weight monitor that is integrated with other …
monitoring clusters. LIMITLESS is a scalable light-weight monitor that is integrated with other …
Data Flow Lifecycles for Optimizing Workflow Coordination
A critical performance challenge in distributed scientific workflows is coordinating tasks and
data flows on distributed resources. To guide these decisions, this paper introduces data …
data flows on distributed resources. To guide these decisions, this paper introduces data …
Scheduling periodic I/O access with bi-colored chains: models and algorithms
Observations show that some HPC applications periodically alternate between (i) operations
(computations, local data accesses) executed on the compute nodes, and (ii) I/O transfers of …
(computations, local data accesses) executed on the compute nodes, and (ii) I/O transfers of …
NORNS: extending Slurm to support data-driven workflows through asynchronous data staging
As HPC systems move into the Exascale era, parallel file systems are struggling to keep up
with the I/O requirements from data-intensive problems. While the inclusion of burst buffers …
with the I/O requirements from data-intensive problems. While the inclusion of burst buffers …
IO-aware Job-Scheduling: Exploiting the Impacts of Workload Characterizations to select the Mapping Strategy
In high performance, computing concurrent applications are sharing the same file system.
However, the bandwidth which provides access to the storage is limited. Therefore, too …
However, the bandwidth which provides access to the storage is limited. Therefore, too …