Fully Sharded Data Parallel (FSDP) technology achieves higher performance by scaling out
data-parallel training of Deep Learning (DL) models. It shards the model parameters …

Cross-application interference can drastically affect performance of HPC applications
executed in clouds. The problem is caused by concurrent access of co-located applications …

Data employed in computations amongst multiple processors in a computing system is
processed so that less bits than a full representation of the data needs to be communicated …

With the rapidly increasing model sizes, state-of-the-art Deep Learning (DL) models rely on
multiple GPU nodes to run distributed training. Large message communication of GPU data …

As the number of nodes in high performance computing (HPC) systems increases, collective
I/O becomes an important issue and I/O aggregators are the key factors in improving the …

Data employed in computations is processed so that during computations more of the data
can be fit into or maintained in a smaller but higher speed memory than an original source of …

Ultrascale computing systems are likely to reach speeds of two or three orders of magnitude
greater than today's computing systems. However, to achieve this level of performance, we …

Communication overheads in distributed systems constitute a large fraction of the total
execution time, and limit the scalability of applications running on these systems. We …

Due to the increase and complexity of computer systems, reducing the overhead of fault
tolerance techniques has become important in recent years. One technique in fault tolerance …

Energy consumption is currently one of the main issues in large distributed systems. More
specifically, the efficient management of energy without losing performance has become a …