High-ratio lossy compression: Exploring the autoencoder to compress scientific data

T Liu, J Wang, Q Liu, S Alibhai, T Lu… - IEEE Transactions on …, 2021 - ieeexplore.ieee.org
Scientific simulations on high-performance computing (HPC) systems can generate large
amounts of floating-point data per run. To mitigate the data storage bottleneck and lower the …

Cusz: An efficient gpu-based error-bounded lossy compression framework for scientific data

J Tian, S Di, K Zhao, C Rivera, MH Fulp… - Proceedings of the …, 2020 - dl.acm.org
Error-bounded lossy compression is a state-of-the-art data reduction technique for HPC
applications because it not only significantly reduces storage overhead but also can retain …

Multifacets of lossy compression for scientific data in the Joint-Laboratory of Extreme Scale Computing

F Cappello, S Di, R Underwood, D Tao… - Future Generation …, 2024 - Elsevier
Abstract The Joint Laboratory on Extreme-Scale Computing (JLESC) was initiated at the
same time lossy compression for scientific data became an important topic for the scientific …

Improving prediction-based lossy compression dramatically via ratio-quality modeling

S Jin, S Di, J Tian, S Byna, D Tao… - 2022 IEEE 38th …, 2022 - ieeexplore.ieee.org
Error-bounded lossy compression is one of the most effective techniques for reducing
scientific data sizes. However, the traditional trial-and-error approach used to configure …

AMRIC: A novel in situ lossy compression framework for efficient I/O in adaptive mesh refinement applications

D Wang, J Pulido, P Grosset, J Tian, S Jin… - Proceedings of the …, 2023 - dl.acm.org
As supercomputers advance towards exascale capabilities, computational intensity
increases significantly, and the volume of data requiring storage and transmission …

Designing high-performance mpi libraries with on-the-fly compression for modern gpu clusters

Q Zhou, C Chu, NS Kumar, P Kousha… - 2021 IEEE …, 2021 - ieeexplore.ieee.org
While the memory bandwidth of accelerators such as GPU has significantly improved over
the last decade, the commodity networks such as Ethernet and InfiniBand are lagging in …

Accelerating parallel write via deeply integrating predictive lossy compression with HDF5

S Jin, D Tao, H Tang, S Di, S Byna… - … Conference for High …, 2022 - ieeexplore.ieee.org
Lossy compression is one of the most efficient solutions to reduce storage overhead and
improve I/O performance for HPC applications. However, existing parallel I/O libraries …

Foresight: analysis that matters for data reduction

P Grosset, CM Biwer, J Pulido… - … Conference for High …, 2020 - ieeexplore.ieee.org
As the computation power of supercomputers increases, so does simulation size, which in
turn produces orders-of-magnitude more data. Because generated data often exceed the …

Accelerating distributed deep learning training with compression assisted allgather and reduce-scatter communication

Q Zhou, Q Anthony, L Xu, A Shafi… - 2023 IEEE …, 2023 - ieeexplore.ieee.org
Fully Sharded Data Parallel (FSDP) technology achieves higher performance by scaling out
data-parallel training of Deep Learning (DL) models. It shards the model parameters …

Fz-gpu: A fast and high-ratio lossy compressor for scientific computing applications on gpus

B Zhang, J Tian, S Di, X Yu, Y Feng, X Liang… - Proceedings of the …, 2023 - dl.acm.org
Today's large-scale scientific applications running on high-performance computing (HPC)
systems generate vast data volumes. Thus, data compression is becoming a critical …