Identifying challenges and opportunities of in-memory computing on large HPC systems

D Huang, Z Qin, Q Liu, N Podhorszki… - Journal of Parallel and …, 2022 - Elsevier
With the increasing fidelity and resolution enabled by high-performance computing systems,
simulation-based scientific discovery is able to model and understand microscopic physical …

Goldrush: Resource efficient in situ scientific data analytics using fine-grained interference aware execution

F Zheng, H Yu, C Hantas, M Wolf… - Proceedings of the …, 2013 - dl.acm.org
Severe I/O bottlenecks on High End Computing platforms call for running data analytics in
situ. Demonstrating that there exist considerable resources in compute nodes un-used by …

Decaf: Decoupled dataflows for in situ high-performance workflows

M Dreher, T Peterka - 2017 - osti.gov
Decaf is a dataflow system for the parallel communication of coupled tasks in an HPC
workflow. The dataflow can perform arbitrary data transformations ranging from simply …

Smart: A mapreduce-like framework for in-situ scientific analytics

Y Wang, G Agrawal, T Bicer, W Jiang - Proceedings of the International …, 2015 - dl.acm.org
In-situ analytics has lately been shown to be an effective approach to reduce both I/O and
storage costs for scientific analytics. Developing an efficient in-situ implementation, however …

An integrated task computation and data management scheduling strategy for workflow applications in cloud environments

L Zeng, B Veeravalli, AY Zomaya - Journal of Network and Computer …, 2015 - Elsevier
A workflow is a systematic computation or a data-intensive application that has a regular
computation and data access patterns. It is a key to design scalable scheduling algorithms in …

A flexible framework for asynchronous in situ and in transit analytics for scientific simulations

M Dreher, B Raffin - … Symposium on Cluster, Cloud and Grid …, 2014 - ieeexplore.ieee.org
High performance computing systems are today composed of tens of thousands of
processors and deep memory hierarchies. The next generation of machines will further …

Scaling embedded in-situ indexing with deltaFS

Q Zheng, CD Cranor, D Guo, GR Ganger… - … Conference for High …, 2018 - ieeexplore.ieee.org
Analysis of large-scale simulation output is a core element of scientific inquiry, but analysis
queries may experience significant I/O overhead when the data is not structured for efficient …

Melissa: large scale in transit sensitivity analysis avoiding intermediate files

T Terraz, A Ribes, Y Fournier, B Iooss… - Proceedings of the …, 2017 - dl.acm.org
Global sensitivity analysis is an important step for analyzing and validating numerical
simulations. One classical approach consists in computing statistics on the outputs from well …

Bootstrapping in-situ workflow auto-tuning via combining performance models of component applications

T Shu, Y Guo, J Wozniak, X Ding, I Foster… - Proceedings of the …, 2021 - dl.acm.org
In an in-situ workflow, multiple components such as simulation and analysis applications are
coupled with streaming data transfers. The multiplicity of possible configurations …

Clarisse: A middleware for data-staging coordination and control on large-scale hpc platforms

F Isaila, J Carretero, R Ross - 2016 16th IEEE/ACM …, 2016 - ieeexplore.ieee.org
On current large-scale HPC platforms the data path from compute nodes to final storage
passes through several networks interconnecting a distributed hierarchy of nodes serving as …