Distributed graph neural network training: A survey

Y Shao, H Li, X Gu, H Yin, Y Li, X Miao… - ACM Computing …, 2024 - dl.acm.org
Graph neural networks (GNNs) are a type of deep learning models that are trained on
graphs and have been successfully applied in various domains. Despite the effectiveness of …

Sancus: staleness-aware communication-avoiding full-graph decentralized training in large-scale graph neural networks

J Peng, Z Chen, Y Shao, Y Shen, L Chen… - Proceedings of the VLDB …, 2022 - dl.acm.org
Graph neural networks (GNNs) have emerged due to their success at modeling graph data.
Yet, it is challenging for GNNs to efficiently scale to large graphs. Thus, distributed GNNs …

Parallel and distributed graph neural networks: An in-depth concurrency analysis

M Besta, T Hoefler - IEEE Transactions on Pattern Analysis and …, 2024 - ieeexplore.ieee.org
Graph neural networks (GNNs) are among the most powerful tools in deep learning. They
routinely solve complex problems on unstructured networks, such as node classification …

Neutronstar: distributed GNN training with hybrid dependency management

Q Wang, Y Zhang, H Wang, C Chen, X Zhang… - Proceedings of the 2022 …, 2022 - dl.acm.org
GNN's training needs to resolve issues of vertex dependencies, ie, each vertex
representation's update depends on its neighbors. Existing distributed GNN systems adopt …

Mariusgnn: Resource-efficient out-of-core training of graph neural networks

R Waleffe, J Mohoney, T Rekatsinas… - Proceedings of the …, 2023 - dl.acm.org
We study training of Graph Neural Networks (GNNs) for large-scale graphs. We revisit the
premise of using distributed training for billion-scale graphs and show that for graphs that fit …

DUCATI: A dual-cache training system for graph neural networks on giant graphs with the GPU

X Zhang, Y Shen, Y Shao, L Chen - … of the ACM on Management of Data, 2023 - dl.acm.org
Recently Graph Neural Networks (GNNs) have achieved great success in many
applications. The mini-batch training has become the de-facto way to train GNNs on giant …

Wholegraph: A fast graph neural network training framework with multi-gpu distributed shared memory architecture

D Yang, J Liu, J Qi, J Lai - SC22: International Conference for …, 2022 - ieeexplore.ieee.org
Graph neural networks (GNNs) are prevalent to deal with graph-structured datasets,
encoding graph data into low dimensional vectors. In this paper, we present a fast training …

Relhd: A graph-based learning on fefet with hyperdimensional computing

J Kang, M Zhou, A Bhansali, W Xu… - 2022 IEEE 40th …, 2022 - ieeexplore.ieee.org
Advances in graph neural network (GNN)-based algorithms enable machine learning on
relational data. GNNs are computationally demanding since they rely upon backpropagation …

Hyperscale FPGA-as-a-service architecture for large-scale distributed graph neural network

S Li, D Niu, Y Wang, W Han, Z Zhang, T Guan… - Proceedings of the 49th …, 2022 - dl.acm.org
Graph neural network (GNN) is a promising emerging application for link prediction,
recommendation, etc. Existing hardware innovation is limited to single-machine GNN (SM …

Fast and efficient model serving using multi-GPUs with direct-host-access

J Jeong, S Baek, J Ahn - … of the Eighteenth European Conference on …, 2023 - dl.acm.org
As deep learning (DL) inference has been widely adopted for building user-facing
applications in many domains, it is increasingly important for DL inference servers to …