The evolution of distributed systems for graph neural networks and their origin in graph processing and deep learning: A survey

J Vatter, R Mayer, HA Jacobsen - ACM Computing Surveys, 2023 - dl.acm.org
Graph neural networks (GNNs) are an emerging research field. This specialized deep
neural network architecture is capable of processing graph structured data and bridges the …

Computing graph neural networks: A survey from algorithms to accelerators

S Abadal, A Jain, R Guirado, J López-Alonso… - ACM Computing …, 2021 - dl.acm.org
Graph Neural Networks (GNNs) have exploded onto the machine learning scene in recent
years owing to their capability to model and learn from graph-structured data. Such an ability …

Torchsparse: Efficient point cloud inference engine

H Tang, Z Liu, X Li, Y Lin, S Han - Proceedings of Machine …, 2022 - proceedings.mlsys.org
Deep learning on point clouds has received increased attention thanks to its wide
applications in AR/VR and autonomous driving. These applications require low latency and …

Ansor: Generating {High-Performance} tensor programs for deep learning

L Zheng, C Jia, M Sun, Z Wu, CH Yu, A Haj-Ali… - … USENIX symposium on …, 2020 - usenix.org
High-performance tensor programs are crucial to guarantee efficient execution of deep
neural networks. However, obtaining performant tensor programs for different operators on …

Distgnn: Scalable distributed training for large-scale graph neural networks

V Md, S Misra, G Ma, R Mohanty, E Georganas… - Proceedings of the …, 2021 - dl.acm.org
Full-batch training on Graph Neural Networks (GNN) to learn the structure of large graphs is
a critical problem that needs to scale to hundreds of compute nodes to be feasible. It is …

Sparsetir: Composable abstractions for sparse compilation in deep learning

Z Ye, R Lai, J Shao, T Chen, L Ceze - Proceedings of the 28th ACM …, 2023 - dl.acm.org
Sparse tensors are rapidly becoming critical components of modern deep learning
workloads. However, developing high-performance sparse operators can be difficult and …

Distributed graph neural network training: A survey

Y Shao, H Li, X Gu, H Yin, Y Li, X Miao… - ACM Computing …, 2024 - dl.acm.org
Graph neural networks (GNNs) are a type of deep learning models that are trained on
graphs and have been successfully applied in various domains. Despite the effectiveness of …

Parallel and distributed graph neural networks: An in-depth concurrency analysis

M Besta, T Hoefler - IEEE Transactions on Pattern Analysis and …, 2024 - ieeexplore.ieee.org
Graph neural networks (GNNs) are among the most powerful tools in deep learning. They
routinely solve complex problems on unstructured networks, such as node classification …

GNN at the edge: Cost-efficient graph neural network processing over distributed edge servers

L Zeng, C Yang, P Huang, Z Zhou… - IEEE Journal on …, 2022 - ieeexplore.ieee.org
Edge intelligence has arisen as a promising computing paradigm for supporting
miscellaneous smart applications that rely on machine learning techniques. While the …

High-Performance and Programmable Attentional Graph Neural Networks with Global Tensor Formulations

M Besta, P Renc, R Gerstenberger… - Proceedings of the …, 2023 - dl.acm.org
Graph attention models (A-GNNs), a type of Graph Neural Networks (GNNs), have been
shown to be more powerful than simpler convolutional GNNs (C-GNNs). However, A-GNNs …