A distributed data-parallel pytorch implementation of the distributed shampoo optimizer for training neural networks at-scale

HJM Shi, TH Lee, S Iwasaki, J Gallego-Posada… - arXiv preprint arXiv …, 2023 - arxiv.org
Shampoo is an online and stochastic optimization algorithm belonging to the AdaGrad
family of methods for training neural networks. It constructs a block-diagonal preconditioner …

Disaggregated Multi-Tower: Topology-aware Modeling Technique for Efficient Large Scale Recommendation

L Luo, B Zhang, M Tsang, Y Ma… - Proceedings of …, 2024 - proceedings.mlsys.org
We study a mismatch between the deep learning recommendation models' flat architecture,
common distributedtraining paradigm and hierarchical data center topology. To address the …

PreSto: An In-Storage Data Preprocessing System for Training Recommendation Models

Y Lee, H Kim, M Rhu - 2024 ACM/IEEE 51st Annual …, 2024 - ieeexplore.ieee.org
Training recommendation systems (RecSys) faces several challenges as it requires the
“data preprocessing” stage to preprocess an ample amount of raw data and feed them to the …

Benchmarking News Recommendation in the Era of Green AI

Q Liu, J Zhu, Q Dai, XM Wu - Companion Proceedings of the ACM on …, 2024 - dl.acm.org
Over recent years, news recommender systems have gained significant attention in both
academia and industry, emphasizing the need for a standardized benchmark to evaluate …

POSTER: Pattern-Aware Sparse Communication for Scalable Recommendation Model Training

J He, S Chen, J Zhai - Proceedings of the 29th ACM SIGPLAN Annual …, 2024 - dl.acm.org
Recommendation models are an important category of deep learning models whose size is
growing enormous. They consist of a sparse part with TBs of memory footprint and a dense …

RecWizard: A Toolkit for Conversational Recommendation with Modular, Portable Models and Interactive User Interface

Z Zhang, T Laud, Z He, X Chen, X Liu, Z Xie… - Proceedings of the …, 2024 - ojs.aaai.org
We present a new Python toolkit called RecWizard for Conversational Recommender
Systems (CRS). RecWizard offers support for development of models and interactive user …

Generalize for Future: Slow and Fast Trajectory Learning for CTR Prediction

J Zhu, C Liu, X Jiang, C Peng, Z Lin… - Proceedings of the AAAI …, 2024 - ojs.aaai.org
Deep neural networks (DNNs) have achieved significant advancements in click-through rate
(CTR) prediction by demonstrating strong generalization on training data. However, in real …

Embedding Optimization for Training Large-scale Deep Learning Recommendation Systems with EMBark

S Liu, N Zheng, H Kang, X Simmons, J Zhang… - Proceedings of the 18th …, 2024 - dl.acm.org
Training large-scale deep learning recommendation models (DLRMs) with embedding
tables stretching across multiple GPUs in a cluster presents a unique challenge, demanding …

FEC: Efficient Deep Recommendation Model Training with Flexible Embedding Communication

K Ma, X Yan, Z Cai, Y Huang, Y Wu… - Proceedings of the ACM on …, 2023 - dl.acm.org
Embedding-based deep recommendation models (EDRMs), which contain small dense
models and large embedding tables, are widely used in industry. Embedding …

Scaling New Frontiers: Insights into Large Recommendation Models

W Guo, H Wang, L Zhang, JY Chin, Z Liu… - arXiv preprint arXiv …, 2024 - arxiv.org
Recommendation systems are essential for filtering data and retrieving relevant information
across various applications. Recent advancements have seen these systems incorporate …