A distributed data-parallel pytorch implementation of the distributed shampoo optimizer for training neural networks at-scale
Shampoo is an online and stochastic optimization algorithm belonging to the AdaGrad
family of methods for training neural networks. It constructs a block-diagonal preconditioner …
family of methods for training neural networks. It constructs a block-diagonal preconditioner …
Disaggregated Multi-Tower: Topology-aware Modeling Technique for Efficient Large Scale Recommendation
We study a mismatch between the deep learning recommendation models' flat architecture,
common distributedtraining paradigm and hierarchical data center topology. To address the …
common distributedtraining paradigm and hierarchical data center topology. To address the …
PreSto: An In-Storage Data Preprocessing System for Training Recommendation Models
Training recommendation systems (RecSys) faces several challenges as it requires the
“data preprocessing” stage to preprocess an ample amount of raw data and feed them to the …
“data preprocessing” stage to preprocess an ample amount of raw data and feed them to the …
Benchmarking News Recommendation in the Era of Green AI
Over recent years, news recommender systems have gained significant attention in both
academia and industry, emphasizing the need for a standardized benchmark to evaluate …
academia and industry, emphasizing the need for a standardized benchmark to evaluate …
POSTER: Pattern-Aware Sparse Communication for Scalable Recommendation Model Training
Recommendation models are an important category of deep learning models whose size is
growing enormous. They consist of a sparse part with TBs of memory footprint and a dense …
growing enormous. They consist of a sparse part with TBs of memory footprint and a dense …
RecWizard: A Toolkit for Conversational Recommendation with Modular, Portable Models and Interactive User Interface
We present a new Python toolkit called RecWizard for Conversational Recommender
Systems (CRS). RecWizard offers support for development of models and interactive user …
Systems (CRS). RecWizard offers support for development of models and interactive user …
Generalize for Future: Slow and Fast Trajectory Learning for CTR Prediction
Deep neural networks (DNNs) have achieved significant advancements in click-through rate
(CTR) prediction by demonstrating strong generalization on training data. However, in real …
(CTR) prediction by demonstrating strong generalization on training data. However, in real …
Embedding Optimization for Training Large-scale Deep Learning Recommendation Systems with EMBark
S Liu, N Zheng, H Kang, X Simmons, J Zhang… - Proceedings of the 18th …, 2024 - dl.acm.org
Training large-scale deep learning recommendation models (DLRMs) with embedding
tables stretching across multiple GPUs in a cluster presents a unique challenge, demanding …
tables stretching across multiple GPUs in a cluster presents a unique challenge, demanding …
FEC: Efficient Deep Recommendation Model Training with Flexible Embedding Communication
Embedding-based deep recommendation models (EDRMs), which contain small dense
models and large embedding tables, are widely used in industry. Embedding …
models and large embedding tables, are widely used in industry. Embedding …
Scaling New Frontiers: Insights into Large Recommendation Models
Recommendation systems are essential for filtering data and retrieving relevant information
across various applications. Recent advancements have seen these systems incorporate …
across various applications. Recent advancements have seen these systems incorporate …