On the Opportunities and Risks of Foundation Models R Bommasani, DA Hudson, E Adeli, R Altman, S Arora, S von Arx, ... arXiv preprint arXiv:2108.07258, 2021 | 3246 | 2021 |
BLOOM: A 176B-Parameter Open-Access Multilingual Language Model TL Scao, A Fan, C Akiki, E Pavlick, S Ilić, D Hesslow, R Castagné, ... arXiv preprint arXiv:2211.05100, 2022 | 1307 | 2022 |
PipeDream: Generalized Pipeline Parallelism for DNN Training D Narayanan, A Harlap, A Phanishayee, V Seshadri, NR Devanur, ... 27th ACM Symposium on Operating Systems Principles, 1-15, 2019 | 758 | 2019 |
Holistic Evaluation of Language Models P Liang, R Bommasani, T Lee, D Tsipras, D Soylu, M Yasunaga, Y Zhang, ... arXiv preprint arXiv:2211.09110, 2022 | 757 | 2022 |
Efficient Large-Scale Language Model Training on GPU Clusters using Megatron-LM D Narayanan, M Shoeybi, J Casper, P LeGresley, M Patwary, ... Proceedings of the International Conference for High Performance Computing …, 2021 | 485 | 2021 |
DAWNBench: An End-to-End Deep Learning Benchmark and Competition C Coleman, D Narayanan, D Kang, T Zhao, J Zhang, L Nardi, P Bailis, ... NeurIPS Workshop on Systems for Machine Learning, 2017 | 379 | 2017 |
MLPerf Training Benchmark P Mattson, C Cheng, C Coleman, G Diamos, P Micikevicius, D Patterson, ... Third Conference on Machine Learning and Systems, 2020 | 312 | 2020 |
PipeDream: Fast and Efficient Pipeline Parallel DNN Training A Harlap, D Narayanan, A Phanishayee, V Seshadri, N Devanur, ... arXiv preprint arXiv:1806.03377, 2018 | 253 | 2018 |
Heterogeneity-Aware Cluster Scheduling Policies for Deep Learning Workloads D Narayanan, K Santhanam, F Kazhamiaka, A Phanishayee, M Zaharia 14th USENIX Symposium on Operating Systems Design and Implementation (OSDI …, 2020 | 187 | 2020 |
MacroBase: Prioritizing Attention in Fast Data P Bailis, E Gan, S Madden, D Narayanan, K Rong, S Suri Proceedings of the 2017 ACM International Conference on Management of Data …, 2017 | 182 | 2017 |
Weld: A Common Runtime for High Performance Data Analytics S Palkar, JJ Thomas, A Shanbhag, D Narayanan, H Pirk, M Schwarzkopf, ... Conference on Innovative Data Systems Research (CIDR), 2017 | 177 | 2017 |
Memory-Efficient Pipeline-Parallel DNN Training D Narayanan, A Phanishayee, K Shi, X Chen, M Zaharia International Conference on Machine Learning, 7937-7947, 2021 | 176 | 2021 |
Analysis of DAWNBench, a Time-to-Accuracy Machine Learning Performance Benchmark C Coleman, D Kang, D Narayanan, L Nardi, T Zhao, J Zhang, P Bailis, ... ACM SIGOPS Operating Systems Review 53 (1), 14-25, 2019 | 138 | 2019 |
Evaluating End-to-End Optimization for Data Analytics Applications in Weld S Palkar, J Thomas, D Narayanan, P Thaker, R Palamuttam, P Negi, ... Proceedings of the VLDB Endowment 11 (9), 1002-1015, 2018 | 96 | 2018 |
Accelerating Deep Learning Workloads through Efficient Multi-Model Execution D Narayanan, K Santhanam, A Phanishayee, M Zaharia NeurIPS Workshop on Systems for Machine Learning, 2018 | 57 | 2018 |
Holistic Evaluation of Text-to-Image Models T Lee, M Yasunaga, C Meng, Y Mai, JS Park, A Gupta, Y Zhang, ... arXiv preprint arXiv:2311.04287, 2023 | 44 | 2023 |
Solving Large-Scale Granular Resource Allocation Problems Efficiently with POP D Narayanan, F Kazhamiaka, F Abuzaid, P Kraft, A Agrawal, S Kandula, ... 28th ACM Symposium on Operating Systems Principles, 2021 | 43 | 2021 |
Piper: Multidimensional Planner for DNN Parallelization JM Tarnawski, D Narayanan, A Phanishayee Advances in Neural Information Processing Systems 34, 24829-24840, 2021 | 38 | 2021 |
MegaBlocks: Efficient Sparse Training with Mixture-of-Experts T Gale, D Narayanan, C Young, M Zaharia Proceedings of Machine Learning and Systems 5, 2023 | 35 | 2023 |
Weld: Rethinking the Interface Between Data-Intensive Applications S Palkar, JJ Thomas, D Narayanan, A Shanbhag, R Palamuttam, H Pirk, ... arXiv preprint arXiv:1709.06416, 2017 | 31 | 2017 |