A survey on evolutionary neural architecture search

Y Liu, Y Sun, B Xue, M Zhang, GG Yen… - IEEE transactions on …, 2021 - ieeexplore.ieee.org
Deep neural networks (DNNs) have achieved great success in many applications. The
architectures of DNNs play a crucial role in their performance, which is usually manually …

Weight-sharing neural architecture search: A battle to shrink the optimization gap

L Xie, X Chen, K Bi, L Wei, Y Xu, L Wang… - ACM Computing …, 2021 - dl.acm.org
Neural architecture search (NAS) has attracted increasing attention. In recent years,
individual search methods have been replaced by weight-sharing search methods for higher …

Bananas: Bayesian optimization with neural architectures for neural architecture search

C White, W Neiswanger, Y Savani - … of the AAAI conference on artificial …, 2021 - ojs.aaai.org
Over the past half-decade, many methods have been considered for neural architecture
search (NAS). Bayesian optimization (BO), which has long had success in hyperparameter …

How powerful are performance predictors in neural architecture search?

C White, A Zela, R Ru, Y Liu… - Advances in Neural …, 2021 - proceedings.neurips.cc
Early methods in the rapidly developing field of neural architecture search (NAS) required
fully training thousands of neural networks. To reduce this extreme computational cost …

Nas-bench-suite-zero: Accelerating research on zero cost proxies

A Krishnakumar, C White, A Zela… - Advances in …, 2022 - proceedings.neurips.cc
Zero-cost proxies (ZC proxies) are a recent architecture performance prediction technique
aiming to significantly speed up algorithms for neural architecture search (NAS). Recent …

A study on encodings for neural architecture search

C White, W Neiswanger, S Nolen… - Advances in neural …, 2020 - proceedings.neurips.cc
Neural architecture search (NAS) has been extensively studied in the past few years. A
popular approach is to represent each neural architecture in the search space as a directed …

Renas: Relativistic evaluation of neural architecture search

Y Xu, Y Wang, K Han, Y Tang, S Jui… - Proceedings of the …, 2021 - openaccess.thecvf.com
An effective and efficient architecture performance evaluation scheme is essential for the
success of Neural Architecture Search (NAS). To save computational cost, most of existing …

Nas-bench-suite: Nas evaluation is (now) surprisingly easy

Y Mehta, C White, A Zela, A Krishnakumar… - arXiv preprint arXiv …, 2022 - arxiv.org
The release of tabular benchmarks, such as NAS-Bench-101 and NAS-Bench-201, has
significantly lowered the computational overhead for conducting scientific research in neural …

EMONAS-Net: Efficient multiobjective neural architecture search using surrogate-assisted evolutionary algorithm for 3D medical image segmentation

MB Calisto, SK Lai-Yuen - Artificial intelligence in medicine, 2021 - Elsevier
Deep learning plays a critical role in medical image segmentation. Nevertheless, manually
designing a neural network for a specific segmentation problem is a very difficult and time …

Efficient federated learning for modern nlp

D Cai, Y Wu, S Wang, FX Lin, M Xu - Proceedings of the 29th Annual …, 2023 - dl.acm.org
Transformer-based pre-trained models have revolutionized NLP for superior performance
and generality. Fine-tuning pre-trained models for downstream tasks often requires private …