Abstract Deep Neural Networks have received considerable attention in recent years. As the
complexity of network architecture increases in relation to the task complexity, it becomes …

Vision transformers (ViTs) inherited the success of NLP but their structures have not been
sufficiently investigated and optimized for visual tasks. One of the simplest solutions is to …

Variational quantum algorithms (VQAs) are expected to be a path to quantum advantages
on noisy intermediate-scale quantum devices. However, both empirical and theoretical …

Distilling knowledge from an ensemble of teacher models is expected to have a more
promising performance than that from a single one. Current methods mainly adopt a vanilla …

Recent advances on Out-of-Distribution (OoD) generalization reveal the robustness of deep
learning models against distribution shifts. However, existing works focus on OoD …

One-shot neural architecture search (NAS) methods significantly reduce the search cost by
considering the whole search space as one network, which only needs to be trained once …

Most differentiable neural architecture search methods construct a super-net for search and
derive a target-net as its sub-graph for evaluation. There exists a significant gap between the …

Gradient Descent, an effective way to search for the local minimum of a function, can
minimize training and validation loss of neural architectures and also be incited in an …

In one-shot weight sharing for NAS, the weights of each operation (at each layer) are
supposed to be identical for all architectures (paths) in the supernet. However, this rules out …

Differentiable architecture search (DARTS) has been a popular one-shot paradigm for NAS
due to its high efficiency. It introduces trainable architecture parameters to represent the …