Recent advances and future prospects for memristive materials, devices, and systems

MK Song, JH Kang, X Zhang, W Ji, A Ascoli… - ACS …, 2023 - ACS Publications
Memristive technology has been rapidly emerging as a potential alternative to traditional
CMOS technology, which is facing fundamental limitations in its development. Since oxide …

A survey on deep neural network pruning: Taxonomy, comparison, analysis, and recommendations

H Cheng, M Zhang, JQ Shi - IEEE Transactions on Pattern …, 2024 - ieeexplore.ieee.org
Modern deep neural networks, particularly recent large language models, come with
massive model sizes that require significant computational and storage resources. To …

A survey on model compression for large language models

X Zhu, J Li, Y Liu, C Ma, W Wang - Transactions of the Association for …, 2024 - direct.mit.edu
Abstract Large Language Models (LLMs) have transformed natural language processing
tasks successfully. Yet, their large size and high computational needs pose challenges for …

Thousands of conductance levels in memristors integrated on CMOS

M Rao, H Tang, J Wu, W Song, M Zhang, W Yin… - Nature, 2023 - nature.com
Neural networks based on memristive devices,–have the ability to improve throughput and
energy efficiency for machine learning, and artificial intelligence, especially in edge …

Beyond transmitting bits: Context, semantics, and task-oriented communications

D Gündüz, Z Qin, IE Aguerri, HS Dhillon… - IEEE Journal on …, 2022 - ieeexplore.ieee.org
Communication systems to date primarily aim at reliably communicating bit sequences.
Such an approach provides efficient engineering designs that are agnostic to the meanings …

Feature dimensionality reduction: a review

W Jia, M Sun, J Lian, S Hou - Complex & Intelligent Systems, 2022 - Springer
As basic research, it has also received increasing attention from people that the “curse of
dimensionality” will lead to increase the cost of data storage and computing; it also …

Sheared llama: Accelerating language model pre-training via structured pruning

M Xia, T Gao, Z Zeng, D Chen - arXiv preprint arXiv:2310.06694, 2023 - arxiv.org
The popularity of LLaMA (Touvron et al., 2023a; b) and other recently emerged moderate-
sized large language models (LLMs) highlights the potential of building smaller yet powerful …

Digital twin enhanced federated reinforcement learning with lightweight knowledge distillation in mobile networks

X Zhou, X Zheng, X Cui, J Shi, W Liang… - IEEE Journal on …, 2023 - ieeexplore.ieee.org
The high-speed mobile networks offer great potentials to many future intelligent applications,
such as autonomous vehicles in smart transportation systems. Such networks provide the …

R-drop: Regularized dropout for neural networks

L Wu, J Li, Y Wang, Q Meng, T Qin… - Advances in …, 2021 - proceedings.neurips.cc
Dropout is a powerful and widely used technique to regularize the training of deep neural
networks. Though effective and performing well, the randomness introduced by dropout …

Edgevits: Competing light-weight cnns on mobile devices with vision transformers

J Pan, A Bulat, F Tan, X Zhu, L Dudziak, H Li… - … on Computer Vision, 2022 - Springer
Self-attention based models such as vision transformers (ViTs) have emerged as a very
competitive architecture alternative to convolutional neural networks (CNNs) in computer …