Abstract Large Language Models (LLMs) have demonstrated remarkable capabilities in
important tasks such as natural language understanding, language generation, and …

The burgeoning field of Large Language Models (LLMs), exemplified by sophisticated
models like OpenAI's ChatGPT, represents a significant advancement in artificial …

Recent advances in Transformers have come with a huge requirement on computing
resources, highlighting the importance of developing efficient training techniques to make …

Large foundation models, including large language models (LLMs), vision transformers
(ViTs), diffusion, and LLM-based multimodal models, are revolutionizing the entire machine …

Optimizer states are a major source of memory consumption for training neural networks,
limiting the maximum trainable model within given memory budget. Compressing the …

As the model size grows rapidly, fine-tuning the large pre-trained language model has
become increasingly difficult due to its extensive memory usage. Previous works usually …

On-device training is essential for user personalisation and privacy. With the pervasiveness
of IoT devices and microcontroller units (MCU), this task becomes more challenging due to …

There has been an explosion of interest in designing various Knowledge Graph Neural
Networks (KGNNs), which achieve state-of-the-art performance and provide great …

Graph learning is becoming increasingly popular due to its superior performance in tackling
many grand challenges. While quantization is widely used to accelerate Graph Neural …

Activation compressed training provides a solution towards reducing the memory cost of
training deep neural networks (DNNs). However, state-of-the-art work combines a search of …