The integration of Large Language Models (LLMs) and Edge Intelligence (EI) introduces a
groundbreaking paradigm for intelligent edge devices. With their capacity for human-like …

The computational challenges of Large Language Model (LLM) inference remain a
significant barrier to their widespread deployment, especially as prompt lengths continue to …

On-device large language models (LLMs), referring to running LLMs on edge devices, have
raised considerable interest owing to their superior privacy, reduced latency, and bandwidth …

LLMs are seeing growing use for applications such as document analysis and
summarization which require large context windows, and with these large context windows …

The widespread of Large Language Models (LLMs) marks a significant milestone in
generative AI. Nevertheless, the increasing context length and batch size in offline LLM …

Transformer-based large Language Models (LLMs) become increasingly important in
various domains. However, the quadratic time complexity of attention operation poses a …

The inference process for large language models is slow and memory-intensive, with one of
the most critical bottlenecks being excessive Key-Value (KV) cache accesses. This paper …

We present UncertaintyRAG, a novel approach for long-context Retrieval-Augmented
Generation (RAG) that utilizes Signal-to-Noise Ratio (SNR)-based span uncertainty to …

This paper tackles the memory hurdle of processing long context sequences in Large
Language Models (LLMs), by presenting a novel approach, Dropping In Convolutions for …

Large language models (LLMs) now support extremely long context windows, but the
quadratic complexity of vanilla attention results in significantly long Time-to-First-Token …