Large Language Models (LLMs) have demonstrated remarkable capabilities across various
fields, from natural language understanding to text generation. Compared to non-generative …

The widespread of Large Language Models (LLMs) marks a significant milestone in
generative AI. Nevertheless, the increasing context length and batch size in offline LLM …

Large language models (LLMs) have emerged due to their capability to generate high-
quality content across diverse contexts. To reduce their explosively increasing demands for …

Modern transformer-based Large Language Models (LLMs) are constructed with a series of
decoder blocks. Each block comprises three key components:(1) QKV generation,(2) multi …

Transformer-based large language models (LLMs) such as Generative Pre-trained
Transformer (GPT) have become popular due to their remarkable performance across …

The Transformer-based generative model (TbGM), comprising summarization (Sum) and
generation (Gen) stages, has demonstrated unprecedented generative performance across …

In medical research, the rapid proliferation of condition-specific studies has led to an
information overload, making it challenging for researchers and practitioners to stay abreast …

Processing-in-memory (PIM) has emerged as a promising solution for accelerating memory-
intensive workloads as they provide high memory bandwidth to the processing units. This …

Generative large language models'(LLMs) inference suffers from inefficiency because of the
token dependency brought by autoregressive decoding. Recently, speculative inference has …

The widespread adoption of LLMs has driven an exponential rise in their deployment,
imposing substantial demands on inference clusters. These clusters must handle numerous …