Understanding knowledge mechanisms in Large Language Models (LLMs) is crucial for
advancing towards trustworthy AGI. This paper reviews knowledge mechanism analysis …

The remarkable performance of large language models (LLMs) in content generation,
coding, and common-sense reasoning has spurred widespread integration into many facets …

Despite the recent proliferation of large language models (LLMs), their training recipes--
model architecture, pre-training data and optimization algorithm--are often very similar. This …

Sparse autoencoders (SAEs) have become a core ingredient in the reverse engineering of
large-language models (LLMs). For LLMs, they have been shown to decompose …

In recent years, machine learning models have made strides towards human-like reasoning
capabilities from several directions. In this work, we review the current state of the literature …

Assessing the degree of similarity of code fragments is crucial for ensuring software quality,
but it remains challenging due to the need to capture the deeper semantic aspects of code …

Research and the scientific activity are widely seen as an area where the current trends in
AI, namely the development of deep learning models (including large language models), are …

We introduce ElastiFormer, a post-training technique that adapts pretrained Transformer
models into an elastic counterpart with variable inference time compute. ElastiFormer …

Recent large language models (LLMs) have excelled across a wide range of tasks, but their
use in high-stakes and compute-limited settings has intensified the demand for …

Despite their impressive capabilities, large language models (LLMs) often lack
interpretability and can generate toxic content. While using LLMs as foundation models and …